gdb: Replace operator new / operator new[]

  If xmalloc fails allocating memory, usually because something tried a
huge allocation, like xmalloc(-1) or some such, GDB asks the user what
to do:

  .../src/gdb/utils.c:1079: internal-error: virtual memory exhausted.
  A problem internal to GDB has been detected,
  further debugging may prove unreliable.
  Quit this debugging session? (y or n)

If the user says "n", that throws a QUIT exception, which is caught by
one of the multiple CATCH(RETURN_MASK_ALL) blocks somewhere up the
stack.

The default implementations of operator new / operator new[] call
malloc directly, and on memory allocation throw std::bad_alloc.
Currently, if that happens, since nothing catches std::bad_alloc, the
exception escapes out of main, and GDB aborts from unhandled
exception.

This patch replaces the default operator new variants with versions
that, just like xmalloc:

 #1 - Raise an internal-error on memory allocation failure.

 #2 - Throw a QUIT gdb_exception, so that the exact same CATCH blocks
      continue handling memory allocation problems.

A minor complication of #2 is that operator new can _only_ throw
std:bad_alloc, or something that extends it:

  void* operator new (std::size_t size) throw (std::bad_alloc);

That means that if we let a gdb QUIT exception escape from within
operator new, the C++ runtime aborts due to unexpected exception
thrown.

So to bridge the gap, this patch adds a new gdb_quit_bad_alloc
exception type that inherits both std::bad_alloc and gdb_exception,
and throws _that_.

If we decide that we should be catching memory allocation errors in
fewer places than all the places we currently catch them (everywhere
we use RETURN_MASK_ALL currently), then we could change operator new
to throw plain std::bad_alloc then.  But I'm considering such a change
as separate matter from this one -- it'd make sense to do the same to
xmalloc at the same time, for instance.

Meanwhile, this allows using new/new[] instead of xmalloc/XNEW/etc.
without losing the "virtual memory exhausted" internal-error
safeguard.

Tested on x86_64 Fedora 23.

gdb/ChangeLog:
2016-09-22  Pedro Alves  <palves@redhat.com>

	* Makefile.in (SFILES): Add common/new-op.c.
	(COMMON_OBS): Add common/new-op.o.
	(new-op.o): New rule.
	* common/common-exceptions.h: Include <new>.
	(struct gdb_quit_bad_alloc): New type.
	* common/new-op.c: New file.

gdb/gdbserver/ChangeLog:
2016-09-22  Pedro Alves  <palves@redhat.com>

	* Makefile.in (SFILES): Add common/new-op.c.
	(OBS): Add common/new-op.o.
	(new-op.o): New rule.
---
 gdb/Makefile.in                |  8 +++-
 gdb/common/common-exceptions.h | 20 ++++++++++
 gdb/common/new-op.c            | 85 ++++++++++++++++++++++++++++++++++++++++++
 gdb/gdbserver/Makefile.in      |  8 +++-
 4 files changed, 117 insertions(+), 4 deletions(-)
 create mode 100644 gdb/common/new-op.c

gdb: Replace operator new / operator new[]

Commit Message

Comments

Patch