diff mbox series

[RFC] Fix AIX core file handling: prevent crashes during GDB quit

Message ID 20260324143835.39727-2-akamath996@gmail.com
State New
Headers
Series [RFC] Fix AIX core file handling: prevent crashes during GDB quit |

Checks

Context Check Description
linaro-tcwg-bot/tcwg_binutils_build--master-aarch64 warning Skipped because it is an RFC
linaro-tcwg-bot/tcwg_binutils_build--master-arm warning Skipped because it is an RFC

Commit Message

Aditya Vidyadhar Kamath March 24, 2026, 2:38 p.m. UTC 
  From: Aditya Vidyadhar Kamath <aditya.kamath1@ibm.com>

When quitting GDB after analyzing a core file, _bfd_coff_free_cached_info()
crashed with a segmentation fault. 

We can see this problem when quit after debugging a large core file
in AIX 7.3.

Ex:
Program terminated with signal SIGSEGV, Segmentation fault.
   from /opt/freeware/lib/libpython3.9.a(libpython3.9.so)
(gdb)q

Fatal signal: Segmentation fault
----- Backtrace -----
0x1009fbffb ???
0x1009fc11f ???
0x1005c3587 ???
0x1005c3833 ???
0x4fdf ???

The backtrace showed:

  htab_delete(0x0000bc18) at 0x1000e2e4
  _bfd_coff_free_cached_info(abfd = 0x20023a90) at 0xd02e5a58
  bfd_cache_close(abfd = 0x20023a90) at 0xd02c0e7c
  bfd_cache_close_all() at 0xd02c0f48
  bfd_close_all_done(abfd = 0x20023a90) at 0xd02c1234
  bfd_close(abfd = 0x20023a90) at 0xd02c1180

My understanding so is as follows:
1. AIX core files store a pointer to 'struct core_dumpxx' in abfd->tdata.any
2. COFF object files store a pointer to 'struct coff_data_type' in the same field
3. The condition checked: (bfd_get_format(abfd) == bfd_object ||
                            bfd_get_format(abfd) == bfd_core)
4. For core files, the code cast 'core_dumpxx *' to 'coff_data_type *'
5. It then tried to read 'section_by_target_index' hash table pointer
6. Reading from the wrong offset in core_dumpxx returned garbage (0xbc18)
7. This garbage pointer passed the NULL check but crashed in htab_delete()

I think the hash tables are only allocated when processing COFF symbols, which
   never happens for core files.

This patch is a fix to the same.
---
 bfd/coffgen.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

Comments

Alan Modra March 25, 2026, 12:14 p.m. UTC | #1 
On Tue, Mar 24, 2026 at 08:08:36PM +0530, Aditya Vidyadhar Kamath wrote:
> 1. AIX core files store a pointer to 'struct core_dumpxx' in abfd->tdata.any
> 2. COFF object files store a pointer to 'struct coff_data_type' in the same field

Correct.  However other COFF/PE core files can be read as for object
files, for instance bfd/coff-x86_64.c has as part of its bfd_target:

  /* Note that we allow an object file to be treated as a core file as well.  */
  {				/* bfd_check_format.  */
    _bfd_dummy_target,
    amd64coff_object_p,
    bfd_generic_archive_p,
    amd64coff_object_p
  },

This means a bfd_core will have the expected coff_data_type for
x86_64, and thus

> 3. The condition checked: (bfd_get_format(abfd) == bfd_object ||
>                             bfd_get_format(abfd) == bfd_core)

is correct for x86_64 and other targets.  xcoff is the odd one out
here.  The proper fix will involve an xcoff specific free_cached_info.

void
_bfd_xcoff_bfd_free_cached_info (bfd *abfd)
{
  /* xcoff bfd_core does not have a coff_tdata.  */
  if (bfd_get_format (abfd) == bfd_object)
    return _bfd_coff_free_cached_info (abfd);

  return _bfd_generic_bfd_free_cached_info (abfd);
}

I have a patch in my tree which I'll push after testing, probably
tomorrow.
diff mbox series

Patch

diff --git a/bfd/coffgen.c b/bfd/coffgen.c
index 030dbc1dc79..ca555647592 100644
--- a/bfd/coffgen.c
+++ b/bfd/coffgen.c
@@ -3310,8 +3310,7 @@  _bfd_coff_free_cached_info (bfd *abfd)
   struct coff_tdata *tdata;
 
   if (bfd_family_coff (abfd)
-      && (bfd_get_format (abfd) == bfd_object
-	  || bfd_get_format (abfd) == bfd_core)
+      && (bfd_get_format (abfd) == bfd_object)
       && (tdata = coff_data (abfd)) != NULL)
     {
       if (tdata->section_by_index)