[v2] PE/COFF: auto-promote oversized objects to bigobj

Message ID 20260623213900.2000-2-oleg.tolmatcev@gmail.com
State New
Headers
Series [v2] PE/COFF: auto-promote oversized objects to bigobj |

Commit Message

Oleg Tolmatcev June 23, 2026, 9:39 p.m. UTC
  PE/COFF object files use 16-bit section numbers in normal symbols.
When an object grows past that limit, current BFD already rejects it with
a "too many sections" / "file too big" error.

BFD already has PE bigobj targets for the backends that support them, but
oversized PE/COFF output still requires selecting bigobj explicitly.

Teach coff_compute_section_file_positions to switch PE/COFF output to the
corresponding bigobj target when the normal section-number limit is
exceeded. This preserves the existing hard error for targets without
bigobj support, but lets supported PE targets transparently emit a valid
bigobj object instead of requiring manual selection.

bfd/
	* coffcode.h (coff_compute_section_file_positions): Auto-promote
	oversized PE/COFF objects to the bigobj target.

gas/
	* testsuite/gas/pe/big-obj-auto.d: New test.
	* testsuite/gas/pe/big-obj-auto.s: New test.
	* testsuite/gas/pe/pe.exp: Run big-obj-auto.

Signed-off-by: Oleg Tolmatcev <oleg.tolmatcev@gmail.com>
---
 bfd/coffcode.h                      | 14 ++++++++++++++
 gas/testsuite/gas/pe/big-obj-auto.d |  9 +++++++++
 gas/testsuite/gas/pe/big-obj-auto.s | 16 ++++++++++++++++
 gas/testsuite/gas/pe/pe.exp         |  4 +++-
 4 files changed, 42 insertions(+), 1 deletion(-)
 create mode 100644 gas/testsuite/gas/pe/big-obj-auto.d
 create mode 100644 gas/testsuite/gas/pe/big-obj-auto.s
  

Comments

Jan Beulich June 26, 2026, 6:59 a.m. UTC | #1
On 23.06.2026 23:39, Oleg Tolmatcev wrote:
> @@ -3154,6 +3158,16 @@ coff_compute_section_file_positions (bfd * abfd)
>  
>    if (target_index >= bfd_coff_max_nscns (abfd))
>      {
> +#ifdef COFF_WITH_PE_BIGOBJ
> +      if (abfd->xvec != &TARGET_SYM_BIG)
> +	{
> +	  /* Regular PE/COFF uses 16-bit section numbers in symbols.
> +	     Promote oversized objects to the matching bigobj target
> +	     before any headers or symbols are finalized.  */
> +	  abfd->xvec = &TARGET_SYM_BIG;
> +	  return coff_compute_section_file_positions (abfd);
> +	}
> +#endif
>        bfd_set_error (bfd_error_file_too_big);
>        _bfd_error_handler
>  	/* xgettext:c-format */

It occurs to me only now that this new behavior affects not only COFF
objects, but also PE binaries. For the former the defaulting is imo okay
(as previously indicated), as you'll notice when linking that the linker
can't deal with the objects (and a newer linker needs using, or section
count needs bringing down). For the latter, however, an executable may
be produced which you have no way of knowing that it won't work with
certain loaders (because all you can ever test with is a limited set of
loaders). IOW I think the conditional may want to be

#if defined(COFF_WITH_PE_BIGOBJ) && !defined(COFF_IMAGE_WITH_PE)

Thoughts? I can certainly make that adjustment while committing, as
long as we can come to an agreement.

Jan
  
Oleg Tolmatcev June 26, 2026, 7:37 a.m. UTC | #2
Am Fr., 26. Juni 2026 um 08:59 Uhr schrieb Jan Beulich <jbeulich@suse.com>:
>
> On 23.06.2026 23:39, Oleg Tolmatcev wrote:
> > @@ -3154,6 +3158,16 @@ coff_compute_section_file_positions (bfd * abfd)
> >
> >    if (target_index >= bfd_coff_max_nscns (abfd))
> >      {
> > +#ifdef COFF_WITH_PE_BIGOBJ
> > +      if (abfd->xvec != &TARGET_SYM_BIG)
> > +     {
> > +       /* Regular PE/COFF uses 16-bit section numbers in symbols.
> > +          Promote oversized objects to the matching bigobj target
> > +          before any headers or symbols are finalized.  */
> > +       abfd->xvec = &TARGET_SYM_BIG;
> > +       return coff_compute_section_file_positions (abfd);
> > +     }
> > +#endif
> >        bfd_set_error (bfd_error_file_too_big);
> >        _bfd_error_handler
> >       /* xgettext:c-format */
>
> It occurs to me only now that this new behavior affects not only COFF
> objects, but also PE binaries. For the former the defaulting is imo okay
> (as previously indicated), as you'll notice when linking that the linker
> can't deal with the objects (and a newer linker needs using, or section
> count needs bringing down). For the latter, however, an executable may
> be produced which you have no way of knowing that it won't work with
> certain loaders (because all you can ever test with is a limited set of
> loaders). IOW I think the conditional may want to be
>
> #if defined(COFF_WITH_PE_BIGOBJ) && !defined(COFF_IMAGE_WITH_PE)
>
> Thoughts? I can certainly make that adjustment while committing, as
> long as we can come to an agreement.

I agree. The COFF objects case is the important one for me. Thanks.

Oleg.
  
Jan Beulich June 29, 2026, 10:54 a.m. UTC | #3
On 23.06.2026 23:39, Oleg Tolmatcev wrote:
> --- a/gas/testsuite/gas/pe/pe.exp
> +++ b/gas/testsuite/gas/pe/pe.exp
> @@ -68,6 +68,8 @@ if {[istarget "aarch64-*-pe*"] || [istarget "aarch64-*-mingw*"]} {
>  # Big obj
>  
>  
> -if ([istarget "*-*-mingw*"]) then {
> +if { [istarget "aarch64-*-*"] || [istarget "i*86-*-*"]
> +    || [istarget "x86_64-*-*"] } then {

This went too far: bfd/config.bfd confirms me observing both tests to fail on
i?86-interix, and it further suggests i?86-beospe* to also be affected. Or wait,
*-*-beos* is covered by the "target removed" switch higher up in the file, so
the individual entries later are all stale. I'll prune that, but for committing
this patch I'll also need to exclude *-*-interix* above.

Jan

>  	run_dump_test "big-obj"
> +	run_dump_test "big-obj-auto"
>  }
  

Patch

diff --git a/bfd/coffcode.h b/bfd/coffcode.h
index 7439fca676..5c9722a15e 100644
--- a/bfd/coffcode.h
+++ b/bfd/coffcode.h
@@ -360,6 +360,10 @@  CODE_FRAGMENT
 #include "coffswap.h"
 #endif
 
+#ifdef COFF_WITH_PE_BIGOBJ
+extern const bfd_target TARGET_SYM_BIG;
+#endif
+
 #define STRING_SIZE_SIZE 4
 
 #define DOT_DEBUG	".debug"
@@ -3154,6 +3158,16 @@  coff_compute_section_file_positions (bfd * abfd)
 
   if (target_index >= bfd_coff_max_nscns (abfd))
     {
+#ifdef COFF_WITH_PE_BIGOBJ
+      if (abfd->xvec != &TARGET_SYM_BIG)
+	{
+	  /* Regular PE/COFF uses 16-bit section numbers in symbols.
+	     Promote oversized objects to the matching bigobj target
+	     before any headers or symbols are finalized.  */
+	  abfd->xvec = &TARGET_SYM_BIG;
+	  return coff_compute_section_file_positions (abfd);
+	}
+#endif
       bfd_set_error (bfd_error_file_too_big);
       _bfd_error_handler
 	/* xgettext:c-format */
diff --git a/gas/testsuite/gas/pe/big-obj-auto.d b/gas/testsuite/gas/pe/big-obj-auto.d
new file mode 100644
index 0000000000..985c1270e2
--- /dev/null
+++ b/gas/testsuite/gas/pe/big-obj-auto.d
@@ -0,0 +1,9 @@ 
+#objdump: -h
+#name: PE big obj auto-promotion
+
+.*: *file format pe-bigobj-.*
+
+Sections:
+#...
+80002 +\.data\$a79999  .*
+                  CONTENTS, ALLOC, LOAD, DATA
diff --git a/gas/testsuite/gas/pe/big-obj-auto.s b/gas/testsuite/gas/pe/big-obj-auto.s
new file mode 100644
index 0000000000..8c010716c6
--- /dev/null
+++ b/gas/testsuite/gas/pe/big-obj-auto.s
@@ -0,0 +1,16 @@ 
+	.file "big-obj-auto.s"
+
+	.irp n,0,1,2,3,4,5,6,7
+	.irp m,0,1,2,3,4,5,6,7,8,9
+	.irp c,0,1,2,3,4,5,6,7,8,9
+	.irp d,0,1,2,3,4,5,6,7,8,9
+	.irp u,0,1,2,3,4,5,6,7,8,9
+	.globl a\n\m\c\d\u
+	.section .data$a\n\m\c\d\u,"w"
+a\n\m\c\d\u :
+	.byte 1
+	.endr
+	.endr
+	.endr
+	.endr
+	.endr
diff --git a/gas/testsuite/gas/pe/pe.exp b/gas/testsuite/gas/pe/pe.exp
index e25de3cfa9..2aba5ecb3d 100644
--- a/gas/testsuite/gas/pe/pe.exp
+++ b/gas/testsuite/gas/pe/pe.exp
@@ -68,6 +68,8 @@  if {[istarget "aarch64-*-pe*"] || [istarget "aarch64-*-mingw*"]} {
 # Big obj
 
 
-if ([istarget "*-*-mingw*"]) then {
+if { [istarget "aarch64-*-*"] || [istarget "i*86-*-*"]
+    || [istarget "x86_64-*-*"] } then {
 	run_dump_test "big-obj"
+	run_dump_test "big-obj-auto"
 }