[1/4] Makeconfig: Add libgcc directory to rtld-prefix search path

  From: Andreas K. Hüttel <dilfridge@gentoo.org>

Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
---
 Makeconfig | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
  

On 22/01/25 13:12, dilfridge@gentoo.org wrote:
> From: Andreas K. Hüttel <dilfridge@gentoo.org>
> 
> Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
> ---
>  Makeconfig | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/Makeconfig b/Makeconfig
> index d0108d2caa..6b4ebd1320 100644
> --- a/Makeconfig
> +++ b/Makeconfig
> @@ -692,6 +692,11 @@ link-libc-static-tests = -Wl,--start-group $(common-objpfx)libc.a $(static-gnuli
>  # some cases and it is preferable to link with libgcc_eh or libgcc_s
>  # so that the testing is as similar as possible to how programs will
>  # be built with the installed glibc.
> +# This leads to moderate difficulties, also since distributions may
> +# install libgcc_s.so in directories only found via ld.so.conf, e.g.
> +# to be able to switch between gcc versions. We need to add the
> +# corresponding directory to the library search path to make sure
> +# our test programs can find it.
>  #
>  # Some architectures have architecture-specific systems for exception
>  # handling that may involve undefined references to
> @@ -709,6 +714,7 @@ libgcc_eh := -Wl,--as-needed -lgcc_s $(libunwind) -Wl,--no-as-needed
>  gnulib-arch =
>  gnulib = -lgcc $(gnulib-arch)
>  gnulib-tests := -lgcc $(libgcc_eh)
> +gnulib-extralibdir := $(shell $(CC) -print-libgcc-file-name | sed -e 's:/libgcc.*$$::')

From gcc documentation [1], -print-libgcc-file-name is the same as -print-file-name=libgcc.a
which might be different than the place libgcc_s.so is installed.

On a toolchain built with build-many-glibc.py, I see:

$ arm-glibc-linux-gnueabihf-gcc -print-libgcc-file-name | xargs realpath
<path>/install/compilers/14/arm-linux-gnueabihf/lib/gcc/arm-glibc-linux-gnueabihf/14.2.1/libgcc.a

$ arm-glibc-linux-gnueabihf-gcc -print-file-name=libgcc_s.so | xargs realpath
<path>/install/compilers/14/arm-linux-gnueabihf/arm-glibc-linux-gnueabihf/lib/libgcc_s.so

So I think we should -print-file-name=libgcc_s.so instead here, and maybe 'dirname'
instead of sed (just to be a bit more clear).

>  static-gnulib-arch =
>  # By default, elf/static-stubs.o, instead of -lgcc_eh, is used to
>  # statically link programs.  When --disable-shared is used, we use
> @@ -781,10 +787,12 @@ endif
>  
>  # How to run a program we just linked with our library.
>  # The program binary is assumed to be $(word 2,$^).
> +# We may require additional libraries from gcc (e.g. libgcc_s.so for exception
> +# handling), which unfortunately somewhat breaks the isolation.
>  built-program-file = $(dir $(word 2,$^))$(notdir $(word 2,$^))
>  rtld-prefix = $(elf-objpfx)$(rtld-installed-name)			      \
>  	      --library-path						      \
> -	      $(rpath-link)$(patsubst %,:%,$(sysdep-library-path))
> +	      $(rpath-link)$(patsubst %,:%,$(sysdep-library-path)):$(gnulib-extralibdir)

It solves for the default case where tests are run explicit by issuing the loader,
we still need to add on RPATH for --enable-hardcoded-path-in-tests:

diff --git a/Makeconfig b/Makeconfig
index 6b4ebd1320..0008b8bec0 100644
--- a/Makeconfig
+++ b/Makeconfig
@@ -635,7 +635,7 @@ link-libc-printers-tests = $(link-libc-rpath) \
 # This is how to find at build-time things that will be installed there.
 rpath-dirs = math elf dlfcn nss nis rt resolv mathvec support
 rpath-link = \
-$(common-objdir):$(subst $(empty) ,:,$(patsubst ../$(subdir),.,$(rpath-dirs:%=$(common-objpfx)%)))
+$(common-objdir):$(subst $(empty) ,:,$(patsubst ../$(subdir),.,$(rpath-dirs:%=$(common-objpfx)%))):$(gnulib-extralibdir)
 else  # build-static
 link-libc = $(common-objpfx)libc.a $(otherlibs) $(gnulib) $(common-objpfx)libc.a $(gnulib)
 link-libc-tests = $(common-objpfx)libc.a $(otherlibs) $(gnulib-tests) $(common-objpfx)libc.a $(gnulib-tests)

>  ifeq (yes,$(build-shared))
>  comma = ,
>  sysdep-library-path = \

[1] https://gcc.gnu.org/onlinedocs/gcc/Developer-Options.html
  

* Adhemerval Zanella Netto:

> On 22/01/25 13:12, dilfridge@gentoo.org wrote:
>> From: Andreas K. Hüttel <dilfridge@gentoo.org>
>> 
>> Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
>> ---
>>  Makeconfig | 10 +++++++++-
>>  1 file changed, 9 insertions(+), 1 deletion(-)
>> 
>> diff --git a/Makeconfig b/Makeconfig
>> index d0108d2caa..6b4ebd1320 100644
>> --- a/Makeconfig
>> +++ b/Makeconfig
>> @@ -692,6 +692,11 @@ link-libc-static-tests = -Wl,--start-group $(common-objpfx)libc.a $(static-gnuli
>>  # some cases and it is preferable to link with libgcc_eh or libgcc_s
>>  # so that the testing is as similar as possible to how programs will
>>  # be built with the installed glibc.
>> +# This leads to moderate difficulties, also since distributions may
>> +# install libgcc_s.so in directories only found via ld.so.conf, e.g.
>> +# to be able to switch between gcc versions. We need to add the
>> +# corresponding directory to the library search path to make sure
>> +# our test programs can find it.

This also happens when configuring (accidentally or deliberately)
without --prefix=/usr, so we should really fix that case.

>>  # Some architectures have architecture-specific systems for exception
>>  # handling that may involve undefined references to
>> @@ -709,6 +714,7 @@ libgcc_eh := -Wl,--as-needed -lgcc_s $(libunwind) -Wl,--no-as-needed
>>  gnulib-arch =
>>  gnulib = -lgcc $(gnulib-arch)
>>  gnulib-tests := -lgcc $(libgcc_eh)
>> +gnulib-extralibdir := $(shell $(CC) -print-libgcc-file-name | sed -e 's:/libgcc.*$$::')
>
> From gcc documentation [1], -print-libgcc-file-name is the same as
> -print-file-name=libgcc.a which might be different than the place
> libgcc_s.so is installed.

Not only that.  On many architectures, libgcc_s.so is a linker script in
the same directory as libgcc.a.

If we want to automatically follow what ldconfig does, I think we should
teach ldconfig to print the information we need.  Then this gets solved
after a couple of releases.

Thanks,
Florian
  

On 30/01/25 03:12, Florian Weimer wrote:
> * Adhemerval Zanella Netto:
> 
>> On 22/01/25 13:12, dilfridge@gentoo.org wrote:
>>> From: Andreas K. Hüttel <dilfridge@gentoo.org>
>>>
>>> Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
>>> ---
>>>  Makeconfig | 10 +++++++++-
>>>  1 file changed, 9 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/Makeconfig b/Makeconfig
>>> index d0108d2caa..6b4ebd1320 100644
>>> --- a/Makeconfig
>>> +++ b/Makeconfig
>>> @@ -692,6 +692,11 @@ link-libc-static-tests = -Wl,--start-group $(common-objpfx)libc.a $(static-gnuli
>>>  # some cases and it is preferable to link with libgcc_eh or libgcc_s
>>>  # so that the testing is as similar as possible to how programs will
>>>  # be built with the installed glibc.
>>> +# This leads to moderate difficulties, also since distributions may
>>> +# install libgcc_s.so in directories only found via ld.so.conf, e.g.
>>> +# to be able to switch between gcc versions. We need to add the
>>> +# corresponding directory to the library search path to make sure
>>> +# our test programs can find it.
> 
> This also happens when configuring (accidentally or deliberately)
> without --prefix=/usr, so we should really fix that case.
> 
>>>  # Some architectures have architecture-specific systems for exception
>>>  # handling that may involve undefined references to
>>> @@ -709,6 +714,7 @@ libgcc_eh := -Wl,--as-needed -lgcc_s $(libunwind) -Wl,--no-as-needed
>>>  gnulib-arch =
>>>  gnulib = -lgcc $(gnulib-arch)
>>>  gnulib-tests := -lgcc $(libgcc_eh)
>>> +gnulib-extralibdir := $(shell $(CC) -print-libgcc-file-name | sed -e 's:/libgcc.*$$::')
>>
>> From gcc documentation [1], -print-libgcc-file-name is the same as
>> -print-file-name=libgcc.a which might be different than the place
>> libgcc_s.so is installed.
> 
> Not only that.  On many architectures, libgcc_s.so is a linker script in
> the same directory as libgcc.a.
> 
> If we want to automatically follow what ldconfig does, I think we should
> teach ldconfig to print the information we need.  Then this gets solved
> after a couple of releases.

I think this does not solve if you use a non-default compiler that has its
own libgcc_s.so, for instance if you build it with bmg. For the issue of
the extra libgcc_s.so dependency, I see the most straightforward fix just
to copy it on the builddir (maybe along with libstdc++.so so tests will
also use the library g++ linked against).
  

* Adhemerval Zanella Netto:

>> Not only that.  On many architectures, libgcc_s.so is a linker script in
>> the same directory as libgcc.a.
>> 
>> If we want to automatically follow what ldconfig does, I think we should
>> teach ldconfig to print the information we need.  Then this gets solved
>> after a couple of releases.
>
> I think this does not solve if you use a non-default compiler that has its
> own libgcc_s.so, for instance if you build it with bmg. For the issue of
> the extra libgcc_s.so dependency, I see the most straightforward fix just
> to copy it on the builddir (maybe along with libstdc++.so so tests will
> also use the library g++ linked against).

If we want to copy the file, we should use

  -print-file-name=libgcc_s.so.$(libgcc_s-version)

where $(libgcc_s-version) comes form shlib-versions (see
sysdeps/hppa/shlib-versions for an example).  This way, we get the real
object, and not the linker script.

I think this should workk in a Makefile:

libgcc_s-version = $(shell $(AWK) -F= '/^libgcc_s=/{print $2}' < $(common-objpfx)shlib-versions.v)

Not sure if there's a nicer way to get that version.

Thanks,
Florian
  

On Jan 30 2025, Florian Weimer wrote:

> libgcc_s-version = $(shell $(AWK) -F= '/^libgcc_s=/{print $2}' < $(common-objpfx)shlib-versions.v)

This is already defined by soversions.mk.
  

On 30/01/25 10:33, Florian Weimer wrote:
> * Adhemerval Zanella Netto:
> 
>>> Not only that.  On many architectures, libgcc_s.so is a linker script in
>>> the same directory as libgcc.a.
>>>
>>> If we want to automatically follow what ldconfig does, I think we should
>>> teach ldconfig to print the information we need.  Then this gets solved
>>> after a couple of releases.
>>
>> I think this does not solve if you use a non-default compiler that has its
>> own libgcc_s.so, for instance if you build it with bmg. For the issue of
>> the extra libgcc_s.so dependency, I see the most straightforward fix just
>> to copy it on the builddir (maybe along with libstdc++.so so tests will
>> also use the library g++ linked against).
> 
> If we want to copy the file, we should use
> 
>   -print-file-name=libgcc_s.so.$(libgcc_s-version)
> 
> where $(libgcc_s-version) comes form shlib-versions (see
> sysdeps/hppa/shlib-versions for an example).  This way, we get the real
> object, and not the linker script.


We can also use '-print-file-name=libgcc_s.so.$(libgcc_s-version)' with
the original approach to add in on either the --library-path or RPATH.

I think we will need add a bit more machinery to handle the container
tests (since we will need an extra copy rule to handle non system
libraries).

> 
> I think this should workk in a Makefile:
> 
> libgcc_s-version = $(shell $(AWK) -F= '/^libgcc_s=/{print $2}' < $(common-objpfx)shlib-versions.v)
> 
> Not sure if there's a nicer way to get that version.
> 
> Thanks,
> Florian
>
  

Am Mittwoch, 29. Januar 2025, 20:55:28 Mitteleuropäische Normalzeit schrieb Adhemerval Zanella Netto:
> 
> On 22/01/25 13:12, dilfridge@gentoo.org wrote:
> > From: Andreas K. Hüttel <dilfridge@gentoo.org>
> > 
> > Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
> > ---
> >  Makeconfig | 10 +++++++++-
> >  1 file changed, 9 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Makeconfig b/Makeconfig
> > index d0108d2caa..6b4ebd1320 100644
> > --- a/Makeconfig
> > +++ b/Makeconfig
> > @@ -692,6 +692,11 @@ link-libc-static-tests = -Wl,--start-group $(common-objpfx)libc.a $(static-gnuli
> >  # some cases and it is preferable to link with libgcc_eh or libgcc_s
> >  # so that the testing is as similar as possible to how programs will
> >  # be built with the installed glibc.
> > +# This leads to moderate difficulties, also since distributions may
> > +# install libgcc_s.so in directories only found via ld.so.conf, e.g.
> > +# to be able to switch between gcc versions. We need to add the
> > +# corresponding directory to the library search path to make sure
> > +# our test programs can find it.
> >  #
> >  # Some architectures have architecture-specific systems for exception
> >  # handling that may involve undefined references to
> > @@ -709,6 +714,7 @@ libgcc_eh := -Wl,--as-needed -lgcc_s $(libunwind) -Wl,--no-as-needed
> >  gnulib-arch =
> >  gnulib = -lgcc $(gnulib-arch)
> >  gnulib-tests := -lgcc $(libgcc_eh)
> > +gnulib-extralibdir := $(shell $(CC) -print-libgcc-file-name | sed -e 's:/libgcc.*$$::')
> 
> From gcc documentation [1], -print-libgcc-file-name is the same as -print-file-name=libgcc.a
> which might be different than the place libgcc_s.so is installed.
> 
> On a toolchain built with build-many-glibc.py, I see:
> 
> $ arm-glibc-linux-gnueabihf-gcc -print-libgcc-file-name | xargs realpath
> <path>/install/compilers/14/arm-linux-gnueabihf/lib/gcc/arm-glibc-linux-gnueabihf/14.2.1/libgcc.a
> 
> $ arm-glibc-linux-gnueabihf-gcc -print-file-name=libgcc_s.so | xargs realpath
> <path>/install/compilers/14/arm-linux-gnueabihf/arm-glibc-linux-gnueabihf/lib/libgcc_s.so
> 
> So I think we should -print-file-name=libgcc_s.so instead here, and maybe 'dirname'
> instead of sed (just to be a bit more clear).

Makes sense.

> 
> >  static-gnulib-arch =
> >  # By default, elf/static-stubs.o, instead of -lgcc_eh, is used to
> >  # statically link programs.  When --disable-shared is used, we use
> > @@ -781,10 +787,12 @@ endif
> >  
> >  # How to run a program we just linked with our library.
> >  # The program binary is assumed to be $(word 2,$^).
> > +# We may require additional libraries from gcc (e.g. libgcc_s.so for exception
> > +# handling), which unfortunately somewhat breaks the isolation.
> >  built-program-file = $(dir $(word 2,$^))$(notdir $(word 2,$^))
> >  rtld-prefix = $(elf-objpfx)$(rtld-installed-name)			      \
> >  	      --library-path						      \
> > -	      $(rpath-link)$(patsubst %,:%,$(sysdep-library-path))
> > +	      $(rpath-link)$(patsubst %,:%,$(sysdep-library-path)):$(gnulib-extralibdir)
> 
> It solves for the default case where tests are run explicit by issuing the loader,
> we still need to add on RPATH for --enable-hardcoded-path-in-tests:
> 
> diff --git a/Makeconfig b/Makeconfig
> index 6b4ebd1320..0008b8bec0 100644
> --- a/Makeconfig
> +++ b/Makeconfig
> @@ -635,7 +635,7 @@ link-libc-printers-tests = $(link-libc-rpath) \
>  # This is how to find at build-time things that will be installed there.
>  rpath-dirs = math elf dlfcn nss nis rt resolv mathvec support
>  rpath-link = \
> -$(common-objdir):$(subst $(empty) ,:,$(patsubst ../$(subdir),.,$(rpath-dirs:%=$(common-objpfx)%)))
> +$(common-objdir):$(subst $(empty) ,:,$(patsubst ../$(subdir),.,$(rpath-dirs:%=$(common-objpfx)%))):$(gnulib-extralibdir)
>  else  # build-static
>  link-libc = $(common-objpfx)libc.a $(otherlibs) $(gnulib) $(common-objpfx)libc.a $(gnulib)
>  link-libc-tests = $(common-objpfx)libc.a $(otherlibs) $(gnulib-tests) $(common-objpfx)libc.a $(gnulib-tests)
> 

Ah, OK, this was not in my field of vision. Makes sense.

> >  ifeq (yes,$(build-shared))
> >  comma = ,
> >  sysdep-library-path = \
> 
> [1] https://gcc.gnu.org/onlinedocs/gcc/Developer-Options.html
>
  

Am Donnerstag, 30. Januar 2025, 17:26:30 Mitteleuropäische Normalzeit schrieb Adhemerval Zanella Netto:
> 
> On 30/01/25 10:33, Florian Weimer wrote:
> > * Adhemerval Zanella Netto:
> > 
> >>> Not only that.  On many architectures, libgcc_s.so is a linker script in
> >>> the same directory as libgcc.a.
> >>>
> >>> If we want to automatically follow what ldconfig does, I think we should
> >>> teach ldconfig to print the information we need.  Then this gets solved
> >>> after a couple of releases.
> >>
> >> I think this does not solve if you use a non-default compiler that has its
> >> own libgcc_s.so, for instance if you build it with bmg. For the issue of
> >> the extra libgcc_s.so dependency, I see the most straightforward fix just
> >> to copy it on the builddir (maybe along with libstdc++.so so tests will
> >> also use the library g++ linked against).
> > 
> > If we want to copy the file, we should use
> > 
> >   -print-file-name=libgcc_s.so.$(libgcc_s-version)
> > 
> > where $(libgcc_s-version) comes form shlib-versions (see
> > sysdeps/hppa/shlib-versions for an example).  This way, we get the real
> > object, and not the linker script.
> 
> 
> We can also use '-print-file-name=libgcc_s.so.$(libgcc_s-version)' with
> the original approach to add in on either the --library-path or RPATH.
> 
> I think we will need add a bit more machinery to handle the container
> tests (since we will need an extra copy rule to handle non system
> libraries).

Actually this patch series was for (and works with) the container tests. 

The trick is the following:
* Patch 1/4 adds the directory where libgcc_s.so is to the search path of
  the loader invocation used in the test suite.
* The container tests work as follows, they run ldd and dump the list of 
  needed libraries, then copy these into the container.
* Without 1/4, ldd does not find libgcc_s.so, resulting in some "not found"
  output and no copying of the library.
* With 1/4, the library is picked up independent of its location (as long
  as the directory is provided) and copied into the testroot.

So extra logic for copying the library is not needed as long as we tell
ldd where it can search (via the ld.so --libdir).

Since the container technique replicates the paths of the main system, 
patches 2/4 and 3/4 ensure the library is also found in the container then
(its location may not be in the ld.so default search path there).


> 
> > 
> > I think this should workk in a Makefile:
> > 
> > libgcc_s-version = $(shell $(AWK) -F= '/^libgcc_s=/{print $2}' < $(common-objpfx)shlib-versions.v)
> > 
> > Not sure if there's a nicer way to get that version.
> > 
> > Thanks,
> > Florian
> > 
> 
>