test-container: disable ld.so system cache on DSO detection

Message ID 20231005125431.3958485-1-simon.chopin@canonical.com
State Committed
Commit 97290559c3b497fb9012c3f6248cb30afb26da7c
Headers
Series test-container: disable ld.so system cache on DSO detection |

Checks

Context Check Description
redhat-pt-bot/TryBot-apply_patch success Patch applied to master at the time it was sent
redhat-pt-bot/TryBot-32bit success Build for i686
linaro-tcwg-bot/tcwg_glibc_build--master-arm success Testing passed
linaro-tcwg-bot/tcwg_glibc_check--master-aarch64 success Testing passed
linaro-tcwg-bot/tcwg_glibc_check--master-arm success Testing passed
linaro-tcwg-bot/tcwg_glibc_build--master-aarch64 success Testing passed

Commit Message

Simon Chopin Oct. 5, 2023, 12:54 p.m. UTC
  When building the testroot, the script runs the newly built ld.so on a
couple of binaries in order to copy over any additional libararies
needed. However, if the dependencies are found in the system cache, it
will be copied over using that path.

This is problematic if the system ld.so and the one built don't have the
exact same search configuration. We encountered this in Ubuntu, where we
build a variant of libc with -fno-omit-frame-pointer for accurate
performance profiling.

This variant is built using a non-standard slibdir to be able to be
co-installed with the default library (e.g. slibdir = /lib/libc6-prof).
Since we have /lib pointing to /usr/lib, any additional dependency
should still be reachable via /usr. However, resolving via the cache
might result in the additional DSOs being copied into $testroot/lib, out
of the search path in the container.

The problem has been triggered by 1d5024f4f052c12e404d42d3b5bfe9c3e9fd27c4
("support: Build with exceptions and asynchronous unwind tables [BZ #30587]")
which introduced a dependency on libgcc_s.so.1 under some circumstances.

Downstream bug: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/2031495
---
 Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


base-commit: 1056e5b4c3f2d90ed2b4a55f96add28da2f4c8fa
  

Comments

Adhemerval Zanella Netto Oct. 17, 2023, 2:48 p.m. UTC | #1
On 05/10/23 09:54, Simon Chopin wrote:
> When building the testroot, the script runs the newly built ld.so on a
> couple of binaries in order to copy over any additional libararies

s/libararies/libraries

> needed. However, if the dependencies are found in the system cache, it
> will be copied over using that path.
> 
> This is problematic if the system ld.so and the one built don't have the
> exact same search configuration. We encountered this in Ubuntu, where we
> build a variant of libc with -fno-omit-frame-pointer for accurate
> performance profiling.
> 
> This variant is built using a non-standard slibdir to be able to be
> co-installed with the default library (e.g. slibdir = /lib/libc6-prof).
> Since we have /lib pointing to /usr/lib, any additional dependency
> should still be reachable via /usr. However, resolving via the cache
> might result in the additional DSOs being copied into $testroot/lib, out
> of the search path in the container.
> 
> The problem has been triggered by 1d5024f4f052c12e404d42d3b5bfe9c3e9fd27c4
> ("support: Build with exceptions and asynchronous unwind tables [BZ #30587]")
> which introduced a dependency on libgcc_s.so.1 under some circumstances.
> 
> Downstream bug: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/2031495

It makes sense to inhibit cache on testroot creation, although default
system dirs will always be used.

LGTM, thanks.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>

> ---
>  Makefile | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index c6d4817a9e..b938721166 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -624,7 +624,7 @@ $(objpfx)testroot.pristine/install.stamp :
>  ifeq ($(run-built-tests),yes)
>  	# Copy these DSOs first so we can overwrite them with our own.
>  	for dso in `$(test-wrapper-env) LD_TRACE_LOADED_OBJECTS=1  \
> -		$(rtld-prefix) \
> +		$(rtld-prefix) --inhibit-cache \
>  		$(objpfx)testroot.pristine/bin/sh \
>  	        | sed -n '/\//{s@.*=> /@/@;s/^[^/]*//;s/ .*//p;}'` ;\
>  	  do \
> @@ -633,7 +633,7 @@ ifeq ($(run-built-tests),yes)
>  	    $(test-wrapper) cp $$dso $(objpfx)testroot.pristine$$dso ;\
>  	  done
>  	for dso in `$(test-wrapper-env) LD_TRACE_LOADED_OBJECTS=1  \
> -		$(rtld-prefix) \
> +		$(rtld-prefix) --inhibit-cache \
>  		$(objpfx)support/$(LINKS_DSO_PROGRAM) \
>  	        | sed -n '/\//{s@.*=> /@/@;s/^[^/]*//;s/ .*//p;}'` ;\
>  	  do \
> 
> base-commit: 1056e5b4c3f2d90ed2b4a55f96add28da2f4c8fa
  
Simon Chopin Oct. 23, 2023, 10:11 a.m. UTC | #2
Hi!

Quoting Adhemerval Zanella Netto (2023-10-17 16:48:55)
>
>
> On 05/10/23 09:54, Simon Chopin wrote:
> > When building the testroot, the script runs the newly built ld.so on a
> > couple of binaries in order to copy over any additional libararies
>
> s/libararies/libraries
>
> > needed. However, if the dependencies are found in the system cache, it
> > will be copied over using that path.
> >
> > This is problematic if the system ld.so and the one built don't have the
> > exact same search configuration. We encountered this in Ubuntu, where we
> > build a variant of libc with -fno-omit-frame-pointer for accurate
> > performance profiling.
> >
> > This variant is built using a non-standard slibdir to be able to be
> > co-installed with the default library (e.g. slibdir = /lib/libc6-prof).
> > Since we have /lib pointing to /usr/lib, any additional dependency
> > should still be reachable via /usr. However, resolving via the cache
> > might result in the additional DSOs being copied into $testroot/lib, out
> > of the search path in the container.
> >
> > The problem has been triggered by 1d5024f4f052c12e404d42d3b5bfe9c3e9fd27c4
> > ("support: Build with exceptions and asynchronous unwind tables [BZ #30587]")
> > which introduced a dependency on libgcc_s.so.1 under some circumstances.
> >
> > Downstream bug: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/2031495
>
> It makes sense to inhibit cache on testroot creation, although default
> system dirs will always be used.
>
> LGTM, thanks.
>
> Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>

Sorry, my experience of ML-based contributions is pretty limited. Am I
expected to send a V2 to fix the typo in the commit log?

If I need to send a V2, should it include the Reviewed-by tag?
  
Adhemerval Zanella Netto Oct. 23, 2023, 2:08 p.m. UTC | #3
On 23/10/23 07:11, Simon Chopin wrote:
> Hi!
> 
> Quoting Adhemerval Zanella Netto (2023-10-17 16:48:55)
>>
>>
>> On 05/10/23 09:54, Simon Chopin wrote:
>>> When building the testroot, the script runs the newly built ld.so on a
>>> couple of binaries in order to copy over any additional libararies
>>
>> s/libararies/libraries
>>
>>> needed. However, if the dependencies are found in the system cache, it
>>> will be copied over using that path.
>>>
>>> This is problematic if the system ld.so and the one built don't have the
>>> exact same search configuration. We encountered this in Ubuntu, where we
>>> build a variant of libc with -fno-omit-frame-pointer for accurate
>>> performance profiling.
>>>
>>> This variant is built using a non-standard slibdir to be able to be
>>> co-installed with the default library (e.g. slibdir = /lib/libc6-prof).
>>> Since we have /lib pointing to /usr/lib, any additional dependency
>>> should still be reachable via /usr. However, resolving via the cache
>>> might result in the additional DSOs being copied into $testroot/lib, out
>>> of the search path in the container.
>>>
>>> The problem has been triggered by 1d5024f4f052c12e404d42d3b5bfe9c3e9fd27c4
>>> ("support: Build with exceptions and asynchronous unwind tables [BZ #30587]")
>>> which introduced a dependency on libgcc_s.so.1 under some circumstances.
>>>
>>> Downstream bug: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/2031495
>>
>> It makes sense to inhibit cache on testroot creation, although default
>> system dirs will always be used.
>>
>> LGTM, thanks.
>>
>> Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
> 
> Sorry, my experience of ML-based contributions is pretty limited. Am I
> expected to send a V2 to fix the typo in the commit log?
> 
> If I need to send a V2, should it include the Reviewed-by tag?

No need, I will commit in your behalf with the commit message fixed.
  

Patch

diff --git a/Makefile b/Makefile
index c6d4817a9e..b938721166 100644
--- a/Makefile
+++ b/Makefile
@@ -624,7 +624,7 @@  $(objpfx)testroot.pristine/install.stamp :
 ifeq ($(run-built-tests),yes)
 	# Copy these DSOs first so we can overwrite them with our own.
 	for dso in `$(test-wrapper-env) LD_TRACE_LOADED_OBJECTS=1  \
-		$(rtld-prefix) \
+		$(rtld-prefix) --inhibit-cache \
 		$(objpfx)testroot.pristine/bin/sh \
 	        | sed -n '/\//{s@.*=> /@/@;s/^[^/]*//;s/ .*//p;}'` ;\
 	  do \
@@ -633,7 +633,7 @@  ifeq ($(run-built-tests),yes)
 	    $(test-wrapper) cp $$dso $(objpfx)testroot.pristine$$dso ;\
 	  done
 	for dso in `$(test-wrapper-env) LD_TRACE_LOADED_OBJECTS=1  \
-		$(rtld-prefix) \
+		$(rtld-prefix) --inhibit-cache \
 		$(objpfx)support/$(LINKS_DSO_PROGRAM) \
 	        | sed -n '/\//{s@.*=> /@/@;s/^[^/]*//;s/ .*//p;}'` ;\
 	  do \