[RFC,v11,0/7] Implementation of RTLD_SHARED for dlmopen

  This is a revision of a previous patchset that I posted here
regarding https://sourceware.org/bugzilla/show_bug.cgi?id=22745 

Introduction:

=======================================================================
  As discussed in the URL above dlmopen requires a mechanism for
  [optionally] sharing some objects between more than one namespace.

  The following patchset provides an implementation for this: If an
  object is loaded with the new RTLD_SHARED flag we instead ensure
  that a "master" copy exists (and is flagged as no-delete) in the
  main namespace and a thin wrapper or clone is placed in the target
  namespace.

  This patch series should address all the comments received on the
  earlier (v1-v10) series.

=======================================================================

Changes from v10:

 - A segfault in a dlmopen error pathway (which does not seem to have existed
   when v10 was applied to the then-HEAD commit) has been fixed.

 - The fallback mechanism for adding DT_GNU_FLAGS_1 sections to the required
   binaries has had some infrastructure moved to the elf/ directory

 - The runstatedir setting introduced by autoconf has been omitted from
   the patchset as it is not relevant to this feature.

Not changed:

 - There is still some diagnostic info in the config.log when the linker
   layout is acceptable but -z unique is not yet supported. I believe this
   _is_ useful diagnostic information as a developer might otherwise wonder
   why the vanilla linker was being rejected when its layout output seemed
   fine.

 - libpthread still has the new section applied as I'm not 100% sure it doesn't
   still need this treatment. I would like to investigate further before
   removing the new section from libpthread.

I have not yet implemented, but plan to address once this series is
accepted/acceptable:

 - Sensible RTLD_GLOBAL semantics for dlmopened DSOs in non-base namespaces

 - dl_iterate_ns_phdr (cf dl_iterate_phdr but taking a namespace argument)

Vivek Das Mohapatra (7):
  Define a new dynamic section tag - DT_GNU_FLAGS_1 (bug 22745)
  Abstract loaded-DSO search code into a helper function
  Use the new DSO finder helper function
  Add DT_GNU_FLAGS_1/DF_GNU_1_UNIQUE to glibc DSOs (bug 22745)
  Implement dlmopen RTLD_SHARED flag (bug 22745)
  Add dlmopen / RTLD_SHARED tests
  Restore separate libc loading for the TLS/namespace storage test

 Makeconfig                           |   3 +
 Makerules                            |  18 +-
 bits/dlfcn.h                         |  10 +
 config.make.in                       |   1 +
 configure                            |  42 +-
 configure.ac                         |  31 +-
 elf/Makefile                         | 104 +++-
 elf/dl-close.c                       |  43 +-
 elf/dl-fini.c                        |   6 +-
 elf/dl-init.c                        |   4 +-
 elf/dl-load.c                        | 223 ++++++-
 elf/dl-object.c                      |  78 +++
 elf/dl-open.c                        | 116 +++-
 elf/dl-sym.c                         |  14 +
 elf/dynamic-notes.c                  |   4 +
 elf/elf.h                            |   7 +-
 elf/get-dynamic-info.h               |  12 +
 elf/rtld.c                           |   2 +-
 elf/tst-dlmopen-auditmod.c           |  23 +
 elf/tst-dlmopen-common.h             |  32 +
 elf/tst-dlmopen-main.h               | 873 +++++++++++++++++++++++++++
 elf/tst-dlmopen-modules.h            |  21 +
 elf/tst-dlmopen-rtld-audit-shared1.c |   8 +
 elf/tst-dlmopen-rtld-audit-shared2.c |   8 +
 elf/tst-dlmopen-rtld-audit-shared3.c |   7 +
 elf/tst-dlmopen-rtld-audit-shared4.c |   8 +
 elf/tst-dlmopen-rtld-audit-shared5.c |   8 +
 elf/tst-dlmopen-rtld-audit-shared6.c |   8 +
 elf/tst-dlmopen-rtld-audit-unique1.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique2.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique3.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique4.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique5.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique6.c |   7 +
 elf/tst-dlmopen-rtld-shared1.c       |   7 +
 elf/tst-dlmopen-rtld-shared1.h       |  64 ++
 elf/tst-dlmopen-rtld-shared2.c       |   7 +
 elf/tst-dlmopen-rtld-shared2.h       |  66 ++
 elf/tst-dlmopen-rtld-shared3.c       |   7 +
 elf/tst-dlmopen-rtld-shared3.h       |  43 ++
 elf/tst-dlmopen-rtld-shared4.c       |   7 +
 elf/tst-dlmopen-rtld-shared4.h       |  14 +
 elf/tst-dlmopen-rtld-shared5.c       |   7 +
 elf/tst-dlmopen-rtld-shared5.h       |  25 +
 elf/tst-dlmopen-rtld-shared6.c       |   7 +
 elf/tst-dlmopen-rtld-shared6.h       |  36 ++
 elf/tst-dlmopen-rtld-unique1.c       |   7 +
 elf/tst-dlmopen-rtld-unique1.h       |  86 +++
 elf/tst-dlmopen-rtld-unique2.c       |   7 +
 elf/tst-dlmopen-rtld-unique2.h       |  25 +
 elf/tst-dlmopen-rtld-unique3.c       |   7 +
 elf/tst-dlmopen-rtld-unique3.h       |  13 +
 elf/tst-dlmopen-rtld-unique4.c       |   7 +
 elf/tst-dlmopen-rtld-unique4.h       |  14 +
 elf/tst-dlmopen-rtld-unique5.c       |   7 +
 elf/tst-dlmopen-rtld-unique5.h       |  58 ++
 elf/tst-dlmopen-rtld-unique6.c       |   7 +
 elf/tst-dlmopen-rtld-unique6.h       |  51 ++
 elf/tst-dlmopen-sharedmod-norm.c     |  11 +
 elf/tst-dlmopen-sharedmod-uniq.c     |  11 +
 elf/tst-dlmopen-std-do-test.c        |  12 +
 elf/tst-tls-ie-dlmopen.c             |   4 +-
 extra-lib.mk                         |   3 +
 htl/Makefile                         |   3 +
 iconvdata/Makefile                   |   3 +
 iconvdata/extra-module.mk            |   4 +
 include/elf.h                        |   2 +
 include/link.h                       |   7 +-
 nptl/Makefile                        |   7 +-
 sysdeps/generic/ldsodefs.h           |   9 +
 sysdeps/mips/bits/dlfcn.h            |  10 +
 71 files changed, 2349 insertions(+), 72 deletions(-)
 create mode 100644 elf/dynamic-notes.c
 create mode 100644 elf/tst-dlmopen-auditmod.c
 create mode 100644 elf/tst-dlmopen-common.h
 create mode 100644 elf/tst-dlmopen-main.h
 create mode 100644 elf/tst-dlmopen-modules.h
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared1.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared2.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared3.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared4.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared5.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared6.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique1.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique2.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique3.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique4.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique5.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique6.c
 create mode 100644 elf/tst-dlmopen-rtld-shared1.c
 create mode 100644 elf/tst-dlmopen-rtld-shared1.h
 create mode 100644 elf/tst-dlmopen-rtld-shared2.c
 create mode 100644 elf/tst-dlmopen-rtld-shared2.h
 create mode 100644 elf/tst-dlmopen-rtld-shared3.c
 create mode 100644 elf/tst-dlmopen-rtld-shared3.h
 create mode 100644 elf/tst-dlmopen-rtld-shared4.c
 create mode 100644 elf/tst-dlmopen-rtld-shared4.h
 create mode 100644 elf/tst-dlmopen-rtld-shared5.c
 create mode 100644 elf/tst-dlmopen-rtld-shared5.h
 create mode 100644 elf/tst-dlmopen-rtld-shared6.c
 create mode 100644 elf/tst-dlmopen-rtld-shared6.h
 create mode 100644 elf/tst-dlmopen-rtld-unique1.c
 create mode 100644 elf/tst-dlmopen-rtld-unique1.h
 create mode 100644 elf/tst-dlmopen-rtld-unique2.c
 create mode 100644 elf/tst-dlmopen-rtld-unique2.h
 create mode 100644 elf/tst-dlmopen-rtld-unique3.c
 create mode 100644 elf/tst-dlmopen-rtld-unique3.h
 create mode 100644 elf/tst-dlmopen-rtld-unique4.c
 create mode 100644 elf/tst-dlmopen-rtld-unique4.h
 create mode 100644 elf/tst-dlmopen-rtld-unique5.c
 create mode 100644 elf/tst-dlmopen-rtld-unique5.h
 create mode 100644 elf/tst-dlmopen-rtld-unique6.c
 create mode 100644 elf/tst-dlmopen-rtld-unique6.h
 create mode 100644 elf/tst-dlmopen-sharedmod-norm.c
 create mode 100644 elf/tst-dlmopen-sharedmod-uniq.c
 create mode 100644 elf/tst-dlmopen-std-do-test.c
  

* Vivek Das Mohapatra via Libc-alpha:

>  - libpthread still has the new section applied as I'm not 100% sure it doesn't
>    still need this treatment. I would like to investigate further before
>    removing the new section from libpthread.

libpthread does not need this anymore.  We can safely load it as many
times as we want.

librt still needs it because it's not empty and has global state (for
handling the thread notifications).

Thanks,
Florian
  

On Wed, 9 Jun 2021, Florian Weimer wrote:

>>  - libpthread still has the new section applied as I'm not 100% sure it doesn't
>>    still need this treatment. I would like to investigate further before
>
> libpthread does not need this anymore.  We can safely load it as many
> times as we want.

Ok, thanks. I'll update the patch series.
  

I'm testing these patches for our use case in GraalVM. We use dlmopen to load several isolated JVMs within the same process.

With the patches I'm hitting the following error:
Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assert
ion `needed != NULL' failed!

Here's the tail of the LD_DEBUG=all log:
   222389:       trying file=./nptl/tls/x86_64/libpthread.so.0
   222389:       trying file=./nptl/tls/libpthread.so.0
   222389:       trying file=./nptl/x86_64/x86_64/libpthread.so.0
   222389:       trying file=./nptl/x86_64/libpthread.so.0
   222389:       trying file=./nptl/x86_64/libpthread.so.0
   222389:       trying file=./nptl/libpthread.so.0
   222389:
   222389:     checking for version `GLIBC_2.2.5' in file ./dlfcn/libdl.so.2 [0]
required by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_E
SPRESSO_JVM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1]
   222389:     checking for version `GLIBC_2.2.5' in file ./nptl/libpthread.so.0
[0] required by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAAL
VM_ESPRESSO_JVM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so
[1]
   222389:     checking for version `GLIBC_2.3' in file ./libc.so.6 [0] required
by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_ESPRESSO_J
VM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1]
   222389:     checking for version `GLIBC_2.3.2' in file ./libc.so.6 [0] require
d by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_ESPRESSO
_JVM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1]
   222389:     checking for version `GLIBC_2.4' in file ./libc.so.6 [0] required
by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_ESPRESSO_J
VM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1]
   222389:     checking for version `GLIBC_2.2.5' in file ./libc.so.6 [0] require
d by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_ESPRESSO
_JVM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1]
Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assert
ion `needed != NULL' failed!


We use dlmopen a lot, but never seen this error before, hopefully you can pinpoint the issue. The program in question runs fine without the patches.
I'm running with ./testrun.sh .
I can provide a reproducer, but the setup is quite cumbersome. I can also test any further patches locally if needed.
We've been following this feature for a long time (with great excitement) and just wanted to make sure it works (or that at least doesn't break us) before it's merged.

Best,
Alfonso
  

On Wed, 16 Jun 2021, Alfonso Alfonso Peterssen wrote:

> I'm testing these patches for our use case in GraalVM. We use dlmopen to load several isolated JVMs within the same process.
>
> With the patches I'm hitting the following error:
> Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assert
> ion `needed != NULL' failed!

Digging into it.

If I can't figure it out I'll ask you for a reproducer.