[v14,0/7] Implementation of RTLD_SHARED for dlmopen

  This is a revision of a previous patchset that I posted here
regarding https://sourceware.org/bugzilla/show_bug.cgi?id=22745 

Introduction:

=======================================================================
  As discussed in the URL above dlmopen requires a mechanism for
  [optionally] sharing some objects between more than one namespace.

  The following patchset provides an implementation for this: If an
  object is loaded with the new RTLD_SHARED flag we instead ensure
  that a "master" copy exists (and is flagged as no-delete) in the
  main namespace and a thin wrapper or clone is placed in the target
  namespace.

  In addition a new ELF dynamic section and flag are introduced which
  indicate that the DSO in question should be shared between namespaces
  by default, and the core libc DSOs are tagged as such by default.

  This patch series should address all the comments received on the
  earlier (v1-v13) series.

=======================================================================

Changes from v13:

 - The following DSOs no longer have the new flag since they are stubs:
   nptl/libpthread, dlfcn/libdl, login/libutil, resolv/libanl

 - htl/libpthread still flagged, since htl pthread has not been merged

 - NOTE: resolv/libanl depends on {htl,nptl}/pthread but is not flagged
   which I believe to be the correct state of affairs

 - Tests now use a purpose built DT_GNU_FLAGS_1/DF_GNU_1_UNIQUE
   library to make sure DT_NEEDED support works (they used librt before
   but we no longer assume librt will keep being flagged)

 - Patchset updated to cope with dl_open_worker being split into two
   functions.

 - Incorporated most of the changes from azanella/rtld-shared, EXCEPT:

   - Makeconfig variable ld-zunique/ld_zunique change propagated to the
     rest of the build system so that DT_GNU_FLAGS_1 is added when the
     linker has -z unique support (as well).

   - dl-lookup.c had: current_value.m->l_name == LM_ID_BASE
     corrected to:    current_value.m->l_ns == LM_ID_BASE

   - dl-open,c: A check on whether GL(dl_ns)[args->nsid].libc_map had been set
     had been removed, with the comment: "dlopen cannot be used to load an 
     initial libc by design", but that was not what this check was for.

     In a secondary namespace it is possible for there to be no libc_map stored
     at that location, so we should not unconditionally call libc init on it.

   - Not all changes to the tests incorporated as they would have inverted
     the return values of at least one helper function, with knock-on changes
     being required as a result.

Changes from v12:

 - Added NEWS entry.

 - Updated patch series to glibc 2.35 as a base.

 - The following DSOs no longer have the new flag since they are stubs:
   {htl,nptl}/libpthread, dlfcn/libdl, login/libutil, resolv/libanl

Changes from v11:

 - If a DSO is required in a non-base namespace because it is mentioned
   in a DT_NEEDED entry and it is itself flagged DF_GNU_1_UNIQUE then
   a proxy is generated for it.

 - Relocations via non-base namespace proxies work reliably (some code
   paths did not do the address calculation relative to the DSO base
   correctly when a proxy was involved).

 - Tests extended cover the above two scenarios.

Not changed:

 - There is still some diagnostic info in the config.log when the linker
   layout is acceptable but -z unique is not yet supported. I believe this
   _is_ useful diagnostic information as a developer might otherwise wonder
   why the vanilla linker was being rejected when its layout output seemed
   fine.

I have not yet implemented, but plan to address once this series is
accepted/acceptable:

 - dl_iterate_ns_phdr (cf dl_iterate_phdr but taking a namespace argument)

 - Check RTLD_GLOBAL interacts properly and unsurprisingly with RTLD_SHARED.

Vivek Das Mohapatra (7):
  Define a new dynamic section tag - DT_GNU_FLAGS_1 (bug 22745)
  Abstract loaded-DSO search code into a helper function
  Use the new DSO finder helper function
  Add DT_GNU_FLAGS_1/DF_GNU_1_UNIQUE to glibc DSOs (bug 22745)
  Implement dlmopen RTLD_SHARED flag (bug 22745)
  Add dlmopen / RTLD_SHARED tests
  NEWS for RTLD_SHARED, RTLD_ISOLATE & DT_GNU_FLAGS_1/DF_GNU_1_UNIQUE

 Makeconfig                           |   3 +
 Makerules                            |  18 +-
 NEWS                                 |  17 +
 bits/dlfcn.h                         |  10 +
 configure                            |  50 +-
 configure.ac                         |  24 +-
 dlfcn/Makefile                       |   1 +
 elf/Makefile                         | 110 +++-
 elf/dl-close.c                       |  41 +-
 elf/dl-deps.c                        |  17 +
 elf/dl-fini.c                        |   6 +-
 elf/dl-init.c                        |   4 +-
 elf/dl-load.c                        | 236 +++++--
 elf/dl-lookup.c                      |  23 +-
 elf/dl-object.c                      |  83 +++
 elf/dl-open.c                        | 162 ++++-
 elf/dl-sym.c                         |  14 +
 elf/dynamic-notes.c                  |   4 +
 elf/elf.h                            |   7 +-
 elf/get-dynamic-info.h               |  12 +
 elf/rtld.c                           |   2 +-
 elf/tst-dlmopen-auditmod.c           |  23 +
 elf/tst-dlmopen-common.h             |  38 ++
 elf/tst-dlmopen-main.h               | 928 +++++++++++++++++++++++++++
 elf/tst-dlmopen-rtld-audit-shared1.c |   7 +
 elf/tst-dlmopen-rtld-audit-shared2.c |   7 +
 elf/tst-dlmopen-rtld-audit-shared3.c |   7 +
 elf/tst-dlmopen-rtld-audit-shared4.c |   7 +
 elf/tst-dlmopen-rtld-audit-shared5.c |   7 +
 elf/tst-dlmopen-rtld-audit-shared6.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique1.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique2.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique3.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique4.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique5.c |   7 +
 elf/tst-dlmopen-rtld-audit-unique6.c |   7 +
 elf/tst-dlmopen-rtld-shared1.c       |   7 +
 elf/tst-dlmopen-rtld-shared1.h       |  64 ++
 elf/tst-dlmopen-rtld-shared2.c       |   7 +
 elf/tst-dlmopen-rtld-shared2.h       |  66 ++
 elf/tst-dlmopen-rtld-shared3.c       |   7 +
 elf/tst-dlmopen-rtld-shared3.h       |  43 ++
 elf/tst-dlmopen-rtld-shared4.c       |   7 +
 elf/tst-dlmopen-rtld-shared4.h       |  14 +
 elf/tst-dlmopen-rtld-shared5.c       |   7 +
 elf/tst-dlmopen-rtld-shared5.h       |  25 +
 elf/tst-dlmopen-rtld-shared6.c       |   7 +
 elf/tst-dlmopen-rtld-shared6.h       |  36 ++
 elf/tst-dlmopen-rtld-unique1.c       |   7 +
 elf/tst-dlmopen-rtld-unique1.h       |  86 +++
 elf/tst-dlmopen-rtld-unique2.c       |   7 +
 elf/tst-dlmopen-rtld-unique2.h       |  25 +
 elf/tst-dlmopen-rtld-unique3.c       |   7 +
 elf/tst-dlmopen-rtld-unique3.h       |  13 +
 elf/tst-dlmopen-rtld-unique4.c       |   7 +
 elf/tst-dlmopen-rtld-unique4.h       |  14 +
 elf/tst-dlmopen-rtld-unique5.c       |   7 +
 elf/tst-dlmopen-rtld-unique5.h       |  58 ++
 elf/tst-dlmopen-rtld-unique6.c       |   7 +
 elf/tst-dlmopen-rtld-unique6.h       |  51 ++
 elf/tst-dlmopen-sharedmod-norm.c     |  37 ++
 elf/tst-dlmopen-sharedmod-uniq.c     |  37 ++
 elf/tst-dlmopen-std-do-test.c        |  12 +
 elf/tst-dlmopen-unique-dep.c         |   6 +
 elf/tst-dlmopen-unique-dep.h         |   6 +
 elf/tst-tls-ie-dlmopen.c             |   7 +-
 extra-lib.mk                         |  31 +
 htl/Makefile                         |   2 +
 iconvdata/Makefile                   |   3 +
 iconvdata/extra-module.mk            |   4 +
 include/elf.h                        |   2 +
 include/link.h                       |   7 +-
 login/Makefile                       |   1 +
 nptl/Makefile                        |   5 +-
 resolv/Makefile                      |   2 +
 sysdeps/generic/ldsodefs.h           |  11 +
 sysdeps/mips/bits/dlfcn.h            |  10 +
 77 files changed, 2570 insertions(+), 109 deletions(-)
 create mode 100644 elf/dynamic-notes.c
 create mode 100644 elf/tst-dlmopen-auditmod.c
 create mode 100644 elf/tst-dlmopen-common.h
 create mode 100644 elf/tst-dlmopen-main.h
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared1.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared2.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared3.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared4.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared5.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-shared6.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique1.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique2.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique3.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique4.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique5.c
 create mode 100644 elf/tst-dlmopen-rtld-audit-unique6.c
 create mode 100644 elf/tst-dlmopen-rtld-shared1.c
 create mode 100644 elf/tst-dlmopen-rtld-shared1.h
 create mode 100644 elf/tst-dlmopen-rtld-shared2.c
 create mode 100644 elf/tst-dlmopen-rtld-shared2.h
 create mode 100644 elf/tst-dlmopen-rtld-shared3.c
 create mode 100644 elf/tst-dlmopen-rtld-shared3.h
 create mode 100644 elf/tst-dlmopen-rtld-shared4.c
 create mode 100644 elf/tst-dlmopen-rtld-shared4.h
 create mode 100644 elf/tst-dlmopen-rtld-shared5.c
 create mode 100644 elf/tst-dlmopen-rtld-shared5.h
 create mode 100644 elf/tst-dlmopen-rtld-shared6.c
 create mode 100644 elf/tst-dlmopen-rtld-shared6.h
 create mode 100644 elf/tst-dlmopen-rtld-unique1.c
 create mode 100644 elf/tst-dlmopen-rtld-unique1.h
 create mode 100644 elf/tst-dlmopen-rtld-unique2.c
 create mode 100644 elf/tst-dlmopen-rtld-unique2.h
 create mode 100644 elf/tst-dlmopen-rtld-unique3.c
 create mode 100644 elf/tst-dlmopen-rtld-unique3.h
 create mode 100644 elf/tst-dlmopen-rtld-unique4.c
 create mode 100644 elf/tst-dlmopen-rtld-unique4.h
 create mode 100644 elf/tst-dlmopen-rtld-unique5.c
 create mode 100644 elf/tst-dlmopen-rtld-unique5.h
 create mode 100644 elf/tst-dlmopen-rtld-unique6.c
 create mode 100644 elf/tst-dlmopen-rtld-unique6.h
 create mode 100644 elf/tst-dlmopen-sharedmod-norm.c
 create mode 100644 elf/tst-dlmopen-sharedmod-uniq.c
 create mode 100644 elf/tst-dlmopen-std-do-test.c
 create mode 100644 elf/tst-dlmopen-unique-dep.c
 create mode 100644 elf/tst-dlmopen-unique-dep.h
  

Looks like this has regressed wrt v12 - I suspect the reorganisation of the 
has_gnu_unique code, but not sure yet. Looking into it.