[v3,00/37] libpthread removal: NPTL forwarders are gone

  This series gets us to the point where no libc → libpthread function
forwarding exists anymore.  It has been tested on i686-linux-gnu and
x86_64-linux-gnu, and built by build-many-glibcs.py.  (A full bootstrap
cycle is still under way.)

I incorporated Adhemerval feedback and his suggestion for keeping a
static assert.

The TSD changes sort-of depend on the dlerror cleanup I originally
posted separately, so I'm including them here as well.

The _Unwind_GetCFA cleanup is gone because I've pushed that separately.

The additional patches are basically more of the same, except that I had
to move some of the libpthread early initialization into ld.so, so that
previously impossible scenarios (like calling pthread_key_create without
linking in libpthread) become supported.  This series does not add tests
to cover those scenarios.  Similarly, I have to initialize more libc
configuration things from __libc_early_init because those variables live
in libc, not ld.so.

I had to remove the x86 low-level lock optimization, and redid it at the
mutex level.  This way, it now works even if libpthread is linked in.
This was a change I wanted to make eventually anyway, but it turns out
the libpthread move invalidated the optimization (similar to the generic
optimization we had for private mutexes), so I had to do something about
it.

I moved the mutex and conditional variable functions in two blocks
because of the many interdependencies, and the potential breakage from
real/stub implementation mismatches.

Obviously, a lot of work still remains to do, but I believe at this
stage, we are already in a much better shape when it comes to late
loading of libpthread because we no longer use any stub implementations
that produce state the real implementations cannot consume.

Thanks,
Florian

Adhemerval Zanella (1):
  x86: Restore compile-time check for shadow stack pointer in longjmp

Florian Weimer (36):
  nptl: Move pthread_mutex_consistent into libc
  nptl: Move __pthread_cleanup_routine into libc
  nptl: Move legacy unwinding implementation into libc
  nptl: Move legacy cancelation handling into libc as compat symbols
  nptl: Remove longjmp, siglongjmp from libpthread
  nptl: Move __pthread_cleanup_upto into libc
  nptl: Move pthread_once and __pthread_once into libc
  nptl: Move __pthread_unwind_next into libc
  csu: Move calling main out of __libc_start_main_impl
  nptl: Move internal __nptl_nthreads variable into libc
  nptl_db: Introduce DB_MAIN_ARRAY_VARIABLE
  nptl: Move __pthread_keys global variable into libc
  nptl: Move __nptl_deallocate_tsd into libc
  nptl: Move pthread_exit into libc
  nptl: Move pthread_setcancelstate into libc
  nptl: Move pthread_setcanceltype into libc
  nptl: Invoke the set_robust_list system call directly in fork
  dlfcn: Failures after dlmopen should not terminate process [BZ #24772]
  dlfcn: dlerror needs to call free from the base namespace [BZ #24773]
  Remove pthread_key_create-related internals from libc-lock.h
  elf: Introduce __tls_init_tp for second-phase TCB initialization
  nptl: Move part of TCB initialization from libpthread to __tls_init_tp
  nptl: Move pthread_key_create, __pthread_key_create into libc
  nptl: Move pthread_getspecific, __pthread_getspecific into libc
  nptl: Move pthread_setspecific, __pthread_setspecific into libc
  nptl: Move pthread_key_delete into libc.
  nptl: Move rwlock functions with forwarders into libc
  nptl: Move the internal thread priority protection symbols into libc
  pthread: Introduce __pthread_early_init
  nptl: Move internal symbol __mutex_aconf into libc
  nptl: pthread_mutex_lock, pthread_mutex_unock single-threaded
    optimization
  x86: Remove low-level lock optimization
  nptl: Move core mutex functions into libc
  nptl: Move core condition variable functions into libc
  nptl: Move setxid broadcast implementation into libc
  nptl: Remove remnants of the libc/libpthread forwarder interface

 NEWS                                          |   4 +-
 csu/libc-start.c                              |  74 +----
 csu/libc-tls.c                                |   8 +-
 dlfcn/Makefile                                |   3 +-
 dlfcn/Versions                                |   6 +-
 dlfcn/dlerror.c                               | 297 ++++++++----------
 dlfcn/dlerror.h                               |  74 +++++
 dlfcn/libc_dlerror_result.c                   |  39 +++
 elf/Makefile                                  |  12 +-
 elf/dl-error-skeleton.c                       |  12 +
 elf/dl-exception.c                            |  11 +
 elf/dl-tls_init_tp.c                          |  24 ++
 elf/libc_early_init.c                         |   3 +
 elf/rtld.c                                    |  16 +-
 elf/tst-dlmopen-dlerror-mod.c                 |  58 ++++
 elf/tst-dlmopen-dlerror.c                     |  49 +++
 include/dlfcn.h                               |   2 -
 include/setjmp.h                              |   4 +-
 libio/iopopen.c                               |   5 +-
 malloc/set-freeres.c                          |  10 +-
 malloc/thread-freeres.c                       |   2 +
 misc/error.c                                  |  19 +-
 nptl/Makefile                                 |  92 +++---
 nptl/Versions                                 | 138 +++++---
 nptl/allocatestack.c                          | 209 ------------
 nptl/cleanup_compat.c                         |  16 +-
 nptl/cleanup_defer_compat.c                   |  72 +----
 nptl/cleanup_routine.c                        |  12 +-
 nptl/forward.c                                | 131 --------
 nptl/libc-cleanup.c                           |  64 +++-
 nptl/libc_pthread_init.c                      |  36 +--
 nptl/libpthread-compat.c                      |   5 +
 nptl/nptl-init.c                              | 152 +--------
 nptl/nptl_deallocate_tsd.c                    | 112 +++++++
 nptl/nptl_nthreads.c                          |  23 ++
 nptl/nptl_setxid.c                            | 279 ++++++++++++++++
 nptl/old_pthread_cond_broadcast.c             |   4 +-
 nptl/old_pthread_cond_signal.c                |   4 +-
 nptl/old_pthread_cond_timedwait.c             |   4 +-
 nptl/old_pthread_cond_wait.c                  |   4 +-
 nptl/pt-longjmp.c                             |  49 ---
 nptl/pthreadP.h                               | 106 ++++---
 nptl/pthread_cancel.c                         |  10 +-
 nptl/{pt-cleanup.c => pthread_cleanup_upto.c} |   2 +-
 nptl/pthread_cond_broadcast.c                 |  11 +-
 nptl/pthread_cond_signal.c                    |   9 +-
 nptl/pthread_cond_wait.c                      |  66 ++--
 nptl/pthread_create.c                         | 102 +-----
 nptl/pthread_exit.c                           |  16 +-
 nptl/pthread_getspecific.c                    |  22 +-
 nptl/pthread_key_create.c                     |  21 +-
 nptl/pthread_key_delete.c                     |  17 +-
 nptl/pthread_keys.c                           |  24 ++
 nptl/pthread_mutex_cond_lock.c                |   4 +-
 nptl/pthread_mutex_conf.c                     |   1 +
 nptl/pthread_mutex_consistent.c               |  22 +-
 nptl/pthread_mutex_destroy.c                  |  18 +-
 nptl/pthread_mutex_init.c                     |  18 +-
 nptl/pthread_mutex_lock.c                     |  47 ++-
 nptl/pthread_mutex_unlock.c                   |  36 ++-
 nptl/pthread_once.c                           |  21 +-
 nptl/pthread_rwlock_rdlock.c                  |  22 +-
 nptl/pthread_rwlock_unlock.c                  |  22 +-
 nptl/pthread_rwlock_wrlock.c                  |  22 +-
 nptl/pthread_setcancelstate.c                 |   6 +-
 nptl/pthread_setcanceltype.c                  |   3 +-
 nptl/pthread_setspecific.c                    |  22 +-
 nptl/tpp.c                                    |   6 +-
 nptl/tst-cleanup4.c                           |   5 +
 nptl/tst-cleanup4aux.c                        |   5 +
 nptl/tst-pthread_exit-nothreads-static.c      |  19 ++
 nptl/tst-pthread_exit-nothreads.c             |  28 ++
 nptl/unwind.c                                 |  14 +-
 nptl/vars.c                                   |   5 -
 nptl_db/db-symbols.awk                        |   1 +
 nptl_db/structs.def                           |   9 +-
 setjmp/Versions                               |   4 -
 stdlib/fmtmsg.c                               |   9 +-
 sysdeps/arm/Makefile                          |   5 -
 sysdeps/arm/nptl/unwind-forcedunwind.c        |  25 --
 sysdeps/arm/pt-arm-unwind-resume.S            |  20 --
 sysdeps/generic/ldsodefs.h                    |  21 ++
 sysdeps/generic/libc-lock.h                   |  12 -
 sysdeps/generic/libc_start_call_main.h        |  24 ++
 sysdeps/generic/unwind-resume.c               |  14 +
 sysdeps/htl/libc-lockP.h                      |  13 -
 sysdeps/i386/nptl/pthread_spin_lock.S         |   1 -
 .../unwind-resume.c}                          |   5 +-
 sysdeps/nptl/dl-tls_init_tp.c                 |  67 ++++
 sysdeps/nptl/fork.c                           |  14 +-
 sysdeps/nptl/fork.h                           |   3 -
 sysdeps/nptl/jmp-unwind.c                     |   8 +-
 sysdeps/nptl/libc-lock.h                      |  59 ++--
 sysdeps/nptl/libc-lockP.h                     | 154 ++-------
 sysdeps/nptl/libc_start_call_main.h           |  74 +++++
 sysdeps/nptl/pthread-functions.h              |  88 ------
 sysdeps/nptl/pthread.h                        |  10 +-
 .../nptl/pthread_early_init.h                 |  22 +-
 {nptl => sysdeps/nptl}/pthread_mutex_conf.h   |   9 +-
 sysdeps/nptl/setxid.h                         |  26 +-
 sysdeps/nptl/unwind-forcedunwind.c            |  64 ----
 sysdeps/posix/system.c                        |   5 +-
 sysdeps/pthread/Makefile                      |   2 +-
 sysdeps/pthread/pthread_early_init.h          |  29 ++
 .../pthread/pthread_mutex_conf.h              |  17 +-
 sysdeps/pthread/tst-robust1.c                 |   4 +-
 sysdeps/pthread/tst-robust8.c                 |   2 +-
 sysdeps/pthread/tst-robust9.c                 |   4 +-
 sysdeps/unix/sysv/linux/aarch64/libc.abilist  |  51 +++
 .../sysv/linux/aarch64/libpthread.abilist     |  41 ---
 sysdeps/unix/sysv/linux/alpha/libc.abilist    |  51 +++
 .../unix/sysv/linux/alpha/libpthread.abilist  |  46 +--
 sysdeps/unix/sysv/linux/arc/libc.abilist      |  51 +++
 .../unix/sysv/linux/arc/libpthread.abilist    |  39 ---
 sysdeps/unix/sysv/linux/arm/be/libc.abilist   |  51 +++
 .../unix/sysv/linux/arm/be/libpthread.abilist |  41 ---
 sysdeps/unix/sysv/linux/arm/le/libc.abilist   |  51 +++
 .../unix/sysv/linux/arm/le/libpthread.abilist |  41 ---
 sysdeps/unix/sysv/linux/csky/libc.abilist     |  51 +++
 .../unix/sysv/linux/csky/libpthread.abilist   |  39 ---
 sysdeps/unix/sysv/linux/fatal-prepare.h       |   4 +-
 sysdeps/unix/sysv/linux/hppa/libc.abilist     |  51 +++
 .../unix/sysv/linux/hppa/libpthread.abilist   |  46 +--
 sysdeps/unix/sysv/linux/i386/libc.abilist     |  51 +++
 .../unix/sysv/linux/i386/libpthread.abilist   |  46 +--
 sysdeps/unix/sysv/linux/ia64/Makefile         |   2 +-
 sysdeps/unix/sysv/linux/ia64/libc.abilist     |  51 +++
 .../unix/sysv/linux/ia64/libpthread.abilist   |  46 +--
 sysdeps/unix/sysv/linux/internal-signals.h    |   4 -
 .../sysv/linux/m68k/coldfire/libc.abilist     |  51 +++
 .../linux/m68k/coldfire/libpthread.abilist    |  41 ---
 .../unix/sysv/linux/m68k/m680x0/libc.abilist  |  51 +++
 .../sysv/linux/m68k/m680x0/libpthread.abilist |  46 +--
 .../sysv/linux/microblaze/be/libc.abilist     |  51 +++
 .../linux/microblaze/be/libpthread.abilist    |  41 ---
 .../sysv/linux/microblaze/le/libc.abilist     |  51 +++
 .../linux/microblaze/le/libpthread.abilist    |  41 ---
 .../sysv/linux/mips/mips32/fpu/libc.abilist   |  51 +++
 .../sysv/linux/mips/mips32/libpthread.abilist |  46 +--
 .../sysv/linux/mips/mips32/nofpu/libc.abilist |  51 +++
 .../sysv/linux/mips/mips64/libpthread.abilist |  46 +--
 .../sysv/linux/mips/mips64/n32/libc.abilist   |  51 +++
 .../sysv/linux/mips/mips64/n64/libc.abilist   |  51 +++
 sysdeps/unix/sysv/linux/nios2/libc.abilist    |  51 +++
 .../unix/sysv/linux/nios2/libpthread.abilist  |  41 ---
 .../linux/powerpc/powerpc32/fpu/libc.abilist  |  51 +++
 .../powerpc/powerpc32/libpthread.abilist      |  48 +--
 .../powerpc/powerpc32/nofpu/libc.abilist      |  51 +++
 .../linux/powerpc/powerpc64/be/libc.abilist   |  51 +++
 .../powerpc/powerpc64/be/libpthread.abilist   |  48 +--
 .../linux/powerpc/powerpc64/le/libc.abilist   |  51 +++
 .../powerpc/powerpc64/le/libpthread.abilist   |  41 ---
 sysdeps/unix/sysv/linux/powerpc/pt-longjmp.c  |  61 ----
 sysdeps/unix/sysv/linux/pthread-pids.h        |  29 --
 .../unix/sysv/linux/riscv/rv32/libc.abilist   |  51 +++
 .../sysv/linux/riscv/rv32/libpthread.abilist  |  39 ---
 .../unix/sysv/linux/riscv/rv64/libc.abilist   |  51 +++
 .../sysv/linux/riscv/rv64/libpthread.abilist  |  39 ---
 sysdeps/unix/sysv/linux/s390/Versions         |   2 +-
 sysdeps/unix/sysv/linux/s390/jmp-unwind.c     |   7 +-
 .../unix/sysv/linux/s390/libpthread-compat.c  |  25 ++
 sysdeps/unix/sysv/linux/s390/pt-longjmp.c     |  33 --
 .../unix/sysv/linux/s390/s390-32/libc.abilist |  51 +++
 .../linux/s390/s390-32/libpthread.abilist     |  49 +--
 .../unix/sysv/linux/s390/s390-64/libc.abilist |  51 +++
 .../linux/s390/s390-64/libpthread.abilist     |  49 +--
 sysdeps/unix/sysv/linux/sh/be/libc.abilist    |  51 +++
 .../unix/sysv/linux/sh/be/libpthread.abilist  |  46 +--
 sysdeps/unix/sysv/linux/sh/le/libc.abilist    |  51 +++
 .../unix/sysv/linux/sh/le/libpthread.abilist  |  46 +--
 .../sysv/linux/sparc/sparc32/libc.abilist     |  51 +++
 .../linux/sparc/sparc32/libpthread.abilist    |  46 +--
 .../sysv/linux/sparc/sparc64/libc.abilist     |  51 +++
 .../linux/sparc/sparc64/libpthread.abilist    |  46 +--
 sysdeps/unix/sysv/linux/spawni.c              |   5 +-
 sysdeps/unix/sysv/linux/x86/longjmp.c         |  39 +++
 sysdeps/unix/sysv/linux/x86/lowlevellock.h    |  87 -----
 .../unix/sysv/linux/x86_64/64/libc.abilist    |  51 +++
 .../sysv/linux/x86_64/64/libpthread.abilist   |  46 +--
 .../unix/sysv/linux/x86_64/x32/libc.abilist   |  51 +++
 .../sysv/linux/x86_64/x32/libpthread.abilist  |  41 ---
 sysdeps/x86/nptl/pt-longjmp.c                 |  71 -----
 sysdeps/x86_64/nptl/pthread_spin_lock.S       |   1 -
 183 files changed, 3682 insertions(+), 3305 deletions(-)
 create mode 100644 dlfcn/dlerror.h
 create mode 100644 dlfcn/libc_dlerror_result.c
 create mode 100644 elf/dl-tls_init_tp.c
 create mode 100644 elf/tst-dlmopen-dlerror-mod.c
 create mode 100644 elf/tst-dlmopen-dlerror.c
 delete mode 100644 nptl/forward.c
 create mode 100644 nptl/nptl_deallocate_tsd.c
 create mode 100644 nptl/nptl_nthreads.c
 create mode 100644 nptl/nptl_setxid.c
 delete mode 100644 nptl/pt-longjmp.c
 rename nptl/{pt-cleanup.c => pthread_cleanup_upto.c} (98%)
 create mode 100644 nptl/pthread_keys.c
 create mode 100644 nptl/tst-pthread_exit-nothreads-static.c
 create mode 100644 nptl/tst-pthread_exit-nothreads.c
 delete mode 100644 sysdeps/arm/nptl/unwind-forcedunwind.c
 delete mode 100644 sysdeps/arm/pt-arm-unwind-resume.S
 create mode 100644 sysdeps/generic/libc_start_call_main.h
 rename sysdeps/{unix/sysv/linux/ia64/unwind-forcedunwind.c => ia64/unwind-resume.c} (87%)
 create mode 100644 sysdeps/nptl/dl-tls_init_tp.c
 create mode 100644 sysdeps/nptl/libc_start_call_main.h
 delete mode 100644 sysdeps/nptl/pthread-functions.h
 rename nptl/pthread-pids.h => sysdeps/nptl/pthread_early_init.h (61%)
 rename {nptl => sysdeps/nptl}/pthread_mutex_conf.h (86%)
 delete mode 100644 sysdeps/nptl/unwind-forcedunwind.c
 create mode 100644 sysdeps/pthread/pthread_early_init.h
 rename dlfcn/dlfreeres.c => sysdeps/pthread/pthread_mutex_conf.h (74%)
 delete mode 100644 sysdeps/unix/sysv/linux/powerpc/pt-longjmp.c
 delete mode 100644 sysdeps/unix/sysv/linux/pthread-pids.h
 create mode 100644 sysdeps/unix/sysv/linux/s390/libpthread-compat.c
 delete mode 100644 sysdeps/unix/sysv/linux/s390/pt-longjmp.c
 create mode 100644 sysdeps/unix/sysv/linux/x86/longjmp.c
 delete mode 100644 sysdeps/unix/sysv/linux/x86/lowlevellock.h
 delete mode 100644 sysdeps/x86/nptl/pt-longjmp.c
  

I've pushed a rebase to fw/libpthread-20180318b.  Apart from abilist
merges (automatically resolved) and various nptl/Versions and
nptl/Makefile conflicts (manually resolved), the main difference is
this:

diff --git a/nptl/pthread_exit.c b/nptl/pthread_exit.c
index 1a93f371a9..6abf66463e 100644
--- a/nptl/pthread_exit.c
+++ b/nptl/pthread_exit.c
@@ -25,16 +25,22 @@
 void
 __pthread_exit (void *value)
 {
   {
     struct unwind_link *unwind_link = __libc_unwind_link_get ();
     if (unwind_link == NULL)
       __libc_fatal (LIBGCC_S_SO
                     " must be installed for pthread_exit to work\n");
   }
 
   THREAD_SETMEM (THREAD_SELF, result, value);
 
   __do_cancel ();
 }
 libc_hidden_def (__pthread_exit)
 weak_alias (__pthread_exit, pthread_exit)
+
+/* Ensure that the unwinder is always linked in (the __pthread_unwind
+   reference from __do_cancel is weak).  Use ___pthread_unwind_next
+   (three underscores) to produce a strong reference to the same
+   file.  */
+PTHREAD_STATIC_FN_REQUIRE (___pthread_unwind_next)

Without this, nptl/tst-pthread_exit-nothreads-static fails.  After
Adhemerval's cleanups, we no longer link in the unwinder in that minimal
test program, so the call to __pthread_unwind turns into a call to a
null pointer.

Thanks,
Florian
  

On 16/03/2021 14:27, Florian Weimer via Libc-alpha wrote:
> This series gets us to the point where no libc → libpthread function
> forwarding exists anymore.  It has been tested on i686-linux-gnu and
> x86_64-linux-gnu, and built by build-many-glibcs.py.  (A full bootstrap
> cycle is still under way.)
> 
> I incorporated Adhemerval feedback and his suggestion for keeping a
> static assert.
> 
> The TSD changes sort-of depend on the dlerror cleanup I originally
> posted separately, so I'm including them here as well.
> 
> The _Unwind_GetCFA cleanup is gone because I've pushed that separately.
> 
> The additional patches are basically more of the same, except that I had
> to move some of the libpthread early initialization into ld.so, so that
> previously impossible scenarios (like calling pthread_key_create without
> linking in libpthread) become supported.  This series does not add tests
> to cover those scenarios.  Similarly, I have to initialize more libc
> configuration things from __libc_early_init because those variables live
> in libc, not ld.so.
> 
> I had to remove the x86 low-level lock optimization, and redid it at the
> mutex level.  This way, it now works even if libpthread is linked in.
> This was a change I wanted to make eventually anyway, but it turns out
> the libpthread move invalidated the optimization (similar to the generic
> optimization we had for private mutexes), so I had to do something about
> it.
> 
> I moved the mutex and conditional variable functions in two blocks
> because of the many interdependencies, and the potential breakage from
> real/stub implementation mismatches.
> 
> Obviously, a lot of work still remains to do, but I believe at this
> stage, we are already in a much better shape when it comes to late
> loading of libpthread because we no longer use any stub implementations
> that produce state the real implementations cannot consume.

Hi Florian, 

I think I reviewed all the patches on the set and it looks good in general.
Most changes are related to to your own patch that add the multiple
reference support to compat_symbol, the remaining as mostly some question
regarding the code and small nits.

From my side there is no need to resend the patches that need rebase, so
I think the set should be good to push upstream once you adjust against
master.
  

* Adhemerval Zanella via Libc-alpha:

> I think I reviewed all the patches on the set and it looks good in general.
> Most changes are related to to your own patch that add the multiple
> reference support to compat_symbol, the remaining as mostly some question
> regarding the code and small nits.
>
> From my side there is no need to resend the patches that need rebase, so
> I think the set should be good to push upstream once you adjust against
> master.

I'm sort of on vacation, which is why I haven't made much progress.

The next version will have quite a few changes, mostly related to
OTHER_SHLIB_COMPAT.  I also need to fix a localplt issue around
longjmp because a previously-hidden symbol is no longer hidden.  The
changes are all quite straightforward, I think.

I will come back to this next week.  I hope it does not block
anything.