[v8,00/10] malloc hooks removal

  This patchset removes the malloc hooks __malloc_hook, __free_hook,
__realloc_hook and __memalign_hook from the API and leaves compatibility
symbols so that existing applications can continue to link to them.  The
reading and execution of the hooks has been moved to a DSO
libc_malloc_debug.so, which can be preloaded for applications that need
it.  By default these hooks no longer have any effect in the library.

Further, debugging features such as MALLOC_CHECK_, mcheck() and mtrace
have been weaned away from these hooks and also moved to
libc_malloc_debug.so.  With this change, these features are only enabled
when libc_malloc_debug.so is preloaded using LD_PRELOAD.

Finally, the __morecore, __morecore_after_hook and __default_morecore
hooks have also been moved to compat symbols and removed from the API.
Existing applications will continue to link to them but they won't have
any effect on malloc behaviour.

Testing:

The patchset has gone through a full build and test on x86_64, i686,
s390x, ppc64le, armv7l and aarch64 through a scratch build on Fedora
rawhide.  I also installed a test package on x86_64 and rebooted to make
sure the system boots to shell.  I did a build-many-glibcs run with
the patchset and it ran clean on all targets.

Changes from v7:
- Made mcheck tests conditional on GLIBC_2.24
- Added SHLIB_COMPAT guard around __malloc_initialize_hook usage in
  libc_malloc_debug.so
- Fixed hurd build failure.
- Add another patch to fix malloc_usable_size to mcheck

Changes from v6:
- Moved malloc-check into libc_malloc_debug.so. Tweaked malloc.c to
  allow building twice, once inside libc.so and next in
  libc_malloc_debug.so
- Included morecore.c into malloc.c
- Moved hook initialization too into libc_malloc_debug.so.
- Interposed more functions in libc_malloc_debug.so
- Versioned all symbols exported from libc_malloc_debug.so and finalized
  them so that the library cannot be linked against

Changes from v5:
- Rebased on latest master
- Fixed realloc-mcheck
- Removed residual mention of libmalloc_compathooks
- Removed LD_PRELOAD from elf/tst-setuid

Changes from v4:
- Patchset has a different approach, starting with moving out hooks
  first to restrict all major malloc.c changes to the first patch
- Renamed libmalloc_compathooks.so to libc_malloc_debug.so
- Moved all debugging features into libc_malloc_debug.so
- Made more documentation updates
- Simplified __malloc_initialized variable use
- Removed debugging tests on static variables since that is no longer
  supported

Changes from v3:
- Remove source file dependencies
- Commit mcheck tests

Changes from v2:
- Move hooks dependencies to malloc.o{,sS}

Changes from v1:

- Added makefile dependencies for the new hooks files
- Fixed memset call in calloc debugging hooks
- Added the tr_break deprecation patch and mcheck test patch to this
  series


Siddhesh Poyarekar (10):
  Make mcheck tests conditional on GLIBC_2.24 or earlier
  Remove __after_morecore_hook
  Remove __morecore and __default_morecore
  Move malloc hooks into a compat DSO
  mcheck: Wean away from malloc hooks
  Simplify __malloc_initialized
  mtrace: Wean away from malloc hooks
  glibc.malloc.check: Wean away from malloc hooks
  Remove malloc hooks
  mcheck Fix malloc_usable_size [BZ #22057]

 NEWS                                          |  21 +
 Rules                                         |   9 +-
 catgets/Makefile                              |   4 +-
 elf/Makefile                                  |  15 +-
 elf/tst-leaks1-static.c                       |   1 -
 iconvdata/Makefile                            |   3 +-
 include/malloc.h                              |   6 -
 include/mcheck.h                              |   4 -
 include/stdlib.h                              |   3 -
 intl/tst-gettext.sh                           |   1 +
 libio/Makefile                                |  12 +-
 localedata/Makefile                           |   3 +-
 malloc/Makefile                               |  56 +-
 malloc/Versions                               |  44 ++
 malloc/arena.c                                |  48 +-
 malloc/hooks.c                                |  71 ++-
 malloc/malloc-check.c                         |  63 ++-
 malloc/malloc-debug.c                         | 522 ++++++++++++++++++
 malloc/malloc-hooks.h                         |  24 -
 malloc/malloc.c                               | 163 ++----
 malloc/malloc.h                               |  27 -
 malloc/mcheck-impl.c                          | 412 ++++++++++++++
 malloc/mcheck.c                               | 397 +------------
 malloc/morecore.c                             |  34 +-
 malloc/mtrace-impl.c                          | 226 ++++++++
 malloc/mtrace.c                               | 313 +----------
 malloc/tst-compathooks-off.c                  | 145 +++++
 malloc/tst-compathooks-on.c                   |   2 +
 malloc/tst-malloc-usable-static-tunables.c    |   1 -
 malloc/tst-malloc-usable-static.c             |   1 -
 malloc/tst-mtrace.sh                          |   1 +
 manual/memory.texi                            | 207 +------
 manual/tunables.texi                          |   4 +-
 misc/Makefile                                 |   6 +-
 nptl/Makefile                                 |   3 +-
 posix/Makefile                                |  48 +-
 resolv/Makefile                               |   9 +-
 shlib-versions                                |   3 +
 stdio-common/Makefile                         |  15 +-
 sysdeps/aarch64/Makefile                      |   3 +
 sysdeps/generic/libc_malloc_debug.abilist     |   0
 sysdeps/generic/localplt.data                 |   1 -
 sysdeps/mach/hurd/Makefile                    |   1 +
 .../mach/hurd/i386/libc_malloc_debug.abilist  |  26 +
 sysdeps/mach/hurd/i386/localplt.data          |   1 -
 sysdeps/pthread/Makefile                      |   3 +-
 .../linux/aarch64/libc_malloc_debug.abilist   |  26 +
 sysdeps/unix/sysv/linux/aarch64/localplt.data |   1 -
 .../linux/alpha/libc_malloc_debug.abilist     |  26 +
 sysdeps/unix/sysv/linux/alpha/localplt.data   |   1 -
 .../sysv/linux/arc/libc_malloc_debug.abilist  |  26 +
 sysdeps/unix/sysv/linux/arc/localplt.data     |   1 -
 .../linux/arm/be/libc_malloc_debug.abilist    |  26 +
 .../linux/arm/le/libc_malloc_debug.abilist    |  26 +
 sysdeps/unix/sysv/linux/arm/localplt.data     |   1 -
 .../sysv/linux/csky/libc_malloc_debug.abilist |  26 +
 sysdeps/unix/sysv/linux/csky/localplt.data    |   1 -
 .../sysv/linux/hppa/libc_malloc_debug.abilist |  26 +
 sysdeps/unix/sysv/linux/hppa/localplt.data    |   1 -
 sysdeps/unix/sysv/linux/hppa/shlib-versions   |   2 +
 .../sysv/linux/i386/libc_malloc_debug.abilist |  26 +
 sysdeps/unix/sysv/linux/i386/localplt.data    |   1 -
 .../sysv/linux/ia64/libc_malloc_debug.abilist |  26 +
 sysdeps/unix/sysv/linux/ia64/localplt.data    |   1 -
 sysdeps/unix/sysv/linux/ia64/shlib-versions   |   2 +
 .../m68k/coldfire/libc_malloc_debug.abilist   |  26 +
 .../sysv/linux/m68k/coldfire/localplt.data    |   1 -
 .../m68k/m680x0/libc_malloc_debug.abilist     |  26 +
 .../unix/sysv/linux/m68k/m680x0/localplt.data |   1 -
 .../microblaze/be/libc_malloc_debug.abilist   |  26 +
 .../microblaze/le/libc_malloc_debug.abilist   |  26 +
 .../unix/sysv/linux/microblaze/localplt.data  |   1 -
 .../mips/mips32/fpu/libc_malloc_debug.abilist |  26 +
 .../mips32/nofpu/libc_malloc_debug.abilist    |  26 +
 .../mips/mips64/n32/libc_malloc_debug.abilist |  26 +
 .../mips/mips64/n64/libc_malloc_debug.abilist |  26 +
 sysdeps/unix/sysv/linux/mips/shlib-versions   |   2 +
 .../linux/nios2/libc_malloc_debug.abilist     |  26 +
 sysdeps/unix/sysv/linux/nios2/localplt.data   |   1 -
 .../powerpc32/fpu/libc_malloc_debug.abilist   |  26 +
 .../linux/powerpc/powerpc32/fpu/localplt.data |   1 -
 .../powerpc32/nofpu/libc_malloc_debug.abilist |  26 +
 .../powerpc/powerpc32/nofpu/localplt.data     |   1 -
 .../powerpc64/be/libc_malloc_debug.abilist    |  26 +
 .../powerpc64/le/libc_malloc_debug.abilist    |  26 +
 .../linux/powerpc/powerpc64/localplt.data     |   1 -
 sysdeps/unix/sysv/linux/riscv/localplt.data   |   1 -
 .../riscv/rv32/libc_malloc_debug.abilist      |  26 +
 .../riscv/rv64/libc_malloc_debug.abilist      |  26 +
 sysdeps/unix/sysv/linux/s390/localplt.data    |   1 -
 .../s390/s390-32/libc_malloc_debug.abilist    |  26 +
 .../s390/s390-64/libc_malloc_debug.abilist    |  26 +
 .../linux/sh/be/libc_malloc_debug.abilist     |  26 +
 .../linux/sh/le/libc_malloc_debug.abilist     |  26 +
 sysdeps/unix/sysv/linux/sh/localplt.data      |   1 -
 sysdeps/unix/sysv/linux/sh/shlib-versions     |   1 +
 .../sparc/sparc32/libc_malloc_debug.abilist   |  26 +
 .../sysv/linux/sparc/sparc32/localplt.data    |   1 -
 .../sparc/sparc64/libc_malloc_debug.abilist   |  26 +
 .../sysv/linux/sparc/sparc64/localplt.data    |   1 -
 .../sysv/linux/sparc/sparc64/shlib-versions   |   1 +
 .../linux/x86_64/64/libc_malloc_debug.abilist |  26 +
 .../x86_64/x32/libc_malloc_debug.abilist      |  26 +
 sysdeps/x86_64/localplt.data                  |   1 -
 104 files changed, 2557 insertions(+), 1266 deletions(-)
 delete mode 100644 elf/tst-leaks1-static.c
 create mode 100644 malloc/malloc-debug.c
 delete mode 100644 malloc/malloc-hooks.h
 create mode 100644 malloc/mcheck-impl.c
 create mode 100644 malloc/mtrace-impl.c
 create mode 100644 malloc/tst-compathooks-off.c
 create mode 100644 malloc/tst-compathooks-on.c
 delete mode 100644 malloc/tst-malloc-usable-static-tunables.c
 delete mode 100644 malloc/tst-malloc-usable-static.c
 create mode 100644 sysdeps/generic/libc_malloc_debug.abilist
 create mode 100644 sysdeps/mach/hurd/i386/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/aarch64/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/alpha/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/arc/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/arm/be/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/arm/le/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/csky/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/hppa/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/i386/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/ia64/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/m68k/coldfire/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/m68k/m680x0/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/microblaze/be/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/microblaze/le/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/mips/mips32/fpu/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/mips/mips64/n32/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/mips/mips64/n64/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/nios2/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc64/be/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc64/le/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv64/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/s390/s390-32/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/s390/s390-64/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/sh/be/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/sh/le/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/sparc/sparc32/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/sparc/sparc64/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/x86_64/64/libc_malloc_debug.abilist
 create mode 100644 sysdeps/unix/sysv/linux/x86_64/x32/libc_malloc_debug.abilist
  

On Tue, Jul 13, 2021 at 12:39 AM Siddhesh Poyarekar via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> This patchset removes the malloc hooks __malloc_hook, __free_hook,
> __realloc_hook and __memalign_hook from the API and leaves compatibility
> symbols so that existing applications can continue to link to them.  The
> reading and execution of the hooks has been moved to a DSO
> libc_malloc_debug.so, which can be preloaded for applications that need
> it.  By default these hooks no longer have any effect in the library.
>
> Further, debugging features such as MALLOC_CHECK_, mcheck() and mtrace
> have been weaned away from these hooks and also moved to
> libc_malloc_debug.so.  With this change, these features are only enabled
> when libc_malloc_debug.so is preloaded using LD_PRELOAD.
>
> Finally, the __morecore, __morecore_after_hook and __default_morecore
> hooks have also been moved to compat symbols and removed from the API.
> Existing applications will continue to link to them but they won't have
> any effect on malloc behaviour.
>
> Testing:
>
> The patchset has gone through a full build and test on x86_64, i686,
> s390x, ppc64le, armv7l and aarch64 through a scratch build on Fedora
> rawhide.  I also installed a test package on x86_64 and rebooted to make
> sure the system boots to shell.  I did a build-many-glibcs run with
> the patchset and it ran clean on all targets.

Does this series fix posix_memalign:

https://sourceware.org/bugzilla/show_bug.cgi?id=23489

> Changes from v7:
> - Made mcheck tests conditional on GLIBC_2.24
> - Added SHLIB_COMPAT guard around __malloc_initialize_hook usage in
>   libc_malloc_debug.so
> - Fixed hurd build failure.
> - Add another patch to fix malloc_usable_size to mcheck
>
> Changes from v6:
> - Moved malloc-check into libc_malloc_debug.so. Tweaked malloc.c to
>   allow building twice, once inside libc.so and next in
>   libc_malloc_debug.so
> - Included morecore.c into malloc.c
> - Moved hook initialization too into libc_malloc_debug.so.
> - Interposed more functions in libc_malloc_debug.so
> - Versioned all symbols exported from libc_malloc_debug.so and finalized
>   them so that the library cannot be linked against
>
> Changes from v5:
> - Rebased on latest master
> - Fixed realloc-mcheck
> - Removed residual mention of libmalloc_compathooks
> - Removed LD_PRELOAD from elf/tst-setuid
>
> Changes from v4:
> - Patchset has a different approach, starting with moving out hooks
>   first to restrict all major malloc.c changes to the first patch
> - Renamed libmalloc_compathooks.so to libc_malloc_debug.so
> - Moved all debugging features into libc_malloc_debug.so
> - Made more documentation updates
> - Simplified __malloc_initialized variable use
> - Removed debugging tests on static variables since that is no longer
>   supported
>
> Changes from v3:
> - Remove source file dependencies
> - Commit mcheck tests
>
> Changes from v2:
> - Move hooks dependencies to malloc.o{,sS}
>
> Changes from v1:
>
> - Added makefile dependencies for the new hooks files
> - Fixed memset call in calloc debugging hooks
> - Added the tr_break deprecation patch and mcheck test patch to this
>   series
>
>
> Siddhesh Poyarekar (10):
>   Make mcheck tests conditional on GLIBC_2.24 or earlier
>   Remove __after_morecore_hook
>   Remove __morecore and __default_morecore
>   Move malloc hooks into a compat DSO
>   mcheck: Wean away from malloc hooks
>   Simplify __malloc_initialized
>   mtrace: Wean away from malloc hooks
>   glibc.malloc.check: Wean away from malloc hooks
>   Remove malloc hooks
>   mcheck Fix malloc_usable_size [BZ #22057]
>
>  NEWS                                          |  21 +
>  Rules                                         |   9 +-
>  catgets/Makefile                              |   4 +-
>  elf/Makefile                                  |  15 +-
>  elf/tst-leaks1-static.c                       |   1 -
>  iconvdata/Makefile                            |   3 +-
>  include/malloc.h                              |   6 -
>  include/mcheck.h                              |   4 -
>  include/stdlib.h                              |   3 -
>  intl/tst-gettext.sh                           |   1 +
>  libio/Makefile                                |  12 +-
>  localedata/Makefile                           |   3 +-
>  malloc/Makefile                               |  56 +-
>  malloc/Versions                               |  44 ++
>  malloc/arena.c                                |  48 +-
>  malloc/hooks.c                                |  71 ++-
>  malloc/malloc-check.c                         |  63 ++-
>  malloc/malloc-debug.c                         | 522 ++++++++++++++++++
>  malloc/malloc-hooks.h                         |  24 -
>  malloc/malloc.c                               | 163 ++----
>  malloc/malloc.h                               |  27 -
>  malloc/mcheck-impl.c                          | 412 ++++++++++++++
>  malloc/mcheck.c                               | 397 +------------
>  malloc/morecore.c                             |  34 +-
>  malloc/mtrace-impl.c                          | 226 ++++++++
>  malloc/mtrace.c                               | 313 +----------
>  malloc/tst-compathooks-off.c                  | 145 +++++
>  malloc/tst-compathooks-on.c                   |   2 +
>  malloc/tst-malloc-usable-static-tunables.c    |   1 -
>  malloc/tst-malloc-usable-static.c             |   1 -
>  malloc/tst-mtrace.sh                          |   1 +
>  manual/memory.texi                            | 207 +------
>  manual/tunables.texi                          |   4 +-
>  misc/Makefile                                 |   6 +-
>  nptl/Makefile                                 |   3 +-
>  posix/Makefile                                |  48 +-
>  resolv/Makefile                               |   9 +-
>  shlib-versions                                |   3 +
>  stdio-common/Makefile                         |  15 +-
>  sysdeps/aarch64/Makefile                      |   3 +
>  sysdeps/generic/libc_malloc_debug.abilist     |   0
>  sysdeps/generic/localplt.data                 |   1 -
>  sysdeps/mach/hurd/Makefile                    |   1 +
>  .../mach/hurd/i386/libc_malloc_debug.abilist  |  26 +
>  sysdeps/mach/hurd/i386/localplt.data          |   1 -
>  sysdeps/pthread/Makefile                      |   3 +-
>  .../linux/aarch64/libc_malloc_debug.abilist   |  26 +
>  sysdeps/unix/sysv/linux/aarch64/localplt.data |   1 -
>  .../linux/alpha/libc_malloc_debug.abilist     |  26 +
>  sysdeps/unix/sysv/linux/alpha/localplt.data   |   1 -
>  .../sysv/linux/arc/libc_malloc_debug.abilist  |  26 +
>  sysdeps/unix/sysv/linux/arc/localplt.data     |   1 -
>  .../linux/arm/be/libc_malloc_debug.abilist    |  26 +
>  .../linux/arm/le/libc_malloc_debug.abilist    |  26 +
>  sysdeps/unix/sysv/linux/arm/localplt.data     |   1 -
>  .../sysv/linux/csky/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/csky/localplt.data    |   1 -
>  .../sysv/linux/hppa/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/hppa/localplt.data    |   1 -
>  sysdeps/unix/sysv/linux/hppa/shlib-versions   |   2 +
>  .../sysv/linux/i386/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/i386/localplt.data    |   1 -
>  .../sysv/linux/ia64/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/ia64/localplt.data    |   1 -
>  sysdeps/unix/sysv/linux/ia64/shlib-versions   |   2 +
>  .../m68k/coldfire/libc_malloc_debug.abilist   |  26 +
>  .../sysv/linux/m68k/coldfire/localplt.data    |   1 -
>  .../m68k/m680x0/libc_malloc_debug.abilist     |  26 +
>  .../unix/sysv/linux/m68k/m680x0/localplt.data |   1 -
>  .../microblaze/be/libc_malloc_debug.abilist   |  26 +
>  .../microblaze/le/libc_malloc_debug.abilist   |  26 +
>  .../unix/sysv/linux/microblaze/localplt.data  |   1 -
>  .../mips/mips32/fpu/libc_malloc_debug.abilist |  26 +
>  .../mips32/nofpu/libc_malloc_debug.abilist    |  26 +
>  .../mips/mips64/n32/libc_malloc_debug.abilist |  26 +
>  .../mips/mips64/n64/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/mips/shlib-versions   |   2 +
>  .../linux/nios2/libc_malloc_debug.abilist     |  26 +
>  sysdeps/unix/sysv/linux/nios2/localplt.data   |   1 -
>  .../powerpc32/fpu/libc_malloc_debug.abilist   |  26 +
>  .../linux/powerpc/powerpc32/fpu/localplt.data |   1 -
>  .../powerpc32/nofpu/libc_malloc_debug.abilist |  26 +
>  .../powerpc/powerpc32/nofpu/localplt.data     |   1 -
>  .../powerpc64/be/libc_malloc_debug.abilist    |  26 +
>  .../powerpc64/le/libc_malloc_debug.abilist    |  26 +
>  .../linux/powerpc/powerpc64/localplt.data     |   1 -
>  sysdeps/unix/sysv/linux/riscv/localplt.data   |   1 -
>  .../riscv/rv32/libc_malloc_debug.abilist      |  26 +
>  .../riscv/rv64/libc_malloc_debug.abilist      |  26 +
>  sysdeps/unix/sysv/linux/s390/localplt.data    |   1 -
>  .../s390/s390-32/libc_malloc_debug.abilist    |  26 +
>  .../s390/s390-64/libc_malloc_debug.abilist    |  26 +
>  .../linux/sh/be/libc_malloc_debug.abilist     |  26 +
>  .../linux/sh/le/libc_malloc_debug.abilist     |  26 +
>  sysdeps/unix/sysv/linux/sh/localplt.data      |   1 -
>  sysdeps/unix/sysv/linux/sh/shlib-versions     |   1 +
>  .../sparc/sparc32/libc_malloc_debug.abilist   |  26 +
>  .../sysv/linux/sparc/sparc32/localplt.data    |   1 -
>  .../sparc/sparc64/libc_malloc_debug.abilist   |  26 +
>  .../sysv/linux/sparc/sparc64/localplt.data    |   1 -
>  .../sysv/linux/sparc/sparc64/shlib-versions   |   1 +
>  .../linux/x86_64/64/libc_malloc_debug.abilist |  26 +
>  .../x86_64/x32/libc_malloc_debug.abilist      |  26 +
>  sysdeps/x86_64/localplt.data                  |   1 -
>  104 files changed, 2557 insertions(+), 1266 deletions(-)
>  delete mode 100644 elf/tst-leaks1-static.c
>  create mode 100644 malloc/malloc-debug.c
>  delete mode 100644 malloc/malloc-hooks.h
>  create mode 100644 malloc/mcheck-impl.c
>  create mode 100644 malloc/mtrace-impl.c
>  create mode 100644 malloc/tst-compathooks-off.c
>  create mode 100644 malloc/tst-compathooks-on.c
>  delete mode 100644 malloc/tst-malloc-usable-static-tunables.c
>  delete mode 100644 malloc/tst-malloc-usable-static.c
>  create mode 100644 sysdeps/generic/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/mach/hurd/i386/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/aarch64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/alpha/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/arc/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/arm/be/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/arm/le/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/csky/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/hppa/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/i386/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/ia64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/m68k/coldfire/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/m68k/m680x0/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/microblaze/be/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/microblaze/le/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/mips/mips32/fpu/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/mips/mips64/n32/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/mips/mips64/n64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/nios2/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc64/be/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc64/le/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/s390/s390-32/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/s390/s390-64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/sh/be/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/sh/le/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/sparc/sparc32/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/sparc/sparc64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/x86_64/64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/x86_64/x32/libc_malloc_debug.abilist
>
> --
> 2.31.1
>
  

On 7/13/21 8:18 PM, H.J. Lu wrote:
> Does this series fix posix_memalign:
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=23489
> 

The series does fix a memalign+posix_memalign bug for less than 
MALLOC_ALIGNMENT alignment (patch 5/10), so I think it should.  If it 
doesn't, I'll track it down and fix it after the series is pushed.

Thanks,
Siddhesh
  

On 7/13/21 3:38 AM, Siddhesh Poyarekar wrote:
> This patchset removes the malloc hooks __malloc_hook, __free_hook,
> __realloc_hook and __memalign_hook from the API and leaves compatibility
> symbols so that existing applications can continue to link to them.  The
> reading and execution of the hooks has been moved to a DSO
> libc_malloc_debug.so, which can be preloaded for applications that need
> it.  By default these hooks no longer have any effect in the library.

OK for 2.34.

Tested without regression on x86_64 and i686.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>

Awesome!

Thank you for taclking this.

I appreciate you sticking with the guidance and feedback on the patch series.
 
> Further, debugging features such as MALLOC_CHECK_, mcheck() and mtrace
> have been weaned away from these hooks and also moved to
> libc_malloc_debug.so.  With this change, these features are only enabled
> when libc_malloc_debug.so is preloaded using LD_PRELOAD.

As expected.

> Finally, the __morecore, __morecore_after_hook and __default_morecore
> hooks have also been moved to compat symbols and removed from the API.
> Existing applications will continue to link to them but they won't have
> any effect on malloc behaviour.

As expected.

> Testing:
> 
> The patchset has gone through a full build and test on x86_64, i686,
> s390x, ppc64le, armv7l and aarch64 through a scratch build on Fedora
> rawhide.  I also installed a test package on x86_64 and rebooted to make
> sure the system boots to shell.  I did a build-many-glibcs run with
> the patchset and it ran clean on all targets.

Awesome.

> Changes from v7:
> - Made mcheck tests conditional on GLIBC_2.24
> - Added SHLIB_COMPAT guard around __malloc_initialize_hook usage in
>   libc_malloc_debug.so
> - Fixed hurd build failure.
> - Add another patch to fix malloc_usable_size to mcheck
> 
> Changes from v6:
> - Moved malloc-check into libc_malloc_debug.so. Tweaked malloc.c to
>   allow building twice, once inside libc.so and next in
>   libc_malloc_debug.so
> - Included morecore.c into malloc.c
> - Moved hook initialization too into libc_malloc_debug.so.
> - Interposed more functions in libc_malloc_debug.so
> - Versioned all symbols exported from libc_malloc_debug.so and finalized
>   them so that the library cannot be linked against
> 
> Changes from v5:
> - Rebased on latest master
> - Fixed realloc-mcheck
> - Removed residual mention of libmalloc_compathooks
> - Removed LD_PRELOAD from elf/tst-setuid
> 
> Changes from v4:
> - Patchset has a different approach, starting with moving out hooks
>   first to restrict all major malloc.c changes to the first patch
> - Renamed libmalloc_compathooks.so to libc_malloc_debug.so
> - Moved all debugging features into libc_malloc_debug.so
> - Made more documentation updates
> - Simplified __malloc_initialized variable use
> - Removed debugging tests on static variables since that is no longer
>   supported
> 
> Changes from v3:
> - Remove source file dependencies
> - Commit mcheck tests
> 
> Changes from v2:
> - Move hooks dependencies to malloc.o{,sS}
> 
> Changes from v1:
> 
> - Added makefile dependencies for the new hooks files
> - Fixed memset call in calloc debugging hooks
> - Added the tr_break deprecation patch and mcheck test patch to this
>   series
> 
> 
> Siddhesh Poyarekar (10):
>   Make mcheck tests conditional on GLIBC_2.24 or earlier
>   Remove __after_morecore_hook
>   Remove __morecore and __default_morecore
>   Move malloc hooks into a compat DSO
>   mcheck: Wean away from malloc hooks
>   Simplify __malloc_initialized
>   mtrace: Wean away from malloc hooks
>   glibc.malloc.check: Wean away from malloc hooks
>   Remove malloc hooks
>   mcheck Fix malloc_usable_size [BZ #22057]
> 
>  NEWS                                          |  21 +
>  Rules                                         |   9 +-
>  catgets/Makefile                              |   4 +-
>  elf/Makefile                                  |  15 +-
>  elf/tst-leaks1-static.c                       |   1 -
>  iconvdata/Makefile                            |   3 +-
>  include/malloc.h                              |   6 -
>  include/mcheck.h                              |   4 -
>  include/stdlib.h                              |   3 -
>  intl/tst-gettext.sh                           |   1 +
>  libio/Makefile                                |  12 +-
>  localedata/Makefile                           |   3 +-
>  malloc/Makefile                               |  56 +-
>  malloc/Versions                               |  44 ++
>  malloc/arena.c                                |  48 +-
>  malloc/hooks.c                                |  71 ++-
>  malloc/malloc-check.c                         |  63 ++-
>  malloc/malloc-debug.c                         | 522 ++++++++++++++++++
>  malloc/malloc-hooks.h                         |  24 -
>  malloc/malloc.c                               | 163 ++----
>  malloc/malloc.h                               |  27 -
>  malloc/mcheck-impl.c                          | 412 ++++++++++++++
>  malloc/mcheck.c                               | 397 +------------
>  malloc/morecore.c                             |  34 +-
>  malloc/mtrace-impl.c                          | 226 ++++++++
>  malloc/mtrace.c                               | 313 +----------
>  malloc/tst-compathooks-off.c                  | 145 +++++
>  malloc/tst-compathooks-on.c                   |   2 +
>  malloc/tst-malloc-usable-static-tunables.c    |   1 -
>  malloc/tst-malloc-usable-static.c             |   1 -
>  malloc/tst-mtrace.sh                          |   1 +
>  manual/memory.texi                            | 207 +------
>  manual/tunables.texi                          |   4 +-
>  misc/Makefile                                 |   6 +-
>  nptl/Makefile                                 |   3 +-
>  posix/Makefile                                |  48 +-
>  resolv/Makefile                               |   9 +-
>  shlib-versions                                |   3 +
>  stdio-common/Makefile                         |  15 +-
>  sysdeps/aarch64/Makefile                      |   3 +
>  sysdeps/generic/libc_malloc_debug.abilist     |   0
>  sysdeps/generic/localplt.data                 |   1 -
>  sysdeps/mach/hurd/Makefile                    |   1 +
>  .../mach/hurd/i386/libc_malloc_debug.abilist  |  26 +
>  sysdeps/mach/hurd/i386/localplt.data          |   1 -
>  sysdeps/pthread/Makefile                      |   3 +-
>  .../linux/aarch64/libc_malloc_debug.abilist   |  26 +
>  sysdeps/unix/sysv/linux/aarch64/localplt.data |   1 -
>  .../linux/alpha/libc_malloc_debug.abilist     |  26 +
>  sysdeps/unix/sysv/linux/alpha/localplt.data   |   1 -
>  .../sysv/linux/arc/libc_malloc_debug.abilist  |  26 +
>  sysdeps/unix/sysv/linux/arc/localplt.data     |   1 -
>  .../linux/arm/be/libc_malloc_debug.abilist    |  26 +
>  .../linux/arm/le/libc_malloc_debug.abilist    |  26 +
>  sysdeps/unix/sysv/linux/arm/localplt.data     |   1 -
>  .../sysv/linux/csky/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/csky/localplt.data    |   1 -
>  .../sysv/linux/hppa/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/hppa/localplt.data    |   1 -
>  sysdeps/unix/sysv/linux/hppa/shlib-versions   |   2 +
>  .../sysv/linux/i386/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/i386/localplt.data    |   1 -
>  .../sysv/linux/ia64/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/ia64/localplt.data    |   1 -
>  sysdeps/unix/sysv/linux/ia64/shlib-versions   |   2 +
>  .../m68k/coldfire/libc_malloc_debug.abilist   |  26 +
>  .../sysv/linux/m68k/coldfire/localplt.data    |   1 -
>  .../m68k/m680x0/libc_malloc_debug.abilist     |  26 +
>  .../unix/sysv/linux/m68k/m680x0/localplt.data |   1 -
>  .../microblaze/be/libc_malloc_debug.abilist   |  26 +
>  .../microblaze/le/libc_malloc_debug.abilist   |  26 +
>  .../unix/sysv/linux/microblaze/localplt.data  |   1 -
>  .../mips/mips32/fpu/libc_malloc_debug.abilist |  26 +
>  .../mips32/nofpu/libc_malloc_debug.abilist    |  26 +
>  .../mips/mips64/n32/libc_malloc_debug.abilist |  26 +
>  .../mips/mips64/n64/libc_malloc_debug.abilist |  26 +
>  sysdeps/unix/sysv/linux/mips/shlib-versions   |   2 +
>  .../linux/nios2/libc_malloc_debug.abilist     |  26 +
>  sysdeps/unix/sysv/linux/nios2/localplt.data   |   1 -
>  .../powerpc32/fpu/libc_malloc_debug.abilist   |  26 +
>  .../linux/powerpc/powerpc32/fpu/localplt.data |   1 -
>  .../powerpc32/nofpu/libc_malloc_debug.abilist |  26 +
>  .../powerpc/powerpc32/nofpu/localplt.data     |   1 -
>  .../powerpc64/be/libc_malloc_debug.abilist    |  26 +
>  .../powerpc64/le/libc_malloc_debug.abilist    |  26 +
>  .../linux/powerpc/powerpc64/localplt.data     |   1 -
>  sysdeps/unix/sysv/linux/riscv/localplt.data   |   1 -
>  .../riscv/rv32/libc_malloc_debug.abilist      |  26 +
>  .../riscv/rv64/libc_malloc_debug.abilist      |  26 +
>  sysdeps/unix/sysv/linux/s390/localplt.data    |   1 -
>  .../s390/s390-32/libc_malloc_debug.abilist    |  26 +
>  .../s390/s390-64/libc_malloc_debug.abilist    |  26 +
>  .../linux/sh/be/libc_malloc_debug.abilist     |  26 +
>  .../linux/sh/le/libc_malloc_debug.abilist     |  26 +
>  sysdeps/unix/sysv/linux/sh/localplt.data      |   1 -
>  sysdeps/unix/sysv/linux/sh/shlib-versions     |   1 +
>  .../sparc/sparc32/libc_malloc_debug.abilist   |  26 +
>  .../sysv/linux/sparc/sparc32/localplt.data    |   1 -
>  .../sparc/sparc64/libc_malloc_debug.abilist   |  26 +
>  .../sysv/linux/sparc/sparc64/localplt.data    |   1 -
>  .../sysv/linux/sparc/sparc64/shlib-versions   |   1 +
>  .../linux/x86_64/64/libc_malloc_debug.abilist |  26 +
>  .../x86_64/x32/libc_malloc_debug.abilist      |  26 +
>  sysdeps/x86_64/localplt.data                  |   1 -
>  104 files changed, 2557 insertions(+), 1266 deletions(-)
>  delete mode 100644 elf/tst-leaks1-static.c
>  create mode 100644 malloc/malloc-debug.c
>  delete mode 100644 malloc/malloc-hooks.h
>  create mode 100644 malloc/mcheck-impl.c
>  create mode 100644 malloc/mtrace-impl.c
>  create mode 100644 malloc/tst-compathooks-off.c
>  create mode 100644 malloc/tst-compathooks-on.c
>  delete mode 100644 malloc/tst-malloc-usable-static-tunables.c
>  delete mode 100644 malloc/tst-malloc-usable-static.c
>  create mode 100644 sysdeps/generic/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/mach/hurd/i386/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/aarch64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/alpha/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/arc/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/arm/be/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/arm/le/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/csky/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/hppa/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/i386/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/ia64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/m68k/coldfire/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/m68k/m680x0/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/microblaze/be/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/microblaze/le/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/mips/mips32/fpu/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/mips/mips64/n32/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/mips/mips64/n64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/nios2/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc64/be/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/powerpc/powerpc64/le/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/s390/s390-32/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/s390/s390-64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/sh/be/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/sh/le/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/sparc/sparc32/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/sparc/sparc64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/x86_64/64/libc_malloc_debug.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/x86_64/x32/libc_malloc_debug.abilist
>