diff mbox series

[v8,5/6] elf: Align large load segments to PMD huge page size for THP

Message ID 20260405035555.558396-1-wangrui@loongson.cn (mailing list archive)
State Superseded
Headers
Series elf: THP-aware load segment alignment |

Checks

Context Check Description
redhat-pt-bot/TryBot-apply_patch success Patch applied to master at the time it was sent
linaro-tcwg-bot/tcwg_glibc_build--master-aarch64 success Build passed
linaro-tcwg-bot/tcwg_glibc_check--master-aarch64 success Test passed
linaro-tcwg-bot/tcwg_glibc_build--master-arm success Build passed
linaro-tcwg-bot/tcwg_glibc_check--master-arm success Test passed

Commit Message

WANG Rui April 5, 2026, 3:55 a.m. UTC 
  Mapping segments that are at least the size of a PMD huge page to
huge-page-aligned addresses helps make them eligible for Transparent
Huge Pages (THP).

This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
to determine an appropriate maximum alignment for ELF load segments based
on the system THP policy. The optimization is enabled only when the glibc
tunable `glibc.elf.thp=1` is set and THP is configured to be used
unconditionally.

The optimization depends on Linux kernel support for file-backed THP,
specifically:

* `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
* `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.

When enabled, the helper queries the default THP page size and uses it
to align sufficiently large load segments that are already properly
aligned in both virtual address and file offset (e.g., zero).

For eligible segments, the alignment is bumped to the THP page size,
which improves THP eligibility, reduces TLB pressure, and improves
performance for large objects. To avoid excessive address space padding
on systems with very large THP sizes, the alignment is capped at 32MB.
The optimization is applied only to non-writable segments, matching
typical THP usage.

Signed-off-by: WANG Rui <wangrui@loongson.cn>
---
 sysdeps/unix/sysv/linux/Makefile              |  1 +
 .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
 .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
 3 files changed, 81 insertions(+)
 create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
 create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h

Comments

H.J. Lu April 6, 2026, 10:45 p.m. UTC | #1 
On Sun, Apr 5, 2026 at 11:56 AM WANG Rui <wangrui@loongson.cn> wrote:
>
> Mapping segments that are at least the size of a PMD huge page to
> huge-page-aligned addresses helps make them eligible for Transparent
> Huge Pages (THP).
>
> This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
> to determine an appropriate maximum alignment for ELF load segments based
> on the system THP policy. The optimization is enabled only when the glibc
> tunable `glibc.elf.thp=1` is set and THP is configured to be used
> unconditionally.
>
> The optimization depends on Linux kernel support for file-backed THP,
> specifically:
>
> * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
>
> When enabled, the helper queries the default THP page size and uses it
> to align sufficiently large load segments that are already properly
> aligned in both virtual address and file offset (e.g., zero).
>
> For eligible segments, the alignment is bumped to the THP page size,
> which improves THP eligibility, reduces TLB pressure, and improves
> performance for large objects. To avoid excessive address space padding
> on systems with very large THP sizes, the alignment is capped at 32MB.
> The optimization is applied only to non-writable segments, matching
> typical THP usage.
>
> Signed-off-by: WANG Rui <wangrui@loongson.cn>
> ---
>  sysdeps/unix/sysv/linux/Makefile              |  1 +
>  .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
>  .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
>  3 files changed, 81 insertions(+)
>  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
>  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h
>
> diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
> index c6bd97abf1a..9879fec9a93 100644
> --- a/sysdeps/unix/sysv/linux/Makefile
> +++ b/sysdeps/unix/sysv/linux/Makefile
> @@ -673,6 +673,7 @@ endif
>
>  ifeq ($(subdir),elf)
>  dl-routines += \
> +  dl-map-segment-align \
>    dl-rseq-symbols \
>    # dl-routines
>
> diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> new file mode 100644
> index 00000000000..75fb0496730
> --- /dev/null
> +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> @@ -0,0 +1,53 @@
> +/* _dl_map_segment_align.  Linux version.
> +   Copyright (C) 2026 Free Software Foundation, Inc.
> +   Copyright The GNU Toolchain Authors.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <dl-map-segment-align.h>
> +#include <dl-tunables.h>
> +#include <hugepages.h>
> +
> +ElfW (Addr)
> +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> +{
> +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> +  static unsigned long int thp_pagesize;
> +
> +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> +    return p_align_max;
> +
> +  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
> +                       || thp_pagesize == 0))
> +    {
> +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
> +      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();
> +    }
> +
> +  /* Aligning load segments that are large enough to the PMD size helps
> +     improve THP eligibility and reduces TLB pressure.
> +     We cap the huge page size at MAX_THP_PAGESIZE to avoid over-aligning
> +     on systems with very large normal pages (like 64K pages with 512M
> +     huge pages). */
> +  if (thp_mode == thp_mode_always && thp_pagesize <= MAX_THP_PAGESIZE
> +      && ((c->mapstart | c->mapoff) & (thp_pagesize - 1)) == 0
> +      && (c->mapend - c->mapstart) >= thp_pagesize
> +      && p_align_max < thp_pagesize && (c->prot & PROT_WRITE) == 0)
> +    return thp_pagesize;
> +
> +  return p_align_max;
> +}
> diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.h b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> new file mode 100644
> index 00000000000..83794a03986
> --- /dev/null
> +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> @@ -0,0 +1,27 @@
> +/* _dl_map_segment_align.  Linux version.
> +   Copyright (C) 2026 Free Software Foundation, Inc.
> +   Copyright The GNU Toolchain Authors.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <dl-load.h>
> +
> +#ifndef DL_MAP_DEFAULT_THP_PAGESIZE
> +# define DL_MAP_DEFAULT_THP_PAGESIZE   0
> +#endif
> +
> +ElfW (Addr)
> +    _dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max);

extern ElfW (Addr) _dl_map_segment_align (const struct loadcmd *, ElfW (Addr));

> --
> 2.53.0
>
H.J. Lu April 6, 2026, 11:10 p.m. UTC | #2 
On Tue, Apr 7, 2026 at 6:45 AM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Sun, Apr 5, 2026 at 11:56 AM WANG Rui <wangrui@loongson.cn> wrote:
> >
> > Mapping segments that are at least the size of a PMD huge page to
> > huge-page-aligned addresses helps make them eligible for Transparent
> > Huge Pages (THP).
> >
> > This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
> > to determine an appropriate maximum alignment for ELF load segments based
> > on the system THP policy. The optimization is enabled only when the glibc
> > tunable `glibc.elf.thp=1` is set and THP is configured to be used
> > unconditionally.
> >
> > The optimization depends on Linux kernel support for file-backed THP,
> > specifically:
> >
> > * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> > * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
> >
> > When enabled, the helper queries the default THP page size and uses it
> > to align sufficiently large load segments that are already properly
> > aligned in both virtual address and file offset (e.g., zero).
> >
> > For eligible segments, the alignment is bumped to the THP page size,
> > which improves THP eligibility, reduces TLB pressure, and improves
> > performance for large objects. To avoid excessive address space padding
> > on systems with very large THP sizes, the alignment is capped at 32MB.
> > The optimization is applied only to non-writable segments, matching
> > typical THP usage.
> >
> > Signed-off-by: WANG Rui <wangrui@loongson.cn>
> > ---
> >  sysdeps/unix/sysv/linux/Makefile              |  1 +
> >  .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
> >  .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
> >  3 files changed, 81 insertions(+)
> >  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
> >  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h
> >
> > diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
> > index c6bd97abf1a..9879fec9a93 100644
> > --- a/sysdeps/unix/sysv/linux/Makefile
> > +++ b/sysdeps/unix/sysv/linux/Makefile
> > @@ -673,6 +673,7 @@ endif
> >
> >  ifeq ($(subdir),elf)
> >  dl-routines += \
> > +  dl-map-segment-align \
> >    dl-rseq-symbols \
> >    # dl-routines
> >
> > diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> > new file mode 100644
> > index 00000000000..75fb0496730
> > --- /dev/null
> > +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> > @@ -0,0 +1,53 @@
> > +/* _dl_map_segment_align.  Linux version.
> > +   Copyright (C) 2026 Free Software Foundation, Inc.
> > +   Copyright The GNU Toolchain Authors.
> > +   This file is part of the GNU C Library.
> > +
> > +   The GNU C Library is free software; you can redistribute it and/or
> > +   modify it under the terms of the GNU Lesser General Public
> > +   License as published by the Free Software Foundation; either
> > +   version 2.1 of the License, or (at your option) any later version.
> > +
> > +   The GNU C Library is distributed in the hope that it will be useful,
> > +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +   Lesser General Public License for more details.
> > +
> > +   You should have received a copy of the GNU Lesser General Public
> > +   License along with the GNU C Library; if not, see
> > +   <https://www.gnu.org/licenses/>.  */
> > +
> > +#include <dl-map-segment-align.h>
> > +#include <dl-tunables.h>
> > +#include <hugepages.h>
> > +
> > +ElfW (Addr)
> > +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> > +{
> > +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> > +  static unsigned long int thp_pagesize;
> > +
> > +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> > +    return p_align_max;
> > +
> > +  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
> > +                       || thp_pagesize == 0))
> > +    {
> > +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> > +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
> > +      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();
> > +    }
> > +
> > +  /* Aligning load segments that are large enough to the PMD size helps
> > +     improve THP eligibility and reduces TLB pressure.
> > +     We cap the huge page size at MAX_THP_PAGESIZE to avoid over-aligning
> > +     on systems with very large normal pages (like 64K pages with 512M
> > +     huge pages). */
> > +  if (thp_mode == thp_mode_always && thp_pagesize <= MAX_THP_PAGESIZE
> > +      && ((c->mapstart | c->mapoff) & (thp_pagesize - 1)) == 0
> > +      && (c->mapend - c->mapstart) >= thp_pagesize
> > +      && p_align_max < thp_pagesize && (c->prot & PROT_WRITE) == 0)
> > +    return thp_pagesize;
> > +
> > +  return p_align_max;
> > +}
> > diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.h b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> > new file mode 100644
> > index 00000000000..83794a03986
> > --- /dev/null
> > +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> > @@ -0,0 +1,27 @@
> > +/* _dl_map_segment_align.  Linux version.
> > +   Copyright (C) 2026 Free Software Foundation, Inc.
> > +   Copyright The GNU Toolchain Authors.
> > +   This file is part of the GNU C Library.
> > +
> > +   The GNU C Library is free software; you can redistribute it and/or
> > +   modify it under the terms of the GNU Lesser General Public
> > +   License as published by the Free Software Foundation; either
> > +   version 2.1 of the License, or (at your option) any later version.
> > +
> > +   The GNU C Library is distributed in the hope that it will be useful,
> > +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +   Lesser General Public License for more details.
> > +
> > +   You should have received a copy of the GNU Lesser General Public
> > +   License along with the GNU C Library; if not, see
> > +   <https://www.gnu.org/licenses/>.  */
> > +
> > +#include <dl-load.h>
> > +
> > +#ifndef DL_MAP_DEFAULT_THP_PAGESIZE
> > +# define DL_MAP_DEFAULT_THP_PAGESIZE   0
> > +#endif
> > +
> > +ElfW (Addr)
> > +    _dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max);
>
> extern ElfW (Addr) _dl_map_segment_align (const struct loadcmd *, ElfW (Addr));
>

extern ElfW (Addr) _dl_map_segment_align
  (const struct loadcmd *, ElfW (Addr)) attribute_hidden;
H.J. Lu April 9, 2026, 10:38 p.m. UTC | #3 
On Sun, Apr 5, 2026 at 11:56 AM WANG Rui <wangrui@loongson.cn> wrote:
>
> Mapping segments that are at least the size of a PMD huge page to
> huge-page-aligned addresses helps make them eligible for Transparent
> Huge Pages (THP).
>
> This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
> to determine an appropriate maximum alignment for ELF load segments based
> on the system THP policy. The optimization is enabled only when the glibc
> tunable `glibc.elf.thp=1` is set and THP is configured to be used
> unconditionally.
>
> The optimization depends on Linux kernel support for file-backed THP,
> specifically:
>
> * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
>
> When enabled, the helper queries the default THP page size and uses it
> to align sufficiently large load segments that are already properly
> aligned in both virtual address and file offset (e.g., zero).
>
> For eligible segments, the alignment is bumped to the THP page size,
> which improves THP eligibility, reduces TLB pressure, and improves
> performance for large objects. To avoid excessive address space padding
> on systems with very large THP sizes, the alignment is capped at 32MB.
> The optimization is applied only to non-writable segments, matching
> typical THP usage.
>
> Signed-off-by: WANG Rui <wangrui@loongson.cn>
> ---
>  sysdeps/unix/sysv/linux/Makefile              |  1 +
>  .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
>  .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
>  3 files changed, 81 insertions(+)
>  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
>  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h
>
> diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
> index c6bd97abf1a..9879fec9a93 100644
> --- a/sysdeps/unix/sysv/linux/Makefile
> +++ b/sysdeps/unix/sysv/linux/Makefile
> @@ -673,6 +673,7 @@ endif
>
>  ifeq ($(subdir),elf)
>  dl-routines += \
> +  dl-map-segment-align \
>    dl-rseq-symbols \
>    # dl-routines
>
> diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> new file mode 100644
> index 00000000000..75fb0496730
> --- /dev/null
> +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> @@ -0,0 +1,53 @@
> +/* _dl_map_segment_align.  Linux version.
> +   Copyright (C) 2026 Free Software Foundation, Inc.
> +   Copyright The GNU Toolchain Authors.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <dl-map-segment-align.h>
> +#include <dl-tunables.h>
> +#include <hugepages.h>
> +
> +ElfW (Addr)
> +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> +{
> +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> +  static unsigned long int thp_pagesize;
> +
> +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> +    return p_align_max;
> +
> +  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
> +                       || thp_pagesize == 0))
> +    {
> +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
> +      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();

We should call __get_hugepage_config to get both the THP page size as well
as the THP flags.  The THP flags should be passed to mmap.

> +    }
> +
> +  /* Aligning load segments that are large enough to the PMD size helps
> +     improve THP eligibility and reduces TLB pressure.
> +     We cap the huge page size at MAX_THP_PAGESIZE to avoid over-aligning
> +     on systems with very large normal pages (like 64K pages with 512M
> +     huge pages). */
> +  if (thp_mode == thp_mode_always && thp_pagesize <= MAX_THP_PAGESIZE
> +      && ((c->mapstart | c->mapoff) & (thp_pagesize - 1)) == 0
> +      && (c->mapend - c->mapstart) >= thp_pagesize
> +      && p_align_max < thp_pagesize && (c->prot & PROT_WRITE) == 0)

Please put one condition per line.

> +    return thp_pagesize;
> +
> +  return p_align_max;
> +}
> diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.h b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> new file mode 100644
> index 00000000000..83794a03986
> --- /dev/null
> +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> @@ -0,0 +1,27 @@
> +/* _dl_map_segment_align.  Linux version.
> +   Copyright (C) 2026 Free Software Foundation, Inc.
> +   Copyright The GNU Toolchain Authors.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <dl-load.h>
> +
> +#ifndef DL_MAP_DEFAULT_THP_PAGESIZE
> +# define DL_MAP_DEFAULT_THP_PAGESIZE   0
> +#endif
> +
> +ElfW (Addr)
> +    _dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max);
> --
> 2.53.0
>
H.J. Lu April 9, 2026, 10:43 p.m. UTC | #4 
On Fri, Apr 10, 2026 at 6:38 AM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Sun, Apr 5, 2026 at 11:56 AM WANG Rui <wangrui@loongson.cn> wrote:
> >
> > Mapping segments that are at least the size of a PMD huge page to
> > huge-page-aligned addresses helps make them eligible for Transparent
> > Huge Pages (THP).
> >
> > This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
> > to determine an appropriate maximum alignment for ELF load segments based
> > on the system THP policy. The optimization is enabled only when the glibc
> > tunable `glibc.elf.thp=1` is set and THP is configured to be used
> > unconditionally.
> >
> > The optimization depends on Linux kernel support for file-backed THP,
> > specifically:
> >
> > * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> > * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
> >
> > When enabled, the helper queries the default THP page size and uses it
> > to align sufficiently large load segments that are already properly
> > aligned in both virtual address and file offset (e.g., zero).
> >
> > For eligible segments, the alignment is bumped to the THP page size,
> > which improves THP eligibility, reduces TLB pressure, and improves
> > performance for large objects. To avoid excessive address space padding
> > on systems with very large THP sizes, the alignment is capped at 32MB.
> > The optimization is applied only to non-writable segments, matching
> > typical THP usage.
> >
> > Signed-off-by: WANG Rui <wangrui@loongson.cn>
> > ---
> >  sysdeps/unix/sysv/linux/Makefile              |  1 +
> >  .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
> >  .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
> >  3 files changed, 81 insertions(+)
> >  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
> >  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h
> >
> > diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
> > index c6bd97abf1a..9879fec9a93 100644
> > --- a/sysdeps/unix/sysv/linux/Makefile
> > +++ b/sysdeps/unix/sysv/linux/Makefile
> > @@ -673,6 +673,7 @@ endif
> >
> >  ifeq ($(subdir),elf)
> >  dl-routines += \
> > +  dl-map-segment-align \
> >    dl-rseq-symbols \
> >    # dl-routines
> >
> > diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> > new file mode 100644
> > index 00000000000..75fb0496730
> > --- /dev/null
> > +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> > @@ -0,0 +1,53 @@
> > +/* _dl_map_segment_align.  Linux version.
> > +   Copyright (C) 2026 Free Software Foundation, Inc.
> > +   Copyright The GNU Toolchain Authors.
> > +   This file is part of the GNU C Library.
> > +
> > +   The GNU C Library is free software; you can redistribute it and/or
> > +   modify it under the terms of the GNU Lesser General Public
> > +   License as published by the Free Software Foundation; either
> > +   version 2.1 of the License, or (at your option) any later version.
> > +
> > +   The GNU C Library is distributed in the hope that it will be useful,
> > +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +   Lesser General Public License for more details.
> > +
> > +   You should have received a copy of the GNU Lesser General Public
> > +   License along with the GNU C Library; if not, see
> > +   <https://www.gnu.org/licenses/>.  */
> > +
> > +#include <dl-map-segment-align.h>
> > +#include <dl-tunables.h>
> > +#include <hugepages.h>
> > +
> > +ElfW (Addr)
> > +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> > +{
> > +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> > +  static unsigned long int thp_pagesize;
> > +
> > +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> > +    return p_align_max;
> > +
> > +  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
> > +                       || thp_pagesize == 0))
> > +    {
> > +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> > +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
> > +      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();
>
> We should call __get_hugepage_config to get both the THP page size as well
> as the THP flags.  The THP flags should be passed to mmap.
>
> > +    }
> > +
> > +  /* Aligning load segments that are large enough to the PMD size helps
> > +     improve THP eligibility and reduces TLB pressure.
> > +     We cap the huge page size at MAX_THP_PAGESIZE to avoid over-aligning
> > +     on systems with very large normal pages (like 64K pages with 512M
> > +     huge pages). */
> > +  if (thp_mode == thp_mode_always && thp_pagesize <= MAX_THP_PAGESIZE
> > +      && ((c->mapstart | c->mapoff) & (thp_pagesize - 1)) == 0
> > +      && (c->mapend - c->mapstart) >= thp_pagesize
> > +      && p_align_max < thp_pagesize && (c->prot & PROT_WRITE) == 0)
>
> Please put one condition per line.
>
> > +    return thp_pagesize;
> > +
> > +  return p_align_max;
> > +}
> > diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.h b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> > new file mode 100644
> > index 00000000000..83794a03986
> > --- /dev/null
> > +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> > @@ -0,0 +1,27 @@
> > +/* _dl_map_segment_align.  Linux version.
> > +   Copyright (C) 2026 Free Software Foundation, Inc.
> > +   Copyright The GNU Toolchain Authors.
> > +   This file is part of the GNU C Library.
> > +
> > +   The GNU C Library is free software; you can redistribute it and/or
> > +   modify it under the terms of the GNU Lesser General Public
> > +   License as published by the Free Software Foundation; either
> > +   version 2.1 of the License, or (at your option) any later version.
> > +
> > +   The GNU C Library is distributed in the hope that it will be useful,
> > +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +   Lesser General Public License for more details.
> > +
> > +   You should have received a copy of the GNU Lesser General Public
> > +   License along with the GNU C Library; if not, see
> > +   <https://www.gnu.org/licenses/>.  */
> > +
> > +#include <dl-load.h>
> > +
> > +#ifndef DL_MAP_DEFAULT_THP_PAGESIZE
> > +# define DL_MAP_DEFAULT_THP_PAGESIZE   0

Can we drop DL_MAP_DEFAULT_THP_PAGESIZE and call
__get_hugepage_config to get the THP page size?

> > +#endif
> > +
> > +ElfW (Addr)
> > +    _dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max);
> > --
> > 2.53.0
> >
>
>
> --
> H.J.
Wilco Dijkstra April 9, 2026, 10:45 p.m. UTC | #5 
Hi HJ,

> +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
> +      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();

> We should call __get_hugepage_config to get both the THP page size as well
> as the THP flags.  The THP flags should be passed to mmap.

No - this is about THP, not hugetlbfs.

Cheers,
Wilco
H.J. Lu April 9, 2026, 11:47 p.m. UTC | #6 
On Fri, Apr 10, 2026 at 6:46 AM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
> Hi HJ,
>
> > +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> > +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
> > +      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();
>
> > We should call __get_hugepage_config to get both the THP page size as well
> > as the THP flags.  The THP flags should be passed to mmap.
>
> No - this is about THP, not hugetlbfs.
>
> Cheers,
> Wilco

Do we need to call __madvise with MADV_HUGEPAGE?
H.J. Lu April 9, 2026, 11:58 p.m. UTC | #7 
On Sun, Apr 5, 2026 at 11:56 AM WANG Rui <wangrui@loongson.cn> wrote:
>
> Mapping segments that are at least the size of a PMD huge page to
> huge-page-aligned addresses helps make them eligible for Transparent
> Huge Pages (THP).
>
> This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
> to determine an appropriate maximum alignment for ELF load segments based
> on the system THP policy. The optimization is enabled only when the glibc
> tunable `glibc.elf.thp=1` is set and THP is configured to be used
> unconditionally.
>
> The optimization depends on Linux kernel support for file-backed THP,
> specifically:
>
> * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
>
> When enabled, the helper queries the default THP page size and uses it
> to align sufficiently large load segments that are already properly
> aligned in both virtual address and file offset (e.g., zero).
>
> For eligible segments, the alignment is bumped to the THP page size,
> which improves THP eligibility, reduces TLB pressure, and improves
> performance for large objects. To avoid excessive address space padding
> on systems with very large THP sizes, the alignment is capped at 32MB.
> The optimization is applied only to non-writable segments, matching
> typical THP usage.
>
> Signed-off-by: WANG Rui <wangrui@loongson.cn>
> ---
>  sysdeps/unix/sysv/linux/Makefile              |  1 +
>  .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
>  .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
>  3 files changed, 81 insertions(+)
>  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
>  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h
>
> diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
> index c6bd97abf1a..9879fec9a93 100644
> --- a/sysdeps/unix/sysv/linux/Makefile
> +++ b/sysdeps/unix/sysv/linux/Makefile
> @@ -673,6 +673,7 @@ endif
>
>  ifeq ($(subdir),elf)
>  dl-routines += \
> +  dl-map-segment-align \
>    dl-rseq-symbols \
>    # dl-routines
>
> diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> new file mode 100644
> index 00000000000..75fb0496730
> --- /dev/null
> +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> @@ -0,0 +1,53 @@
> +/* _dl_map_segment_align.  Linux version.
> +   Copyright (C) 2026 Free Software Foundation, Inc.
> +   Copyright The GNU Toolchain Authors.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <dl-map-segment-align.h>
> +#include <dl-tunables.h>
> +#include <hugepages.h>
> +
> +ElfW (Addr)
> +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> +{
> +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> +  static unsigned long int thp_pagesize;
> +
> +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> +    return p_align_max;
> +
> +  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
> +                       || thp_pagesize == 0))
> +    {
> +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();

Isn't THP mode controlled by kernel?  I think we should also call
__madvise with MADV_HUGEPAGE to enable THP.

> +      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();
> +    }
> +
> +  /* Aligning load segments that are large enough to the PMD size helps
> +     improve THP eligibility and reduces TLB pressure.
> +     We cap the huge page size at MAX_THP_PAGESIZE to avoid over-aligning
> +     on systems with very large normal pages (like 64K pages with 512M
> +     huge pages). */
> +  if (thp_mode == thp_mode_always && thp_pagesize <= MAX_THP_PAGESIZE
> +      && ((c->mapstart | c->mapoff) & (thp_pagesize - 1)) == 0
> +      && (c->mapend - c->mapstart) >= thp_pagesize
> +      && p_align_max < thp_pagesize && (c->prot & PROT_WRITE) == 0)
> +    return thp_pagesize;
> +
> +  return p_align_max;
> +}
> diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.h b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> new file mode 100644
> index 00000000000..83794a03986
> --- /dev/null
> +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> @@ -0,0 +1,27 @@
> +/* _dl_map_segment_align.  Linux version.
> +   Copyright (C) 2026 Free Software Foundation, Inc.
> +   Copyright The GNU Toolchain Authors.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <dl-load.h>
> +
> +#ifndef DL_MAP_DEFAULT_THP_PAGESIZE
> +# define DL_MAP_DEFAULT_THP_PAGESIZE   0
> +#endif
> +
> +ElfW (Addr)
> +    _dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max);
> --
> 2.53.0
>
H.J. Lu April 10, 2026, 12:57 a.m. UTC | #8 
On Fri, Apr 10, 2026 at 7:58 AM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Sun, Apr 5, 2026 at 11:56 AM WANG Rui <wangrui@loongson.cn> wrote:
> >
> > Mapping segments that are at least the size of a PMD huge page to
> > huge-page-aligned addresses helps make them eligible for Transparent
> > Huge Pages (THP).
> >
> > This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
> > to determine an appropriate maximum alignment for ELF load segments based
> > on the system THP policy. The optimization is enabled only when the glibc
> > tunable `glibc.elf.thp=1` is set and THP is configured to be used
> > unconditionally.
> >
> > The optimization depends on Linux kernel support for file-backed THP,
> > specifically:
> >
> > * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> > * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
> >
> > When enabled, the helper queries the default THP page size and uses it
> > to align sufficiently large load segments that are already properly
> > aligned in both virtual address and file offset (e.g., zero).
> >
> > For eligible segments, the alignment is bumped to the THP page size,
> > which improves THP eligibility, reduces TLB pressure, and improves
> > performance for large objects. To avoid excessive address space padding
> > on systems with very large THP sizes, the alignment is capped at 32MB.
> > The optimization is applied only to non-writable segments, matching
> > typical THP usage.
> >
> > Signed-off-by: WANG Rui <wangrui@loongson.cn>
> > ---
> >  sysdeps/unix/sysv/linux/Makefile              |  1 +
> >  .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
> >  .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
> >  3 files changed, 81 insertions(+)
> >  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
> >  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h
> >
> > diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
> > index c6bd97abf1a..9879fec9a93 100644
> > --- a/sysdeps/unix/sysv/linux/Makefile
> > +++ b/sysdeps/unix/sysv/linux/Makefile
> > @@ -673,6 +673,7 @@ endif
> >
> >  ifeq ($(subdir),elf)
> >  dl-routines += \
> > +  dl-map-segment-align \
> >    dl-rseq-symbols \
> >    # dl-routines
> >
> > diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> > new file mode 100644
> > index 00000000000..75fb0496730
> > --- /dev/null
> > +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> > @@ -0,0 +1,53 @@
> > +/* _dl_map_segment_align.  Linux version.
> > +   Copyright (C) 2026 Free Software Foundation, Inc.
> > +   Copyright The GNU Toolchain Authors.
> > +   This file is part of the GNU C Library.
> > +
> > +   The GNU C Library is free software; you can redistribute it and/or
> > +   modify it under the terms of the GNU Lesser General Public
> > +   License as published by the Free Software Foundation; either
> > +   version 2.1 of the License, or (at your option) any later version.
> > +
> > +   The GNU C Library is distributed in the hope that it will be useful,
> > +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +   Lesser General Public License for more details.
> > +
> > +   You should have received a copy of the GNU Lesser General Public
> > +   License along with the GNU C Library; if not, see
> > +   <https://www.gnu.org/licenses/>.  */
> > +
> > +#include <dl-map-segment-align.h>
> > +#include <dl-tunables.h>
> > +#include <hugepages.h>
> > +
> > +ElfW (Addr)
> > +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> > +{
> > +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> > +  static unsigned long int thp_pagesize;
> > +
> > +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> > +    return p_align_max;
> > +
> > +  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
> > +                       || thp_pagesize == 0))
> > +    {
> > +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> > +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
>
> Isn't THP mode controlled by kernel?  I think we should also call

We should always do

thp_mode = __get_thp_mode ();

> __madvise with MADV_HUGEPAGE to enable THP.

We can call  __madvise after _dl_postprocess_loadcmd is called if
thp_mode == thp_mode_madvise.

>
> > +      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();
> > +    }
> > +
> > +  /* Aligning load segments that are large enough to the PMD size helps
> > +     improve THP eligibility and reduces TLB pressure.
> > +     We cap the huge page size at MAX_THP_PAGESIZE to avoid over-aligning
> > +     on systems with very large normal pages (like 64K pages with 512M
> > +     huge pages). */
> > +  if (thp_mode == thp_mode_always && thp_pagesize <= MAX_THP_PAGESIZE
> > +      && ((c->mapstart | c->mapoff) & (thp_pagesize - 1)) == 0
> > +      && (c->mapend - c->mapstart) >= thp_pagesize
> > +      && p_align_max < thp_pagesize && (c->prot & PROT_WRITE) == 0)
> > +    return thp_pagesize;
> > +
> > +  return p_align_max;
> > +}
> > diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.h b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> > new file mode 100644
> > index 00000000000..83794a03986
> > --- /dev/null
> > +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
> > @@ -0,0 +1,27 @@
> > +/* _dl_map_segment_align.  Linux version.
> > +   Copyright (C) 2026 Free Software Foundation, Inc.
> > +   Copyright The GNU Toolchain Authors.
> > +   This file is part of the GNU C Library.
> > +
> > +   The GNU C Library is free software; you can redistribute it and/or
> > +   modify it under the terms of the GNU Lesser General Public
> > +   License as published by the Free Software Foundation; either
> > +   version 2.1 of the License, or (at your option) any later version.
> > +
> > +   The GNU C Library is distributed in the hope that it will be useful,
> > +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +   Lesser General Public License for more details.
> > +
> > +   You should have received a copy of the GNU Lesser General Public
> > +   License along with the GNU C Library; if not, see
> > +   <https://www.gnu.org/licenses/>.  */
> > +
> > +#include <dl-load.h>
> > +
> > +#ifndef DL_MAP_DEFAULT_THP_PAGESIZE
> > +# define DL_MAP_DEFAULT_THP_PAGESIZE   0
> > +#endif
> > +
> > +ElfW (Addr)
> > +    _dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max);
> > --
> > 2.53.0
> >
>
>
> --
> H.J.
WANG Rui April 10, 2026, 2:51 a.m. UTC | #9 
On Fri, Apr 10, 2026 at 8:58 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > +ElfW (Addr)
> > > +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> > > +{
> > > +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> > > +  static unsigned long int thp_pagesize;
> > > +
> > > +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> > > +    return p_align_max;
> > > +
> > > +  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
> > > +                       || thp_pagesize == 0))
> > > +    {
> > > +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> > > +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
> >
> > Isn't THP mode controlled by kernel?  I think we should also call
>
> We should always do
>
> thp_mode = __get_thp_mode ();
>
> > __madvise with MADV_HUGEPAGE to enable THP.
>
> We can call  __madvise after _dl_postprocess_loadcmd is called if
> thp_mode == thp_mode_madvise.

The goal of this patch is simply to be aware of THP being enabled and,
when possible, increase the alignment of LOAD segments so THP can work
more effectively. Overall it's a passive approach, the decision of
whether a mapping actually ends up using huge pages is entirely left
to THP.

From an impl point of view, it tries to avoid introducing any
additional overhead as much as possible, including extra syscalls and
the like.

Thanks,
Rui
H.J. Lu April 10, 2026, 3:22 a.m. UTC | #10 
On Fri, Apr 10, 2026 at 10:59 AM WANG Rui <wangrui@loongson.cn> wrote:
>
> On Fri, Apr 10, 2026 at 8:58 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > > +ElfW (Addr)
> > > > +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> > > > +{
> > > > +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> > > > +  static unsigned long int thp_pagesize;
> > > > +
> > > > +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> > > > +    return p_align_max;
> > > > +
> > > > +  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
> > > > +                       || thp_pagesize == 0))
> > > > +    {
> > > > +      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
> > > > +      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
> > >
> > > Isn't THP mode controlled by kernel?  I think we should also call
> >
> > We should always do
> >
> > thp_mode = __get_thp_mode ();
> >
> > > __madvise with MADV_HUGEPAGE to enable THP.
> >
> > We can call  __madvise after _dl_postprocess_loadcmd is called if
> > thp_mode == thp_mode_madvise.
>
> The goal of this patch is simply to be aware of THP being enabled and,
> when possible, increase the alignment of LOAD segments so THP can work
> more effectively. Overall it's a passive approach, the decision of
> whether a mapping actually ends up using huge pages is entirely left
> to THP.
>
> From an impl point of view, it tries to avoid introducing any
> additional overhead as much as possible, including extra syscalls and
> the like.
>

My kernel has

[hjl@gnu-tgl-3 linux]$ cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
[hjl@gnu-tgl-3 linux]$

Why can't ld.so enable THP?  A single __madvise call won't kill the THP
performance.
Wilco Dijkstra April 10, 2026, 10:18 a.m. UTC | #11 
Hi HJ,

>> The goal of this patch is simply to be aware of THP being enabled and,
>> when possible, increase the alignment of LOAD segments so THP can work
>> more effectively. Overall it's a passive approach, the decision of
>> whether a mapping actually ends up using huge pages is entirely left
>> to THP.
>>
>> From an impl point of view, it tries to avoid introducing any
>> additional overhead as much as possible, including extra syscalls and
>> the like.

Indeed. And not just because it is unnecessary overhead, but also because
there were bug reports where containers crashed on the expensive
__get_thp_mode() call. Hence in malloc we don't call it by default.

> Why can't ld.so enable THP?  A single __madvise call won't kill the THP
> performance.

It won't, but my question is whether it applies to file mappings in the same
way it works on anonymous mappings. Do you have evidence it does?

Rui's patch is conservative on purpose because it is a new feature. If after
some testing we conclude it works really well, we can improve it further.

Cheers,
Wilco
H.J. Lu April 11, 2026, 12:29 a.m. UTC | #12 
On Fri, Apr 10, 2026 at 6:19 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
> Hi HJ,
>
> >> The goal of this patch is simply to be aware of THP being enabled and,
> >> when possible, increase the alignment of LOAD segments so THP can work
> >> more effectively. Overall it's a passive approach, the decision of
> >> whether a mapping actually ends up using huge pages is entirely left
> >> to THP.
> >>
> >> From an impl point of view, it tries to avoid introducing any
> >> additional overhead as much as possible, including extra syscalls and
> >> the like.
>
> Indeed. And not just because it is unnecessary overhead, but also because
> there were bug reports where containers crashed on the expensive
> __get_thp_mode() call. Hence in malloc we don't call it by default.
>
> > Why can't ld.so enable THP?  A single __madvise call won't kill the THP
> > performance.
>
> It won't, but my question is whether it applies to file mappings in the same
> way it works on anonymous mappings. Do you have evidence it does?
>
> Rui's patch is conservative on purpose because it is a new feature. If after
> some testing we conclude it works really well, we can improve it further.
>

A couple issues:

1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
add a system call or AT_XXX values to get them.
2.  We should support THP enabled with madvise.  If the current kernel doesn't
support it, it should be updated.
3. Glibc can map PIE and DSO to address aligned to THP page size after

commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
Date:   Fri Dec 10 20:39:10 2021 +0800

    elf: Properly align PT_LOAD segments [BZ #28676]

If THP is enabled with madvise, should kernel map PIE to address aligned to
THP page size just like THP enabled with always?  If not,  should we add a
marker in PIE to let kernel know that a PIE should be mapped to address
aligned to THP page size?
Wilco Dijkstra April 11, 2026, 12:13 p.m. UTC | #13 
Hi HJ,

> A couple issues:
>
> 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> add a system call or AT_XXX values to get them.

Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
a good idea - it's cheap for the loader to process a few extra entries.

However until we have a way to easily get the THP settings at startup, we need an
alternative without reading those files. We also need to override the PMD size if
it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
to 2MB for all base page sizes.

> 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> support it, it should be updated.

In the future, sure, but it is not a requirement for Rui's patch series.

> 3. Glibc can map PIE and DSO to address aligned to THP page size after
>
> commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> Date:   Fri Dec 10 20:39:10 2021 +0800
>
>    elf: Properly align PT_LOAD segments [BZ #28676]
>
> If THP is enabled with madvise, should kernel map PIE to address aligned to
> THP page size just like THP enabled with always?  If not,  should we add a
> marker in PIE to let kernel know that a PIE should be mapped to address
> aligned to THP page size?

The kernel should do the alignment when it believes there is an expected benefit -
the exact cases for which that is true are likely to change over time depending on how
it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
give GLIBC what it believes is the best alignment for THP (potentially the alignment for
file and anonymous mappings might be different).

Cheers,
Wilco
H.J. Lu April 11, 2026, 11:05 p.m. UTC | #14 
On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
> Hi HJ,
>
> > A couple issues:
> >
> > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > add a system call or AT_XXX values to get them.
>
> Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> a good idea - it's cheap for the loader to process a few extra entries.
>
> However until we have a way to easily get the THP settings at startup, we need an
> alternative without reading those files. We also need to override the PMD size if
> it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> to 2MB for all base page sizes.

There is an AT_PAGE_SHIFT_MASK proposal for x86:

https://lore.kernel.org/lkml/ecb049aa-bcac-45c7-bbb1-4612d094935a@p183/

which covers

/sys/kernel/mm/hugepages

It can be used by target to decide the optimal huge page size.  But we
need to cover
all targets as well as

/sys/kernel/mm/transparent_hugepage/enabled

> > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > support it, it should be updated.
>
> In the future, sure, but it is not a requirement for Rui's patch series.

Since THP has significant drawbacks for certain workloads, some applications,
like databases, recommend not setting THP to always.  We should support madvise
THP to get it right from the start.

> > 3. Glibc can map PIE and DSO to address aligned to THP page size after
> >
> > commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> > Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> > Date:   Fri Dec 10 20:39:10 2021 +0800
> >
> >    elf: Properly align PT_LOAD segments [BZ #28676]
> >
> > If THP is enabled with madvise, should kernel map PIE to address aligned to
> > THP page size just like THP enabled with always?  If not,  should we add a
> > marker in PIE to let kernel know that a PIE should be mapped to address
> > aligned to THP page size?
>
> The kernel should do the alignment when it believes there is an expected benefit -
> the exact cases for which that is true are likely to change over time depending on how
> it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
> give GLIBC what it believes is the best alignment for THP (potentially the alignment for
> file and anonymous mappings might be different).
>

But kernel may not know if THP is good for performance for every
application.  Al
mode of "google linux transparent huge pages" shows

Common Issues & Trade-offs

Latency Spikes: The background process (khugepaged) that consolidates pages
can cause temporary system freezes or high CPU usage.
Memory Waste: Using a 2MB page for only 4KB of data can lead to internal
fragmentation and higher memory consumption.
Database Incompatibility: Many databases (e.g., Oracle, MongoDB, SAP ASE)
recommend disabling THP because their non-contiguous access patterns often
trigger performance degradations.

A madvise THP kernel aligning binaries to the optimal huge page size only for
binaries with a THP marker can mitigate the THP performance issues.


--
H.J.
H.J. Lu April 12, 2026, 3:42 a.m. UTC | #15 
On Sun, Apr 5, 2026 at 11:56 AM WANG Rui <wangrui@loongson.cn> wrote:
>
> Mapping segments that are at least the size of a PMD huge page to
> huge-page-aligned addresses helps make them eligible for Transparent
> Huge Pages (THP).
>
> This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
> to determine an appropriate maximum alignment for ELF load segments based
> on the system THP policy. The optimization is enabled only when the glibc
> tunable `glibc.elf.thp=1` is set and THP is configured to be used
> unconditionally.
>
> The optimization depends on Linux kernel support for file-backed THP,
> specifically:
>
> * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
>
> When enabled, the helper queries the default THP page size and uses it
> to align sufficiently large load segments that are already properly
> aligned in both virtual address and file offset (e.g., zero).
>
> For eligible segments, the alignment is bumped to the THP page size,
> which improves THP eligibility, reduces TLB pressure, and improves
> performance for large objects. To avoid excessive address space padding
> on systems with very large THP sizes, the alignment is capped at 32MB.
> The optimization is applied only to non-writable segments, matching
> typical THP usage.
>
> Signed-off-by: WANG Rui <wangrui@loongson.cn>
> ---
>  sysdeps/unix/sysv/linux/Makefile              |  1 +
>  .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
>  .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
>  3 files changed, 81 insertions(+)
>  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
>  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h
>
> diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
> index c6bd97abf1a..9879fec9a93 100644
> --- a/sysdeps/unix/sysv/linux/Makefile
> +++ b/sysdeps/unix/sysv/linux/Makefile
> @@ -673,6 +673,7 @@ endif
>
>  ifeq ($(subdir),elf)
>  dl-routines += \
> +  dl-map-segment-align \
>    dl-rseq-symbols \
>    # dl-routines
>
> diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> new file mode 100644
> index 00000000000..75fb0496730
> --- /dev/null
> +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> @@ -0,0 +1,53 @@
> +/* _dl_map_segment_align.  Linux version.
> +   Copyright (C) 2026 Free Software Foundation, Inc.
> +   Copyright The GNU Toolchain Authors.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <dl-map-segment-align.h>
> +#include <dl-tunables.h>
> +#include <hugepages.h>
> +
> +ElfW (Addr)
> +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> +{
> +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> +  static unsigned long int thp_pagesize;
> +
> +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> +    return p_align_max;
> +

We should add GLRO(dl_thp_control) and set it in _dl_sysdep_start
to avoid calling TUNABLE_GET for every PT_LOAD segment.

H.J.
WANG Rui April 12, 2026, 10:06 a.m. UTC | #16 
On Sun, Apr 12, 2026 at 7:05 AM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
> >
> > Hi HJ,
> >
> > > A couple issues:
> > >
> > > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > > add a system call or AT_XXX values to get them.
> >
> > Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> > a good idea - it's cheap for the loader to process a few extra entries.
> >
> > However until we have a way to easily get the THP settings at startup, we need an
> > alternative without reading those files. We also need to override the PMD size if
> > it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> > to 2MB for all base page sizes.
>
> There is an AT_PAGE_SHIFT_MASK proposal for x86:
>
> https://lore.kernel.org/lkml/ecb049aa-bcac-45c7-bbb1-4612d094935a@p183/
>
> which covers
>
> /sys/kernel/mm/hugepages
>
> It can be used by target to decide the optimal huge page size.  But we
> need to cover
> all targets as well as
>
> /sys/kernel/mm/transparent_hugepage/enabled
>
> > > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > > support it, it should be updated.
> >
> > In the future, sure, but it is not a requirement for Rui's patch series.
>
> Since THP has significant drawbacks for certain workloads, some applications,
> like databases, recommend not setting THP to always.  We should support madvise
> THP to get it right from the start.
>
> > > 3. Glibc can map PIE and DSO to address aligned to THP page size after
> > >
> > > commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> > > Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> > > Date:   Fri Dec 10 20:39:10 2021 +0800
> > >
> > >    elf: Properly align PT_LOAD segments [BZ #28676]
> > >
> > > If THP is enabled with madvise, should kernel map PIE to address aligned to
> > > THP page size just like THP enabled with always?  If not,  should we add a
> > > marker in PIE to let kernel know that a PIE should be mapped to address
> > > aligned to THP page size?
> >
> > The kernel should do the alignment when it believes there is an expected benefit -
> > the exact cases for which that is true are likely to change over time depending on how
> > it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
> > give GLIBC what it believes is the best alignment for THP (potentially the alignment for
> > file and anonymous mappings might be different).
> >
>
> But kernel may not know if THP is good for performance for every
> application.  Al

From my perspective, it is actually harder for applications or
programmers to make a well-informed decision about adding such a
marker than it is for the kernel.
The kernel has a more comprehensive view of the system's runtime
state, memory pressure, workload patterns, and overall system
behavior, while an individual application typically lacks that global
context.

> mode of "google linux transparent huge pages" shows
>
> Common Issues & Trade-offs
>
> Latency Spikes: The background process (khugepaged) that consolidates pages
> can cause temporary system freezes or high CPU usage.

Regarding the concerns cited from the general discussions around THP
trade-offs, I think we should be careful not to apply those
conclusions too broadly without considering the specific details of
this patch.

> Memory Waste: Using a 2MB page for only 4KB of data can lead to internal
> fragmentation and higher memory consumption.

In the current impl, there is already a filtering condition that
requires the text segment length to exceed the huge page size.
So the scenario of using a 2MB page to map only 4KB of data does not apply here.

> Database Incompatibility: Many databases (e.g., Oracle, MongoDB, SAP ASE)
> recommend disabling THP because their non-contiguous access patterns often
> trigger performance degradations.

Similarly, the database examples commonly refer to memory mappings of
database **data**, which often have non-contiguous access patterns.
This patch, however, targets executable **code** segments, whose
access characteristics are fundamentally different.

Therefore, rather than relying on high-level summaries of THP
behavior, it would be more productive to evaluate the impact based on
the concrete behavior of code mappings in this specific context.

>
> A madvise THP kernel aligning binaries to the optimal huge page size only for
> binaries with a THP marker can mitigate the THP performance issues.

Thanks,
Rui
WANG Rui April 12, 2026, 10:21 a.m. UTC | #17 
On Sun, Apr 12, 2026 at 11:51 AM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Sun, Apr 5, 2026 at 11:56 AM WANG Rui <wangrui@loongson.cn> wrote:
> >
> > Mapping segments that are at least the size of a PMD huge page to
> > huge-page-aligned addresses helps make them eligible for Transparent
> > Huge Pages (THP).
> >
> > This patch introduces a Linux-specific helper, `_dl_map_segment_align`,
> > to determine an appropriate maximum alignment for ELF load segments based
> > on the system THP policy. The optimization is enabled only when the glibc
> > tunable `glibc.elf.thp=1` is set and THP is configured to be used
> > unconditionally.
> >
> > The optimization depends on Linux kernel support for file-backed THP,
> > specifically:
> >
> > * `CONFIG_READ_ONLY_THP_FOR_FS` (available since Linux kernel 5.4), and
> > * `CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS`.
> >
> > When enabled, the helper queries the default THP page size and uses it
> > to align sufficiently large load segments that are already properly
> > aligned in both virtual address and file offset (e.g., zero).
> >
> > For eligible segments, the alignment is bumped to the THP page size,
> > which improves THP eligibility, reduces TLB pressure, and improves
> > performance for large objects. To avoid excessive address space padding
> > on systems with very large THP sizes, the alignment is capped at 32MB.
> > The optimization is applied only to non-writable segments, matching
> > typical THP usage.
> >
> > Signed-off-by: WANG Rui <wangrui@loongson.cn>
> > ---
> >  sysdeps/unix/sysv/linux/Makefile              |  1 +
> >  .../unix/sysv/linux/dl-map-segment-align.c    | 53 +++++++++++++++++++
> >  .../unix/sysv/linux/dl-map-segment-align.h    | 27 ++++++++++
> >  3 files changed, 81 insertions(+)
> >  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.c
> >  create mode 100644 sysdeps/unix/sysv/linux/dl-map-segment-align.h
> >
> > diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
> > index c6bd97abf1a..9879fec9a93 100644
> > --- a/sysdeps/unix/sysv/linux/Makefile
> > +++ b/sysdeps/unix/sysv/linux/Makefile
> > @@ -673,6 +673,7 @@ endif
> >
> >  ifeq ($(subdir),elf)
> >  dl-routines += \
> > +  dl-map-segment-align \
> >    dl-rseq-symbols \
> >    # dl-routines
> >
> > diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> > new file mode 100644
> > index 00000000000..75fb0496730
> > --- /dev/null
> > +++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
> > @@ -0,0 +1,53 @@
> > +/* _dl_map_segment_align.  Linux version.
> > +   Copyright (C) 2026 Free Software Foundation, Inc.
> > +   Copyright The GNU Toolchain Authors.
> > +   This file is part of the GNU C Library.
> > +
> > +   The GNU C Library is free software; you can redistribute it and/or
> > +   modify it under the terms of the GNU Lesser General Public
> > +   License as published by the Free Software Foundation; either
> > +   version 2.1 of the License, or (at your option) any later version.
> > +
> > +   The GNU C Library is distributed in the hope that it will be useful,
> > +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +   Lesser General Public License for more details.
> > +
> > +   You should have received a copy of the GNU Lesser General Public
> > +   License along with the GNU C Library; if not, see
> > +   <https://www.gnu.org/licenses/>.  */
> > +
> > +#include <dl-map-segment-align.h>
> > +#include <dl-tunables.h>
> > +#include <hugepages.h>
> > +
> > +ElfW (Addr)
> > +_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
> > +{
> > +  static enum thp_mode_t thp_mode = thp_mode_not_supported;
> > +  static unsigned long int thp_pagesize;
> > +
> > +  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
> > +    return p_align_max;
> > +
>
> We should add GLRO(dl_thp_control) and set it in _dl_sysdep_start
> to avoid calling TUNABLE_GET for every PT_LOAD segment.

Since tunables are already almost for free once initialized,
essentially just a global variable access.
I'm wondering if adding a separate GLRO(dl_thp_control) is really
necessary in this patch.

Thanks,
Rui
WANG Rui April 12, 2026, 10:51 a.m. UTC | #18 
On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
> Hi HJ,
>
> > A couple issues:
> >
> > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > add a system call or AT_XXX values to get them.
>
> Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> a good idea - it's cheap for the loader to process a few extra entries.
>
> However until we have a way to easily get the THP settings at startup, we need an
> alternative without reading those files. We also need to override the PMD size if
> it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> to 2MB for all base page sizes.
>
> > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > support it, it should be updated.
>
> In the future, sure, but it is not a requirement for Rui's patch series.

I agree. This patch is intentionally conservative and only improves
segment alignment while passively working with THP in "always" mode.

We can move forward incrementally and evaluate the behavior in
practice before considering a more proactive use of madvise.

Thanks,
Rui

>
> > 3. Glibc can map PIE and DSO to address aligned to THP page size after
> >
> > commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> > Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> > Date:   Fri Dec 10 20:39:10 2021 +0800
> >
> >    elf: Properly align PT_LOAD segments [BZ #28676]
> >
> > If THP is enabled with madvise, should kernel map PIE to address aligned to
> > THP page size just like THP enabled with always?  If not,  should we add a
> > marker in PIE to let kernel know that a PIE should be mapped to address
> > aligned to THP page size?
>
> The kernel should do the alignment when it believes there is an expected benefit -
> the exact cases for which that is true are likely to change over time depending on how
> it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
> give GLIBC what it believes is the best alignment for THP (potentially the alignment for
> file and anonymous mappings might be different).
>
> Cheers,
> Wilco
H.J. Lu April 12, 2026, 11:26 a.m. UTC | #19 
On Sun, Apr 12, 2026 at 6:14 PM WANG Rui <wangrui@loongson.cn> wrote:
>
> On Sun, Apr 12, 2026 at 7:05 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> >
> > On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
> > >
> > > Hi HJ,
> > >
> > > > A couple issues:
> > > >
> > > > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > > > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > > > add a system call or AT_XXX values to get them.
> > >
> > > Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> > > a good idea - it's cheap for the loader to process a few extra entries.
> > >
> > > However until we have a way to easily get the THP settings at startup, we need an
> > > alternative without reading those files. We also need to override the PMD size if
> > > it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> > > to 2MB for all base page sizes.
> >
> > There is an AT_PAGE_SHIFT_MASK proposal for x86:
> >
> > https://lore.kernel.org/lkml/ecb049aa-bcac-45c7-bbb1-4612d094935a@p183/
> >
> > which covers
> >
> > /sys/kernel/mm/hugepages
> >
> > It can be used by target to decide the optimal huge page size.  But we
> > need to cover
> > all targets as well as
> >
> > /sys/kernel/mm/transparent_hugepage/enabled
> >
> > > > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > > > support it, it should be updated.
> > >
> > > In the future, sure, but it is not a requirement for Rui's patch series.
> >
> > Since THP has significant drawbacks for certain workloads, some applications,
> > like databases, recommend not setting THP to always.  We should support madvise
> > THP to get it right from the start.
> >
> > > > 3. Glibc can map PIE and DSO to address aligned to THP page size after
> > > >
> > > > commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> > > > Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> > > > Date:   Fri Dec 10 20:39:10 2021 +0800
> > > >
> > > >    elf: Properly align PT_LOAD segments [BZ #28676]
> > > >
> > > > If THP is enabled with madvise, should kernel map PIE to address aligned to
> > > > THP page size just like THP enabled with always?  If not,  should we add a
> > > > marker in PIE to let kernel know that a PIE should be mapped to address
> > > > aligned to THP page size?
> > >
> > > The kernel should do the alignment when it believes there is an expected benefit -
> > > the exact cases for which that is true are likely to change over time depending on how
> > > it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
> > > give GLIBC what it believes is the best alignment for THP (potentially the alignment for
> > > file and anonymous mappings might be different).
> > >
> >
> > But kernel may not know if THP is good for performance for every
> > application.  Al
>
> From my perspective, it is actually harder for applications or
> programmers to make a well-informed decision about adding such a
> marker than it is for the kernel.
> The kernel has a more comprehensive view of the system's runtime
> state, memory pressure, workload patterns, and overall system
> behavior, while an individual application typically lacks that global
> context.
>
> > mode of "google linux transparent huge pages" shows
> >
> > Common Issues & Trade-offs
> >
> > Latency Spikes: The background process (khugepaged) that consolidates pages
> > can cause temporary system freezes or high CPU usage.
>
> Regarding the concerns cited from the general discussions around THP
> trade-offs, I think we should be careful not to apply those
> conclusions too broadly without considering the specific details of
> this patch.
>
> > Memory Waste: Using a 2MB page for only 4KB of data can lead to internal
> > fragmentation and higher memory consumption.
>
> In the current impl, there is already a filtering condition that
> requires the text segment length to exceed the huge page size.
> So the scenario of using a 2MB page to map only 4KB of data does not apply here.
>
> > Database Incompatibility: Many databases (e.g., Oracle, MongoDB, SAP ASE)
> > recommend disabling THP because their non-contiguous access patterns often
> > trigger performance degradations.
>
> Similarly, the database examples commonly refer to memory mappings of
> database **data**, which often have non-contiguous access patterns.
> This patch, however, targets executable **code** segments, whose
> access characteristics are fundamentally different.
>
> Therefore, rather than relying on high-level summaries of THP
> behavior, it would be more productive to evaluate the impact based on
> the concrete behavior of code mappings in this specific context.

The issue is that the current implementation requires and assumes
THP enabled with always which impacts ALL memory mapping,
including data.

>
> >
> > A madvise THP kernel aligning binaries to the optimal huge page size only for
> > binaries with a THP marker can mitigate the THP performance issues.
>
> Thanks,
> Rui
>
H.J. Lu April 12, 2026, 11:49 a.m. UTC | #20 
On Sun, Apr 12, 2026 at 6:51 PM WANG Rui <wangrui@loongson.cn> wrote:
>
> On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
> >
> > Hi HJ,
> >
> > > A couple issues:
> > >
> > > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > > add a system call or AT_XXX values to get them.
> >
> > Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> > a good idea - it's cheap for the loader to process a few extra entries.
> >
> > However until we have a way to easily get the THP settings at startup, we need an
> > alternative without reading those files. We also need to override the PMD size if
> > it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> > to 2MB for all base page sizes.
> >
> > > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > > support it, it should be updated.
> >
> > In the future, sure, but it is not a requirement for Rui's patch series.
>
> I agree. This patch is intentionally conservative and only improves
> segment alignment while passively working with THP in "always" mode.
>
> We can move forward incrementally and evaluate the behavior in
> practice before considering a more proactive use of madvise.

1. Since the THP setting can be changed at run-time in the user space,
it is wrong for glibc to assume that THP is enabled with always.
2. There are potential performance issues with THP enabled with always.

I propose the following:

1.  Always check the THP setting at load-time.
2.  Check the tunable once at load-time.
3.  If THP isn't disabled by the tunable,
      3a.  Load shared libraries at THP compatible addresses.
      3b.  If THP is enabled with madvise, issue madvise on all THP
             compatible executable load segments, including main
             executable and shared libraries.
4.  Select an application and build it as PDE with THP compatible executable
load segments.  Use PDE, instead of PIE, ensures that executable load
segments have addresses suitable for THP under the existing kernel.
5. Compare its performance under never THP kernel, always kernel and
madvise THP kernel.

I can provide such glibc changes on top of yours.

madvise THP kernel should have similar iTLB performance as always THP
kernel.

If the results are positive, we can make kernel changes to provide faster
THP setting access and load PIE at THP compatible addresses.

> Thanks,
> Rui
>
> >
> > > 3. Glibc can map PIE and DSO to address aligned to THP page size after
> > >
> > > commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> > > Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> > > Date:   Fri Dec 10 20:39:10 2021 +0800
> > >
> > >    elf: Properly align PT_LOAD segments [BZ #28676]
> > >
> > > If THP is enabled with madvise, should kernel map PIE to address aligned to
> > > THP page size just like THP enabled with always?  If not,  should we add a
> > > marker in PIE to let kernel know that a PIE should be mapped to address
> > > aligned to THP page size?
> >
> > The kernel should do the alignment when it believes there is an expected benefit -
> > the exact cases for which that is true are likely to change over time depending on how
> > it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
> > give GLIBC what it believes is the best alignment for THP (potentially the alignment for
> > file and anonymous mappings might be different).
> >
> > Cheers,
> > Wilco
>
WANG Rui April 12, 2026, 12:50 p.m. UTC | #21 
On Sun, Apr 12, 2026 at 7:26 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Sun, Apr 12, 2026 at 6:14 PM WANG Rui <wangrui@loongson.cn> wrote:
> >
> > On Sun, Apr 12, 2026 at 7:05 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> > >
> > > On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
> > > >
> > > > Hi HJ,
> > > >
> > > > > A couple issues:
> > > > >
> > > > > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > > > > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > > > > add a system call or AT_XXX values to get them.
> > > >
> > > > Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> > > > a good idea - it's cheap for the loader to process a few extra entries.
> > > >
> > > > However until we have a way to easily get the THP settings at startup, we need an
> > > > alternative without reading those files. We also need to override the PMD size if
> > > > it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> > > > to 2MB for all base page sizes.
> > >
> > > There is an AT_PAGE_SHIFT_MASK proposal for x86:
> > >
> > > https://lore.kernel.org/lkml/ecb049aa-bcac-45c7-bbb1-4612d094935a@p183/
> > >
> > > which covers
> > >
> > > /sys/kernel/mm/hugepages
> > >
> > > It can be used by target to decide the optimal huge page size.  But we
> > > need to cover
> > > all targets as well as
> > >
> > > /sys/kernel/mm/transparent_hugepage/enabled
> > >
> > > > > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > > > > support it, it should be updated.
> > > >
> > > > In the future, sure, but it is not a requirement for Rui's patch series.
> > >
> > > Since THP has significant drawbacks for certain workloads, some applications,
> > > like databases, recommend not setting THP to always.  We should support madvise
> > > THP to get it right from the start.
> > >
> > > > > 3. Glibc can map PIE and DSO to address aligned to THP page size after
> > > > >
> > > > > commit 718fdd87b1b98ef88e883a37d9c18867256fa5a4
> > > > > Author: Rongwei Wang <rongwei.wang@linux.alibaba.com>
> > > > > Date:   Fri Dec 10 20:39:10 2021 +0800
> > > > >
> > > > >    elf: Properly align PT_LOAD segments [BZ #28676]
> > > > >
> > > > > If THP is enabled with madvise, should kernel map PIE to address aligned to
> > > > > THP page size just like THP enabled with always?  If not,  should we add a
> > > > > marker in PIE to let kernel know that a PIE should be mapped to address
> > > > > aligned to THP page size?
> > > >
> > > > The kernel should do the alignment when it believes there is an expected benefit -
> > > > the exact cases for which that is true are likely to change over time depending on how
> > > > it is implemented in the kernel. If we use AT_XXX as the mechanism, it could actually
> > > > give GLIBC what it believes is the best alignment for THP (potentially the alignment for
> > > > file and anonymous mappings might be different).
> > > >
> > >
> > > But kernel may not know if THP is good for performance for every
> > > application.  Al
> >
> > From my perspective, it is actually harder for applications or
> > programmers to make a well-informed decision about adding such a
> > marker than it is for the kernel.
> > The kernel has a more comprehensive view of the system's runtime
> > state, memory pressure, workload patterns, and overall system
> > behavior, while an individual application typically lacks that global
> > context.
> >
> > > mode of "google linux transparent huge pages" shows
> > >
> > > Common Issues & Trade-offs
> > >
> > > Latency Spikes: The background process (khugepaged) that consolidates pages
> > > can cause temporary system freezes or high CPU usage.
> >
> > Regarding the concerns cited from the general discussions around THP
> > trade-offs, I think we should be careful not to apply those
> > conclusions too broadly without considering the specific details of
> > this patch.
> >
> > > Memory Waste: Using a 2MB page for only 4KB of data can lead to internal
> > > fragmentation and higher memory consumption.
> >
> > In the current impl, there is already a filtering condition that
> > requires the text segment length to exceed the huge page size.
> > So the scenario of using a 2MB page to map only 4KB of data does not apply here.
> >
> > > Database Incompatibility: Many databases (e.g., Oracle, MongoDB, SAP ASE)
> > > recommend disabling THP because their non-contiguous access patterns often
> > > trigger performance degradations.
> >
> > Similarly, the database examples commonly refer to memory mappings of
> > database **data**, which often have non-contiguous access patterns.
> > This patch, however, targets executable **code** segments, whose
> > access characteristics are fundamentally different.
> >
> > Therefore, rather than relying on high-level summaries of THP
> > behavior, it would be more productive to evaluate the impact based on
> > the concrete behavior of code mappings in this specific context.
>
> The issue is that the current implementation requires and assumes
> THP enabled with always which impacts ALL memory mapping,
> including data.
>

The current implementation is designed only for the scenario where
users have already chosen to run with THP in "always" mode.

Thanks,
Rui
WANG Rui April 12, 2026, 12:50 p.m. UTC | #22 
On Sun, Apr 12, 2026 at 7:49 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Sun, Apr 12, 2026 at 6:51 PM WANG Rui <wangrui@loongson.cn> wrote:
> >
> > On Sat, Apr 11, 2026 at 8:15 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
> > >
> > > Hi HJ,
> > >
> > > > A couple issues:
> > > >
> > > > 1.  We shouldn't assume what are in /sys/kernel/mm/transparent_hugepage/enabled
> > > > nor /sys/kernel/mm/transparent_hugepage/hpage_pmd_size.   We can
> > > > add a system call or AT_XXX values to get them.
> > >
> > > Indeed, reading those files turned out to be a bad idea. Adding AT_XXX sounds like
> > > a good idea - it's cheap for the loader to process a few extra entries.
> > >
> > > However until we have a way to easily get the THP settings at startup, we need an
> > > alternative without reading those files. We also need to override the PMD size if
> > > it is too large - the current patch limits it to 32MB and for AArch64 I force the THP size
> > > to 2MB for all base page sizes.
> > >
> > > > 2.  We should support THP enabled with madvise.  If the current kernel doesn't
> > > > support it, it should be updated.
> > >
> > > In the future, sure, but it is not a requirement for Rui's patch series.
> >
> > I agree. This patch is intentionally conservative and only improves
> > segment alignment while passively working with THP in "always" mode.
> >
> > We can move forward incrementally and evaluate the behavior in
> > practice before considering a more proactive use of madvise.
>
> 1. Since the THP setting can be changed at run-time in the user space,
> it is wrong for glibc to assume that THP is enabled with always.

If we take into account that the THP setting can change at run time,
then strictly speaking we can't reliably depend on the THP mode at
load time anyway.

In practice, the check here is more of a pragmatic safeguard, mainly
to avoid unnecessarily reducing ASLR entropy in cases where THP is
clearly not in use.

> 2. There are potential performance issues with THP enabled with always.
>
> I propose the following:
>
> 1.  Always check the THP setting at load-time.
> 2.  Check the tunable once at load-time.
> 3.  If THP isn't disabled by the tunable,
>       3a.  Load shared libraries at THP compatible addresses.
>       3b.  If THP is enabled with madvise, issue madvise on all THP
>              compatible executable load segments, including main
>              executable and shared libraries.
> 4.  Select an application and build it as PDE with THP compatible executable
> load segments.  Use PDE, instead of PIE, ensures that executable load
> segments have addresses suitable for THP under the existing kernel.
> 5. Compare its performance under never THP kernel, always kernel and
> madvise THP kernel.
>
> I can provide such glibc changes on top of yours.
>
> madvise THP kernel should have similar iTLB performance as always THP
> kernel.
>
> If the results are positive, we can make kernel changes to provide faster
> THP setting access and load PIE at THP compatible addresses.

That sounds like a useful direction to explore, and I'd definitely be
interested in collaborating on that.

For now, I suggest landing the current patch based on the THP "always"
case first, since it is relatively conservative and has a well-defined
scope.

Follow-up patches could then extend the behavior to cover the madvise
case, based on the results of the experiments you described.

Does that sound like a reasonable way to proceed?

Thanks,
Rui
Wilco Dijkstra April 12, 2026, 2:01 p.m. UTC | #23 
Hi HJ/Rui,

>> The issue is that the current implementation requires and assumes
>> THP enabled with always which impacts ALL memory mapping,
>> including data.
>>
>
> The current implementation is designed only for the scenario where
> users have already chosen to run with THP in "always" mode.

Indeed, when things are faster in some cases but not in every case, it's
not an "issue". It's QoI.

I expect various targets want to benchmark this feature and enable it
by default if it works out well. Once we reach that point, it is worth
improving it further.

Cheers,
Wilco
H.J. Lu April 12, 2026, 2:18 p.m. UTC | #24 
On Sun, Apr 12, 2026 at 10:02 PM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
> Hi HJ/Rui,
>
> >> The issue is that the current implementation requires and assumes
> >> THP enabled with always which impacts ALL memory mapping,
> >> including data.
> >>
> >
> > The current implementation is designed only for the scenario where
> > users have already chosen to run with THP in "always" mode.

Glibc can't make such an assumption which it has no control over.
It needs to check THP status at run-time.  I will have an incremental
patch to do that in a few days.

> Indeed, when things are faster in some cases but not in every case, it's
> not an "issue". It's QoI.
>
> I expect various targets want to benchmark this feature and enable it
> by default if it works out well. Once we reach that point, it is worth
> improving it further.
>
> Cheers,
> Wilco
diff mbox series

Patch

diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
index c6bd97abf1a..9879fec9a93 100644
--- a/sysdeps/unix/sysv/linux/Makefile
+++ b/sysdeps/unix/sysv/linux/Makefile
@@ -673,6 +673,7 @@  endif
 
 ifeq ($(subdir),elf)
 dl-routines += \
+  dl-map-segment-align \
   dl-rseq-symbols \
   # dl-routines
 
diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.c b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
new file mode 100644
index 00000000000..75fb0496730
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.c
@@ -0,0 +1,53 @@ 
+/* _dl_map_segment_align.  Linux version.
+   Copyright (C) 2026 Free Software Foundation, Inc.
+   Copyright The GNU Toolchain Authors.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-map-segment-align.h>
+#include <dl-tunables.h>
+#include <hugepages.h>
+
+ElfW (Addr)
+_dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max)
+{
+  static enum thp_mode_t thp_mode = thp_mode_not_supported;
+  static unsigned long int thp_pagesize;
+
+  if (TUNABLE_GET (glibc, elf, thp, int32_t, NULL) == 0)
+    return p_align_max;
+
+  if (__glibc_unlikely (thp_mode == thp_mode_not_supported
+			|| thp_pagesize == 0))
+    {
+      unsigned long int default_thp_pagesize = DL_MAP_DEFAULT_THP_PAGESIZE;
+      thp_mode = default_thp_pagesize ? thp_mode_always : __get_thp_mode ();
+      thp_pagesize = default_thp_pagesize ? : __get_thp_size ();
+    }
+
+  /* Aligning load segments that are large enough to the PMD size helps
+     improve THP eligibility and reduces TLB pressure.
+     We cap the huge page size at MAX_THP_PAGESIZE to avoid over-aligning
+     on systems with very large normal pages (like 64K pages with 512M
+     huge pages). */
+  if (thp_mode == thp_mode_always && thp_pagesize <= MAX_THP_PAGESIZE
+      && ((c->mapstart | c->mapoff) & (thp_pagesize - 1)) == 0
+      && (c->mapend - c->mapstart) >= thp_pagesize
+      && p_align_max < thp_pagesize && (c->prot & PROT_WRITE) == 0)
+    return thp_pagesize;
+
+  return p_align_max;
+}
diff --git a/sysdeps/unix/sysv/linux/dl-map-segment-align.h b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
new file mode 100644
index 00000000000..83794a03986
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/dl-map-segment-align.h
@@ -0,0 +1,27 @@ 
+/* _dl_map_segment_align.  Linux version.
+   Copyright (C) 2026 Free Software Foundation, Inc.
+   Copyright The GNU Toolchain Authors.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-load.h>
+
+#ifndef DL_MAP_DEFAULT_THP_PAGESIZE
+# define DL_MAP_DEFAULT_THP_PAGESIZE	0
+#endif
+
+ElfW (Addr)
+    _dl_map_segment_align (const struct loadcmd *c, ElfW (Addr) p_align_max);