[0/2] Make tunable a default feature

  It is enabled as default since 2.26, some features option require it
(such as hwcap mask, huge pages support, and lock elisition tuning),
and assuming its support also simplifies the build permutation.

Adhemerval Zanella (2):
  elf: Remove glibc.rtld.dynamic_sort tunable
  Remove --enable-tunables configure option

 INSTALL                                       |  19 ---
 Makeconfig                                    |   4 -
 NEWS                                          |   3 +
 config.h.in                                   |   3 -
 config.make.in                                |   1 -
 configure                                     |  17 ---
 configure.ac                                  |  10 --
 elf/Makefile                                  |   8 +-
 elf/dl-cache.c                                |   2 +-
 elf/dl-hwcaps.h                               |  11 --
 elf/dl-sort-maps.c                            | 139 +-----------------
 elf/dl-support.c                              |  15 --
 elf/dl-tls.c                                  |   6 -
 elf/dl-tunables.h                             |  35 ++---
 elf/dl-tunables.list                          |   9 --
 elf/dl-usage.c                                |   8 +-
 elf/dso-sort-tests-1.def                      |  15 +-
 elf/dso-sort-tests-2.def                      |  10 +-
 elf/rtld.c                                    |  23 +--
 elf/tst-env-setuid-tunables.c                 |  11 --
 elf/tst-rtld-list-tunables.exp                |   1 -
 gmon/gmon.c                                   |  12 +-
 malloc/Makefile                               |   2 -
 malloc/arena.c                                | 118 +--------------
 malloc/malloc-check.c                         |   8 -
 malloc/malloc.c                               |  14 +-
 manual/install.texi                           |  19 ---
 manual/tunables.texi                          |  13 --
 nptl/pthread_mutex_conf.c                     |   4 +-
 nptl/tst-mutex8.c                             |   2 -
 scripts/build-many-glibcs.py                  |   1 -
 sysdeps/generic/ldsodefs.h                    |  17 ---
 sysdeps/generic/unsecvars.h                   |   8 +-
 sysdeps/mach/hurd/dl-sysdep.c                 |   3 -
 sysdeps/nptl/dl-tls_init_tp.c                 |   2 -
 sysdeps/nptl/pthreadP.h                       |   4 -
 sysdeps/nptl/pthread_early_init.h             |   2 -
 sysdeps/nptl/pthread_mutex_conf.h             |   8 -
 sysdeps/powerpc/cpu-features.c                |   9 +-
 sysdeps/s390/cpu-features.c                   |  14 +-
 sysdeps/sparc/sparc32/dl-machine.h            |   5 +-
 sysdeps/unix/sysv/linux/Makefile              |   4 +-
 .../unix/sysv/linux/aarch64/cpu-features.c    |   6 -
 sysdeps/unix/sysv/linux/dl-sysdep.c           |   3 -
 .../unix/sysv/linux/powerpc/elision-conf.c    |   8 +-
 sysdeps/unix/sysv/linux/s390/elision-conf.c   |   8 +-
 sysdeps/unix/sysv/linux/x86/elision-conf.c    |   8 +-
 sysdeps/unix/sysv/linux/x86_64/64/Makefile    |   2 -
 sysdeps/x86/Makefile                          |  10 --
 sysdeps/x86/cacheinfo.h                       |   8 +-
 sysdeps/x86/cpu-features.c                    |  24 +--
 sysdeps/x86/cpu-tunables.c                    |  45 +++---
 sysdeps/x86/dl-cacheinfo.h                    |  10 --
 sysdeps/x86_64/Makefile                       |   2 -
 54 files changed, 80 insertions(+), 673 deletions(-)
  

On 2023-03-13 15:06, Adhemerval Zanella wrote:
> It is enabled as default since 2.26, some features option require it
> (such as hwcap mask, huge pages support, and lock elisition tuning),
> and assuming its support also simplifies the build permutation.

from a historical perspective, there was not just the question of 
enabling tunables but also providing multiple ways to read tunables, 
each having its own parser.  That is why there's a 'valstring' as an 
option for --enable-tunables.

Over the years there haven't really been any other ideas to read 
tunables.  There's the idea of systemwide tunables through, e.g. 
ld.so.conf that we'd like to have at some point, but that is more 
effective alongside valstring than as an either-or feature.

There's also conceivably a use case for having only systemwide tunables 
without any valstring override but maybe that should get supported when 
it's actually needed by someone.  So in summary, I'm not aware of 
anybody using --disable-tunables, so I'm not opposed to removing the 
configure flag altogether and simplifying the code.  Hopefully if 
someone is using it they'll come forward before this patchset gets accepted.

Thanks,
Sid
  

On 14/03/23 15:02, Siddhesh Poyarekar wrote:
> On 2023-03-13 15:06, Adhemerval Zanella wrote:
>> It is enabled as default since 2.26, some features option require it
>> (such as hwcap mask, huge pages support, and lock elisition tuning),
>> and assuming its support also simplifies the build permutation.
> 
> from a historical perspective, there was not just the question of enabling tunables but also providing multiple ways to read tunables, each having its own parser.  That is why there's a 'valstring' as an option for --enable-tunables.

Does it really make sense to provide such option, where multiple distributions
would support different tunables? I don't this much as way forward, it would require
additional effort to document, along with some prudence to make it concise. And
its has the inherent problem of fragmentation.

> 
> Over the years there haven't really been any other ideas to read tunables.  There's the idea of systemwide tunables through, e.g. ld.so.conf that we'd like to have at some point, but that is more effective alongside valstring than as an either-or feature.

For ld.so cache, the program can use DF_1_NODEFLIB to either avoid its costs or
not use system cache (if its uses RUNPATH).  I think a system-wide tunable would
require a similar scheme, where the program can opt-out if required.

> 
> There's also conceivably a use case for having only systemwide tunables without any valstring override but maybe that should get supported when it's actually needed by someone.  So in summary, I'm not aware of anybody using --disable-tunables, so I'm not opposed to removing the configure flag altogether and simplifying the code.  Hopefully if someone is using it they'll come forward before this patchset gets accepted.

Alright, so I think it should be good to setup this for next release then.
  

On 2023-03-15 16:05, Adhemerval Zanella Netto wrote:
> On 14/03/23 15:02, Siddhesh Poyarekar wrote:
>> On 2023-03-13 15:06, Adhemerval Zanella wrote:
>>> It is enabled as default since 2.26, some features option require it
>>> (such as hwcap mask, huge pages support, and lock elisition tuning),
>>> and assuming its support also simplifies the build permutation.
>>
>> from a historical perspective, there was not just the question of enabling tunables but also providing multiple ways to read tunables, each having its own parser.  That is why there's a 'valstring' as an option for --enable-tunables.
> 
> Does it really make sense to provide such option, where multiple distributions
> would support different tunables? I don't this much as way forward, it would require
> additional effort to document, along with some prudence to make it concise. And
> its has the inherent problem of fragmentation.

Yeah I agree it doesn't make a lot of sense anymore; I think we made it 
like that to bake in some flexibility for use cases we weren't aware of 
yet.  I mentioned it here in the hope that folks who were part of the 
conversation then and have better memory than I would be able to pitch 
in with additional context :)

>> Over the years there haven't really been any other ideas to read tunables.  There's the idea of systemwide tunables through, e.g. ld.so.conf that we'd like to have at some point, but that is more effective alongside valstring than as an either-or feature.
> 
> For ld.so cache, the program can use DF_1_NODEFLIB to either avoid its costs or
> not use system cache (if its uses RUNPATH).  I think a system-wide tunable would
> require a similar scheme, where the program can opt-out if required.

Actually I was thinking of systemwide tunables as the canonical way to 
apply default rules that *won't* get overridden by users.  That could 
allow users to, e.g. lock down memory tagging on a systemwide basis if 
needed, or put a hard upper or lower limit on some of the malloc 
tunables.  The scheme would work similar to rlimit, but it shouldn't be 
misunderstood to be a security feature in that sense; applications can 
always work around a malloc tunable limit by rolling their own allocator.

>> There's also conceivably a use case for having only systemwide tunables without any valstring override but maybe that should get supported when it's actually needed by someone.  So in summary, I'm not aware of anybody using --disable-tunables, so I'm not opposed to removing the configure flag altogether and simplifying the code.  Hopefully if someone is using it they'll come forward before this patchset gets accepted.
> 
> Alright, so I think it should be good to setup this for next release then.

Yep.  I don't know how we can make this more widely visible before we 
drop the option.  Maybe @gnutools could tweet about it.  If there are no 
major objections in the next week or so I'll do a proper review of the 
patchset.

Thanks,
Sid
  

On 17/03/23 08:15, Siddhesh Poyarekar wrote:
> On 2023-03-15 16:05, Adhemerval Zanella Netto wrote:
>> On 14/03/23 15:02, Siddhesh Poyarekar wrote:
>>> On 2023-03-13 15:06, Adhemerval Zanella wrote:
>>>> It is enabled as default since 2.26, some features option require it
>>>> (such as hwcap mask, huge pages support, and lock elisition tuning),
>>>> and assuming its support also simplifies the build permutation.
>>>
>>> from a historical perspective, there was not just the question of enabling tunables but also providing multiple ways to read tunables, each having its own parser.  That is why there's a 'valstring' as an option for --enable-tunables.
>>
>> Does it really make sense to provide such option, where multiple distributions
>> would support different tunables? I don't this much as way forward, it would require
>> additional effort to document, along with some prudence to make it concise. And
>> its has the inherent problem of fragmentation.
> 
> Yeah I agree it doesn't make a lot of sense anymore; I think we made it like that to bake in some flexibility for use cases we weren't aware of yet.  I mentioned it here in the hope that folks who were part of the conversation then and have better memory than I would be able to pitch in with additional context :)
> 
>>> Over the years there haven't really been any other ideas to read tunables.  There's the idea of systemwide tunables through, e.g. ld.so.conf that we'd like to have at some point, but that is more effective alongside valstring than as an either-or feature.
>>
>> For ld.so cache, the program can use DF_1_NODEFLIB to either avoid its costs or
>> not use system cache (if its uses RUNPATH).  I think a system-wide tunable would
>> require a similar scheme, where the program can opt-out if required.
> 
> Actually I was thinking of systemwide tunables as the canonical way to apply default rules that *won't* get overridden by users.  That could allow users to, e.g. lock down memory tagging on a systemwide basis if needed, or put a hard upper or lower limit on some of the malloc tunables.  The scheme would work similar to rlimit, but it shouldn't be misunderstood to be a security feature in that sense; applications can always work around a malloc tunable limit by rolling their own allocator.

I think it might make sense for tunables that change the program semantic, such
as security hardening; although I don't think it really fits for performance
oriented ones (such malloc or pthread tuning).  So maybe we can define a global
file format where the administrator can set where setuid binaries can use it,
and if uses can overwrite it.  My initial idea would be something quite simple,
similar to sysctl.conf:

$ cat /etc/glibc-tunables.conf
# tunable                       setuid   override
glibc.cpu.hwcaps=-AVX512F,-AVX2      1          0
glibc.malloc.trim_threshold=128      1          1
glibc.malloc.tcache_count=2          0          1

And I was thinking about an DF_1_NODEFLIB analogous so the program can opt-out
any performance or behavior difference any tunable might incur.  Although with
your idea of enforceable tunable, I think it does not make much sense.

> 
>>> There's also conceivably a use case for having only systemwide tunables without any valstring override but maybe that should get supported when it's actually needed by someone.  So in summary, I'm not aware of anybody using --disable-tunables, so I'm not opposed to removing the configure flag altogether and simplifying the code.  Hopefully if someone is using it they'll come forward before this patchset gets accepted.
>>
>> Alright, so I think it should be good to setup this for next release then.
> 
> Yep.  I don't know how we can make this more widely visible before we drop the option.  Maybe @gnutools could tweet about it.  If there are no major objections in the next week or so I'll do a proper review of the patchset.
> 
> Thanks,
> Sid
  

On 2023-03-20 12:59, Adhemerval Zanella Netto wrote:
> I think it might make sense for tunables that change the program semantic, such
> as security hardening; although I don't think it really fits for performance
> oriented ones (such malloc or pthread tuning).  So maybe we can define a global

Administrative level performance tuning defaults for setuid binaries?

> file format where the administrator can set where setuid binaries can use it,
> and if uses can overwrite it.  My initial idea would be something quite simple,
> similar to sysctl.conf:

Yes, I think Florian suggested reusing ld.so.conf instead.  I don't have 
a strong preference either way so y'all can fight that one out - potato 
potato ;)

> And I was thinking about an DF_1_NODEFLIB analogous so the program can opt-out
> any performance or behavior difference any tunable might incur.  Although with
> your idea of enforceable tunable, I think it does not make much sense.

Yeah an ELF flag to override all tunables seems counter to the whole 
idea, but ELF flags to override specific tunables may make sense.  E.g. 
memory tagging enabled by default in the system and a program built with 
DF_NO_MEMTAG overrides that systemwide setting.

Sid
  

On 20/03/23 20:15, Siddhesh Poyarekar wrote:
> On 2023-03-20 12:59, Adhemerval Zanella Netto wrote:
>> I think it might make sense for tunables that change the program semantic, such
>> as security hardening; although I don't think it really fits for performance
>> oriented ones (such malloc or pthread tuning).  So maybe we can define a global
> 
> Administrative level performance tuning defaults for setuid binaries?

I was thinking more of opt-in security features (such as mte_state on aarch64),
although it might fit better on some ABI extension (such as done by cet).

> 
>> file format where the administrator can set where setuid binaries can use it,
>> and if uses can overwrite it.  My initial idea would be something quite simple,
>> similar to sysctl.conf:
> 
> Yes, I think Florian suggested reusing ld.so.conf instead.  I don't have a strong preference either way so y'all can fight that one out - potato potato ;)

Did he mean add the tunable information on ld.so.conf or using a similar scheme
where the file is preprocessed by a tool so loader can just mmap a file without
the need any parsing?

For later, I don't have a strong opinion.  I assume that a global tunable won't
be a default configuration, like loader cache; and parsing should really easy
(it a ini like file).

> 
>> And I was thinking about an DF_1_NODEFLIB analogous so the program can opt-out
>> any performance or behavior difference any tunable might incur.  Although with
>> your idea of enforceable tunable, I think it does not make much sense.
> 
> Yeah an ELF flag to override all tunables seems counter to the whole idea, but ELF flags to override specific tunables may make sense.  E.g. memory tagging enabled by default in the system and a program built with DF_NO_MEMTAG overrides that systemwide setting.

The issue of a per-tunable flag is it will require to settle a minimum ABI for
tunable, or adding a ELF extension with a string blob that is only understable
by an specific glibc version.  I am not sure, maybe we can it only if required.
  

On 2023-03-22 10:40, Adhemerval Zanella Netto wrote:
> 
> 
> On 20/03/23 20:15, Siddhesh Poyarekar wrote:
>> On 2023-03-20 12:59, Adhemerval Zanella Netto wrote:
>>> I think it might make sense for tunables that change the program semantic, such
>>> as security hardening; although I don't think it really fits for performance
>>> oriented ones (such malloc or pthread tuning).  So maybe we can define a global
>>
>> Administrative level performance tuning defaults for setuid binaries?
> 
> I was thinking more of opt-in security features (such as mte_state on aarch64),
> although it might fit better on some ABI extension (such as done by cet).

Yeah, that too.

>>> file format where the administrator can set where setuid binaries can use it,
>>> and if uses can overwrite it.  My initial idea would be something quite simple,
>>> similar to sysctl.conf:
>>
>> Yes, I think Florian suggested reusing ld.so.conf instead.  I don't have a strong preference either way so y'all can fight that one out - potato potato ;)
> 
> Did he mean add the tunable information on ld.so.conf or using a similar scheme
> where the file is preprocessed by a tool so loader can just mmap a file without
> the need any parsing?
> 
> For later, I don't have a strong opinion.  I assume that a global tunable won't
> be a default configuration, like loader cache; and parsing should really easy
> (it a ini like file).

Just the former IIRC, i.e. extending ld.so.conf to add a section for 
systemwide tunables.

>>
>>> And I was thinking about an DF_1_NODEFLIB analogous so the program can opt-out
>>> any performance or behavior difference any tunable might incur.  Although with
>>> your idea of enforceable tunable, I think it does not make much sense.
>>
>> Yeah an ELF flag to override all tunables seems counter to the whole idea, but ELF flags to override specific tunables may make sense.  E.g. memory tagging enabled by default in the system and a program built with DF_NO_MEMTAG overrides that systemwide setting.
> 
> The issue of a per-tunable flag is it will require to settle a minimum ABI for
> tunable, or adding a ELF extension with a string blob that is only understable
> by an specific glibc version.  I am not sure, maybe we can it only if required.

We shouldn't need a flag for *every* tunable, only those that would have 
practical benefit from having ELF overrides, e.g. those that control 
architecture-specific features.

That is, the ELF flag should not be tied to a tunable, but a tunable 
could change behaviour based on an ELF flag.

Sid
  

On 22/03/23 12:21, Siddhesh Poyarekar wrote:
> On 2023-03-22 10:40, Adhemerval Zanella Netto wrote:
>>
>>
>> On 20/03/23 20:15, Siddhesh Poyarekar wrote:
>>> On 2023-03-20 12:59, Adhemerval Zanella Netto wrote:
>>>> I think it might make sense for tunables that change the program semantic, such
>>>> as security hardening; although I don't think it really fits for performance
>>>> oriented ones (such malloc or pthread tuning).  So maybe we can define a global
>>>
>>> Administrative level performance tuning defaults for setuid binaries?
>>
>> I was thinking more of opt-in security features (such as mte_state on aarch64),
>> although it might fit better on some ABI extension (such as done by cet).
> 
> Yeah, that too.
> 
>>>> file format where the administrator can set where setuid binaries can use it,
>>>> and if uses can overwrite it.  My initial idea would be something quite simple,
>>>> similar to sysctl.conf:
>>>
>>> Yes, I think Florian suggested reusing ld.so.conf instead.  I don't have a strong preference either way so y'all can fight that one out - potato potato ;)
>>
>> Did he mean add the tunable information on ld.so.conf or using a similar scheme
>> where the file is preprocessed by a tool so loader can just mmap a file without
>> the need any parsing?
>>
>> For later, I don't have a strong opinion.  I assume that a global tunable won't
>> be a default configuration, like loader cache; and parsing should really easy
>> (it a ini like file).
> 
> Just the former IIRC, i.e. extending ld.so.conf to add a section for systemwide tunables.

I am not very found of mixing too different libc facilities in the same config
file and the syntax won't be the same and there are two different

> 
>>>
>>>> And I was thinking about an DF_1_NODEFLIB analogous so the program can opt-out
>>>> any performance or behavior difference any tunable might incur.  Although with
>>>> your idea of enforceable tunable, I think it does not make much sense.
>>>
>>> Yeah an ELF flag to override all tunables seems counter to the whole idea, but ELF flags to override specific tunables may make sense.  E.g. memory tagging enabled by default in the system and a program built with DF_NO_MEMTAG overrides that systemwide setting.
>>
>> The issue of a per-tunable flag is it will require to settle a minimum ABI for
>> tunable, or adding a ELF extension with a string blob that is only understable
>> by an specific glibc version.  I am not sure, maybe we can it only if required.
> 
> We shouldn't need a flag for *every* tunable, only those that would have practical benefit from having ELF overrides, e.g. those that control architecture-specific features.
> 
> That is, the ELF flag should not be tied to a tunable, but a tunable could change behaviour based on an ELF flag.

I think it might be an option if the flag is opaque to ELF itself, like a string
that glibc would parse.  But I don't have a strong preference.

I still think the simplest solution would to have a textual glibc-tunables.conf
similar to sysctl.conf.