uapi/auxvec: Define AT_HWCAP3 and AT_HWCAP4 aux vector, entries
Checks
Commit Message
The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast
access by our __builtin_cpu_supports built-in function. The TCB space for
the HWCAP entries - which are created in pairs - is an ABI extension, so
waiting to create the space for HWCAP3 and HWCAP4 until we need them is
problematical, given distro unwillingness to apply ABI modifying patches
to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic
uapi header so they can be used in GLIBC to reserve space in the powerpc
TCB for their future use.
I scanned both the Linux and GLIBC source codes looking for unused AT_*
values and 29 and 30 did not seem to be used, so they are what I went
with. If anyone sees a problem with using those specific values, I'm
amenable to using other values, just let me know what would be better.
Peter
Signed-off-by: Peter Bergner <bergner@linux.ibm.com>
---
include/uapi/linux/auxvec.h | 2 ++
1 file changed, 2 insertions(+)
Comments
On 26/09/23 19:02, Peter Bergner wrote:
> The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast
> access by our __builtin_cpu_supports built-in function. The TCB space for
> the HWCAP entries - which are created in pairs - is an ABI extension, so
> waiting to create the space for HWCAP3 and HWCAP4 until we need them is
> problematical, given distro unwillingness to apply ABI modifying patches
> to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic
> uapi header so they can be used in GLIBC to reserve space in the powerpc
> TCB for their future use.
This is different than previously exported auxv, where each AT_* constant
would have a auxv entry. On glibc it would require changing _dl_parse_auxv
to iterate over the auxv_values to find AT_HWCAP3/AT_HWCAP4 (not ideal,
but doable).
Wouldn't be better to always export it on fs/binfmt_elf.c, along with all
the machinery to setup it (ELF_HWCAP3, etc), along with proper documentation?
>
> I scanned both the Linux and GLIBC source codes looking for unused AT_*
> values and 29 and 30 did not seem to be used, so they are what I went
> with. If anyone sees a problem with using those specific values, I'm
> amenable to using other values, just let me know what would be better.
>
> Peter
>
>
> Signed-off-by: Peter Bergner <bergner@linux.ibm.com>
> ---
> include/uapi/linux/auxvec.h | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/include/uapi/linux/auxvec.h b/include/uapi/linux/auxvec.h
> index 6991c4b8ab18..cc61cb9b3e9a 100644
> --- a/include/uapi/linux/auxvec.h
> +++ b/include/uapi/linux/auxvec.h
> @@ -32,6 +32,8 @@
> #define AT_HWCAP2 26 /* extension of AT_HWCAP */
> #define AT_RSEQ_FEATURE_SIZE 27 /* rseq supported feature size */
> #define AT_RSEQ_ALIGN 28 /* rseq allocation alignment */
> +#define AT_HWCAP3 29 /* extension of AT_HWCAP */
> +#define AT_HWCAP4 30 /* extension of AT_HWCAP */
>
> #define AT_EXECFN 31 /* filename of program */
>
Hi Adhemerval, sorry for the delay in replying, I was a little under the
weather last week.
On 9/27/23 11:03 AM, Adhemerval Zanella Netto wrote:
> On 26/09/23 19:02, Peter Bergner wrote:
>> The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast
>> access by our __builtin_cpu_supports built-in function. The TCB space for
>> the HWCAP entries - which are created in pairs - is an ABI extension, so
>> waiting to create the space for HWCAP3 and HWCAP4 until we need them is
>> problematical, given distro unwillingness to apply ABI modifying patches
>> to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic
>> uapi header so they can be used in GLIBC to reserve space in the powerpc
>> TCB for their future use.
>
> This is different than previously exported auxv, where each AT_* constant
> would have a auxv entry. On glibc it would require changing _dl_parse_auxv
> to iterate over the auxv_values to find AT_HWCAP3/AT_HWCAP4 (not ideal,
> but doable).
When you say different, do you mean because all AUXVs exported by the kernel
*will* have an AT_HWCAP and AT_HWCAP2 entry and AT_HWCAP3/AT_HWCAP4 won't?
I don't think that's a problem for either kernel or glibc side of things.
I'm not even sure there is a guarantee that every AT_* value *must* be
present in the exported AUXV.
The new AT_HWCAP3/AT_HWCAP4 defines are less than AT_MINSIGSTKSZ, so they
don't affect the size of _dl_parse_auxv's auxv_values array size and the
AT_HWCAP3 and AT_HWCAP4 entries in auxv_values[] are already initialized
to zero today. Additionally, the loop in _dl_parse_auxv already parses
the entire AUXV, so there is no extra work for it to do, unless and until
AT_HWCAP3 and AT_HWCAP4 start being exported by the kernel. Really, the
only extra work _dl_parse_auxv would need to do, is add two new stores:
GLRO(dl_hwcap3) = auxv_values[AT_HWCAP3];
GLRO(dl_hwcap4) = auxv_values[AT_HWCAP4];
> Wouldn't be better to always export it on fs/binfmt_elf.c, along with all
> the machinery to setup it (ELF_HWCAP3, etc), along with proper documentation?
You mean modify the kernel now to export AT_HWCAP3 and AT_HWCAP4 as zero
masks? Is that really necessary since we don't need or have any features
defined in them yet? GLIBC's _dl_parse_auxv doesn't really need them to
be exported either, since in the absence of the entries, it will just end
up using zero masks for dl_hwcap3 and dl_hwcap4, so everything is copacetic
even without any kernel changes.
As I mentioned, our real problem is the lead time for getting changes that
affect the user ABI into a distro release, and ppc's copy/cache of the HWCAP
masks is an ABI change. If we wait to add this support until when we
actually have a need for HWCAP3, then we won't have any support until
the next major distro release. However, if we can add this support now,
which I don't think is an onerous change on glibc's part, then we can
start using it immediately when Linux starts exporting them.
Peter
On 02/10/23 18:19, Peter Bergner wrote:
> Hi Adhemerval, sorry for the delay in replying, I was a little under the
> weather last week.
>
>
> On 9/27/23 11:03 AM, Adhemerval Zanella Netto wrote:
>> On 26/09/23 19:02, Peter Bergner wrote:
>>> The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast
>>> access by our __builtin_cpu_supports built-in function. The TCB space for
>>> the HWCAP entries - which are created in pairs - is an ABI extension, so
>>> waiting to create the space for HWCAP3 and HWCAP4 until we need them is
>>> problematical, given distro unwillingness to apply ABI modifying patches
>>> to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic
>>> uapi header so they can be used in GLIBC to reserve space in the powerpc
>>> TCB for their future use.
>>
>> This is different than previously exported auxv, where each AT_* constant
>> would have a auxv entry. On glibc it would require changing _dl_parse_auxv
>> to iterate over the auxv_values to find AT_HWCAP3/AT_HWCAP4 (not ideal,
>> but doable).
>
> When you say different, do you mean because all AUXVs exported by the kernel
> *will* have an AT_HWCAP and AT_HWCAP2 entry and AT_HWCAP3/AT_HWCAP4 won't?
> I don't think that's a problem for either kernel or glibc side of things.
> I'm not even sure there is a guarantee that every AT_* value *must* be
> present in the exported AUXV.
>
> The new AT_HWCAP3/AT_HWCAP4 defines are less than AT_MINSIGSTKSZ, so they
> don't affect the size of _dl_parse_auxv's auxv_values array size and the
> AT_HWCAP3 and AT_HWCAP4 entries in auxv_values[] are already initialized
> to zero today. Additionally, the loop in _dl_parse_auxv already parses
> the entire AUXV, so there is no extra work for it to do, unless and until
> AT_HWCAP3 and AT_HWCAP4 start being exported by the kernel. Really, the
> only extra work _dl_parse_auxv would need to do, is add two new stores:
>
> GLRO(dl_hwcap3) = auxv_values[AT_HWCAP3];
> GLRO(dl_hwcap4) = auxv_values[AT_HWCAP4];
>
Indeed you are right, I wrong assumed that the AT_HWCAP3/AT_HWCAP4 would
be higher than AT_MINSIGSTKSZ.
>
>
>> Wouldn't be better to always export it on fs/binfmt_elf.c, along with all
>> the machinery to setup it (ELF_HWCAP3, etc), along with proper documentation?
>
> You mean modify the kernel now to export AT_HWCAP3 and AT_HWCAP4 as zero
> masks? Is that really necessary since we don't need or have any features
> defined in them yet? GLIBC's _dl_parse_auxv doesn't really need them to
> be exported either, since in the absence of the entries, it will just end
> up using zero masks for dl_hwcap3 and dl_hwcap4, so everything is copacetic
> even without any kernel changes.
>
> As I mentioned, our real problem is the lead time for getting changes that
> affect the user ABI into a distro release, and ppc's copy/cache of the HWCAP
> masks is an ABI change. If we wait to add this support until when we
> actually have a need for HWCAP3, then we won't have any support until
> the next major distro release. However, if we can add this support now,
> which I don't think is an onerous change on glibc's part, then we can
> start using it immediately when Linux starts exporting them.
What it is not clear to me is what kind of ABI boundary you are trying to
preemptively add support here. The TCB ABI for __builtin_cpu_supports is
userland only, so if your intention is just to allow gcc to work on older
glibcs, it should be a matter to just reserve the space on tcbhead_t. Once
kernel actually provides AT_HWCAP3/AT_HWCAP4, backporting should be
straightforward.
If your intention is to also add support on glibc, it makes more sense to
already reserve it. For __builtin_cpu_supports it should work, although
for glibc itself some backporting would be required (to correctly showing
the bits with LD_SHOW_AUXV).
On 10/3/23 9:08 AM, Adhemerval Zanella Netto wrote:
> What it is not clear to me is what kind of ABI boundary you are trying to
> preemptively add support here. The TCB ABI for __builtin_cpu_supports is
> userland only, so if your intention is just to allow gcc to work on older
> glibcs, it should be a matter to just reserve the space on tcbhead_t.
Yes, extending tcbhead_t to contain the slots for hwcap3 and hwcap4 are the
ABI extensions we are interested in, and not something that can be backported
into a distro point release. Yes, we don't strictly need the AT_HWCAP3 and
AT_HWCAP4 kernel defines to reserve (and clear) that space in glibc, but....
> If your intention is to also add support on glibc, it makes more sense to
> already reserve it. For __builtin_cpu_supports it should work, although
> for glibc itself some backporting would be required (to correctly showing
> the bits with LD_SHOW_AUXV).
Our intention is to also add the glibc support too once we have the
AT_HWCAP3 and AT_HWCAP4 kernel macros defined. 1) Once the defines are
there, adding the support should be pretty straight forward, so why wait?
And 2) part of the glibc and compiler support introduces a new symbol
that is exported by glibc and referenced by the compilers to ensure the
compilers *never* access the hwcap* fields in the TCB unless the glibc
supports them. See the symbol __parse_hwcap_and_convert_at_platform used
for HWCAP/HWCAP2. We'll need a similar one for HWCAP3/HWCAP4 and I'm
doubtful whether the distros will allow the backport of a patch that
introduces a new exported symbol from glibc in a distro point release.
Peter
On 03/10/23 19:12, Peter Bergner wrote:
> On 10/3/23 9:08 AM, Adhemerval Zanella Netto wrote:
>> What it is not clear to me is what kind of ABI boundary you are trying to
>> preemptively add support here. The TCB ABI for __builtin_cpu_supports is
>> userland only, so if your intention is just to allow gcc to work on older
>> glibcs, it should be a matter to just reserve the space on tcbhead_t.
>
> Yes, extending tcbhead_t to contain the slots for hwcap3 and hwcap4 are the
> ABI extensions we are interested in, and not something that can be backported
> into a distro point release. Yes, we don't strictly need the AT_HWCAP3 and
> AT_HWCAP4 kernel defines to reserve (and clear) that space in glibc, but....
>
>
>
>> If your intention is to also add support on glibc, it makes more sense to
>> already reserve it. For __builtin_cpu_supports it should work, although
>> for glibc itself some backporting would be required (to correctly showing
>> the bits with LD_SHOW_AUXV).
>
> Our intention is to also add the glibc support too once we have the
> AT_HWCAP3 and AT_HWCAP4 kernel macros defined. 1) Once the defines are
> there, adding the support should be pretty straight forward, so why wait?
> And 2) part of the glibc and compiler support introduces a new symbol
> that is exported by glibc and referenced by the compilers to ensure the
> compilers *never* access the hwcap* fields in the TCB unless the glibc
> supports them. See the symbol __parse_hwcap_and_convert_at_platform used
> for HWCAP/HWCAP2. We'll need a similar one for HWCAP3/HWCAP4 and I'm
> doubtful whether the distros will allow the backport of a patch that
> introduces a new exported symbol from glibc in a distro point release.
Alright, I makes more sense it now. And indeed backporting a __parse_hwcap
for HWCAP3/HWCAP4 will be frown upon.
CCing linux-kernel for more exposure.
PING. I'm waiting on a reply from anyone on the kernel side of things
to see whether they have an issue with reserving values for AT_HWCAP3
and AT_HWCAP4.
I'll note reviews from the GLIBC camp did not have an issue with the below patch.
Thanks.
Peter
On 9/26/23 5:02 PM, Peter Bergner wrote:
> The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast
> access by our __builtin_cpu_supports built-in function. The TCB space for
> the HWCAP entries - which are created in pairs - is an ABI extension, so
> waiting to create the space for HWCAP3 and HWCAP4 until we need them is
> problematical, given distro unwillingness to apply ABI modifying patches
> to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic
> uapi header so they can be used in GLIBC to reserve space in the powerpc
> TCB for their future use.
>
> I scanned both the Linux and GLIBC source codes looking for unused AT_*
> values and 29 and 30 did not seem to be used, so they are what I went
> with. If anyone sees a problem with using those specific values, I'm
> amenable to using other values, just let me know what would be better.
>
> Peter
>
>
> Signed-off-by: Peter Bergner <bergner@linux.ibm.com>
> ---
> include/uapi/linux/auxvec.h | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/include/uapi/linux/auxvec.h b/include/uapi/linux/auxvec.h
> index 6991c4b8ab18..cc61cb9b3e9a 100644
> --- a/include/uapi/linux/auxvec.h
> +++ b/include/uapi/linux/auxvec.h
> @@ -32,6 +32,8 @@
> #define AT_HWCAP2 26 /* extension of AT_HWCAP */
> #define AT_RSEQ_FEATURE_SIZE 27 /* rseq supported feature size */
> #define AT_RSEQ_ALIGN 28 /* rseq allocation alignment */
> +#define AT_HWCAP3 29 /* extension of AT_HWCAP */
> +#define AT_HWCAP4 30 /* extension of AT_HWCAP */
>
> #define AT_EXECFN 31 /* filename of program */
>
The 10/17/2023 18:14, Peter Bergner wrote:
> CCing linux-kernel for more exposure.
>
> PING. I'm waiting on a reply from anyone on the kernel side of things
> to see whether they have an issue with reserving values for AT_HWCAP3
> and AT_HWCAP4.
>
> I'll note reviews from the GLIBC camp did not have an issue with the below patch.
fwiw, aarch64 is quickly filling up AT_HWCAP2 so this will be
useful for arm64 too eventually, but we are not in a hurry.
> > +#define AT_HWCAP3 29 /* extension of AT_HWCAP */
> > +#define AT_HWCAP4 30 /* extension of AT_HWCAP */
@@ -32,6 +32,8 @@
#define AT_HWCAP2 26 /* extension of AT_HWCAP */
#define AT_RSEQ_FEATURE_SIZE 27 /* rseq supported feature size */
#define AT_RSEQ_ALIGN 28 /* rseq allocation alignment */
+#define AT_HWCAP3 29 /* extension of AT_HWCAP */
+#define AT_HWCAP4 30 /* extension of AT_HWCAP */
#define AT_EXECFN 31 /* filename of program */