math: Relax the fast-math gate for vector math declarations
Checks
| Context |
Check |
Description |
| redhat-pt-bot/TryBot-apply_patch |
success
|
Patch applied to master at the time it was sent
|
| linaro-tcwg-bot/tcwg_glibc_build--master-arm |
success
|
Build passed
|
| redhat-pt-bot/TryBot-32bit |
success
|
Build for i686
|
| linaro-tcwg-bot/tcwg_glibc_build--master-aarch64 |
success
|
Build passed
|
| linaro-tcwg-bot/tcwg_glibc_check--master-aarch64 |
success
|
Test passed
|
| linaro-tcwg-bot/tcwg_glibc_check--master-arm |
success
|
Test passed
|
Commit Message
The aarch64 and x86_64 <bits/math-vector.h> headers declare the libmvec
variants only under __FAST_MATH__. GCC does not define that macro for
-ffast-math -fno-finite-math-only, so code built that way to keep NaN
and Inf semantics gets no vectorized math calls. For example, GCC 16
on AArch64 vectorizes this loop with _ZGVnN4v_expf at -O3 -ffast-math,
but not once -fno-finite-math-only is added:
#include <math.h>
void f (float *restrict y, const float *restrict x, int n)
{
for (int i = 0; i < n; i++)
y[i] = expf (x[i]);
}
The vector routines handle NaN and Inf. They do not set errno or
exception flags, do not handle signed zeros or directed rounding, and
can be less accurate than the scalar functions. So also enable the
declarations when GCC defines __NO_MATH_ERRNO__, __NO_TRAPPING_MATH__
and __ASSOCIATIVE_MATH__ (which already permits results that change
with optimization, and implies -fno-signed-zeros), but not
__ROUNDING_MATH__. The rounding check is needed because the AArch64
tan and tanf vector routines can give wrong results in directed
rounding modes.
The __FAST_MATH__ condition is unchanged, so the patch only enables the
declarations in more cases. Strict modes, glibc's own build and
compilers before GCC 12, which do not define all of these macros, are
unaffected.
The motivating case is 736.ocio_r from SPEC CPU 2026, which relies on
NaN handling and so cannot use -ffinite-math-only.
Tested on aarch64-linux-gnu and x86_64-linux-gnu.
Co-authored-by: Konstantinos Eleftheriou <konstantinos.eleftheriou@vrull.eu>
---
sysdeps/aarch64/fpu/bits/math-vector.h | 11 ++++++++++-
sysdeps/x86/fpu/bits/math-vector.h | 11 ++++++++++-
2 files changed, 20 insertions(+), 2 deletions(-)
Comments
Hi,
For AArch64 at least, this patch is essentially making
-fassociative-math permit lower accuracy math routines. I agree that
-fassociative-math does suggest that the user is happy with degraded
accuracy, but is it OK to add this extra (undocumented) bit of behaviour
onto associative-math? Would it be possible to add a define on the
compiler side, either for unsafe-math-optimizations, or specifically
targeted at allowing low-accuracy math routines, and allow vectorisation
based on that instead?
Thanks,
Joe
On 01/10/2026 12:21, Philipp Tomsich wrote:
> The aarch64 and x86_64 <bits/math-vector.h> headers declare the libmvec
> variants only under __FAST_MATH__. GCC does not define that macro for
> -ffast-math -fno-finite-math-only, so code built that way to keep NaN
> and Inf semantics gets no vectorized math calls. For example, GCC 16
> on AArch64 vectorizes this loop with _ZGVnN4v_expf at -O3 -ffast-math,
> but not once -fno-finite-math-only is added:
>
> #include <math.h>
> void f (float *restrict y, const float *restrict x, int n)
> {
> for (int i = 0; i < n; i++)
> y[i] = expf (x[i]);
> }
>
> The vector routines handle NaN and Inf. They do not set errno or
> exception flags, do not handle signed zeros or directed rounding, and
> can be less accurate than the scalar functions. So also enable the
> declarations when GCC defines __NO_MATH_ERRNO__, __NO_TRAPPING_MATH__
> and __ASSOCIATIVE_MATH__ (which already permits results that change
> with optimization, and implies -fno-signed-zeros), but not
> __ROUNDING_MATH__. The rounding check is needed because the AArch64
> tan and tanf vector routines can give wrong results in directed
> rounding modes.
>
> The __FAST_MATH__ condition is unchanged, so the patch only enables the
> declarations in more cases. Strict modes, glibc's own build and
> compilers before GCC 12, which do not define all of these macros, are
> unaffected.
>
> The motivating case is 736.ocio_r from SPEC CPU 2026, which relies on
> NaN handling and so cannot use -ffinite-math-only.
>
> Tested on aarch64-linux-gnu and x86_64-linux-gnu.
>
> Co-authored-by: Konstantinos Eleftheriou <konstantinos.eleftheriou@vrull.eu>
> ---
> sysdeps/aarch64/fpu/bits/math-vector.h | 11 ++++++++++-
> sysdeps/x86/fpu/bits/math-vector.h | 11 ++++++++++-
> 2 files changed, 20 insertions(+), 2 deletions(-)
>
> diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h
> index db218eedf9..75393c06c1 100644
> --- a/sysdeps/aarch64/fpu/bits/math-vector.h
> +++ b/sysdeps/aarch64/fpu/bits/math-vector.h
> @@ -25,7 +25,16 @@
> /* Get default empty definitions for simd declarations. */
> #include <bits/libm-simd-decl-stubs.h>
>
> -#if defined __aarch64__ && defined __FAST_MATH__ && __GNUC_PREREQ (6, 0)
> +/* The vector routines handle NaN and Inf but not errno, exception
> + flags, signed zeros or directed rounding, and can be less accurate
> + than the scalar functions. Besides -ffast-math, enable the
> + declarations when GCC's predefined macros show that the code does
> + not rely on any of these (__ASSOCIATIVE_MATH__ implies
> + -fno-signed-zeros), as with -ffast-math -fno-finite-math-only. */
> +#if defined __aarch64__ && __GNUC_PREREQ (6, 0) \
> + && (defined __FAST_MATH__ \
> + || (defined __NO_MATH_ERRNO__ && defined __NO_TRAPPING_MATH__ \
> + && defined __ASSOCIATIVE_MATH__ && !defined __ROUNDING_MATH__))
> /* Requires GCC >= 6 for __attribute__ ((__simd__)). */
> # define __DECL_SIMD_aarch64 __attribute__ ((__simd__ ("notinbranch"), const))
>
> diff --git a/sysdeps/x86/fpu/bits/math-vector.h b/sysdeps/x86/fpu/bits/math-vector.h
> index f72bab1609..f7ba2f16c9 100644
> --- a/sysdeps/x86/fpu/bits/math-vector.h
> +++ b/sysdeps/x86/fpu/bits/math-vector.h
> @@ -24,7 +24,16 @@
> /* Get default empty definitions for simd declarations. */
> #include <bits/libm-simd-decl-stubs.h>
>
> -#if defined __x86_64__ && defined __FAST_MATH__
> +/* The vector routines handle NaN and Inf but not errno, exception
> + flags, signed zeros or directed rounding, and can be less accurate
> + than the scalar functions. Besides -ffast-math, enable the
> + declarations when GCC's predefined macros show that the code does
> + not rely on any of these (__ASSOCIATIVE_MATH__ implies
> + -fno-signed-zeros), as with -ffast-math -fno-finite-math-only. */
> +#if defined __x86_64__ \
> + && (defined __FAST_MATH__ \
> + || (defined __NO_MATH_ERRNO__ && defined __NO_TRAPPING_MATH__ \
> + && defined __ASSOCIATIVE_MATH__ && !defined __ROUNDING_MATH__))
> # if defined _OPENMP && _OPENMP >= 201307
> /* OpenMP case. */
> # define __DECL_SIMD_x86_64 _Pragma ("omp declare simd notinbranch")
@@ -25,7 +25,16 @@
/* Get default empty definitions for simd declarations. */
#include <bits/libm-simd-decl-stubs.h>
-#if defined __aarch64__ && defined __FAST_MATH__ && __GNUC_PREREQ (6, 0)
+/* The vector routines handle NaN and Inf but not errno, exception
+ flags, signed zeros or directed rounding, and can be less accurate
+ than the scalar functions. Besides -ffast-math, enable the
+ declarations when GCC's predefined macros show that the code does
+ not rely on any of these (__ASSOCIATIVE_MATH__ implies
+ -fno-signed-zeros), as with -ffast-math -fno-finite-math-only. */
+#if defined __aarch64__ && __GNUC_PREREQ (6, 0) \
+ && (defined __FAST_MATH__ \
+ || (defined __NO_MATH_ERRNO__ && defined __NO_TRAPPING_MATH__ \
+ && defined __ASSOCIATIVE_MATH__ && !defined __ROUNDING_MATH__))
/* Requires GCC >= 6 for __attribute__ ((__simd__)). */
# define __DECL_SIMD_aarch64 __attribute__ ((__simd__ ("notinbranch"), const))
@@ -24,7 +24,16 @@
/* Get default empty definitions for simd declarations. */
#include <bits/libm-simd-decl-stubs.h>
-#if defined __x86_64__ && defined __FAST_MATH__
+/* The vector routines handle NaN and Inf but not errno, exception
+ flags, signed zeros or directed rounding, and can be less accurate
+ than the scalar functions. Besides -ffast-math, enable the
+ declarations when GCC's predefined macros show that the code does
+ not rely on any of these (__ASSOCIATIVE_MATH__ implies
+ -fno-signed-zeros), as with -ffast-math -fno-finite-math-only. */
+#if defined __x86_64__ \
+ && (defined __FAST_MATH__ \
+ || (defined __NO_MATH_ERRNO__ && defined __NO_TRAPPING_MATH__ \
+ && defined __ASSOCIATIVE_MATH__ && !defined __ROUNDING_MATH__))
# if defined _OPENMP && _OPENMP >= 201307
/* OpenMP case. */
# define __DECL_SIMD_x86_64 _Pragma ("omp declare simd notinbranch")