diff mbox

[RFC,1/2] Aarch64: Add simd exp/expf ABI symbols

Message ID 0ae3af1b-cb2b-367f-8b86-f39971498291@arm.com
State Dropped
Headers

Commit Message

Szabolcs Nagy July 4, 2019, 11:09 a.m. UTC 
  The implementation is in assembly and just calls the scalar math code.
This ensures that old compiler without vector call abi support can
build libmvec. The abi is supported since GCC 9.1, the specification is

https://developer.arm.com/tools-and-software/server-and-hpc/arm-architecture-tools/arm-compiler-for-hpc/vector-function-abi

Vector functions require a STO_AARCH64_VARIANT_PCS marking in the
dynamic symbol table for lazy bound calls to work. This will be
missing in libmvec, which works because the marking only affects
the behaviour if there are calls to the symbols in the binary.

(TODO: detect .variant_pcs asm support and use the directive if available.)

Testing requires vector call abi support, which is detected.

Header declarations are not added yet, so the symbols will not be used
by the compiler: they are just added so the abi is in place which
enables backporting later. Currently we cannot add correct declarations
that only declare the specific symbols we provide: the OpenMP pragma
mechanism would declare both AdvSIMD and SVE variants.

(TODO: figure out the c and fortran header magic that works and backportable)

this is a bit late for 2.30, but wanted to show how i planed to add
libmvec symbols for backporting.

2019-07-04  Steve Ellcey  <sellcey@marvell.com>
	    Szabolcs Nagy  <szabolcs.nagy@arm.com>

	* sysdeps/aarch64/configure.ac (build_mathvec): Enable.
	(test-mathvec): Enable if ABI is supported.
	* sysdeps/aarch64/configure: Regenerate.
	* sysdeps/aarch64/fpu/Makefile
	(libmvec-support): Add libmvec_double_vlen2_exp,
	libmvec_float_vlen4_expf to list.
	(libmvec-tests): Add double-vlen2, float-vlen4 to list.
	(double-vlen2-funcs): Add new vector function name.
	(float-vlen4-funcs): Add new vector function name.
	* sysdeps/aarch64/fpu/Versions: New file.
	* sysdeps/aarch64/fpu/libmvec_double_vlen2.h: New file.
	* sysdeps/aarch64/fpu/libmvec_double_vlen2_exp.S: New file.
	* sysdeps/aarch64/fpu/libmvec_float_vlen4.h: New file.
	* sysdeps/aarch64/fpu/libmvec_float_vlen4_expf.S: New file.
	* sysdeps/aarch64/fpu/test-double-vlen2-wrappers.c: New file.
	* sysdeps/aarch64/fpu/test-float-vlen4-wrappers.c: New file.
	* sysdeps/aarch64/libm-test-ulps (exp_vlen2): New entry.
	(exp_vlen4): Likewise.
	* sysdeps/unix/sysv/linux/aarch64/libmvec.abilist: New file.
---
 sysdeps/aarch64/configure                     | 31 +++++++++
 sysdeps/aarch64/configure.ac                  | 24 +++++++
 sysdeps/aarch64/fpu/Makefile                  | 17 +++++
 sysdeps/aarch64/fpu/Versions                  |  5 ++
 sysdeps/aarch64/fpu/libmvec_double_vlen2.h    | 59 +++++++++++++++++
 .../aarch64/fpu/libmvec_double_vlen2_exp.S    | 21 ++++++
 sysdeps/aarch64/fpu/libmvec_float_vlen4.h     | 65 +++++++++++++++++++
 .../aarch64/fpu/libmvec_float_vlen4_expf.S    | 21 ++++++
 .../aarch64/fpu/test-double-vlen2-wrappers.c  | 28 ++++++++
 .../aarch64/fpu/test-float-vlen4-wrappers.c   | 28 ++++++++
 sysdeps/aarch64/libm-test-ulps                |  6 ++
 .../unix/sysv/linux/aarch64/libmvec.abilist   |  2 +
 12 files changed, 307 insertions(+)
 create mode 100644 sysdeps/aarch64/fpu/Versions
 create mode 100644 sysdeps/aarch64/fpu/libmvec_double_vlen2.h
 create mode 100644 sysdeps/aarch64/fpu/libmvec_double_vlen2_exp.S
 create mode 100644 sysdeps/aarch64/fpu/libmvec_float_vlen4.h
 create mode 100644 sysdeps/aarch64/fpu/libmvec_float_vlen4_expf.S
 create mode 100644 sysdeps/aarch64/fpu/test-double-vlen2-wrappers.c
 create mode 100644 sysdeps/aarch64/fpu/test-float-vlen4-wrappers.c
 create mode 100644 sysdeps/unix/sysv/linux/aarch64/libmvec.abilist

Comments

Tulio Magno Quites Machado Filho July 8, 2019, 6 p.m. UTC | #1 
Szabolcs Nagy <Szabolcs.Nagy@arm.com> writes:

> diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions
> new file mode 100644
> index 0000000000..da36f3c495
> --- /dev/null
> +++ b/sysdeps/aarch64/fpu/Versions
> @@ -0,0 +1,5 @@
> +libmvec {
> +  GLIBC_2.30 {
> +    _ZGVnN2v_exp; _ZGVnN4v_expf;

The prefix _ZGV is reserved for guard variables in the C++ ABI [1] [2].
How do you plan to avoid a conflict there?

[1] https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/cp/mangle.c;#l4242
[2] https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/testsuite/g%2B%2B.dg/abi/aarch64_guard1.C
Szabolcs Nagy July 9, 2019, 8:52 a.m. UTC | #2 
On 08/07/2019 19:00, Tulio Magno Quites Machado Filho wrote:
> Szabolcs Nagy <Szabolcs.Nagy@arm.com> writes:

> 

>> diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions

>> new file mode 100644

>> index 0000000000..da36f3c495

>> --- /dev/null

>> +++ b/sysdeps/aarch64/fpu/Versions

>> @@ -0,0 +1,5 @@

>> +libmvec {

>> +  GLIBC_2.30 {

>> +    _ZGVnN2v_exp; _ZGVnN4v_expf;

> 

> The prefix _ZGV is reserved for guard variables in the C++ ABI [1] [2].

> How do you plan to avoid a conflict there?

> 

> [1] https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/cp/mangle.c;#l4242

> [2] https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/testsuite/g%2B%2B.dg/abi/aarch64_guard1.C


i noticed that too when this was designed, but it
was decided that it's better to use the intel vector
function mangling on aarch64 too than to introduce
target specific mangling.

i don't know how serious such a conflict is, nor why
intel chose a clearly conflicting prefix. i assumed
c++ folks were aware of the issue and accepted it
when the intel vector abi was introduced.
Florian Weimer July 9, 2019, 10 a.m. UTC | #3 
* Szabolcs Nagy:

> On 08/07/2019 19:00, Tulio Magno Quites Machado Filho wrote:
>> Szabolcs Nagy <Szabolcs.Nagy@arm.com> writes:
>> 
>>> diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions
>>> new file mode 100644
>>> index 0000000000..da36f3c495
>>> --- /dev/null
>>> +++ b/sysdeps/aarch64/fpu/Versions
>>> @@ -0,0 +1,5 @@
>>> +libmvec {
>>> +  GLIBC_2.30 {
>>> +    _ZGVnN2v_exp; _ZGVnN4v_expf;
>> 
>> The prefix _ZGV is reserved for guard variables in the C++ ABI [1] [2].
>> How do you plan to avoid a conflict there?
>> 
>> [1] https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/cp/mangle.c;#l4242
>> [2] https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/testsuite/g%2B%2B.dg/abi/aarch64_guard1.C
>
> i noticed that too when this was designed, but it
> was decided that it's better to use the intel vector
> function mangling on aarch64 too than to introduce
> target specific mangling.
>
> i don't know how serious such a conflict is, nor why
> intel chose a clearly conflicting prefix. i assumed
> c++ folks were aware of the issue and accepted it
> when the intel vector abi was introduced.

I think the mangled names of objects always start with a capital letter,
so there is no conflict in practice because the libmvec names use a
lower-case letter after the _ZGV prefix.

It may make sense to ask the C++ developers for clarification.

Thanks,
Florian
Steve Ellcey July 12, 2019, 9:51 p.m. UTC | #4 
Szabolcs,

I applied your patches (1 and 2) to a glibc tree and tried doing a full
build (built ToT binutils, then partial ToT GCC, then ToT glibc with
your patches, then full GCC) and I got an error when building the final
GCC:

/extra/sellcey/gcc-vect/install/aarch64-linux-gnu/bin/ld: cannot find /lib/libmvec_nonshared.a inside /extra/sellcey/gcc-vect/obj-gcc/gcc/../../install
collect2: error: ld returned 1 exit status
Makefile:1704: recipe for target 'libgfortran.la' failed

I don't see a libmvec_nonshared.a object in my build or install
directory.  There are libmvec.so and libmvec.a libraries.  Have you run
into this problem?  You may not have hit it if you don't config GCC
with the Fortran compiler.

I think it is necessary to have some object (even if it is not used)
that is put in the libmvec-static-only-routines list of objects in
sysdeps/aarch64/fpu/Makefile in order to build libmvec_nonshared.a.  If
you don't have that library the linker script fails when it cannot find
it.

Steve Ellcey
sellcey@marvell.com
Szabolcs Nagy July 15, 2019, 10:33 a.m. UTC | #5 
On 12/07/2019 22:51, Steve Ellcey wrote:
> Szabolcs,

> 

> I applied your patches (1 and 2) to a glibc tree and tried doing a full

> build (built ToT binutils, then partial ToT GCC, then ToT glibc with

> your patches, then full GCC) and I got an error when building the final

> GCC:

> 

> /extra/sellcey/gcc-vect/install/aarch64-linux-gnu/bin/ld: cannot find /lib/libmvec_nonshared.a inside /extra/sellcey/gcc-vect/obj-gcc/gcc/../../install

> collect2: error: ld returned 1 exit status

> Makefile:1704: recipe for target 'libgfortran.la' failed

> 

> I don't see a libmvec_nonshared.a object in my build or install

> directory.  There are libmvec.so and libmvec.a libraries.  Have you run

> into this problem?  You may not have hit it if you don't config GCC

> with the Fortran compiler.

> 

> I think it is necessary to have some object (even if it is not used)

> that is put in the libmvec-static-only-routines list of objects in

> sysdeps/aarch64/fpu/Makefile in order to build libmvec_nonshared.a.  If

> you don't have that library the linker script fails when it cannot find

> it.


you are right, i think

libmvec-static-only-routines = non-existing-routine

in that Makefile would fix it: creates an empty libmvec_nonshared.a.

i made this change and put the patches on the nsz/mathvec branch.
Florian Weimer July 15, 2019, 10:55 a.m. UTC | #6 
* Steve Ellcey:

> I think it is necessary to have some object (even if it is not used)
> that is put in the libmvec-static-only-routines list of objects in
> sysdeps/aarch64/fpu/Makefile in order to build libmvec_nonshared.a.  If
> you don't have that library the linker script fails when it cannot find
> it.

Maybe you could make the generation of the linker script optional in
math/Makefile?

Thanks,
Florian
Szabolcs Nagy July 15, 2019, 11:04 a.m. UTC | #7 
On 15/07/2019 11:55, Florian Weimer wrote:
> * Steve Ellcey:

> 

>> I think it is necessary to have some object (even if it is not used)

>> that is put in the libmvec-static-only-routines list of objects in

>> sysdeps/aarch64/fpu/Makefile in order to build libmvec_nonshared.a.  If

>> you don't have that library the linker script fails when it cannot find

>> it.

> 

> Maybe you could make the generation of the linker script optional in

> math/Makefile?


the linker script is needed anyway to add libmvec.so,
just the libmvec_nonshared.a part can be dropped on
targets that don't need it.

but it's easier to have an empty libmvec_nonshared.a

(it would be nicer if all symbols were in libm.so
without any linker script and nonshared hack, but
i guess it's too late for that)
Steve Ellcey July 15, 2019, 5:14 p.m. UTC | #8 
On Mon, 2019-07-15 at 10:33 +0000, Szabolcs Nagy wrote:
> 

> > 

> > I think it is necessary to have some object (even if it is not used)

> > that is put in the libmvec-static-only-routines list of objects in

> > sysdeps/aarch64/fpu/Makefile in order to build libmvec_nonshared.a.  If

> > you don't have that library the linker script fails when it cannot find

> > it.

> 

> you are right, i think

> 

> libmvec-static-only-routines = non-existing-routine

> 

> in that Makefile would fix it: creates an empty libmvec_nonshared.a.

> 

> i made this change and put the patches on the nsz/mathvec branch.


Thanks, the other thing I noticed when I tried building with the
patches was that no functions got declared as SIMD in math-vector.h
or math-vector-fortran.h.  Not even exp or expf.

Steve Ellcey
sellcey@marvell.com
Andreas Schwab July 15, 2019, 5:25 p.m. UTC | #9 
On Jul 15 2019, Szabolcs Nagy <Szabolcs.Nagy@arm.com> wrote:

> you are right, i think
>
> libmvec-static-only-routines = non-existing-routine
>
> in that Makefile would fix it: creates an empty libmvec_nonshared.a.

You should use make-dummy-lib.

Andreas.
Szabolcs Nagy July 15, 2019, 6:28 p.m. UTC | #10 
On 15/07/2019 18:14, Steve Ellcey wrote:
> On Mon, 2019-07-15 at 10:33 +0000, Szabolcs Nagy wrote:

>>

>>>

>>> I think it is necessary to have some object (even if it is not used)

>>> that is put in the libmvec-static-only-routines list of objects in

>>> sysdeps/aarch64/fpu/Makefile in order to build libmvec_nonshared.a.  If

>>> you don't have that library the linker script fails when it cannot find

>>> it.

>>

>> you are right, i think

>>

>> libmvec-static-only-routines = non-existing-routine

>>

>> in that Makefile would fix it: creates an empty libmvec_nonshared.a.

>>

>> i made this change and put the patches on the nsz/mathvec branch.

> 

> Thanks, the other thing I noticed when I tried building with the

> patches was that no functions got declared as SIMD in math-vector.h

> or math-vector-fortran.h.  Not even exp or expf.



yes, because currently it's not possible to declare them correctly.
(adding omp declare simd should work today, but when gcc gets sve
support that will be incorrect, so you can add such declarations
for testing purposes, but it's not valid to distribute glibc headers
that way)

(and wilco's finite math removal patches will be needed once we
add declarations to the headers.)
Szabolcs Nagy July 15, 2019, 6:29 p.m. UTC | #11 
On 15/07/2019 18:25, Andreas Schwab wrote:
> On Jul 15 2019, Szabolcs Nagy <Szabolcs.Nagy@arm.com> wrote:

> 

>> you are right, i think

>>

>> libmvec-static-only-routines = non-existing-routine

>>

>> in that Makefile would fix it: creates an empty libmvec_nonshared.a.

> 

> You should use make-dummy-lib.


thanks.
diff mbox

Patch

diff --git a/sysdeps/aarch64/configure b/sysdeps/aarch64/configure
index 5bd355a691..df15cdb02a 100644
--- a/sysdeps/aarch64/configure
+++ b/sysdeps/aarch64/configure
@@ -172,3 +172,34 @@  else
   config_vars="$config_vars
 default-abi = lp64"
 fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for pcs attribute support" >&5
+$as_echo_n "checking for pcs attribute support... " >&6; }
+if ${libc_cv_gcc_pcs_attribute+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  cat > conftest.c <<EOF
+__attribute__((aarch64_vector_pcs)) extern void foo (void);
+EOF
+libc_cv_gcc_pcs_attribute=no
+if ${CC-cc} -c -Wall -Werror conftest.c -o conftest.o 1>&5 \
+   2>&5 ; then
+  libc_cv_gcc_pcs_attribute=yes
+fi
+rm -f conftest*
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_gcc_pcs_attribute" >&5
+$as_echo "$libc_cv_gcc_pcs_attribute" >&6; }
+
+# Enable libmvec by default.
+if test x"$build_mathvec" = xnotset; then
+  build_mathvec=yes
+fi
+
+# Only test libmvec if the compiler supports aarch64_vector_pcs.
+if test x"$build_mathvec" = xyes; then
+  if test $libc_cv_gcc_pcs_attribute = yes; then
+    config_vars="$config_vars
+test-mathvec = yes"
+  fi
+fi
diff --git a/sysdeps/aarch64/configure.ac b/sysdeps/aarch64/configure.ac
index 7851dd4dac..eab411cad4 100644
--- a/sysdeps/aarch64/configure.ac
+++ b/sysdeps/aarch64/configure.ac
@@ -20,3 +20,27 @@  if test $libc_cv_aarch64_be = yes; then
 else
   LIBC_CONFIG_VAR([default-abi], [lp64])
 fi
+
+AC_CACHE_CHECK([for pcs attribute support],
+               libc_cv_gcc_pcs_attribute, [dnl
+cat > conftest.c <<EOF
+__attribute__((aarch64_vector_pcs)) extern void foo (void);
+EOF
+libc_cv_gcc_pcs_attribute=no
+if ${CC-cc} -c -Wall -Werror conftest.c -o conftest.o 1>&AS_MESSAGE_LOG_FD \
+   2>&AS_MESSAGE_LOG_FD ; then
+  libc_cv_gcc_pcs_attribute=yes
+fi
+rm -f conftest*])
+
+# Enable libmvec by default.
+if test x"$build_mathvec" = xnotset; then
+  build_mathvec=yes
+fi
+
+# Only test libmvec if the compiler supports aarch64_vector_pcs.
+if test x"$build_mathvec" = xyes; then
+  if test $libc_cv_gcc_pcs_attribute = yes; then
+    LIBC_CONFIG_VAR([test-mathvec], [yes])
+  fi
+fi
diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile
index 4a182bd6d6..220b664323 100644
--- a/sysdeps/aarch64/fpu/Makefile
+++ b/sysdeps/aarch64/fpu/Makefile
@@ -12,3 +12,20 @@  CFLAGS-s_fmaxf.c += -ffinite-math-only
 CFLAGS-s_fmin.c += -ffinite-math-only
 CFLAGS-s_fminf.c += -ffinite-math-only
 endif
+
+ifeq ($(subdir),mathvec)
+libmvec-support += \
+  libmvec_double_vlen2_exp \
+  libmvec_float_vlen4_expf \
+
+endif
+
+ifeq ($(subdir),math)
+ifeq ($(build-mathvec),yes)
+double-vlen2-funcs = exp
+float-vlen4-funcs = exp
+ifeq ($(test-mathvec),yes)
+libmvec-tests += double-vlen2 float-vlen4
+endif
+endif
+endif
diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions
new file mode 100644
index 0000000000..da36f3c495
--- /dev/null
+++ b/sysdeps/aarch64/fpu/Versions
@@ -0,0 +1,5 @@ 
+libmvec {
+  GLIBC_2.30 {
+    _ZGVnN2v_exp; _ZGVnN4v_expf;
+  }
+}
diff --git a/sysdeps/aarch64/fpu/libmvec_double_vlen2.h b/sysdeps/aarch64/fpu/libmvec_double_vlen2.h
new file mode 100644
index 0000000000..383980d6ef
--- /dev/null
+++ b/sysdeps/aarch64/fpu/libmvec_double_vlen2.h
@@ -0,0 +1,59 @@ 
+/* Double-precision 2 element vector function template.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <sysdep.h>
+
+ENTRY (VECTOR_FUNCTION)
+	stp	x29, x30, [sp, -288]!
+	cfi_adjust_cfa_offset (288)
+	cfi_rel_offset (x29, 0)
+	cfi_rel_offset (x30, 8)
+	mov	x29, sp
+	stp	 q8,  q9, [sp, 16]
+	stp	q10, q11, [sp, 48]
+	stp	q12, q13, [sp, 80]
+	stp	q14, q15, [sp, 112]
+	stp	q16, q17, [sp, 144]
+	stp	q18, q19, [sp, 176]
+	stp	q20, q21, [sp, 208]
+	stp	q22, q23, [sp, 240]
+
+	// Use per lane load/store to avoid endianness issues.
+	str	q0, [sp, 272]
+	ldr	d0, [sp, 272]
+	bl SCALAR_FUNCTION
+	str	d0, [sp, 272]
+	ldr	d0, [sp, 280]
+	bl SCALAR_FUNCTION
+	str	d0, [sp, 280]
+	ldr	q0, [sp, 272]
+
+	ldp	q8, q9, [sp, 16]
+	ldp	q10, q11, [sp, 48]
+	ldp	q12, q13, [sp, 80]
+	ldp	q14, q15, [sp, 112]
+	ldp	q16, q17, [sp, 144]
+	ldp	q18, q19, [sp, 176]
+	ldp	q20, q21, [sp, 208]
+	ldp	q22, q23, [sp, 240]
+	ldp	x29, x30, [sp], 288
+	cfi_adjust_cfa_offset (288)
+	cfi_restore (x29)
+	cfi_restore (x30)
+	ret
+END (VECTOR_FUNCTION)
diff --git a/sysdeps/aarch64/fpu/libmvec_double_vlen2_exp.S b/sysdeps/aarch64/fpu/libmvec_double_vlen2_exp.S
new file mode 100644
index 0000000000..644405cc4f
--- /dev/null
+++ b/sysdeps/aarch64/fpu/libmvec_double_vlen2_exp.S
@@ -0,0 +1,21 @@ 
+/* Double-precision 2 element vector e^x function.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#define SCALAR_FUNCTION exp
+#define VECTOR_FUNCTION _ZGVnN2v_exp
+#include "libmvec_double_vlen2.h"
diff --git a/sysdeps/aarch64/fpu/libmvec_float_vlen4.h b/sysdeps/aarch64/fpu/libmvec_float_vlen4.h
new file mode 100644
index 0000000000..2450309c13
--- /dev/null
+++ b/sysdeps/aarch64/fpu/libmvec_float_vlen4.h
@@ -0,0 +1,65 @@ 
+/* Single-precision 4 element vector function template.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <sysdep.h>
+
+ENTRY (VECTOR_FUNCTION)
+	stp	x29, x30, [sp, -288]!
+	cfi_adjust_cfa_offset (288)
+	cfi_rel_offset (x29, 0)
+	cfi_rel_offset (x30, 8)
+	mov	x29, sp
+	stp	 q8,  q9, [sp, 16]
+	stp	q10, q11, [sp, 48]
+	stp	q12, q13, [sp, 80]
+	stp	q14, q15, [sp, 112]
+	stp	q16, q17, [sp, 144]
+	stp	q18, q19, [sp, 176]
+	stp	q20, q21, [sp, 208]
+	stp	q22, q23, [sp, 240]
+
+	// Use per lane load/store to avoid endianness issues.
+	str	q0, [sp, 272]
+	ldr	s0, [sp, 272]
+	bl SCALAR_FUNCTION
+	str	s0, [sp, 272]
+	ldr	s0, [sp, 276]
+	bl SCALAR_FUNCTION
+	str	s0, [sp, 276]
+	ldr	s0, [sp, 280]
+	bl SCALAR_FUNCTION
+	str	s0, [sp, 280]
+	ldr	s0, [sp, 284]
+	bl SCALAR_FUNCTION
+	str	s0, [sp, 284]
+	ldr	q0, [sp, 272]
+
+	ldp	q8, q9, [sp, 16]
+	ldp	q10, q11, [sp, 48]
+	ldp	q12, q13, [sp, 80]
+	ldp	q14, q15, [sp, 112]
+	ldp	q16, q17, [sp, 144]
+	ldp	q18, q19, [sp, 176]
+	ldp	q20, q21, [sp, 208]
+	ldp	q22, q23, [sp, 240]
+	ldp	x29, x30, [sp], 288
+	cfi_adjust_cfa_offset (288)
+	cfi_restore (x29)
+	cfi_restore (x30)
+	ret
+END (VECTOR_FUNCTION)
diff --git a/sysdeps/aarch64/fpu/libmvec_float_vlen4_expf.S b/sysdeps/aarch64/fpu/libmvec_float_vlen4_expf.S
new file mode 100644
index 0000000000..ab76ea0c77
--- /dev/null
+++ b/sysdeps/aarch64/fpu/libmvec_float_vlen4_expf.S
@@ -0,0 +1,21 @@ 
+/* Single-precision 4 element vector e^x function.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#define SCALAR_FUNCTION expf
+#define VECTOR_FUNCTION _ZGVnN4v_expf
+#include "libmvec_float_vlen4.h"
diff --git a/sysdeps/aarch64/fpu/test-double-vlen2-wrappers.c b/sysdeps/aarch64/fpu/test-double-vlen2-wrappers.c
new file mode 100644
index 0000000000..6c6c44d6b5
--- /dev/null
+++ b/sysdeps/aarch64/fpu/test-double-vlen2-wrappers.c
@@ -0,0 +1,28 @@ 
+/* Wrapper part of tests for aarch64 double vector math functions.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <arm_neon.h>
+#include "test-double-vlen2.h"
+
+#define VEC_TYPE float64x2_t
+
+/* Hack: VECTOR_WRAPPER declares the vector function without the pcs attribute,
+   placing it here happens to work, should be fixed in test-math-vector.h.  */
+__attribute__ ((aarch64_vector_pcs))
+
+VECTOR_WRAPPER (WRAPPER_NAME (exp), _ZGVnN2v_exp)
diff --git a/sysdeps/aarch64/fpu/test-float-vlen4-wrappers.c b/sysdeps/aarch64/fpu/test-float-vlen4-wrappers.c
new file mode 100644
index 0000000000..5117633f1f
--- /dev/null
+++ b/sysdeps/aarch64/fpu/test-float-vlen4-wrappers.c
@@ -0,0 +1,28 @@ 
+/* Wrapper part of tests for float aarch64 vector math functions.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <arm_neon.h>
+#include "test-float-vlen4.h"
+
+#define VEC_TYPE float32x4_t
+
+/* Hack: VECTOR_WRAPPER declares the vector function without the pcs attribute,
+   placing it here happens to work, should be fixed in test-math-vector.h.  */
+__attribute__ ((aarch64_vector_pcs))
+
+VECTOR_WRAPPER (WRAPPER_NAME (expf), _ZGVnN4v_expf)
diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps
index 585e5bbce7..1ed4af9e55 100644
--- a/sysdeps/aarch64/libm-test-ulps
+++ b/sysdeps/aarch64/libm-test-ulps
@@ -1601,6 +1601,12 @@  float: 1
 idouble: 1
 ifloat: 1
 
+Function: "exp_vlen2":
+double: 1
+
+Function: "exp_vlen4":
+float: 1
+
 Function: "expm1":
 double: 1
 float: 1
diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist
new file mode 100644
index 0000000000..9e178253f7
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist
@@ -0,0 +1,2 @@ 
+GLIBC_2.30 _ZGVnN2v_exp F
+GLIBC_2.30 _ZGVnN4v_expf F