diff mbox

Add ifunc memcpy and memmove for aarch64

Message ID 1485214395.4759.44.camel@caviumnetworks.com
State New, archived
Headers show

Commit Message

Steve Ellcey Jan. 23, 2017, 11:33 p.m. UTC
On Thu, 2017-01-19 at 17:41 -0200, Adhemerval Zanella wrote:

> I think to avoid potentially multiple kernel traps at loading or plt resolve time,
> a better solution would be issue the mrs instruction once at loader/program startup,
> fill in an internal structure with the required information and use it later on
> ifunc resolution.  This is similar the cpu-features/cacheinfo strategy for x86.
> 
> From last patch iteration [1] documentation, kernel provides the HWCAP_CPUID
> bit on hwcap to indication it supports the mrs emulation.  So using my previous
> suggestion I would recommend:
> 
>   1. Remove any configure check or restriction.
>   2. Add a cpu_features module similar to x86 that set a global state with
>      the cpu information obtained from kernel.  It will first check HWCAP_CPUID
>      bit on hwcap and if it is set then issue the mrs instruction.  It will
>      then populate the global state with the required cpu information.
>   3. Use the cpu information to select the correct ifunc.
> 
> It has another advantage of avoid more complexity with different glibc
> with different minimum required kernels.


Adhemerval,

I am looking at the cpu-features setup from x86 and trying to implement
that for aarch64 but there are some things I don't understand about the
code and I was hoping you (or someone else on the list) could help me.
I have attached the patch I have so far, this code doesn't contain any
use of the cpu features code but is just the code that tries to initialize
it on start up.  Right now it doesn't build and I am not sure what I am
missing.

Specifically I have these questions.

How is cpu-features-offsets.sym used and what do I need in this file?
I think this may be how _dl_aarch64_cpu_features is supposed to be
defined but I am not sure.

I obviously need something in init_cpu_features to check if mrs is
emulated in the kernel but I am not sure how to do that.  I know it
involves the HWCAPs but I am not sure how to access them, do I need a
sym file to get access to that too?  Something like
sysdeps/arm/rtld-global-offsets.sym?

Right now my build dies with:

<stdin>:2:102: error: implicit declaration of function ‘rtld_global_ro_offsetof’ [-Werror=implicit-function-declaration]
<stdin>:2:127: error: ‘_dl_aarch64_cpu_features’ undeclared (first use in this function)
<stdin>:2:127: note: each undeclared identifier is reported only once for each function it appears in
<stdin>:3:82: error: invalid application of ‘sizeof’ to incomplete type ‘struct cpu_features’
cc1: all warnings being treated as errors
../Makerules:266: recipe for target '/home/ubuntu/sellcey/glibc-ifunc-new/obj-glibc64/cpu-features-offsets.h' failed
make[2]: *** [/home/ubuntu/sellcey/glibc-ifunc-new/obj-glibc64/cpu-features-offsets.h] Error 1

Steve Ellcey
sellcey@caviumnetworks.com

Comments

Florian Weimer Jan. 24, 2017, 9:37 a.m. UTC | #1
On 01/24/2017 12:33 AM, Steve Ellcey wrote:
> How is cpu-features-offsets.sym used and what do I need in this file?
> I think this may be how _dl_aarch64_cpu_features is supposed to be
> defined but I am not sure.

It allows the assembler to use the values of C constant expressions. 
Commit 67aae64512cb42332f76a83e84ac2bc608ad4ad2 is an aarch64 example of 
its use.

Florian
Adhemerval Zanella Jan. 24, 2017, 2:09 p.m. UTC | #2
On 23/01/2017 21:33, Steve Ellcey wrote:
> On Thu, 2017-01-19 at 17:41 -0200, Adhemerval Zanella wrote:
>>  
>> I think to avoid potentially multiple kernel traps at loading or plt resolve time,
>> a better solution would be issue the mrs instruction once at loader/program startup,
>> fill in an internal structure with the required information and use it later on
>> ifunc resolution.  This is similar the cpu-features/cacheinfo strategy for x86.
>>
>> From last patch iteration [1] documentation, kernel provides the HWCAP_CPUID
>> bit on hwcap to indication it supports the mrs emulation.  So using my previous
>> suggestion I would recommend:
>>
>>   1. Remove any configure check or restriction.
>>   2. Add a cpu_features module similar to x86 that set a global state with
>>      the cpu information obtained from kernel.  It will first check HWCAP_CPUID
>>      bit on hwcap and if it is set then issue the mrs instruction.  It will
>>      then populate the global state with the required cpu information.
>>   3. Use the cpu information to select the correct ifunc.
>>
>> It has another advantage of avoid more complexity with different glibc
>> with different minimum required kernels.
> 
> 
> Adhemerval,
> 
> I am looking at the cpu-features setup from x86 and trying to implement
> that for aarch64 but there are some things I don't understand about the
> code and I was hoping you (or someone else on the list) could help me.
> I have attached the patch I have so far, this code doesn't contain any
> use of the cpu features code but is just the code that tries to initialize
> it on start up.  Right now it doesn't build and I am not sure what I am
> missing.
> 
> Specifically I have these questions.
> 
> How is cpu-features-offsets.sym used and what do I need in this file?
> I think this may be how _dl_aarch64_cpu_features is supposed to be
> defined but I am not sure.
> 

The .sym files are a trick glibc uses to basically define struct or tls
offsets so use on assembly implementations. x86 uses it because it
originally implemented most of ifunc resolvers directly in assembly
(back when compiler support was lacking).

Since you are implementing directly on C, these files are unnecessary.

> I obviously need something in init_cpu_features to check if mrs is
> emulated in the kernel but I am not sure how to do that.  I know it
> involves the HWCAPs but I am not sure how to access them, do I need a
> sym file to get access to that too?  Something like
> sysdeps/arm/rtld-global-offsets.sym?
> 
> Right now my build dies with:
> 
> <stdin>:2:102: error: implicit declaration of function ‘rtld_global_ro_offsetof’ [-Werror=implicit-function-declaration]
> <stdin>:2:127: error: ‘_dl_aarch64_cpu_features’ undeclared (first use in this function)
> <stdin>:2:127: note: each undeclared identifier is reported only once for each function it appears in
> <stdin>:3:82: error: invalid application of ‘sizeof’ to incomplete type ‘struct cpu_features’
> cc1: all warnings being treated as errors
> ../Makerules:266: recipe for target '/home/ubuntu/sellcey/glibc-ifunc-new/obj-glibc64/cpu-features-offsets.h' failed
> make[2]: *** [/home/ubuntu/sellcey/glibc-ifunc-new/obj-glibc64/cpu-features-offsets.h] Error 1
> 
> Steve Ellcey
> sellcey@caviumnetworks.com
> 

This branch in my personal repo [1] have a workable draft version for
aarch64.  It contains 2 patches, one that implements the cpu-features.c
for aarch64 and another one that actually uses it to implement the
thundex ifunc.

On the first patch I would like to remove the sysdeps/aarch64/ldsodefs.h and
make only Linux specific, because of hwcap. I will try to cleanup this up
later.

[1] https://github.com/zatrazz/glibc/tree/master-aarch64-ifunc
diff mbox

Patch

diff --git a/sysdeps/aarch64/Makefile b/sysdeps/aarch64/Makefile
index 562c137..e1d47fd 100644
--- a/sysdeps/aarch64/Makefile
+++ b/sysdeps/aarch64/Makefile
@@ -5,11 +5,13 @@  CFLAGS-backtrace.c += -funwind-tables
 endif
 
 ifeq ($(subdir),elf)
+sysdep-dl-routines += dl-get-cpu-features
 sysdep-dl-routines += tlsdesc dl-tlsdesc
 gen-as-const-headers += dl-link.sym
 endif
 
 ifeq ($(subdir),csu)
+gen-as-const-headers += cpu-features-offsets.sym
 gen-as-const-headers += tlsdesc.sym
 endif
 
diff --git a/sysdeps/aarch64/cpu-features-offsets.sym b/sysdeps/aarch64/cpu-features-offsets.sym
index e69de29..ad33818 100644
--- a/sysdeps/aarch64/cpu-features-offsets.sym
+++ b/sysdeps/aarch64/cpu-features-offsets.sym
@@ -0,0 +1,4 @@ 
+
+RTLD_GLOBAL_RO_DL_AARCH64_CPU_FEATURES_OFFSET rtld_global_ro_offsetof (_dl_aarch64_cpu_features)
+
+CPU_FEATURES_SIZE	sizeof (struct cpu_features)
diff --git a/sysdeps/aarch64/cpu-features.c b/sysdeps/aarch64/cpu-features.c
index e69de29..6c8f065 100644
--- a/sysdeps/aarch64/cpu-features.c
+++ b/sysdeps/aarch64/cpu-features.c
@@ -0,0 +1,30 @@ 
+/* Initialize CPU feature data.
+   This file is part of the GNU C Library.
+   Copyright (C) 2017 Free Software Foundation, Inc.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <stdint.h>
+#include <cpu-features.h>
+
+static inline void
+init_cpu_features (struct cpu_features *cpu_features)
+{
+  /* What do I do on a kernel that does not support mrs.  */
+
+  register uint64_t id = 0;
+  asm volatile ("mrs %0, midr_el1" : "=r"(id));
+  cpu_features->midr_el1 = id;
+}
diff --git a/sysdeps/aarch64/cpu-features.h b/sysdeps/aarch64/cpu-features.h
index e69de29..a2d0786 100644
--- a/sysdeps/aarch64/cpu-features.h
+++ b/sysdeps/aarch64/cpu-features.h
@@ -0,0 +1,48 @@ 
+/* This file is part of the GNU C Library.
+   Copyright (C) 2017 Free Software Foundation, Inc.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef cpu_features_h
+#define cpu_features_h
+
+#include <stdint.h>
+
+#define MIDR_PARTNUM_SHIFT	4
+#define MIDR_PARTNUM_MASK	(0xfff << MIDR_PARTNUM_SHIFT)
+#define MIDR_PARTNUM(midr)	\
+	(((midr) & MIDR_PARTNUM_MASK) >> MIDR_PARTNUM_SHIFT)
+#define MIDR_ARCHITECTURE_SHIFT	16
+#define MIDR_ARCHITECTURE_MASK	(0xf << MIDR_ARCHITECTURE_SHIFT)
+#define MIDR_ARCHITECTURE(midr)	\
+	(((midr) & MIDR_ARCHITECTURE_MASK) >> MIDR_ARCHITECTURE_SHIFT)
+#define MIDR_VARIANT_SHIFT	20
+#define MIDR_VARIANT_MASK	(0xf << MIDR_VARIANT_SHIFT)
+#define MIDR_VARIANT(midr)	\
+	(((midr) & MIDR_VARIANT_MASK) >> MIDR_VARIANT_SHIFT)
+#define MIDR_IMPLEMENTOR_SHIFT	24
+#define MIDR_IMPLEMENTOR_MASK	(0xff << MIDR_IMPLEMENTOR_SHIFT)
+#define MIDR_IMPLEMENTOR(midr)	\
+	(((midr) & MIDR_IMPLEMENTOR_MASK) >> MIDR_IMPLEMENTOR_SHIFT)
+
+#define IS_THUNDERX(midr) (MIDR_IMPLEMENTOR(midr) == 'C'	\
+			   && MIDR_PARTNUM(midr) == 0x0a1)
+
+struct cpu_features
+{
+  uint64_t midr_el1;
+};
+
+#endif /* cpu_features_h */
diff --git a/sysdeps/aarch64/dl-get-cpu-features.c b/sysdeps/aarch64/dl-get-cpu-features.c
index e69de29..1581c75 100644
--- a/sysdeps/aarch64/dl-get-cpu-features.c
+++ b/sysdeps/aarch64/dl-get-cpu-features.c
@@ -0,0 +1,26 @@ 
+/* This file is part of the GNU C Library.
+   Copyright (C) 2017 Free Software Foundation, Inc.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <ldsodefs.h>
+
+#undef __get_cpu_features
+
+const struct cpu_features *
+__get_cpu_features (void)
+{
+  return &GLRO(dl_aarch64_cpu_features);
+}
diff --git a/sysdeps/aarch64/dl-machine.h b/sysdeps/aarch64/dl-machine.h
index 84b8aec..4e34b13 100644
--- a/sysdeps/aarch64/dl-machine.h
+++ b/sysdeps/aarch64/dl-machine.h
@@ -25,6 +25,7 @@ 
 #include <tls.h>
 #include <dl-tlsdesc.h>
 #include <dl-irel.h>
+#include <cpu-features.c>
 
 /* Return nonzero iff ELF header is compatible with the running host.  */
 static inline int __attribute__ ((unused))
@@ -426,4 +427,20 @@  elf_machine_lazy_rel (struct link_map *map,
     _dl_reloc_bad_type (map, r_type, 1);
 }
 
+#define DL_PLATFORM_INIT dl_platform_init ()
+
+static inline void __attribute__ ((unused))
+dl_platform_init (void)
+{
+  if (GLRO(dl_platform) != NULL && *GLRO(dl_platform) == '\0')
+    /* Avoid an empty string which would disturb us.  */
+    GLRO(dl_platform) = NULL;
+
+#ifdef SHARED
+  /* init_cpu_features has been called early from __libc_start_main in
+     static executable.  */
+  init_cpu_features (&GLRO(dl_aarch64_cpu_features));
+#endif
+}
+
 #endif
diff --git a/sysdeps/aarch64/dl-procinfo.c b/sysdeps/aarch64/dl-procinfo.c
index e69de29..8d477d5 100644
--- a/sysdeps/aarch64/dl-procinfo.c
+++ b/sysdeps/aarch64/dl-procinfo.c
@@ -0,0 +1,57 @@ 
+/* Data for Aarch64 version of processor capability information.
+   Copyright (C) 2017 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+/* If anything should be added here check whether the size of each string
+   is still ok with the given array size.
+
+   All the #ifdefs in the definitions are quite irritating but
+   necessary if we want to avoid duplicating the information.  There
+   are three different modes:
+
+   - PROCINFO_DECL is defined.  This means we are only interested in
+     declarations.
+
+   - PROCINFO_DECL is not defined:
+
+     + if SHARED is defined the file is included in an array
+       initializer.  The .element = { ... } syntax is needed.
+
+     + if SHARED is not defined a normal array initialization is
+       needed.
+  */
+
+#ifndef PROCINFO_CLASS
+# define PROCINFO_CLASS
+#endif
+
+#if !defined PROCINFO_DECL && defined SHARED
+  ._dl_aarch64_cpu_features
+#else
+PROCINFO_CLASS struct cpu_features _dl_aarch64_cpu_features
+#endif
+#ifndef PROCINFO_DECL
+= { }
+#endif
+#if !defined SHARED || defined PROCINFO_DECL
+;
+#else
+,
+#endif
+
+#undef PROCINFO_DECL
+#undef PROCINFO_CLASS
diff --git a/sysdeps/aarch64/ldsodefs.h b/sysdeps/aarch64/ldsodefs.h
index f277074..ba4ada3 100644
--- a/sysdeps/aarch64/ldsodefs.h
+++ b/sysdeps/aarch64/ldsodefs.h
@@ -20,6 +20,7 @@ 
 #define _AARCH64_LDSODEFS_H 1
 
 #include <elf.h>
+#include <cpu-features.h>
 
 struct La_aarch64_regs;
 struct La_aarch64_retval;
diff --git a/sysdeps/aarch64/libc-start.c b/sysdeps/aarch64/libc-start.c
index e69de29..49d5f4a 100644
--- a/sysdeps/aarch64/libc-start.c
+++ b/sysdeps/aarch64/libc-start.c
@@ -0,0 +1,41 @@ 
+/* Copyright (C) 2017 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifdef SHARED
+# include <csu/libc-start.c>
+# else
+/* The main work is done in the generic function.  */
+# define LIBC_START_DISABLE_INLINE
+# define LIBC_START_MAIN generic_start_main
+# include <csu/libc-start.c>
+# include <cpu-features.h>
+# include <cpu-features.c>
+
+extern struct cpu_features _dl_aarch64_cpu_features;
+
+int
+__libc_start_main (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL),
+		   int argc, char **argv,
+		   __typeof (main) init,
+		   void (*fini) (void),
+		   void (*rtld_fini) (void), void *stack_end)
+{
+  init_cpu_features (&_dl_aarch64_cpu_features);
+  return generic_start_main (main, argc, argv, init, fini, rtld_fini,
+			     stack_end);
+}
+#endif