From patchwork Mon Aug 23 04:36:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fangrui Song X-Patchwork-Id: 44738 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 471A83857020 for ; Mon, 23 Aug 2021 04:37:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 471A83857020 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1629693449; bh=r6BIEu1L6n7cCs089av2Y5P8aWHvA2gk12yyTpF4Sbg=; h=Date:Subject:To:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=Q+yh79fJd3jjDdAtdrqIWVKsUc/m4Knr9/Q3/pho3qyJcVP2oq5BguYFlj4Xo1iKX FLteC09/BBZC9k3r3DXI+TWP2oaS/Xp8ocy4HjdoKcVROsQs8aVkDq3+2s91tWhSWS gpryjdpGuxn8Z+IXftRnIgLiyBQF49hCWpXQl8Sw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by sourceware.org (Postfix) with ESMTPS id A416C3858417 for ; Mon, 23 Aug 2021 04:36:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A416C3858417 Received: by mail-qt1-x84a.google.com with SMTP id v28-20020a05622a189cb029028e697f617dso7847608qtc.20 for ; Sun, 22 Aug 2021 21:36:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=r6BIEu1L6n7cCs089av2Y5P8aWHvA2gk12yyTpF4Sbg=; b=IXgC3lWx9oig7QH3N4Qy3sBqzrKZPmrDDIPQtxPZdHSWNUdIxMZyAV65YAdDf8Pf5D 3QzPY/u9aUSBGTu75IMznl65LUWg2/34FdBzM6PQ1tM1zABP1vTiJaLZEbiEV+vQHzo0 y/gaeiDTX/GqUHaQqMROe5r2FvqYsovz/DjUPM7B6/uH/2qCwMfmgQzA/ABb7DwkfTTp AQoiHZKzSsVPzcubmHXazXSDxVlF35A6Mcmhw9cSwM/80NsxczDv9XX8rQCL9qTZl5sV ggXEhvkrxxwsfAfVhprXwYJGSjnsPnktNOh1WQmDBEfGbA/n5mB0z/a/teFR87rrS2uH bo8w== X-Gm-Message-State: AOAM533mbJ/a/+hZPoNalw4ySJBv8Wh44jprgHc0UjrQTzCjd1EqS/U3 jNnazMlQerEkEtaDGG0b6RigOSkFjF8IMHdkfZDcwW9HYrT5LFuBPd6kqfXxQUoTBKCUaZc062w dCFCvoqn45ijdANlSsbLt/53gy5GSANYA9CvsaWZOdw8BLceYn6zwZ+wFP0nei75ZmfXW X-Google-Smtp-Source: ABdhPJyCNorl6MYkyk4D+/+ZBN7Ghzrs1b2Iab6w+R5nToEYkwoPWCPUYVZj4kb3niEzkOwTDG2Ctnv15rVw X-Received: from maskray1.svl.corp.google.com ([2620:15c:2ce:200:d371:40c2:7638:dd06]) (user=maskray job=sendgmr) by 2002:a05:6214:3018:: with SMTP id ke24mr29612386qvb.53.1629693415186; Sun, 22 Aug 2021 21:36:55 -0700 (PDT) Date: Sun, 22 Aug 2021 21:36:48 -0700 Message-Id: <20210823043648.2648608-1-maskray@google.com> Mime-Version: 1.0 Subject: [PATCH] elf: Avoid nested functions in the loader (all ports) [BZ #27220] To: libc-alpha@sourceware.org, "Carlos O'Donell" , Adhemerval Zanella X-Spam-Status: No, score=-18.8 required=5.0 tests=BAYES_00, DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_STOCKGEN, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Fangrui Song via Libc-alpha From: Fangrui Song Reply-To: Fangrui Song Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" [Alternative to https://sourceware.org/pipermail/libc-alpha/2021-August/130340.html This version fixes all ports and doesn't add NESTING dispatches.] dynamic-link.h is included more than once in some elf/ files (rtld.c, dl-conflict.c, dl-reloc.c, dl-reloc-static-pie.c) and uses GCC nested functions. This harms readability and the nested functions usage is the biggest obstacle prevents CC=clang (which doesn't support the feature). To un-nest elf_machine_rela, the key idea is to pass the variable in the containing scope as an extra argument. Stan Shebs implemented ppc64/x86-64 parts in the google/grte/v5-2.27/master branch. This patch squashes and cleans up the commits, and fixes all ports. Tested on aarch64-linux-gnu and x86_64-linux-gnu. On x86_64, ld.so size is slightly larger (0.25% .text size). The performance is unrelated because the very few extra instructions in RESOLVE_MAP are negligible compared with expensive symbol lookup costs. On aarch64, ld.so size is slightly smaller. The nested functions appear to need more .rodata and .eh_frame space. Tested build-many-glibcs.py with {alpha,arc,csky,hppa,ia64,microblaze,s390,s390x,sh,sparc64,sparcv9}-linux-gnu, arm-linux-gnueabi{,hf}, mips64{,el}-linux-gnu{,-n32,-n64}, and riscv64-linux-gnu-rv64imafdc-lp64d. --- elf/dl-conflict.c | 35 +++++++------- elf/dl-reloc-static-pie.c | 12 ++--- elf/dl-reloc.c | 66 +++++++++++++++----------- elf/do-rel.h | 12 ++--- elf/dynamic-link.h | 40 ++++++++-------- elf/get-dynamic-info.h | 5 ++ elf/rtld.c | 10 ++-- sysdeps/aarch64/dl-machine.h | 16 +++++-- sysdeps/alpha/dl-machine.h | 13 +++-- sysdeps/arc/dl-machine.h | 8 +++- sysdeps/arm/dl-machine.h | 22 +++++---- sysdeps/csky/dl-machine.h | 12 +++-- sysdeps/hppa/dl-machine.h | 10 ++-- sysdeps/i386/dl-machine.h | 22 ++++----- sysdeps/ia64/dl-machine.h | 16 ++++--- sysdeps/m68k/dl-machine.h | 12 +++-- sysdeps/microblaze/dl-machine.h | 12 +++-- sysdeps/mips/dl-machine.h | 45 +++++++++++------- sysdeps/nios2/dl-machine.h | 14 ++++-- sysdeps/powerpc/powerpc32/dl-machine.h | 12 +++-- sysdeps/powerpc/powerpc64/dl-machine.h | 16 ++++--- sysdeps/riscv/dl-machine.h | 10 ++-- sysdeps/s390/s390-32/dl-machine.h | 8 ++-- sysdeps/s390/s390-64/dl-machine.h | 8 ++-- sysdeps/sh/dl-machine.h | 12 +++-- sysdeps/sparc/sparc32/dl-machine.h | 12 +++-- sysdeps/sparc/sparc64/dl-machine.h | 12 +++-- sysdeps/x86_64/dl-machine.h | 16 +++++-- 28 files changed, 291 insertions(+), 197 deletions(-) diff --git a/elf/dl-conflict.c b/elf/dl-conflict.c index 31a2f90770..d7219b726a 100644 --- a/elf/dl-conflict.c +++ b/elf/dl-conflict.c @@ -27,17 +27,9 @@ #include #include "dynamic-link.h" -void -_dl_resolve_conflicts (struct link_map *l, ElfW(Rela) *conflict, - ElfW(Rela) *conflictend) -{ -#if ! ELF_MACHINE_NO_RELA - if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_RELOC)) - _dl_debug_printf ("\nconflict processing: %s\n", DSO_FILENAME (l->l_name)); - - { - /* Do the conflict relocation of the object and library GOT and other - data. */ +/* Used by RESOLVE_CONFLICT_FIND_MAP at init time. Cannot be accessed + * concurrently. */ +static struct link_map *resolve_conflict_map __attribute__ ((__unused__)); /* This macro is used as a callback from the ELF_DYNAMIC_RELOCATE code. */ #define RESOLVE_MAP(ref, version, flags) (*ref = NULL, NULL) @@ -51,12 +43,23 @@ _dl_resolve_conflicts (struct link_map *l, ElfW(Rela) *conflict, (map) = resolve_conflict_map; \ } while (0) +#include "dynamic-link.h" + +void +_dl_resolve_conflicts (struct link_map *l, ElfW(Rela) *conflict, + ElfW(Rela) *conflictend) +{ +#if ! ELF_MACHINE_NO_RELA + if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_RELOC)) + _dl_debug_printf ("\nconflict processing: %s\n", DSO_FILENAME (l->l_name)); + + { + /* Do the conflict relocation of the object and library GOT and other + data. */ + /* Prelinking makes no sense for anything but the main namespace. */ assert (l->l_ns == LM_ID_BASE); - struct link_map *resolve_conflict_map __attribute__ ((__unused__)) - = GL(dl_ns)[LM_ID_BASE]._ns_loaded; - -#include "dynamic-link.h" + resolve_conflict_map = GL(dl_ns)[LM_ID_BASE]._ns_loaded; /* Override these, defined in dynamic-link.h. */ #undef CHECK_STATIC_TLS @@ -68,7 +71,7 @@ _dl_resolve_conflicts (struct link_map *l, ElfW(Rela) *conflict, for (; conflict < conflictend; ++conflict) elf_machine_rela (l, conflict, NULL, NULL, (void *) conflict->r_offset, - 0); + 0, NULL); } #endif } diff --git a/elf/dl-reloc-static-pie.c b/elf/dl-reloc-static-pie.c index d5bd2f31e9..25d1c53aa2 100644 --- a/elf/dl-reloc-static-pie.c +++ b/elf/dl-reloc-static-pie.c @@ -23,6 +23,11 @@ #include #include "dynamic-link.h" +#define STATIC_PIE_BOOTSTRAP +#define BOOTSTRAP_MAP (main_map) +#define RESOLVE_MAP(sym, version, flags) BOOTSTRAP_MAP +#include "dynamic-link.h" + /* Relocate static executable with PIE. */ void @@ -30,11 +35,6 @@ _dl_relocate_static_pie (void) { struct link_map *main_map = _dl_get_dl_main_map (); -# define STATIC_PIE_BOOTSTRAP -# define BOOTSTRAP_MAP (main_map) -# define RESOLVE_MAP(sym, version, flags) BOOTSTRAP_MAP -# include "dynamic-link.h" - /* Figure out the run-time load address of static PIE. */ main_map->l_addr = elf_machine_load_address (); @@ -48,7 +48,7 @@ _dl_relocate_static_pie (void) /* Relocate ourselves so we can do normal function calls and data access using the global offset table. */ - ELF_DYNAMIC_RELOCATE (main_map, 0, 0, 0); + ELF_DYNAMIC_RELOCATE (main_map, 0, 0, 0, main_map); main_map->l_relocated = 1; /* Initialize _r_debug. */ diff --git a/elf/dl-reloc.c b/elf/dl-reloc.c index e13a672ade..35cab08683 100644 --- a/elf/dl-reloc.c +++ b/elf/dl-reloc.c @@ -162,6 +162,39 @@ _dl_nothread_init_static_tls (struct link_map *map) } #endif /* !THREAD_GSCOPE_IN_TCB */ +/* Used by RESOLVE_MAP. _dl_relocate_object is either called at init time or + * by dlopen with a global lock, so the variables cannot be accessed + * concurrently. */ +static struct link_map *cur_l; +static struct r_scope_elem **cur_scope; +static const char *cur_strtab; + +/* This macro is used as a callback from the ELF_DYNAMIC_RELOCATE code. */ +#define RESOLVE_MAP(ref, version, r_type) \ + ({ struct link_map *l = cur_l; \ + (ELFW(ST_BIND) ((*ref)->st_info) != STB_LOCAL \ + && __glibc_likely (!dl_symbol_visibility_binds_local_p (*ref))) \ + ? ((__builtin_expect ((*ref) == l->l_lookup_cache.sym, 0) \ + && elf_machine_type_class (r_type) == l->l_lookup_cache.type_class) \ + ? (bump_num_cache_relocations (), \ + (*ref) = l->l_lookup_cache.ret, \ + l->l_lookup_cache.value) \ + : ({ lookup_t _lr; \ + int _tc = elf_machine_type_class (r_type); \ + l->l_lookup_cache.type_class = _tc; \ + l->l_lookup_cache.sym = (*ref); \ + const struct r_found_version *v = NULL; \ + if ((version) != NULL && (version)->hash != 0) \ + v = (version); \ + _lr = _dl_lookup_symbol_x (cur_strtab + (*ref)->st_name, l, \ + (ref), cur_scope, v, _tc, \ + DL_LOOKUP_ADD_DEPENDENCY, NULL); \ + l->l_lookup_cache.ret = (*ref); \ + l->l_lookup_cache.value = _lr; })) \ + : l; }) + +#include "dynamic-link.h" + void _dl_relocate_object (struct link_map *l, struct r_scope_elem *scope[], int reloc_mode, int consider_profiling) @@ -243,36 +276,11 @@ _dl_relocate_object (struct link_map *l, struct r_scope_elem *scope[], { /* Do the actual relocation of the object's GOT and other data. */ - /* String table object symbols. */ - const char *strtab = (const void *) D_PTR (l, l_info[DT_STRTAB]); + cur_l = l; + cur_scope = scope; + cur_strtab = (const void *) D_PTR (cur_l, l_info[DT_STRTAB]); - /* This macro is used as a callback from the ELF_DYNAMIC_RELOCATE code. */ -#define RESOLVE_MAP(ref, version, r_type) \ - ((ELFW(ST_BIND) ((*ref)->st_info) != STB_LOCAL \ - && __glibc_likely (!dl_symbol_visibility_binds_local_p (*ref))) \ - ? ((__builtin_expect ((*ref) == l->l_lookup_cache.sym, 0) \ - && elf_machine_type_class (r_type) == l->l_lookup_cache.type_class) \ - ? (bump_num_cache_relocations (), \ - (*ref) = l->l_lookup_cache.ret, \ - l->l_lookup_cache.value) \ - : ({ lookup_t _lr; \ - int _tc = elf_machine_type_class (r_type); \ - l->l_lookup_cache.type_class = _tc; \ - l->l_lookup_cache.sym = (*ref); \ - const struct r_found_version *v = NULL; \ - if ((version) != NULL && (version)->hash != 0) \ - v = (version); \ - _lr = _dl_lookup_symbol_x (strtab + (*ref)->st_name, l, (ref), \ - scope, v, _tc, \ - DL_LOOKUP_ADD_DEPENDENCY \ - | DL_LOOKUP_FOR_RELOCATE, NULL); \ - l->l_lookup_cache.ret = (*ref); \ - l->l_lookup_cache.value = _lr; })) \ - : l) - -#include "dynamic-link.h" - - ELF_DYNAMIC_RELOCATE (l, lazy, consider_profiling, skip_ifunc); + ELF_DYNAMIC_RELOCATE (l, lazy, consider_profiling, skip_ifunc, NULL); #ifndef PROF if (__glibc_unlikely (consider_profiling) diff --git a/elf/do-rel.h b/elf/do-rel.h index 321ac2b359..19fc6c085e 100644 --- a/elf/do-rel.h +++ b/elf/do-rel.h @@ -37,11 +37,11 @@ relocations; they should be set up to call _dl_runtime_resolve, rather than fully resolved now. */ -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_dynamic_do_Rel (struct link_map *map, ElfW(Addr) reladdr, ElfW(Addr) relsize, __typeof (((ElfW(Dyn) *) 0)->d_un.d_val) nrelative, - int lazy, int skip_ifunc) + int lazy, int skip_ifunc, struct link_map *boot_map) { const ElfW(Rel) *r = (const void *) reladdr; const ElfW(Rel) *end = (const void *) (reladdr + relsize); @@ -136,7 +136,7 @@ elf_dynamic_do_Rel (struct link_map *map, ElfW(Half) ndx = version[ELFW(R_SYM) (r->r_info)] & 0x7fff; elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)], &map->l_versions[ndx], - (void *) (l_addr + r->r_offset), skip_ifunc); + (void *) (l_addr + r->r_offset), skip_ifunc, boot_map); } #if defined ELF_MACHINE_IRELATIVE && !defined RTLD_BOOTSTRAP @@ -150,7 +150,7 @@ elf_dynamic_do_Rel (struct link_map *map, &symtab[ELFW(R_SYM) (r2->r_info)], &map->l_versions[ndx], (void *) (l_addr + r2->r_offset), - skip_ifunc); + skip_ifunc, boot_map); } #endif } @@ -168,7 +168,7 @@ elf_dynamic_do_Rel (struct link_map *map, else # endif elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)], NULL, - (void *) (l_addr + r->r_offset), skip_ifunc); + (void *) (l_addr + r->r_offset), skip_ifunc, boot_map); # ifdef ELF_MACHINE_IRELATIVE if (r2 != NULL) @@ -176,7 +176,7 @@ elf_dynamic_do_Rel (struct link_map *map, if (ELFW(R_TYPE) (r2->r_info) == ELF_MACHINE_IRELATIVE) elf_machine_rel (map, r2, &symtab[ELFW(R_SYM) (r2->r_info)], NULL, (void *) (l_addr + r2->r_offset), - skip_ifunc); + skip_ifunc, boot_map); # endif } #endif diff --git a/elf/dynamic-link.h b/elf/dynamic-link.h index 3eb24ba3a6..6d71427675 100644 --- a/elf/dynamic-link.h +++ b/elf/dynamic-link.h @@ -59,30 +59,30 @@ int _dl_try_allocate_static_tls (struct link_map *map, bool optional) copying memory, breaking the very code written to handle the unaligned cases. */ # if ! ELF_MACHINE_NO_REL -auto inline void __attribute__((always_inline)) +static inline void __attribute__((always_inline)) elf_machine_rel (struct link_map *map, const ElfW(Rel) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr, int skip_ifunc); -auto inline void __attribute__((always_inline)) + void *const reloc_addr, int skip_ifunc, struct link_map *boot_map); +static inline void __attribute__((always_inline)) elf_machine_rel_relative (ElfW(Addr) l_addr, const ElfW(Rel) *reloc, void *const reloc_addr); # endif # if ! ELF_MACHINE_NO_RELA -auto inline void __attribute__((always_inline)) +static inline void __attribute__((always_inline)) elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr, int skip_ifunc); -auto inline void __attribute__((always_inline)) + void *const reloc_addr, int skip_ifunc, struct link_map *boot_map); +static inline void __attribute__((always_inline)) elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, void *const reloc_addr); # endif # if ELF_MACHINE_NO_RELA || defined ELF_MACHINE_PLT_REL -auto inline void __attribute__((always_inline)) +static inline void __attribute__((always_inline)) elf_machine_lazy_rel (struct link_map *map, ElfW(Addr) l_addr, const ElfW(Rel) *reloc, int skip_ifunc); # else -auto inline void __attribute__((always_inline)) +static inline void __attribute__((always_inline)) elf_machine_lazy_rel (struct link_map *map, ElfW(Addr) l_addr, const ElfW(Rela) *reloc, int skip_ifunc); @@ -114,7 +114,7 @@ elf_machine_lazy_rel (struct link_map *map, consumes precisely the very end of the DT_REL*, or DT_JMPREL and DT_REL* are completely separate and there is a gap between them. */ -# define _ELF_DYNAMIC_DO_RELOC(RELOC, reloc, map, do_lazy, skip_ifunc, test_rel) \ +# define _ELF_DYNAMIC_DO_RELOC(RELOC, reloc, map, do_lazy, skip_ifunc, test_rel, boot_map) \ do { \ struct { ElfW(Addr) start, size; \ __typeof (((ElfW(Dyn) *) 0)->d_un.d_val) nrelative; int lazy; } \ @@ -153,7 +153,7 @@ elf_machine_lazy_rel (struct link_map *map, \ if (ELF_DURING_STARTUP) \ elf_dynamic_do_##reloc ((map), ranges[0].start, ranges[0].size, \ - ranges[0].nrelative, 0, skip_ifunc); \ + ranges[0].nrelative, 0, skip_ifunc, boot_map); \ else \ { \ int ranges_index; \ @@ -163,7 +163,7 @@ elf_machine_lazy_rel (struct link_map *map, ranges[ranges_index].size, \ ranges[ranges_index].nrelative, \ ranges[ranges_index].lazy, \ - skip_ifunc); \ + skip_ifunc, boot_map); \ } \ } while (0) @@ -175,29 +175,29 @@ elf_machine_lazy_rel (struct link_map *map, # if ! ELF_MACHINE_NO_REL # include "do-rel.h" -# define ELF_DYNAMIC_DO_REL(map, lazy, skip_ifunc) \ - _ELF_DYNAMIC_DO_RELOC (REL, Rel, map, lazy, skip_ifunc, _ELF_CHECK_REL) +# define ELF_DYNAMIC_DO_REL(map, lazy, skip_ifunc, boot_map) \ + _ELF_DYNAMIC_DO_RELOC (REL, Rel, map, lazy, skip_ifunc, _ELF_CHECK_REL, boot_map) # else -# define ELF_DYNAMIC_DO_REL(map, lazy, skip_ifunc) /* Nothing to do. */ +# define ELF_DYNAMIC_DO_REL(map, lazy, skip_ifunc, boot_map) /* Nothing to do. */ # endif # if ! ELF_MACHINE_NO_RELA # define DO_RELA # include "do-rel.h" -# define ELF_DYNAMIC_DO_RELA(map, lazy, skip_ifunc) \ - _ELF_DYNAMIC_DO_RELOC (RELA, Rela, map, lazy, skip_ifunc, _ELF_CHECK_REL) +# define ELF_DYNAMIC_DO_RELA(map, lazy, skip_ifunc, boot_map) \ + _ELF_DYNAMIC_DO_RELOC (RELA, Rela, map, lazy, skip_ifunc, _ELF_CHECK_REL, boot_map) # else -# define ELF_DYNAMIC_DO_RELA(map, lazy, skip_ifunc) /* Nothing to do. */ +# define ELF_DYNAMIC_DO_RELA(map, lazy, skip_ifunc, boot_map) /* Nothing to do. */ # endif /* This can't just be an inline function because GCC is too dumb to inline functions containing inlines themselves. */ -# define ELF_DYNAMIC_RELOCATE(map, lazy, consider_profile, skip_ifunc) \ +# define ELF_DYNAMIC_RELOCATE(map, lazy, consider_profile, skip_ifunc, boot_map) \ do { \ int edr_lazy = elf_machine_runtime_setup ((map), (lazy), \ (consider_profile)); \ - ELF_DYNAMIC_DO_REL ((map), edr_lazy, skip_ifunc); \ - ELF_DYNAMIC_DO_RELA ((map), edr_lazy, skip_ifunc); \ + ELF_DYNAMIC_DO_REL ((map), edr_lazy, skip_ifunc, boot_map); \ + ELF_DYNAMIC_DO_RELA ((map), edr_lazy, skip_ifunc, boot_map); \ } while (0) #endif diff --git a/elf/get-dynamic-info.h b/elf/get-dynamic-info.h index d8ec32377d..5827ae6e24 100644 --- a/elf/get-dynamic-info.h +++ b/elf/get-dynamic-info.h @@ -19,6 +19,8 @@ /* This file is included multiple times and therefore lacks a header file inclusion guard. */ +#ifndef _GET_DYNAMIC_INFO_H + #include #include @@ -180,3 +182,6 @@ elf_get_dynamic_info (struct link_map *l, ElfW(Dyn) *temp) info[DT_RPATH] = NULL; #endif } + +#define _GET_DYNAMIC_INFO_H +#endif diff --git a/elf/rtld.c b/elf/rtld.c index 878e6480f4..c62efa7308 100644 --- a/elf/rtld.c +++ b/elf/rtld.c @@ -499,13 +499,9 @@ _dl_start_final (void *arg, struct dl_start_final_info *info) return start_addr; } -static ElfW(Addr) __attribute_used__ -_dl_start (void *arg) -{ #ifdef DONT_USE_BOOTSTRAP_MAP # define bootstrap_map GL(dl_rtld_map) #else - struct dl_start_final_info info; # define bootstrap_map info.l #endif @@ -518,9 +514,13 @@ _dl_start (void *arg) #define RESOLVE_MAP(sym, version, flags) BOOTSTRAP_MAP #include "dynamic-link.h" +static ElfW(Addr) __attribute_used__ +_dl_start (void *arg) +{ #ifdef DONT_USE_BOOTSTRAP_MAP rtld_timer_start (&start_time); #else + struct dl_start_final_info info; rtld_timer_start (&info.start_time); #endif @@ -561,7 +561,7 @@ _dl_start (void *arg) /* Relocate ourselves so we can do normal function calls and data access using the global offset table. */ - ELF_DYNAMIC_RELOCATE (&bootstrap_map, 0, 0, 0); + ELF_DYNAMIC_RELOCATE (&bootstrap_map, 0, 0, 0, &bootstrap_map); } bootstrap_map.l_relocated = 1; diff --git a/sysdeps/aarch64/dl-machine.h b/sysdeps/aarch64/dl-machine.h index 3e10cb462f..ab9e5c4d4b 100644 --- a/sysdeps/aarch64/dl-machine.h +++ b/sysdeps/aarch64/dl-machine.h @@ -237,11 +237,12 @@ elf_machine_plt_value (struct link_map *map, #ifdef RESOLVE_MAP -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, + struct link_map *boot_map) { ElfW(Addr) *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELFW (R_TYPE) (reloc->r_info); @@ -253,7 +254,11 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, else { const ElfW(Sym) *const refsym = sym; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif ElfW(Addr) value = SYMBOL_ADDRESS (sym_map, sym, true); if (sym != NULL @@ -377,7 +382,7 @@ elf_machine_rela_relative (ElfW(Addr) l_addr, *reloc_addr = l_addr + reloc->r_addend; } -inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, ElfW(Addr) l_addr, @@ -407,7 +412,7 @@ elf_machine_lazy_rel (struct link_map *map, version = &map->l_versions[vernum[symndx] & 0x7fff]; } elf_machine_rela (map, reloc, sym, version, reloc_addr, - skip_ifunc); + skip_ifunc, NULL); return; } } @@ -433,7 +438,8 @@ elf_machine_lazy_rel (struct link_map *map, /* Always initialize TLS descriptors completely, because lazy initialization requires synchronization at every TLS access. */ - elf_machine_rela (map, reloc, sym, version, reloc_addr, skip_ifunc); + elf_machine_rela (map, reloc, sym, version, reloc_addr, skip_ifunc, + NULL); } else if (__glibc_unlikely (r_type == AARCH64_R(IRELATIVE))) { diff --git a/sysdeps/alpha/dl-machine.h b/sysdeps/alpha/dl-machine.h index 2cd2213d9a..dc22baa68a 100644 --- a/sysdeps/alpha/dl-machine.h +++ b/sysdeps/alpha/dl-machine.h @@ -361,14 +361,15 @@ elf_machine_plt_value (struct link_map *map, const Elf64_Rela *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc, const Elf64_Sym *sym, const struct r_found_version *version, void *const reloc_addr_arg, - int skip_ifunc) + int skip_ifunc, + struct link_map *boot_map) { Elf64_Addr *const reloc_addr = reloc_addr_arg; unsigned long int const r_type = ELF64_R_TYPE (reloc->r_info); @@ -411,7 +412,11 @@ elf_machine_rela (struct link_map *map, return; else { +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif Elf64_Addr sym_value; Elf64_Addr sym_raw_value; @@ -489,7 +494,7 @@ elf_machine_rela (struct link_map *map, can be skipped. */ #define ELF_MACHINE_REL_RELATIVE 1 -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, void *const reloc_addr_arg) @@ -506,7 +511,7 @@ elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, memcpy (reloc_addr_arg, &reloc_addr_val, 8); } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf64_Addr l_addr, const Elf64_Rela *reloc, diff --git a/sysdeps/arc/dl-machine.h b/sysdeps/arc/dl-machine.h index e6ce7f0ff6..1054530321 100644 --- a/sysdeps/arc/dl-machine.h +++ b/sysdeps/arc/dl-machine.h @@ -228,11 +228,11 @@ elf_machine_fixup_plt (struct link_map *map, lookup_t t, #ifdef RESOLVE_MAP -inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { ElfW(Addr) r_info = reloc->r_info; const unsigned long int r_type = ELFW (R_TYPE) (r_info); @@ -245,7 +245,11 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, else { const ElfW(Sym) *const refsym = sym; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif ElfW(Addr) value = SYMBOL_ADDRESS (sym_map, sym, true); switch (r_type) diff --git a/sysdeps/arm/dl-machine.h b/sysdeps/arm/dl-machine.h index eb13cb8b57..84f84a1965 100644 --- a/sysdeps/arm/dl-machine.h +++ b/sysdeps/arm/dl-machine.h @@ -276,7 +276,7 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rel *reloc, #ifdef RESOLVE_MAP /* Handle a PC24 reloc, including the out-of-range case. */ -auto void +static void relocate_pc24 (struct link_map *map, Elf32_Addr value, Elf32_Addr *const reloc_addr, Elf32_Sword addend) { @@ -330,11 +330,11 @@ relocate_pc24 (struct link_map *map, Elf32_Addr value, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -364,7 +364,11 @@ elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc, #endif { const Elf32_Sym *const refsym = sym; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif Elf32_Addr value = SYMBOL_ADDRESS (sym_map, sym, true); if (sym != NULL @@ -508,11 +512,11 @@ elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc, } # ifndef RTLD_BOOTSTRAP -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -601,7 +605,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } # endif -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rel_relative (Elf32_Addr l_addr, const Elf32_Rel *reloc, void *const reloc_addr_arg) @@ -611,7 +615,7 @@ elf_machine_rel_relative (Elf32_Addr l_addr, const Elf32_Rel *reloc, } # ifndef RTLD_BOOTSTRAP -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) @@ -621,7 +625,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, } # endif -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rel *reloc, @@ -653,7 +657,7 @@ elf_machine_lazy_rel (struct link_map *map, /* Always initialize TLS descriptors completely, because lazy initialization requires synchronization at every TLS access. */ - elf_machine_rel (map, reloc, sym, version, reloc_addr, skip_ifunc); + elf_machine_rel (map, reloc, sym, version, reloc_addr, skip_ifunc, NULL); } else _dl_reloc_bad_type (map, r_type, 1); diff --git a/sysdeps/csky/dl-machine.h b/sysdeps/csky/dl-machine.h index b08f06d74c..8702fe9917 100644 --- a/sysdeps/csky/dl-machine.h +++ b/sysdeps/csky/dl-machine.h @@ -215,10 +215,10 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rela *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void __attribute__ ((unused, always_inline)) +static inline void __attribute__ ((unused, always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -230,7 +230,11 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, else { const Elf32_Sym *const refsym = sym; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif ElfW(Addr) value = SYMBOL_ADDRESS (sym_map, sym, true); opcode16_addr = (unsigned short *)reloc_addr; @@ -331,7 +335,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } } -auto inline void __attribute__ ((unused, always_inline)) +static inline void __attribute__ ((unused, always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) { @@ -339,7 +343,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, *reloc_addr = l_addr + reloc->r_addend; } -auto inline void __attribute__ ((unused, always_inline)) +static inline void __attribute__ ((unused, always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, int skip_ifunc) diff --git a/sysdeps/hppa/dl-machine.h b/sysdeps/hppa/dl-machine.h index f048fd2072..4aa159cfbd 100644 --- a/sysdeps/hppa/dl-machine.h +++ b/sysdeps/hppa/dl-machine.h @@ -549,13 +549,13 @@ dl_platform_init (void) ( (((as14) & 0x1fff) << 1) \ | (((as14) & 0x2000) >> 13)) -auto void __attribute__((always_inline)) +static void __attribute__((always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, void *const reloc_addr_arg, - int skip_ifunc) + int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const Elf32_Sym *const refsym = sym; @@ -581,7 +581,7 @@ elf_machine_rela (struct link_map *map, # ifdef RTLD_BOOTSTRAP /* RESOLVE_MAP in rtld.c doesn't have the local sym test. */ sym_map = (ELF32_ST_BIND (sym->st_info) != STB_LOCAL - ? RESOLVE_MAP (&sym, version, r_type) : map); + ? boot_map : map); # else sym_map = RESOLVE_MAP (&sym, version, r_type); # endif @@ -741,7 +741,7 @@ elf_machine_rela (struct link_map *map, /* hppa doesn't have an R_PARISC_RELATIVE reloc, but uses relocs with ELF32_R_SYM (info) == 0 for a similar purpose. */ -auto void __attribute__((always_inline)) +static void __attribute__((always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) @@ -794,7 +794,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, *reloc_addr = value; } -auto void __attribute__((always_inline)) +static void __attribute__((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, int skip_ifunc) diff --git a/sysdeps/i386/dl-machine.h b/sysdeps/i386/dl-machine.h index 590b41d8d7..9028e53ff1 100644 --- a/sysdeps/i386/dl-machine.h +++ b/sysdeps/i386/dl-machine.h @@ -291,11 +291,11 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rel *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute ((always_inline)) elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -498,11 +498,11 @@ and creates an unsatisfiable circular dependency.\n", } # ifndef RTLD_BOOTSTRAP -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -647,7 +647,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } # endif /* !RTLD_BOOTSTRAP */ -auto inline void +static inline void __attribute ((always_inline)) elf_machine_rel_relative (Elf32_Addr l_addr, const Elf32_Rel *reloc, void *const reloc_addr_arg) @@ -658,7 +658,7 @@ elf_machine_rel_relative (Elf32_Addr l_addr, const Elf32_Rel *reloc, } # ifndef RTLD_BOOTSTRAP -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) @@ -668,7 +668,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, } # endif /* !RTLD_BOOTSTRAP */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rel *reloc, @@ -707,12 +707,12 @@ elf_machine_lazy_rel (struct link_map *map, ElfW(Half) ndx = version[ELFW(R_SYM) (r->r_info)] & 0x7fff; elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)], &map->l_versions[ndx], - (void *) (l_addr + r->r_offset), skip_ifunc); + (void *) (l_addr + r->r_offset), skip_ifunc, NULL); } # ifndef RTLD_BOOTSTRAP else elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)], NULL, - (void *) (l_addr + r->r_offset), skip_ifunc); + (void *) (l_addr + r->r_offset), skip_ifunc, NULL); # endif } else if (__glibc_unlikely (r_type == R_386_IRELATIVE)) @@ -728,7 +728,7 @@ elf_machine_lazy_rel (struct link_map *map, # ifndef RTLD_BOOTSTRAP -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rela (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, @@ -754,7 +754,7 @@ elf_machine_lazy_rela (struct link_map *map, /* Always initialize TLS descriptors completely at load time, in case static TLS is allocated for it that requires locking. */ - elf_machine_rela (map, reloc, sym, version, reloc_addr, skip_ifunc); + elf_machine_rela (map, reloc, sym, version, reloc_addr, skip_ifunc, NULL); } else if (__glibc_unlikely (r_type == R_386_IRELATIVE)) { diff --git a/sysdeps/ia64/dl-machine.h b/sysdeps/ia64/dl-machine.h index 4403e7767a..8390148134 100644 --- a/sysdeps/ia64/dl-machine.h +++ b/sysdeps/ia64/dl-machine.h @@ -371,14 +371,14 @@ elf_machine_plt_value (struct link_map *map, const Elf64_Rela *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute ((always_inline)) elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc, const Elf64_Sym *sym, const struct r_found_version *version, void *const reloc_addr_arg, - int skip_ifunc) + int skip_ifunc, struct link_map *boot_map) { Elf64_Addr *const reloc_addr = reloc_addr_arg; const unsigned long int r_type = ELF64_R_TYPE (reloc->r_info); @@ -414,10 +414,14 @@ elf_machine_rela (struct link_map *map, return; else { - struct link_map *sym_map; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else + struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif /* RESOLVE_MAP() will return NULL if it fail to locate the symbol. */ - if ((sym_map = RESOLVE_MAP (&sym, version, r_type))) + if (sym_map != NULL) { value = SYMBOL_ADDRESS (sym_map, sym, true) + reloc->r_addend; @@ -476,7 +480,7 @@ elf_machine_rela (struct link_map *map, can be skipped. */ #define ELF_MACHINE_REL_RELATIVE 1 -auto inline void +static inline void __attribute ((always_inline)) elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, void *const reloc_addr_arg) @@ -489,7 +493,7 @@ elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, } /* Perform a RELATIVE reloc on the .got entry that transfers to the .plt. */ -auto inline void +static inline void __attribute ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf64_Addr l_addr, const Elf64_Rela *reloc, diff --git a/sysdeps/m68k/dl-machine.h b/sysdeps/m68k/dl-machine.h index 86a8c67e2a..9c7657c636 100644 --- a/sysdeps/m68k/dl-machine.h +++ b/sysdeps/m68k/dl-machine.h @@ -215,10 +215,10 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rela *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void __attribute__ ((unused, always_inline)) +static inline void __attribute__ ((unused, always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -228,7 +228,11 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, else { const Elf32_Sym *const refsym = sym; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif Elf32_Addr value = SYMBOL_ADDRESS (sym_map, sym, true); switch (r_type) @@ -303,7 +307,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } } -auto inline void __attribute__ ((unused, always_inline)) +static inline void __attribute__ ((unused, always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) { @@ -311,7 +315,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, *reloc_addr = l_addr + reloc->r_addend; } -auto inline void __attribute__ ((unused, always_inline)) +static inline void __attribute__ ((unused, always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, int skip_ifunc) diff --git a/sysdeps/microblaze/dl-machine.h b/sysdeps/microblaze/dl-machine.h index e460f6f195..3e54676afe 100644 --- a/sysdeps/microblaze/dl-machine.h +++ b/sysdeps/microblaze/dl-machine.h @@ -207,10 +207,10 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rela *reloc, ((unsigned short *)(rel_addr))[3] = (val) & 0xffff; \ } while (0) -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const int r_type = ELF32_R_TYPE (reloc->r_info); @@ -222,7 +222,11 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, else { const Elf32_Sym *const refsym = sym; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif Elf32_Addr value = SYMBOL_ADDRESS (sym_map, sym, true); value += reloc->r_addend; @@ -277,7 +281,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } } -auto inline void +static inline void elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) { @@ -285,7 +289,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, PUT_REL_64 (reloc_addr, l_addr + reloc->r_addend); } -auto inline void +static inline void elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, int skip_ifunc) diff --git a/sysdeps/mips/dl-machine.h b/sysdeps/mips/dl-machine.h index d9c6d33d0c..2293ea0b3e 100644 --- a/sysdeps/mips/dl-machine.h +++ b/sysdeps/mips/dl-machine.h @@ -475,11 +475,12 @@ elf_machine_plt_value (struct link_map *map, const ElfW(Rel) *reloc, by RELOC_ADDR. SYM is the relocation symbol specified by R_INFO and MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_reloc (struct link_map *map, ElfW(Addr) r_info, const ElfW(Sym) *sym, const struct r_found_version *version, - void *reloc_addr, ElfW(Addr) r_addend, int inplace_p) + void *reloc_addr, ElfW(Addr) r_addend, int inplace_p, + struct link_map *boot_map) { const unsigned long int r_type = ELFW(R_TYPE) (r_info); ElfW(Addr) *addr_field = (ElfW(Addr) *) reloc_addr; @@ -507,7 +508,11 @@ elf_machine_reloc (struct link_map *map, ElfW(Addr) r_info, case R_MIPS_TLS_TPREL32: # endif { - struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else + struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif switch (r_type) { @@ -647,7 +652,11 @@ elf_machine_reloc (struct link_map *map, ElfW(Addr) r_info, _dl_signal_error (0, map->l_name, NULL, "found jump slot relocation with non-zero addend"); - sym_map = RESOLVE_MAP (&sym, version, r_type); +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + sym_map = boot_map; +#else + sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif value = SYMBOL_ADDRESS (sym_map, sym, true); *addr_field = value; @@ -661,7 +670,11 @@ elf_machine_reloc (struct link_map *map, ElfW(Addr) r_info, ElfW(Addr) value; /* Calculate the address of the symbol. */ - sym_map = RESOLVE_MAP (&sym, version, r_type); +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + sym_map = boot_map; +#else + sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif value = SYMBOL_ADDRESS (sym_map, sym, true); if (__builtin_expect (sym == NULL, 0)) @@ -708,16 +721,16 @@ elf_machine_reloc (struct link_map *map, ElfW(Addr) r_info, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rel (struct link_map *map, const ElfW(Rel) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr, int skip_ifunc) + void *const reloc_addr, int skip_ifunc, struct link_map *boot_map) { - elf_machine_reloc (map, reloc->r_info, sym, version, reloc_addr, 0, 1); + elf_machine_reloc (map, reloc->r_info, sym, version, reloc_addr, 0, 1, boot_map); } -auto inline void +static inline void __attribute__((always_inline)) elf_machine_rel_relative (ElfW(Addr) l_addr, const ElfW(Rel) *reloc, void *const reloc_addr) @@ -725,7 +738,7 @@ elf_machine_rel_relative (ElfW(Addr) l_addr, const ElfW(Rel) *reloc, /* XXX Nothing to do. There is no relative relocation, right? */ } -auto inline void +static inline void __attribute__((always_inline)) elf_machine_lazy_rel (struct link_map *map, ElfW(Addr) l_addr, const ElfW(Rel) *reloc, @@ -748,17 +761,17 @@ elf_machine_lazy_rel (struct link_map *map, _dl_reloc_bad_type (map, r_type, 1); } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr, int skip_ifunc) + void *const reloc_addr, int skip_ifunc, struct link_map *boot_map) { elf_machine_reloc (map, reloc->r_info, sym, version, reloc_addr, - reloc->r_addend, 0); + reloc->r_addend, 0, boot_map); } -auto inline void +static inline void __attribute__((always_inline)) elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, void *const reloc_addr) @@ -767,7 +780,7 @@ elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, #ifndef RTLD_BOOTSTRAP /* Relocate GOT. */ -auto inline void +static inline void __attribute__((always_inline)) elf_machine_got_rel (struct link_map *map, int lazy) { @@ -868,7 +881,7 @@ elf_machine_got_rel (struct link_map *map, int lazy) /* Set up the loaded object described by L so its stub function will jump to the on-demand fixup code __dl_runtime_resolve. */ -auto inline int +static inline int __attribute__((always_inline)) elf_machine_runtime_setup (struct link_map *l, int lazy, int profile) { diff --git a/sysdeps/nios2/dl-machine.h b/sysdeps/nios2/dl-machine.h index e000cd081f..cd03ea93bd 100644 --- a/sysdeps/nios2/dl-machine.h +++ b/sysdeps/nios2/dl-machine.h @@ -234,10 +234,10 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rela *reloc, LOADADDR is the load address of the object; INFO is an array indexed by DT_* of the .dynamic section info. */ -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -249,7 +249,11 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, else { const Elf32_Sym *const refsym = sym; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif Elf32_Addr value = SYMBOL_ADDRESS (sym_map, sym, true); switch (r_type) @@ -314,7 +318,7 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, } } -auto inline void __attribute__((always_inline)) +static inline void __attribute__((always_inline)) elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, void *const reloc_addr_arg) { @@ -322,10 +326,10 @@ elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, *reloc_addr = l_addr + reloc->r_addend; } -auto inline void __attribute__((always_inline)) +static inline void __attribute__((always_inline)) elf_machine_lazy_rel (struct link_map *map, ElfW(Addr) l_addr, const ElfW(Rela) *reloc, - int skip_ifunc) + int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = (void *) (l_addr + reloc->r_offset); if (ELF32_R_TYPE (reloc->r_info) == R_NIOS2_JUMP_SLOT) diff --git a/sysdeps/powerpc/powerpc32/dl-machine.h b/sysdeps/powerpc/powerpc32/dl-machine.h index ced3a7b659..eaf3a1202f 100644 --- a/sysdeps/powerpc/powerpc32/dl-machine.h +++ b/sysdeps/powerpc/powerpc32/dl-machine.h @@ -286,10 +286,10 @@ extern void _dl_reloc_overflow (struct link_map *map, LOADADDR is the load address of the object; INFO is an array indexed by DT_* of the .dynamic section info. */ -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const Elf32_Sym *const refsym = sym; @@ -317,7 +317,11 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } else { +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + sym_map = boot_map; +#else sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif value = SYMBOL_ADDRESS (sym_map, sym, true); } value += reloc->r_addend; @@ -441,7 +445,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } } -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) { @@ -449,7 +453,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, *reloc_addr = l_addr + reloc->r_addend; } -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, int skip_ifunc) diff --git a/sysdeps/powerpc/powerpc64/dl-machine.h b/sysdeps/powerpc/powerpc64/dl-machine.h index b90f407119..b88d9d7912 100644 --- a/sysdeps/powerpc/powerpc64/dl-machine.h +++ b/sysdeps/powerpc/powerpc64/dl-machine.h @@ -620,7 +620,7 @@ extern void attribute_hidden _dl_reloc_overflow (struct link_map *map, Elf64_Addr *const reloc_addr, const Elf64_Sym *refsym); -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, void *const reloc_addr_arg) { @@ -629,7 +629,7 @@ elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, } /* This computes the value used by TPREL* relocs. */ -auto inline Elf64_Addr __attribute__ ((always_inline, const)) +static inline Elf64_Addr __attribute__ ((always_inline, const)) elf_machine_tprel (struct link_map *map, struct link_map *sym_map, const Elf64_Sym *sym, @@ -648,7 +648,7 @@ elf_machine_tprel (struct link_map *map, } /* Call function at address VALUE (an OPD entry) to resolve ifunc relocs. */ -auto inline Elf64_Addr __attribute__ ((always_inline)) +static inline Elf64_Addr __attribute__ ((always_inline)) resolve_ifunc (Elf64_Addr value, const struct link_map *map, const struct link_map *sym_map) { @@ -678,13 +678,13 @@ resolve_ifunc (Elf64_Addr value, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc, const Elf64_Sym *sym, const struct r_found_version *version, void *const reloc_addr_arg, - int skip_ifunc) + int skip_ifunc, struct link_map *boot_map) { Elf64_Addr *const reloc_addr = reloc_addr_arg; const int r_type = ELF64_R_TYPE (reloc->r_info); @@ -707,7 +707,11 @@ elf_machine_rela (struct link_map *map, /* We need SYM_MAP even in the absence of TLS, for elf_machine_fixup_plt and STT_GNU_IFUNC. */ +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif Elf64_Addr value = SYMBOL_ADDRESS (sym_map, sym, true) + reloc->r_addend; if (sym != NULL @@ -1037,7 +1041,7 @@ elf_machine_rela (struct link_map *map, MODIFIED_CODE_NOQUEUE (reloc_addr); } -auto inline void __attribute__ ((always_inline)) +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf64_Addr l_addr, const Elf64_Rela *reloc, int skip_ifunc) diff --git a/sysdeps/riscv/dl-machine.h b/sysdeps/riscv/dl-machine.h index 5b0746175c..8e759ac8d3 100644 --- a/sysdeps/riscv/dl-machine.h +++ b/sysdeps/riscv/dl-machine.h @@ -161,11 +161,11 @@ elf_machine_fixup_plt (struct link_map *map, lookup_t t, by RELOC_ADDR. SYM is the relocation symbol specified by R_INFO and MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr, int skip_ifunc) + void *const reloc_addr, int skip_ifunc, struct link_map *boot_map) { ElfW(Addr) r_info = reloc->r_info; const unsigned long int r_type = ELFW (R_TYPE) (r_info); @@ -279,7 +279,7 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, } } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, void *const reloc_addr) @@ -287,7 +287,7 @@ elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, *(ElfW(Addr) *) reloc_addr = l_addr + reloc->r_addend; } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, ElfW(Addr) l_addr, const ElfW(Rela) *reloc, int skip_ifunc) @@ -320,7 +320,7 @@ elf_machine_lazy_rel (struct link_map *map, ElfW(Addr) l_addr, /* Set up the loaded object described by L so its stub function will jump to the on-demand fixup code __dl_runtime_resolve. */ -auto inline int +static inline int __attribute__ ((always_inline)) elf_machine_runtime_setup (struct link_map *l, int lazy, int profile) { diff --git a/sysdeps/s390/s390-32/dl-machine.h b/sysdeps/s390/s390-32/dl-machine.h index d0ccd69261..712367fa16 100644 --- a/sysdeps/s390/s390-32/dl-machine.h +++ b/sysdeps/s390/s390-32/dl-machine.h @@ -321,11 +321,11 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rela *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -484,7 +484,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) @@ -493,7 +493,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, *reloc_addr = l_addr + reloc->r_addend; } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, diff --git a/sysdeps/s390/s390-64/dl-machine.h b/sysdeps/s390/s390-64/dl-machine.h index 543361c836..77a658e17f 100644 --- a/sysdeps/s390/s390-64/dl-machine.h +++ b/sysdeps/s390/s390-64/dl-machine.h @@ -268,11 +268,11 @@ elf_machine_plt_value (struct link_map *map, const Elf64_Rela *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc, const Elf64_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf64_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF64_R_TYPE (reloc->r_info); @@ -438,7 +438,7 @@ elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc, } } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, void *const reloc_addr_arg) @@ -447,7 +447,7 @@ elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, *reloc_addr = l_addr + reloc->r_addend; } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf64_Addr l_addr, const Elf64_Rela *reloc, diff --git a/sysdeps/sh/dl-machine.h b/sysdeps/sh/dl-machine.h index 122b417a17..9c3c0ba099 100644 --- a/sysdeps/sh/dl-machine.h +++ b/sysdeps/sh/dl-machine.h @@ -259,11 +259,11 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rela *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute ((always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; const unsigned int r_type = ELF32_R_TYPE (reloc->r_info); @@ -318,7 +318,11 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, else { const Elf32_Sym *const refsym = sym; +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif value = SYMBOL_ADDRESS (sym_map, sym, true); value += reloc->r_addend; @@ -424,7 +428,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) @@ -443,7 +447,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, #undef COPY_UNALIGNED_WORD } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, diff --git a/sysdeps/sparc/sparc32/dl-machine.h b/sysdeps/sparc/sparc32/dl-machine.h index 0269e458ea..20f60b2a3b 100644 --- a/sysdeps/sparc/sparc32/dl-machine.h +++ b/sysdeps/sparc/sparc32/dl-machine.h @@ -327,11 +327,11 @@ elf_machine_plt_value (struct link_map *map, const Elf32_Rela *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, const Elf32_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf32_Addr *const reloc_addr = reloc_addr_arg; #if !defined RTLD_BOOTSTRAP && !defined RESOLVE_CONFLICT_FIND_MAP @@ -381,7 +381,11 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } else { +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + sym_map = boot_map; +#else sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif value = SYMBOL_ADDRESS (sym_map, sym, true); } #else @@ -536,7 +540,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc, } } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, void *const reloc_addr_arg) @@ -545,7 +549,7 @@ elf_machine_rela_relative (Elf32_Addr l_addr, const Elf32_Rela *reloc, *reloc_addr += l_addr + reloc->r_addend; } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf32_Addr l_addr, const Elf32_Rela *reloc, diff --git a/sysdeps/sparc/sparc64/dl-machine.h b/sysdeps/sparc/sparc64/dl-machine.h index bbd4566d8a..4d47b3a150 100644 --- a/sysdeps/sparc/sparc64/dl-machine.h +++ b/sysdeps/sparc/sparc64/dl-machine.h @@ -354,11 +354,11 @@ elf_machine_runtime_setup (struct link_map *l, int lazy, int profile) /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc, const Elf64_Sym *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, struct link_map *boot_map) { Elf64_Addr *const reloc_addr = reloc_addr_arg; #if !defined RTLD_BOOTSTRAP && !defined RESOLVE_CONFLICT_FIND_MAP @@ -408,7 +408,11 @@ elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc, } else { +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + sym_map = boot_map; +#else sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif value = SYMBOL_ADDRESS (sym_map, sym, true); } #else @@ -646,7 +650,7 @@ elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc, } } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, void *const reloc_addr_arg) @@ -655,7 +659,7 @@ elf_machine_rela_relative (Elf64_Addr l_addr, const Elf64_Rela *reloc, *reloc_addr = l_addr + reloc->r_addend; } -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_lazy_rel (struct link_map *map, Elf64_Addr l_addr, const Elf64_Rela *reloc, diff --git a/sysdeps/x86_64/dl-machine.h b/sysdeps/x86_64/dl-machine.h index ceee50734e..738663ec5d 100644 --- a/sysdeps/x86_64/dl-machine.h +++ b/sysdeps/x86_64/dl-machine.h @@ -251,11 +251,12 @@ elf_machine_plt_value (struct link_map *map, const ElfW(Rela) *reloc, /* Perform the relocation specified by RELOC and SYM (which is fully resolved). MAP is the object containing the reloc. */ -auto inline void +static inline void __attribute__ ((always_inline)) elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, const ElfW(Sym) *sym, const struct r_found_version *version, - void *const reloc_addr_arg, int skip_ifunc) + void *const reloc_addr_arg, int skip_ifunc, + struct link_map *boot_map) { ElfW(Addr) *const reloc_addr = reloc_addr_arg; const unsigned long int r_type = ELFW(R_TYPE) (reloc->r_info); @@ -293,7 +294,11 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc, # ifndef RTLD_BOOTSTRAP const ElfW(Sym) *const refsym = sym; # endif +#if defined RTLD_BOOTSTRAP || defined STATIC_PIE_BOOTSTRAP + struct link_map *sym_map = boot_map; +#else struct link_map *sym_map = RESOLVE_MAP (&sym, version, r_type); +#endif ElfW(Addr) value = SYMBOL_ADDRESS (sym_map, sym, true); if (sym != NULL @@ -518,7 +523,7 @@ and creates an unsatisfiable circular dependency.\n", } } -auto inline void +static inline void __attribute ((always_inline)) elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, void *const reloc_addr_arg) @@ -537,7 +542,7 @@ elf_machine_rela_relative (ElfW(Addr) l_addr, const ElfW(Rela) *reloc, } } -auto inline void +static inline void __attribute ((always_inline)) elf_machine_lazy_rel (struct link_map *map, ElfW(Addr) l_addr, const ElfW(Rela) *reloc, @@ -573,7 +578,8 @@ elf_machine_lazy_rel (struct link_map *map, /* Always initialize TLS descriptors completely at load time, in case static TLS is allocated for it that requires locking. */ - elf_machine_rela (map, reloc, sym, version, reloc_addr, skip_ifunc); + elf_machine_rela (map, reloc, sym, version, reloc_addr, skip_ifunc, + NULL); } else if (__glibc_unlikely (r_type == R_X86_64_IRELATIVE)) {