From patchwork Fri Oct 6 14:24:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 23379 Received: (qmail 22699 invoked by alias); 6 Oct 2017 14:24:26 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 22624 invoked by uid 89); 6 Oct 2017 14:24:20 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-24.8 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 spammy=ndx X-HELO: EUR02-VE1-obe.outbound.protection.outlook.com Message-ID: <59D7920A.6050205@arm.com> Date: Fri, 06 Oct 2017 15:24:10 +0100 From: Szabolcs Nagy User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0 MIME-Version: 1.0 To: GNU C Library CC: nd@arm.com Subject: [PATCH 2/2] aarch64: Remove barriers from TLS descriptor functions References: <59D791A6.10507@arm.com> In-Reply-To: <59D791A6.10507@arm.com> X-ClientProxiedBy: HE1PR07CA0009.eurprd07.prod.outlook.com (2603:10a6:7:67::19) To VI1PR0802MB2494.eurprd08.prod.outlook.com (2603:10a6:800:b6::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 11567adb-797e-4563-bcf6-08d50cc5ebaa X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(49563074)(201703131423075)(201703031133081)(201702281549075); SRVR:VI1PR0802MB2494; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0802MB2494; 3:8lLFxZmHGUa0QamDBl4ZN7fds84CP4Qr2Ix04scX/RklBU51x0lOuavulPWyiudeFwTqrCI2g5+3Z9jP1mz2GNLgB/0fvl9WfozGMXbfD6F9gkhhdFpC2q32jC2MF/cuX1crF823K+okYVdeYA/m82gAMCiHvQMpZxynW72AVerqpbNnzpEJP//yg+AC/WUmAT0WyHoVMgzbIC/eIZJheqHMvxmmRV16WspjnI7ZvU4HKrx1QgxDjFHM+YmS268c; 25:Y4c6aIp7F0uUFgS30GQ0Y89BK0jJ5vLUZ1AV7dvg84sc5PoXxfv4uYJkIt5Kx2LuU/1sQIhBIbbK+YX75jZW0u3IvN9IuRnDa336zHV77HM10qY57bN5QQ5myMJQfYRbjb3K9ezbw7LK2IJiCk59zH4a/S5NY0zW3xxr55gCUwhc2I9wX6+1CFBBEJKDmk7v/YfXMs2veoveI0WARET8YAxXoFNko8zjFKNAP4y6HQM=; 31:gPWIRR10EjiN+bT3DNopAmUAY2P6fkz46DvSttW0u42eNBQ6jF2GEDdrSANgzW9bp2Kdc1TSTa7uyYTZajyqC3RBmQWnvyDK04yDNQRq4eqFznZjvaZ/VtFbXHs6eMcV; 20:1uIuZ5sywSLfPVr3eDzyeoemUFncChW4iE3ri+hlwTxUc7uQoQ01N8B1Mm/xgvlrg8YBRhYYy7WycD9mVZHdwGUEYVPxcNERcS+rQTLy61+GzPllHN0rKCHkCJSv7GqQsHzPktsMZGD0NKZY+VP8jcQpCSq6yYTE5DpJfMkRMC8= X-MS-TrafficTypeDiagnostic: VI1PR0802MB2494: NoDisclaimer: True X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(102415395)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(100000703101)(100105400095)(3002001)(10201501046)(6055026)(6041248)(20161123562025)(20161123560025)(20161123558100)(20161123555025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:VI1PR0802MB2494; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:VI1PR0802MB2494; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0802MB2494; 4:BJG8LwJnTwRpheE5olHPx4tbyVilX+gZcrzEFYhjAEc970JYGepR7exAeSjObD1dLUKK9f8pN9g+hRF+rulHy6kP02YgT2v/CUbsXJHZhT7iWQCNMrSaso5CjM3UBfZKBbhHWtkSuruwzy2/rKRAZQ== X-Forefront-PRVS: 0452022BE1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(6049001)(376002)(39860400002)(346002)(199003)(189002)(97736004)(564344004)(106356001)(84326002)(36756003)(66066001)(573454002)(4610100001)(58126008)(5000100001)(65956001)(65806001)(7736002)(68736007)(4326008)(6666003)(5890100001)(83506001)(568964002)(16526018)(6116002)(16586007)(2906002)(3846002)(562524006)(270700001)(478600001)(2476003)(101416001)(64126003)(77096006)(16576012)(6486002)(59896002)(6916009)(25786009)(305945005)(105586002)(8676002)(2950100002)(33656002)(65816999)(316002)(8936002)(86362001)(53936002)(80316001)(87266999)(72206003)(50986999)(76176999)(81156014)(81166006)(54356999); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0802MB2494; H:[10.2.206.69]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: arm.com does not designate permitted sender hosts) Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Szabolcs.Nagy@arm.com; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0802MB2494; 23:lyzhdPrzkg6YaCP3Quc/ApMdOLsQ/zNuv9hf67H?= =?us-ascii?Q?DaG9/HEOA7r4Wu8FAsswPYO4CT6tamyQbIXatBUZMSQqnZWAi3bGC4xjb03U?= =?us-ascii?Q?JbZa3bhHB6L298JRdW4CkrbkYdN3kbs0a5CyJyzAb4YUhTL1hk2cqcLoKixJ?= =?us-ascii?Q?Z5BOGYaSdYeJOcztJCn4DkqF8uLcRnG8jPRtajU3n1rtHhUNTnAVyGjSi7ki?= =?us-ascii?Q?PNvGGtXmDc38dkODUZxSGgsr0BLzHRULk01r2NN7/AYvVLiIif0oimzwthmJ?= =?us-ascii?Q?BsFCDut8cGL9u5xtblUZCe8cXIyTL5rBWyvmvslDDoXd2THCQLGbpzde7nZV?= =?us-ascii?Q?CNpLEfowT8OfUcmuz4uNd2ZNWTTDxJOKY6eGYnYMiooUPj2ARt+AqQ9ttKd6?= =?us-ascii?Q?EzChTGN5V6FQwsCPRFbe4L803Lv4hohRuy+oR0HwR+TQVCYqqJ8KMbNX5VQg?= =?us-ascii?Q?Jm2DWvsuSjosikRCOQ7ratLgzWpy4m6ggC800RbQ/Z4M/ZikzgqZWe3cX6Xq?= =?us-ascii?Q?7zOyWtdcWDsI/aPR96Nx58On59OeuzgS5mOCnm8wi5EbDXwemyIaoOJONW+e?= =?us-ascii?Q?COxttHzIe3Bt9Ur0P8+rvxa7gIMLX/pX2CSnQCRj7+6T9Miokj7HpplXM3rP?= =?us-ascii?Q?8kKu//wvnVgk3/2brCNmLukX3UCcTMHJT8zg4AymXRytsn90qWPdeAK46RN+?= =?us-ascii?Q?EsPXCkf0bIS2I3GEY1h+dVj52O0WHeBghrGueptJJlPL8r5/woNx2cJWFF3o?= =?us-ascii?Q?JFi/6mvEtze7p6RfzH6j/sWqaISDXI9FJunSmTT2O1Lcj99S+WSxHBNgNdOc?= =?us-ascii?Q?ZDkENag4TAH3XtMYileqN7sAbjr+u74NNLbra/t5XTkvzIFvRh32eWVmlAMB?= =?us-ascii?Q?xMecK/pdrr0WUXmWhM+macU3oYL0T4N4HjzxzrKyjgV9brSF+utnZ/rQE6f2?= =?us-ascii?Q?pFsHn9X2RpPXqR+SSDk18EzegnT3fSezcUihUAKRnaZymDYuWjY2prfn3FOw?= =?us-ascii?Q?a/0uSZNJ1kXfv445EY6+NrKUbgik76srdHDZmkpfWpyDxfFY9ZWmAdI8ADBJ?= =?us-ascii?Q?wgvVWVlVBwOeuefLhvpfJK0yzA0xKJkabLqPl6SZvKYOXP3om60c53MXqkaY?= =?us-ascii?Q?vSV84/AVNATb7TVB9UnrK5YL1JpKSPX2mE+i4AL2IJXyeb1ip8tQzja2znzt?= =?us-ascii?Q?7IabuBjptQ3znJIGKh+x9770ZBsfIAVH3mCSJ/OXOlWIsquPfk5QjXluDqKU?= =?us-ascii?Q?ATyDO9JfGuH95DhJSbJlAiEvanrzJpKmX1VdjBJqMkmzDLoshLh9+TPaLDjV?= =?us-ascii?Q?fbSm5/XB3kq3V4XL+3M95OPxGQx0Z/cdljH/mDNEq9ISjRJ8Q8gB8Tch+XsQ?= =?us-ascii?Q?GLEYyX1UPUhNknjsqP3N3lcLA8zV7tuCgs27Ke9DypyrADU/n9UeiqQ+qNs3?= =?us-ascii?Q?wymCVTYzBLg=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; VI1PR0802MB2494; 6:NFWObRFVQ2KnNFAMEjxeih9wmxOshYYUUbeupdG/3qZR0K4HoX4MGOERHr3D1Xsxj1iBzEVmrY5w4AGIaubsqB9rL2mLVaIElY0bOk/GQMRoxOqk4vC6MZIpkOuZvq6b2u+T08moxmRcLvf4lgfuBycd1EhfMR8PcbB/2o5u5meCe+bn53nUaEvxkZBAxcMvaDt7ZmWZFEemGTJK4OZcbLywxeKVgQXh/t/dkLwlM20=; 5:Txw13yd980kdFyZ8xFYBINqhK/nmYVrWu5neo771qbV2D8rNswWu/BCafUJRS66G9N7wwc51UAyOtC6cwdKjr3UaZi86+P45Z3DXqS8SzPv38HQgvjx04Qb2XBGAfq3EFntAGHXts/cYwcgKTqm+Ww==; 24:t+YK+HDEbDD8e5m0sHsYmz0ME2DAUuD+03WCr99Pe6ly6quV15H0LS8rJHNP5oNlyxHjfcSzH1Iiaz0dyS0oOt9RevdLJs7X7CVUp7DC12g=; 7:fm9iICGZrl6h8o9ahGPyaTWzcrUyKhp4b8sGOYS1hPdSoKoAK3O7beDAshYFBGe3R+EWiPCxZKVyLQf3N9i/ltMSEkuAOyWWGk+kaJJAx6bsz0tAhJ+i/mmPWikFAzJCpYt7PeOWtMWx0mrzTgGuVuGjbJjlGhAZ1ZAit2E/+BB5WI/yzeUPsB6gjFFZB4mGRtNGXb+9ru4sfUWJM0Q0I0N8VXYOLpVWV52koexM5Jg= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Oct 2017 14:24:14.1125 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2494 From 8a98ef1313d620a0c3fbda4dffcb9df9e47adba0 Mon Sep 17 00:00:00 2001 From: Szabolcs Nagy Date: Wed, 27 Sep 2017 18:14:21 +0100 Subject: [PATCH 2/2] aarch64: Remove barriers from TLS descriptor functions Remove ldar synchronization and most lazy TLSDESC initialization related code. 2017-09-29 Szabolcs Nagy * elf/tlsdeschtab.h (_dl_tls_resolve_early_return_p): Mark unused. (_dl_tlsdesc_wake_up_held_fixups): Likewise. * sysdeps/aarch64/dl-machine.h (elf_machine_runtime_setup): Remove DT_TLSDESC_GOT initialization. * sysdeps/aarch64/dl-tlsdesc.S (_dl_tlsdesc_return_lazy): Remove. (_dl_tlsdesc_resolve_rela): Likewise. (_dl_tlsdesc_resolve_hold): Likewise. (_dl_tlsdesc_undefweak): Remove ldar. (_dl_tlsdesc_dynamic): Likewise. * sysdeps/aarch64/dl-tlsdesc.h (_dl_tlsdesc_return_lazy): Remove. (_dl_tlsdesc_resolve_rela): Likewise. (_dl_tlsdesc_resolve_hold): Likewise. * sysdeps/aarch64/tlsdesc.c (_dl_tlsdesc_resolve_rela_fixup): Remove. (_dl_tlsdesc_resolve_hold_fixup): Likewise. (_dl_tlsdesc_resolve_rela): Likewise. (_dl_tlsdesc_resolve_hold): Likewise. --- elf/tlsdeschtab.h | 2 + sysdeps/aarch64/dl-machine.h | 4 - sysdeps/aarch64/dl-tlsdesc.S | 203 ------------------------------------------- sysdeps/aarch64/dl-tlsdesc.h | 9 -- sysdeps/aarch64/tlsdesc.c | 127 +-------------------------- 5 files changed, 3 insertions(+), 342 deletions(-) diff --git a/elf/tlsdeschtab.h b/elf/tlsdeschtab.h index ad3001dac5..879631897c 100644 --- a/elf/tlsdeschtab.h +++ b/elf/tlsdeschtab.h @@ -137,6 +137,7 @@ _dl_make_tlsdesc_dynamic (struct link_map *map, size_t ti_offset) avoid introducing such dependencies. */ static int +__attribute__ ((unused)) _dl_tlsdesc_resolve_early_return_p (struct tlsdesc volatile *td, void *caller) { if (caller != atomic_load_relaxed (&td->entry)) @@ -155,6 +156,7 @@ _dl_tlsdesc_resolve_early_return_p (struct tlsdesc volatile *td, void *caller) } static void +__attribute__ ((unused)) _dl_tlsdesc_wake_up_held_fixups (void) { __rtld_lock_unlock_recursive (GL(dl_load_lock)); diff --git a/sysdeps/aarch64/dl-machine.h b/sysdeps/aarch64/dl-machine.h index 9bd48752e5..89d0fe87db 100644 --- a/sysdeps/aarch64/dl-machine.h +++ b/sysdeps/aarch64/dl-machine.h @@ -131,10 +131,6 @@ elf_machine_runtime_setup (struct link_map *l, int lazy, int profile) } } - if (l->l_info[ADDRIDX (DT_TLSDESC_GOT)] && lazy) - *(ElfW(Addr)*)(D_PTR (l, l_info[ADDRIDX (DT_TLSDESC_GOT)]) + l->l_addr) - = (ElfW(Addr)) &_dl_tlsdesc_resolve_rela; - return lazy; } diff --git a/sysdeps/aarch64/dl-tlsdesc.S b/sysdeps/aarch64/dl-tlsdesc.S index 3b2da62f39..70550c7ce0 100644 --- a/sysdeps/aarch64/dl-tlsdesc.S +++ b/sysdeps/aarch64/dl-tlsdesc.S @@ -80,30 +80,6 @@ _dl_tlsdesc_return: cfi_endproc .size _dl_tlsdesc_return, .-_dl_tlsdesc_return - /* Same as _dl_tlsdesc_return but with synchronization for - lazy relocation. - Prototype: - _dl_tlsdesc_return_lazy (tlsdesc *) ; - */ - .hidden _dl_tlsdesc_return_lazy - .global _dl_tlsdesc_return_lazy - .type _dl_tlsdesc_return_lazy,%function - cfi_startproc - .align 2 -_dl_tlsdesc_return_lazy: - /* The ldar here happens after the load from [x0] at the call site - (that is generated by the compiler as part of the TLS access ABI), - so it reads the same value (this function is the final value of - td->entry) and thus it synchronizes with the release store to - td->entry in _dl_tlsdesc_resolve_rela_fixup ensuring that the load - from [x0,#PTR_SIZE] here happens after the initialization of td->arg. */ - DELOUSE (0) - ldar PTR_REG (zr), [x0] - ldr PTR_REG (0), [x0, #PTR_SIZE] - RET - cfi_endproc - .size _dl_tlsdesc_return_lazy, .-_dl_tlsdesc_return_lazy - /* Handler for undefined weak TLS symbols. Prototype: _dl_tlsdesc_undefweak (tlsdesc *); @@ -121,14 +97,7 @@ _dl_tlsdesc_return_lazy: _dl_tlsdesc_undefweak: str x1, [sp, #-16]! cfi_adjust_cfa_offset (16) - /* The ldar here happens after the load from [x0] at the call site - (that is generated by the compiler as part of the TLS access ABI), - so it reads the same value (this function is the final value of - td->entry) and thus it synchronizes with the release store to - td->entry in _dl_tlsdesc_resolve_rela_fixup ensuring that the load - from [x0,#8] here happens after the initialization of td->arg. */ DELOUSE (0) - ldar PTR_REG (zr), [x0] ldr PTR_REG (0), [x0, #PTR_SIZE] mrs x1, tpidr_el0 sub PTR_REG (0), PTR_REG (0), PTR_REG (1) @@ -192,13 +161,6 @@ _dl_tlsdesc_dynamic: cfi_rel_offset (x4, 32+24) mrs x4, tpidr_el0 - /* The ldar here happens after the load from [x0] at the call site - (that is generated by the compiler as part of the TLS access ABI), - so it reads the same value (this function is the final value of - td->entry) and thus it synchronizes with the release store to - td->entry in _dl_tlsdesc_resolve_rela_fixup ensuring that the load - from [x0,#PTR_SIZE] here happens after the initialization of td->arg. */ - ldar PTR_REG (zr), [x0] ldr PTR_REG (1), [x0,#TLSDESC_ARG] ldr PTR_REG (0), [x4,#TCBHEAD_DTV] ldr PTR_REG (3), [x1,#TLSDESC_GEN_COUNT] @@ -276,168 +238,3 @@ _dl_tlsdesc_dynamic: .size _dl_tlsdesc_dynamic, .-_dl_tlsdesc_dynamic # undef NSAVEXREGPAIRS #endif - - /* This function is a wrapper for a lazy resolver for TLS_DESC - RELA relocations. - When the actual resolver returns, it will have adjusted the - TLS descriptor such that we can tail-call it for it to return - the TP offset of the symbol. */ - - .hidden _dl_tlsdesc_resolve_rela - .global _dl_tlsdesc_resolve_rela - .type _dl_tlsdesc_resolve_rela,%function - cfi_startproc - .align 2 -_dl_tlsdesc_resolve_rela: -#define NSAVEXREGPAIRS 9 - /* The tlsdesc PLT entry pushes x2 and x3 to the stack. */ - cfi_adjust_cfa_offset (16) - cfi_rel_offset (x2, 0) - cfi_rel_offset (x3, 8) - stp x29, x30, [sp, #-(32+16*NSAVEXREGPAIRS)]! - cfi_adjust_cfa_offset (32+16*NSAVEXREGPAIRS) - cfi_rel_offset (x29, 0) - cfi_rel_offset (x30, 8) - mov x29, sp - stp x1, x4, [sp, #32+16*0] - stp x5, x6, [sp, #32+16*1] - stp x7, x8, [sp, #32+16*2] - stp x9, x10, [sp, #32+16*3] - stp x11, x12, [sp, #32+16*4] - stp x13, x14, [sp, #32+16*5] - stp x15, x16, [sp, #32+16*6] - stp x17, x18, [sp, #32+16*7] - str x0, [sp, #32+16*8] - cfi_rel_offset (x1, 32) - cfi_rel_offset (x4, 32+8) - cfi_rel_offset (x5, 32+16) - cfi_rel_offset (x6, 32+16+8) - cfi_rel_offset (x7, 32+16*2) - cfi_rel_offset (x8, 32+16*2+8) - cfi_rel_offset (x9, 32+16*3) - cfi_rel_offset (x10, 32+16*3+8) - cfi_rel_offset (x11, 32+16*4) - cfi_rel_offset (x12, 32+16*4+8) - cfi_rel_offset (x13, 32+16*5) - cfi_rel_offset (x14, 32+16*5+8) - cfi_rel_offset (x15, 32+16*6) - cfi_rel_offset (x16, 32+16*6+8) - cfi_rel_offset (x17, 32+16*7) - cfi_rel_offset (x18, 32+16*7+8) - cfi_rel_offset (x0, 32+16*8) - - SAVE_Q_REGISTERS - - DELOUSE (3) - ldr PTR_REG (1), [x3, #PTR_SIZE] - bl _dl_tlsdesc_resolve_rela_fixup - - RESTORE_Q_REGISTERS - - ldr x0, [sp, #32+16*8] - DELOUSE (0) - ldr PTR_REG (1), [x0] - blr x1 - - ldp x1, x4, [sp, #32+16*0] - ldp x5, x6, [sp, #32+16*1] - ldp x7, x8, [sp, #32+16*2] - ldp x9, x10, [sp, #32+16*3] - ldp x11, x12, [sp, #32+16*4] - ldp x13, x14, [sp, #32+16*5] - ldp x15, x16, [sp, #32+16*6] - ldp x17, x18, [sp, #32+16*7] - ldp x29, x30, [sp], #(32+16*NSAVEXREGPAIRS) - cfi_adjust_cfa_offset (-32-16*NSAVEXREGPAIRS) - cfi_restore (x29) - cfi_restore (x30) - ldp x2, x3, [sp], #16 - cfi_adjust_cfa_offset (-16) - RET -#undef NSAVEXREGPAIRS - cfi_endproc - .size _dl_tlsdesc_resolve_rela, .-_dl_tlsdesc_resolve_rela - - /* This function is a placeholder for lazy resolving of TLS - relocations. Once some thread starts resolving a TLS - relocation, it sets up the TLS descriptor to use this - resolver, such that other threads that would attempt to - resolve it concurrently may skip the call to the original lazy - resolver and go straight to a condition wait. - - When the actual resolver returns, it will have adjusted the - TLS descriptor such that we can tail-call it for it to return - the TP offset of the symbol. */ - - .hidden _dl_tlsdesc_resolve_hold - .global _dl_tlsdesc_resolve_hold - .type _dl_tlsdesc_resolve_hold,%function - cfi_startproc - .align 2 -_dl_tlsdesc_resolve_hold: -#define NSAVEXREGPAIRS 10 -1: - stp x29, x30, [sp, #-(32+16*NSAVEXREGPAIRS)]! - cfi_adjust_cfa_offset (32+16*NSAVEXREGPAIRS) - cfi_rel_offset (x29, 0) - cfi_rel_offset (x30, 8) - mov x29, sp - stp x1, x2, [sp, #32+16*0] - stp x3, x4, [sp, #32+16*1] - stp x5, x6, [sp, #32+16*2] - stp x7, x8, [sp, #32+16*3] - stp x9, x10, [sp, #32+16*4] - stp x11, x12, [sp, #32+16*5] - stp x13, x14, [sp, #32+16*6] - stp x15, x16, [sp, #32+16*7] - stp x17, x18, [sp, #32+16*8] - str x0, [sp, #32+16*9] - cfi_rel_offset (x1, 32) - cfi_rel_offset (x2, 32+8) - cfi_rel_offset (x3, 32+16) - cfi_rel_offset (x4, 32+16+8) - cfi_rel_offset (x5, 32+16*2) - cfi_rel_offset (x6, 32+16*2+8) - cfi_rel_offset (x7, 32+16*3) - cfi_rel_offset (x8, 32+16*3+8) - cfi_rel_offset (x9, 32+16*4) - cfi_rel_offset (x10, 32+16*4+8) - cfi_rel_offset (x11, 32+16*5) - cfi_rel_offset (x12, 32+16*5+8) - cfi_rel_offset (x13, 32+16*6) - cfi_rel_offset (x14, 32+16*6+8) - cfi_rel_offset (x15, 32+16*7) - cfi_rel_offset (x16, 32+16*7+8) - cfi_rel_offset (x17, 32+16*8) - cfi_rel_offset (x18, 32+16*8+8) - cfi_rel_offset (x0, 32+16*9) - - SAVE_Q_REGISTERS - - adr x1, 1b - bl _dl_tlsdesc_resolve_hold_fixup - - RESTORE_Q_REGISTERS - - ldr x0, [sp, #32+16*9] - DELOUSE (0) - ldr PTR_REG (1), [x0] - blr x1 - - ldp x1, x2, [sp, #32+16*0] - ldp x3, x4, [sp, #32+16*1] - ldp x5, x6, [sp, #32+16*2] - ldp x7, x8, [sp, #32+16*3] - ldp x9, x10, [sp, #32+16*4] - ldp x11, x12, [sp, #32+16*5] - ldp x13, x14, [sp, #32+16*6] - ldp x15, x16, [sp, #32+16*7] - ldp x17, x18, [sp, #32+16*8] - ldp x29, x30, [sp], #(32+16*NSAVEXREGPAIRS) - cfi_adjust_cfa_offset (-32-16*NSAVEXREGPAIRS) - cfi_restore (x29) - cfi_restore (x30) - RET - cfi_endproc - .size _dl_tlsdesc_resolve_hold, .-_dl_tlsdesc_resolve_hold -#undef NSAVEXREGPAIRS diff --git a/sysdeps/aarch64/dl-tlsdesc.h b/sysdeps/aarch64/dl-tlsdesc.h index 66ec0dea7f..25b5b64e6e 100644 --- a/sysdeps/aarch64/dl-tlsdesc.h +++ b/sysdeps/aarch64/dl-tlsdesc.h @@ -46,17 +46,8 @@ extern ptrdiff_t attribute_hidden _dl_tlsdesc_return (struct tlsdesc *); extern ptrdiff_t attribute_hidden -_dl_tlsdesc_return_lazy (struct tlsdesc *); - -extern ptrdiff_t attribute_hidden _dl_tlsdesc_undefweak (struct tlsdesc *); -extern ptrdiff_t attribute_hidden -_dl_tlsdesc_resolve_rela (struct tlsdesc *); - -extern ptrdiff_t attribute_hidden -_dl_tlsdesc_resolve_hold (struct tlsdesc *); - # ifdef SHARED extern void *_dl_make_tlsdesc_dynamic (struct link_map *, size_t); diff --git a/sysdeps/aarch64/tlsdesc.c b/sysdeps/aarch64/tlsdesc.c index 4c8a5a81d1..bd1356fa8e 100644 --- a/sysdeps/aarch64/tlsdesc.c +++ b/sysdeps/aarch64/tlsdesc.c @@ -18,137 +18,12 @@ License along with the GNU C Library; if not, see . */ -#include #include -#include #include #include #include +#define _dl_tlsdesc_resolve_hold 0 #include -#include - -/* The following functions take an entry_check_offset argument. It's - computed by the caller as an offset between its entry point and the - call site, such that by adding the built-in return address that is - implicitly passed to the function with this offset, we can easily - obtain the caller's entry point to compare with the entry point - given in the TLS descriptor. If it's changed, we want to return - immediately. */ - -/* This function is used to lazily resolve TLS_DESC RELA relocations. - The argument location is used to hold a pointer to the relocation. */ - -void -attribute_hidden -_dl_tlsdesc_resolve_rela_fixup (struct tlsdesc *td, struct link_map *l) -{ - const ElfW(Rela) *reloc = atomic_load_relaxed (&td->arg); - - /* After GL(dl_load_lock) is grabbed only one caller can see td->entry in - initial state in _dl_tlsdesc_resolve_early_return_p, other concurrent - callers will return and retry calling td->entry. The updated td->entry - synchronizes with the single writer so all read accesses here can use - relaxed order. */ - if (_dl_tlsdesc_resolve_early_return_p - (td, (void*)(D_PTR (l, l_info[ADDRIDX (DT_TLSDESC_PLT)]) + l->l_addr))) - return; - - /* The code below was borrowed from _dl_fixup(), - except for checking for STB_LOCAL. */ - const ElfW(Sym) *const symtab - = (const void *) D_PTR (l, l_info[DT_SYMTAB]); - const char *strtab = (const void *) D_PTR (l, l_info[DT_STRTAB]); - const ElfW(Sym) *sym = &symtab[ELFW(R_SYM) (reloc->r_info)]; - lookup_t result; - - /* Look up the target symbol. If the normal lookup rules are not - used don't look in the global scope. */ - if (ELFW(ST_BIND) (sym->st_info) != STB_LOCAL - && __builtin_expect (ELFW(ST_VISIBILITY) (sym->st_other), 0) == 0) - { - const struct r_found_version *version = NULL; - - if (l->l_info[VERSYMIDX (DT_VERSYM)] != NULL) - { - const ElfW(Half) *vernum = - (const void *) D_PTR (l, l_info[VERSYMIDX (DT_VERSYM)]); - ElfW(Half) ndx = vernum[ELFW(R_SYM) (reloc->r_info)] & 0x7fff; - version = &l->l_versions[ndx]; - if (version->hash == 0) - version = NULL; - } - - result = _dl_lookup_symbol_x (strtab + sym->st_name, l, &sym, - l->l_scope, version, ELF_RTYPE_CLASS_PLT, - DL_LOOKUP_ADD_DEPENDENCY, NULL); - } - else - { - /* We already found the symbol. The module (and therefore its load - address) is also known. */ - result = l; - } - - if (!sym) - { - atomic_store_relaxed (&td->arg, (void *) reloc->r_addend); - /* This release store synchronizes with the ldar acquire load - instruction in _dl_tlsdesc_undefweak. */ - atomic_store_release (&td->entry, _dl_tlsdesc_undefweak); - } - else - { -# ifndef SHARED - CHECK_STATIC_TLS (l, result); -# else - if (!TRY_STATIC_TLS (l, result)) - { - void *p = _dl_make_tlsdesc_dynamic (result, sym->st_value - + reloc->r_addend); - atomic_store_relaxed (&td->arg, p); - /* This release store synchronizes with the ldar acquire load - instruction in _dl_tlsdesc_dynamic. */ - atomic_store_release (&td->entry, _dl_tlsdesc_dynamic); - } - else -# endif - { - void *p = (void*) (sym->st_value + result->l_tls_offset - + reloc->r_addend); - atomic_store_relaxed (&td->arg, p); - /* This release store synchronizes with the ldar acquire load - instruction in _dl_tlsdesc_return_lazy. */ - atomic_store_release (&td->entry, _dl_tlsdesc_return_lazy); - } - } - - _dl_tlsdesc_wake_up_held_fixups (); -} - -/* This function is used to avoid busy waiting for other threads to - complete the lazy relocation. Once another thread wins the race to - relocate a TLS descriptor, it sets the descriptor up such that this - function is called to wait until the resolver releases the - lock. */ - -void -attribute_hidden -_dl_tlsdesc_resolve_hold_fixup (struct tlsdesc *td, void *caller) -{ - /* Maybe we're lucky and can return early. */ - if (caller != atomic_load_relaxed (&td->entry)) - return; - - /* Locking here will stop execution until the running resolver runs - _dl_tlsdesc_wake_up_held_fixups(), releasing the lock. - - FIXME: We'd be better off waiting on a condition variable, such - that we didn't have to hold the lock throughout the relocation - processing. */ - __rtld_lock_lock_recursive (GL(dl_load_lock)); - __rtld_lock_unlock_recursive (GL(dl_load_lock)); -} - /* Unmap the dynamic object, but also release its TLS descriptor table if there is one. */ -- 2.11.0