From patchwork Tue Apr 13 08:19:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 42966 X-Patchwork-Delegate: azanella@linux.vnet.ibm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CEDD0393C85F; Tue, 13 Apr 2021 08:19:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CEDD0393C85F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1618301983; bh=LaKYoWqKLO5NDHPyszwt0zNEKgRV794+tn5Hd1VfQTg=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ImUQKw9WBCNT8AZE3+mp6PB0B0vzp3mfLaw57Hn6ZcLTn+NRJEBa50pIi2CCmfMcH pXCX+6/rXj81RvlzunF3LxSUR2QkdcQBIs1oX9p9vIyqmfVpNGJd9DoknWpVeZkmtI UGaoKi6Ub/MAppEuT4HJUCwJOkr9+HBB6JcSQ9Oo= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80048.outbound.protection.outlook.com [40.107.8.48]) by sourceware.org (Postfix) with ESMTPS id A51C4393C85F for ; Tue, 13 Apr 2021 08:19:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A51C4393C85F Received: from AS8PR04CA0067.eurprd04.prod.outlook.com (2603:10a6:20b:313::12) by VI1PR0801MB2096.eurprd08.prod.outlook.com (2603:10a6:800:8b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.17; Tue, 13 Apr 2021 08:19:37 +0000 Received: from AM5EUR03FT032.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:313:cafe::10) by AS8PR04CA0067.outlook.office365.com (2603:10a6:20b:313::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.17 via Frontend Transport; Tue, 13 Apr 2021 08:19:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; sourceware.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;sourceware.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT032.mail.protection.outlook.com (10.152.16.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.17 via Frontend Transport; Tue, 13 Apr 2021 08:19:36 +0000 Received: ("Tessian outbound 34291f7cb530:v90"); Tue, 13 Apr 2021 08:19:36 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: bd4747229966f4c5 X-CR-MTA-TID: 64aa7808 Received: from 29273e85ca4c.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 36108BE1-4026-41BF-B9E2-23D4FAA79E5F.1; Tue, 13 Apr 2021 08:19:27 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 29273e85ca4c.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 13 Apr 2021 08:19:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gMwBuV2ulh4r2TnU1V1j4sUr/H2S8nNahkLYYagT4YWRvSdUYZ5iM3RlY1AChKtNZ4mg5m9tiwje22EyMroCkDrrckfuRzIkx2L5t9iPZVLIfc67nFPmRKkcoLGHl/8sE1EEUEfDQ7VeY+mJ1MKY7Edzwnxm/Z9FMeWZoCAxml4WtIVxPrFL/hqxjogoQQCIkMuHihMFZKVsX4logHeR6llo+p2NI/1BGz/XkoZWX+86nFZFgQ/bM6MxU10zbIKwmflrPz6ac0DnBIXnuZsBYabGvDI3KAmBwIFYAo/bKuN86Lr54JIj4aB3QcCONahD+pMHpuOdg7z+oULswQ0Lag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LaKYoWqKLO5NDHPyszwt0zNEKgRV794+tn5Hd1VfQTg=; b=bS+oRs052A8ilmseNtGjYYSiPPRzu3y5OgpKkw9YnEJpbs1E7n0+5SYuvD2lKVQfi/zJxCqKTTsfurX0Z6ULPxKbzg8kucxyy8zXbWMy/T1IdXjS3rBF9ISTJsppLoNWixmi0QtNqmNxHGY/9VaQOJ8fB7Kvy4XZ/D07vKZ17D0WztEETWVcUXvyDjdMDPhJ4syZMC+VIGlO0m1ysxJUZFGj4SYxvDaIAQwx5IrefgJMetFDZUgSSKDvaEzRxlPSV0Pov/DKpSB6Bz5XzUDD5462amosCPKKS42mXHwRwc3ePKYr4uIwgwU/VsceqmKqXLWsQRJ6A+pLPrDkbKafng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: sourceware.org; dkim=none (message not signed) header.d=none;sourceware.org; dmarc=none action=none header.from=arm.com; Received: from PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) by PAXPR08MB6397.eurprd08.prod.outlook.com (2603:10a6:102:12f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Tue, 13 Apr 2021 08:19:25 +0000 Received: from PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::c99f:671d:bb2c:f20b]) by PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::c99f:671d:bb2c:f20b%7]) with mapi id 15.20.4020.022; Tue, 13 Apr 2021 08:19:25 +0000 To: libc-alpha@sourceware.org Subject: [PATCH v2 05/14] elf: Fix data races in pthread_create and TLS access [BZ #19329] Date: Tue, 13 Apr 2021 09:19:13 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: X-Originating-IP: [217.140.106.55] X-ClientProxiedBy: SN7PR04CA0232.namprd04.prod.outlook.com (2603:10b6:806:127::27) To PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (217.140.106.55) by SN7PR04CA0232.namprd04.prod.outlook.com (2603:10b6:806:127::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.16 via Frontend Transport; Tue, 13 Apr 2021 08:19:24 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2c753614-fe34-4c45-7c33-08d8fe54e07e X-MS-TrafficTypeDiagnostic: PAXPR08MB6397:|VI1PR0801MB2096: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 70Yfzl/Gsq+xK3fgV9eQgxEA3cg2sZ3WKxnXkl0XZAk6tJHYhqaun17Z8fAT6TCul8cFiLwtX+2EKZcvqLBKARgoCDIl1cyQVjT7qXTNtQlJa5XAWoOx4Uh45uYky/oguFFXscYxQuWHZfACJz2XoCN6DFs/D+dYDEmT4dbQfScDBmUylgcQCAynIvIoUtyjAsyC968nayOnx8IaTplgKgmfl461hIxzhT+sBfFTcNX4Wi7fsM5c3CLlrPjnVAlkeMi2BxD88aqOPBQxuKmDYf5bDCxaabLLMl2HpLEmFnMGmXvO1XDSvRERrnLOVNN1I4EfjMybhVzLzeGq3HrPcv+qB1ZdJha66YGOKTxDd05InsEugZ5l1yw/XM35jdVlxp7+iosQt+89kkI4485LPeaWufBV0he3XU6fjcZggq8zhzCDKptchL+r6Ej4Mut5VlPK/cBoXgTN+P5Y4hKK/8LYElGaYoEH52dzq8CWZSgJx6UUu3ruHzumELxWd4xi2L4vDN/k+x4MqeDn333l2UvUQVV+KXAB8dRWY+8QNQt8rHMaLwjvhiuckB/zd82Y0iYSIjvjkFRtchyRnWLDHxsM4IHrD1xE6ORCapsghzEBu1dFybdZLS0+IrKHr74sz2pYkh1ijmmNwqWan6qKdDi0GXXmg4bbZRd+aDxKD2W6V+xWocY3Tr+9GPXWMDlr X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PA4PR08MB6320.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(346002)(396003)(39860400002)(376002)(366004)(2906002)(6666004)(66556008)(6506007)(36756003)(26005)(66946007)(69590400012)(44832011)(38350700002)(83380400001)(956004)(86362001)(6486002)(16526019)(8676002)(316002)(8936002)(478600001)(6512007)(52116002)(38100700002)(5660300002)(2616005)(186003)(66476007)(6916009); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: gZr2ZoAJRBY4lZGghKz53HNeQM+EBad8gD9MCIFoX6kTFK1UDo6scRY4c5n5hNliOLkP39q3s9DkkNx9qOus4JTm5d7MRWChuVA8OVT7xFR1UP/GJ4p0hgY5T2Uw294k8j5LLuGmOJp4KC88gjU9xE9ev0372SYYmtZyBkg+OnXKiLZYuv58K2oTC2Re+uvHOdIX9L194yqEN23XKs9GZmBVHC867+7/ztSvuNgMMyG2ahhf2VyX2hZ+Pbth+Y6zSklZ+LfMe998tw44PmQrks0dwfXpU5AKtV1rzhjesvUxkiPJMGmE9J1145C37IPrqBQYNEfGCYjh/gTy+ZQXdruDXe/931vh4ENY+C+tiGjkp34dlG9BHXDgo2e5tYqP50YqK8EMoEUAKmBA9Kr0GzAfCyPPkcDRXV9SLsoeoulmGv43xN38X+Phn2WuCeb+Us0eJ/uBvHoMzol/ViF7oFMy2tvabVvGl/q8wXGlkYzTDWF/kkmOzLsUXwR0lLvn7KDeKxxxUTNMUMGfr7e/yk7KVCILs2W4QOaNrUFFduBGwW61DuVfegEqM/9HTnRuCF59kVvCBZTG+EkjuSViYw/tgUVNG25IVslXdCfM5x7Vh4sGNoffowc6AMuDNXsoNEU7R5iOl4j7jpsTYdmq9XjzSpsyHSOYuR7bSO8/boi18cmAuwQ06wlF6rpshEYyY9TB+eeXY008Nnw+D/qnpND9vatrES+zXDACqUi24m19VooI4qp58cP0QpDA68cuwRwazauqFMY6emqO+108pgDRtxGY2fqy5YmX+uchWkudqAIqjVUe4I7gZp1cYCSiJ9pnNtUDB2xkDsENaW8GpDNpczYXmfUY3YaN/Ls+E49tbr+xiafIuLpMjfBlwN+n3fq5f4mk9oe4Z3w5FHWdyz1KsFuWeS9hFZBhOxKphvcKanBF870uUKGG+4BbXrdUIJrGWpRbcUNdRZ/KqP9ndqe7CQA4xdk0vqvp6wRaMC3cXFULl7hk+ydMi+yxmL7Z8AW3ySzaZVVnm7gXLbfn8qYxCDwG1snKIiKltwRsoue/6KMtQ46gWGjMHJ+Ipy3o8C12YnDaS19p51jhk09GMD1HcvhlJwZbwSDMKtEggQLGzdtIpSb8DKH5gJl6h5PGLdKHOXhuqYlFI+xZ19BdeUPJUrIT+EI39LXLQk6GoxBbRbsTJnvewErhASTjGPXSKYpLr/31GRny+2o7qAXV74g4GDO5qBKTuXdXLmgJ4YvgF2EvckzYq0xKJkdw9TxIvbbjRuY/L0Y4ppUbGGTeb2PwQYj7dK0nplA+LQ1VwOq9qR0PKlMgwzK/HMrzFDaA X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6397 Original-Authentication-Results: sourceware.org; dkim=none (message not signed) header.d=none; sourceware.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT032.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: feac154d-7ac5-4081-7a68-08d8fe54d93b X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +jhlBRxk7ni1txnAq8Rq49oYEMopE39JZhWzsti6voZm4vk6UUezG/3gpiUk1t9Nf0PtqFvednMG5DdgUU1ezYXFhC6rDHJIL+YczgECLavct92tip91xBFX+F3BQ3AYBP3UOYguqCT7gX4zS+o921TeFRV+Jdjnn0kLeMTOrtc+qz5rCVjlS1FjUL276NpcaDCr9V1SCPXTpNbCzBdwq0T7sI0GOZ87xQ40OjUYBJIA/QNM6xE08jQ8n0JXmGjzqG0vWttnDfYn1rT2wYiQ78EBJiJQ61JKhjeYB0Se3xrHe8AxhivCKwnb69/tSN8StsX+8fN0S4HS2ybPv4j5pUlENhzfNurcBl9TkEA2L9KArtWHZVKS5ez1sqh+0wQboURTe0oT9qZGwZxnXzd+PJj/PCt1fZdHJmAfAfIKnCIgt5aYypsB0ZNvl+vUKNNGAVn9LgTgjK7BeAH3qXsD1Qh1EW0VoGV8JwMoHkIXpRPhgjo8vHHx6FQf5vtvVHxVCIHf8U40Im+L67rbcfdWntzFto8noBk81a+Hyf+w6AYgyxWbE/V3dJV9f7+C7FkoVLLpX9GrYTpHGC0rkaTb/NZ2+hUzCLfqZVtc7Axo5/BrQOhWHk1bsIyWR3yS/qYR8ULxiMoJ7ssAteZxTIttwvDE4+z9IJJ/SBP0LMDpEvLJxCPBSGb9dknNir7cNUeq X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(346002)(396003)(376002)(136003)(39860400002)(36840700001)(46966006)(6486002)(82740400003)(356005)(70206006)(26005)(44832011)(6666004)(70586007)(36860700001)(6512007)(8676002)(478600001)(81166007)(82310400003)(316002)(2616005)(2906002)(186003)(956004)(83380400001)(47076005)(86362001)(36756003)(8936002)(6916009)(6506007)(336012)(5660300002)(69590400012)(16526019); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2021 08:19:36.9159 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2c753614-fe34-4c45-7c33-08d8fe54e07e X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT032.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB2096 X-Spam-Status: No, score=-13.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Szabolcs Nagy via Libc-alpha From: Szabolcs Nagy Reply-To: Szabolcs Nagy Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" DTV setup at thread creation (_dl_allocate_tls_init) is changed to take the dlopen lock, GL(dl_load_lock). Avoiding data races here without locks would require design changes: the map that is accessed for static TLS initialization here may be concurrently freed by dlclose. That use after free may be solved by only locking around static TLS setup or by ensuring dlclose does not free modules with static TLS, however currently every link map with TLS has to be accessed at least to see if it needs static TLS. And even if that's solved, still a lot of atomics would be needed to synchronize DTV related globals without a lock. So fix both bug 19329 and bug 27111 with a lock that prevents DTV setup running concurrently with dlopen or dlclose. _dl_update_slotinfo at TLS access still does not use any locks so CONCURRENCY NOTES are added to explain the synchronization. The early exit from the slotinfo walk when max_modid is reached is not strictly necessary, but does not hurt either. An incorrect acquire load was removed from _dl_resize_dtv: it did not synchronize with any release store or fence and synchronization is now handled separately at thread creation and TLS access time. There are still a number of racy read accesses to globals that will be changed to relaxed MO atomics in a followup patch. This should not introduce regressions compared to existing behaviour and avoid cluttering the main part of the fix. Not all TLS access related data races got fixed here: there are additional races at lazy tlsdesc relocations see bug 27137. Reviewed-by: Adhemerval Zanella --- elf/dl-tls.c | 63 +++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 47 insertions(+), 16 deletions(-) diff --git a/elf/dl-tls.c b/elf/dl-tls.c index f8b32b3ecb..33c06782b1 100644 --- a/elf/dl-tls.c +++ b/elf/dl-tls.c @@ -471,14 +471,11 @@ extern dtv_t _dl_static_dtv[]; #endif static dtv_t * -_dl_resize_dtv (dtv_t *dtv) +_dl_resize_dtv (dtv_t *dtv, size_t max_modid) { /* Resize the dtv. */ dtv_t *newp; - /* Load GL(dl_tls_max_dtv_idx) atomically since it may be written to by - other threads concurrently. */ - size_t newsize - = atomic_load_acquire (&GL(dl_tls_max_dtv_idx)) + DTV_SURPLUS; + size_t newsize = max_modid + DTV_SURPLUS; size_t oldsize = dtv[-1].counter; if (dtv == GL(dl_initial_dtv)) @@ -524,11 +521,14 @@ _dl_allocate_tls_init (void *result) size_t total = 0; size_t maxgen = 0; + /* Protects global dynamic TLS related state. */ + __rtld_lock_lock_recursive (GL(dl_load_lock)); + /* Check if the current dtv is big enough. */ if (dtv[-1].counter < GL(dl_tls_max_dtv_idx)) { /* Resize the dtv. */ - dtv = _dl_resize_dtv (dtv); + dtv = _dl_resize_dtv (dtv, GL(dl_tls_max_dtv_idx)); /* Install this new dtv in the thread data structures. */ INSTALL_DTV (result, &dtv[-1]); @@ -596,6 +596,7 @@ _dl_allocate_tls_init (void *result) listp = listp->next; assert (listp != NULL); } + __rtld_lock_unlock_recursive (GL(dl_load_lock)); /* The DTV version is up-to-date now. */ dtv[0].counter = maxgen; @@ -730,12 +731,29 @@ _dl_update_slotinfo (unsigned long int req_modid) if (dtv[0].counter < listp->slotinfo[idx].gen) { - /* The generation counter for the slot is higher than what the - current dtv implements. We have to update the whole dtv but - only those entries with a generation counter <= the one for - the entry we need. */ + /* CONCURRENCY NOTES: + + Here the dtv needs to be updated to new_gen generation count. + + This code may be called during TLS access when GL(dl_load_lock) + is not held. In that case the user code has to synchrnize with + dlopen and dlclose calls of relevant modules. A module m is + relevant if the generation of m <= new_gen and dlclose of m is + synchronized: a memory access here happens after the dlopen and + before the dlclose of relevant modules. The dtv entries for + relevant modules need to be updated, other entries can be + arbitrary. + + This e.g. means that the first part of the slotinfo list can be + accessed race free, but the tail may be concurrently extended. + Similarly relevant slotinfo entries can be read race free, but + other entries are racy. However updating a non-relevant dtv + entry does not affect correctness. For a relevant module m, + max_modid >= modid of m. */ size_t new_gen = listp->slotinfo[idx].gen; size_t total = 0; + size_t max_modid = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); + assert (max_modid >= req_modid); /* We have to look through the entire dtv slotinfo list. */ listp = GL(dl_tls_dtv_slotinfo_list); @@ -745,12 +763,14 @@ _dl_update_slotinfo (unsigned long int req_modid) { size_t modid = total + cnt; + /* Later entries are not relevant. */ + if (modid > max_modid) + break; + size_t gen = listp->slotinfo[cnt].gen; if (gen > new_gen) - /* This is a slot for a generation younger than the - one we are handling now. It might be incompletely - set up so ignore it. */ + /* Not relevant. */ continue; /* If the entry is older than the current dtv layout we @@ -767,7 +787,7 @@ _dl_update_slotinfo (unsigned long int req_modid) continue; /* Resize the dtv. */ - dtv = _dl_resize_dtv (dtv); + dtv = _dl_resize_dtv (dtv, max_modid); assert (modid <= dtv[-1].counter); @@ -789,8 +809,17 @@ _dl_update_slotinfo (unsigned long int req_modid) } total += listp->len; + if (total > max_modid) + break; + + /* Synchronize with _dl_add_to_slotinfo. Ideally this would + be consume MO since we only need to order the accesses to + the next node after the read of the address and on most + hardware (other than alpha) a normal load would do that + because of the address dependency. */ + listp = atomic_load_acquire (&listp->next); } - while ((listp = listp->next) != NULL); + while (listp != NULL); /* This will be the new maximum generation counter. */ dtv[0].counter = new_gen; @@ -982,7 +1011,7 @@ _dl_add_to_slotinfo (struct link_map *l, bool do_add) the first slot. */ assert (idx == 0); - listp = prevp->next = (struct dtv_slotinfo_list *) + listp = (struct dtv_slotinfo_list *) malloc (sizeof (struct dtv_slotinfo_list) + TLS_SLOTINFO_SURPLUS * sizeof (struct dtv_slotinfo)); if (listp == NULL) @@ -996,6 +1025,8 @@ cannot create TLS data structures")); listp->next = NULL; memset (listp->slotinfo, '\0', TLS_SLOTINFO_SURPLUS * sizeof (struct dtv_slotinfo)); + /* Synchronize with _dl_update_slotinfo. */ + atomic_store_release (&prevp->next, listp); } /* Add the information into the slotinfo data structure. */