From patchwork Tue Apr 13 08:21:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 42975 X-Patchwork-Delegate: azanella@linux.vnet.ibm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C50523968C0B; Tue, 13 Apr 2021 08:22:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C50523968C0B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1618302141; bh=sddYzLpHANSzgprb5caHzWCdxa83Q2vJxLwexOolJfM=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=KpltnXzCUs+4hpb5FJm12FK2q1tstDBf5jTn0gr+pFtviGj5w4cGqbpKD4QAUmcBU R5moU5EWJIqbtMvWobkYLb9QN0VwVXuGmqUj3Niat+Ad7JhJkg84+0n11hecPuDpdi LRjfMwKX4H+65d6XAxIN+7iXMQUWaRV7ODaVVDcE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140082.outbound.protection.outlook.com [40.107.14.82]) by sourceware.org (Postfix) with ESMTPS id 181CB3968C0B for ; Tue, 13 Apr 2021 08:22:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 181CB3968C0B Received: from DB6PR0801CA0049.eurprd08.prod.outlook.com (2603:10a6:4:2b::17) by AM6PR08MB4470.eurprd08.prod.outlook.com (2603:10a6:20b:b5::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Tue, 13 Apr 2021 08:22:13 +0000 Received: from DB5EUR03FT051.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:2b:cafe::8) by DB6PR0801CA0049.outlook.office365.com (2603:10a6:4:2b::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.16 via Frontend Transport; Tue, 13 Apr 2021 08:22:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; sourceware.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;sourceware.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT051.mail.protection.outlook.com (10.152.21.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.17 via Frontend Transport; Tue, 13 Apr 2021 08:22:13 +0000 Received: ("Tessian outbound 82c2d58b350b:v90"); Tue, 13 Apr 2021 08:22:13 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 230df4b0c435bb40 X-CR-MTA-TID: 64aa7808 Received: from 39efff76f572.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id DCC7E601-A10E-4E70-BEF8-C832CF13FA2A.1; Tue, 13 Apr 2021 08:22:06 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 39efff76f572.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 13 Apr 2021 08:22:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NgcGCpy9Qc4luuwZEBuMKSyitglk8uCH4eddp/6VVVdTvXhiS+NaeHJde9Ht2EcUbKcHHQN4s5E0WOfZx5i4O7UioNcaNw3Be0kmsA5LCGfl/DLuFM7pxQcI5wCIJJKa3LntXgULxKDacFxs0qVSb2hmoGYQeL+B5W9tQ/zj0M1s2JTltgXFlMTyscGj3DDRqVF7sn5mhHd/9wFJPmAD2xbSASPVgJgx8trlgDPnHsGXsmWbu7e4PNECKLM5B2CwyF0O2VPynTkhDSDrPxu3Ur9vxOS+eDomCBMsxygRHdvrvpqv+DRHSkwwhQCQAD9foxtuOTbamvuNdsR6eUZknw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sddYzLpHANSzgprb5caHzWCdxa83Q2vJxLwexOolJfM=; b=UtXPrV6R3pFe8qMiz5wZOTBplVQCPWYbPVT5SUR3c2yo86FlbBruuh+6sMlzdUpmtxhypdm1hwW1ibUEXCdE+SGg0+0jw9YOpkWSGYsP2MmNVnHi76eb1Cmbn5b/deYxLKn/f3tUpVKyJeUGkw+JaX9GJHDGxClMOdK8GwmEvkm/gBoqTgvm6ncZDLmwygifX1KkeEbA3leBim9vP5hyupXM+AEsW9o+/SosvQ1c5teODTnGRL7uxMe8/T2ISHZCAbRnGNAyhy84sNHn0Jfj/PI2FhufvR9HsUj0z+N6DsEBKtSP6cNNClu4o7owy89iDMCzdtR/KONKDUygVY2Ocg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: sourceware.org; dkim=none (message not signed) header.d=none;sourceware.org; dmarc=none action=none header.from=arm.com; Received: from PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) by PA4PR08MB6237.eurprd08.prod.outlook.com (2603:10a6:102:f3::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22; Tue, 13 Apr 2021 08:22:04 +0000 Received: from PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::c99f:671d:bb2c:f20b]) by PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::c99f:671d:bb2c:f20b%7]) with mapi id 15.20.4020.022; Tue, 13 Apr 2021 08:22:04 +0000 To: libc-alpha@sourceware.org Subject: [PATCH v2 14/14] RFC elf: Fix slow tls access after dlopen [BZ #19924] Date: Tue, 13 Apr 2021 09:21:58 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: X-Originating-IP: [217.140.106.55] X-ClientProxiedBy: LO4P123CA0401.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:189::10) To PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (217.140.106.55) by LO4P123CA0401.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:189::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Tue, 13 Apr 2021 08:22:04 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 577d4e60-6922-46f3-f1be-08d8fe553d83 X-MS-TrafficTypeDiagnostic: PA4PR08MB6237:|AM6PR08MB4470: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:8882;OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: tfw2lvZgR0EHQA9eBBkELzTHLhvwoQhzUn38LXXx7bZGSki7rXMh1fnHbhxtDubQOvKV7hCNl1NRDvP5HR9GyNijCXRGzuxpDVeQYMPUG6bVt0QaOjaAEu1/uaGmmxWw6Ur3dULrlkuhts8cXhGyX3RE/KVGb0YDFGYHzaY28T6cjnl6LVY1Ls3ucOs2UAddLhfSkIIpC3AJrVmTxtSlxDhMIrPQs+vxolVuUedHwkRfcc/2mZ6II4O3S1JVHmlIE99yMiZht9N9HZt5MsmZSU9AtcDvDw0OPnx24B/5e5muIanwYpFyhwhLUPtLqhJqtwYtf05879WvdOqOqFwJdCuQ1fOiqL0CBoTGxZGHn1+j8YNSk0PmVHIQ72jBxG/QTkm7UvfR8pXvlqxBi2KAcMfBguCqiRVsuOnRu3XxfT1sn4zPiodP4OEQwJm2nFSRGQJPRzHzeMDUNfZeAWy2+2s/gJbWZN9q72j2tjJz8LZQMtlhbL6kw+8w7Ib4JFf0413ISIsEBB++iwQ1+myj/3KrTAzWN6+0l/yeS5W82bCfab2oDIIdRK5oyR9rPv1QHT6eec+UidK1dtnvxYOGkk5GRNPwf8QN13s7JHFzdTF6SY5dsxgpdNdKn9WAKRz7Q9gk7lsJNl0suR2j3ZxEeaeh0pSQfXl+E9/e1GJTWTsxqudx/HCXgFs8/H9WuuJK X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PA4PR08MB6320.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(39860400002)(376002)(396003)(136003)(8936002)(86362001)(2906002)(6486002)(66946007)(16526019)(8676002)(38100700002)(38350700002)(316002)(478600001)(44832011)(6506007)(186003)(6666004)(66476007)(6916009)(2616005)(36756003)(5660300002)(66556008)(6512007)(83380400001)(956004)(69590400012)(52116002)(26005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 5naDOs9/91wVNqlj2mQaX9APgHwxHQuPOCI23XQd3ZjKTIKLlIqFRDsWsB/w1EwbJVMkblz9IifZjgAgUtMsnUvp3dmqDDvzJQJ2+gu/qg4eZOOtT8qng6j5w+WC7hzeQRpRU9eL7rZWEr/NGULBfcDZ8C/dNJvE6pTZZ86a2QDl/3Seq6XcLeDeI3VB6JCRX+g/45iWPsWWbOj9M48dN+omHiSJzh5p0s542QAKJ1BzDgJ7Q1TUowvyGjzS6iHXpTx3G/CYHrkvbDmTJoFgcSJxE0VNjzz1MPP8lmttBEnf5+YvN5G9isaMUAnfxRhzcYNHQdeM5typ5j/u4aINKP5teZdodBVhoPhgcDTOIUgUGcHOPawAAc8jOftEBKXWm7fCDjGiR9hlKUXaTwourIuPn1v3MvtKS+SBj8fa7J957C1ArlnhiWeSIttIS4vzAuRFlBd22Pt8fj3sHiKpAL1p9dmOMwqiwx9PTwJraiWq40t2MvRfhmTqEfcbPesgLkEePzLVZRk1o3ixP7nxRwfq3DnhBRzvW2ghwie+0/W39iKcfONqLK2W8uf8gnUslWAyR4y63wDrANhPrPoZ+ZlLhTqpspOQcNqo3A9aau36b0mDIoIvk06ITvHlof9AOVRjzw/8OVE7blQOYBWb9J5ZJNsJnBMglEU0crnbnWWbo6543YKxgpR8s3xH1qpvq+gMrjav5ZNlGSTn3DpD78g+XYTo8RAwANTVO3pYsMZZjazO0ntL27ieONg+w1Fk7A6j28FX+ejYtgVR6nlp5fvdWZeLRf8th8Ej7kCFOcZTUw2qVx0nCJDcF+xT4wXQyDN8atGzJ37/IVQ9S9GpD+einhGL+evMqUoSc3sWsZoUm1fJPf67JDvlKgZMRTrAzLmppZqb2VtuGEq0FEsLZAsZZBvsBR6g1aQZ9oBvlidR/GeKrlp2mkFC/CruIaQq/ccq8oxMTvIClNmBc05VDPfw9yKda6OZdJMcq4KYZUnEPcLTfXKp51XfoQ6z8zisAD9dC/TYlqUzaaxK6MLjRTRpnZ546PPaklEyPgJ8v9QsY1Wy1DuxS4i4cY/7+8imK+KTa1ir9cDefXxyBiYiez7YyNaVnI9aAt8eVjMRJtz+xKQ5247ul4jJE8aOwN9RPezydDMOtQc/q12E14tQybmZpTwTucQ5YbS3B8KAGhIGjNGW9tYK5HQNKht1JLRF21OV/eUgFkMwzyHqXjT/vugdimXCB4ZTFXhO0pJescJJ8gDxdP7Rw09jkJWxFmBmv9yUI6N3gdzP2c/Zay+a8+N0xMjw+UooEpf5eFd7geU1DjHrqvvYNf9iH2NmYK8U X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6237 Original-Authentication-Results: sourceware.org; dkim=none (message not signed) header.d=none; sourceware.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT051.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 11b6dc1e-bd25-45f2-d739-08d8fe553833 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sgmQI1nKEgJzs3s8YguEMoqm2gy3QoyD2Pe+adUMdQcuSX18q6neYXr69euhmItGFv19fPPq4ki4LnuJ28xVrjdOHmgGcxJoWSSp8xbAygO8p2boo/GZBCAV7A3PvHZx3fYP65gfgXRE09IMU7Qoa//tranxJfxZTFNhHfJWbzQDfevYZHdcsZO8WKHRANiivZFdF8P8uGl9OZC84KN4bpZOhgCbImn8N2bIdgBxUunVh49es3MJV4Ka5STNoj8/JRYklH1CUr/inc/3b5biCzS6p8GaTVLDPVyxznRWDxwCgMl36GZG8tKeP/8KbBB7MXMkeC9D8UQZEK5VUuhpC/sJQUtw/okVQeCfwFNqAjsFTXSrQGL7ZiPf9O41nEmu2vdHQG82HhrlDhoo77YYF0mTMJN8BHdH0qnNvjuQHja6Sf2WZyq0AGrDMtnPxWF39Sm/+y7ln6YJ7zpMBjiTpzCvtO1ZPB6IBiNOXlGc7/iB8SQnyfyrEQIYLpvZapJRHtze1hl+0QWnn1dISh9wy33O0MbceZqUDZe8t9ePY9/uxZu0kok3CLFTJT0966S6n9A0PDhpfX3I48I5I2dpkNgOq55sx95NPMrmz5JO2cYdQb3UkbydC77Vqo02Fna8INCtIj3OwQnifguzz9KtyEMRb2xgEe1sihZPiQ1Y0GZKl9kko+qSe5iYdgH2K/n+ X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(396003)(346002)(136003)(39860400002)(376002)(36840700001)(46966006)(2906002)(6506007)(8936002)(8676002)(86362001)(36756003)(956004)(6512007)(16526019)(478600001)(5660300002)(70586007)(336012)(6486002)(69590400012)(2616005)(26005)(6916009)(6666004)(70206006)(82310400003)(44832011)(83380400001)(316002)(36860700001)(82740400003)(186003)(47076005)(81166007)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2021 08:22:13.0577 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 577d4e60-6922-46f3-f1be-08d8fe553d83 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT051.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4470 X-Spam-Status: No, score=-13.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Szabolcs Nagy via Libc-alpha From: Szabolcs Nagy Reply-To: Szabolcs Nagy Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" In short: __tls_get_addr checks the global generation counter, _dl_update_slotinfo updates up to the generation of the accessed module. If the global generation is newer than geneneration of the module then __tls_get_addr keeps hitting the slow path that updates the dtv. Possible approaches i can see: 1. update to global generation instead of module, 2. check the module generation in the fast path. This patch is 1.: it needs additional sync (load acquire) so the slotinfo list is up to date with the observed global generation. Approach 2. would require walking the slotinfo list at all times. I don't know how to make that fast with many modules. Note: in the x86_64 version of dl-tls.c the generation is only loaded once, since relaxed mo is not faster than acquire mo load. I have not benchmarked this yet. --- elf/dl-close.c | 2 +- elf/dl-open.c | 8 ++++---- elf/dl-reloc.c | 5 ++--- elf/dl-tls.c | 28 ++++++++++++---------------- sysdeps/generic/ldsodefs.h | 3 ++- sysdeps/x86_64/dl-tls.c | 4 ++-- 6 files changed, 23 insertions(+), 27 deletions(-) diff --git a/elf/dl-close.c b/elf/dl-close.c index 9f31532f41..45f8a7fe31 100644 --- a/elf/dl-close.c +++ b/elf/dl-close.c @@ -780,7 +780,7 @@ _dl_close_worker (struct link_map *map, bool force) if (__glibc_unlikely (newgen == 0)) _dl_fatal_printf ("TLS generation counter wrapped! Please report as described in "REPORT_BUGS_TO".\n"); /* Can be read concurrently. */ - atomic_store_relaxed (&GL(dl_tls_generation), newgen); + atomic_store_release (&GL(dl_tls_generation), newgen); if (tls_free_end == GL(dl_tls_static_used)) GL(dl_tls_static_used) = tls_free_start; diff --git a/elf/dl-open.c b/elf/dl-open.c index 661f26977e..5b9816e4e8 100644 --- a/elf/dl-open.c +++ b/elf/dl-open.c @@ -400,7 +400,7 @@ update_tls_slotinfo (struct link_map *new) _dl_fatal_printf (N_("\ TLS generation counter wrapped! Please report this.")); /* Can be read concurrently. */ - atomic_store_relaxed (&GL(dl_tls_generation), newgen); + atomic_store_release (&GL(dl_tls_generation), newgen); /* We need a second pass for static tls data, because _dl_update_slotinfo must not be run while calls to @@ -417,8 +417,8 @@ TLS generation counter wrapped! Please report this.")); now, but we can delay updating the DTV. */ imap->l_need_tls_init = 0; #ifdef SHARED - /* Update the slot information data for at least the - generation of the DSO we are allocating data for. */ + /* Update the slot information data for the current + generation. */ /* FIXME: This can terminate the process on memory allocation failure. It is not possible to raise @@ -426,7 +426,7 @@ TLS generation counter wrapped! Please report this.")); _dl_update_slotinfo would have to be split into two operations, similar to resize_scopes and update_scopes above. This is related to bug 16134. */ - _dl_update_slotinfo (imap->l_tls_modid); + _dl_update_slotinfo (imap->l_tls_modid, newgen); #endif GL(dl_init_static_tls) (imap); diff --git a/elf/dl-reloc.c b/elf/dl-reloc.c index c2df26deea..427669d769 100644 --- a/elf/dl-reloc.c +++ b/elf/dl-reloc.c @@ -111,11 +111,10 @@ _dl_try_allocate_static_tls (struct link_map *map, bool optional) if (map->l_real->l_relocated) { #ifdef SHARED +// TODO: it is not clear why we need to update the DTV here, add comment if (__builtin_expect (THREAD_DTV()[0].counter != GL(dl_tls_generation), 0)) - /* Update the slot information data for at least the generation of - the DSO we are allocating data for. */ - (void) _dl_update_slotinfo (map->l_tls_modid); + (void) _dl_update_slotinfo (map->l_tls_modid, GL(dl_tls_generation)); #endif GL(dl_init_static_tls) (map); diff --git a/elf/dl-tls.c b/elf/dl-tls.c index b0257185e9..b51a4f3a19 100644 --- a/elf/dl-tls.c +++ b/elf/dl-tls.c @@ -701,7 +701,7 @@ allocate_and_init (struct link_map *map) struct link_map * -_dl_update_slotinfo (unsigned long int req_modid) +_dl_update_slotinfo (unsigned long int req_modid, size_t new_gen) { struct link_map *the_map = NULL; dtv_t *dtv = THREAD_DTV (); @@ -718,19 +718,12 @@ _dl_update_slotinfo (unsigned long int req_modid) code and therefore add to the slotinfo list. This is a problem since we must not pick up any information about incomplete work. The solution to this is to ignore all dtv slots which were - created after the one we are currently interested. We know that - dynamic loading for this module is completed and this is the last - load operation we know finished. */ - unsigned long int idx = req_modid; + created after the generation we are interested in. We know that + dynamic loading for this generation is completed and this is the + last load operation we know finished. */ struct dtv_slotinfo_list *listp = GL(dl_tls_dtv_slotinfo_list); - while (idx >= listp->len) - { - idx -= listp->len; - listp = listp->next; - } - - if (dtv[0].counter < listp->slotinfo[idx].gen) + if (dtv[0].counter < new_gen) { /* CONCURRENCY NOTES: @@ -751,7 +744,6 @@ _dl_update_slotinfo (unsigned long int req_modid) other entries are racy. However updating a non-relevant dtv entry does not affect correctness. For a relevant module m, max_modid >= modid of m. */ - size_t new_gen = listp->slotinfo[idx].gen; size_t total = 0; size_t max_modid = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); assert (max_modid >= req_modid); @@ -894,9 +886,9 @@ tls_get_addr_tail (GET_ADDR_ARGS, dtv_t *dtv, struct link_map *the_map) static struct link_map * __attribute_noinline__ -update_get_addr (GET_ADDR_ARGS) +update_get_addr (GET_ADDR_ARGS, size_t gen) { - struct link_map *the_map = _dl_update_slotinfo (GET_ADDR_MODULE); + struct link_map *the_map = _dl_update_slotinfo (GET_ADDR_MODULE, gen); dtv_t *dtv = THREAD_DTV (); void *p = dtv[GET_ADDR_MODULE].pointer.val; @@ -931,7 +923,11 @@ __tls_get_addr (GET_ADDR_ARGS) by user code, see CONCURRENCY NOTES in _dl_update_slotinfo. */ size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); if (__glibc_unlikely (dtv[0].counter != gen)) - return update_get_addr (GET_ADDR_PARAM); + { +// TODO: needs comment update if we rely on consistent generation with slotinfo + gen = atomic_load_acquire (&GL(dl_tls_generation)); + return update_get_addr (GET_ADDR_PARAM, gen); + } void *p = dtv[GET_ADDR_MODULE].pointer.val; diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h index ea3f7a69d0..614463f016 100644 --- a/sysdeps/generic/ldsodefs.h +++ b/sysdeps/generic/ldsodefs.h @@ -1224,7 +1224,8 @@ extern void _dl_add_to_slotinfo (struct link_map *l, bool do_add) /* Update slot information data for at least the generation of the module with the given index. */ -extern struct link_map *_dl_update_slotinfo (unsigned long int req_modid) +extern struct link_map *_dl_update_slotinfo (unsigned long int req_modid, + size_t gen) attribute_hidden; /* Look up the module's TLS block as for __tls_get_addr, diff --git a/sysdeps/x86_64/dl-tls.c b/sysdeps/x86_64/dl-tls.c index 24ef560b71..4ded8dd6b9 100644 --- a/sysdeps/x86_64/dl-tls.c +++ b/sysdeps/x86_64/dl-tls.c @@ -40,9 +40,9 @@ __tls_get_addr_slow (GET_ADDR_ARGS) { dtv_t *dtv = THREAD_DTV (); - size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); + size_t gen = atomic_load_acquire (&GL(dl_tls_generation)); if (__glibc_unlikely (dtv[0].counter != gen)) - return update_get_addr (GET_ADDR_PARAM); + return update_get_addr (GET_ADDR_PARAM, gen); return tls_get_addr_tail (GET_ADDR_PARAM, dtv, NULL); }