From patchwork Mon Feb 15 12:01:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 42037 X-Patchwork-Delegate: azanella@linux.vnet.ibm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2CAB13951847; Mon, 15 Feb 2021 12:01:28 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2CAB13951847 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1613390488; bh=LaKYoWqKLO5NDHPyszwt0zNEKgRV794+tn5Hd1VfQTg=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ChlpN6nxT+HEujSmJket/kCAZElfoIfT4iaX1OQzamOzam1sovmWftrrmdkRXdDQS h9QYidDwqkF8cgNo68bNQxU1CH+XC7SX+R8xjQFE7lVsuHXSOPoKT7ZJpKazGhBqU6 5lWoUGpGrzCQ2BdnnVTqJDi6/BEr3bMPhzZZzP8k= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60066.outbound.protection.outlook.com [40.107.6.66]) by sourceware.org (Postfix) with ESMTPS id 4AC073951825 for ; Mon, 15 Feb 2021 12:01:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 4AC073951825 Received: from AM5PR04CA0027.eurprd04.prod.outlook.com (2603:10a6:206:1::40) by VI1PR08MB3792.eurprd08.prod.outlook.com (2603:10a6:803:bf::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25; Mon, 15 Feb 2021 12:01:23 +0000 Received: from AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:1:cafe::59) by AM5PR04CA0027.outlook.office365.com (2603:10a6:206:1::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Mon, 15 Feb 2021 12:01:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; sourceware.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;sourceware.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT042.mail.protection.outlook.com (10.152.17.168) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Mon, 15 Feb 2021 12:01:22 +0000 Received: ("Tessian outbound 28c96a6c9d2e:v71"); Mon, 15 Feb 2021 12:01:21 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 8a7a57cc342bb582 X-CR-MTA-TID: 64aa7808 Received: from e360f0d04376.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 9BBDF676-E178-4854-9D83-2D92A5420DA7.1; Mon, 15 Feb 2021 12:01:12 +0000 Received: from FRA01-MR2-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e360f0d04376.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 15 Feb 2021 12:01:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GfAeTdmMZIIspcAb6f6TpCZYDbCGLrXY/vw8NPRPT/UFd3F2Bg6kRsJACvDlIShUCs3HsW//75l8n+VKZVRJK7J65/P1F19khsiHxQUI2Edb3c6AWSwanD+7QIXw6TxulLb/WXlbEOD5mpfPYZaJxGnblfseNJVyoTBqM+jn3sTP1ygixi1vRvRetuMseRRLcEt4mmxKLKC1IAGDlGhocUKaNO/owMFAKoi1okC4NZZQF7HGQhFggMZbSgAZ+K8s2vb6lBK3G4ge+S80PBEfemjTTa/IAwCeioAgdUZiQ1+7KZOZzLaU/GfBysD7HiADXbjOmjCylckpa4VefWNYBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LaKYoWqKLO5NDHPyszwt0zNEKgRV794+tn5Hd1VfQTg=; b=ipUUCYG9frATpJUmCdjX3aKvx/zSqGXmVRCs4ADavxXh4woIX6dG2KWDk7eJ67uF4nLLoqjjXL1GMS7Pk97hcchl5YhaWPA6dc/ld+4kh+i9esiJ4VoA3i8wy1fG3Ozb0HCbvoTiQM1re/X4FKxW9PHhHCCCLkNs8OS3COkm09PY23gZxcjMsRlrhNZkQAQw88+JmylKNgB0B2jLgHhc0r2PFHDRcNu7iKDDajhT0vYHXCGBq9noXTngYq8g/CsB8AqZIs07z4D3gfcsnAjenB+551cJIh6L6hct5W9CCNJeWJdOVojmC5NpXL4DTZKdjSvdkTqfkt0I0U8khZeQ2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: sourceware.org; dkim=none (message not signed) header.d=none;sourceware.org; dmarc=none action=none header.from=arm.com; Received: from PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) by PR2PR08MB4809.eurprd08.prod.outlook.com (2603:10a6:101:1a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.27; Mon, 15 Feb 2021 12:01:10 +0000 Received: from PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::60f0:3773:69b8:e336]) by PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::60f0:3773:69b8:e336%2]) with mapi id 15.20.3846.042; Mon, 15 Feb 2021 12:01:10 +0000 To: libc-alpha@sourceware.org Subject: [PATCH 08/15] elf: Fix data races in pthread_create and TLS access [BZ #19329] Date: Mon, 15 Feb 2021 12:01:03 +0000 Message-Id: <63e1d0eb071f68e74ba10d4532fb486b5aec8fd7.1613390045.git.szabolcs.nagy@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: X-Originating-IP: [217.140.106.49] X-ClientProxiedBy: LO4P123CA0500.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1ab::19) To PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (217.140.106.49) by LO4P123CA0500.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1ab::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.38 via Frontend Transport; Mon, 15 Feb 2021 12:01:09 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b365fecf-e324-4633-c680-08d8d1a969d9 X-MS-TrafficTypeDiagnostic: PR2PR08MB4809:|VI1PR08MB3792: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: MW2f2EsIX6ZXCoD4Fge/+IBBhygcplICPeKWwIP1gwr7zL8dU4hpSNHvYh5kiR12TPfRlYoAkxqlPz2GQhLUD80AvaSywiXCebOTS0TlGRC3BpwxByOSBy1ZWb7phHpk4TQatuCn19Vkq6cxm+CBqI0yVrUyR6Bxr+BL0OKoPKVMDS0fCrtQkD3sdwakD+vl7Wec+oAhOmVU4vgWXJ96Lkr6YsniWLwAPOImTHHWRzANrGPbw2g1nVr8wlS/c464tGt0PExtGDqRbwL5uF9USYzKL4Q/ISqwCVm0QeZh/BE1O4ZF5xsBb7yTYdjIBOrm2UyQBsJLMIBRFVtddE0ZIGN6Q5R6QyBdx9x4UwWMMmDgucMml14mFildH91E2dfbKSrfk2I3JIvZS//O4K/ww5BaJ+n5iapq8l4D5dGs9qKJACt+C8ZaufyYqx8s/0uHlN+Qgm7KnlQNVYrx+8NyH2FP0hOnRL0Dc+NRv1kZV/ieRKLJ8nyMehCBr5d76RDaOTs7BEXXvbfj65wcBjwrKeBPPrsmyBRKUT/hERhwNUITHNxKGQu6hCxOirKDEIBMi06eo2Ud3upKGXL8YqKlTg== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PA4PR08MB6320.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(376002)(346002)(136003)(39850400004)(396003)(8676002)(6486002)(6512007)(478600001)(66556008)(6916009)(2616005)(8936002)(66946007)(66476007)(86362001)(2906002)(69590400012)(6666004)(52116002)(26005)(44832011)(6506007)(956004)(36756003)(83380400001)(186003)(16526019)(5660300002)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: eLZVHhhCOc9EisQwKdheTcxZtSX0bIVIOFX8lB/iMXTlyGoj3hLRy1ZyfpgZH7zo94+fvzrkR2DMTcoazJ3+DFg77dVVBfhsCBGKrDduntkes6V4/R8YrEeWkIO70scwxzjWF2nUFGTM6BoFeNzv+NpFth53lIYAz8b1oUPUbfP6uP5OlU0i+R7wlbnIJQ2FKvNA7eBIFEFi9u4BrMb774JKCcYGsxm4YIzPv6xhb9j4xIkK3VGXzWVU5YbZ1cWQZ0517ua9lnd6yFpavEf7GnB8Ncr35ckTOW2IoAcHIMyC3YrPE/fN3g9sS86eh+bijv34hh+XBn1IK4/MGfkC0zS1VK1ksaWtYRdmOG+9wTZramB3o5pxMMIYoRW4lXEThCQXDRAQ42BZpjL6m4me6lTXQ4+WnqPxC0CuzlqSWQ4Xy4D9laxkpxwSqT4yx0WRt/sLbVDQqydzw1bkYOSLUHecI5uzs9LX6iDWfMDheHpHza28LKuaNOgI0OG2LMd5F/BOmLVp0EmFDGyEw8bQVLU+jqJpuuDdh+uymnk7U9xbjLkiwXf2xK59E+czh1vJGlefulzan3cQk9MbBnuhnuCDA/UNG4deziecGFEq/SClxjKGggeTWB7LkNVqEdNzPuYv8qeDPb9OTdWYPokoJySJ2mLx/3HY6hzq2BrHkEUk2IU1dI/deqVhk6OFSGZbAlMAvr4A2JiquHw9oX1aKC4Z9B4tb1GsL5fudCT55p/vM5G15kLwqtj8atDBjTwpOUXfjk6cpaHNSfOvrY8gS5hhv6px1C4Mfcsa+1hcjvTWzVhRiAczUkr7Xle5/QIOWoxR7ZBmbKvOANEbzNsUFWmZmp+itoa4tAQp5MpGzvIItrz3YD/rHC9hou6Vx5n0uC7H2tzn0Lp2F3Z22EeuWgVwzsIPY61XCjWPsI6PlJbyjNuL8Z9muRMfJKpH2kJb7lp6CN64l8U8ZShdImEWOcMf/fN1FsJeR6qPhVWZ3LgjV25uG52QGGlCEPTXQQD4HEw6TCJfkcJaHpAGdZV61g+gJWoIYZsKWBKl4BXjEu6pLz/ianmklJNz0ZMZ+Ise363+lDwY5R3UBQOn5hoKAc8qrmc/tEMmGzX/osHpJNWn/58VYEAhUOnqvdO0/io7E9s5AV8CdL+9a74tDyyRefl9dmCGTXH2n+SwIviB5mGsX2ZsahWZxcH3SNIsnxYDiA0l78su02Oig5DOGyanTjlDPZo1Bgq04lOpFOIrEIKVJb6jLfKV8onz2HQFwGXWB39NGZw9zljJKDoGBh83mzj7kv6TJluKK+wKgtqmsaSDplidAkI77Ycb/uvCopRq X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4809 Original-Authentication-Results: sourceware.org; dkim=none (message not signed) header.d=none; sourceware.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 93e758bd-46d2-44f9-a656-08d8d1a9620f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5GPmBo2iut0bnlURFcdaddtUwLpwuZfBVurxD5nopclXgz+aIZyqE1ULaV4W4JhSXyNM53RIEfdGAKirFFtxmOoiFeY8WTz87hDKvlahE332Jy1mO/P97SYrPZJkSIcw/75wRKZOguN/IrRM1VpROifiCSnNKwFuukHSJyzpnR9iK01xDsiXYvoux9MrPoTNhPXWpCk5gEbRc2LrqgfvSbPCgzC5RkfDRQUyvbj/rVLl2CEveDZIes9PSd8v6rIeabHQjVezjo0NujA6HD7Kzgeeg3WdZ4NN9u9w0CKoFI/w/uB9zWJ3aqayiCP07+S5LedkBL5S1QLdw8b9Y1T0ANzutvV6BL1spNSKtrwmqbSevaQFkJAWTjRdQuCKxDO1oe08qbc5HVgF78YpGVrnb+ZwHhMKOfOdautYtMIUBoiJUhG0AofPJBal7Q/QV5UheTE//hL2kc2lS4QioJ2oiCur6u/1CU6gnupLgojxpCiDCF8VoveRdso/n/P42LY2RJhzkt21eQriKwWDqEb8Gqq/iBo3wjCgaTGv1OW2R855C2a3lwOZsT1gPS7c1NU7fkAT2Ajzka/R53TurPkzen+zUqkqyddNUzlAxKxuUljB02l7zfrV7/hODusyXmmQNKgyQdkRpWo+ZXeshYdF1k1w6eidH5K3CgNXev6iN4lzZzhMATN1f94i3q7DKC5e X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(346002)(376002)(136003)(39850400004)(396003)(46966006)(36840700001)(36756003)(8936002)(6486002)(16526019)(36860700001)(336012)(2616005)(86362001)(26005)(956004)(6666004)(44832011)(8676002)(2906002)(478600001)(82310400003)(5660300002)(6506007)(82740400003)(186003)(70206006)(83380400001)(6512007)(69590400012)(6916009)(47076005)(81166007)(356005)(70586007)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2021 12:01:22.7743 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b365fecf-e324-4633-c680-08d8d1a969d9 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT042.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3792 X-Spam-Status: No, score=-14.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Szabolcs Nagy via Libc-alpha From: Szabolcs Nagy Reply-To: Szabolcs Nagy Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" DTV setup at thread creation (_dl_allocate_tls_init) is changed to take the dlopen lock, GL(dl_load_lock). Avoiding data races here without locks would require design changes: the map that is accessed for static TLS initialization here may be concurrently freed by dlclose. That use after free may be solved by only locking around static TLS setup or by ensuring dlclose does not free modules with static TLS, however currently every link map with TLS has to be accessed at least to see if it needs static TLS. And even if that's solved, still a lot of atomics would be needed to synchronize DTV related globals without a lock. So fix both bug 19329 and bug 27111 with a lock that prevents DTV setup running concurrently with dlopen or dlclose. _dl_update_slotinfo at TLS access still does not use any locks so CONCURRENCY NOTES are added to explain the synchronization. The early exit from the slotinfo walk when max_modid is reached is not strictly necessary, but does not hurt either. An incorrect acquire load was removed from _dl_resize_dtv: it did not synchronize with any release store or fence and synchronization is now handled separately at thread creation and TLS access time. There are still a number of racy read accesses to globals that will be changed to relaxed MO atomics in a followup patch. This should not introduce regressions compared to existing behaviour and avoid cluttering the main part of the fix. Not all TLS access related data races got fixed here: there are additional races at lazy tlsdesc relocations see bug 27137. --- elf/dl-tls.c | 63 +++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 47 insertions(+), 16 deletions(-) diff --git a/elf/dl-tls.c b/elf/dl-tls.c index f8b32b3ecb..33c06782b1 100644 --- a/elf/dl-tls.c +++ b/elf/dl-tls.c @@ -471,14 +471,11 @@ extern dtv_t _dl_static_dtv[]; #endif static dtv_t * -_dl_resize_dtv (dtv_t *dtv) +_dl_resize_dtv (dtv_t *dtv, size_t max_modid) { /* Resize the dtv. */ dtv_t *newp; - /* Load GL(dl_tls_max_dtv_idx) atomically since it may be written to by - other threads concurrently. */ - size_t newsize - = atomic_load_acquire (&GL(dl_tls_max_dtv_idx)) + DTV_SURPLUS; + size_t newsize = max_modid + DTV_SURPLUS; size_t oldsize = dtv[-1].counter; if (dtv == GL(dl_initial_dtv)) @@ -524,11 +521,14 @@ _dl_allocate_tls_init (void *result) size_t total = 0; size_t maxgen = 0; + /* Protects global dynamic TLS related state. */ + __rtld_lock_lock_recursive (GL(dl_load_lock)); + /* Check if the current dtv is big enough. */ if (dtv[-1].counter < GL(dl_tls_max_dtv_idx)) { /* Resize the dtv. */ - dtv = _dl_resize_dtv (dtv); + dtv = _dl_resize_dtv (dtv, GL(dl_tls_max_dtv_idx)); /* Install this new dtv in the thread data structures. */ INSTALL_DTV (result, &dtv[-1]); @@ -596,6 +596,7 @@ _dl_allocate_tls_init (void *result) listp = listp->next; assert (listp != NULL); } + __rtld_lock_unlock_recursive (GL(dl_load_lock)); /* The DTV version is up-to-date now. */ dtv[0].counter = maxgen; @@ -730,12 +731,29 @@ _dl_update_slotinfo (unsigned long int req_modid) if (dtv[0].counter < listp->slotinfo[idx].gen) { - /* The generation counter for the slot is higher than what the - current dtv implements. We have to update the whole dtv but - only those entries with a generation counter <= the one for - the entry we need. */ + /* CONCURRENCY NOTES: + + Here the dtv needs to be updated to new_gen generation count. + + This code may be called during TLS access when GL(dl_load_lock) + is not held. In that case the user code has to synchrnize with + dlopen and dlclose calls of relevant modules. A module m is + relevant if the generation of m <= new_gen and dlclose of m is + synchronized: a memory access here happens after the dlopen and + before the dlclose of relevant modules. The dtv entries for + relevant modules need to be updated, other entries can be + arbitrary. + + This e.g. means that the first part of the slotinfo list can be + accessed race free, but the tail may be concurrently extended. + Similarly relevant slotinfo entries can be read race free, but + other entries are racy. However updating a non-relevant dtv + entry does not affect correctness. For a relevant module m, + max_modid >= modid of m. */ size_t new_gen = listp->slotinfo[idx].gen; size_t total = 0; + size_t max_modid = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); + assert (max_modid >= req_modid); /* We have to look through the entire dtv slotinfo list. */ listp = GL(dl_tls_dtv_slotinfo_list); @@ -745,12 +763,14 @@ _dl_update_slotinfo (unsigned long int req_modid) { size_t modid = total + cnt; + /* Later entries are not relevant. */ + if (modid > max_modid) + break; + size_t gen = listp->slotinfo[cnt].gen; if (gen > new_gen) - /* This is a slot for a generation younger than the - one we are handling now. It might be incompletely - set up so ignore it. */ + /* Not relevant. */ continue; /* If the entry is older than the current dtv layout we @@ -767,7 +787,7 @@ _dl_update_slotinfo (unsigned long int req_modid) continue; /* Resize the dtv. */ - dtv = _dl_resize_dtv (dtv); + dtv = _dl_resize_dtv (dtv, max_modid); assert (modid <= dtv[-1].counter); @@ -789,8 +809,17 @@ _dl_update_slotinfo (unsigned long int req_modid) } total += listp->len; + if (total > max_modid) + break; + + /* Synchronize with _dl_add_to_slotinfo. Ideally this would + be consume MO since we only need to order the accesses to + the next node after the read of the address and on most + hardware (other than alpha) a normal load would do that + because of the address dependency. */ + listp = atomic_load_acquire (&listp->next); } - while ((listp = listp->next) != NULL); + while (listp != NULL); /* This will be the new maximum generation counter. */ dtv[0].counter = new_gen; @@ -982,7 +1011,7 @@ _dl_add_to_slotinfo (struct link_map *l, bool do_add) the first slot. */ assert (idx == 0); - listp = prevp->next = (struct dtv_slotinfo_list *) + listp = (struct dtv_slotinfo_list *) malloc (sizeof (struct dtv_slotinfo_list) + TLS_SLOTINFO_SURPLUS * sizeof (struct dtv_slotinfo)); if (listp == NULL) @@ -996,6 +1025,8 @@ cannot create TLS data structures")); listp->next = NULL; memset (listp->slotinfo, '\0', TLS_SLOTINFO_SURPLUS * sizeof (struct dtv_slotinfo)); + /* Synchronize with _dl_update_slotinfo. */ + atomic_store_release (&prevp->next, listp); } /* Add the information into the slotinfo data structure. */