From patchwork Mon Feb 15 12:01:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 42038 X-Patchwork-Delegate: azanella@linux.vnet.ibm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2FCE53951863; Mon, 15 Feb 2021 12:01:52 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2FCE53951863 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1613390512; bh=FCtZAFvh+McMUjGW5VDz18+0p1vLtTuD4PBZpyIQ1gc=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=u2Td9rpsYo3mMYz6Q4GtELizGNJ+hrj7pI/45Ko+ElLkQNJRKyuuSNCopjNmCpFjP dhmKNOSmmu1DLL6F6PpWmSsAOZHumOAr8dv/+KV73jaSscGpis/TPWuu9yZr6Cotgo zJP4VYIcyqMXiJrFwxWHQVbKkfXGwHgx3c80hGR4= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2072.outbound.protection.outlook.com [40.107.22.72]) by sourceware.org (Postfix) with ESMTPS id 2768C3951825 for ; Mon, 15 Feb 2021 12:01:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 2768C3951825 Received: from DB7PR05CA0057.eurprd05.prod.outlook.com (2603:10a6:10:2e::34) by AM6PR08MB4997.eurprd08.prod.outlook.com (2603:10a6:20b:e8::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.27; Mon, 15 Feb 2021 12:01:43 +0000 Received: from DB5EUR03FT047.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:2e:cafe::34) by DB7PR05CA0057.outlook.office365.com (2603:10a6:10:2e::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.26 via Frontend Transport; Mon, 15 Feb 2021 12:01:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; sourceware.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;sourceware.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT047.mail.protection.outlook.com (10.152.21.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Mon, 15 Feb 2021 12:01:43 +0000 Received: ("Tessian outbound 4d8113405d55:v71"); Mon, 15 Feb 2021 12:01:43 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 58758d6e978af435 X-CR-MTA-TID: 64aa7808 Received: from b39eb2cf04d3.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id E2168774-2FBC-4356-93B8-0F990026E2F5.1; Mon, 15 Feb 2021 12:01:34 +0000 Received: from FRA01-MR2-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b39eb2cf04d3.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 15 Feb 2021 12:01:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ez232BQ8XOLdqI7+RKYW3XACqx+BQiGSjYLpSG6l9dRTilq557fEXF5aSzODEzhA+qMm6ndq/HYbuMZAbtTLsE4FS4OUN6Fj4/1MtAsSmdvbbs634rr3ZuuFaGiqw2MZmLYR9z1rg3142aY9lcq6q/HmcewKqjA33WiXB4z1LqFutej0dkOCiWvt4uUrWqMg3E5sJBQHVipi3PxW6KXeK503ESDBwEM8mrWdM/JmWl/MMuHSL3h/h3xON0HXd8UbDjCuuRmeaX8NnGgKd3yFkmiuyYIMQCpCY7ZyyBbzpCleJeUkOY5t26sYULUXW6Vrf2AmVgOJLsEwUih0h+tKqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FCtZAFvh+McMUjGW5VDz18+0p1vLtTuD4PBZpyIQ1gc=; b=Scb1XDrPKtkfMfSLjpkeoYaJJL2sfJDQR9YErop1/mGjbw56z9R80jscj193xOLp89G2wBwijj1/eM1lmFzetZk6QpPdvCdpGkBFT4lPxn66UdpitM76ThK+w07ASJxrZbt7LHoRTGUsDb/iacoAi7VzmwEnv90uP93aIqXQI4BEyf5IlOjFAZc/bNn9oiDUMa3XMyu8eTvy41BRcnUrHQRZSXolIAra7UbQ1AEKGVmx1P9xOJyjcC56VNfEgOxyPETmZEhppVbT5mTX53OIjyaaNA1LAddfXimYSUB1cprt3pdRbZb43k/Hjg9u2Q6QSdZezg/+1/xDgIrL6StouQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: sourceware.org; dkim=none (message not signed) header.d=none;sourceware.org; dmarc=none action=none header.from=arm.com; Received: from PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) by PR2PR08MB4809.eurprd08.prod.outlook.com (2603:10a6:101:1a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.27; Mon, 15 Feb 2021 12:01:32 +0000 Received: from PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::60f0:3773:69b8:e336]) by PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::60f0:3773:69b8:e336%2]) with mapi id 15.20.3846.042; Mon, 15 Feb 2021 12:01:32 +0000 To: libc-alpha@sourceware.org Subject: [PATCH 09/15] elf: Use relaxed atomics for racy accesses [BZ #19329] Date: Mon, 15 Feb 2021 12:01:21 +0000 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: X-Originating-IP: [217.140.106.49] X-ClientProxiedBy: SA9PR11CA0023.namprd11.prod.outlook.com (2603:10b6:806:6e::28) To PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (217.140.106.49) by SA9PR11CA0023.namprd11.prod.outlook.com (2603:10b6:806:6e::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Mon, 15 Feb 2021 12:01:31 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: a49a31b1-f671-454a-d066-08d8d1a97635 X-MS-TrafficTypeDiagnostic: PR2PR08MB4809:|AM6PR08MB4997: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: chrLEU2/sqNdX6NwzOP/bQhksa+WS9U2VnYNAAvK47z0Dr/zotXcEX084Hy5goeKmiaWrOkLXMQ3stlxyk9ylhqoFg60yEqq7nlNITA7ItFjzd+oWJ6UWXytQS8AkFQ8sjSvxlEsM5AOWeIsFsEoExTDUsnYosGSc+CwjtzpeufTO1apjavv7L8fkwXhoXcoIS7xOkn34yZ3dn5CWb/cEbEK/Q3AtxDx8OfhZW6zRl0LQrxI87b0W6wAXxLdOHl+3Fsw/j8JcqDaOzvR1aIxGSuLpElchf0OPu4/TTA1iC2hl/cHFkgAfPQGfXTwUwae1Fx4ZKQpgDUlmz47DX7OQC44eKfHddULG8zVwfZR1ngQLMDA4ceUGYwjP/8hfymf1Gq6CBbG64uC/56GXHh1fWwGohqions2Gv0Mo+9oYbpEBOzZYqojk6mnVSbDh5JSwyHoB9EYE85SJg1xScU3XG6fMGXY6PARDOBZAMQWdUo0VRN+zIsHrKdG+mYKu5gT/gdky4IOZFubBzTfxaerUJ/WS3/f2xx4rKXkU7RtiSCjYwl9auKQF0KuFS5jAx23K8nmzG22Zq7xa9/3wzoYLw== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PA4PR08MB6320.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(39850400004)(136003)(376002)(366004)(346002)(44832011)(52116002)(26005)(186003)(83380400001)(5660300002)(316002)(16526019)(956004)(36756003)(6506007)(6916009)(2616005)(66556008)(8936002)(66946007)(6486002)(6512007)(8676002)(478600001)(66476007)(69590400012)(6666004)(2906002)(86362001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: fnOegUOsXkzIoChtE+NzwgKBnBOmixxoQRPNBk8RDJQQSTHimji+G1R56gqx5HtGDFgH3w/zfO6m8RNiyaFZM3sQ88xPeGB8CjDj46gypdbSZdtR8X+42OtUlNJETEKSyJBLYI1/Yy8S34G1mcUIHomfNfcOjGJ8ksoY/rbXEOKMSvODFfgmikXu12AVGaACrRAhYe+4vYZWtJWen74Y2nSOzXeNRw5do1SLQulaZi05XU6cy5SxEhDShWeWDxaoA3PWFgyPTPjWY+0JKgSlEgCWXEwbHrwtso8kIRXggqR1TeciOFNJFm0NnCe8xD60J/y4Cp/ZpUDm1gAJyTU6n5T+TDmgJ97nKO+qLSLK5Cppxl+vTOtwYlMAALFq8973zLb9PgYww3p4Z8VBNoVk8CwBX/bY4wWoJ3u0+cb95ziTdFl7U1NqcO90l7g9lLP823EpVEo3mlbSz9CjwoI7W1D2z/u9n6Cv8RgUcNHhNhGhVnQQ9I1yVHv1fV07bQkz4S4onTBlV6Ib6kfRlBFP2H4BpmQ5voJ2RIRIr5M7c0fwmMFjSBSAQz2oHr7ljGN8B7UhYM1+5dc1t6mCOqdekK5tUkXVEIofqB8s/+v/OOpiRs+a2Fw5qoqC2dp9AIVavMBEWdwOBp9fync+Jzamr8avp4p6MENecVfqMPMgLqyNr5qkC5kMBLVOEN37TOQM0oAC/8R1gWz86FkMFkM9j4NW2yg78h1ADZBJ4dbOiir6DNCaZhWDySgFfTJvYTXDdwdjYKc61jtrEWWAD1msDKp2tRi6oh/ek3EsYqZqGLQpKg+kCJyqeRVvASSovUIKHuH8G74BdYNTsWJ2wnsnNzEc2Rtrzuj8w17NYTsW5fCMSVuYuuhq+ZaHHqX7CNrbeV0EHBQsMLMNBp7Ypfqpa6iNPLQjf6pUibOkOxKK+u7aMqvWFiZx3y+mPEd1ixJwaTURCF0+QFcCEqOVRoLp1mZaN0KAxbnfmuzZsAPpoJtm98/EEMhr/4Daj4VlDvTwe1FsCB22vmsxM9qR2mvoPQjOnnoKqYGdEr/diVkuAx73NaIOqm/eij2Y0soXPTFdmEDZkt0DNtVK9e9Y1VYgSpAATWBlXX0S7dNR2pGjbwBO/cRgIwMgUN8sVI5nJZo8a0y5vmMqR8VQb7CGOYL7JM6B732UzpNzuUSg5Jz+IPp2oEuYJgk8XH0qH/KyWaieQAb0vuoupqYepztucmGkSqxppQkzy2ors31Tt4jTla8M1RpkUxn/NZS1SqXcCdUoGuTFRzHABDgtZZDsaVtWZOwoWtGWjr7ZFBgrgvrv5bR5YjrLkNfqUfzoXwBAumx1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4809 Original-Authentication-Results: sourceware.org; dkim=none (message not signed) header.d=none; sourceware.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT047.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 87073d25-2dfe-4ba6-93b6-08d8d1a96f7f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XeqphuACTVzLwvFbJtAqoftjbwIVX8ikh0I9NmtuhaoeZ/3NIKwDwzA46eEUKMclXE7Rlx0C472KGYgcRkCmw7OTdWXckFUE6p6Im0RpOSTklz2Z1ykgsYo6llb4XTXqZt3XWAYJMbvhUa48+tOyILk0pxhisRHCOXU2mWIftzGOcdGsxicQLOLJR07GreAti3GvChWkTGGGNYG1BIGVXEUNsRHkMqGa5PkD/Xzx9vqNQIxrgINlGGCjoAtP3uukriJpTOT1yuCqWmKsZox2bmmX3OEx/agW8zO3DuK+uSdtihg7fFxqTX6eP1v0orxfbubZst+kqn7Z8mh1hMHI3h9dymX+wOnnHEcGeaa32RjJMaSY/FKUvwtbQTMke/gb1mu/Zh3trBUBUv1TrDKmp0T5b6Yq6oFT5TDUdGDrOGVc311DENdRNuxU0ltrsU7oK/2WbbU1Pn6u3AQFd9EU9LY4MtLxWelSYs27vP75x/ULcn/bP45W7iz0qoshSKkdUDZ+jbMF3lvi3EiXDEsQfJ5KLoUPvA8OD263eJeOtx8Uo99desG8p9ZWlH4n6UyDmE4abX9wJXqRsBZutdoPIl3yE0MShR6wVYOsmkNSd9DOUJr4wrztLLZr1nflHf7izzPpM6Rzng4V91UmKpTpnbouI/WHnp+DJ9Okh7EXNYDRjyb303O0Ej+m7r4g8Api X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(39850400004)(376002)(396003)(136003)(346002)(46966006)(36840700001)(16526019)(6506007)(186003)(6512007)(8936002)(26005)(36756003)(2906002)(47076005)(86362001)(36860700001)(8676002)(478600001)(82310400003)(5660300002)(70206006)(6486002)(81166007)(82740400003)(83380400001)(2616005)(6666004)(336012)(44832011)(70586007)(956004)(6916009)(316002)(69590400012)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2021 12:01:43.5761 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a49a31b1-f671-454a-d066-08d8d1a97635 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT047.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4997 X-Spam-Status: No, score=-14.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Szabolcs Nagy via Libc-alpha From: Szabolcs Nagy Reply-To: Szabolcs Nagy Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" This is a follow up patch to the fix for bug 19329. This adds relaxed MO atomics to accesses that are racy, but relaxed MO is enough. --- elf/dl-close.c | 20 +++++++++++++------- elf/dl-open.c | 5 ++++- elf/dl-tls.c | 31 +++++++++++++++++++++++-------- 3 files changed, 40 insertions(+), 16 deletions(-) diff --git a/elf/dl-close.c b/elf/dl-close.c index c51becd06b..3720e47dd1 100644 --- a/elf/dl-close.c +++ b/elf/dl-close.c @@ -79,9 +79,10 @@ remove_slotinfo (size_t idx, struct dtv_slotinfo_list *listp, size_t disp, { assert (old_map->l_tls_modid == idx); - /* Mark the entry as unused. */ - listp->slotinfo[idx - disp].gen = GL(dl_tls_generation) + 1; - listp->slotinfo[idx - disp].map = NULL; + /* Mark the entry as unused. These can be read concurrently. */ + atomic_store_relaxed (&listp->slotinfo[idx - disp].gen, + GL(dl_tls_generation) + 1); + atomic_store_relaxed (&listp->slotinfo[idx - disp].map, NULL); } /* If this is not the last currently used entry no need to look @@ -96,8 +97,8 @@ remove_slotinfo (size_t idx, struct dtv_slotinfo_list *listp, size_t disp, if (listp->slotinfo[idx - disp].map != NULL) { - /* Found a new last used index. */ - GL(dl_tls_max_dtv_idx) = idx; + /* Found a new last used index. This can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), idx); return true; } } @@ -571,7 +572,9 @@ _dl_close_worker (struct link_map *map, bool force) GL(dl_tls_dtv_slotinfo_list), 0, imap->l_init_called)) /* All dynamically loaded modules with TLS are unloaded. */ - GL(dl_tls_max_dtv_idx) = GL(dl_tls_static_nelem); + /* Can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), + GL(dl_tls_static_nelem)); if (imap->l_tls_offset != NO_TLS_OFFSET && imap->l_tls_offset != FORCED_DYNAMIC_TLS_OFFSET) @@ -769,8 +772,11 @@ _dl_close_worker (struct link_map *map, bool force) /* If we removed any object which uses TLS bump the generation counter. */ if (any_tls) { - if (__glibc_unlikely (++GL(dl_tls_generation) == 0)) + size_t newgen = GL(dl_tls_generation) + 1; + if (__glibc_unlikely (newgen == 0)) _dl_fatal_printf ("TLS generation counter wrapped! Please report as described in "REPORT_BUGS_TO".\n"); + /* Can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_generation), newgen); if (tls_free_end == GL(dl_tls_static_used)) GL(dl_tls_static_used) = tls_free_start; diff --git a/elf/dl-open.c b/elf/dl-open.c index ab7aaa345e..83b8e96a5c 100644 --- a/elf/dl-open.c +++ b/elf/dl-open.c @@ -395,9 +395,12 @@ update_tls_slotinfo (struct link_map *new) } } - if (__builtin_expect (++GL(dl_tls_generation) == 0, 0)) + size_t newgen = GL(dl_tls_generation) + 1; + if (__builtin_expect (newgen == 0, 0)) _dl_fatal_printf (N_("\ TLS generation counter wrapped! Please report this.")); + /* Can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_generation), newgen); /* We need a second pass for static tls data, because _dl_update_slotinfo must not be run while calls to diff --git a/elf/dl-tls.c b/elf/dl-tls.c index 33c06782b1..c4466bd9fc 100644 --- a/elf/dl-tls.c +++ b/elf/dl-tls.c @@ -175,7 +175,9 @@ _dl_next_tls_modid (void) /* No gaps, allocate a new entry. */ nogaps: - result = ++GL(dl_tls_max_dtv_idx); + result = GL(dl_tls_max_dtv_idx) + 1; + /* Can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), result); } return result; @@ -359,10 +361,12 @@ allocate_dtv (void *result) dtv_t *dtv; size_t dtv_length; + /* Relaxed MO, because the dtv size is later rechecked, not relied on. */ + size_t max_modid = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); /* We allocate a few more elements in the dtv than are needed for the initial set of modules. This should avoid in most cases expansions of the dtv. */ - dtv_length = GL(dl_tls_max_dtv_idx) + DTV_SURPLUS; + dtv_length = max_modid + DTV_SURPLUS; dtv = calloc (dtv_length + 2, sizeof (dtv_t)); if (dtv != NULL) { @@ -767,7 +771,7 @@ _dl_update_slotinfo (unsigned long int req_modid) if (modid > max_modid) break; - size_t gen = listp->slotinfo[cnt].gen; + size_t gen = atomic_load_relaxed (&listp->slotinfo[cnt].gen); if (gen > new_gen) /* Not relevant. */ @@ -779,7 +783,8 @@ _dl_update_slotinfo (unsigned long int req_modid) continue; /* If there is no map this means the entry is empty. */ - struct link_map *map = listp->slotinfo[cnt].map; + struct link_map *map + = atomic_load_relaxed (&listp->slotinfo[cnt].map); /* Check whether the current dtv array is large enough. */ if (dtv[-1].counter < modid) { @@ -923,7 +928,12 @@ __tls_get_addr (GET_ADDR_ARGS) { dtv_t *dtv = THREAD_DTV (); - if (__glibc_unlikely (dtv[0].counter != GL(dl_tls_generation))) + /* Update is needed if dtv[0].counter < the generation of the accessed + module. The global generation counter is used here as it is easier + to check. Synchronization for the relaxed MO access is guaranteed + by user code, see CONCURRENCY NOTES in _dl_update_slotinfo. */ + size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); + if (__glibc_unlikely (dtv[0].counter != gen)) return update_get_addr (GET_ADDR_PARAM); void *p = dtv[GET_ADDR_MODULE].pointer.val; @@ -946,7 +956,10 @@ _dl_tls_get_addr_soft (struct link_map *l) return NULL; dtv_t *dtv = THREAD_DTV (); - if (__glibc_unlikely (dtv[0].counter != GL(dl_tls_generation))) + /* This may be called without holding the GL(dl_load_lock). Reading + arbitrary gen value is fine since this is best effort code. */ + size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); + if (__glibc_unlikely (dtv[0].counter != gen)) { /* This thread's DTV is not completely current, but it might already cover this module. */ @@ -1032,7 +1045,9 @@ cannot create TLS data structures")); /* Add the information into the slotinfo data structure. */ if (do_add) { - listp->slotinfo[idx].map = l; - listp->slotinfo[idx].gen = GL(dl_tls_generation) + 1; + /* Can be read concurrently. See _dl_update_slotinfo. */ + atomic_store_relaxed (&listp->slotinfo[idx].map, l); + atomic_store_relaxed (&listp->slotinfo[idx].gen, + GL(dl_tls_generation) + 1); } }