From patchwork Wed Oct 19 11:14:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 59052 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B68E33858295 for ; Wed, 19 Oct 2022 11:15:23 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B68E33858295 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1666178123; bh=8QlRJk3ItP746v81GIxt+n5aN+tSIJ9EDkQHt8NtRI0=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=fjIUSDl+M+t8X5Zq1j1QO4aWO0xFDUTP30PhxizdfvLXD5tzFUIaMfaw60Ey9z+Kl k6D6JcAoDiSQg9FF3FCMfXcn1RYM4bIDHG9E0RVgyStJPAKy4hUeUVeBeE8pYXxaM3 JhURCafhtpNf4Vg0N/0TnEtkaA21s8tIFcU90e6Q= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70087.outbound.protection.outlook.com [40.107.7.87]) by sourceware.org (Postfix) with ESMTPS id 667D9385800B for ; Wed, 19 Oct 2022 11:14:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 667D9385800B ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=Bs88ypX+5djno/zLwm32+LrP6WylO+tgEuj6l7eg4J3BXT2BUUUro5o9XrlClwbaOd5oWsaxerg0hghHs5ijXv82luCEeCXI6Ss1ofrhZ6XQflPu70qMr4rSImiSRIuF0POyF0E6VIYK7hwSumuvfPYA9ytw1kYi55kgG/VJc08SMmyO1+FBBTHmGN6JipTQfPZVoi+eZGTGP0yxhGv9owkiOzDbV41TV1mungu8hVTcgnVpD+7dV37LZPO+xnhMUNfbNue0OLNRoj+JwWze3Om0yQRi/i0rIVpnIRB/RTJZ1YuSMiCRhTR9+xRM3DQ+mxvKmlYq6ifUk6j/x8i5QQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8QlRJk3ItP746v81GIxt+n5aN+tSIJ9EDkQHt8NtRI0=; b=F7WWEhYDmsDae39zx9Zd/Ma++mxBXl7fCrbO3wR6pc4OxEQdgiRTD1hp6dhLLi6YqcbbcmxLLQP9cmvEo5w9rFUHIKr9UZClM/g74WyUVBO4QayF++Ri0Lm1CePobkYjrQYHjSR4adEnQZBqRGNpBpdADOE+/ziGleD5kKhGJh43QnhRwYk4KEmvUGZtatx1KhHup9q2goaWQ2dhpEHj4dMxWbJnGgh+cFYtIdghJz/qLo+4waFqeE6vOZnAIbs0slMiPcN2AWonpuUICQmpfCzqheMZWkzwnSs6K467KuuU3stgAh5zBbmUU5Vzso5cwCV8fCOcVZY4yv2uVjoE0w== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1, 1, smtp.mailfrom=arm.com] dmarc=[1, 1, header.from=arm.com]) Received: from FR3P281CA0135.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:94::19) by AS8PR08MB9196.eurprd08.prod.outlook.com (2603:10a6:20b:5a0::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.34; Wed, 19 Oct 2022 11:14:55 +0000 Received: from VI1EUR03FT016.eop-EUR03.prod.protection.outlook.com (2603:10a6:d10:94:cafe::f4) by FR3P281CA0135.outlook.office365.com (2603:10a6:d10:94::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.17 via Frontend Transport; Wed, 19 Oct 2022 11:14:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VI1EUR03FT016.mail.protection.outlook.com (100.127.144.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.32 via Frontend Transport; Wed, 19 Oct 2022 11:14:54 +0000 Received: ("Tessian outbound 2ff13c8f2c05:v130"); Wed, 19 Oct 2022 11:14:54 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: f3d14c9ed6cd3f68 X-CR-MTA-TID: 64aa7808 Received: from 202537e8219b.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 26429A8E-ECD9-4F23-AEED-CF8DCDCC96C2.1; Wed, 19 Oct 2022 11:14:47 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 202537e8219b.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 19 Oct 2022 11:14:47 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Chld8u9hmYNZowa2AUW5fUD1oXZlvWECdoEqBbcAng2DvJMXrqgacef8+MybutxnOfVHnoyNdaPNXtxq5EKrRHBJZt5fOpTg6zTF4Hk2G2YVefZwXS9HV2/UHEkdRAfsPvcLZXBOCiPnnHopLFCYW0tyJs+D0X0flhctNU1jMqglxh/iK8wh7yJWzytp/r2mPz9SccDXQ9I8uaLXJj5zdEfWayIqLMEpIOuXsWFtEiR1uxTWmAbijpjzwQdH34AyYxtnAnpbu6OQmiElo0RvVwamgbhr6dsiePrPUjQW0yaQU+1IwCJQPIa9GwlN/+u6B0CbWl0SkCMpQMzfK1heZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8QlRJk3ItP746v81GIxt+n5aN+tSIJ9EDkQHt8NtRI0=; b=che4c515cAnqMAkhpx8vPoReWCDqeUdik3Zqlytu3bwDsB0Q27pbOQoI9HpAnko1foMEtuopSavL5DUpFrbMcLH71cuLjYfXu/lDcs6we7/dx42nQGRVuMUxPzgh2zQaPbkiRnj2ZNM08DCWQEFd1Veieul2sgFKCI9GBiP97KDOppx/Ucj/XUK8pwHByRjTsQBr111iYepHpv5HiB5IZWrxf1791I2lCAf4zbtwxc7N67mwE/KpwVEc1MyFFuOyU0vyc4LQ/USaWcOdUORR302WxRgwU9ZyOVNwPha85ExkranynFBfIujoLnmkr4dCJofzH0AhchKvfLlPD/G4og== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from AS9PR06CA0118.eurprd06.prod.outlook.com (2603:10a6:20b:465::20) by DB9PR08MB9658.eurprd08.prod.outlook.com (2603:10a6:10:461::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.26; Wed, 19 Oct 2022 11:14:43 +0000 Received: from AM7EUR03FT030.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:465:cafe::5a) by AS9PR06CA0118.outlook.office365.com (2603:10a6:20b:465::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.32 via Frontend Transport; Wed, 19 Oct 2022 11:14:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT030.mail.protection.outlook.com (100.127.140.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5723.32 via Frontend Transport; Wed, 19 Oct 2022 11:14:40 +0000 Received: from AZ-NEU-EX03.Arm.com (10.251.24.31) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.12; Wed, 19 Oct 2022 11:14:39 +0000 Received: from armchair.cambridge.arm.com (10.2.80.71) by mail.arm.com (10.251.24.31) with Microsoft SMTP Server id 15.1.2507.12 via Frontend Transport; Wed, 19 Oct 2022 11:14:39 +0000 To: Subject: [PATCH v2] elf: Fix slow tls access after dlopen [BZ #19924] Date: Wed, 19 Oct 2022 12:14:39 +0100 Message-ID: <20221019111439.63501-1-szabolcs.nagy@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT030:EE_|DB9PR08MB9658:EE_|VI1EUR03FT016:EE_|AS8PR08MB9196:EE_ X-MS-Office365-Filtering-Correlation-Id: 8a146e70-8df8-4737-8f71-08dab1c3269c x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: S9ecbO3LTi3LuaU0hTr5zj7IDrV3cpnetxKFxTD/xQE1hKzITpYUJ6MdSk1ttGG1Zbcd9+Qv24CbvpxtuX47EvbwPYA+6fbPRvPQelfurLgaVVAVcHeNOf3rmEWFF4HHZYuJGmDsQLrNz7wDKSmQvwO9LeJGsTwVCXcz992pYAklDX+qIJ+XMEre32/tJVKCy7FKx4Z7KBB9c5RGJGtEJcFQ4sSBoDe+ONI72Ru7gegChPn8pFpF2KdgsCckjX8SG/zQINRxs3xgZ2u1cpy0Cj5iyJKBITdVOpkshy3SSCQ7KYGaGQbaeloY3BqsNd/L5v/I9h6gTBB/0R80Ta0sa5SSaON6p4/e0O2sxapfm26EqZ3G7EGyAgchyIIaONck+BMa6Zf7ese79tPMQ5STRZQWGbs6so2ELp+k/RcAYV9AiyV/0gUZbubwH6dGqymUKOpU8JaZjPZsEiJ1A53FeijI1kX2WbTc/MyPdGMqDxR3tuoU7QmvQixx1WKjYy/r9GUuNpmj4Gc2CxbZInM0E5RL6s/GqwZ3lpJF6SSfKl/D9W8fjWjEEcbesdkPdmFyMCBdOBNshEzATzQkR6Y75AJvGIYHiU/iR+wtW0W5uArv8lBw71a9fIOdyEL4i/FlscnBfVHQ0RBTg++1jYXSHCA36VS7Z33LSb8xRdr2HfnpS+N8/kL7qkZe1lMaqZwlo2Ms26d2V6phedhiaBhQBh65ZpQepr++kx09fkzQrLxtCRPQ3koVP1M+P2PA3DYjGIZzSAyWvYt97ZW7Q7XbiXAVJ8rAS8rBZhhAscgp8RbAA57VBoquwioyhgdo4wru X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230022)(4636009)(346002)(396003)(39860400002)(376002)(136003)(451199015)(36840700001)(40470700004)(46966006)(6916009)(44832011)(8676002)(316002)(26005)(7696005)(41300700001)(478600001)(1076003)(336012)(2616005)(8936002)(186003)(2906002)(5660300002)(81166007)(82310400005)(36756003)(356005)(82740400003)(47076005)(426003)(70206006)(36860700001)(83380400001)(70586007)(40460700003)(86362001)(40480700001)(36900700001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB9658 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VI1EUR03FT016.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 38758487-c4a1-4daa-cafe-08dab1c31db6 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KYYZ6Dh/MkYe478aMCjUJWRW2+rvBwLeNopfibQ7YQmIUktHwhNpLIDEq4MgYJb/7NBmJ4m61PVux8Oeh3pgBE9AJmdURrAO5adi7qAe0VY3jcspcODCeLopcPrB+sArI9ZVBz5MaKMpJX/PnoKaOYTuDRzv2MJebPavH/wfil0R4CjDRXNO8FRDLvMcnodiGNb6zZzrZ74NTXWh/CbR0PeWiUNDmygpLoZfJGYZXFUKAhMNq7NQtciNbPlWScjCOpVYFPYn+GNfmSyahaRZQE20LuBDiM71NbKN9gmJyso820hKomXJIHo9rmM4DNJ0LKfArlNhc8eirrvrUgwBNVgaOe9PsiPyHj5dbU3jEI64XhtHnz9mdv32voMgx73xCB4oMsRhKmllSk/TuRNC9/+6QwaXCF6j3xyyhi9sd0d7u88hXUYXkKt1rUWvwR+0Jta5EsEAlvZVClHR6t+joV7nkipcj3knse2L3vO6tJRkZlu8aJLOY5jtdADlACBF4VoZt1sw/v/ve0aDkwV37uaq6zNZlq+xhYkMyN9hoIXydkCTb8cLxsbn3uF2etgNZQ2s8mBYBQos9t2ik6Jz/GOFk59DC+TyYTkhkQYIfJODZBSawjlizpOlRuAFZVvC/JSuRHQU3MrvE3OL5xge+0z2/xcDHb0EQaEuVpyw34Pm3gMe0xdOZdayjMO8fd0tOuIcu8pDysMW07Maq8y0AhqiOIlxvWTv4Y2H3eDhOliAJSCceapxBrnG/v3prghEQM+lpASzRddoYLxERA75sw== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230022)(4636009)(376002)(396003)(346002)(39860400002)(136003)(451199015)(36840700001)(40470700004)(46966006)(82740400003)(81166007)(47076005)(86362001)(5660300002)(8936002)(41300700001)(70206006)(7696005)(70586007)(26005)(40460700003)(40480700001)(8676002)(336012)(426003)(82310400005)(44832011)(1076003)(186003)(6916009)(2906002)(2616005)(316002)(36860700001)(36756003)(478600001)(83380400001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2022 11:14:54.9473 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8a146e70-8df8-4737-8f71-08dab1c3269c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VI1EUR03FT016.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB9196 X-Spam-Status: No, score=-11.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Szabolcs Nagy via Libc-alpha From: Szabolcs Nagy Reply-To: Szabolcs Nagy Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" In short: __tls_get_addr checks the global generation counter and if the current dtv is older then _dl_update_slotinfo updates dtv up to the generation of the accessed module. So if the global generation is newer than geneneration of the module then __tls_get_addr keeps hitting the slow dtv update path. The dtv update path includes a number of checks to see if any update is needed and this already causes measurable tls access slow down after dlopen. Possible approaches i can see: 1. update dtv to the global generation instead of module, 2. bail out faster if dtv is up to date for the module. This patch is 1.: It needs additional sync (load acquire) so the slotinfo list is up to date with the observed global generation. (before the patch we could rely on that the tls access was already synced with the dlopen of the module that is accessed.) Approach 2. Would require walking the slotinfo list to see if dtv is up to date to the module generation. This should be quick if less than TLS_SLOTINFO_SURPLUS modules are dlopened, but can take long with arbitrary many dlopens. Note: in the x86_64 version of dl-tls.c the generation is only loaded once, since relaxed mo is not faster than acquire mo load. I have not benchmarked this yet. TODO: some comments need further update. --- v2: rebased and updated the commit message a bit. still RFC quality. --- elf/dl-close.c | 2 +- elf/dl-open.c | 8 ++++---- elf/dl-reloc.c | 5 ++--- elf/dl-tls.c | 28 ++++++++++++---------------- sysdeps/generic/ldsodefs.h | 3 ++- sysdeps/x86_64/dl-tls.c | 4 ++-- 6 files changed, 23 insertions(+), 27 deletions(-) diff --git a/elf/dl-close.c b/elf/dl-close.c index bcd6e206e9..8e81cee8e7 100644 --- a/elf/dl-close.c +++ b/elf/dl-close.c @@ -743,7 +743,7 @@ _dl_close_worker (struct link_map *map, bool force) if (__glibc_unlikely (newgen == 0)) _dl_fatal_printf ("TLS generation counter wrapped! Please report as described in "REPORT_BUGS_TO".\n"); /* Can be read concurrently. */ - atomic_store_relaxed (&GL(dl_tls_generation), newgen); + atomic_store_release (&GL(dl_tls_generation), newgen); if (tls_free_end == GL(dl_tls_static_used)) GL(dl_tls_static_used) = tls_free_start; diff --git a/elf/dl-open.c b/elf/dl-open.c index e7db5e9642..734a0bee4e 100644 --- a/elf/dl-open.c +++ b/elf/dl-open.c @@ -405,7 +405,7 @@ update_tls_slotinfo (struct link_map *new) _dl_fatal_printf (N_("\ TLS generation counter wrapped! Please report this.")); /* Can be read concurrently. */ - atomic_store_relaxed (&GL(dl_tls_generation), newgen); + atomic_store_release (&GL(dl_tls_generation), newgen); /* We need a second pass for static tls data, because _dl_update_slotinfo must not be run while calls to @@ -422,8 +422,8 @@ TLS generation counter wrapped! Please report this.")); now, but we can delay updating the DTV. */ imap->l_need_tls_init = 0; #ifdef SHARED - /* Update the slot information data for at least the - generation of the DSO we are allocating data for. */ + /* Update the slot information data for the current + generation. */ /* FIXME: This can terminate the process on memory allocation failure. It is not possible to raise @@ -431,7 +431,7 @@ TLS generation counter wrapped! Please report this.")); _dl_update_slotinfo would have to be split into two operations, similar to resize_scopes and update_scopes above. This is related to bug 16134. */ - _dl_update_slotinfo (imap->l_tls_modid); + _dl_update_slotinfo (imap->l_tls_modid, newgen); #endif dl_init_static_tls (imap); diff --git a/elf/dl-reloc.c b/elf/dl-reloc.c index 756bf950f6..e391ed47b0 100644 --- a/elf/dl-reloc.c +++ b/elf/dl-reloc.c @@ -112,11 +112,10 @@ _dl_try_allocate_static_tls (struct link_map *map, bool optional) if (map->l_real->l_relocated) { #ifdef SHARED +// TODO: it is not clear why we need to update the DTV here, add comment if (__builtin_expect (THREAD_DTV()[0].counter != GL(dl_tls_generation), 0)) - /* Update the slot information data for at least the generation of - the DSO we are allocating data for. */ - (void) _dl_update_slotinfo (map->l_tls_modid); + (void) _dl_update_slotinfo (map->l_tls_modid, GL(dl_tls_generation)); #endif dl_init_static_tls (map); diff --git a/elf/dl-tls.c b/elf/dl-tls.c index 093cdddb7e..3d1ae00b61 100644 --- a/elf/dl-tls.c +++ b/elf/dl-tls.c @@ -721,7 +721,7 @@ allocate_and_init (struct link_map *map) struct link_map * -_dl_update_slotinfo (unsigned long int req_modid) +_dl_update_slotinfo (unsigned long int req_modid, size_t new_gen) { struct link_map *the_map = NULL; dtv_t *dtv = THREAD_DTV (); @@ -738,19 +738,12 @@ _dl_update_slotinfo (unsigned long int req_modid) code and therefore add to the slotinfo list. This is a problem since we must not pick up any information about incomplete work. The solution to this is to ignore all dtv slots which were - created after the one we are currently interested. We know that - dynamic loading for this module is completed and this is the last - load operation we know finished. */ - unsigned long int idx = req_modid; + created after the generation we are interested in. We know that + dynamic loading for this generation is completed and this is the + last load operation we know finished. */ struct dtv_slotinfo_list *listp = GL(dl_tls_dtv_slotinfo_list); - while (idx >= listp->len) - { - idx -= listp->len; - listp = listp->next; - } - - if (dtv[0].counter < listp->slotinfo[idx].gen) + if (dtv[0].counter < new_gen) { /* CONCURRENCY NOTES: @@ -771,7 +764,6 @@ _dl_update_slotinfo (unsigned long int req_modid) other entries are racy. However updating a non-relevant dtv entry does not affect correctness. For a relevant module m, max_modid >= modid of m. */ - size_t new_gen = listp->slotinfo[idx].gen; size_t total = 0; size_t max_modid = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); assert (max_modid >= req_modid); @@ -914,9 +906,9 @@ tls_get_addr_tail (GET_ADDR_ARGS, dtv_t *dtv, struct link_map *the_map) static struct link_map * __attribute_noinline__ -update_get_addr (GET_ADDR_ARGS) +update_get_addr (GET_ADDR_ARGS, size_t gen) { - struct link_map *the_map = _dl_update_slotinfo (GET_ADDR_MODULE); + struct link_map *the_map = _dl_update_slotinfo (GET_ADDR_MODULE, gen); dtv_t *dtv = THREAD_DTV (); void *p = dtv[GET_ADDR_MODULE].pointer.val; @@ -951,7 +943,11 @@ __tls_get_addr (GET_ADDR_ARGS) by user code, see CONCURRENCY NOTES in _dl_update_slotinfo. */ size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); if (__glibc_unlikely (dtv[0].counter != gen)) - return update_get_addr (GET_ADDR_PARAM); + { +// TODO: needs comment update if we rely on consistent generation with slotinfo + gen = atomic_load_acquire (&GL(dl_tls_generation)); + return update_get_addr (GET_ADDR_PARAM, gen); + } void *p = dtv[GET_ADDR_MODULE].pointer.val; diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h index 6b256b8388..8ba8e75c3d 100644 --- a/sysdeps/generic/ldsodefs.h +++ b/sysdeps/generic/ldsodefs.h @@ -1259,7 +1259,8 @@ extern void _dl_add_to_slotinfo (struct link_map *l, bool do_add) /* Update slot information data for at least the generation of the module with the given index. */ -extern struct link_map *_dl_update_slotinfo (unsigned long int req_modid) +extern struct link_map *_dl_update_slotinfo (unsigned long int req_modid, + size_t gen) attribute_hidden; /* Look up the module's TLS block as for __tls_get_addr, diff --git a/sysdeps/x86_64/dl-tls.c b/sysdeps/x86_64/dl-tls.c index 24a6e643b0..f74b20dd8d 100644 --- a/sysdeps/x86_64/dl-tls.c +++ b/sysdeps/x86_64/dl-tls.c @@ -40,9 +40,9 @@ __tls_get_addr_slow (GET_ADDR_ARGS) { dtv_t *dtv = THREAD_DTV (); - size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); + size_t gen = atomic_load_acquire (&GL(dl_tls_generation)); if (__glibc_unlikely (dtv[0].counter != gen)) - return update_get_addr (GET_ADDR_PARAM); + return update_get_addr (GET_ADDR_PARAM, gen); return tls_get_addr_tail (GET_ADDR_PARAM, dtv, NULL); }