From patchwork Mon Mar 18 14:14:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 87315 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 76AA03858433 for ; Mon, 18 Mar 2024 14:14:53 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR02-VI1-obe.outbound.protection.outlook.com (mail-vi1eur02on2069.outbound.protection.outlook.com [40.107.241.69]) by sourceware.org (Postfix) with ESMTPS id 2ECA5385840A for ; Mon, 18 Mar 2024 14:14:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2ECA5385840A Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2ECA5385840A Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.241.69 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1710771271; cv=pass; b=WHojgecM2CTN88egIf6rqgK8rBnBiqIQ10l/fyjzVmh+AludqUBsCiuMVcZBKK2uTtp0eVI9vWli0GFJ+Q9o1yFTTfE8EFAlxX7FUCYezZclmcLquswuIgHaQFZclz7C8yqiDh1pYwZiD0QXprfUMmNWueJ2C5d6MPzcOLWjU8w= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1710771271; c=relaxed/simple; bh=+Y0y3PXO8HE072q7kqDlOfVcOHvvSi29dO4A7oqd/YI=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=hCXHBbGwHpl3iv5H6GZgRwDXUJAY+jTniHBeIcRJo3CTwFxbU03d62ElJ5BWFQeMpfD9zuhvZ23RKuMUc3a1W030diht2diB8XW6n2mz/0O40Y0JPd27hJkeBxb9nNGnvp0CW81o/ARdI2lFdfRavTOA1KznAF0d3UmDR/YClc4= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=jcUxlboOZRGDtrDO//gYWVLnDN9WBBu5BIKehdn1w4eBZScvsWbBS8sf4/c9bmt/jzZ7gcaMq6R32F2VNzvYim3fNZnKeP5/SPU7rkK9FtLU/0jf8N8ioHJdHbTLQsZ8QJy0E7TNRkAtVAJBbTdFNIYHW8U/qFMGibj8wBy/mOoFllndJ1GCY7kYOtn7uIiT4vj+AUXHFsHBWXV2y35pR+Sf4ZBQyeZAnyUM192rPMHihD6bXEAl1qEzlpqTeWmZAac4QOqg3U0RJDFs1SutdKqEzcSgiJX03UJm34A0oqYPXMo1O2jVtqie/XJ//itaVSmt+Hc+3BRB3jt4M/Ykow== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wzmmBRBv4b5xUnB7lhDielwHLpY+umSOm49PiK3/yRE=; b=GYIHR+9MURRBYr+v2D7MDoN4XkictSr/e3UweS68giZwqXn5uhMiDgQfi4xxk+wWLhHhZpJ9zIGjzP69rVNQKte9MDEWjo4ueJknPfvcFj1RuRFxhD0n/fsF13z8ohOok7pomdTQt8xemr9VDNy4LErK4AX/tT7pY9FbiklxnL9NNYl5LcrmvtNXIwkT2IqV/FMywKnuDBnABa16D6Dif0YETDLHQvCGQG8Rw9VZ8z8TWpvvDNZAmW42Pk99tdnfhD23rzjSx+vG82nyjO7Th9W9Bk8sL877vojKEc+gT7O95pBS82QVh+TGYjflnllkyEvmpn3IzOoQMgHgJwVCcQ== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wzmmBRBv4b5xUnB7lhDielwHLpY+umSOm49PiK3/yRE=; b=ITQ1UVoAd/krpyeyBkA5Y2j+R/b//8NrNi19oo357PjgcoVubCqGA2wfE00W6tk6X2rvUydSCUxX2ZccLndwfscvgfECtzRWwUmf/CFMdq/pjg2at7bk9SrrseUgkZ1Ev0fkJtMJ+hvhYn00r/eqnJun0/R553CBINe2ZJGQx9g= Received: from DUZPR01CA0028.eurprd01.prod.exchangelabs.com (2603:10a6:10:46b::7) by AS8PR08MB8827.eurprd08.prod.outlook.com (2603:10a6:20b:5ba::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.26; Mon, 18 Mar 2024 14:14:24 +0000 Received: from DB1PEPF000509FF.eurprd03.prod.outlook.com (2603:10a6:10:46b:cafe::32) by DUZPR01CA0028.outlook.office365.com (2603:10a6:10:46b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.26 via Frontend Transport; Mon, 18 Mar 2024 14:14:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB1PEPF000509FF.mail.protection.outlook.com (10.167.242.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.12 via Frontend Transport; Mon, 18 Mar 2024 14:14:23 +0000 Received: ("Tessian outbound 5ad6c4395be7:v300"); Mon, 18 Mar 2024 14:14:23 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: a3f672e88e3df810 X-CR-MTA-TID: 64aa7808 Received: from 70663d311e9d.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id E3A84600-3C64-408D-B81D-A7FFECF750AC.1; Mon, 18 Mar 2024 14:14:16 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 70663d311e9d.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 18 Mar 2024 14:14:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UjYrnPFhOOGiurYRNa6Hi6i8FAioIsxd6D4Kh8BYKX6IpVRASmw7+/2qGNSvN7mA/CpiQEvpYDVEKZIC2l2abW4dbSNvMVb3KPBg2+Auk9G2hD4LfYTwn+up4uTSk5i1ERAJBMrhgskHqt1XPnao78IGqAPvblLU2NAi3p9pCcJd8+rOm7Ap0VCZ0kghakty4vYzhf6fvzr72jJ9g+2uyXM06igWvKyAeULtGbhdaZmuhJx8pvNJDmhmBC2NhQCua3Dr2wKtYyslz55HFD0GpT18y0XTS9pfPoKhHBotxPGaY7/5l6ot72zUzLgLlmyd5I217u7Yi9Zg4xnugq2Gjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wzmmBRBv4b5xUnB7lhDielwHLpY+umSOm49PiK3/yRE=; b=PC/zHT2Gcdqilmnxvm2D6RcykL2DH6oYBGqhRuWtgII0bdWVYWdIRjalh6WvY4UUKP3aIcbsvLjD2C/SIsJFl3VXtK1vPB7YR8xK2LFwwfh8s8xqHsLBh+jYEFMWDa6cstAvor0Ttp2IrfMNB6YvrOZCMDNxaxGAoLLuUbeIUyH2IiqMxqrXUcC8As+Yfi21nUzgCipSKaFSbCrPM6zzyp+YpQwbQvXof9kV79ppG5vjeUmo7L1vYye0/jabXg6EBnQNV9WLEbhni+kUG9eiRjdKGin+eWYGF/3N8GVRHDs0LdgQjxPZZvSFsUx6idCJQdoJUKtYjkDOG6WNJYRW4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wzmmBRBv4b5xUnB7lhDielwHLpY+umSOm49PiK3/yRE=; b=ITQ1UVoAd/krpyeyBkA5Y2j+R/b//8NrNi19oo357PjgcoVubCqGA2wfE00W6tk6X2rvUydSCUxX2ZccLndwfscvgfECtzRWwUmf/CFMdq/pjg2at7bk9SrrseUgkZ1Ev0fkJtMJ+hvhYn00r/eqnJun0/R553CBINe2ZJGQx9g= Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20) by AS8PR08MB7838.eurprd08.prod.outlook.com (2603:10a6:20b:529::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.26; Mon, 18 Mar 2024 14:14:15 +0000 Received: from PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::8b1b:5f28:5006:ac18]) by PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::8b1b:5f28:5006:ac18%4]) with mapi id 15.20.7386.023; Mon, 18 Mar 2024 14:14:15 +0000 From: Wilco Dijkstra To: 'GNU C Library' CC: Florian Weimer , Adhemerval Zanella , Szabolcs Nagy Subject: [PATCH v2] AArch64: Check kernel version for SVE ifuncs Thread-Topic: [PATCH v2] AArch64: Check kernel version for SVE ifuncs Thread-Index: AQHaeT40wcb7wZHXdEObt8aMsmautg== Date: Mon, 18 Mar 2024 14:14:15 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: PAWPR08MB8982:EE_|AS8PR08MB7838:EE_|DB1PEPF000509FF:EE_|AS8PR08MB8827:EE_ X-MS-Office365-Filtering-Correlation-Id: d9dbf3af-6311-4967-8a3c-08dc4755b660 x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: ytsINsFZQk31lnW6ZaBWjzAwqLSKkgfVi3LBHAX3G4C+msHb5voyNajIr+dLw8+4hPsGthfw5mk789xVq8epesjNCaiOuLFWTy8w/GvcHMQw7YcHgUHUprShMllU2RSiyzymcQ6wRaamGxOFvAV9zpRvnTSrPZn1GS7eDbjeeNm7xdRLTBHiOsksoj29MAPfyI3TuYTjZgT6LsDk/WI6eSWG33/eeuGqYzaMwEEDhobGcfewG8n/xOfQ+ke7A0nywJ8GSUqlSiXcg0erXruM7CcM+wq2ojPWb3l+cNo7MPw0BJqdvvHMU7nYYwYbZOehHVefWAYkCrjWXkCsxzACmbn/+BttA+GpcocIkqnEKL1YQUQm32a6TUgDD5usVxXAuCVoB+iVp6yONIbIVdYDsVYGjWMD8eo7N6QLrh4QwQOVy6rp8opofFLDbLke4MqfjdbZQ2TpPNyuUVx3b0gMcBJtPa8LtbVsnpFcBo5FPUb52a++IpU07q4bVjpm2jdyNj1aRU1mGDp2yRntYe4EHl3n2qPB3wffSL7S7+O8ArNKQwfOIJVAW2j9e/2EtdWJuhCznlM/OM3lXujyljqboqiOOzqodmnUD4AFalpHqJcpeOOTHrJFr9gpWzkSFJfkz2uWyoLC2C88i8KfLz4EAGGA+alDqkKABdR5tgEtI/PSVtgMMDRq6vNPFEJSYmJWQ6mk5Zt4SOIyZ7sK4DIOYCeN8vlZk4MAMN3DGVTI6Xo= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAWPR08MB8982.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366007)(1800799015)(376005)(38070700009); DIR:OUT; SFP:1101; MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7838 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB1PEPF000509FF.eurprd03.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 96f0bdc4-63c3-49b3-ba53-08dc4755b14b X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qQyt3YqydLZL7Dp3KaIaiYJNBb+Oa9XVmt7uB0Oq+65+tWkCxSYZaA8E6kkmkIC9NOUAraQ8vD8MTa4usyABJoj32Bl7hQI5EeXm/JnIoWyc8lQ6DFoap3l3T14X8qnSKwI8nIlba6YNzQOYctJVzfm530eNJA1tweEsFm9Ef2NrgGSeFUk4lfGD50qOyUaxXRTkCs6c/Fd3tAMf9wW9wybrs8tSiqo5wG4lbSv9v+iZMYlrRCJglc0rIJqulMwMaisZ/wR4W0m93xcq1Vge8LvUIS1xppJGYMDVU/uDhlyZ46+Nv5bDHRIfFoV5PpiKD70KLAyY5K/XWeBl6+KZTQQlXckY4vDXHoA/QxhrGSBYF881jDgtTj7YRR/F89grypy9vYYApQ1Spx5OGkLrYqVcFlrMEHGdA0ccVwS1JomzsvGY8UQ3c5gmUKW5t/jJ0BqiAIQH61ifpoZGEU3IlyM8voOcyQd1geKfqTbrwXQaCRmvteJIYKRC8HqvXaBJ+pE6YjvZfkc+PoNgy4LimTvsT5OCKqTcxM1krg96rRV7HU7uiif8/4QwekXcGQjkkFdnpiPF1VLZuyYWh5I6c5n7UyDiOqREs0RGUZfQZfcvwAgqhHU2oriW7677i/Uk9dz7PyMp1k6vT/FLyQaJ4N9FF+T56kUEs/xjP8N2XRJtxT6Jr+aOnnLYFUdfgCqReBXH6BXGnd3hOqvJcrerjGy/1VoQKQZV1BnSZW5Q97jH1lj4ZnyzWpY12IyAiZz9 X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(1800799015)(376005)(82310400014)(36860700004); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Mar 2024 14:14:23.7183 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d9dbf3af-6311-4967-8a3c-08dc4755b660 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509FF.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8827 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org v2: Add __LINUX_KERNEL_VERSION #ifdefs, improve parser, use 8:8:8 format. Old Linux kernels disable SVE after every system call. Calling the SVE-optimized memcpy afterwards will then cause a trap to reenable SVE. As a result, applications with a high use of syscalls may run slower with the SVE memcpy. This is true for kernels between 4.15.0 and before 6.2.0, except for 5.14.0 which was patched. Avoid this by checking the kernel version and selecting the SVE ifunc on modern kernels. Parse the kernel version reported by uname() into a 24-bit kernel.major.minor value without calling any library functions. If uname() is not supported or if the version format is not recognized, assume the kernel is modern. Passes regress, OK for commit? Reviewed-by: Szabolcs Nagy Tested-by: Florian Weimer diff --git a/sysdeps/aarch64/cpu-features.h b/sysdeps/aarch64/cpu-features.h index 77a782422af1b6e4b2af32bfebfda37874111510..5f2da91ebbd0adafb0d84ec503b0f902f566da5a 100644 --- a/sysdeps/aarch64/cpu-features.h +++ b/sysdeps/aarch64/cpu-features.h @@ -71,6 +71,7 @@ struct cpu_features /* Currently, the GLIBC memory tagging tunable only defines 8 bits. */ uint8_t mte_state; bool sve; + bool prefer_sve_ifuncs; bool mops; }; diff --git a/sysdeps/aarch64/multiarch/init-arch.h b/sysdeps/aarch64/multiarch/init-arch.h index c52860efb22d70eb4bdf356781f51c7de8ec67dc..61dc40088f4d9e5e06b57bdc7d54bde1e2a686a4 100644 --- a/sysdeps/aarch64/multiarch/init-arch.h +++ b/sysdeps/aarch64/multiarch/init-arch.h @@ -36,5 +36,7 @@ MTE_ENABLED (); \ bool __attribute__((unused)) sve = \ GLRO(dl_aarch64_cpu_features).sve; \ + bool __attribute__((unused)) prefer_sve_ifuncs = \ + GLRO(dl_aarch64_cpu_features).prefer_sve_ifuncs; \ bool __attribute__((unused)) mops = \ GLRO(dl_aarch64_cpu_features).mops; diff --git a/sysdeps/aarch64/multiarch/memcpy.c b/sysdeps/aarch64/multiarch/memcpy.c index d12eccfca51f4bcfef6ccf5aa286edb301e361ac..ce53567dab33c2f00b89b4069235abd4651811a6 100644 --- a/sysdeps/aarch64/multiarch/memcpy.c +++ b/sysdeps/aarch64/multiarch/memcpy.c @@ -47,7 +47,7 @@ select_memcpy_ifunc (void) { if (IS_A64FX (midr)) return __memcpy_a64fx; - return __memcpy_sve; + return prefer_sve_ifuncs ? __memcpy_sve : __memcpy_generic; } if (IS_THUNDERX (midr)) diff --git a/sysdeps/aarch64/multiarch/memmove.c b/sysdeps/aarch64/multiarch/memmove.c index 2081eeb4d40e0240e67a7b26b64576f44eaf18e3..fe95037be391896c7670ef606bf4d3ba7dfb6a00 100644 --- a/sysdeps/aarch64/multiarch/memmove.c +++ b/sysdeps/aarch64/multiarch/memmove.c @@ -47,7 +47,7 @@ select_memmove_ifunc (void) { if (IS_A64FX (midr)) return __memmove_a64fx; - return __memmove_sve; + return prefer_sve_ifuncs ? __memmove_sve : __memmove_generic; } if (IS_THUNDERX (midr)) diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c index b1a3f673f067280bdacfddd92723a81e418023e5..c0b047bc0dbeae428c89e12688b7d802e4cb3a43 100644 --- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c +++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #define DCZID_DZP_MASK (1 << 4) @@ -62,6 +63,46 @@ get_midr_from_mcpu (const struct tunable_str_t *mcpu) return UINT64_MAX; } +#if __LINUX_KERNEL_VERSION < 0x060200 + +/* Return true if we prefer using SVE in string ifuncs. Old kernels disable + SVE after every system call which results in unnecessary traps if memcpy + uses SVE. This is true for kernels between 4.15.0 and before 6.2.0, except + for 5.14.0 which was patched. For these versions return false to avoid using + SVE ifuncs. + Parse the kernel version into a 24-bit kernel.major.minor value without + calling any library functions. If uname() is not supported or if the version + format is not recognized, assume the kernel is modern and return true. */ + +static inline bool +prefer_sve_ifuncs (void) +{ + struct utsname buf; + const char *p = &buf.release[0]; + int kernel = 0; + int val; + + if (__uname (&buf) < 0) + return true; + + for (int shift = 16; shift >= 0; shift -= 8) + { + for (val = 0; *p >= '0' && *p <= '9'; p++) + val = val * 10 + *p - '0'; + kernel |= (val & 255) << shift; + if (*p++ != '.') + break; + } + + if (kernel >= 0x060200 || kernel == 0x050e00) + return true; + if (kernel >= 0x040f00) + return false; + return true; +} + +#endif + static inline void init_cpu_features (struct cpu_features *cpu_features) { @@ -126,6 +167,13 @@ init_cpu_features (struct cpu_features *cpu_features) /* Check if SVE is supported. */ cpu_features->sve = GLRO (dl_hwcap) & HWCAP_SVE; + cpu_features->prefer_sve_ifuncs = cpu_features->sve; + +#if __LINUX_KERNEL_VERSION < 0x060200 + if (cpu_features->sve) + cpu_features->prefer_sve_ifuncs = prefer_sve_ifuncs (); +#endif + /* Check if MOPS is supported. */ cpu_features->mops = GLRO (dl_hwcap2) & HWCAP2_MOPS; }