From patchwork Mon Aug 3 15:48:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 140497 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 35E364BA2E3C for ; Mon, 3 Aug 2026 15:50:05 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 35E364BA2E3C Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=Zu6jpcC1; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=Zu6jpcC1 X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from GVXPR05CU001.outbound.protection.outlook.com (mail-swedencentralazlp170130007.outbound.protection.outlook.com [IPv6:2a01:111:f403:c202::7]) by sourceware.org (Postfix) with ESMTPS id 4CD4D4BA799F for ; Mon, 3 Aug 2026 15:49:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4CD4D4BA799F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4CD4D4BA799F Authentication-Results: sourceware.org; arc=pass smtp.remote-ip=2a01:111:f403:c202::7 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1785772166; cv=pass; b=Em/65ePu2mlEfhljDlcG2V3H00JIaL4E6fI/pLpQ5zeLdja9y7E6p2pNupLgqD+Xp3XdoaBvnjavDNm4PmSposh+kjwYAtuSsQNWhfg5jxTzHJbbKTmfmnAzuDeWT/rMrhS0PBe+kI4LVaI3GLrqc8Kwcb2XRD6Xuhu/azo4lnk= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1785772166; c=relaxed/simple; bh=kFX+d7L1lm+k267lIanO0QdgLbYfNXQgarIXaLvU920=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=rkD/CcoIi/A7bU5wXSvMLoFaQAH7upYiLUvH8jqrAPRmKJ3ScctNuRtK2jLovqXXhIfNN3CPvAkA2U/rI4nVEE1bKW03cYW4p3Z4XKfZy/vYCigD3So0uc5ES4qDJbV3XhxxQ2Xo8l5GvP+Ilg8clOGUwlO1nD3Xt4nRu1JqjpE= ARC-Authentication-Results: i=3; sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=Zu6jpcC1; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=Zu6jpcC1 DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4CD4D4BA799F ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=bsqBF2/dtCWCl6Ctq9hdtqmXu9BK9hgdQdZVUWMEvrUO0vnzRK6jsBVAw+Cbr4DTV6Yt4OatMrP81b2ueL5uD/Q41MRCkK4Sd5t89B5/3VP3diWG1r3zOHoJsgv6hfLEXBwfZ4wsr0vrqhK/2D3iR5rDoXMdDCvHWoHK7j5Oqh+2fF6sArFGoahTd1LQYY1lm4U1YdQgwHM30A1Ah3yujFU3b8/7opl8VmiflBVAawU6kkqsVCbXxsu0MGYWKdsxVTgL/27VT9UnxnySf1Pqp6T4iRKSYem6uXy9PGphCBdFbqNFGk8Kg0zSh/sg2Hv4l3xhVEkP7R4G1fGPwbRVCA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m0Og/4qqdb0chijflyzw2Jw12DLOvn8TRF8MAijyacM=; b=x3IRpnK827EDA0DIoiEFe2E+Gnal3DVSThTriqQLSRtmhvLJYZttBaT2UYiAdB0mqClJDLH6SshCitw5CVh9tQx1L707Dea1S/SpTYVUWA1BiG6vUcC8TI/E2R7qPw72tvgYXr2ciClr9h+8NEX04Sl5MRM4Wgr2d91uIUNo0n1dKNwlySTqjyYJlXML9PhTJH2tF5aILVPHG1kIL/26tYxjyYfGihaiaH3Ro+EVeDE5ao7IXhInWPN44Keu0+orqJeNoVWqZ9TGceiEZdvTzLtMKdh10q6PaYkHPJJWcluOctsp+MkKl84Zy4nLjcys52yCorp1nfBQxWH2v3BN0A== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m0Og/4qqdb0chijflyzw2Jw12DLOvn8TRF8MAijyacM=; b=Zu6jpcC1icK53nQp8OIPbwCn97MzfiETsRij1olaA8oq00BGM1pXPk1DCdoTdvomzQVn66OXd7DC1gPJoEQgVTyvXjjVq/fPKBE//mW7HPgRPzBQaGNZAi9Sn5vKEvMLis2iHrRBHKQDbRXjeOuAbdfg+3/guvgYl4gMUXFICCs= Received: from AS4P189CA0015.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:5d7::19) by DU0PR08MB9558.eurprd08.prod.outlook.com (2603:10a6:10:44d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.17; Mon, 3 Aug 2026 15:49:10 +0000 Received: from AMS1EPF0000008F.eurprd05.prod.outlook.com (2603:10a6:20b:5d7:cafe::7e) by AS4P189CA0015.outlook.office365.com (2603:10a6:20b:5d7::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.270.18 via Frontend Transport; Mon, 3 Aug 2026 15:49:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AMS1EPF0000008F.mail.protection.outlook.com (10.167.242.86) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.292.8 via Frontend Transport; Mon, 3 Aug 2026 15:49:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yFWFk2T75+PpPsy4ibHSDbiFEUlmx8PLGQMORyg+ePu9u98dcSSblOA54HwSqLoYqYNlA4WbJsbqm30OuTfH2yoUNrfCIjftaTJdPBXwL2MguaYPDgN/jP+VAwUKhEZnQitMzqxFL87hcqplh5Ebs723YWv+igBnemiRu+dLouNQVsHtNl9eAl6xIAi/IaNwHITYKL/lxQcO2TC/h2gweJbkewW9M/LUuzMdl72aSy0AuEolki+n63ThJM48qBbyaIkEHJhNUGOoBImp3e94YRT0VbnQBGD1QTP/FT/vdCcbfUy7OSQzoOEolEfPeHHp+1rDp2nrcGkTD/OM+LwZmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m0Og/4qqdb0chijflyzw2Jw12DLOvn8TRF8MAijyacM=; b=O8yhXlG5mdsXLe+V5q9ru6WpVQyCS+s1y4w8rqTE68Mee8dbTLd5MuzhKUTSAZjgGU4W5R+2bAd3MiovccGb01gOAMqklpzqOY8MgAEL8ydxK/HWOU2j+DzoNuAxevOwhMSSRTYVgXCqeds9Y21hP18pQUGKtMUdR5e4C8bB2hQv/SB98D3cPYjWWozucEH0HGA0299HXtOIMLfMcSdZOSQDUvTd2N8oKAH9uT5ODIpEGLi5CDhb5wp56F+dpzkGjAdmKUWttK9kr/stXzkF6RBGlNgS388aQAZKVpvQVYeww+dmE+VMfcl1eyNQNqqF+WHCTovtp4qBGZY9SgARWA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m0Og/4qqdb0chijflyzw2Jw12DLOvn8TRF8MAijyacM=; b=Zu6jpcC1icK53nQp8OIPbwCn97MzfiETsRij1olaA8oq00BGM1pXPk1DCdoTdvomzQVn66OXd7DC1gPJoEQgVTyvXjjVq/fPKBE//mW7HPgRPzBQaGNZAi9Sn5vKEvMLis2iHrRBHKQDbRXjeOuAbdfg+3/guvgYl4gMUXFICCs= Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20) by PA4PR08MB7617.eurprd08.prod.outlook.com (2603:10a6:102:272::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.17; Mon, 3 Aug 2026 15:48:15 +0000 Received: from PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::5821:707a:c30b:dd9a]) by PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::5821:707a:c30b:dd9a%5]) with mapi id 15.21.0270.016; Mon, 3 Aug 2026 15:48:14 +0000 From: Wilco Dijkstra To: GNU C Library CC: Wilco Dijkstra Subject: [PATCH 1/2] AArch64: Add SVE2 strchr Thread-Topic: [PATCH 1/2] AArch64: Add SVE2 strchr Thread-Index: AQHdI19MU8LWAn4+AUieLfVb/H8Mtw== Date: Mon, 3 Aug 2026 15:48:14 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: PAWPR08MB8982:EE_|PA4PR08MB7617:EE_|AMS1EPF0000008F:EE_|DU0PR08MB9558:EE_ X-MS-Office365-Filtering-Correlation-Id: 73e1115d-de81-491c-49ee-08def176c279 x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|376014|366016|1800799024|23010399003|38070700021|13003099007|56012099006|10067099003|11063799006|5023799004|18002099003; X-Microsoft-Antispam-Message-Info-Original: kJzU5FNPmoIrITnifTYNTRzxwRN7Ek1ft8/3BmeI+9GXhZNZrEu2xL/NGVKzrpJvvo4KdpdLXRPxVkS1mQZA+xiELWGdM447CxbOqd7KvUaw+Ic0BKGHMtO8l4MKeRlhDZVcSy7FqMc9la/y5CofDEdnCyF6F9Jaqpy5FKQ/6w3DB0E9fW0vcRex+GUr/Ym+IvAQ9zgzzexllmUAia7YiHtHL/hB1bKtnuDHxiu/6fcMpd0Rccx3K7Per78Lbn7JC4jkzn+JJ7ovW7Dk5aTCTVPsE8IEMuRe3biq3QU2i7nhoWE1RUGj4ewiX6cWHCYmyTFpjPihs3bKEovtw5CuZ+Wpg5CQeAHmy+xRe6jlrnbgHnmUhL5gt7uNV9YARqKLd2lNChOH5YR9KBDHuFNwrkPA0lhhMX2PQgN0Gfn0CXe7LZdyadI4YzVBXZwREqqeRM5AGhU0tsDS9TV7wfKu1ESeGzpIGKr3x8xzdwuWgpPiN6T9Gg9EDwgVvRKUKQcRXBOQbiymDOgXsDlAcNhxAsl75pB1qZlrDoLekwCNcw/qX6zz2qi8iQ3HEuAelQQzgEwmEA9qeZph2WvffQ4gXqRw9mCKwNB1VMP0JA/0tDprbfbPh4UfWtJiSPlrmI32lghvmYTGT+25qhJm9Sh3bREzxmiLcuIs0JQDf5Y6MLlPZi4BUhlBUfSgQq+sAtSBKVJSmFnOl0w/6wnHtTNmfP9EEJMouysQjEW9CfgaZJk= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAWPR08MB8982.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024)(23010399003)(38070700021)(13003099007)(56012099006)(10067099003)(11063799006)(5023799004)(18002099003); DIR:OUT; SFP:1101; MIME-Version: 1.0 X-Exchange-RoutingPolicyChecked: cjW3rRG3KUQkyUSRr7YCNXDX2FLobq6UzhWl2MxOm35wPQco/3zWDUlktwFwiGbR6ULLdpapev9QmBBwyGj3jXA0aXe7i52OE9kgEvDHQl+0utZX2GW2Y390Q77uxq7rnMk0ZXJMOkY8XzwK3pTJlGHULwYtWWvAUyTjOz8Y4oFcADHNV7bPpEvTVqFO7jdXjecEYzeRYfQF96yQ1fxozAPxuxpaxiuaLtse41rR4FWWBbs5rrmui21sxz5Xoz2i7DyNVY3AIvjlAOEXJS944gHm5VJSVMVaX3y45SVZlOgKfJE0OVtiAhOTcBI814EpaWkvhtGx+hvlmTDj3FYPIg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB7617 X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS1EPF0000008F.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 70d5aa4a-7f70-44fe-bfa1-08def176a12a X-Microsoft-Antispam: BCL:0; ARA:13230040|35042699022|1800799024|14060799003|376014|36860700016|23010399003|82310400026|18002099003|13003099007|10067099003|11063799006|5023799004|56012099006; X-Microsoft-Antispam-Message-Info: mKmXz27ZfKIuVn0YyjsO/74AV9eqtuiKNKzK0iLJBFV9wNTQQR7Fjd3GAJS3oMr+CumaJGEwMK4DrUXt75HYbQVlfRPXxl51u4LnHuNFOy+mvM6cDVLgSVp0eIe7Rd7TUcVFbuEXjtlm1+euAPYMhT5hNzgiotr00/mzLWYje8ak78gao1loxk3q0Tp5KBcuLAd2sGhR73cB6vfCBiKopGJ3axqq2di1VDTKo82GwjCX+U1nso9rY/14jHhC8C5u3xG6de9LC1fqWKd6tqYP7Hh10CiTfkB3uLwfOtgpGBUeoy91uDdSCgejFHFK4uaxW8rr5jIhVSSu7CZhp0ECEwsdGKyfwYJzkYLYnywY+56IP/hu4TnI/VB1gAqSIaK+wqgb+jdav3tgTU+rLzTiU/k9wfD9W9B5QRFZpAIkbkyLQV47ztXzvu709tj6vlMDUaoqn+6iDosn0IYmhDsKbH1G6XDXoVvkVpH3K/mT2Q9rrZLvCqFDN+QRw89l58xuQt9soVgZ5LPSJ9sG7prtp6W/VriTIcg/p/5EIhMKNLPPrOt45Xs4fpph/xgQJqOomlACOhbriFDOUQ6vOvSdsx1OzoOEYQdx3cgBFIHnc/FLHtJBwAVhN7bsdG8uTCeKaH4CSnEwyi++lGveOAC1zrhHMzr+yVYSYv5SeIbdRI2OYZCzmTxFoAduLK6I7tCWXd60NT5FEfx98mpVXS1FzA== X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(35042699022)(1800799024)(14060799003)(376014)(36860700016)(23010399003)(82310400026)(18002099003)(13003099007)(10067099003)(11063799006)(5023799004)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +h3mkw2mSvJbVm7+ErDUdOOb5Yaj8rrPhzn+Bd6WzWx3RP9Lz0slc7SzXgIytK+cjfhmjJPcv59aFmDKnyhnbEdo7vrHr1LrXrEYgB7XWkEpivhv1sP3jkU8u9OYEY3vc7Kff37QY9qnNH7aDp5PeebGgx+bMLr+b7hwkji+z1CTUASexAvi8bv5U/sIe/w3nt2xuftd8pc4TpODh2U1eJz7ymJ8nhz0S3J8Eydecqx+T8/M2Jq75J9mNH471NFMoY8atTWr2uKBYfZ0wS6WN98DKFfXCxPR2wKAzjwFks5TjEZlTFzQAEXARlRM0k1vmXbOefHBNVk4HVOpjKQjRLLj/zBbjO7i0ymEhKEy50DNOCCjawVO0n6klr13w0k2f5T4mm3JrSTkDujRq1XMTcsGZJEZpPedtIl8uaZWaNXN8X/lKIyLDId5cCYQp2zF X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2026 15:49:10.3970 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 73e1115d-de81-491c-49ee-08def176c279 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF0000008F.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9558 X-Spam-Status: No, score=-10.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_LOTSOFHASH, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org Add an SVE2 strchr implementation and ifunc support. Use MATCH to check for both the input char and NUL. Performance of bench-strchr improves ~28% on Neoverse V2. diff --git a/sysdeps/aarch64/multiarch/Makefile b/sysdeps/aarch64/multiarch/Makefile index 6294a70c252af39964a8750749db34da413e6656..9616231c042c03088384b07ea8c5465d413048d1 100644 --- a/sysdeps/aarch64/multiarch/Makefile +++ b/sysdeps/aarch64/multiarch/Makefile @@ -16,6 +16,8 @@ sysdep_routines += \ memset_oryon1 \ memset_sve_zva64 \ memset_zva64 \ + strchr_generic \ + strchr_sve2 \ strlen_asimd \ strlen_generic \ # sysdep_routines diff --git a/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h index 0910e321d24d84db951cf498c3e883c2cfdd01f5..50b2581455c94dc2bd38f8e5ba9a357d1b79e89f 100644 --- a/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h +++ b/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h @@ -22,6 +22,7 @@ asm ("memset = __memset_generic"); asm ("strlen = __strlen_generic"); #ifndef SHARED +asm ("strchr = __strchr_generic"); asm ("memcpy = __memcpy_generic"); asm ("memmove = __memmove_generic"); asm ("memcmp = __memcmp_generic"); diff --git a/sysdeps/aarch64/multiarch/ifunc-impl-list.c b/sysdeps/aarch64/multiarch/ifunc-impl-list.c index d43f6b58ee075c815b93ab2f42735e7e9f315d32..a0c93147c532d935d562a66ac67ebaa5326dd3bd 100644 --- a/sysdeps/aarch64/multiarch/ifunc-impl-list.c +++ b/sysdeps/aarch64/multiarch/ifunc-impl-list.c @@ -33,7 +33,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, INIT_ARCH (); - /* Support sysdeps/aarch64/multiarch/memcmp.c, memcpy.c, memmove.c and memset.c. */ IFUNC_IMPL (i, name, memcmp, IFUNC_IMPL_ADD (array, i, memcmp, sve, __memcmp_kunpeng950) IFUNC_IMPL_ADD (array, i, memcmp, 1, __memcmp_generic)) @@ -61,6 +60,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, IFUNC_IMPL (i, name, strlen, IFUNC_IMPL_ADD (array, i, strlen, !mte, __strlen_asimd) IFUNC_IMPL_ADD (array, i, strlen, 1, __strlen_generic)) - + IFUNC_IMPL (i, name, strchr, + IFUNC_IMPL_ADD (array, i, strchr, sve2 && !mte, __strchr_sve2) + IFUNC_IMPL_ADD (array, i, strchr, 1, __strchr_generic)) return 0; } diff --git a/sysdeps/aarch64/multiarch/strchr.c b/sysdeps/aarch64/multiarch/strchr.c new file mode 100644 index 0000000000000000000000000000000000000000..12f4e10bbe034ab462a4e578c69fd1d9b4afd9ed --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchr.c @@ -0,0 +1,38 @@ +/* Multiple versions of strchr. AArch64 version. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Define multiple versions only for the definition in libc. */ + +#if IS_IN (libc) +/* Redefine strchr so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef strchr +# define strchr __redirect_strchr +# include +# include + +extern __typeof (__redirect_strchr) __libc_strchr; + +extern __typeof (__redirect_strchr) __strchr_generic attribute_hidden; +extern __typeof (__redirect_strchr) __strchr_sve2 attribute_hidden; + +libc_ifunc (__libc_strchr, (sve2 && !mte ? __strchr_sve2 : __strchr_generic)); + +# undef strchr +strong_alias (__libc_strchr, strchr); +#endif diff --git a/sysdeps/aarch64/multiarch/strchr_generic.S b/sysdeps/aarch64/multiarch/strchr_generic.S new file mode 100644 index 0000000000000000000000000000000000000000..d72b0be62a89f4e5b090f9d5c1973088bad89704 --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchr_generic.S @@ -0,0 +1,35 @@ +/* A generic optimized strchr implementation for AArch64. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN (libc) + +# define STRCHR __strchr_generic + +/* Do not hide the generic version of strchr, we use it internally. */ +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) + +# ifdef SHARED +/* It doesn't make sense to send libc-internal strchr calls through a PLT. */ + .globl __GI_strchr; __GI_strchr = __strchr_generic +# endif +#endif + +#include "../strchr.S" diff --git a/sysdeps/aarch64/multiarch/strchr_sve2.S b/sysdeps/aarch64/multiarch/strchr_sve2.S new file mode 100644 index 0000000000000000000000000000000000000000..4f24c65e2915a14286a0ea922ae8325bde3561c1 --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchr_sve2.S @@ -0,0 +1,116 @@ +/* Optimized strchr for SVE2. + Copyright (C) 2026 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#include + +/* Assumptions: + * + * SVE2, unaligned accesses. + * + */ + +.arch armv9-a+sve2 + +#if defined (BUILD_STRCHRNUL) +#define STRCHR __strchrnul_sve2 +#else +#define STRCHR __strchr_sve2 +#endif + +#define chrin w1 // int chr. +#define result x0 // Return. + +#define src x0 // Current data pointer. +#define vlalign x2 // Alignment mask. +#define tmp x3 + +#define zdata z0 // Data. +#define zchr_nul z1 // chr and NUL data for search. + +#define pall p0 // All active predicate. +#define pchr_nul p1 // chr and NUL search result. +#define pchr p2 // chr search result. +#define psrc p3 // Mask of chars before srcin. +#define pfirst p4 // First found. + +ENTRY (STRCHR) + ptrue pall.b + and chrin, chrin, 0xff // Set zeros after first byte. + dup zchr_nul.h, chrin // Initialize chr and NUL for search. + + and tmp, src, 0xfff // Check for pagecross of 2VL reads. + incb tmp, all, mul #2 + tbnz tmp, 12, L(pagecross) + + ld1b {zdata.b}, pall/z, [src] + match pchr_nul.b, pall/z, zdata.b, zchr_nul.b + b.any L(firstmatch) + ld1b {zdata.b}, pall/z, [src, 1, mul vl] + match pchr_nul.b, pall/z, zdata.b, zchr_nul.b + b.none L(loop_start) + incb src +L(firstmatch): + brkb pfirst.b, pall/z, pchr_nul.b // Find first element. + incp src, pfirst.b // Increment by element index. +#if !defined (BUILD_STRCHRNUL) + cbz chrin, 1f // Special case NUL chrin. + cmpne pchr.b, pchr_nul/z, zdata.b, 0 // Matched chrin? + csel result, src, xzr, first // If it is NUL, return null. +1: ret + nop +#else + ret +#endif +L(loop_start): + mov vlalign, 0 + decb vlalign + and src, src, vlalign // Align src to VL. +L(loop): + ld1b {zdata.b}, pall/z, [src, 2, mul vl] + incb src, all, mul #2 + match pchr_nul.b, pall/z, zdata.b, zchr_nul.b + b.any L(loop_end) +L(loop2): + ld1b {zdata.b}, pall/z, [src, 1, mul vl] + match pchr_nul.b, pall/z, zdata.b, zchr_nul.b + b.none L(loop) + incb src +L(loop_end): + brkb pfirst.b, pall/z, pchr_nul.b // Find first element. + incp src, pfirst.b // Increment by element index. +#if !defined (BUILD_STRCHRNUL) + cbz chrin, 2f // Special case NUL chrin. + cmpne pchr.b, pchr_nul/z, zdata.b, 0 // Matched chrin? + csel result, src, xzr, first // If it is NUL, return null. +#endif +2: ret + + .p2align 3 +L(pagecross): + mov vlalign, 0 + decb vlalign + orn tmp, src, vlalign // Find end of VL aligned src. + whilels psrc.b, src, tmp // Mask off elements. + ld1b {zdata.b}, psrc/z, [src] + match pchr_nul.b, psrc/z, zdata.b, zchr_nul.b + b.any L(firstmatch) + and src, src, vlalign // Align src to VL. + b L(loop2) + +END (STRCHR) diff --git a/sysdeps/aarch64/strchr.S b/sysdeps/aarch64/strchr.S index e793fa92f15b63495eab94c83534c5830258bfec..daa28acbfc8b278897d47ec403dc945eb3ba3555 100644 --- a/sysdeps/aarch64/strchr.S +++ b/sysdeps/aarch64/strchr.S @@ -26,6 +26,10 @@ * MTE compatible. */ +#ifndef STRCHR +# define STRCHR strchr +#endif + #define srcin x0 #define chrin w1 #define result x0 @@ -51,7 +55,7 @@ zeroes gives the position of the matching byte if it is a multiple of 4. If it is not a multiple of 4, there was no match. */ -ENTRY (strchr) +ENTRY (STRCHR) bic src, srcin, 15 dup vrepchr.16b, chrin ld1 {vdata.16b}, [src] @@ -109,6 +113,6 @@ L(end): csel result, result, xzr, eq ret -END (strchr) -libc_hidden_builtin_def (strchr) -weak_alias (strchr, index) +END (STRCHR) +libc_hidden_builtin_def (STRCHR) +weak_alias (STRCHR, index) From patchwork Mon Aug 3 15:49:59 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 140498 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A41564BAE7F9 for ; Mon, 3 Aug 2026 15:51:27 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A41564BAE7F9 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=bnkDJMdB; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=bnkDJMdB X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from DB3PR0202CU003.outbound.protection.outlook.com (mail-northeuropeazon11010002.outbound.protection.outlook.com [52.101.84.2]) by sourceware.org (Postfix) with ESMTPS id C9E794BA2E37 for ; Mon, 3 Aug 2026 15:50:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C9E794BA2E37 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org C9E794BA2E37 Authentication-Results: sourceware.org; arc=pass smtp.remote-ip=52.101.84.2 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1785772251; cv=pass; b=KP5W8tV/JK7CBvA2hbr/1KZNrqp5nZTHgs1vV3ETLXC72KLQS5UV+jjP0hhEYNbLLpdIpRr7oNh1HFLo28qwfCqPJdjfsbOTbaMNC5vl+QQI2uTCJqbPw1qcSLmNYY1Ldabg9mzO8CSieXuZaTArI7pfYSdKG0SWjqJkaSWohIQ= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1785772251; c=relaxed/simple; bh=2imDXFFNWcI1Ol+QAaKQA66X31F5p2VFkBNQhdaDiQc=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=XaJUIJQCbupTF5D7wjxIjXkM6fAKAPA//17iSOR4MTfAMNMPko0Lewm3mdheLP6Mu9SqDNXgPuFLHxuVxcbZFjrxvxakMzxDDT+1V9+OmCfdBQ91CXrI7mzFBtZ3yWwzCX5FAr/+58vLdaPmitOfmPghW2ch7nGV+7gcGJDtWqU= ARC-Authentication-Results: i=3; sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=bnkDJMdB; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=bnkDJMdB DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C9E794BA2E37 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=uTKtFgJQSktq8sjdyj/RttMR3aJ+rt8AgtkzTuSJ/bdMVhbWBkuuntKawP9uHJ/ttxUR89xSVq3nJelWaZJsoCRBEbmGjgSH1V4YEoF6OoA893LNtRsJmR4SSwxvQYyx40iE0IwEaKVMwg/dNo/G3z0m4CTp2okVYvHKHJSCvM9b0tj9deL1B8fm6cm4m3dhq3GVY7IDgbcHPfx/JoGEdrzgcNLnRiI55nJsOU+Xe5CQzyE01fGcrlYD4aZ6LAfYaWfbK9pKLO7VRtcoxCxIpvrVg5yjZAOlrI+rCUIoKH4EQr8SGQpLFePKFx0sb8A1QbQ+IMSReoOAIIUG1nj1MQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Rbs7Ci4DyAMolMn0oemM7PFA/wvKqDK6aV9+ymONmQo=; b=qeSoS19iXv8GvuPHTvnWc80fO1nuoZypM+tKbq2PfCdYi75FNn7VdrLVoyPl/NvWoYcr5082IVFRlg6j8CITDAJAVrI7jj4cjemkjszpgdRgIj//AIQa92vWoUpKolegbKFG57Bh6hYQW0cZyo9QfdywAYm7bnk2Bt3zTUVVR1TSVYeoLLdIvSZ73dE8YZNsNp3nlhFdVf0QpS1KCrfz3VsomuzVaGrJJVHI8kLLslzkCOouugBB9QWPzeQrhgcIULV9s3gaGZAwKi9DEbwSrF2UlJXn1oWAUawCXNAuBpQd9mkBFhaUNK9IBKdr7qK9KmGZAYcbBrTJQN2LFSpYYA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Rbs7Ci4DyAMolMn0oemM7PFA/wvKqDK6aV9+ymONmQo=; b=bnkDJMdBppNzIAWc1GT2N+1JeLEuMlMCBcEsDAy8sXao2CdtiKefZZld+Ag3LlZ9kA22YhPzgVchznXc1m/UePxYipVsK7XqraIjxbsvqRkHHx9pfGVt/0fobr0acO9Ajn2wjW2cRe02CWVQ61WqAdYiLpZAtDPzo2/BXi7de80= Received: from DU2PR04CA0172.eurprd04.prod.outlook.com (2603:10a6:10:2b0::27) by AS2PR08MB9428.eurprd08.prod.outlook.com (2603:10a6:20b:5e8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.18; Mon, 3 Aug 2026 15:50:38 +0000 Received: from DU6PEPF0000B620.eurprd02.prod.outlook.com (2603:10a6:10:2b0:cafe::a6) by DU2PR04CA0172.outlook.office365.com (2603:10a6:10:2b0::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.270.17 via Frontend Transport; Mon, 3 Aug 2026 15:50:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by DU6PEPF0000B620.mail.protection.outlook.com (10.167.8.136) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.292.8 via Frontend Transport; Mon, 3 Aug 2026 15:50:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pjeDGQJzQ3zd9t3oLFER1mKdZFMx1pRnQYcw2WURpP6DIp6fLyXZUa4RDNdipXAkECK4CglTsd3nHZeX2ykZ1ZPQnjP/VZ9Ie8em3A8QeJ3MCdm/Fk1Zv8Oqu6BYgzafR0GYNY6SFJEgWxOBCksjPygJWNHi2H/tih/96uZb+0rQOkTqodLi0NKKEZ35rEHpUTSb2fbV3AcRl+PeuRkFP8O05ezDASv+W38HRTuL0gK5zQMwZLf7SeFKdGNjJ8T1RuEdPAx2ZyUkZm1ZBdxV2/x1Ox5pEfRxAszjRW8CqSCH3rFwzViH493M1cD0HlRn2Y4ysASXEhNGGAJFWzk/YA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Rbs7Ci4DyAMolMn0oemM7PFA/wvKqDK6aV9+ymONmQo=; b=bKPH9pkvaavN9XsdoH+xKsgJwNrOAZUzZcy7uUqZbPowzcgF6FVTC1B40rzbORbdBh3xzG1pny/tcXbNgOalgisOPtNQuVF/qVm4Zhx7rRmBcAxPvpL1t1gc1dHohXcjBRfbC7utSLbiQilM7I5jHwjNDFA//4RgrFb8HCHLGK+1ltZ9/bUhMLjAIVc7/NtVVIxnklcBJzTme+wpFHULxvrSg6mCTnbE2XxPEQwQqT3dNxah2uszNfXrq1ppjaKJ4HbeBxAZjojO8586Bkel4+LtDrMt3suIss/MO1ErnbApidwXM+VD9HrnyBfte9ohhq7FZt0nFBuirnQvA66xnQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Rbs7Ci4DyAMolMn0oemM7PFA/wvKqDK6aV9+ymONmQo=; b=bnkDJMdBppNzIAWc1GT2N+1JeLEuMlMCBcEsDAy8sXao2CdtiKefZZld+Ag3LlZ9kA22YhPzgVchznXc1m/UePxYipVsK7XqraIjxbsvqRkHHx9pfGVt/0fobr0acO9Ajn2wjW2cRe02CWVQ61WqAdYiLpZAtDPzo2/BXi7de80= Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20) by PA4PR08MB7617.eurprd08.prod.outlook.com (2603:10a6:102:272::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.17; Mon, 3 Aug 2026 15:50:00 +0000 Received: from PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::5821:707a:c30b:dd9a]) by PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::5821:707a:c30b:dd9a%5]) with mapi id 15.21.0270.016; Mon, 3 Aug 2026 15:50:00 +0000 From: Wilco Dijkstra To: GNU C Library Subject: [PATCH 2/2] AArch64: Add SVE2 strchrnul Thread-Topic: [PATCH 2/2] AArch64: Add SVE2 strchrnul Thread-Index: AQHdI1+9NFrrtQeYEUiFMguDegfMYA== Date: Mon, 3 Aug 2026 15:49:59 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: PAWPR08MB8982:EE_|PA4PR08MB7617:EE_|DU6PEPF0000B620:EE_|AS2PR08MB9428:EE_ X-MS-Office365-Filtering-Correlation-Id: 803f78b6-2d5e-4fc5-f4a1-08def176f6ca x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|376014|366016|1800799024|23010399003|38070700021|13003099007|56012099006|10067099003|11063799006|5023799004|22082099003|18002099003; X-Microsoft-Antispam-Message-Info-Original: skbUF6I7qe9OXEGZhnweoNNirQFtdM7e6pPN4qNnI5KwuCbh47xC5/09RVwFCtNa9ERFTcNabo6hwitqOur3hz3YFEQb3p0PZyNcW9spvoqW/NaEQiRhZnIewYC65iLcb1bwspht6YvSudevWIq7CJDQGTRSNSYangOdp/YgWH1j+LW20xwvxIS0W6rpl1S6DQp/KeR5Z6jhUWzHEYs+GOb+E+UM/kDyQI1MQE6f8XSHVUXJBT6jZcL7HbuMEViavqUvfADhOnD5DALNMw6penO+Ake0h7iB9LTpmLJ185skZi7OJZgut5iz6t1NcpvPtk9UbybRgPd1/HnYGC6WjN/Owd0T0lJpxXYxneGTzqTA7rI1KU32cKTZDjnVa7En7BzR7IsmCxYdB8uHt3x0SqQQ/EIYj8Hw4CO++v1QJktJj3rt1xWV5pupwOC2iA8bGGrqe/20DNM42TypGHe/wIpdgikDykiMx2kKJ9PF3GLuL++8tujvXMTsIAxf6u2j24rA/Bgv0WMJXK2FZXL05K+mPXZ7pSZzucI4sI1InCwC0xCbUFQWPB4obcdGa6W+p2ZIlUop+3EOIoSSoSFo6aimv5bPc5ofFjHH4RWIMBWqJn397rC/vBjT2DiZ7c5HrOanEhCkAvHzbh/pAXmQe1jmIhhOTgjnyHzo5ZQ5jaimKKY7VrbEWiEBvvqu5kqM9PBlDe9vRHXGs/LBjPrNQJ4aRUEjWWT8ZuRbi244VKs= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAWPR08MB8982.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024)(23010399003)(38070700021)(13003099007)(56012099006)(10067099003)(11063799006)(5023799004)(22082099003)(18002099003); DIR:OUT; SFP:1101; MIME-Version: 1.0 X-Exchange-RoutingPolicyChecked: cQ87y/k08L91+xgUhqdSRvlNxEhVGX8Qvu/KHI8lKxm7klmFHpDU/Pq0mNkSyutp3FQRsaBKHOB4DwcqBocZZ2S6WkOgfGqShmztMkgB66jpIzqybm45NpVdUesNTP/pZY9ndWeN+o8xsj3v8+YitlMXpeDpZby1Pb623eHdzhnbqv+1ScHpnNiL9rBLhzAixgIfuVkjaQfX8boFlb6jHJj5d6TcGs3weZcH8djf1uxczbp/Iaz4CopLreKHsO6+n3m8xUkruVeVBY3QetiuL8JAAr5sbCVfGlHgGEoFqFKjE3hVA1MBRedPBE4HElwnYyJy7hcqgSJCM5H4rbeFUQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB7617 X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DU6PEPF0000B620.eurprd02.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 2000aa5b-72fc-416b-98b4-08def176e008 X-Microsoft-Antispam: BCL:0; ARA:13230040|35042699022|376014|23010399003|36860700016|82310400026|1800799024|14060799003|22082099003|18002099003|56012099006|11063799006|5023799004|13003099007|10067099003; X-Microsoft-Antispam-Message-Info: 4mFM6EIQUt1M+IjS8vQ3v1To+mQkKI/4UaaFuB+0GbbkX+WpOEoGwTZ7gjS2kxN3kmu3PcoS1+2TBCVIOs9TvyJIQFVfFVzIqNP5TdGW2XRsLPywZMKehC3vY4Aj75baaIbkZo1cbkL+oQXv/UGzwryTY3oVfgTRWKI1WXYP+Je8lQ8rbYuOogH584JsPnUp5tzRU0+QWfGz0l5kYsOxNEjT01i2PdSvnhoERouv/ifbOOvEPYNH4Wl0VhCJ71WqoZnlvDum6vIiQi3ZrMF/jMs+W1AV+YYaz1KkDYX1lqV+93dHOWLmw7MD8Tjpv28HIZVf8D/EuOqeri1+YXLiXPS00OVPCW37A/51RVhgnFm+W4jgJWyIJ6p4N006qBGKCkLp3TregSIWkrJ13AxPE4943vfFQ2wfzBO2dZe08Klpcf2VmAdyqKO5gmROv3qeaPC/BeOT2lfhWicXvZYvcB2/8GNJ8rRuQVqddMG7sFAmXxlHxPdz8klV87TlzeTXbp6+sVWMiIpTLQPMXkt58onMKiy4uB8huRk1e64Ak0WuJbI4tWfUDl1ZXwVLs8uH3FgFO2izx/iP0Jja2VV3k6sLnJ2RivZJHyaL8FpZPxzhMlu7PPmaXWRTFdKI5AvNIO5BWGUIHKIUYJ+jwkV2qGKhhMfDkVKta32e63SUBpAeU9UzPdmkRKuaGMKepCTjIBTIB4GY1DQ4nYaNdab6qg== X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(35042699022)(376014)(23010399003)(36860700016)(82310400026)(1800799024)(14060799003)(22082099003)(18002099003)(56012099006)(11063799006)(5023799004)(13003099007)(10067099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DLY9F0EuOPkUBCf5zUPOSJ5x3w/NBc5S0M/2I9Jmf2RC/e2LWVciqjlvP+zdHu2c6E/5GbtOgy9tW+BUbpRdxaNliWeMhPLwjMzuZUK/281q2SURNXh4l83aYcY+yIxfSva4QYmOiomL6OMGpxENYv59zChkv2ngbQrB2kVgso7t8igJLTsxvL/v3OvPcfRHsa5ZNzQM7ouPwCr8/SI2AFnKhClyp/Eu2WLDmw5yWRg6zljbE/mN0CTKpxIBM5TkcL57vxTUfco6mtKSUfdyJx98p+vfBH8nfmXIlF2Q8yL3AAQEYz4TKVon0MLZgY2AvIN6pV0UD8a3PltJaffm4nvzhgdK2hPc+GymJ93dAiuoANJbbDaLIqQ9IT/6+xJdVF3UMRbDuQkKtiY38pbHkf8V0rEIC2w8zZoYk0cHPHD5WbTtiETz31bWwSOUm3IX X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2026 15:50:38.1656 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 803f78b6-2d5e-4fc5-f4a1-08def176f6ca X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000B620.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB9428 X-Spam-Status: No, score=-10.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, KAM_LOTSOFHASH, KAM_SHORT, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org Add an SVE2 strchrnul implementation and ifunc support based on strchr-sve.S. Use MATCH to check for both the input char and NUL. Performance of bench-strchrnul improves ~34% on Neoverse V2. diff --git a/sysdeps/aarch64/multiarch/Makefile b/sysdeps/aarch64/multiarch/Makefile index 9616231c042c03088384b07ea8c5465d413048d1..8c279a81f7bd53f07bcab49ba17680355f57b5ab 100644 --- a/sysdeps/aarch64/multiarch/Makefile +++ b/sysdeps/aarch64/multiarch/Makefile @@ -18,6 +18,8 @@ sysdep_routines += \ memset_zva64 \ strchr_generic \ strchr_sve2 \ + strchrnul_generic \ + strchrnul_sve2 \ strlen_asimd \ strlen_generic \ # sysdep_routines diff --git a/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h index 50b2581455c94dc2bd38f8e5ba9a357d1b79e89f..74fdca91baa562316f8f0a4d2450e9dd81aabb9c 100644 --- a/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h +++ b/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h @@ -23,6 +23,8 @@ asm ("memset = __memset_generic"); asm ("strlen = __strlen_generic"); #ifndef SHARED asm ("strchr = __strchr_generic"); +asm ("strchrnul = __strchrnul_generic"); +asm ("__strchrnul = __strchrnul_generic"); asm ("memcpy = __memcpy_generic"); asm ("memmove = __memmove_generic"); asm ("memcmp = __memcmp_generic"); diff --git a/sysdeps/aarch64/multiarch/ifunc-impl-list.c b/sysdeps/aarch64/multiarch/ifunc-impl-list.c index a0c93147c532d935d562a66ac67ebaa5326dd3bd..5f5350850dd5d8270c2675fdbdc3be0f110f7116 100644 --- a/sysdeps/aarch64/multiarch/ifunc-impl-list.c +++ b/sysdeps/aarch64/multiarch/ifunc-impl-list.c @@ -63,5 +63,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, IFUNC_IMPL (i, name, strchr, IFUNC_IMPL_ADD (array, i, strchr, sve2 && !mte, __strchr_sve2) IFUNC_IMPL_ADD (array, i, strchr, 1, __strchr_generic)) + IFUNC_IMPL (i, name, strchrnul, + IFUNC_IMPL_ADD (array, i, strchrnul, sve2 && !mte, __strchrnul_sve2) + IFUNC_IMPL_ADD (array, i, strchrnul, 1, __strchrnul_generic)) return 0; } diff --git a/sysdeps/aarch64/multiarch/strchrnul.c b/sysdeps/aarch64/multiarch/strchrnul.c new file mode 100644 index 0000000000000000000000000000000000000000..d09508880655bada4ca9185a5cedf28714d10a31 --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchrnul.c @@ -0,0 +1,41 @@ +/* Multiple versions of strchrnul. AArch64 version. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Define multiple versions only for the definition in libc. */ + +#if IS_IN (libc) +/* Redefine strchrnul so that the compiler won't complain about the type + mismatch with the IFUNC selector in weak_alias, below. */ +# undef strchrnul +# undef __strchrnul +# define strchrnul __redirect_strchrnul +# define __strchrnul __redirect___strchrnul +# include +# include +# undef strchrnul +# undef __strchrnul + +extern __typeof (__redirect_strchrnul) __strchrnul attribute_hidden; + +extern __typeof (__redirect_strchrnul) __strchrnul_generic attribute_hidden; +extern __typeof (__redirect_strchrnul) __strchrnul_sve2 attribute_hidden; + +libc_ifunc (__strchrnul, (sve2 && !mte ? __strchrnul_sve2 : __strchrnul_generic)); + +weak_alias (__strchrnul, strchrnul); +#endif diff --git a/sysdeps/aarch64/multiarch/strchrnul_generic.S b/sysdeps/aarch64/multiarch/strchrnul_generic.S new file mode 100644 index 0000000000000000000000000000000000000000..e5a6380e775eee3b97db575c1e1ea99860922f11 --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchrnul_generic.S @@ -0,0 +1,38 @@ +/* A generic optimized strchrnul implementation for AArch64. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN (libc) + +# define STRCHRNUL __strchrnul_generic + +/* Do not hide the generic version of strchr, we use it internally. */ +# undef hidden_def +# define hidden_def(name) + +# undef weak_alias +# define weak_alias(a, b) + +# ifdef SHARED +/* It doesn't make sense to send libc-internal strchrnul calls through a PLT. */ + .globl __GI___strchrnul; __GI___strchrnul = __strchrnul_generic +# endif +#endif + +#include "../strchrnul.S" diff --git a/sysdeps/aarch64/multiarch/strchrnul_sve2.S b/sysdeps/aarch64/multiarch/strchrnul_sve2.S new file mode 100644 index 0000000000000000000000000000000000000000..3a037446d851e060790369e7ea78be8573fed56b --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchrnul_sve2.S @@ -0,0 +1,22 @@ +/* Optimized strchrnul for SVE2. + Copyright (C) 2026 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#define BUILD_STRCHRNUL + +#include "strchr_sve2.S" diff --git a/sysdeps/aarch64/strchrnul.S b/sysdeps/aarch64/strchrnul.S index c80807ea73d85493982e60d023c4c137b6c9b311..2bc535fc4f77bed5d558719fad2837a9d9791eef 100644 --- a/sysdeps/aarch64/strchrnul.S +++ b/sysdeps/aarch64/strchrnul.S @@ -26,6 +26,10 @@ * MTE compatible. */ +#ifndef STRCHRNUL +# define STRCHRNUL __strchrnul +#endif + #define srcin x0 #define chrin w1 #define result x0 @@ -50,7 +54,7 @@ which things occur in the original string, counting leading zeros identifies exactly which byte matched. */ -ENTRY (__strchrnul) +ENTRY (STRCHRNUL) bic src, srcin, 15 dup vrepchr.16b, chrin ld1 {vdata.16b}, [src] @@ -93,6 +97,6 @@ L(end): add result, src, tmp1, lsr 2 ret -END(__strchrnul) -libc_hidden_def (__strchrnul) -weak_alias (__strchrnul, strchrnul) +END(STRCHRNUL) +hidden_def (__strchrnul) +weak_alias (STRCHRNUL, strchrnul) diff --git a/sysdeps/aarch64/strspn.S b/sysdeps/aarch64/strspn.S index 42f3b82a944503aa726bc8bdb61105721d0b959a..f70047bec04fe601a942895140e7cd98cdb6a522 100644 --- a/sysdeps/aarch64/strspn.S +++ b/sysdeps/aarch64/strspn.S @@ -51,7 +51,7 @@ #define table_b v5 #define sevens v7.16b -ENTRY(STRSPN) +ENTRY (STRSPN) ldrb w2, [set] cbz w2, L(early) #ifdef USE_AS_STRCSPN @@ -128,7 +128,7 @@ L(early): mov w1, w2 mov fp, sp mov x19, x0 - bl __strchrnul + bl HIDDEN_JUMPTARGET (__strchrnul) sub x0, x0, x19 ldr x19, [sp, 16] ldp fp, lr, [sp], 32 @@ -140,7 +140,7 @@ L(early): mov w0, 0 #endif ret -END(STRSPN) +END (STRSPN) #undef set -libc_hidden_def(STRSPN) +libc_hidden_def (STRSPN)