From patchwork Mon Aug 3 15:48:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 140497 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 35E364BA2E3C for ; Mon, 3 Aug 2026 15:50:05 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 35E364BA2E3C Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=Zu6jpcC1; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=Zu6jpcC1 X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from GVXPR05CU001.outbound.protection.outlook.com (mail-swedencentralazlp170130007.outbound.protection.outlook.com [IPv6:2a01:111:f403:c202::7]) by sourceware.org (Postfix) with ESMTPS id 4CD4D4BA799F for ; Mon, 3 Aug 2026 15:49:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4CD4D4BA799F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4CD4D4BA799F Authentication-Results: sourceware.org; arc=pass smtp.remote-ip=2a01:111:f403:c202::7 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1785772166; cv=pass; b=Em/65ePu2mlEfhljDlcG2V3H00JIaL4E6fI/pLpQ5zeLdja9y7E6p2pNupLgqD+Xp3XdoaBvnjavDNm4PmSposh+kjwYAtuSsQNWhfg5jxTzHJbbKTmfmnAzuDeWT/rMrhS0PBe+kI4LVaI3GLrqc8Kwcb2XRD6Xuhu/azo4lnk= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1785772166; c=relaxed/simple; bh=kFX+d7L1lm+k267lIanO0QdgLbYfNXQgarIXaLvU920=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=rkD/CcoIi/A7bU5wXSvMLoFaQAH7upYiLUvH8jqrAPRmKJ3ScctNuRtK2jLovqXXhIfNN3CPvAkA2U/rI4nVEE1bKW03cYW4p3Z4XKfZy/vYCigD3So0uc5ES4qDJbV3XhxxQ2Xo8l5GvP+Ilg8clOGUwlO1nD3Xt4nRu1JqjpE= ARC-Authentication-Results: i=3; sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=Zu6jpcC1; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=Zu6jpcC1 DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4CD4D4BA799F ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=bsqBF2/dtCWCl6Ctq9hdtqmXu9BK9hgdQdZVUWMEvrUO0vnzRK6jsBVAw+Cbr4DTV6Yt4OatMrP81b2ueL5uD/Q41MRCkK4Sd5t89B5/3VP3diWG1r3zOHoJsgv6hfLEXBwfZ4wsr0vrqhK/2D3iR5rDoXMdDCvHWoHK7j5Oqh+2fF6sArFGoahTd1LQYY1lm4U1YdQgwHM30A1Ah3yujFU3b8/7opl8VmiflBVAawU6kkqsVCbXxsu0MGYWKdsxVTgL/27VT9UnxnySf1Pqp6T4iRKSYem6uXy9PGphCBdFbqNFGk8Kg0zSh/sg2Hv4l3xhVEkP7R4G1fGPwbRVCA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m0Og/4qqdb0chijflyzw2Jw12DLOvn8TRF8MAijyacM=; b=x3IRpnK827EDA0DIoiEFe2E+Gnal3DVSThTriqQLSRtmhvLJYZttBaT2UYiAdB0mqClJDLH6SshCitw5CVh9tQx1L707Dea1S/SpTYVUWA1BiG6vUcC8TI/E2R7qPw72tvgYXr2ciClr9h+8NEX04Sl5MRM4Wgr2d91uIUNo0n1dKNwlySTqjyYJlXML9PhTJH2tF5aILVPHG1kIL/26tYxjyYfGihaiaH3Ro+EVeDE5ao7IXhInWPN44Keu0+orqJeNoVWqZ9TGceiEZdvTzLtMKdh10q6PaYkHPJJWcluOctsp+MkKl84Zy4nLjcys52yCorp1nfBQxWH2v3BN0A== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m0Og/4qqdb0chijflyzw2Jw12DLOvn8TRF8MAijyacM=; b=Zu6jpcC1icK53nQp8OIPbwCn97MzfiETsRij1olaA8oq00BGM1pXPk1DCdoTdvomzQVn66OXd7DC1gPJoEQgVTyvXjjVq/fPKBE//mW7HPgRPzBQaGNZAi9Sn5vKEvMLis2iHrRBHKQDbRXjeOuAbdfg+3/guvgYl4gMUXFICCs= Received: from AS4P189CA0015.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:5d7::19) by DU0PR08MB9558.eurprd08.prod.outlook.com (2603:10a6:10:44d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.17; Mon, 3 Aug 2026 15:49:10 +0000 Received: from AMS1EPF0000008F.eurprd05.prod.outlook.com (2603:10a6:20b:5d7:cafe::7e) by AS4P189CA0015.outlook.office365.com (2603:10a6:20b:5d7::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.270.18 via Frontend Transport; Mon, 3 Aug 2026 15:49:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AMS1EPF0000008F.mail.protection.outlook.com (10.167.242.86) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.292.8 via Frontend Transport; Mon, 3 Aug 2026 15:49:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yFWFk2T75+PpPsy4ibHSDbiFEUlmx8PLGQMORyg+ePu9u98dcSSblOA54HwSqLoYqYNlA4WbJsbqm30OuTfH2yoUNrfCIjftaTJdPBXwL2MguaYPDgN/jP+VAwUKhEZnQitMzqxFL87hcqplh5Ebs723YWv+igBnemiRu+dLouNQVsHtNl9eAl6xIAi/IaNwHITYKL/lxQcO2TC/h2gweJbkewW9M/LUuzMdl72aSy0AuEolki+n63ThJM48qBbyaIkEHJhNUGOoBImp3e94YRT0VbnQBGD1QTP/FT/vdCcbfUy7OSQzoOEolEfPeHHp+1rDp2nrcGkTD/OM+LwZmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m0Og/4qqdb0chijflyzw2Jw12DLOvn8TRF8MAijyacM=; b=O8yhXlG5mdsXLe+V5q9ru6WpVQyCS+s1y4w8rqTE68Mee8dbTLd5MuzhKUTSAZjgGU4W5R+2bAd3MiovccGb01gOAMqklpzqOY8MgAEL8ydxK/HWOU2j+DzoNuAxevOwhMSSRTYVgXCqeds9Y21hP18pQUGKtMUdR5e4C8bB2hQv/SB98D3cPYjWWozucEH0HGA0299HXtOIMLfMcSdZOSQDUvTd2N8oKAH9uT5ODIpEGLi5CDhb5wp56F+dpzkGjAdmKUWttK9kr/stXzkF6RBGlNgS388aQAZKVpvQVYeww+dmE+VMfcl1eyNQNqqF+WHCTovtp4qBGZY9SgARWA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m0Og/4qqdb0chijflyzw2Jw12DLOvn8TRF8MAijyacM=; b=Zu6jpcC1icK53nQp8OIPbwCn97MzfiETsRij1olaA8oq00BGM1pXPk1DCdoTdvomzQVn66OXd7DC1gPJoEQgVTyvXjjVq/fPKBE//mW7HPgRPzBQaGNZAi9Sn5vKEvMLis2iHrRBHKQDbRXjeOuAbdfg+3/guvgYl4gMUXFICCs= Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20) by PA4PR08MB7617.eurprd08.prod.outlook.com (2603:10a6:102:272::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.17; Mon, 3 Aug 2026 15:48:15 +0000 Received: from PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::5821:707a:c30b:dd9a]) by PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::5821:707a:c30b:dd9a%5]) with mapi id 15.21.0270.016; Mon, 3 Aug 2026 15:48:14 +0000 From: Wilco Dijkstra To: GNU C Library CC: Wilco Dijkstra Subject: [PATCH 1/2] AArch64: Add SVE2 strchr Thread-Topic: [PATCH 1/2] AArch64: Add SVE2 strchr Thread-Index: AQHdI19MU8LWAn4+AUieLfVb/H8Mtw== Date: Mon, 3 Aug 2026 15:48:14 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: PAWPR08MB8982:EE_|PA4PR08MB7617:EE_|AMS1EPF0000008F:EE_|DU0PR08MB9558:EE_ X-MS-Office365-Filtering-Correlation-Id: 73e1115d-de81-491c-49ee-08def176c279 x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|376014|366016|1800799024|23010399003|38070700021|13003099007|56012099006|10067099003|11063799006|5023799004|18002099003; X-Microsoft-Antispam-Message-Info-Original: kJzU5FNPmoIrITnifTYNTRzxwRN7Ek1ft8/3BmeI+9GXhZNZrEu2xL/NGVKzrpJvvo4KdpdLXRPxVkS1mQZA+xiELWGdM447CxbOqd7KvUaw+Ic0BKGHMtO8l4MKeRlhDZVcSy7FqMc9la/y5CofDEdnCyF6F9Jaqpy5FKQ/6w3DB0E9fW0vcRex+GUr/Ym+IvAQ9zgzzexllmUAia7YiHtHL/hB1bKtnuDHxiu/6fcMpd0Rccx3K7Per78Lbn7JC4jkzn+JJ7ovW7Dk5aTCTVPsE8IEMuRe3biq3QU2i7nhoWE1RUGj4ewiX6cWHCYmyTFpjPihs3bKEovtw5CuZ+Wpg5CQeAHmy+xRe6jlrnbgHnmUhL5gt7uNV9YARqKLd2lNChOH5YR9KBDHuFNwrkPA0lhhMX2PQgN0Gfn0CXe7LZdyadI4YzVBXZwREqqeRM5AGhU0tsDS9TV7wfKu1ESeGzpIGKr3x8xzdwuWgpPiN6T9Gg9EDwgVvRKUKQcRXBOQbiymDOgXsDlAcNhxAsl75pB1qZlrDoLekwCNcw/qX6zz2qi8iQ3HEuAelQQzgEwmEA9qeZph2WvffQ4gXqRw9mCKwNB1VMP0JA/0tDprbfbPh4UfWtJiSPlrmI32lghvmYTGT+25qhJm9Sh3bREzxmiLcuIs0JQDf5Y6MLlPZi4BUhlBUfSgQq+sAtSBKVJSmFnOl0w/6wnHtTNmfP9EEJMouysQjEW9CfgaZJk= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAWPR08MB8982.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024)(23010399003)(38070700021)(13003099007)(56012099006)(10067099003)(11063799006)(5023799004)(18002099003); DIR:OUT; SFP:1101; MIME-Version: 1.0 X-Exchange-RoutingPolicyChecked: cjW3rRG3KUQkyUSRr7YCNXDX2FLobq6UzhWl2MxOm35wPQco/3zWDUlktwFwiGbR6ULLdpapev9QmBBwyGj3jXA0aXe7i52OE9kgEvDHQl+0utZX2GW2Y390Q77uxq7rnMk0ZXJMOkY8XzwK3pTJlGHULwYtWWvAUyTjOz8Y4oFcADHNV7bPpEvTVqFO7jdXjecEYzeRYfQF96yQ1fxozAPxuxpaxiuaLtse41rR4FWWBbs5rrmui21sxz5Xoz2i7DyNVY3AIvjlAOEXJS944gHm5VJSVMVaX3y45SVZlOgKfJE0OVtiAhOTcBI814EpaWkvhtGx+hvlmTDj3FYPIg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB7617 X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS1EPF0000008F.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 70d5aa4a-7f70-44fe-bfa1-08def176a12a X-Microsoft-Antispam: BCL:0; ARA:13230040|35042699022|1800799024|14060799003|376014|36860700016|23010399003|82310400026|18002099003|13003099007|10067099003|11063799006|5023799004|56012099006; X-Microsoft-Antispam-Message-Info: mKmXz27ZfKIuVn0YyjsO/74AV9eqtuiKNKzK0iLJBFV9wNTQQR7Fjd3GAJS3oMr+CumaJGEwMK4DrUXt75HYbQVlfRPXxl51u4LnHuNFOy+mvM6cDVLgSVp0eIe7Rd7TUcVFbuEXjtlm1+euAPYMhT5hNzgiotr00/mzLWYje8ak78gao1loxk3q0Tp5KBcuLAd2sGhR73cB6vfCBiKopGJ3axqq2di1VDTKo82GwjCX+U1nso9rY/14jHhC8C5u3xG6de9LC1fqWKd6tqYP7Hh10CiTfkB3uLwfOtgpGBUeoy91uDdSCgejFHFK4uaxW8rr5jIhVSSu7CZhp0ECEwsdGKyfwYJzkYLYnywY+56IP/hu4TnI/VB1gAqSIaK+wqgb+jdav3tgTU+rLzTiU/k9wfD9W9B5QRFZpAIkbkyLQV47ztXzvu709tj6vlMDUaoqn+6iDosn0IYmhDsKbH1G6XDXoVvkVpH3K/mT2Q9rrZLvCqFDN+QRw89l58xuQt9soVgZ5LPSJ9sG7prtp6W/VriTIcg/p/5EIhMKNLPPrOt45Xs4fpph/xgQJqOomlACOhbriFDOUQ6vOvSdsx1OzoOEYQdx3cgBFIHnc/FLHtJBwAVhN7bsdG8uTCeKaH4CSnEwyi++lGveOAC1zrhHMzr+yVYSYv5SeIbdRI2OYZCzmTxFoAduLK6I7tCWXd60NT5FEfx98mpVXS1FzA== X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(35042699022)(1800799024)(14060799003)(376014)(36860700016)(23010399003)(82310400026)(18002099003)(13003099007)(10067099003)(11063799006)(5023799004)(56012099006); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +h3mkw2mSvJbVm7+ErDUdOOb5Yaj8rrPhzn+Bd6WzWx3RP9Lz0slc7SzXgIytK+cjfhmjJPcv59aFmDKnyhnbEdo7vrHr1LrXrEYgB7XWkEpivhv1sP3jkU8u9OYEY3vc7Kff37QY9qnNH7aDp5PeebGgx+bMLr+b7hwkji+z1CTUASexAvi8bv5U/sIe/w3nt2xuftd8pc4TpODh2U1eJz7ymJ8nhz0S3J8Eydecqx+T8/M2Jq75J9mNH471NFMoY8atTWr2uKBYfZ0wS6WN98DKFfXCxPR2wKAzjwFks5TjEZlTFzQAEXARlRM0k1vmXbOefHBNVk4HVOpjKQjRLLj/zBbjO7i0ymEhKEy50DNOCCjawVO0n6klr13w0k2f5T4mm3JrSTkDujRq1XMTcsGZJEZpPedtIl8uaZWaNXN8X/lKIyLDId5cCYQp2zF X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2026 15:49:10.3970 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 73e1115d-de81-491c-49ee-08def176c279 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF0000008F.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9558 X-Spam-Status: No, score=-10.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_LOTSOFHASH, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org Add an SVE2 strchr implementation and ifunc support. Use MATCH to check for both the input char and NUL. Performance of bench-strchr improves ~28% on Neoverse V2. diff --git a/sysdeps/aarch64/multiarch/Makefile b/sysdeps/aarch64/multiarch/Makefile index 6294a70c252af39964a8750749db34da413e6656..9616231c042c03088384b07ea8c5465d413048d1 100644 --- a/sysdeps/aarch64/multiarch/Makefile +++ b/sysdeps/aarch64/multiarch/Makefile @@ -16,6 +16,8 @@ sysdep_routines += \ memset_oryon1 \ memset_sve_zva64 \ memset_zva64 \ + strchr_generic \ + strchr_sve2 \ strlen_asimd \ strlen_generic \ # sysdep_routines diff --git a/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h index 0910e321d24d84db951cf498c3e883c2cfdd01f5..50b2581455c94dc2bd38f8e5ba9a357d1b79e89f 100644 --- a/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h +++ b/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h @@ -22,6 +22,7 @@ asm ("memset = __memset_generic"); asm ("strlen = __strlen_generic"); #ifndef SHARED +asm ("strchr = __strchr_generic"); asm ("memcpy = __memcpy_generic"); asm ("memmove = __memmove_generic"); asm ("memcmp = __memcmp_generic"); diff --git a/sysdeps/aarch64/multiarch/ifunc-impl-list.c b/sysdeps/aarch64/multiarch/ifunc-impl-list.c index d43f6b58ee075c815b93ab2f42735e7e9f315d32..a0c93147c532d935d562a66ac67ebaa5326dd3bd 100644 --- a/sysdeps/aarch64/multiarch/ifunc-impl-list.c +++ b/sysdeps/aarch64/multiarch/ifunc-impl-list.c @@ -33,7 +33,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, INIT_ARCH (); - /* Support sysdeps/aarch64/multiarch/memcmp.c, memcpy.c, memmove.c and memset.c. */ IFUNC_IMPL (i, name, memcmp, IFUNC_IMPL_ADD (array, i, memcmp, sve, __memcmp_kunpeng950) IFUNC_IMPL_ADD (array, i, memcmp, 1, __memcmp_generic)) @@ -61,6 +60,8 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, IFUNC_IMPL (i, name, strlen, IFUNC_IMPL_ADD (array, i, strlen, !mte, __strlen_asimd) IFUNC_IMPL_ADD (array, i, strlen, 1, __strlen_generic)) - + IFUNC_IMPL (i, name, strchr, + IFUNC_IMPL_ADD (array, i, strchr, sve2 && !mte, __strchr_sve2) + IFUNC_IMPL_ADD (array, i, strchr, 1, __strchr_generic)) return 0; } diff --git a/sysdeps/aarch64/multiarch/strchr.c b/sysdeps/aarch64/multiarch/strchr.c new file mode 100644 index 0000000000000000000000000000000000000000..12f4e10bbe034ab462a4e578c69fd1d9b4afd9ed --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchr.c @@ -0,0 +1,38 @@ +/* Multiple versions of strchr. AArch64 version. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Define multiple versions only for the definition in libc. */ + +#if IS_IN (libc) +/* Redefine strchr so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef strchr +# define strchr __redirect_strchr +# include +# include + +extern __typeof (__redirect_strchr) __libc_strchr; + +extern __typeof (__redirect_strchr) __strchr_generic attribute_hidden; +extern __typeof (__redirect_strchr) __strchr_sve2 attribute_hidden; + +libc_ifunc (__libc_strchr, (sve2 && !mte ? __strchr_sve2 : __strchr_generic)); + +# undef strchr +strong_alias (__libc_strchr, strchr); +#endif diff --git a/sysdeps/aarch64/multiarch/strchr_generic.S b/sysdeps/aarch64/multiarch/strchr_generic.S new file mode 100644 index 0000000000000000000000000000000000000000..d72b0be62a89f4e5b090f9d5c1973088bad89704 --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchr_generic.S @@ -0,0 +1,35 @@ +/* A generic optimized strchr implementation for AArch64. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN (libc) + +# define STRCHR __strchr_generic + +/* Do not hide the generic version of strchr, we use it internally. */ +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) + +# ifdef SHARED +/* It doesn't make sense to send libc-internal strchr calls through a PLT. */ + .globl __GI_strchr; __GI_strchr = __strchr_generic +# endif +#endif + +#include "../strchr.S" diff --git a/sysdeps/aarch64/multiarch/strchr_sve2.S b/sysdeps/aarch64/multiarch/strchr_sve2.S new file mode 100644 index 0000000000000000000000000000000000000000..4f24c65e2915a14286a0ea922ae8325bde3561c1 --- /dev/null +++ b/sysdeps/aarch64/multiarch/strchr_sve2.S @@ -0,0 +1,116 @@ +/* Optimized strchr for SVE2. + Copyright (C) 2026 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#include + +/* Assumptions: + * + * SVE2, unaligned accesses. + * + */ + +.arch armv9-a+sve2 + +#if defined (BUILD_STRCHRNUL) +#define STRCHR __strchrnul_sve2 +#else +#define STRCHR __strchr_sve2 +#endif + +#define chrin w1 // int chr. +#define result x0 // Return. + +#define src x0 // Current data pointer. +#define vlalign x2 // Alignment mask. +#define tmp x3 + +#define zdata z0 // Data. +#define zchr_nul z1 // chr and NUL data for search. + +#define pall p0 // All active predicate. +#define pchr_nul p1 // chr and NUL search result. +#define pchr p2 // chr search result. +#define psrc p3 // Mask of chars before srcin. +#define pfirst p4 // First found. + +ENTRY (STRCHR) + ptrue pall.b + and chrin, chrin, 0xff // Set zeros after first byte. + dup zchr_nul.h, chrin // Initialize chr and NUL for search. + + and tmp, src, 0xfff // Check for pagecross of 2VL reads. + incb tmp, all, mul #2 + tbnz tmp, 12, L(pagecross) + + ld1b {zdata.b}, pall/z, [src] + match pchr_nul.b, pall/z, zdata.b, zchr_nul.b + b.any L(firstmatch) + ld1b {zdata.b}, pall/z, [src, 1, mul vl] + match pchr_nul.b, pall/z, zdata.b, zchr_nul.b + b.none L(loop_start) + incb src +L(firstmatch): + brkb pfirst.b, pall/z, pchr_nul.b // Find first element. + incp src, pfirst.b // Increment by element index. +#if !defined (BUILD_STRCHRNUL) + cbz chrin, 1f // Special case NUL chrin. + cmpne pchr.b, pchr_nul/z, zdata.b, 0 // Matched chrin? + csel result, src, xzr, first // If it is NUL, return null. +1: ret + nop +#else + ret +#endif +L(loop_start): + mov vlalign, 0 + decb vlalign + and src, src, vlalign // Align src to VL. +L(loop): + ld1b {zdata.b}, pall/z, [src, 2, mul vl] + incb src, all, mul #2 + match pchr_nul.b, pall/z, zdata.b, zchr_nul.b + b.any L(loop_end) +L(loop2): + ld1b {zdata.b}, pall/z, [src, 1, mul vl] + match pchr_nul.b, pall/z, zdata.b, zchr_nul.b + b.none L(loop) + incb src +L(loop_end): + brkb pfirst.b, pall/z, pchr_nul.b // Find first element. + incp src, pfirst.b // Increment by element index. +#if !defined (BUILD_STRCHRNUL) + cbz chrin, 2f // Special case NUL chrin. + cmpne pchr.b, pchr_nul/z, zdata.b, 0 // Matched chrin? + csel result, src, xzr, first // If it is NUL, return null. +#endif +2: ret + + .p2align 3 +L(pagecross): + mov vlalign, 0 + decb vlalign + orn tmp, src, vlalign // Find end of VL aligned src. + whilels psrc.b, src, tmp // Mask off elements. + ld1b {zdata.b}, psrc/z, [src] + match pchr_nul.b, psrc/z, zdata.b, zchr_nul.b + b.any L(firstmatch) + and src, src, vlalign // Align src to VL. + b L(loop2) + +END (STRCHR) diff --git a/sysdeps/aarch64/strchr.S b/sysdeps/aarch64/strchr.S index e793fa92f15b63495eab94c83534c5830258bfec..daa28acbfc8b278897d47ec403dc945eb3ba3555 100644 --- a/sysdeps/aarch64/strchr.S +++ b/sysdeps/aarch64/strchr.S @@ -26,6 +26,10 @@ * MTE compatible. */ +#ifndef STRCHR +# define STRCHR strchr +#endif + #define srcin x0 #define chrin w1 #define result x0 @@ -51,7 +55,7 @@ zeroes gives the position of the matching byte if it is a multiple of 4. If it is not a multiple of 4, there was no match. */ -ENTRY (strchr) +ENTRY (STRCHR) bic src, srcin, 15 dup vrepchr.16b, chrin ld1 {vdata.16b}, [src] @@ -109,6 +113,6 @@ L(end): csel result, result, xzr, eq ret -END (strchr) -libc_hidden_builtin_def (strchr) -weak_alias (strchr, index) +END (STRCHR) +libc_hidden_builtin_def (STRCHR) +weak_alias (STRCHR, index)