| Message ID | PAWPR08MB8982CC8DDEB31EF83DFDA98683FD9@PAWPR08MB8982.eurprd08.prod.outlook.com |
|---|---|
| State | Committed |
| Commit | 00776241776e67fc666b896c1e85770f4f3ec1e1 |
| Headers |
Return-Path: <libc-alpha-bounces+patchwork=sourceware.org@sourceware.org>
X-Original-To: patchwork@sourceware.org
Delivered-To: patchwork@sourceware.org
Received: from server2.sourceware.org (localhost [IPv6:::1])
by sourceware.org (Postfix) with ESMTP id CA2D73854380
for <patchwork@sourceware.org>; Thu, 12 Jan 2023 15:58:10 +0000 (GMT)
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CA2D73854380
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org;
s=default; t=1673539090;
bh=44LgVpI1hoLG+gQsEXOS4mREcX9dFytACINTxAHnlTM=;
h=To:CC:Subject:Date:List-Id:List-Unsubscribe:List-Archive:
List-Post:List-Help:List-Subscribe:From:Reply-To:From;
b=Vg4Q8ZsnpFHgXDQUmLkHl3ncT+upOVcCtwcbHwQnAxNiC6Udwq0DG5scECGh/TkVW
j/Igi9IZ1b4din+5hN3cMnq9V8Zx6kMMl9dRe1o+MtEvWyBh3li1F0OHyAB3NXjEZU
CMU0W/SiW7+rAahwvmxr3TXW/mhAg3R02wet8h84=
X-Original-To: libc-alpha@sourceware.org
Delivered-To: libc-alpha@sourceware.org
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
(mail-db8eur05on2062.outbound.protection.outlook.com [40.107.20.62])
by sourceware.org (Postfix) with ESMTPS id CF2973858D35
for <libc-alpha@sourceware.org>; Thu, 12 Jan 2023 15:57:47 +0000 (GMT)
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CF2973858D35
Received: from AS9PR05CA0202.eurprd05.prod.outlook.com (2603:10a6:20b:495::31)
by PA4PR08MB6238.eurprd08.prod.outlook.com (2603:10a6:102:e8::9) with
Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13; Thu, 12 Jan
2023 15:57:45 +0000
Received: from AM7EUR03FT041.eop-EUR03.prod.protection.outlook.com
(2603:10a6:20b:495:cafe::b5) by AS9PR05CA0202.outlook.office365.com
(2603:10a6:20b:495::31) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13 via Frontend
Transport; Thu, 12 Jan 2023 15:57:45 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
smtp.mailfrom=arm.com; dkim=pass (signature was verified)
header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
63.35.35.123 as permitted sender) receiver=protection.outlook.com;
client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
pr=C
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
AM7EUR03FT041.mail.protection.outlook.com (100.127.140.233) with
Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.6002.13 via Frontend Transport; Thu, 12 Jan 2023 15:57:45 +0000
Received: ("Tessian outbound 0d7b2ab0f13d:v132");
Thu, 12 Jan 2023 15:57:45 +0000
X-CheckRecipientChecked: true
X-CR-MTA-CID: 212799be7f5ea731
X-CR-MTA-TID: 64aa7808
Received: from ce3b836705db.2
by 64aa7808-outbound-1.mta.getcheckrecipient.com id
CBB250AB-1675-435E-8D95-3E484D92C158.1;
Thu, 12 Jan 2023 15:57:37 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id
ce3b836705db.2
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
Thu, 12 Jan 2023 15:57:37 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=DgroAxjWNN9Xc/Cb4szvJlG0cY2am7PQTTvCP/nNeGHtrwZ57yPzzkizy6gxRlUsFr5Ge8qhFESRyQF08McvdY3WCYe/NsIyTIWP6UQWHzA4Ii/PBe0SfZ1VV5jwKVWKPFWGwSVwUeLYahjvB2dvsIh6aIUAAT6ocEyQKWt7t3K1CphO0rtJC4rLr23dauDns85klxVSkSqI92ssRx7WhM9TB4+iY5dyzAYqHTH4ArCXQ/yJYICyxi3bo/mqddOCaasorz3le80diQu/050bOtJhfrAiSsvpYpDUTVYZUQRmaoqCD4qs9q4O3melbiMhf+/3U+vHHYaKqsqgjjTxxA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=44LgVpI1hoLG+gQsEXOS4mREcX9dFytACINTxAHnlTM=;
b=OENEk3g8eEQlPEaVgc4MnT980gHZkKb3GQQjZEXvLYPfa9v1vMWIvIV3zPluKJ85qat/wwHqsgeD7JDsRVGGEfndpxesbZmxY14Q+2UWXbSVVSzhP6/koJ2XESO8Kssn6Fmz7St6tfcp427ejUkUbq7j1Y3Fphh2hFun8N7OQ89SC717Es0TTe90kjoYGkzHI27++qgqjU0C1L251y7SZ8PTshpOxJE8h5RFIhed5vhLs2It6SsZLUlLXuhDmCHdrgsSI2osvyv10KHEqmsWlr+QJIPn4B39eXKogPGJm5AbAiZpGavmgvVE0dIqyEJ2OU3HRs8tLTm6kT9Av5Rb8w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
header.d=arm.com; arc=none
Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20)
by AS2PR08MB10111.eurprd08.prod.outlook.com (2603:10a6:20b:62d::22)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Thu, 12 Jan
2023 15:57:36 +0000
Received: from PAWPR08MB8982.eurprd08.prod.outlook.com
([fe80::66e4:4940:d096:4f7]) by PAWPR08MB8982.eurprd08.prod.outlook.com
([fe80::66e4:4940:d096:4f7%9]) with mapi id 15.20.5986.018; Thu, 12 Jan 2023
15:57:36 +0000
To: 'GNU C Library' <libc-alpha@sourceware.org>
CC: Szabolcs Nagy <Szabolcs.Nagy@arm.com>
Subject: [PATCH] AArch64: Optimize memrchr
Thread-Topic: [PATCH] AArch64: Optimize memrchr
Thread-Index: AQHZJp5/hJ/UrlI3MUKRFxDXOPvwPQ==
Date: Thu, 12 Jan 2023 15:57:35 +0000
Message-ID:
<PAWPR08MB8982CC8DDEB31EF83DFDA98683FD9@PAWPR08MB8982.eurprd08.prod.outlook.com>
Accept-Language: en-GB, en-US
Content-Language: en-GB
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
Authentication-Results-Original: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
PAWPR08MB8982:EE_|AS2PR08MB10111:EE_|AM7EUR03FT041:EE_|PA4PR08MB6238:EE_
X-MS-Office365-Filtering-Correlation-Id: 2a81bc9f-c497-4ebe-72a7-08daf4b5beec
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
l+7ky9jkHJEob8AjDVzHWvRB0HPmRLEjV483pV7yUxHcWb6UbAW+HL86Pv97CVXNVtMY0uHtmFOaMdcdoR1k8x7rEDu/SF8VubjZsvYeGAOg1d+q4X11TVMbeucRkxkU7a8zrYhe9xV8aA7eBPuvENeAHnfke0wJrGvbZhT6pshIcu1LixioHer4WKs4OdKAWGicC0y2cClXVJ0QGJF87YqZEsATYLtmAh85105pa9mrC51uX6mzgpaL0De5eXaYDmIskOJkr4yzuIIrS7aXxolzvdyX1eaIzycMyHjAIxpX9mSG9YMrTOLmTEt4pJhsgmLdYzwgdsrEciEZG+1MyDDzO46irXtWllopexefUf+/uxgX66YykCSCEzlPuS5n2038zwCdHOxx3QSiqDaZgk+b26OqG/dMF6i2qJMDIiBBheisBnOpM1ik1WL8iLohUW8bgG63x5hUDl4JG0htjWBYFdTWmwIvL9o4y0oxU12rJYZMFX/psGzL4nE8doYBpivP82NewxUayj8Vhk0vkhdxtSM7aaaCKbqZOCfrkvkY9LFMuRumJKlO4XTWizbaFaeVjN+HBU8Cr2lRRFYw08BvgwiiOmq99wwfNwkaek9IW/uQ8WdW6usaraefWStgTLbH+YKv8Jn96sOyErI+S87FIhfxKSRYxLBy8slyNO3eGHu+30ZqEZjOdbHHcPxgUOHvzbliUCVtpGy1h7oSdA==
X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en;
SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAWPR08MB8982.eurprd08.prod.outlook.com;
PTR:; CAT:NONE;
SFS:(13230022)(4636009)(376002)(396003)(39860400002)(366004)(136003)(346002)(451199015)(86362001)(2906002)(5660300002)(8936002)(66446008)(41300700001)(66556008)(91956017)(33656002)(66476007)(52536014)(8676002)(66946007)(64756008)(76116006)(38100700002)(4326008)(316002)(55016003)(6916009)(6506007)(26005)(9686003)(71200400001)(186003)(7696005)(478600001)(122000001)(38070700005);
DIR:OUT; SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB10111
Original-Authentication-Results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
AM7EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
0fa48245-c503-4095-13b6-08daf4b5b931
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
VL/OJLyjT87c8pBqeLRE7YqRG/tBL9VPJh/DWIHAJI8ZjukQ+fgughXllfmQtLR+9DCZs/ya8lfHEOjIS0i+yHBAEe1V9A1HcW0uKWre71JaC8z7T1VETgIKwPFNHSc57d1BOexG9RicBMWdq/CEdnSTbZvns3y0UZz/KBIgMKOgh/tMuxnmvCfkOkBGXVehiYfnYIUtIsmWxsfnr6sVg2gp8YLLZPcG5mWr4qSsqwO2zA7bEnGLLy/yjgqBRcj75EjbBW4WSQJ2wIjUqS0h9TRa89AwUV2iQMrnKTRMl5cQBzvDFZKDCiZPLu5vS4FqR36vSM+RM5dgTto22HmtVFbvttfMP9CgBJXWfzbBAzYveTkal5JyVpOSkG6c7+aLbrHzbQ6IQn9pJu9U5UJaZ5Wgo2LrCylLhrfFhderbjU3xbUSMVMpSJylPRIvFHzTubwgNF5if4hMz7m7bTiU/3DRh9RQbtKwZ8e9JWOwmVW7mHN9nzi10XY+wNCUviEReJ9wdhJeSaNKz3AfHoBTvIhaeXEK++u80STKaAUelBfo4efQCrcEY0mOQzbEw8yDVfGIXDu5dT6zBIQtFXMUhkaSR37Cyla1XmLzd6PDIN7cKGOg9AoJjcSKTJL/pjblLkhtEEZHqs1VDPy66D6lci41dG8vsJr92You35ibe1bQIe/14xvT8A/3djnu951Z5aqaKUIm11OVB2RGocXlGA==
X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:;
IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com;
PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE;
SFS:(13230022)(4636009)(396003)(39860400002)(346002)(136003)(376002)(451199015)(40470700004)(36840700001)(46966006)(70586007)(70206006)(8676002)(6916009)(4326008)(52536014)(8936002)(316002)(478600001)(5660300002)(2906002)(6506007)(26005)(9686003)(186003)(336012)(47076005)(33656002)(55016003)(40480700001)(36860700001)(40460700003)(82310400005)(81166007)(356005)(86362001)(41300700001)(7696005)(82740400003);
DIR:OUT; SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jan 2023 15:57:45.5302 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id:
2a81bc9f-c497-4ebe-72a7-08daf4b5beec
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp:
TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123];
Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
AM7EUR03FT041.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6238
X-Spam-Status: No, score=-11.0 required=5.0 tests=BAYES_00, DKIM_SIGNED,
DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, RCVD_IN_DNSWL_NONE,
RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP,
UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
server2.sourceware.org
X-BeenThere: libc-alpha@sourceware.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Libc-alpha mailing list <libc-alpha.sourceware.org>
List-Unsubscribe: <https://sourceware.org/mailman/options/libc-alpha>,
<mailto:libc-alpha-request@sourceware.org?subject=unsubscribe>
List-Archive: <https://sourceware.org/pipermail/libc-alpha/>
List-Post: <mailto:libc-alpha@sourceware.org>
List-Help: <mailto:libc-alpha-request@sourceware.org?subject=help>
List-Subscribe: <https://sourceware.org/mailman/listinfo/libc-alpha>,
<mailto:libc-alpha-request@sourceware.org?subject=subscribe>
From: Wilco Dijkstra via Libc-alpha <libc-alpha@sourceware.org>
Reply-To: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org
Sender: "Libc-alpha"
<libc-alpha-bounces+patchwork=sourceware.org@sourceware.org>
|
| Series |
AArch64: Optimize memrchr
|
|
Checks
| Context | Check | Description |
|---|---|---|
| dj/TryBot-apply_patch | success | Patch applied to master at the time it was sent |
| dj/TryBot-32bit | success | Build for i686 |
Commit Message
Wilco Dijkstra
Jan. 12, 2023, 3:57 p.m. UTC
Optimize the main loop - large strings are 43% faster on modern CPUs. Passes regress. ---
Comments
The 01/12/2023 15:57, Wilco Dijkstra wrote: > Optimize the main loop - large strings are 43% faster on modern CPUs. > Passes regress. please commit it, thanks. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com> > > --- > > diff --git a/sysdeps/aarch64/memrchr.S b/sysdeps/aarch64/memrchr.S > index 9d2d29a396d46d6c2e74e3ca637091e2f3d68d5e..621fc65109736646b74900db8d15c6f8a7c68895 100644 > --- a/sysdeps/aarch64/memrchr.S > +++ b/sysdeps/aarch64/memrchr.S > @@ -26,7 +26,6 @@ > * MTE compatible. > */ > > -/* Arguments and results. */ > #define srcin x0 > #define chrin w1 > #define cntin x2 > @@ -77,31 +76,34 @@ ENTRY (__memrchr) > csel result, result, xzr, hi > ret > > + nop > L(start_loop): > - sub tmp, end, src > - subs cntrem, cntin, tmp > + subs cntrem, src, srcin > b.ls L(nomatch) > > /* Make sure that it won't overread by a 16-byte chunk */ > - add tmp, cntrem, 15 > - tbnz tmp, 4, L(loop32_2) > + sub cntrem, cntrem, 1 > + tbz cntrem, 4, L(loop32_2) > + add src, src, 16 > > - .p2align 4 > + .p2align 5 > L(loop32): > - ldr qdata, [src, -16]! > + ldr qdata, [src, -32]! > cmeq vhas_chr.16b, vdata.16b, vrepchr.16b > umaxp vend.16b, vhas_chr.16b, vhas_chr.16b /* 128->64 */ > fmov synd, dend > cbnz synd, L(end) > > L(loop32_2): > - ldr qdata, [src, -16]! > + ldr qdata, [src, -16] > subs cntrem, cntrem, 32 > cmeq vhas_chr.16b, vdata.16b, vrepchr.16b > - b.ls L(end) > + b.lo L(end_2) > umaxp vend.16b, vhas_chr.16b, vhas_chr.16b /* 128->64 */ > fmov synd, dend > cbz synd, L(loop32) > +L(end_2): > + sub src, src, 16 > L(end): > shrn vend.8b, vhas_chr.8h, 4 /* 128->64 */ > fmov synd, dend
diff --git a/sysdeps/aarch64/memrchr.S b/sysdeps/aarch64/memrchr.S index 9d2d29a396d46d6c2e74e3ca637091e2f3d68d5e..621fc65109736646b74900db8d15c6f8a7c68895 100644 --- a/sysdeps/aarch64/memrchr.S +++ b/sysdeps/aarch64/memrchr.S @@ -26,7 +26,6 @@ * MTE compatible. */ -/* Arguments and results. */ #define srcin x0 #define chrin w1 #define cntin x2 @@ -77,31 +76,34 @@ ENTRY (__memrchr) csel result, result, xzr, hi ret + nop L(start_loop): - sub tmp, end, src - subs cntrem, cntin, tmp + subs cntrem, src, srcin b.ls L(nomatch) /* Make sure that it won't overread by a 16-byte chunk */ - add tmp, cntrem, 15 - tbnz tmp, 4, L(loop32_2) + sub cntrem, cntrem, 1 + tbz cntrem, 4, L(loop32_2) + add src, src, 16 - .p2align 4 + .p2align 5 L(loop32): - ldr qdata, [src, -16]! + ldr qdata, [src, -32]! cmeq vhas_chr.16b, vdata.16b, vrepchr.16b umaxp vend.16b, vhas_chr.16b, vhas_chr.16b /* 128->64 */ fmov synd, dend cbnz synd, L(end) L(loop32_2): - ldr qdata, [src, -16]! + ldr qdata, [src, -16] subs cntrem, cntrem, 32 cmeq vhas_chr.16b, vdata.16b, vrepchr.16b - b.ls L(end) + b.lo L(end_2) umaxp vend.16b, vhas_chr.16b, vhas_chr.16b /* 128->64 */ fmov synd, dend cbz synd, L(loop32) +L(end_2): + sub src, src, 16 L(end): shrn vend.8b, vhas_chr.8h, 4 /* 128->64 */ fmov synd, dend