From patchwork Tue Jun 28 16:39:34 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 13441 Received: (qmail 34164 invoked by alias); 28 Jun 2016 16:43:13 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 33866 invoked by uid 89); 28 Jun 2016 16:43:10 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.8 required=5.0 tests=AWL, BAYES_00, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS autolearn=ham version=3.3.2 spammy=sk:perform, locals, strrchr X-HELO: na01-bn1-obe.outbound.protection.outlook.com Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Yuri.Norov@caviumnetworks.com; From: Yury Norov To: , CC: , , , , , , , , , , , , , , , , Yury Norov Subject: [PATCH 19/23] [AARCH64] delouse input arguments in system functions Date: Tue, 28 Jun 2016 19:39:34 +0300 Message-ID: <1467131978-669-20-git-send-email-ynorov@caviumnetworks.com> In-Reply-To: <1467131978-669-1-git-send-email-ynorov@caviumnetworks.com> References: <1467131978-669-1-git-send-email-ynorov@caviumnetworks.com> MIME-Version: 1.0 X-ClientProxiedBy: AM5PR0901CA0004.eurprd09.prod.outlook.com (10.164.186.142) To SN1PR07MB2256.namprd07.prod.outlook.com (10.164.47.150) X-MS-Office365-Filtering-Correlation-Id: da9c6ae9-f61c-43a3-bc43-08d39f7345fd X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2256; 2:mpCQeIgW586bqSCch2pRc4/E/5KBU1WngPDjP0VjZvSonNCJpv/ZF7hP9/ifnPAZilNsqqPGGiPQ3/5ygJJVRpKiA97vLk1vY1muvHMEhwkwDGpQ6A317Mdo45l/af7F8bxm0xQ3EgoXVhRMR/z0oSJruCSWqBILyOUYy2b987DPN+KXQYFTxwZFfVaNQ/CV; 3:qUx8PB+IdagvPrNdRXAmQBqyyyEZKPqUoJIurEcu3LlXDTvfQd8nHRs8L2K3DPzC6oyua80BStWCZl+Kou1mRuORn6jEV2VYUpsJoobxKhYUo1msJw0mzC42HQtdReS/ X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN1PR07MB2256; X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2256; 25:jFuAKt8x0XCvaE4DE4uT0n70x70/TNw6NJ9c5OCdYkl3qjFomNwDQXmUJCS+2SXWImmo92ZRFjv9zGLT9px+8u2XoUP9EJF/Y8hn3NydbuGwwfUV/WRf3nsXNX+9/H8WoE5+vm2F+DRBDBBi6Jne08sTqGzolFVwlU4Qd2FFHxxWKmn0TElMLG8cPFDDtkITuQdwNVDkX8DWfgd9KgoxyREKHXxVZSjk4mDPhNre3gkO23FFiAOaIiQC6bIURqgmoLlhq39VVSkCsm1gvIP4CHgjlhn5jRJFOPjFzHAFFibb3qMMEQiT8zgJUMC94Eam8Ow2wCwOIz3oh1cQblyJFOz4LPl+tERX8NJUHfCLv/Z612BO6zGGt0e+YIqQPtjg71osgtjBZMEd0eWB4DMlcLhxWEeIigPs+ThtIH/7yBbfVAe+B9cImLo5obuyDXiGc+XFJa4foil06Dn7gahaXR3Er6Un/GD0ryJlfvRBaFPm/NDdiEzyjmn2RxS46XCAIQazXae6v4hXrxHjKh1zoXsiXLnoxxPc2C8tcFylAcyXxvr5jg0oGSJzENG/pnT3phgf4l8Ca2Dwt1uhjLDk8g2BaV9PpAv0wV9z+DT/r2RBi3RW5TPjlxri7J1O1Z1j5GxQ68wDltz/yMLi4Q15sZF2D8ANyK6reYVVa42g3FOEi3KQrRPUAr1HWhY1HAlbX4W8XVbclfwWQaUu/u6IsY3dcR3kdz6h6oTpBBaxD0nBjpWAaU91v1WIaSen97spUMeTZcOfTAfugytsG4Q4kA== X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2256; 31:057cjiPemuC73O8u8tre5zX4QbR0yDLiktckk0oZ5XUbnuZ6z9DLmFE8SdAwprbmg2VCWlKU/Zl3Sl5uIBqB0OgvPOW9vYcSoW4N5YH4nQ9h06HXwjrIq2q3nueZlkuotJCrHlC4aNXqjuXW/FKAq+MdgeOdVBkMP35FiMqhM8oMChUpWGs+VFCIubSIFzRfz7KKovZeHvOvDbr2AWN6og==; 20:EgG/7xiTMRSTMvwsNUX8gVfE1k84bQ6NuvhvyMljPiPx0moW/dvf/X6OfL0TaK/HZVlDA9MeBPOIzDmtt8wrmnpObvjKXnKlNIY+iGkn2Bxs2ueW0ekZUXM42Vz39nxZd1QjwfAJ98GEz8MQ57Zf+68ZICgopovCJmatEcAlON0VQECXb2XVS1ASll/XqvQ82QYhV/HjoGle/oe6NRQTplycjr6frx11J1ezr44DK5zyNsRQqVmSsevhgodfqE6gOLZ7KDqNF5Pf/Ge3UZqtsBHHhc4f6CHgwA3bT2UfELw7umUpTagdnwi1vwcC2+G7YeX/Topk0y5oEK03Lg7dB0YrNKp/JW0tPd739wEEZedk6hBiKvRhRTtrihsz2yDsdoBcq0rBmMdhOv7LLG+qb2IJetiz6QTFWiqci7L7YKb+KXxsU1RLYpS3zig26lRf7gYr99Z2oerSm4VMYQD6C0W4kl1ByU1vTywwJGqNnu6QEwq5U4rO4Ra0dfJidgGfWroUOWXSfyVhgPlCiukTghP7lvJdWRjIkTYLQfquLPqJeswKbDGe63H+43PVH4Q556k8COqHN0FrSr6VtPUxdoLM7az8KTAs2lMub0KIdRQ= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001); SRVR:SN1PR07MB2256; BCL:0; PCL:0; RULEID:; SRVR:SN1PR07MB2256; X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2256; 4:xMSoijJVwCTdxAUi6nEuUoFKtFpb3woU+vRWeWULEgL5iDf5Qzz8uQzHvNklB5B6fHj6JHJaAvjghjpfeMGqA0wVDyqKixH1z0AMsX5e8H1hcJ+d0+5GRvI5ccpJzj24jLDzQujbLGj8jIFngMaMFyqcZ+eUDBgRhUik3XcmFqo2Wd2C9KmcAhH8ElSN9qtkumA7Vq9CLTjobNhO17VppmKPvA4Ef2ZKpyM9LzSflAwazvP6dQRfqpIbHiPOLGdGdx9z4sgmmh4Pqc55V4ssEdNFFgpUph8RGZh6ahvjtK0UYlvuLEp7OzpkjmWULi+UiT9A55JkM4eZCs5M4oEamFdoak/v4CqfcVLA5DFU8Y3Tyeu43gYHepJCZMLNB4S6 X-Forefront-PRVS: 0987ACA2E2 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(7916002)(199003)(189002)(47776003)(92566002)(5003940100001)(2950100001)(48376002)(3846002)(586003)(6116002)(36756003)(50466002)(50226002)(81156014)(66066001)(81166006)(189998001)(68736007)(77096005)(4001430100002)(8676002)(105586002)(76506005)(42186005)(19580405001)(106356001)(229853001)(5001770100001)(97736004)(101416001)(7846002)(107886002)(76176999)(575784001)(50986999)(19580395003)(4326007)(7736002)(2906002)(305945005)(33646002)(2101003); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR07MB2256; H:localhost; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: caviumnetworks.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN1PR07MB2256; 23:3BdeMdJLTOMdPGN2+wCaP0cbOQ1qnuWrjPE0knIq7?= =?us-ascii?Q?ioztpkjJ/Ui7sZY1iJi822QmBPVUSwsVQMcPwY6BYP22ZACUh0z1wPJ51f9a?= =?us-ascii?Q?NKDtXS11UBFknM5yfK+OLLs0fifhbP+YXgIoVPPXUFx2IV8vssbrCmWzPHBM?= =?us-ascii?Q?T7m8U8b+aELdgo8iyiL0sYZzbVybvFHGSYTsyGJXhSHEcZJcMe9cwGkhB6/X?= =?us-ascii?Q?ExAM5/0P8yCZH+mwMWrltpiCibhEBdqLtrtEcUFIms8vffm54fGDzWC78mEy?= =?us-ascii?Q?RH3FIaFnNxnik0fRVFXKqHpigF9UDybQuU780iEnmIy8ToiRjkEkXUmMjpMv?= =?us-ascii?Q?LLKuefy7jxmloX+GLgX7YUYFHx3UY60JRrsWa6qIOTUqpWCDqRzeBOGjb90J?= =?us-ascii?Q?4NFrUc0WQylXLdEcbxkKg9bTkd//NkYjUe3ni82QBQQRiANgUPg+X0//iO81?= =?us-ascii?Q?g5lRmWMgUP3beTxTaZ8bPrrGei8wJI2lCysmA5u3lgJacI1EHHvxki8PMkVS?= =?us-ascii?Q?cJlypVX53f17SIUp26XBH4J5IkHGHDL/5ayKYupmcQu1Bq1ImQ7svNhAlB66?= =?us-ascii?Q?FRRHuEGyrwT0HPt4FgD6vR2OPLAuTqtTczWmb5yadIsPEo3SfEBXOAfnBUWY?= =?us-ascii?Q?DmVXdiZoSHTE0nk+e1UiIzJmnKckHOU3W+G/NvGwUQ1oXrDl8M334jwRj1E+?= =?us-ascii?Q?jYdXTt5tRzCKYECYT7bOCySraC48hq2fFvOW+jY1NQaVUIob0dUyStBw0hpY?= =?us-ascii?Q?9pG9ntIf21jLFHzskhRboBFgSr0LmYwajx81fQXKFV58jUOPA3lAJ6lJPMlo?= =?us-ascii?Q?y/e8kbuj1TxD67ApkFsq2nFBBeHL0HMooYqVlUnEAVhcJo0GEd+XgG76aU0V?= =?us-ascii?Q?9Nxi3UqbsALDMSi4I794FwcgIZeQK3XvXR5NG8DuqBSGN8LHCXhvJ/H4R3ht?= =?us-ascii?Q?ZRDkVXRzHn/QZFjozNwBXsazatCt1x+aCZRKxp+YZKEm0F/11mNWIU15VqzV?= =?us-ascii?Q?kpbfL2IWiAt6H+ypB+Yu7UHIoMAJystIpQ2K/+D76tRcp1yUclkX5AJrwQk5?= =?us-ascii?Q?vOKZEkoMmRsmSF0kLVITXAv1lMVkLPp7fwnnr0xD9Z9GLJPWUnXhd5gQiiH0?= =?us-ascii?Q?/+TMPHzIfW00D4EGMoseVQGQ+sYVcmR4RzTaeiP2JmdnwgWjuO9a6iZEXa1R?= =?us-ascii?Q?c1/qMCWhyWE8ok=3D?= X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2256; 6:7N3XgMBD4npJy+uqt5MNaHrgeCJCb7WpJTFB4IhyXqjbQCAuyUYOj70M0WqOfsHDBTL/RkhAatH2M0H9jK+RB/qYK3fYrw3SiS1yuYu2iNA+3DsRnlxNewKExPANaDWp8Q0tVWe60Y1jCA4Hl8hMAvNcoMELHPN7BVE6dceopWmSV6Qp7UzPXTqI5OupQyV2eFibCtsaIfkUCbmtGCKRlUx8NKmMGAPCOCwWR+vulIJRUuYvLeJLaEUleCW7iYUHCZ1beqvoJoxFVCrmh2MG8rN6Occsx10CGV+EeWuKqkU=; 5:vgIIIoVBDMrgPKS8QAt2M2w0Oh8dKLqJZL7EhDlBzV2ysIo5ttsLLjqOXefw+BshYuBGzNXoD5DQ/KnkS14b7ALQ2SGZnko/JWTsauNQj+ahC2ynWDM6lMhxhZ1rkbikikd3XY5kmZDsco5mMyWVXw==; 24:oyxYBAMYDEuOSP7dXDj/yw0ofZxw2D/Ic8uNdtE+s9nhw5NfeLnUIilDVx4g112LfZCv2QDAGMlVvVzQ2zR/oqkTZTzGNP7UnbsUVM+pf00=; 7:gos+z95m27/0i4FjcEGB1k+KShdR1A903kH5VakBVaTxD68M55a0RHb2fNZHUKVcbd57vaecapIGwUJzRRC6P/34PnOTCo31EcbS89QPzKZHUGLF3QVHFH0RjskhO/MD7sILbHebkmooaMJnBFFtblLsnjFWPjXJTSSw8ZGR6L833ijlJlMf3XaxbdmZ+fZIoqHIoS7NsFpJ9+WxfYc8OjDgnz8U+FNKn6NutmyPNlmcCBNSXky/Ci+JZGifEjFG SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2016 16:43:03.0271 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR07MB2256 This patch introduces DELOUSE() macro which does nothing for lp64, and clears top bits of registers holding inputs, where needed. AARCH64/ILP32 needs it because top bits condition is undefined according to ABI. 'Delouse' term comes from Linux kernel where similar macro does the same for compat syscall wrapper. Signed-off-by: Yury Norov --- sysdeps/aarch64/__longjmp.S | 2 ++ sysdeps/aarch64/dl-tlsdesc.S | 8 ++++++++ sysdeps/aarch64/memcmp.S | 3 +++ sysdeps/aarch64/memcpy.S | 8 ++++++++ sysdeps/aarch64/memset.S | 3 +++ sysdeps/aarch64/setjmp.S | 1 + sysdeps/aarch64/strchr.S | 1 + sysdeps/aarch64/strchrnul.S | 1 + sysdeps/aarch64/strcmp.S | 2 ++ sysdeps/aarch64/strcpy.S | 2 ++ sysdeps/aarch64/strlen.S | 2 ++ sysdeps/aarch64/strncmp.S | 3 +++ sysdeps/aarch64/strnlen.S | 3 +++ sysdeps/aarch64/strrchr.S | 1 + sysdeps/aarch64/sysdep.h | 2 ++ sysdeps/unix/sysv/linux/aarch64/clone.S | 7 +++++++ sysdeps/unix/sysv/linux/aarch64/getcontext.S | 1 + sysdeps/unix/sysv/linux/aarch64/setcontext.S | 1 + sysdeps/unix/sysv/linux/aarch64/swapcontext.S | 1 + 19 files changed, 52 insertions(+) diff --git a/sysdeps/aarch64/__longjmp.S b/sysdeps/aarch64/__longjmp.S index 58332be..0377715 100644 --- a/sysdeps/aarch64/__longjmp.S +++ b/sysdeps/aarch64/__longjmp.S @@ -46,6 +46,8 @@ ENTRY (__longjmp) cfi_offset(d14, JB_D14<<3) cfi_offset(d15, JB_D15<<3) + DELOUSE(0) + ldp x19, x20, [x0, #JB_X19<<3] ldp x21, x22, [x0, #JB_X21<<3] ldp x23, x24, [x0, #JB_X23<<3] diff --git a/sysdeps/aarch64/dl-tlsdesc.S b/sysdeps/aarch64/dl-tlsdesc.S index 09cd158..24a110e 100644 --- a/sysdeps/aarch64/dl-tlsdesc.S +++ b/sysdeps/aarch64/dl-tlsdesc.S @@ -74,6 +74,7 @@ cfi_startproc .align 2 _dl_tlsdesc_return: + DELOUSE(0) ldr PTR_REG (0), [x0, #PTR_SIZE] RET cfi_endproc @@ -96,6 +97,7 @@ _dl_tlsdesc_return_lazy: td->entry) and thus it synchronizes with the release store to td->entry in _dl_tlsdesc_resolve_rela_fixup ensuring that the load from [x0,#PTR_SIZE] here happens after the initialization of td->arg. */ + DELOUSE(0) ldar PTR_REG (zr), [x0] ldr PTR_REG (0), [x0, #PTR_SIZE] RET @@ -125,6 +127,7 @@ _dl_tlsdesc_undefweak: td->entry) and thus it synchronizes with the release store to td->entry in _dl_tlsdesc_resolve_rela_fixup ensuring that the load from [x0,#8] here happens after the initialization of td->arg. */ + DELOUSE(0) ldar PTR_REG (zr), [x0] ldr PTR_REG (0), [x0, #PTR_SIZE] mrs x1, tpidr_el0 @@ -174,6 +177,7 @@ _dl_tlsdesc_dynamic: stp x29, x30, [sp,#-(32+16*NSAVEXREGPAIRS)]! cfi_adjust_cfa_offset (32+16*NSAVEXREGPAIRS) mov x29, sp + DELOUSE(0) /* Save just enough registers to support fast path, if we fall into slow path we will save additional registers. */ @@ -188,6 +192,7 @@ _dl_tlsdesc_dynamic: td->entry) and thus it synchronizes with the release store to td->entry in _dl_tlsdesc_resolve_rela_fixup ensuring that the load from [x0,#PTR_SIZE] here happens after the initialization of td->arg. */ + DELOUSE(0) ldar PTR_REG (zr), [x0] ldr PTR_REG (1), [x0,#PTR_SIZE] ldr PTR_REG (0), [x4] @@ -279,12 +284,14 @@ _dl_tlsdesc_resolve_rela: SAVE_Q_REGISTERS + DELOUSE(3) ldr PTR_REG (1), [x3, #PTR_SIZE] bl _dl_tlsdesc_resolve_rela_fixup RESTORE_Q_REGISTERS ldr x0, [sp, #32+16*8] + DELOUSE(0) ldr PTR_REG (1), [x0] blr x1 @@ -346,6 +353,7 @@ _dl_tlsdesc_resolve_hold: RESTORE_Q_REGISTERS ldr x0, [sp, #32+16*9] + DELOUSE(0) ldr PTR_REG (1), [x0] blr x1 diff --git a/sysdeps/aarch64/memcmp.S b/sysdeps/aarch64/memcmp.S index ae2d997..982aa02 100644 --- a/sysdeps/aarch64/memcmp.S +++ b/sysdeps/aarch64/memcmp.S @@ -47,6 +47,9 @@ #define mask x13 ENTRY_ALIGN (memcmp, 6) + DELOUSE(0) + DELOUSE(1) + DELOUSE(2) cbz limit, L(ret0) eor tmp1, src1, src2 tst tmp1, #7 diff --git a/sysdeps/aarch64/memcpy.S b/sysdeps/aarch64/memcpy.S index de73f0f..38a29d6 100644 --- a/sysdeps/aarch64/memcpy.S +++ b/sysdeps/aarch64/memcpy.S @@ -61,6 +61,10 @@ ENTRY_ALIGN (memmove, 6) + DELOUSE(0) + DELOUSE(1) + DELOUSE(2) + sub tmp1, dstin, src cmp count, 96 ccmp tmp1, count, 2, hi @@ -71,6 +75,10 @@ END (memmove) libc_hidden_builtin_def (memmove) ENTRY (memcpy) + DELOUSE(0) + DELOUSE(1) + DELOUSE(2) + prfm PLDL1KEEP, [src] add srcend, src, count add dstend, dstin, count diff --git a/sysdeps/aarch64/memset.S b/sysdeps/aarch64/memset.S index 4d222c5..1cad5c6 100644 --- a/sysdeps/aarch64/memset.S +++ b/sysdeps/aarch64/memset.S @@ -39,6 +39,9 @@ ENTRY_ALIGN (__memset, 6) + DELOUSE(0) + DELOUSE(2) + dup v0.16B, valw add dstend, dstin, count diff --git a/sysdeps/aarch64/setjmp.S b/sysdeps/aarch64/setjmp.S index da83f19..d608660 100644 --- a/sysdeps/aarch64/setjmp.S +++ b/sysdeps/aarch64/setjmp.S @@ -33,6 +33,7 @@ END (_setjmp) libc_hidden_def (_setjmp) ENTRY (__sigsetjmp) + DELOUSE(0) 1: stp x19, x20, [x0, #JB_X19<<3] diff --git a/sysdeps/aarch64/strchr.S b/sysdeps/aarch64/strchr.S index 5e3aecf..838384c 100644 --- a/sysdeps/aarch64/strchr.S +++ b/sysdeps/aarch64/strchr.S @@ -62,6 +62,7 @@ /* Locals and temporaries. */ ENTRY (strchr) + DELOUSE(0) mov wtmp2, #0x0401 movk wtmp2, #0x4010, lsl #16 dup vrepchr.16b, chrin diff --git a/sysdeps/aarch64/strchrnul.S b/sysdeps/aarch64/strchrnul.S index a624c8d..b60df26 100644 --- a/sysdeps/aarch64/strchrnul.S +++ b/sysdeps/aarch64/strchrnul.S @@ -60,6 +60,7 @@ identify exactly which byte is causing the termination. */ ENTRY (__strchrnul) + DELOUSE(0) /* Magic constant 0x40100401 to allow us to identify which lane matches the termination condition. */ mov wtmp2, #0x0401 diff --git a/sysdeps/aarch64/strcmp.S b/sysdeps/aarch64/strcmp.S index ba0ccb4..ccfe281 100644 --- a/sysdeps/aarch64/strcmp.S +++ b/sysdeps/aarch64/strcmp.S @@ -49,6 +49,8 @@ /* Start of performance-critical section -- one 64B cache line. */ ENTRY_ALIGN(strcmp, 6) + DELOUSE(0) + DELOUSE(1) eor tmp1, src1, src2 mov zeroones, #REP8_01 tst tmp1, #7 diff --git a/sysdeps/aarch64/strcpy.S b/sysdeps/aarch64/strcpy.S index 0694199..2a281b9 100644 --- a/sysdeps/aarch64/strcpy.S +++ b/sysdeps/aarch64/strcpy.S @@ -91,6 +91,8 @@ #define MIN_PAGE_SIZE (1 << MIN_PAGE_P2) ENTRY_ALIGN (STRCPY, 6) + DELOUSE(0) + DELOUSE(1) /* For moderately short strings, the fastest way to do the copy is to calculate the length of the string in the same way as strlen, then essentially do a memcpy of the result. This avoids the need for diff --git a/sysdeps/aarch64/strlen.S b/sysdeps/aarch64/strlen.S index a07834b..d1df0d1 100644 --- a/sysdeps/aarch64/strlen.S +++ b/sysdeps/aarch64/strlen.S @@ -85,6 +85,8 @@ boundary. */ ENTRY_ALIGN (__strlen, 6) + DELOUSE(0) + DELOUSE(1) and tmp1, srcin, MIN_PAGE_SIZE - 1 mov zeroones, REP8_01 cmp tmp1, MIN_PAGE_SIZE - 16 diff --git a/sysdeps/aarch64/strncmp.S b/sysdeps/aarch64/strncmp.S index f6a17fd..a372654 100644 --- a/sysdeps/aarch64/strncmp.S +++ b/sysdeps/aarch64/strncmp.S @@ -51,6 +51,9 @@ #define endloop x15 ENTRY_ALIGN_AND_PAD (strncmp, 6, 7) + DELOUSE(0) + DELOUSE(1) + DELOUSE(2) cbz limit, L(ret0) eor tmp1, src1, src2 mov zeroones, #REP8_01 diff --git a/sysdeps/aarch64/strnlen.S b/sysdeps/aarch64/strnlen.S index 4cce45f..6f67221 100644 --- a/sysdeps/aarch64/strnlen.S +++ b/sysdeps/aarch64/strnlen.S @@ -50,6 +50,9 @@ #define REP8_80 0x8080808080808080 ENTRY_ALIGN_AND_PAD (__strnlen, 6, 9) + DELOUSE(0) + DELOUSE(1) + DELOUSE(2) cbz limit, L(hit_limit) mov zeroones, #REP8_01 bic src, srcin, #15 diff --git a/sysdeps/aarch64/strrchr.S b/sysdeps/aarch64/strrchr.S index 44c1917..bb85a60 100644 --- a/sysdeps/aarch64/strrchr.S +++ b/sysdeps/aarch64/strrchr.S @@ -68,6 +68,7 @@ identify exactly which byte is causing the termination, and why. */ ENTRY(strrchr) + DELOUSE(0) cbz x1, L(null_search) /* Magic constant 0x40100401 to allow us to identify which lane matches the requested byte. Magic constant 0x80200802 used diff --git a/sysdeps/aarch64/sysdep.h b/sysdeps/aarch64/sysdep.h index 4ce0945..c5de4de 100644 --- a/sysdeps/aarch64/sysdep.h +++ b/sysdeps/aarch64/sysdep.h @@ -25,10 +25,12 @@ # define AARCH64_R(NAME) R_AARCH64_ ## NAME # define PTR_REG(n) x##n # define PTR_LOG_SIZE 3 +# define DELOUSE(n) #else # define AARCH64_R(NAME) R_AARCH64_P32_ ## NAME # define PTR_REG(n) w##n # define PTR_LOG_SIZE 2 +# define DELOUSE(n) mov w##n, w##n #endif #define PTR_SIZE (1<