From patchwork Mon Oct 31 15:45:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Engel X-Patchwork-Id: 59669 X-Patchwork-Delegate: rearnsha@gcc.gnu.org Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8D9B73953801 for ; Mon, 31 Oct 2022 15:48:53 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by sourceware.org (Postfix) with ESMTPS id BE8133860777 for ; Mon, 31 Oct 2022 15:46:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org BE8133860777 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=danielengel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=danielengel.com Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 8E0E13200094; Mon, 31 Oct 2022 11:46:55 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 31 Oct 2022 11:46:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=danielengel.com; h=cc:cc:content-transfer-encoding:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm1; t=1667231215; x= 1667317615; bh=NQHA6C/JVA87/+133/0AHHXVcYpD7PJcCLZehkbDAV0=; b=p jgZUOmMEfRXOLpN/SUKk7hQp1TdAAYvTdWouNIqk1YnXOzO1VN6Z2lmmyH3mwnhX EzPrKzqVovcStwXGgsXAhRYvTrp6upw26UftOiornQvyF42rP0wGmUcwqEwh2KmF Y6uygykVkt891icOz/8aXhgPrCFSX81H0+USSFzRCp6tq8155Zem4yLk8180dpK1 FDe7HyIgMYygcckPrGjX2J5kv+Svm/Kg0s7vo5NTMtCaPH+1lqlCfbdoGVmUdIAr 4N+03a22ExwEkOFtOgTr6Pk6RKbWImpTqLMESuEi9SFkptXnrZViuq6Myufssvwp 09HlKxy3DqY+3Wcnw3wAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1667231215; x=1667317615; bh=NQHA6C/JVA87/ +133/0AHHXVcYpD7PJcCLZehkbDAV0=; b=MxbToeHCrNtURPyLSqWHjQKz/2Bot J7woENAKiZ3fFxV3PFLOfbFsaknnPLBWoNMZypz/JODxYjKZ600avGJQp/UzLPag sCl5hmzOWvNytTlrN5vKeQ8iXhgFBVRRYXCy6gs+agJCqG4AFzmaWqZwe3yo5Dtl afwvM6l1AWzO79vYDBjuhkM20fbT9frT2Vk3sCDtGndBYjSxKvDLEROCVdi42ZJU hhP8XBJlSGzV9zYzFVIrRdOu+q1KBfyM4/xoaMVaTMRyEH1TMcIaXLAAzttH3u8D Y5PxbALXznOae2H8OnRFFpyz8BxRpch8r5sjHmQsftkwm1nrNRRQJFCHQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrudefgdektdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpeffrghnihgv lhcugfhnghgvlhcuoehgnhhusegurghnihgvlhgvnhhgvghlrdgtohhmqeenucggtffrrg htthgvrhhnpeetfeefvdelfffhteeiteehfeekuedtteekheevjedtjefhudehheejjedv udffveenucffohhmrghinheptghliidvrdhssgdpghhnuhdrohhrghenucevlhhushhtvg hrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgnhhusegurghnihgvlhgv nhhgvghlrdgtohhm X-ME-Proxy: Feedback-ID: i791144d6:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 31 Oct 2022 11:46:54 -0400 (EDT) Received: from ubuntu.lorien.danielengel.com (ubuntu.lorien.danielengel.com [10.0.0.96]) by sendmail.lorien.danielengel.com (8.15.2/8.15.2) with ESMTP id 29VFkkPu087256; Mon, 31 Oct 2022 08:46:46 -0700 (PDT) (envelope-from gnu@danielengel.com) From: Daniel Engel To: Richard Earnshaw , gcc-patches@gcc.gnu.org Subject: [PATCH v7 09/34] Import 'clz' functions from the CM0 library Date: Mon, 31 Oct 2022 08:45:04 -0700 Message-Id: <20221031154529.3627576-10-gnu@danielengel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221031154529.3627576-1-gnu@danielengel.com> References: <20221031154529.3627576-1-gnu@danielengel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_SHORT, RCVD_IN_DNSWL_LOW, SCC_5_SHORT_WORD_LINES, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Engel , Christophe Lyon Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" On architectures without __ARM_FEATURE_CLZ, this version combines __clzdi2() with __clzsi2() into a single object with an efficient tail call. Also, this version merges the formerly separate Thumb and ARM code implementations into a unified instruction sequence. This change significantly improves Thumb performance without affecting ARM performance. Finally, this version adds a new __OPTIMIZE_SIZE__ build option (binary search loop). There is no change to the code for architectures with __ARM_FEATURE_CLZ. gcc/libgcc/ChangeLog: 2022-10-09 Daniel Engel * config/arm/bits/clz2.S (__clzsi2, __clzdi2): Reduced code size on architectures without __ARM_FEATURE_CLZ. * config/arm/t-elf (LIB1ASMFUNCS): Moved _clzsi2 to new weak roup. --- libgcc/config/arm/clz2.S | 363 +++++++++++++++++++++++++-------------- libgcc/config/arm/t-elf | 7 +- 2 files changed, 237 insertions(+), 133 deletions(-) diff --git a/libgcc/config/arm/clz2.S b/libgcc/config/arm/clz2.S index 439341752ba..ed04698fef4 100644 --- a/libgcc/config/arm/clz2.S +++ b/libgcc/config/arm/clz2.S @@ -1,145 +1,244 @@ -/* Copyright (C) 1995-2022 Free Software Foundation, Inc. +/* clz2.S: Cortex M0 optimized 'clz' functions -This file is free software; you can redistribute it and/or modify it -under the terms of the GNU General Public License as published by the -Free Software Foundation; either version 3, or (at your option) any -later version. + Copyright (C) 2018-2022 Free Software Foundation, Inc. + Contributed by Daniel Engel (gnu@danielengel.com) -This file is distributed in the hope that it will be useful, but -WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -General Public License for more details. + This file is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by the + Free Software Foundation; either version 3, or (at your option) any + later version. -Under Section 7 of GPL version 3, you are granted additional -permissions described in the GCC Runtime Library Exception, version -3.1, as published by the Free Software Foundation. + This file is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. -You should have received a copy of the GNU General Public License and -a copy of the GCC Runtime Library Exception along with this program; -see the files COPYING3 and COPYING.RUNTIME respectively. If not, see -. */ + Under Section 7 of GPL version 3, you are granted additional + permissions described in the GCC Runtime Library Exception, version + 3.1, as published by the Free Software Foundation. + + You should have received a copy of the GNU General Public License and + a copy of the GCC Runtime Library Exception along with this program; + see the files COPYING3 and COPYING.RUNTIME respectively. If not, see + . */ + + +#if defined(__ARM_FEATURE_CLZ) && __ARM_FEATURE_CLZ + +#ifdef L_clzdi2 + +// int __clzdi2(long long) +// Counts leading zero bits in $r1:$r0. +// Returns the result in $r0. +FUNC_START_SECTION clzdi2 .text.sorted.libgcc.clz2.clzdi2 + CFI_START_FUNCTION + + // Moved here from lib1funcs.S + cmp xxh, #0 + do_it eq, et + clzeq r0, xxl + clzne r0, xxh + addeq r0, #32 + RET + + CFI_END_FUNCTION +FUNC_END clzdi2 + +#endif /* L_clzdi2 */ #ifdef L_clzsi2 -#ifdef NOT_ISA_TARGET_32BIT -FUNC_START clzsi2 - movs r1, #28 - movs r3, #1 - lsls r3, r3, #16 - cmp r0, r3 /* 0x10000 */ - bcc 2f - lsrs r0, r0, #16 - subs r1, r1, #16 -2: lsrs r3, r3, #8 - cmp r0, r3 /* #0x100 */ - bcc 2f - lsrs r0, r0, #8 - subs r1, r1, #8 -2: lsrs r3, r3, #4 - cmp r0, r3 /* #0x10 */ - bcc 2f - lsrs r0, r0, #4 - subs r1, r1, #4 -2: adr r2, 1f - ldrb r0, [r2, r0] - adds r0, r0, r1 - bx lr -.align 2 -1: -.byte 4, 3, 2, 2, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0 - FUNC_END clzsi2 -#else -ARM_FUNC_START clzsi2 -# if defined (__ARM_FEATURE_CLZ) - clz r0, r0 - RET -# else - mov r1, #28 - cmp r0, #0x10000 - do_it cs, t - movcs r0, r0, lsr #16 - subcs r1, r1, #16 - cmp r0, #0x100 - do_it cs, t - movcs r0, r0, lsr #8 - subcs r1, r1, #8 - cmp r0, #0x10 - do_it cs, t - movcs r0, r0, lsr #4 - subcs r1, r1, #4 - adr r2, 1f - ldrb r0, [r2, r0] - add r0, r0, r1 - RET -.align 2 -1: -.byte 4, 3, 2, 2, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0 -# endif /* !defined (__ARM_FEATURE_CLZ) */ - FUNC_END clzsi2 -#endif + +// int __clzsi2(int) +// Counts leading zero bits in $r0. +// Returns the result in $r0. +FUNC_START_SECTION clzsi2 .text.sorted.libgcc.clz2.clzsi2 + CFI_START_FUNCTION + + // Moved here from lib1funcs.S + clz r0, r0 + RET + + CFI_END_FUNCTION +FUNC_END clzsi2 + #endif /* L_clzsi2 */ +#else /* !__ARM_FEATURE_CLZ */ + #ifdef L_clzdi2 -#if !defined (__ARM_FEATURE_CLZ) - -# ifdef NOT_ISA_TARGET_32BIT -FUNC_START clzdi2 - push {r4, lr} - cmp xxh, #0 - bne 1f -# ifdef __ARMEB__ - movs r0, xxl - bl __clzsi2 - adds r0, r0, #32 - b 2f -1: - bl __clzsi2 -# else - bl __clzsi2 - adds r0, r0, #32 - b 2f -1: - movs r0, xxh - bl __clzsi2 -# endif -2: - pop {r4, pc} -# else /* NOT_ISA_TARGET_32BIT */ -ARM_FUNC_START clzdi2 - do_push {r4, lr} - cmp xxh, #0 - bne 1f -# ifdef __ARMEB__ - mov r0, xxl - bl __clzsi2 - add r0, r0, #32 - b 2f -1: - bl __clzsi2 -# else - bl __clzsi2 - add r0, r0, #32 - b 2f -1: - mov r0, xxh - bl __clzsi2 -# endif -2: - RETLDM r4 - FUNC_END clzdi2 -# endif /* NOT_ISA_TARGET_32BIT */ - -#else /* defined (__ARM_FEATURE_CLZ) */ - -ARM_FUNC_START clzdi2 - cmp xxh, #0 - do_it eq, et - clzeq r0, xxl - clzne r0, xxh - addeq r0, r0, #32 - RET - FUNC_END clzdi2 -#endif +// int __clzdi2(long long) +// Counts leading zero bits in $r1:$r0. +// Returns the result in $r0. +// Uses $r2 and possibly $r3 as scratch space. +FUNC_START_SECTION clzdi2 .text.sorted.libgcc.clz2.clzdi2 + CFI_START_FUNCTION + + #if defined(__ARMEB__) && __ARMEB__ + // Check if the upper word is zero. + cmp r0, #0 + + // The upper word is non-zero, so calculate __clzsi2(upper). + bne SYM(__clzsi2) + + // The upper word is zero, so calculate 32 + __clzsi2(lower). + movs r2, #64 + movs r0, r1 + b SYM(__internal_clzsi2) + + #else /* !__ARMEB__ */ + // Assume all the bits in the argument are zero. + movs r2, #64 + + // Check if the upper word is zero. + cmp r1, #0 + + // The upper word is zero, so calculate 32 + __clzsi2(lower). + beq SYM(__internal_clzsi2) + + // The upper word is non-zero, so set up __clzsi2(upper). + // Then fall through. + movs r0, r1 + + #endif /* !__ARMEB__ */ + #endif /* L_clzdi2 */ + +// The bitwise implementation of __clzdi2() tightly couples with __clzsi2(), +// such that instructions must appear consecutively in the same memory +// section for proper flow control. However, this construction inhibits +// the ability to discard __clzdi2() when only using __clzsi2(). +// Therefore, this block configures __clzsi2() for compilation twice. +// The first version is a minimal standalone implementation, and the second +// version is the continuation of __clzdi2(). The standalone version must +// be declared WEAK, so that the combined version can supersede it and +// provide both symbols when required. +// '_clzsi2' should appear before '_clzdi2' in LIB1ASMFUNCS. +#if defined(L_clzsi2) || defined(L_clzdi2) + +#ifdef L_clzsi2 +// int __clzsi2(int) +// Counts leading zero bits in $r0. +// Returns the result in $r0. +// Uses $r2 and possibly $r3 as scratch space. +WEAK_START_SECTION clzsi2 .text.sorted.libgcc.clz2.clzsi2 + CFI_START_FUNCTION + +#else /* L_clzdi2 */ +FUNC_ENTRY clzsi2 + +#endif + + // Assume all the bits in the argument are zero + movs r2, #32 + +#ifdef L_clzsi2 + WEAK_ENTRY internal_clzsi2 +#else /* L_clzdi2 */ + FUNC_ENTRY internal_clzsi2 +#endif + + // Size optimized: 22 bytes, 51 cycles + // Speed optimized: 50 bytes, 20 cycles + + #if defined(__OPTIMIZE_SIZE__) && __OPTIMIZE_SIZE__ + + // Binary search starts at half the word width. + movs r3, #16 + + LLSYM(__clz_loop): + // Test the upper 'n' bits of the operand for ZERO. + movs r1, r0 + lsrs r1, r3 + + // When the test fails, discard the lower bits of the register, + // and deduct the count of discarded bits from the result. + #ifdef __HAVE_FEATURE_IT + do_it ne, t + #else + beq LLSYM(__clz_skip) + #endif + + IT(mov,ne) r0, r1 + IT(sub,ne) r2, r3 + + LLSYM(__clz_skip): + // Decrease the shift distance for the next test. + lsrs r3, #1 + bne LLSYM(__clz_loop) + + #else /* __OPTIMIZE_SIZE__ */ + + // Unrolled binary search. + lsrs r1, r0, #16 + + #ifdef __HAVE_FEATURE_IT + do_it ne,t + #else + beq LLSYM(__clz8) + #endif + + // Out of 32 bits, the first '1' is somewhere in the highest 16, + // so the lower 16 bits are no longer interesting. + IT(mov,ne) r0, r1 + IT(sub,ne) r2, #16 + + LLSYM(__clz8): + lsrs r1, r0, #8 + + #ifdef __HAVE_FEATURE_IT + do_it ne,t + #else + beq LLSYM(__clz4) + #endif + + // Out of 16 bits, the first '1' is somewhere in the highest 8, + // so the lower 8 bits are no longer interesting. + IT(mov,ne) r0, r1 + IT(sub,ne) r2, #8 + + LLSYM(__clz4): + lsrs r1, r0, #4 + + #ifdef __HAVE_FEATURE_IT + do_it ne,t + #else + beq LLSYM(__clz2) + #endif + + // Out of 8 bits, the first '1' is somewhere in the highest 4, + // so the lower 4 bits are no longer interesting. + IT(mov,ne) r0, r1 + IT(sub,ne) r2, #4 + + LLSYM(__clz2): + // Load the remainder by index + adr r1, LLSYM(__clz_remainder) + ldrb r0, [r1, r0] + + #endif /* !__OPTIMIZE_SIZE__ */ + + // Account for the remainder. + subs r0, r2, r0 + RET + + #if !defined(__OPTIMIZE_SIZE__) || !__OPTIMIZE_SIZE__ + .align 2 + LLSYM(__clz_remainder): + .byte 0,1,2,2,3,3,3,3,4,4,4,4,4,4,4,4 + #endif + + CFI_END_FUNCTION +FUNC_END internal_clzsi2 +FUNC_END clzsi2 + +#ifdef L_clzdi2 +FUNC_END clzdi2 +#endif + +#endif /* L_clzsi2 || L_clzdi2 */ + +#endif /* !__ARM_FEATURE_CLZ */ + diff --git a/libgcc/config/arm/t-elf b/libgcc/config/arm/t-elf index 93ea1cd8f76..af779afa0a9 100644 --- a/libgcc/config/arm/t-elf +++ b/libgcc/config/arm/t-elf @@ -19,13 +19,18 @@ endif # !__symbian__ # assembly implementation here for double-precision values. +# Group 0: WEAK overridable function objects. +# See respective sources for rationale. +LIB1ASMFUNCS += \ + _clzsi2 \ + + # Group 1: Integer function objects. LIB1ASMFUNCS += \ _ashldi3 \ _ashrdi3 \ _lshrdi3 \ _clzdi2 \ - _clzsi2 \ _ctzsi2 \ _dvmd_tls \ _divsi3 \