From patchwork Thu Oct 5 16:10:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 77182 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A19283888C54 for ; Thu, 5 Oct 2023 16:12:08 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03on2056.outbound.protection.outlook.com [40.107.105.56]) by sourceware.org (Postfix) with ESMTPS id 9D638385773C for ; Thu, 5 Oct 2023 16:11:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9D638385773C Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yexMuPn3UGo/AEeGnyNbrt088CTCuk1RVuNixrhVMp0=; b=bOLsX6Jl4vzsNSduChflsZvmYDq3rLX3Qml4iKeYyc+q4ywhCMKSPXv92EsiF/1iD9Hsx9v4DUTlamOZ//pff2vzdCZWSP6UMpR/7hXzauZMxUOZIM6iImIF6/u+hJCFLqOuNnSR4AUADhso3ZZOSMZR4loCvvvBW5C1d9u3BJk= Received: from AS9PR06CA0549.eurprd06.prod.outlook.com (2603:10a6:20b:485::22) by AM0PR08MB5395.eurprd08.prod.outlook.com (2603:10a6:208:188::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.20; Thu, 5 Oct 2023 16:11:04 +0000 Received: from AM7EUR03FT008.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:485:cafe::d1) by AS9PR06CA0549.outlook.office365.com (2603:10a6:20b:485::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33 via Frontend Transport; Thu, 5 Oct 2023 16:11:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT008.mail.protection.outlook.com (100.127.141.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.28 via Frontend Transport; Thu, 5 Oct 2023 16:11:04 +0000 Received: ("Tessian outbound 6d14f3380669:v211"); Thu, 05 Oct 2023 16:11:04 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 30ec9364c250670a X-CR-MTA-TID: 64aa7808 Received: from e98dc09077db.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id B281D666-ADC4-48D2-8A4E-28CFDA9A7951.1; Thu, 05 Oct 2023 16:10:58 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e98dc09077db.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 05 Oct 2023 16:10:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f+sYcusRFHraHFNcreC9gjyBiwKUp1VTI3+w2pzYCOjNfOAQ/lTVONzMO/r5ysftI3J2kpmIoPZ6vzOkSHfm9LCOK1wEmyy9CKdmAbmcN3rbcrX+pFot/ALLZPBcKHq9kYbhg+vHFz/3HXOVEa9+4yQaZhznZERqrxjXVNiwji3Bunb8ODjQDBFTdY0NsnjJuIsjve3hmkHIeZVMh2wPAEp9RvhIIas7ByEqQTqKSroLmM6fLSrrBLrJIuTqjPHKHgqDKfRFN8qzSh5obVcpuneoSBp7Uyb7nqMFs/AoRHfUlqMq7cN77w0VTJAqFUNsUh4DeNxinqPe6JbhGwqTgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yexMuPn3UGo/AEeGnyNbrt088CTCuk1RVuNixrhVMp0=; b=FqrCdkiZKG884bQNvtpceQkKwsmaPrcOmx0C4c8//uJYKAaBHyn97QZOvhu+Okyczi7cp440w9yKWXxP7MfPy09F28541lc0uJBGboGn0YnnXXyTjE7AlDO/4JeTihdSzaPSqW6mabXUZJ3Mb8Nvowel+K6UtDM1eQ0uvVSO6XWcN00h1MfMthwQS0utpdvHEP6T30cjhCTuFoFC2HYh5tepeMzCqs3iC6ilqizb+UNky8D82zenLPM/j4kCm5Wc1Gz2cQpIAxq2c9wBbXECIYjSE/Kvq4Pn0zr74EwU1eOsENnVw4U/6F6ZrD90IReL7V3kU4aX4UTKKvJW5unCcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yexMuPn3UGo/AEeGnyNbrt088CTCuk1RVuNixrhVMp0=; b=bOLsX6Jl4vzsNSduChflsZvmYDq3rLX3Qml4iKeYyc+q4ywhCMKSPXv92EsiF/1iD9Hsx9v4DUTlamOZ//pff2vzdCZWSP6UMpR/7hXzauZMxUOZIM6iImIF6/u+hJCFLqOuNnSR4AUADhso3ZZOSMZR4loCvvvBW5C1d9u3BJk= Received: from AM5PR1001CA0072.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:15::49) by DB4PR08MB7933.eurprd08.prod.outlook.com (2603:10a6:10:37b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.26; Thu, 5 Oct 2023 16:10:54 +0000 Received: from AM7EUR03FT004.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:15:cafe::c1) by AM5PR1001CA0072.outlook.office365.com (2603:10a6:206:15::49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT004.mail.protection.outlook.com (100.127.140.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6863.29 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 16:10:53 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 5 Oct 2023 16:10:53 +0000 From: Joe Ramsay To: CC: Joe Ramsay Subject: [PATCH v2 1/5] aarch64: Add vector implementations of tan routines Date: Thu, 5 Oct 2023 17:10:48 +0100 Message-ID: <20231005161052.11878-1-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT004:EE_|DB4PR08MB7933:EE_|AM7EUR03FT008:EE_|AM0PR08MB5395:EE_ X-MS-Office365-Filtering-Correlation-Id: 1c0ebe0d-f7ff-4655-a038-08dbc5bdad1f x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: yo5i/m8DHFj6pyK78vHc0Gi5gVgLfmzBflX8yaORJ233Ro4kJJXgMcNBtw9u6zX3NLvcU+7YLOpLz4dIrYpJDpkVrp3J/R/e1EdZDO19Q39V18MT+oegXOmPsd3A20lH4hlvGgTnj8zyOQtaiFEJ4jEKUA8nk5ldhIcrdvOBd/14dDDsiRCpyVERwmKs4BpIqqUWeu/NEswOTxoacjVtR+hs4Bp2J7Y66QCRcgkvQ+yvqnAQjp/OpUwpEaiB6176NX358q4ftKc8uIU9w8s8570Orjk5fZ8ZBxHQ7KC6f/GiL2zli5Jhu6mPWQWQjepzOS2NRzVIJ47aWTo83W8GBVly2uHjIxkfrDuDGQN6yNCG2R++jQyKsff4CjTP5kfng0n19d/cLwEq3x8phLOSnNMG/YX7eqZmKR7pRmUU5CKKuf8ZiSi1wJ09Ssq0A+Ck36zZzRhCHTse3/+UxcMkuOiRWtlGOeWDSLJc+0oaFjAmVVcgdfeD/YOMRGCPD+CahMk1uaGFK/ibjRlOJToQlVuQ4chwrAFpWX46upkhSIMdjFd05pLn84ATULDnsmWClWrCFpmtg4kL9kV3hOkFODsE4POxj3QfUtj3YCjLjTEDXRu2jZKZ/VoCx/CziEkskASk8iFYrCuB9Q9j0zIGk2PZjvo7PQ08PWx+AGLyuTdrCl8ErFrMbl3+uJHsZF9+nDDZ2dKTsr9Kngobb0SvPanu0vSoPFHy4Ca6TglJ208WLkk521v36ygVJ23ZCQmYSBZrlypoIvUcIl4pTJBTew== X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(396003)(136003)(39850400004)(376002)(346002)(230922051799003)(82310400011)(64100799003)(1800799009)(186009)(451199024)(46966006)(36840700001)(41300700001)(70206006)(316002)(8676002)(8936002)(6916009)(4326008)(6666004)(30864003)(5660300002)(2906002)(70586007)(426003)(478600001)(40480700001)(83380400001)(47076005)(86362001)(82740400003)(36860700001)(336012)(36756003)(2616005)(1076003)(26005)(7696005)(81166007)(356005)(2004002)(36900700001)(559001)(579004)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB7933 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT008.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 8df2ab04-2f2c-4d9f-2c3f-08dbc5bda6fd X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qPw2tT9bLMZgglW3QrW/fPbD19VpYSjUa42JQK1YrLE840tZAymYSNGpdKZmTCYdvcN4lBDr6R95W37nL8CNUujjPfwD7WhKvhDkAZsVIQDZop/70cECZHJpcNxhF5Wrgd7/KVoLGWsMSDWtEidiiz0QxbKx2nAd6eALptGiEJvaqyM4KFery09sfVV8FOT9m7VdDHk4twmVzQ6SbjEHsa92p0oPwZJRCairw9BOKs/iJEiSAgnb9R9xbGXIR23JEQIiD0WSkEvJ71zB+RM1vIyqHOmWh1d2yogW+ovjmwIi7IV3XeGsYD4T6Z+IhV79Aigi5piiclRDReVlZOEq9XuyKXW+cGMUApHs0jzRP4WABIFBQ+fp3WPVaNkbRZM47z72d6+u7vytfaebDp5pO5mgGdewmM8C2RR09kY+qAt9yoX1XEoDPTSWTIC6vdpj2RfdE8+3zQkgya/slYAB5qGpgoFPC4K7YiGgnzze/5N43jH1ucPeH443qODCJh9rYWAZVT4EY8mu0O60vnHlK5oOwuIz5LP0SrvWQ9ldobvb82I/OOf+rJ470R1BGJt8yC0ci++zWfvOFj0lMvwdDp+JtTzU4pTJMZALfo3w39wOdimXvcDd9RuEdSdn7X3SQE3PJRhejmBAHKD09cLBH8Q6OFCPBYV3GdNdGItC2baKnvaWM1vJ653ECOaBGAJR0b5cq565jalbK3nS7ORPKhWjY7oUunZ1kdgWiWuMsfmjC7TxdPFvU5ObAr5vxr75C73881BRTEjI7/O6cEAp+Q== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(136003)(396003)(376002)(346002)(230922051799003)(1800799009)(64100799003)(451199024)(82310400011)(186009)(36840700001)(46966006)(40470700004)(426003)(40460700003)(7696005)(6666004)(478600001)(5660300002)(6916009)(36860700001)(82740400003)(336012)(86362001)(81166007)(2616005)(41300700001)(40480700001)(30864003)(2906002)(83380400001)(26005)(47076005)(1076003)(70206006)(316002)(8936002)(36756003)(70586007)(8676002)(4326008)(2004002)(579004)(559001)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 16:11:04.6297 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1c0ebe0d-f7ff-4655-a038-08dbc5bdad1f X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT008.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5395 X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org This includes some utility headers for evaluating polynomials using various schemes. --- Changes from v1: * Silence sign-of-zero check for mathvec tan * Optimise register usage around special-case, and remove -0 check Thanks, Joe math/auto-libm-test-in | 2 +- math/auto-libm-test-out-tan | 50 +-- sysdeps/aarch64/fpu/Makefile | 3 +- sysdeps/aarch64/fpu/Versions | 6 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/poly_advsimd_f32.h | 36 ++ sysdeps/aarch64/fpu/poly_advsimd_f64.h | 36 ++ sysdeps/aarch64/fpu/poly_generic.h | 285 ++++++++++++++++ sysdeps/aarch64/fpu/poly_sve_f32.h | 38 +++ sysdeps/aarch64/fpu/poly_sve_f64.h | 38 +++ sysdeps/aarch64/fpu/poly_sve_generic.h | 313 ++++++++++++++++++ sysdeps/aarch64/fpu/tan_advsimd.c | 123 +++++++ sysdeps/aarch64/fpu/tan_sve.c | 104 ++++++ sysdeps/aarch64/fpu/tanf_advsimd.c | 129 ++++++++ sysdeps/aarch64/fpu/tanf_sve.c | 118 +++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 21 files changed, 1274 insertions(+), 27 deletions(-) create mode 100644 sysdeps/aarch64/fpu/poly_advsimd_f32.h create mode 100644 sysdeps/aarch64/fpu/poly_advsimd_f64.h create mode 100644 sysdeps/aarch64/fpu/poly_generic.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_f32.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_f64.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_generic.h create mode 100644 sysdeps/aarch64/fpu/tan_advsimd.c create mode 100644 sysdeps/aarch64/fpu/tan_sve.c create mode 100644 sysdeps/aarch64/fpu/tanf_advsimd.c create mode 100644 sysdeps/aarch64/fpu/tanf_sve.c diff --git a/math/auto-libm-test-in b/math/auto-libm-test-in index 2672eb1f6a..70892503d6 100644 --- a/math/auto-libm-test-in +++ b/math/auto-libm-test-in @@ -7655,7 +7655,7 @@ sqrt min sqrt min_subnorm tan 0 -tan -0 +tan -0 no-mathvec tan pi/4 tan pi/2 tan -pi/2 diff --git a/math/auto-libm-test-out-tan b/math/auto-libm-test-out-tan index 7d00d03e1d..f46fdc7ec6 100644 --- a/math/auto-libm-test-out-tan +++ b/math/auto-libm-test-out-tan @@ -23,31 +23,31 @@ tan 0 = tan tonearest ibm128 0x0p+0 : 0x0p+0 : inexact-ok = tan towardzero ibm128 0x0p+0 : 0x0p+0 : inexact-ok = tan upward ibm128 0x0p+0 : 0x0p+0 : inexact-ok -tan -0 -= tan downward binary32 -0x0p+0 : -0x0p+0 : inexact-ok -= tan tonearest binary32 -0x0p+0 : -0x0p+0 : inexact-ok -= tan towardzero binary32 -0x0p+0 : -0x0p+0 : inexact-ok -= tan upward binary32 -0x0p+0 : -0x0p+0 : inexact-ok -= tan downward binary64 -0x0p+0 : -0x0p+0 : inexact-ok -= tan tonearest binary64 -0x0p+0 : -0x0p+0 : inexact-ok -= tan towardzero binary64 -0x0p+0 : -0x0p+0 : inexact-ok -= tan upward binary64 -0x0p+0 : -0x0p+0 : inexact-ok -= tan downward intel96 -0x0p+0 : -0x0p+0 : inexact-ok -= tan tonearest intel96 -0x0p+0 : -0x0p+0 : inexact-ok -= tan towardzero intel96 -0x0p+0 : -0x0p+0 : inexact-ok -= tan upward intel96 -0x0p+0 : -0x0p+0 : inexact-ok -= tan downward m68k96 -0x0p+0 : -0x0p+0 : inexact-ok -= tan tonearest m68k96 -0x0p+0 : -0x0p+0 : inexact-ok -= tan towardzero m68k96 -0x0p+0 : -0x0p+0 : inexact-ok -= tan upward m68k96 -0x0p+0 : -0x0p+0 : inexact-ok -= tan downward binary128 -0x0p+0 : -0x0p+0 : inexact-ok -= tan tonearest binary128 -0x0p+0 : -0x0p+0 : inexact-ok -= tan towardzero binary128 -0x0p+0 : -0x0p+0 : inexact-ok -= tan upward binary128 -0x0p+0 : -0x0p+0 : inexact-ok -= tan downward ibm128 -0x0p+0 : -0x0p+0 : inexact-ok -= tan tonearest ibm128 -0x0p+0 : -0x0p+0 : inexact-ok -= tan towardzero ibm128 -0x0p+0 : -0x0p+0 : inexact-ok -= tan upward ibm128 -0x0p+0 : -0x0p+0 : inexact-ok +tan -0 no-mathvec += tan downward binary32 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan tonearest binary32 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan towardzero binary32 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan upward binary32 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan downward binary64 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan tonearest binary64 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan towardzero binary64 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan upward binary64 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan downward intel96 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan tonearest intel96 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan towardzero intel96 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan upward intel96 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan downward m68k96 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan tonearest m68k96 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan towardzero m68k96 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan upward m68k96 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan downward binary128 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan tonearest binary128 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan towardzero binary128 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan upward binary128 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan downward ibm128 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan tonearest ibm128 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan towardzero ibm128 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok += tan upward ibm128 -0x0p+0 : -0x0p+0 : no-mathvec inexact-ok tan pi/4 = tan downward binary32 0xc.90fdbp-4 : 0x1p+0 : inexact-ok = tan tonearest binary32 0xc.90fdbp-4 : 0x1p+0 : inexact-ok diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 04aa2e37ca..a1bbc9bcaa 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,7 +1,8 @@ libmvec-supported-funcs = cos \ exp \ log \ - sin + sin \ + tan float-advsimd-funcs = $(libmvec-supported-funcs) double-advsimd-funcs = $(libmvec-supported-funcs) diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index c85c0f3efb..f0ca0940a9 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -17,4 +17,10 @@ libmvec { _ZGVsMxv_sin; _ZGVsMxv_sinf; } + GLIBC_2.39 { + _ZGVnN4v_tanf; + _ZGVnN2v_tan; + _ZGVsMxv_tanf; + _ZGVsMxv_tan; + } } diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 7c200599c1..6193213147 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,11 +53,13 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); # undef __ADVSIMD_VEC_MATH_SUPPORTED #endif /* __ADVSIMD_VEC_MATH_SUPPORTED */ @@ -68,11 +70,13 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); # undef __SVE_VEC_MATH_SUPPORTED #endif /* __SVE_VEC_MATH_SUPPORTED */ diff --git a/sysdeps/aarch64/fpu/poly_advsimd_f32.h b/sysdeps/aarch64/fpu/poly_advsimd_f32.h new file mode 100644 index 0000000000..9e2ad9ad94 --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_advsimd_f32.h @@ -0,0 +1,36 @@ +/* Helpers for evaluating polynomials on single-precision AdvSIMD input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_ADVSIMD_F32_H +#define AARCH64_FPU_POLY_ADVSIMD_F32_H + +#include + +/* Wrap AdvSIMD f32 helpers: evaluation of some scheme/order has form: + v_[scheme]_[order]_f32. */ +#define VTYPE float32x4_t +#define FMA(x, y, z) vfmaq_f32 (z, x, y) +#define VWRAP(f) v_##f##_f32 +#include "poly_generic.h" +#undef VWRAP +#undef FMA +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_advsimd_f64.h b/sysdeps/aarch64/fpu/poly_advsimd_f64.h new file mode 100644 index 0000000000..955cfc08ce --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_advsimd_f64.h @@ -0,0 +1,36 @@ +/* Helpers for evaluating polynomials on double-precision AdvSIMD input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_ADVSIMD_F64_H +#define AARCH64_FPU_POLY_ADVSIMD_F64_H + +#include + +/* Wrap AdvSIMD f64 helpers: evaluation of some scheme/order has form: + v_[scheme]_[order]_f64. */ +#define VTYPE float64x2_t +#define FMA(x, y, z) vfmaq_f64 (z, x, y) +#define VWRAP(f) v_##f##_f64 +#include "poly_generic.h" +#undef VWRAP +#undef FMA +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_generic.h b/sysdeps/aarch64/fpu/poly_generic.h new file mode 100644 index 0000000000..84f042182b --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_generic.h @@ -0,0 +1,285 @@ +/* Generic helpers for evaluating polynomials with various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + + +#ifndef VTYPE +# error Cannot use poly_generic without defining VTYPE +#endif +#ifndef VWRAP +# error Cannot use poly_generic without defining VWRAP +#endif +#ifndef FMA +# error Cannot use poly_generic without defining FMA +#endif + +static inline VTYPE VWRAP (pairwise_poly_3) (VTYPE x, VTYPE x2, + const VTYPE *poly) +{ + /* At order 3, Estrin and Pairwise Horner are identical. */ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + return FMA (p23, x2, p01); +} + +static inline VTYPE VWRAP (estrin_4) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + return FMA (poly[4], x4, p03); +} +static inline VTYPE VWRAP (estrin_5) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p45 = FMA (poly[5], x, poly[4]); + return FMA (p45, x4, p03); +} +static inline VTYPE VWRAP (estrin_6) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p45 = FMA (poly[5], x, poly[4]); + VTYPE p46 = FMA (poly[6], x2, p45); + return FMA (p46, x4, p03); +} +static inline VTYPE VWRAP (estrin_7) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p47 = VWRAP (pairwise_poly_3) (x, x2, poly + 4); + return FMA (p47, x4, p03); +} +static inline VTYPE VWRAP (estrin_8) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (poly[8], x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_9) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p89 = FMA (poly[9], x, poly[8]); + return FMA (p89, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_10) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p89 = FMA (poly[9], x, poly[8]); + VTYPE p8_10 = FMA (poly[10], x2, p89); + return FMA (p8_10, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_11) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p8_11 = VWRAP (pairwise_poly_3) (x, x2, poly + 8); + return FMA (p8_11, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_12) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_4) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_13) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_5) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_14) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_6) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_15) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_7) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_16) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + return FMA (poly[16], x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_17) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_17 = FMA (poly[17], x, poly[16]); + return FMA (p16_17, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_18) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_17 = FMA (poly[17], x, poly[16]); + VTYPE p16_18 = FMA (poly[18], x2, p16_17); + return FMA (p16_18, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_19) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_19 = VWRAP (pairwise_poly_3) (x, x2, poly + 16); + return FMA (p16_19, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} + +static inline VTYPE VWRAP (horner_3) (VTYPE x, const VTYPE *poly) +{ + VTYPE p = FMA (poly[3], x, poly[2]); + p = FMA (x, p, poly[1]); + p = FMA (x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_4) (VTYPE x, const VTYPE *poly) +{ + VTYPE p = FMA (poly[4], x, poly[3]); + p = FMA (x, p, poly[2]); + p = FMA (x, p, poly[1]); + p = FMA (x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_5) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_4) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_6) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_5) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_7) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_6) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_8) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_7) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_9) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_8) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_10) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_9) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_11) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_10) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_12) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_11) (x, poly + 1), poly[0]); +} + +static inline VTYPE VWRAP (pw_horner_4) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + VTYPE p; + p = FMA (x2, poly[4], p23); + p = FMA (x2, p, p01); + return p; +} +static inline VTYPE VWRAP (pw_horner_5) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + VTYPE p45 = FMA (poly[5], x, poly[4]); + VTYPE p; + p = FMA (x2, p45, p23); + p = FMA (x2, p, p01); + return p; +} +static inline VTYPE VWRAP (pw_horner_6) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p26 = VWRAP (pw_horner_4) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p26, p01); +} +static inline VTYPE VWRAP (pw_horner_7) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p27 = VWRAP (pw_horner_5) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p27, p01); +} +static inline VTYPE VWRAP (pw_horner_8) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p28 = VWRAP (pw_horner_6) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p28, p01); +} +static inline VTYPE VWRAP (pw_horner_9) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p29 = VWRAP (pw_horner_7) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p29, p01); +} +static inline VTYPE VWRAP (pw_horner_10) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_10 = VWRAP (pw_horner_8) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_10, p01); +} +static inline VTYPE VWRAP (pw_horner_11) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_11 = VWRAP (pw_horner_9) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_11, p01); +} +static inline VTYPE VWRAP (pw_horner_12) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_12 = VWRAP (pw_horner_10) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_12, p01); +} +static inline VTYPE VWRAP (pw_horner_13) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_13 = VWRAP (pw_horner_11) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_13, p01); +} +static inline VTYPE VWRAP (pw_horner_14) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_14 = VWRAP (pw_horner_12) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_14, p01); +} +static inline VTYPE VWRAP (pw_horner_15) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_15 = VWRAP (pw_horner_13) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_15, p01); +} +static inline VTYPE VWRAP (pw_horner_16) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_16 = VWRAP (pw_horner_14) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_16, p01); +} +static inline VTYPE VWRAP (pw_horner_17) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_17 = VWRAP (pw_horner_15) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_17, p01); +} +static inline VTYPE VWRAP (pw_horner_18) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_18 = VWRAP (pw_horner_16) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_18, p01); +} diff --git a/sysdeps/aarch64/fpu/poly_sve_f32.h b/sysdeps/aarch64/fpu/poly_sve_f32.h new file mode 100644 index 0000000000..dcf2fab8dd --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_f32.h @@ -0,0 +1,38 @@ +/* Helpers for evaluating polynomials on single-precision SVE input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_SVE_F32_H +#define AARCH64_FPU_POLY_SVE_F32_H + +#include + +/* Wrap SVE f32 helpers: evaluation of some scheme/order has form: + sv_[scheme]_[order]_f32_x. */ +#define VTYPE svfloat32_t +#define STYPE float +#define VWRAP(f) sv_##f##_f32_x +#define DUP svdup_n_f32 +#include "poly_sve_generic.h" +#undef DUP +#undef VWRAP +#undef STYPE +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_sve_f64.h b/sysdeps/aarch64/fpu/poly_sve_f64.h new file mode 100644 index 0000000000..97a0b76637 --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_f64.h @@ -0,0 +1,38 @@ +/* Helpers for evaluating polynomials on double-precision SVE input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_SVE_F64_H +#define AARCH64_FPU_POLY_SVE_F64_H + +#include + +/* Wrap SVE f64 helpers: evaluation of some scheme/order has form: + sv_[scheme]_[order]_f64_x. */ +#define VTYPE svfloat64_t +#define STYPE double +#define VWRAP(f) sv_##f##_f64_x +#define DUP svdup_n_f64 +#include "poly_sve_generic.h" +#undef DUP +#undef VWRAP +#undef STYPE +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_sve_generic.h b/sysdeps/aarch64/fpu/poly_sve_generic.h new file mode 100644 index 0000000000..0ecf5ce45b --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_generic.h @@ -0,0 +1,313 @@ +/* Helpers for evaluating polynomials with various schemes - specific to SVE + but precision-agnostic. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef VTYPE +# error Cannot use poly_generic without defining VTYPE +#endif +#ifndef STYPE +# error Cannot use poly_generic without defining STYPE +#endif +#ifndef VWRAP +# error Cannot use poly_generic without defining VWRAP +#endif +#ifndef DUP +# error Cannot use poly_generic without defining DUP +#endif + +static inline VTYPE VWRAP (pairwise_poly_3) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + /* At order 3, Estrin and Pairwise Horner are identical. */ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + return svmla_x (pg, p01, p23, x2); +} + +static inline VTYPE VWRAP (estrin_4) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + return svmla_x (pg, p03, x4, poly[4]); +} +static inline VTYPE VWRAP (estrin_5) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + return svmla_x (pg, p03, p45, x4); +} +static inline VTYPE VWRAP (estrin_6) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + VTYPE p46 = svmla_x (pg, p45, x, poly[6]); + return svmla_x (pg, p03, p46, x4); +} +static inline VTYPE VWRAP (estrin_7) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p47 = VWRAP (pairwise_poly_3) (pg, x, x2, poly + 4); + return svmla_x (pg, p03, p47, x4); +} +static inline VTYPE VWRAP (estrin_8) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), x8, poly[8]); +} +static inline VTYPE VWRAP (estrin_9) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + VTYPE x8, const STYPE *poly) +{ + VTYPE p89 = svmla_x (pg, DUP (poly[8]), x, poly[9]); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p89, x8); +} +static inline VTYPE VWRAP (estrin_10) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + VTYPE p89 = svmla_x (pg, DUP (poly[8]), x, poly[9]); + VTYPE p8_10 = svmla_x (pg, p89, x2, poly[10]); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p8_10, x8); +} +static inline VTYPE VWRAP (estrin_11) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + VTYPE p8_11 = VWRAP (pairwise_poly_3) (pg, x, x2, poly + 8); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p8_11, x8); +} +static inline VTYPE VWRAP (estrin_12) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_4) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_13) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_5) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_14) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_6) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_15) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_7) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_16) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), x16, + poly[16]); +} +static inline VTYPE VWRAP (estrin_17) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + VTYPE p16_17 = svmla_x (pg, DUP (poly[16]), x, poly[17]); + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), p16_17, + x16); +} +static inline VTYPE VWRAP (estrin_18) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + VTYPE p16_17 = svmla_x (pg, DUP (poly[16]), x, poly[17]); + VTYPE p16_18 = svmla_x (pg, p16_17, x2, poly[18]); + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), p16_18, + x16); +} +static inline VTYPE VWRAP (estrin_19) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), + VWRAP (pairwise_poly_3) (pg, x, x2, poly + 16), x16); +} + +static inline VTYPE VWRAP (horner_3) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + VTYPE p = svmla_x (pg, DUP (poly[2]), x, poly[3]); + p = svmad_x (pg, x, p, poly[1]); + p = svmad_x (pg, x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_4) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + VTYPE p = svmla_x (pg, DUP (poly[3]), x, poly[4]); + p = svmad_x (pg, x, p, poly[2]); + p = svmad_x (pg, x, p, poly[1]); + p = svmad_x (pg, x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_5) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_4) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_6) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_5) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_7) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_6) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_8) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_7) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_9) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_8) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_10_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_9) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_11_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, sv_horner_10_f32_x (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_12_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, sv_horner_11_f32_x (pg, x, poly + 1), poly[0]); +} + +static inline VTYPE VWRAP (pw_horner_4) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + VTYPE p; + p = svmla_x (pg, p23, x2, poly[4]); + p = svmla_x (pg, p01, x2, p); + return p; +} +static inline VTYPE VWRAP (pw_horner_5) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + VTYPE p; + p = svmla_x (pg, p23, x2, p45); + p = svmla_x (pg, p01, x2, p); + return p; +} +static inline VTYPE VWRAP (pw_horner_6) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p26 = VWRAP (pw_horner_4) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p26); +} +static inline VTYPE VWRAP (pw_horner_7) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p27 = VWRAP (pw_horner_5) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p27); +} +static inline VTYPE VWRAP (pw_horner_8) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p28 = VWRAP (pw_horner_6) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p28); +} +static inline VTYPE VWRAP (pw_horner_9) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p29 = VWRAP (pw_horner_7) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p29); +} +static inline VTYPE VWRAP (pw_horner_10) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_10 = VWRAP (pw_horner_8) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_10); +} +static inline VTYPE VWRAP (pw_horner_11) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_11 = VWRAP (pw_horner_9) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_11); +} +static inline VTYPE VWRAP (pw_horner_12) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_12 = VWRAP (pw_horner_10) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_12); +} +static inline VTYPE VWRAP (pw_horner_13) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_13 = VWRAP (pw_horner_11) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_13); +} +static inline VTYPE VWRAP (pw_horner_14) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_14 = VWRAP (pw_horner_12) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_14); +} +static inline VTYPE VWRAP (pw_horner_15) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_15 = VWRAP (pw_horner_13) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_15); +} +static inline VTYPE VWRAP (pw_horner_16) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_16 = VWRAP (pw_horner_14) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_16); +} +static inline VTYPE VWRAP (pw_horner_17) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_17 = VWRAP (pw_horner_15) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_17); +} +static inline VTYPE VWRAP (pw_horner_18) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_18 = VWRAP (pw_horner_16) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_18); +} diff --git a/sysdeps/aarch64/fpu/tan_advsimd.c b/sysdeps/aarch64/fpu/tan_advsimd.c new file mode 100644 index 0000000000..936a0569c8 --- /dev/null +++ b/sysdeps/aarch64/fpu/tan_advsimd.c @@ -0,0 +1,123 @@ +/* Double-precision vector (Advanced SIMD) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +static const struct data +{ + float64x2_t poly[9]; + float64x2_t half_pi_hi, half_pi_lo, two_over_pi, shift; +#if !WANT_SIMD_EXCEPT + float64x2_t range_val; +#endif +} data = { + /* Coefficients generated using FPMinimax. */ + .poly = { V2 (0x1.5555555555556p-2), V2 (0x1.1111111110a63p-3), + V2 (0x1.ba1ba1bb46414p-5), V2 (0x1.664f47e5b5445p-6), + V2 (0x1.226e5e5ecdfa3p-7), V2 (0x1.d6c7ddbf87047p-9), + V2 (0x1.7ea75d05b583ep-10), V2 (0x1.289f22964a03cp-11), + V2 (0x1.4e4fd14147622p-12) }, + .half_pi_hi = V2 (0x1.921fb54442d18p0), + .half_pi_lo = V2 (0x1.1a62633145c07p-54), + .two_over_pi = V2 (0x1.45f306dc9c883p-1), + .shift = V2 (0x1.8p52), +#if !WANT_SIMD_EXCEPT + .range_val = V2 (0x1p23), +#endif +}; + +#define RangeVal 0x4160000000000000 /* asuint64(0x1p23). */ +#define TinyBound 0x3e50000000000000 /* asuint64(2^-26). */ +#define Thresh 0x310000000000000 /* RangeVal - TinyBound. */ + +/* Special cases (fall back to scalar calls). */ +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x) +{ + return v_call_f64 (tan, x, x, v_u64 (-1)); +} + +/* Vector approximation for double-precision tan. + Maximum measured error is 3.48 ULP: + __v_tan(0x1.4457047ef78d8p+20) got -0x1.f6ccd8ecf7dedp+37 + want -0x1.f6ccd8ecf7deap+37. */ +float64x2_t VPCS_ATTR V_NAME_D1 (tan) (float64x2_t x) +{ + const struct data *dat = ptr_barrier (&data); + /* Our argument reduction cannot calculate q with sufficient accuracy for very + large inputs. Fall back to scalar routine for all lanes if any are too + large, or Inf/NaN. If fenv exceptions are expected, also fall back for tiny + input to avoid underflow. */ +#if WANT_SIMD_EXCEPT + uint64x2_t iax = vreinterpretq_u64_f64 (vabsq_f64 (x)); + /* iax - tiny_bound > range_val - tiny_bound. */ + uint64x2_t special + = vcgtq_u64 (vsubq_u64 (iax, v_u64 (TinyBound)), v_u64 (Thresh)); + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x); +#endif + + /* q = nearest integer to 2 * x / pi. */ + float64x2_t q + = vsubq_f64 (vfmaq_f64 (dat->shift, x, dat->two_over_pi), dat->shift); + int64x2_t qi = vcvtq_s64_f64 (q); + + /* Use q to reduce x to r in [-pi/4, pi/4], by: + r = x - q * pi/2, in extended precision. */ + float64x2_t r = x; + r = vfmsq_f64 (r, q, dat->half_pi_hi); + r = vfmsq_f64 (r, q, dat->half_pi_lo); + /* Further reduce r to [-pi/8, pi/8], to be reconstructed using double angle + formula. */ + r = vmulq_n_f64 (r, 0.5); + + /* Approximate tan(r) using order 8 polynomial. + tan(x) is odd, so polynomial has the form: + tan(x) ~= x + C0 * x^3 + C1 * x^5 + C3 * x^7 + ... + Hence we first approximate P(r) = C1 + C2 * r^2 + C3 * r^4 + ... + Then compute the approximation by: + tan(r) ~= r + r^3 * (C0 + r^2 * P(r)). */ + float64x2_t r2 = vmulq_f64 (r, r), r4 = vmulq_f64 (r2, r2), + r8 = vmulq_f64 (r4, r4); + /* Offset coefficients to evaluate from C1 onwards. */ + float64x2_t p = v_estrin_7_f64 (r2, r4, r8, dat->poly + 1); + p = vfmaq_f64 (dat->poly[0], p, r2); + p = vfmaq_f64 (r, r2, vmulq_f64 (p, r)); + + /* Recombination uses double-angle formula: + tan(2x) = 2 * tan(x) / (1 - (tan(x))^2) + and reciprocity around pi/2: + tan(x) = 1 / (tan(pi/2 - x)) + to assemble result using change-of-sign and conditional selection of + numerator/denominator, dependent on odd/even-ness of q (hence quadrant). */ + float64x2_t n = vfmaq_f64 (v_f64 (-1), p, p); + float64x2_t d = vaddq_f64 (p, p); + + uint64x2_t no_recip = vtstq_u64 (vreinterpretq_u64_s64 (qi), v_u64 (1)); + +#if !WANT_SIMD_EXCEPT + uint64x2_t special = vceqzq_u64 (vcaleq_f64 (x, dat->range_val)); + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x); +#endif + + return vdivq_f64 (vbslq_f64 (no_recip, n, vnegq_f64 (d)), + vbslq_f64 (no_recip, d, n)); +} diff --git a/sysdeps/aarch64/fpu/tan_sve.c b/sysdeps/aarch64/fpu/tan_sve.c new file mode 100644 index 0000000000..df9666711d --- /dev/null +++ b/sysdeps/aarch64/fpu/tan_sve.c @@ -0,0 +1,104 @@ +/* Double-precision vector (SVE) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +static const struct data +{ + double poly[9]; + double half_pi_hi, half_pi_lo, inv_half_pi, range_val, shift; +} data = { + /* Polynomial generated with FPMinimax. */ + .poly = { 0x1.5555555555556p-2, 0x1.1111111110a63p-3, 0x1.ba1ba1bb46414p-5, + 0x1.664f47e5b5445p-6, 0x1.226e5e5ecdfa3p-7, 0x1.d6c7ddbf87047p-9, + 0x1.7ea75d05b583ep-10, 0x1.289f22964a03cp-11, + 0x1.4e4fd14147622p-12, }, + .half_pi_hi = 0x1.921fb54442d18p0, + .half_pi_lo = 0x1.1a62633145c07p-54, + .inv_half_pi = 0x1.45f306dc9c883p-1, + .range_val = 0x1p23, + .shift = 0x1.8p52, +}; + +static svfloat64_t NOINLINE +special_case (svfloat64_t x, svfloat64_t y, svbool_t special) +{ + return sv_call_f64 (tan, x, y, special); +} + +/* Vector approximation for double-precision tan. + Maximum measured error is 3.48 ULP: + _ZGVsMxv_tan(0x1.4457047ef78d8p+20) got -0x1.f6ccd8ecf7dedp+37 + want -0x1.f6ccd8ecf7deap+37. */ +svfloat64_t SV_NAME_D1 (tan) (svfloat64_t x, svbool_t pg) +{ + const struct data *dat = ptr_barrier (&data); + + /* Invert condition to catch NaNs and Infs as well as large values. */ + svbool_t special = svnot_z (pg, svaclt (pg, x, dat->range_val)); + + /* q = nearest integer to 2 * x / pi. */ + svfloat64_t shift = sv_f64 (dat->shift); + svfloat64_t q = svmla_x (pg, shift, x, dat->inv_half_pi); + q = svsub_x (pg, q, shift); + svint64_t qi = svcvt_s64_x (pg, q); + + /* Use q to reduce x to r in [-pi/4, pi/4], by: + r = x - q * pi/2, in extended precision. */ + svfloat64_t r = x; + svfloat64_t half_pi = svld1rq (svptrue_b64 (), &dat->half_pi_hi); + r = svmls_lane (r, q, half_pi, 0); + r = svmls_lane (r, q, half_pi, 1); + /* Further reduce r to [-pi/8, pi/8], to be reconstructed using double angle + formula. */ + r = svmul_x (pg, r, 0.5); + + /* Approximate tan(r) using order 8 polynomial. + tan(x) is odd, so polynomial has the form: + tan(x) ~= x + C0 * x^3 + C1 * x^5 + C3 * x^7 + ... + Hence we first approximate P(r) = C1 + C2 * r^2 + C3 * r^4 + ... + Then compute the approximation by: + tan(r) ~= r + r^3 * (C0 + r^2 * P(r)). */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t r4 = svmul_x (pg, r2, r2); + svfloat64_t r8 = svmul_x (pg, r4, r4); + /* Use offset version coeff array by 1 to evaluate from C1 onwards. */ + svfloat64_t p = sv_estrin_7_f64_x (pg, r2, r4, r8, dat->poly + 1); + p = svmad_x (pg, p, r2, dat->poly[0]); + p = svmla_x (pg, r, r2, svmul_x (pg, p, r)); + + /* Recombination uses double-angle formula: + tan(2x) = 2 * tan(x) / (1 - (tan(x))^2) + and reciprocity around pi/2: + tan(x) = 1 / (tan(pi/2 - x)) + to assemble result using change-of-sign and conditional selection of + numerator/denominator dependent on odd/even-ness of q (hence quadrant). */ + svbool_t use_recip + = svcmpeq (pg, svand_x (pg, svreinterpret_u64 (qi), 1), 0); + + svfloat64_t n = svmad_x (pg, p, p, -1); + svfloat64_t d = svmul_x (pg, p, 2); + svfloat64_t swap = n; + n = svneg_m (n, use_recip, d); + d = svsel (use_recip, swap, d); + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svdiv_x (svnot_z (pg, special), n, d), special); + return svdiv_x (pg, n, d); +} diff --git a/sysdeps/aarch64/fpu/tanf_advsimd.c b/sysdeps/aarch64/fpu/tanf_advsimd.c new file mode 100644 index 0000000000..4c8a7f740e --- /dev/null +++ b/sysdeps/aarch64/fpu/tanf_advsimd.c @@ -0,0 +1,129 @@ +/* Single-precision vector (Advanced SIMD) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + float32x4_t poly[6]; + float32x4_t neg_half_pi_1, neg_half_pi_2, neg_half_pi_3, two_over_pi, shift; +#if !WANT_SIMD_EXCEPT + float32x4_t range_val; +#endif +} data = { + /* Coefficients generated using FPMinimax. */ + .poly = { V4 (0x1.55555p-2f), V4 (0x1.11166p-3f), V4 (0x1.b88a78p-5f), + V4 (0x1.7b5756p-6f), V4 (0x1.4ef4cep-8f), V4 (0x1.0e1e74p-7f) }, + .neg_half_pi_1 = V4 (-0x1.921fb6p+0f), + .neg_half_pi_2 = V4 (0x1.777a5cp-25f), + .neg_half_pi_3 = V4 (0x1.ee59dap-50f), + .two_over_pi = V4 (0x1.45f306p-1f), + .shift = V4 (0x1.8p+23f), +#if !WANT_SIMD_EXCEPT + .range_val = V4 (0x1p15f), +#endif +}; + +#define RangeVal v_u32 (0x47000000) /* asuint32(0x1p15f). */ +#define TinyBound v_u32 (0x30000000) /* asuint32 (0x1p-31f). */ +#define Thresh v_u32 (0x16000000) /* asuint32(RangeVal) - TinyBound. */ + +/* Special cases (fall back to scalar calls). */ +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + return v_call_f32 (tanf, x, y, cmp); +} + +/* Use a full Estrin scheme to evaluate polynomial. */ +static inline float32x4_t +eval_poly (float32x4_t z, const struct data *d) +{ + float32x4_t z2 = vmulq_f32 (z, z); +#if WANT_SIMD_EXCEPT + /* Tiny z (<= 0x1p-31) will underflow when calculating z^4. If fp exceptions + are to be triggered correctly, sidestep this by fixing such lanes to 0. */ + uint32x4_t will_uflow + = vcleq_u32 (vreinterpretq_u32_f32 (vabsq_f32 (z)), TinyBound); + if (__glibc_unlikely (v_any_u32 (will_uflow))) + z2 = vbslq_f32 (will_uflow, v_f32 (0), z2); +#endif + float32x4_t z4 = vmulq_f32 (z2, z2); + return v_estrin_5_f32 (z, z2, z4, d->poly); +} + +/* Fast implementation of AdvSIMD tanf. + Maximum error is 3.45 ULP: + __v_tanf(-0x1.e5f0cap+13) got 0x1.ff9856p-1 + want 0x1.ff9850p-1. */ +float32x4_t VPCS_ATTR V_NAME_F1 (tan) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + float32x4_t special_arg = x; + + /* iax >= RangeVal means x, if not inf or NaN, is too large to perform fast + regression. */ +#if WANT_SIMD_EXCEPT + uint32x4_t iax = vreinterpretq_u32_f32 (vabsq_f32 (x)); + /* If fp exceptions are to be triggered correctly, also special-case tiny + input, as this will load to overflow later. Fix any special lanes to 1 to + prevent any exceptions being triggered. */ + uint32x4_t special = vcgeq_u32 (vsubq_u32 (iax, TinyBound), Thresh); + if (__glibc_unlikely (v_any_u32 (special))) + x = vbslq_f32 (special, v_f32 (1.0f), x); +#else + /* Otherwise, special-case large and special values. */ + uint32x4_t special = vcageq_f32 (x, d->range_val); +#endif + + /* n = rint(x/(pi/2)). */ + float32x4_t q = vfmaq_f32 (d->shift, d->two_over_pi, x); + float32x4_t n = vsubq_f32 (q, d->shift); + /* Determine if x lives in an interval, where |tan(x)| grows to infinity. */ + uint32x4_t pred_alt = vtstq_u32 (vreinterpretq_u32_f32 (q), v_u32 (1)); + + /* r = x - n * (pi/2) (range reduction into -pi./4 .. pi/4). */ + float32x4_t r; + r = vfmaq_f32 (x, d->neg_half_pi_1, n); + r = vfmaq_f32 (r, d->neg_half_pi_2, n); + r = vfmaq_f32 (r, d->neg_half_pi_3, n); + + /* If x lives in an interval, where |tan(x)| + - is finite, then use a polynomial approximation of the form + tan(r) ~ r + r^3 * P(r^2) = r + r * r^2 * P(r^2). + - grows to infinity then use symmetries of tangent and the identity + tan(r) = cotan(pi/2 - r) to express tan(x) as 1/tan(-r). Finally, use + the same polynomial approximation of tan as above. */ + + /* Invert sign of r if odd quadrant. */ + float32x4_t z = vmulq_f32 (r, vbslq_f32 (pred_alt, v_f32 (-1), v_f32 (1))); + + /* Evaluate polynomial approximation of tangent on [-pi/4, pi/4]. */ + float32x4_t z2 = vmulq_f32 (r, r); + float32x4_t p = eval_poly (z2, d); + float32x4_t y = vfmaq_f32 (z, vmulq_f32 (z, z2), p); + + /* Compute reciprocal and apply if required. */ + float32x4_t inv_y = vdivq_f32 (v_f32 (1.0f), y); + + if (__glibc_unlikely (v_any_u32 (special))) + return special_case (special_arg, vbslq_f32 (pred_alt, inv_y, y), special); + return vbslq_f32 (pred_alt, inv_y, y); +} diff --git a/sysdeps/aarch64/fpu/tanf_sve.c b/sysdeps/aarch64/fpu/tanf_sve.c new file mode 100644 index 0000000000..856cbece7e --- /dev/null +++ b/sysdeps/aarch64/fpu/tanf_sve.c @@ -0,0 +1,118 @@ +/* Single-precision vector (SVE) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float pio2_1, pio2_2, pio2_3, invpio2; + float c1, c3, c5; + float c0, c2, c4, range_val, shift; +} data = { + /* Coefficients generated using: + poly = fpminimax((tan(sqrt(x))-sqrt(x))/x^(3/2), + deg, + [|single ...|], + [a*a;b*b]); + optimize relative error + final prec : 23 bits + deg : 5 + a : 0x1p-126 ^ 2 + b : ((pi) / 0x1p2) ^ 2 + dirty rel error: 0x1.f7c2e4p-25 + dirty abs error: 0x1.f7c2ecp-25. */ + .c0 = 0x1.55555p-2, .c1 = 0x1.11166p-3, + .c2 = 0x1.b88a78p-5, .c3 = 0x1.7b5756p-6, + .c4 = 0x1.4ef4cep-8, .c5 = 0x1.0e1e74p-7, + + .pio2_1 = 0x1.921fb6p+0f, .pio2_2 = -0x1.777a5cp-25f, + .pio2_3 = -0x1.ee59dap-50f, .invpio2 = 0x1.45f306p-1f, + .range_val = 0x1p15f, .shift = 0x1.8p+23f +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t cmp) +{ + return sv_call_f32 (tanf, x, y, cmp); +} + +/* Fast implementation of SVE tanf. + Maximum error is 3.45 ULP: + SV_NAME_F1 (tan)(-0x1.e5f0cap+13) got 0x1.ff9856p-1 + want 0x1.ff9850p-1. */ +svfloat32_t SV_NAME_F1 (tan) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + + /* Determine whether input is too large to perform fast regression. */ + svbool_t cmp = svacge (pg, x, d->range_val); + + svfloat32_t odd_coeffs = svld1rq (svptrue_b32 (), &d->c1); + svfloat32_t pi_vals = svld1rq (svptrue_b32 (), &d->pio2_1); + + /* n = rint(x/(pi/2)). */ + svfloat32_t q = svmla_lane (sv_f32 (d->shift), x, pi_vals, 3); + svfloat32_t n = svsub_x (pg, q, d->shift); + /* n is already a signed integer, simply convert it. */ + svint32_t in = svcvt_s32_x (pg, n); + /* Determine if x lives in an interval, where |tan(x)| grows to infinity. */ + svint32_t alt = svand_x (pg, in, 1); + svbool_t pred_alt = svcmpne (pg, alt, 0); + + /* r = x - n * (pi/2) (range reduction into 0 .. pi/4). */ + svfloat32_t r; + r = svmls_lane (x, n, pi_vals, 0); + r = svmls_lane (r, n, pi_vals, 1); + r = svmls_lane (r, n, pi_vals, 2); + + /* If x lives in an interval, where |tan(x)| + - is finite, then use a polynomial approximation of the form + tan(r) ~ r + r^3 * P(r^2) = r + r * r^2 * P(r^2). + - grows to infinity then use symmetries of tangent and the identity + tan(r) = cotan(pi/2 - r) to express tan(x) as 1/tan(-r). Finally, use + the same polynomial approximation of tan as above. */ + + /* Perform additional reduction if required. */ + svfloat32_t z = svneg_m (r, pred_alt, r); + + /* Evaluate polynomial approximation of tangent on [-pi/4, pi/4], + using Estrin on z^2. */ + svfloat32_t z2 = svmul_x (pg, z, z); + svfloat32_t p01 = svmla_lane (sv_f32 (d->c0), z2, odd_coeffs, 0); + svfloat32_t p23 = svmla_lane (sv_f32 (d->c2), z2, odd_coeffs, 1); + svfloat32_t p45 = svmla_lane (sv_f32 (d->c4), z2, odd_coeffs, 2); + + svfloat32_t z4 = svmul_x (pg, z2, z2); + svfloat32_t p = svmla_x (pg, p01, z4, p23); + + svfloat32_t z8 = svmul_x (pg, z4, z4); + p = svmla_x (pg, p, z8, p45); + + svfloat32_t y = svmla_x (pg, z, p, svmul_x (pg, z, z2)); + + /* Transform result back, if necessary. */ + svfloat32_t inv_y = svdivr_x (pg, y, 1.0f); + + /* No need to pass pg to specialcase here since cmp is a strict subset, + guaranteed by the cmpge above. */ + if (__glibc_unlikely (svptest_any (pg, cmp))) + return special_case (x, svsel (pred_alt, inv_y, y), cmp); + + return svsel (pred_alt, inv_y, y); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index 3b6b1e343d..f76726e324 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,3 +27,4 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) +VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index d7ac47ca22..5a9e5b552a 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,3 +36,4 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) +SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index d4a9ac7154..66c6227087 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,3 +27,4 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) +VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index d44033eab0..3cbf8b48b7 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,3 +36,4 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) +SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 6b25ed43e0..ec84381c75 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1340,11 +1340,19 @@ Function: "tan": float: 1 ldouble: 1 +Function: "tan_advsimd": +double: 2 +float: 2 + Function: "tan_downward": double: 1 float: 2 ldouble: 1 +Function: "tan_sve": +double: 2 +float: 2 + Function: "tan_towardzero": double: 1 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index ae46ef8c34..0ec668582e 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,3 +14,7 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_tan F +GLIBC_2.39 _ZGVnN4v_tanf F +GLIBC_2.39 _ZGVsMxv_tan F +GLIBC_2.39 _ZGVsMxv_tanf F From patchwork Thu Oct 5 16:10:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 77180 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0013F3836E9D for ; Thu, 5 Oct 2023 16:11:26 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02on2076.outbound.protection.outlook.com [40.107.249.76]) by sourceware.org (Postfix) with ESMTPS id 9F526385842C for ; Thu, 5 Oct 2023 16:11:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9F526385842C Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CI2HFVxOPlTtqXt0cZ67Ej0qrD9gRRA7WA/68AV3NOY=; b=ZOCz5ilAFTuU6FUU80NIJvyCrfJy5q0dOayKrSBNcm3sseXt/r3oqgmtxZiArZU15CydohCv32Ht3FHjjRMBkwBRHFnO0pt4p+y03ZlzOwp1ESc+2XpN5JPY8d8atkMYebepfji6mu4KYezI7On7CgO2mTAmuVRhQ3Lsz5E2jGg= Received: from DU2P250CA0014.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:231::19) by AS8PR08MB8134.eurprd08.prod.outlook.com (2603:10a6:20b:54f::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33; Thu, 5 Oct 2023 16:11:02 +0000 Received: from DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:231:cafe::75) by DU2P250CA0014.outlook.office365.com (2603:10a6:10:231::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33 via Frontend Transport; Thu, 5 Oct 2023 16:11:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT047.mail.protection.outlook.com (100.127.143.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.26 via Frontend Transport; Thu, 5 Oct 2023 16:11:02 +0000 Received: ("Tessian outbound 0ae75d4034ba:v211"); Thu, 05 Oct 2023 16:11:02 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: c89dc6de701afe5b X-CR-MTA-TID: 64aa7808 Received: from 13e37ff05e31.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 4F7D663C-727A-467E-BA8B-840EFB5A7EC0.1; Thu, 05 Oct 2023 16:10:56 +0000 Received: from EUR02-DB5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 13e37ff05e31.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 05 Oct 2023 16:10:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C47+GMGfvK00vy7AyHNFxApZiJRSNdMsQtAZ2otGrw6XY9T2p5erQPFkk8No6r8uDu4JIja19O/zMiDW+Q9dQVI1LK6vddiFOX5enRQXDqcvprI7Z3DTCxWfHtz+lh26OufpDcd61a7zqtFwx4xXWr7D3L8QiPP+BC549xRinsoW/acoXRnAcqn1Kjap1JuGGY+I1x4EUfO2fVkiVaCTD41Thypa66l0gxWwbqv2ykK8S4aevhvdtWFidIai/Hq3YOV9C9aGTrqfX52KcMnNxhQWwUZmFAaoIeDCCNog7d6vCUzB46O+fEp2mEz1pwS3KBUJ56BnOFIpSnCo2tRjlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CI2HFVxOPlTtqXt0cZ67Ej0qrD9gRRA7WA/68AV3NOY=; b=iP9n0OXR9YRx6bf+8uMkFwt/Fpp1/yEKulbhBGuRDdMXKAkd8cxHrFdulHUlyRv8WhuUku70kD5c+QMYzWb/KqwEI4nOSMb719pusLgKP4MlDHBxHcyf7w/jNjEWqaAAX8VRYvW1S2Yqeu7hpGydhgldVZAyuoflpix1eF5PKvGVt4gtWQu607TbySXnlZl258QnC60rjPpzii/lVYU8qnwYjHDIQOu4TEc7SlfNAlKVWUut9gwkBbrfwZlAZVGgeE20G2Rpkv+QLKQjObs45jWKUOmwg4zmQXJBZNYNH6C4zq0PKl/zrM++wl/QqdaSL7PYc61O7iLboW4+xbGhIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CI2HFVxOPlTtqXt0cZ67Ej0qrD9gRRA7WA/68AV3NOY=; b=ZOCz5ilAFTuU6FUU80NIJvyCrfJy5q0dOayKrSBNcm3sseXt/r3oqgmtxZiArZU15CydohCv32Ht3FHjjRMBkwBRHFnO0pt4p+y03ZlzOwp1ESc+2XpN5JPY8d8atkMYebepfji6mu4KYezI7On7CgO2mTAmuVRhQ3Lsz5E2jGg= Received: from AM5PR1001CA0069.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:15::46) by AS8PR08MB10315.eurprd08.prod.outlook.com (2603:10a6:20b:5bd::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.35; Thu, 5 Oct 2023 16:10:54 +0000 Received: from AM7EUR03FT004.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:15:cafe::88) by AM5PR1001CA0069.outlook.office365.com (2603:10a6:206:15::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT004.mail.protection.outlook.com (100.127.140.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6863.29 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 16:10:54 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 5 Oct 2023 16:10:53 +0000 From: Joe Ramsay To: CC: Joe Ramsay Subject: [PATCH v2 2/5] aarch64: Add vector implementations of exp2 routines Date: Thu, 5 Oct 2023 17:10:49 +0100 Message-ID: <20231005161052.11878-2-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231005161052.11878-1-Joe.Ramsay@arm.com> References: <20231005161052.11878-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT004:EE_|AS8PR08MB10315:EE_|DBAEUR03FT047:EE_|AS8PR08MB8134:EE_ X-MS-Office365-Filtering-Correlation-Id: f36c284b-deaa-4657-80e2-08dbc5bdabf4 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: tu34dcN+iEW8Pn0BZEu1T+scG9bHlmoGj1cuOnHSCmVlvhvy0JDTOeTZcstayviZfsU2uwjB1gAAV+nYVgREFGs72PDa47S1oh7UZJPiGoaJ8R2hGISvHllWuFlaq9iKjZ8SQjIqFOJiVA63TWSMGuw16hhFl0NwoGBoOdKNrLakOrSJoksa9d1ppYFAwF0nPrCTGCrBkMUuREXOqK/8eBnNoBoVqe50qb6kWjvRi6qew6HLaNF88v+g/k3nfFbgXasZ87zHLh2Ifyblst//fp25cPvP8uOdD9WQZurkjTlP5OWp4w6NAdpn5jdcUZqvXBSzMvMEKQs2tnRkaJeBEptdN+O+Y01ED6H4am8jb0uPeMWfHDPcTnJe9mNj1wyaUEcy6FsOS8DhpwfrmdKtm2GbC8e+wG1tDkSyQ3m54ocuCNvw2TSQFU8pjrVc0RudLFH6lyYHvCyaY17iwIBsn8prv9P95pY2+pmaO7HJ+Oh6ysPpnW5TFadN8AB04SUlOIvAhDoYWaYG+epKX9SD9w3Tkii9KTAGpAyZ69X4c322czdL3diMFgXLEUI5mq4ZSDiaounokJKePLcCU5RVEPmWUjETG4QYJMNMKHHt8PAvgRbFj/R4fpcGixIMISjOhKOpqirhPmW0rJWcgRk3VG2LyWQZbO4zXJybOcJ6EMjgRohmjEdsVQpXzn6n7GVkw/hFfV5LuclNT4R9FKKZq6VLRKFovKQK1PxpVsidyoofwc6IB3GVuQV6Y0k7x26ilShjtL+9u8FfnhXmtUExNg== X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(376002)(396003)(346002)(136003)(39860400002)(230922051799003)(186009)(64100799003)(451199024)(82310400011)(1800799009)(36840700001)(46966006)(40470700004)(2616005)(70586007)(40460700003)(6916009)(316002)(70206006)(41300700001)(1076003)(26005)(336012)(426003)(6666004)(36756003)(7696005)(478600001)(36860700001)(82740400003)(86362001)(81166007)(356005)(47076005)(83380400001)(40480700001)(2906002)(30864003)(4326008)(5660300002)(8936002)(8676002)(2004002)(36900700001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB10315 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 33a58055-bd8c-4194-3cb8-08dbc5bda72f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FHnDoLEM1H/PbmVKn4a9WaF+izX/LhI+EbF99dx2wkMswx0o28P6uBcRIFnZdqwaVsB8Zq5UorJYA3xVHExllqNMk5w3inSxjGY4dNR4gbd85IuPJQOADLrAT5EZAzjm+RFgGZ62fL/17DWDPlpPuFKMzcvQKcozqLj97d4vY379utTVM78I1TGSK9iXWtQrQ9Ie0hF7HuXZLXfWrwEBYNJVjHWi9GRVm3IjDMKPuLZzwvQC755qD70VlJ0nKOAP4vawbA6m6QQNwpeO6qB3Drzu+Fx9YmQMBpmruq9J/2M2V1Jm+SyvqWTFIjUMDbKEkjGbX1ARwgRoNLtaYw2jEOvdV5y6+Du9JmAXflWM/fr0rmVzFoqjwsboC5DMB9Z3gDGfhxYNJU90+ctx3kltb8KyEXrtc/fY+bzNyrknZIET7L6vU5ZTJXbLwhCt3zlUwNILLFySIqNb5IdDp7Cd5zmGRgLxuCJOBbUbed/NHc90RGvUXE+Ec8YisK3w/uWD6cakuNIA00uJyCqJ8p7PyR4YuIwIZF7WejD32zS20cDlDoEWrKk3aj+P6YeX0udrVexjP6ZXG5f2xAp1RDW6kk1KNVhZTobEez3OWRbfsLv0aZ2mtZiZPqHhBMLTFXNvHMgEGfbNvdbDr2LKyvmp55pxOz1kqUOLWq0I9p7ZQ9TObAvi/iLGXbafKUwnrZqfPoF56wQsU4w5ujmzpNvUO4E582Ugk8rcem39hPkYI6qpARIkhn+2lz1aDishYPgH X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(376002)(136003)(346002)(39860400002)(396003)(230922051799003)(64100799003)(186009)(451199024)(1800799009)(82310400011)(36840700001)(46966006)(40470700004)(40480700001)(40460700003)(7696005)(478600001)(6666004)(82740400003)(36860700001)(86362001)(81166007)(30864003)(2906002)(6916009)(4326008)(336012)(83380400001)(2616005)(426003)(47076005)(26005)(1076003)(36756003)(70206006)(70586007)(8936002)(41300700001)(8676002)(316002)(5660300002)(2004002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 16:11:02.7006 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f36c284b-deaa-4657-80e2-08dbc5bdabf4 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8134 X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Some routines reuse table from v_exp_data.c --- No changes from v1, but re-sent as part of the stack. Thanks, Joe sysdeps/aarch64/fpu/Makefile | 1 + sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/exp2_advsimd.c | 128 ++++++++++++++++++ sysdeps/aarch64/fpu/exp2_sve.c | 111 +++++++++++++++ sysdeps/aarch64/fpu/exp2f_advsimd.c | 124 +++++++++++++++++ sysdeps/aarch64/fpu/exp2f_sve.c | 75 ++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 ++ .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 13 files changed, 463 insertions(+) create mode 100644 sysdeps/aarch64/fpu/exp2_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp2_sve.c create mode 100644 sysdeps/aarch64/fpu/exp2f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp2f_sve.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index a1bbc9bcaa..9c7c768301 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,5 +1,6 @@ libmvec-supported-funcs = cos \ exp \ + exp2 \ log \ sin \ tan diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index f0ca0940a9..05de4325d5 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -18,6 +18,10 @@ libmvec { _ZGVsMxv_sinf; } GLIBC_2.39 { + _ZGVnN4v_exp2f; + _ZGVnN2v_exp2; + _ZGVsMxv_exp2f; + _ZGVsMxv_exp2; _ZGVnN4v_tanf; _ZGVnN2v_tan; _ZGVsMxv_tanf; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 6193213147..50921b22e5 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -51,12 +51,14 @@ typedef __SVBool_t __sv_bool_t; __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -68,12 +70,14 @@ __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/exp2_advsimd.c b/sysdeps/aarch64/fpu/exp2_advsimd.c new file mode 100644 index 0000000000..8e26033cb9 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2_advsimd.c @@ -0,0 +1,128 @@ +/* Double-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +#define N (1 << V_EXP_TABLE_BITS) +#define IndexMask (N - 1) +#define BigBound 1022.0 +#define UOFlowBound 1280.0 + +static const struct data +{ + float64x2_t poly[4]; + float64x2_t shift, scale_big_bound, scale_uoflow_bound; +} data = { + /* Coefficients are computed using Remez algorithm with + minimisation of the absolute error. */ + .poly = { V2 (0x1.62e42fefa3686p-1), V2 (0x1.ebfbdff82c241p-3), + V2 (0x1.c6b09b16de99ap-5), V2 (0x1.3b2abf5571ad8p-7) }, + .shift = V2 (0x1.8p52 / N), + .scale_big_bound = V2 (BigBound), + .scale_uoflow_bound = V2 (UOFlowBound), +}; + +static inline uint64x2_t +lookup_sbits (uint64x2_t i) +{ + return (uint64x2_t){ __v_exp_data[i[0] & IndexMask], + __v_exp_data[i[1] & IndexMask] }; +} + +#if WANT_SIMD_EXCEPT + +# define TinyBound 0x2000000000000000 /* asuint64(0x1p-511). */ +# define Thres 0x2080000000000000 /* asuint64(512.0) - TinyBound. */ + +/* Call scalar exp2 as a fallback. */ +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x) +{ + return v_call_f64 (exp2, x, x, v_u64 (0xffffffffffffffff)); +} + +#else + +# define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +# define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +# define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +static float64x2_t VPCS_ATTR +special_case (float64x2_t s, float64x2_t y, float64x2_t n, + const struct data *d) +{ + /* 2^(n/N) may overflow, break it up into s1*s2. */ + uint64x2_t b = vandq_u64 (vclezq_f64 (n), v_u64 (SpecialOffset)); + float64x2_t s1 = vreinterpretq_f64_u64 (vsubq_u64 (v_u64 (SpecialBias1), b)); + float64x2_t s2 = vreinterpretq_f64_u64 ( + vaddq_u64 (vsubq_u64 (vreinterpretq_u64_f64 (s), v_u64 (SpecialBias2)), b)); + uint64x2_t cmp = vcagtq_f64 (n, d->scale_uoflow_bound); + float64x2_t r1 = vmulq_f64 (s1, s1); + float64x2_t r0 = vmulq_f64 (vfmaq_f64 (s2, s2, y), s1); + return vbslq_f64 (cmp, r1, r0); +} + +#endif + +/* Fast vector implementation of exp2. + Maximum measured error is 1.65 ulp. + _ZGVnN2v_exp2(-0x1.4c264ab5b559bp-6) got 0x1.f8db0d4df721fp-1 + want 0x1.f8db0d4df721dp-1. */ +VPCS_ATTR +float64x2_t V_NAME_D1 (exp2) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t cmp; +#if WANT_SIMD_EXCEPT + uint64x2_t ia = vreinterpretq_u64_f64 (vabsq_f64 (x)); + cmp = vcgeq_u64 (vsubq_u64 (ia, v_u64 (TinyBound)), v_u64 (Thres)); + /* If any special case (inf, nan, small and large x) is detected, + fall back to scalar for all lanes. */ + if (__glibc_unlikely (v_any_u64 (cmp))) + return special_case (x); +#else + cmp = vcagtq_f64 (x, d->scale_big_bound); +#endif + + /* n = round(x/N). */ + float64x2_t z = vaddq_f64 (d->shift, x); + uint64x2_t u = vreinterpretq_u64_f64 (z); + float64x2_t n = vsubq_f64 (z, d->shift); + + /* r = x - n/N. */ + float64x2_t r = vsubq_f64 (x, n); + + /* s = 2^(n/N). */ + uint64x2_t e = vshlq_n_u64 (u, 52 - V_EXP_TABLE_BITS); + u = lookup_sbits (u); + float64x2_t s = vreinterpretq_f64_u64 (vaddq_u64 (u, e)); + + /* y ~ exp2(r) - 1. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t y = v_pairwise_poly_3_f64 (r, r2, d->poly); + y = vmulq_f64 (r, y); + +#if !WANT_SIMD_EXCEPT + if (__glibc_unlikely (v_any_u64 (cmp))) + return special_case (s, y, n, d); +#endif + return vfmaq_f64 (s, s, y); +} diff --git a/sysdeps/aarch64/fpu/exp2_sve.c b/sysdeps/aarch64/fpu/exp2_sve.c new file mode 100644 index 0000000000..c9314a9fee --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2_sve.c @@ -0,0 +1,111 @@ +/* Double-precision vector (SVE) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define N (1 << V_EXP_TABLE_BITS) + +#define BigBound 1022 +#define UOFlowBound 1280 + +static const struct data +{ + double poly[4]; + double shift, big_bound, uoflow_bound; +} data = { + /* Coefficients are computed using Remez algorithm with + minimisation of the absolute error. */ + .poly = { 0x1.62e42fefa3686p-1, 0x1.ebfbdff82c241p-3, 0x1.c6b09b16de99ap-5, + 0x1.3b2abf5571ad8p-7 }, + .shift = 0x1.8p52 / N, + .uoflow_bound = UOFlowBound, + .big_bound = BigBound, +}; + +#define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +#define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +#define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +/* Update of both special and non-special cases, if any special case is + detected. */ +static inline svfloat64_t +special_case (svbool_t pg, svfloat64_t s, svfloat64_t y, svfloat64_t n, + const struct data *d) +{ + /* s=2^n may overflow, break it up into s=s1*s2, + such that exp = s + s*y can be computed as s1*(s2+s2*y) + and s1*s1 overflows only if n>0. */ + + /* If n<=0 then set b to 0x6, 0 otherwise. */ + svbool_t p_sign = svcmple (pg, n, 0.0); /* n <= 0. */ + svuint64_t b = svdup_u64_z (p_sign, SpecialOffset); + + /* Set s1 to generate overflow depending on sign of exponent n. */ + svfloat64_t s1 = svreinterpret_f64 (svsubr_x (pg, b, SpecialBias1)); + /* Offset s to avoid overflow in final result if n is below threshold. */ + svfloat64_t s2 = svreinterpret_f64 ( + svadd_x (pg, svsub_x (pg, svreinterpret_u64 (s), SpecialBias2), b)); + + /* |n| > 1280 => 2^(n) overflows. */ + svbool_t p_cmp = svacgt (pg, n, d->uoflow_bound); + + svfloat64_t r1 = svmul_x (pg, s1, s1); + svfloat64_t r2 = svmla_x (pg, s2, s2, y); + svfloat64_t r0 = svmul_x (pg, r2, s1); + + return svsel (p_cmp, r1, r0); +} + +/* Fast vector implementation of exp2. + Maximum measured error is 1.65 ulp. + _ZGVsMxv_exp2(-0x1.4c264ab5b559bp-6) got 0x1.f8db0d4df721fp-1 + want 0x1.f8db0d4df721dp-1. */ +svfloat64_t SV_NAME_D1 (exp2) (svfloat64_t x, svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svbool_t no_big_scale = svacle (pg, x, d->big_bound); + svbool_t special = svnot_z (pg, no_big_scale); + + /* Reduce x to k/N + r, where k is integer and r in [-1/2N, 1/2N]. */ + svfloat64_t shift = sv_f64 (d->shift); + svfloat64_t kd = svadd_x (pg, x, shift); + svuint64_t ki = svreinterpret_u64 (kd); + /* kd = k/N. */ + kd = svsub_x (pg, kd, shift); + svfloat64_t r = svsub_x (pg, x, kd); + + /* scale ~= 2^(k/N). */ + svuint64_t idx = svand_x (pg, ki, N - 1); + svuint64_t sbits = svld1_gather_index (pg, __v_exp_data, idx); + /* This is only a valid scale when -1023*N < k < 1024*N. */ + svuint64_t top = svlsl_x (pg, ki, 52 - V_EXP_TABLE_BITS); + svfloat64_t scale = svreinterpret_f64 (svadd_x (pg, sbits, top)); + + /* Approximate exp2(r) using polynomial. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t p = sv_pairwise_poly_3_f64_x (pg, r, r2, d->poly); + svfloat64_t y = svmul_x (pg, r, p); + + /* Assemble exp2(x) = exp2(r) * scale. */ + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (pg, scale, y, kd, d); + return svmla_x (pg, scale, scale, y); +} diff --git a/sysdeps/aarch64/fpu/exp2f_advsimd.c b/sysdeps/aarch64/fpu/exp2f_advsimd.c new file mode 100644 index 0000000000..70b3ab66c1 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2f_advsimd.c @@ -0,0 +1,124 @@ +/* Single-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" + +static const struct data +{ + float32x4_t poly[5]; + uint32x4_t exponent_bias; +#if !WANT_SIMD_EXCEPT + float32x4_t special_bound, scale_thresh; +#endif +} data = { + /* maxerr: 1.962 ulp. */ + .poly = { V4 (0x1.59977ap-10f), V4 (0x1.3ce9e4p-7f), V4 (0x1.c6bd32p-5f), + V4 (0x1.ebf9bcp-3f), V4 (0x1.62e422p-1f) }, + .exponent_bias = V4 (0x3f800000), +#if !WANT_SIMD_EXCEPT + .special_bound = V4 (126.0f), + .scale_thresh = V4 (192.0f), +#endif +}; + +#define C(i) d->poly[i] + +#if WANT_SIMD_EXCEPT + +# define TinyBound v_u32 (0x20000000) /* asuint (0x1p-63). */ +# define BigBound v_u32 (0x42800000) /* asuint (0x1p6). */ +# define SpecialBound v_u32 (0x22800000) /* BigBound - TinyBound. */ + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine for special lanes. */ + return v_call_f32 (exp2f, x, y, cmp); +} + +#else + +# define SpecialOffset v_u32 (0x82000000) +# define SpecialBias v_u32 (0x7f000000) + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t poly, float32x4_t n, uint32x4_t e, uint32x4_t cmp1, + float32x4_t scale, const struct data *d) +{ + /* 2^n may overflow, break it up into s1*s2. */ + uint32x4_t b = vandq_u32 (vclezq_f32 (n), SpecialOffset); + float32x4_t s1 = vreinterpretq_f32_u32 (vaddq_u32 (b, SpecialBias)); + float32x4_t s2 = vreinterpretq_f32_u32 (vsubq_u32 (e, b)); + uint32x4_t cmp2 = vcagtq_f32 (n, d->scale_thresh); + float32x4_t r2 = vmulq_f32 (s1, s1); + float32x4_t r1 = vmulq_f32 (vfmaq_f32 (s2, poly, s2), s1); + /* Similar to r1 but avoids double rounding in the subnormal range. */ + float32x4_t r0 = vfmaq_f32 (scale, poly, scale); + float32x4_t r = vbslq_f32 (cmp1, r1, r0); + return vbslq_f32 (cmp2, r2, r); +} + +#endif + +float32x4_t VPCS_ATTR V_NAME_F1 (exp2) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + float32x4_t n, r, r2, scale, p, q, poly; + uint32x4_t cmp, e; + +#if WANT_SIMD_EXCEPT + /* asuint(|x|) - TinyBound >= BigBound - TinyBound. */ + uint32x4_t ia = vreinterpretq_u32_f32 (vabsq_f32 (x)); + cmp = vcgeq_u32 (vsubq_u32 (ia, TinyBound), SpecialBound); + float32x4_t xm = x; + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special_case to fix special lanes later. This is only necessary if fenv + exceptions are to be triggered correctly. */ + if (__glibc_unlikely (v_any_u32 (cmp))) + x = vbslq_f32 (cmp, v_f32 (1), x); +#endif + + /* exp2(x) = 2^n (1 + poly(r)), with 1 + poly(r) in [1/sqrt(2),sqrt(2)] + x = n + r, with r in [-1/2, 1/2]. */ + n = vrndaq_f32 (x); + r = vsubq_f32 (x, n); + e = vshlq_n_u32 (vreinterpretq_u32_s32 (vcvtaq_s32_f32 (x)), 23); + scale = vreinterpretq_f32_u32 (vaddq_u32 (e, d->exponent_bias)); + +#if !WANT_SIMD_EXCEPT + cmp = vcagtq_f32 (n, d->special_bound); +#endif + + r2 = vmulq_f32 (r, r); + p = vfmaq_f32 (C (1), C (0), r); + q = vfmaq_f32 (C (3), C (2), r); + q = vfmaq_f32 (q, p, r2); + p = vmulq_f32 (C (4), r); + poly = vfmaq_f32 (p, q, r2); + + if (__glibc_unlikely (v_any_u32 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f32 (scale, poly, scale), cmp); +#else + return special_case (poly, n, e, cmp, scale, d); +#endif + + return vfmaq_f32 (scale, poly, scale); +} diff --git a/sysdeps/aarch64/fpu/exp2f_sve.c b/sysdeps/aarch64/fpu/exp2f_sve.c new file mode 100644 index 0000000000..88d10daefc --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2f_sve.c @@ -0,0 +1,75 @@ +/* Single-precision vector (SVE) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f32.h" + +static const struct data +{ + float poly[5]; + float shift, thres; +} data = { + /* Coefficients copied from the polynomial in AdvSIMD variant, reversed for + compatibility with polynomial helpers. */ + .poly = { 0x1.62e422p-1f, 0x1.ebf9bcp-3f, 0x1.c6bd32p-5f, 0x1.3ce9e4p-7f, + 0x1.59977ap-10f }, + /* 1.5*2^17 + 127. */ + .shift = 0x1.903f8p17f, + /* Roughly 87.3. For x < -Thres, the result is subnormal and not handled + correctly by FEXPA. */ + .thres = 0x1.5d5e2ap+6f, +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (exp2f, x, y, special); +} + +/* Single-precision SVE exp2f routine. Implements the same algorithm + as AdvSIMD exp2f. + Worst case error is 1.04 ULPs. + SV_NAME_F1 (exp2)(0x1.943b9p-1) got 0x1.ba7eb2p+0 + want 0x1.ba7ebp+0. */ +svfloat32_t SV_NAME_F1 (exp2) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + /* exp2(x) = 2^n (1 + poly(r)), with 1 + poly(r) in [1/sqrt(2),sqrt(2)] + x = n + r, with r in [-1/2, 1/2]. */ + svfloat32_t shift = sv_f32 (d->shift); + svfloat32_t z = svadd_x (pg, x, shift); + svfloat32_t n = svsub_x (pg, z, shift); + svfloat32_t r = svsub_x (pg, x, n); + + svbool_t special = svacgt (pg, x, d->thres); + svfloat32_t scale = svexpa (svreinterpret_u32 (z)); + + /* Polynomial evaluation: poly(r) ~ exp2(r)-1. + Evaluate polynomial use hybrid scheme - offset ESTRIN by 1 for + coefficients 1 to 4, and apply most significant coefficient directly. */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t p14 = sv_pairwise_poly_3_f32_x (pg, r, r2, d->poly + 1); + svfloat32_t p0 = svmul_x (pg, r, d->poly[0]); + svfloat32_t poly = svmla_x (pg, p0, r2, p14); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (pg, scale, scale, poly), special); + + return svmla_x (pg, scale, scale, poly); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index f76726e324..b2b36fd847 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) +VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 5a9e5b552a..88b76ed678 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) +SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 66c6227087..02ab609b5a 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) +VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index 3cbf8b48b7..fa41ce09d8 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) +SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index ec84381c75..a1e5651c87 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -990,11 +990,19 @@ double: 1 float: 1 ldouble: 1 +Function: "exp2_advsimd": +double: 1 +float: 1 + Function: "exp2_downward": double: 1 float: 1 ldouble: 1 +Function: "exp2_sve": +double: 1 +float: 1 + Function: "exp2_towardzero": double: 1 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 0ec668582e..6046c3d046 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,7 +14,11 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_exp2 F GLIBC_2.39 _ZGVnN2v_tan F +GLIBC_2.39 _ZGVnN4v_exp2f F GLIBC_2.39 _ZGVnN4v_tanf F +GLIBC_2.39 _ZGVsMxv_exp2 F +GLIBC_2.39 _ZGVsMxv_exp2f F GLIBC_2.39 _ZGVsMxv_tan F GLIBC_2.39 _ZGVsMxv_tanf F From patchwork Thu Oct 5 16:10:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 77183 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 81EB53888C4A for ; Thu, 5 Oct 2023 16:12:18 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2088.outbound.protection.outlook.com [40.107.20.88]) by sourceware.org (Postfix) with ESMTPS id E36F4385B534 for ; Thu, 5 Oct 2023 16:11:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E36F4385B534 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zXViPjM/s+wwKFjy7zxs01WbVTxAGTc+Of+5EpM5wqU=; b=yYlMXL3wpva445d8VkgnefjQhyvrStbORf2293qE2K9KySKysdIEzLFe3br+Qw9JLtQpGtzPGlb1ZWFO6p0VdYlDuUSVHCIzR4u36a8S9NsbwbPGUA7hDomFqpQQOqshhBzhPI+rF8KkTl4P/8yThSbCrrrjA3IFHNd0eEw1t1Q= Received: from DUZPR01CA0008.eurprd01.prod.exchangelabs.com (2603:10a6:10:3c3::17) by VI1PR08MB5502.eurprd08.prod.outlook.com (2603:10a6:803:12f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33; Thu, 5 Oct 2023 16:11:04 +0000 Received: from DBAEUR03FT061.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:3c3:cafe::72) by DUZPR01CA0008.outlook.office365.com (2603:10a6:10:3c3::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.29 via Frontend Transport; Thu, 5 Oct 2023 16:11:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT061.mail.protection.outlook.com (100.127.143.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.29 via Frontend Transport; Thu, 5 Oct 2023 16:11:04 +0000 Received: ("Tessian outbound ee9c7f88acf7:v211"); Thu, 05 Oct 2023 16:11:04 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 3aa69ba71678c97a X-CR-MTA-TID: 64aa7808 Received: from 2017b245f538.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 21F59D74-74C1-42EA-92CF-7F4758FFB3E7.1; Thu, 05 Oct 2023 16:10:58 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2017b245f538.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 05 Oct 2023 16:10:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HiCLGtDPUYAVvRTlEXBd1wOkNqH97EFwuDRS78hxb5CIwkD4338Pi6+B+R3vgB7FVcrSY7Tl2jIbcpqQ/XnOI5RGrVDxsgyjqbho9eV482KC5aSrQogcw0NmDdOffQwulzsQIPBsdlNr+9NYilC45kGEJ/1vVHQQeIxK4p76vIDuIWlBTZd2TXXY6IgptMGB/+NHH6YfeH+9ecpeipdv68iE6SNKk0ZghUS7jQt7IxDJOGHbgCtphzrxWKehGFKnYsSGpyK0LPRhmBAsKR0lKXWbHE9X9+KqU5Ipy+5zgk7iAS6BRvEtaSXCvQmkl1TqkTiIXeJdpRBeQ7OXfBXrgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zXViPjM/s+wwKFjy7zxs01WbVTxAGTc+Of+5EpM5wqU=; b=NeMi/xQmnvIGEy2knRvNHxxu6wd/n9NXyEKadE6mQoXIefbIpRdhJ8p9a+Swk/liQACRs0/To/J2oHMrSByprB6r+LKZ9VMLvij+DxNE4frx+sPcbm4jjkbRFNJk+jge9ThWAUubvYzLyCAGq9qluCZcfkKFcCWrT6hDeN8m9NpS6nv8hesWLOPYsSnuEosLOoqwNRLaJZ8TvAJZzqMFvh1eHwIQ3baorr87/IbsnR8Ou9IN3aSpk1ZCnzTR3IzC5tE3xh+vaVNkzAG6pBq9178L2Q8bo08BIXth9tYQvbuVV7c7ezj2HP9qFkHjKXd7yde2mVbGg+C+NpNxPLivLg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zXViPjM/s+wwKFjy7zxs01WbVTxAGTc+Of+5EpM5wqU=; b=yYlMXL3wpva445d8VkgnefjQhyvrStbORf2293qE2K9KySKysdIEzLFe3br+Qw9JLtQpGtzPGlb1ZWFO6p0VdYlDuUSVHCIzR4u36a8S9NsbwbPGUA7hDomFqpQQOqshhBzhPI+rF8KkTl4P/8yThSbCrrrjA3IFHNd0eEw1t1Q= Received: from AM5PR1001CA0046.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:15::23) by DU0PR08MB9153.eurprd08.prod.outlook.com (2603:10a6:10:414::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.35; Thu, 5 Oct 2023 16:10:55 +0000 Received: from AM7EUR03FT004.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:15:cafe::e0) by AM5PR1001CA0046.outlook.office365.com (2603:10a6:206:15::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT004.mail.protection.outlook.com (100.127.140.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6863.29 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 16:10:54 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 From: Joe Ramsay To: CC: Joe Ramsay Subject: [PATCH 3/5] aarch64: Add vector implementations of log2 routines Date: Thu, 5 Oct 2023 17:10:50 +0100 Message-ID: <20231005161052.11878-3-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231005161052.11878-1-Joe.Ramsay@arm.com> References: <20231005161052.11878-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT004:EE_|DU0PR08MB9153:EE_|DBAEUR03FT061:EE_|VI1PR08MB5502:EE_ X-MS-Office365-Filtering-Correlation-Id: c9454ea7-3312-45ed-539f-08dbc5bdad0c x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: CxAWvS/rnWF0t2fR6hRxNozka0xUwfj4+LxNtaR4I/ZxncLA2rtXZPt8z17B+Xoq8niwnFGYo7zVwvl2csARpTt9+d7w99Z2fD7gFYcdDYD1Fb2FIN/cZ94uJL1nb7VGxxYclrU+5paLSTUF3cYYN2YL5MG4UMZmrI9kQF/Ii4pkf/8u9IAHh5cx7Emuu1GBxz0530GdkbuVPu5XECbN+ayD40we4n5ktxktliWD4Uzs3x5AQJkUAQrQAfGUsf43Bsk1ZO+7q9PK+oskmz045193qr/UBEeBe206//tBiu3VgJ6wm2B+O91COYMBZjVEK6PCpQCPBwXGE9w4GOjIGteQO5s/EhxxaFYHu5v6G8eHxgNPGOwJtR0nIQxS2N25yEISpfCCULXZyEmXevwMeSrzU85fN9rTqTVVYZtRFrOrIYdKb7eDvRUrWC3z/i24fvQMrTwhMTxAj5uenCKvKpMVtpQx72OXd1svTRLpr53dZnM/12TXqSEZZQahMb+tVG+pfOtwskKWLLzvXS0PaIP+cz2xwt3885NAog5Ku8uUAC+chIEwgS37wy7XEelFEWnmAguSrjszeh30UPrZF/iIBxXUtbk583jcGv3bTVhx9K0dBKVZoqAsYxX+WuLqG2q7ivbqBU8BQQCEHONS/eHZnNu+rg3f3LVVbVeypuDBKumlpqo8O+v2WS/DNVm7Oe90ilKvk3WYhLa59zaLl84P+K4L4P4HXKIlMjQtMEa+SRh8MtlY3OgcbWQVQ8Xb8erNl9s28oQ8+5Z9kiVS3Mbng/6scfJ8xp6ZHDiH8eSYjPXo783jVtfdwA0r4IWD X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(376002)(396003)(346002)(39860400002)(230922051799003)(64100799003)(82310400011)(451199024)(1800799009)(186009)(40470700004)(46966006)(36840700001)(40460700003)(40480700001)(6666004)(478600001)(7696005)(47076005)(86362001)(81166007)(36860700001)(356005)(82740400003)(30864003)(2906002)(41300700001)(336012)(426003)(83380400001)(2616005)(1076003)(26005)(36756003)(70206006)(5660300002)(70586007)(316002)(6916009)(8936002)(4326008)(8676002)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9153 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT061.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: ff399604-a079-4462-3ab8-08dbc5bda74e X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: y975UWc6wZLqjBv598lgxmljZCXkBjgLK0zJiNMB67oV9rtHlQf68D19LVygdFmVAfGOcOYFTuR7W56cLbYao2cONMwqTOcH6ab+lXT6E+A2wFLaId6WatGKI8LW9WtD3SJoZgZOtkurR/VB0OBpHESgXoiqCFfKCiZjeThgBzfpZx768lk3qlg0ZJuxEbB2S7xNbsxkTPy3/R9vQ5uOV1fIIF6pP0fq2do+/iOGtUnGINgdjkIW25solc1OzdLRHlXJ8pi3PXZ01C1aS8VhEdrEALVykTfPakzeSWJD46IJf/aSFnGnyTOqyWrlB8surz0lwRMONWXGfkHJ29jJ3V4TyeM+mJdKO+OW5IUeTEoJXFWHZDLWbam+ajIzuN1SdMSPvhZPKncIXnlN7/5TGpcRzAAMiT8LYZbEk7Bgymyfa0LQTGMrbDSL+xqvL+GBIiYuDa2uj1NbojJ5+og08dyeo/M6Q01PYfRDG82xzDWzD1cmHaB2Vu+JGsMVc8HJykV7SUPkS8eWWJzzqJEF7lJyofI8LbH9SYVaHkQEvq2bsKyhuJKLM5DKo3GXAEYO0qUjD2LRTZbO0a5ry9OmASv+2dmgeXnO4y+cQeAImXOypWGWTSR1m665EiAFwGMY8+C6ntW1NltGsLzlyOKjbaP36/XtjeIaqTPmrZGXw0uXe3ct8AQMIJhani3ZrI1mMSVPdbE7DzebVhCI2Xesp0mBKGN2CoaLDhu3rjKTO9oAELyM5C/sGAdjipN0DWJjRKo/B7AEr9h0oafpDlB2wg== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(39850400004)(136003)(346002)(376002)(396003)(230922051799003)(1800799009)(64100799003)(451199024)(186009)(82310400011)(36840700001)(40470700004)(46966006)(40480700001)(40460700003)(7696005)(478600001)(6666004)(82740400003)(36860700001)(86362001)(81166007)(30864003)(2906002)(6916009)(4326008)(336012)(83380400001)(2616005)(426003)(47076005)(26005)(1076003)(36756003)(70206006)(70586007)(8936002)(41300700001)(8676002)(316002)(5660300002)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 16:11:04.5345 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c9454ea7-3312-45ed-539f-08dbc5bdad0c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT061.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB5502 X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org A table is also added, which is shared between AdvSIMD and SVE log2. --- Changes from v1: * Transpose table layout for zipwise access * Use half-vectors for AdvSIMD special-case comparison * Optimise return values Thanks, Joe sysdeps/aarch64/fpu/Makefile | 4 +- sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/log2_advsimd.c | 109 ++++++++++++ sysdeps/aarch64/fpu/log2_sve.c | 73 ++++++++ sysdeps/aarch64/fpu/log2f_advsimd.c | 77 ++++++++ sysdeps/aarch64/fpu/log2f_sve.c | 86 +++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/fpu/v_log2_data.c | 165 ++++++++++++++++++ sysdeps/aarch64/fpu/vecmath_config.h | 12 ++ sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 15 files changed, 549 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/fpu/log2_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log2_sve.c create mode 100644 sysdeps/aarch64/fpu/log2f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log2f_sve.c create mode 100644 sysdeps/aarch64/fpu/v_log2_data.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 9c7c768301..c3f204ff0d 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -2,6 +2,7 @@ libmvec-supported-funcs = cos \ exp \ exp2 \ log \ + log2 \ sin \ tan @@ -16,7 +17,8 @@ libmvec-support = $(addsuffix f_advsimd,$(float-advsimd-funcs)) \ $(addsuffix f_sve,$(float-sve-funcs)) \ $(addsuffix _sve,$(double-sve-funcs)) \ v_log_data \ - v_exp_data + v_exp_data \ + v_log2_data endif sve-cflags = -march=armv8-a+sve diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index 05de4325d5..ffe62a6f65 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -22,6 +22,10 @@ libmvec { _ZGVnN2v_exp2; _ZGVsMxv_exp2f; _ZGVsMxv_exp2; + _ZGVnN4v_log2f; + _ZGVnN2v_log2; + _ZGVsMxv_log2f; + _ZGVsMxv_log2; _ZGVnN4v_tanf; _ZGVnN2v_tan; _ZGVsMxv_tanf; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 50921b22e5..92f214b194 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,6 +53,7 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_log2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); @@ -60,6 +61,7 @@ __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_log2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -72,6 +74,7 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_log2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); @@ -79,6 +82,7 @@ __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_log2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/log2_advsimd.c b/sysdeps/aarch64/fpu/log2_advsimd.c new file mode 100644 index 0000000000..4f29924bfa --- /dev/null +++ b/sysdeps/aarch64/fpu/log2_advsimd.c @@ -0,0 +1,109 @@ +/* Double-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +#define N (1 << V_LOG2_TABLE_BITS) + +static const struct data +{ + uint64x2_t min_norm; + uint32x4_t special_bound; + float64x2_t poly[5]; + float64x2_t invln2; + uint64x2_t sign_exp_mask; +} data = { + /* Each coefficient was generated to approximate log(r) for |r| < 0x1.fp-9 + and N = 128, then scaled by log2(e) in extended precision and rounded back + to double precision. */ + .poly = { V2 (-0x1.71547652b83p-1), V2 (0x1.ec709dc340953p-2), + V2 (-0x1.71547651c8f35p-2), V2 (0x1.2777ebe12dda5p-2), + V2 (-0x1.ec738d616fe26p-3) }, + .invln2 = V2 (0x1.71547652b82fep0), + .min_norm = V2 (0x0010000000000000), /* asuint64(0x1p-1022). */ + .special_bound = V4 (0x7fe00000), /* asuint64(inf) - min_norm. */ + .sign_exp_mask = V2 (0xfff0000000000000), +}; + +#define Off v_u64 (0x3fe6900900000000) +#define IndexMask (N - 1) + +struct entry +{ + float64x2_t invc; + float64x2_t log2c; +}; + +static inline struct entry +lookup (uint64x2_t i) +{ + struct entry e; + uint64_t i0 = (i[0] >> (52 - V_LOG2_TABLE_BITS)) & IndexMask; + uint64_t i1 = (i[1] >> (52 - V_LOG2_TABLE_BITS)) & IndexMask; + float64x2_t e0 = vld1q_f64 (&__v_log2_data.table[i0].invc); + float64x2_t e1 = vld1q_f64 (&__v_log2_data.table[i1].invc); + e.invc = vuzp1q_f64 (e0, e1); + e.log2c = vuzp2q_f64 (e0, e1); + return e; +} + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x, float64x2_t y, float64x2_t w, float64x2_t r2, + uint32x2_t special) +{ + return v_call_f64 (log2, x, vfmaq_f64 (w, r2, y), vmovl_u32 (special)); +} + +/* Double-precision vector log2 routine. Implements the same algorithm as + vector log10, with coefficients and table entries scaled in extended + precision. The maximum observed error is 2.58 ULP: + _ZGVnN2v_log2(0x1.0b556b093869bp+0) got 0x1.fffb34198d9dap-5 + want 0x1.fffb34198d9ddp-5. */ +float64x2_t VPCS_ATTR V_NAME_D1 (log2) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t ix = vreinterpretq_u64_f64 (x); + uint32x2_t special = vcge_u32 (vsubhn_u64 (ix, d->min_norm), + vget_low_u32 (d->special_bound)); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + uint64x2_t tmp = vsubq_u64 (ix, Off); + int64x2_t k = vshrq_n_s64 (vreinterpretq_s64_u64 (tmp), 52); + uint64x2_t iz = vsubq_u64 (ix, vandq_u64 (tmp, d->sign_exp_mask)); + float64x2_t z = vreinterpretq_f64_u64 (iz); + + struct entry e = lookup (tmp); + + /* log2(x) = log1p(z/c-1)/log(2) + log2(c) + k. */ + + float64x2_t r = vfmaq_f64 (v_f64 (-1.0), z, e.invc); + float64x2_t kd = vcvtq_f64_s64 (k); + float64x2_t w = vfmaq_f64 (e.log2c, r, d->invln2); + + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t y = v_pw_horner_4_f64 (r, r2, d->poly); + w = vaddq_f64 (kd, w); + + if (__glibc_unlikely (v_any_u32h (special))) + return special_case (x, y, w, r2, special); + return vfmaq_f64 (w, r2, y); +} diff --git a/sysdeps/aarch64/fpu/log2_sve.c b/sysdeps/aarch64/fpu/log2_sve.c new file mode 100644 index 0000000000..0ef6669fd5 --- /dev/null +++ b/sysdeps/aarch64/fpu/log2_sve.c @@ -0,0 +1,73 @@ +/* Double-precision vector (SVE) log2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define N (1 << V_LOG2_TABLE_BITS) +#define Off 0x3fe6900900000000 +#define Max (0x7ff0000000000000) +#define Min (0x0010000000000000) +#define Thresh (0x7fe0000000000000) /* Max - Min. */ + +static svfloat64_t NOINLINE +special_case (svfloat64_t x, svfloat64_t y, svbool_t cmp) +{ + return sv_call_f64 (log2, x, y, cmp); +} + +/* Double-precision SVE log2 routine. + Implements the same algorithm as AdvSIMD log10, with coefficients and table + entries scaled in extended precision. + The maximum observed error is 2.58 ULP: + SV_NAME_D1 (log2)(0x1.0b556b093869bp+0) got 0x1.fffb34198d9dap-5 + want 0x1.fffb34198d9ddp-5. */ +svfloat64_t SV_NAME_D1 (log2) (svfloat64_t x, const svbool_t pg) +{ + svuint64_t ix = svreinterpret_u64 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thresh); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + svuint64_t tmp = svsub_x (pg, ix, Off); + svuint64_t i = svlsr_x (pg, tmp, 51 - V_LOG2_TABLE_BITS); + i = svand_x (pg, i, (N - 1) << 1); + svfloat64_t k = svcvt_f64_x (pg, svasr_x (pg, svreinterpret_s64 (tmp), 52)); + svfloat64_t z = svreinterpret_f64 ( + svsub_x (pg, ix, svand_x (pg, tmp, 0xfffULL << 52))); + + svfloat64_t invc = svld1_gather_index (pg, &__v_log2_data.table[0].invc, i); + svfloat64_t log2c + = svld1_gather_index (pg, &__v_log2_data.table[0].log2c, i); + + /* log2(x) = log1p(z/c-1)/log(2) + log2(c) + k. */ + + svfloat64_t r = svmad_x (pg, invc, z, -1.0); + svfloat64_t w = svmla_x (pg, log2c, r, __v_log2_data.invln2); + + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = sv_pw_horner_4_f64_x (pg, r, r2, __v_log2_data.poly); + w = svadd_x (pg, k, w); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (svnot_z (pg, special), w, r2, y), + special); + return svmla_x (pg, w, r2, y); +} diff --git a/sysdeps/aarch64/fpu/log2f_advsimd.c b/sysdeps/aarch64/fpu/log2f_advsimd.c new file mode 100644 index 0000000000..e913bcda18 --- /dev/null +++ b/sysdeps/aarch64/fpu/log2f_advsimd.c @@ -0,0 +1,77 @@ +/* Single-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + uint32x4_t min_norm; + uint16x8_t special_bound; + uint32x4_t off, mantissa_mask; + float32x4_t poly[9]; +} data = { + /* Coefficients generated using Remez algorithm approximate + log2(1+r)/r for r in [ -1/3, 1/3 ]. + rel error: 0x1.c4c4b0cp-26. */ + .poly = { V4 (0x1.715476p0f), /* (float)(1 / ln(2)). */ + V4 (-0x1.715458p-1f), V4 (0x1.ec701cp-2f), V4 (-0x1.7171a4p-2f), + V4 (0x1.27a0b8p-2f), V4 (-0x1.e5143ep-3f), V4 (0x1.9d8ecap-3f), + V4 (-0x1.c675bp-3f), V4 (0x1.9e495p-3f) }, + .min_norm = V4 (0x00800000), + .special_bound = V8 (0x7f00), /* asuint32(inf) - min_norm. */ + .off = V4 (0x3f2aaaab), /* 0.666667. */ + .mantissa_mask = V4 (0x007fffff), +}; + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t n, float32x4_t p, float32x4_t r, + uint16x4_t cmp) +{ + /* Fall back to scalar code. */ + return v_call_f32 (log2f, x, vfmaq_f32 (n, p, r), vmovl_u16 (cmp)); +} + +/* Fast implementation for single precision AdvSIMD log2, + relies on same argument reduction as AdvSIMD logf. + Maximum error: 2.48 ULPs + _ZGVnN4v_log2f(0x1.558174p+0) got 0x1.a9be84p-2 + want 0x1.a9be8p-2. */ +float32x4_t VPCS_ATTR V_NAME_F1 (log2) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + uint32x4_t u = vreinterpretq_u32_f32 (x); + uint16x4_t special = vcge_u16 (vsubhn_u32 (u, d->min_norm), + vget_low_u16 (d->special_bound)); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = vsubq_u32 (u, d->off); + float32x4_t n = vcvtq_f32_s32 ( + vshrq_n_s32 (vreinterpretq_s32_u32 (u), 23)); /* signextend. */ + u = vaddq_u32 (vandq_u32 (u, d->mantissa_mask), d->off); + float32x4_t r = vsubq_f32 (vreinterpretq_f32_u32 (u), v_f32 (1.0f)); + + /* y = log2(1+r) + n. */ + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t p = v_pw_horner_8_f32 (r, r2, d->poly); + + if (__glibc_unlikely (v_any_u16h (special))) + return special_case (x, n, p, r, special); + return vfmaq_f32 (n, p, r); +} diff --git a/sysdeps/aarch64/fpu/log2f_sve.c b/sysdeps/aarch64/fpu/log2f_sve.c new file mode 100644 index 0000000000..d00813ee24 --- /dev/null +++ b/sysdeps/aarch64/fpu/log2f_sve.c @@ -0,0 +1,86 @@ +/* Single-precision vector (SVE) log2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float poly_02468[5]; + float poly_1357[4]; +} data = { + .poly_1357 = { + /* Coefficients copied from the AdvSIMD routine, then rearranged so that coeffs + 1, 3, 5 and 7 can be loaded as a single quad-word, hence used with _lane + variant of MLA intrinsic. */ + -0x1.715458p-1f, -0x1.7171a4p-2f, -0x1.e5143ep-3f, -0x1.c675bp-3f + }, + .poly_02468 = { 0x1.715476p0f, 0x1.ec701cp-2f, 0x1.27a0b8p-2f, + 0x1.9d8ecap-3f, 0x1.9e495p-3f }, +}; + +#define Min (0x00800000) +#define Max (0x7f800000) +#define Thres (0x7f000000) /* Max - Min. */ +#define MantissaMask (0x007fffff) +#define Off (0x3f2aaaab) /* 0.666667. */ + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t cmp) +{ + return sv_call_f32 (log2f, x, y, cmp); +} + +/* Optimised implementation of SVE log2f, using the same algorithm + and polynomial as AdvSIMD log2f. + Maximum error is 2.48 ULPs: + SV_NAME_F1 (log2)(0x1.558174p+0) got 0x1.a9be84p-2 + want 0x1.a9be8p-2. */ +svfloat32_t SV_NAME_F1 (log2) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + + svuint32_t u = svreinterpret_u32 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, u, Min), Thres); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = svsub_x (pg, u, Off); + svfloat32_t n = svcvt_f32_x ( + pg, svasr_x (pg, svreinterpret_s32 (u), 23)); /* Sign-extend. */ + u = svand_x (pg, u, MantissaMask); + u = svadd_x (pg, u, Off); + svfloat32_t r = svsub_x (pg, svreinterpret_f32 (u), 1.0f); + + /* y = log2(1+r) + n. */ + svfloat32_t r2 = svmul_x (pg, r, r); + + /* Evaluate polynomial using pairwise Horner scheme. */ + svfloat32_t p_1357 = svld1rq (svptrue_b32 (), &d->poly_1357[0]); + svfloat32_t q_01 = svmla_lane (sv_f32 (d->poly_02468[0]), r, p_1357, 0); + svfloat32_t q_23 = svmla_lane (sv_f32 (d->poly_02468[1]), r, p_1357, 1); + svfloat32_t q_45 = svmla_lane (sv_f32 (d->poly_02468[2]), r, p_1357, 2); + svfloat32_t q_67 = svmla_lane (sv_f32 (d->poly_02468[3]), r, p_1357, 3); + svfloat32_t y = svmla_x (pg, q_67, r2, sv_f32 (d->poly_02468[4])); + y = svmla_x (pg, q_45, r2, y); + y = svmla_x (pg, q_23, r2, y); + y = svmla_x (pg, q_01, r2, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (svnot_z (pg, special), n, r, y), special); + return svmla_x (pg, n, r, y); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index b2b36fd847..d30dcd6f95 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,5 +27,6 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) +VPCS_VECTOR_WRAPPER (log2_advsimd, _ZGVnN2v_log2) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 88b76ed678..22a8479100 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,5 +36,6 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) +SVE_VECTOR_WRAPPER (log2_sve, _ZGVsMxv_log2) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 02ab609b5a..e8f7f47c67 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,5 +27,6 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) +VPCS_VECTOR_WRAPPER (log2f_advsimd, _ZGVnN4v_log2f) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index fa41ce09d8..f5e9584265 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,5 +36,6 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) +SVE_VECTOR_WRAPPER (log2f_sve, _ZGVsMxv_log2f) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/fpu/v_log2_data.c b/sysdeps/aarch64/fpu/v_log2_data.c new file mode 100644 index 0000000000..4fb126bf31 --- /dev/null +++ b/sysdeps/aarch64/fpu/v_log2_data.c @@ -0,0 +1,165 @@ +/* Coefficients and table entries for vector log2 + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "vecmath_config.h" + +#define N (1 << V_LOG2_TABLE_BITS) + +const struct v_log2_data __v_log2_data = { + + /* Each coefficient was generated to approximate log(r) for |r| < 0x1.fp-9 + and N = 128, then scaled by log2(e) in extended precision and rounded back + to double precision. */ + .poly = { -0x1.71547652b83p-1, 0x1.ec709dc340953p-2, -0x1.71547651c8f35p-2, + 0x1.2777ebe12dda5p-2, -0x1.ec738d616fe26p-3 }, + + .invln2 = 0x1.71547652b82fep0, + + /* Derived from tables in v_log_data.c in a similar way as v_log10_data.c. + This means invc is unchanged and log2c was calculated by scaling log(c) by + log2(e) in extended precision and rounding back to double precision. */ + .table = { { 0x1.6a133d0dec120p+0, -0x1.00130d57f5fadp-1 }, + { 0x1.6815f2f3e42edp+0, -0x1.f802661bd725ep-2 }, + { 0x1.661e39be1ac9ep+0, -0x1.efea1c6f73a5bp-2 }, + { 0x1.642bfa30ac371p+0, -0x1.e7dd1dcd06f05p-2 }, + { 0x1.623f1d916f323p+0, -0x1.dfdb4ae024809p-2 }, + { 0x1.60578da220f65p+0, -0x1.d7e484d101958p-2 }, + { 0x1.5e75349dea571p+0, -0x1.cff8ad452f6ep-2 }, + { 0x1.5c97fd387a75ap+0, -0x1.c817a666c997fp-2 }, + { 0x1.5abfd2981f200p+0, -0x1.c04152d640419p-2 }, + { 0x1.58eca051dc99cp+0, -0x1.b87595a3f64b2p-2 }, + { 0x1.571e526d9df12p+0, -0x1.b0b4526c44d07p-2 }, + { 0x1.5554d555b3fcbp+0, -0x1.a8fd6d1a90f5ep-2 }, + { 0x1.539015e2a20cdp+0, -0x1.a150ca2559fc6p-2 }, + { 0x1.51d0014ee0164p+0, -0x1.99ae4e62cca29p-2 }, + { 0x1.50148538cd9eep+0, -0x1.9215df1a1e842p-2 }, + { 0x1.4e5d8f9f698a1p+0, -0x1.8a8761fe1f0d9p-2 }, + { 0x1.4cab0edca66bep+0, -0x1.8302bd1cc9a54p-2 }, + { 0x1.4afcf1a9db874p+0, -0x1.7b87d6fb437f6p-2 }, + { 0x1.495327136e16fp+0, -0x1.741696673a86dp-2 }, + { 0x1.47ad9e84af28fp+0, -0x1.6caee2b3c6fe4p-2 }, + { 0x1.460c47b39ae15p+0, -0x1.6550a3666c27ap-2 }, + { 0x1.446f12b278001p+0, -0x1.5dfbc08de02a4p-2 }, + { 0x1.42d5efdd720ecp+0, -0x1.56b022766c84ap-2 }, + { 0x1.4140cfe001a0fp+0, -0x1.4f6db1c955536p-2 }, + { 0x1.3fafa3b421f69p+0, -0x1.4834579063054p-2 }, + { 0x1.3e225c9c8ece5p+0, -0x1.4103fd2249a76p-2 }, + { 0x1.3c98ec29a211ap+0, -0x1.39dc8c3fe6dabp-2 }, + { 0x1.3b13442a413fep+0, -0x1.32bdeed4b5c8fp-2 }, + { 0x1.399156baa3c54p+0, -0x1.2ba80f41e20ddp-2 }, + { 0x1.38131639b4cdbp+0, -0x1.249ad8332f4a7p-2 }, + { 0x1.36987540fbf53p+0, -0x1.1d96347e7f3ebp-2 }, + { 0x1.352166b648f61p+0, -0x1.169a0f7d6604ap-2 }, + { 0x1.33adddb3eb575p+0, -0x1.0fa654a221909p-2 }, + { 0x1.323dcd99fc1d3p+0, -0x1.08baefcf8251ap-2 }, + { 0x1.30d129fefc7d2p+0, -0x1.01d7cd14deecdp-2 }, + { 0x1.2f67e6b72fe7dp+0, -0x1.f5f9b1ad55495p-3 }, + { 0x1.2e01f7cf8b187p+0, -0x1.e853ff76a77afp-3 }, + { 0x1.2c9f518ddc86ep+0, -0x1.dabe5d624cba1p-3 }, + { 0x1.2b3fe86e5f413p+0, -0x1.cd38a5cef4822p-3 }, + { 0x1.29e3b1211b25cp+0, -0x1.bfc2b38d315f9p-3 }, + { 0x1.288aa08b373cfp+0, -0x1.b25c61f5edd0fp-3 }, + { 0x1.2734abcaa8467p+0, -0x1.a5058d18e9cacp-3 }, + { 0x1.25e1c82459b81p+0, -0x1.97be1113e47a3p-3 }, + { 0x1.2491eb1ad59c5p+0, -0x1.8a85cafdf5e27p-3 }, + { 0x1.23450a54048b5p+0, -0x1.7d5c97e8fc45bp-3 }, + { 0x1.21fb1bb09e578p+0, -0x1.704255d6486e4p-3 }, + { 0x1.20b415346d8f7p+0, -0x1.6336e2cedd7bfp-3 }, + { 0x1.1f6fed179a1acp+0, -0x1.563a1d9b0cc6ap-3 }, + { 0x1.1e2e99b93c7b3p+0, -0x1.494be541aaa6fp-3 }, + { 0x1.1cf011a7a882ap+0, -0x1.3c6c1964dd0f2p-3 }, + { 0x1.1bb44b97dba5ap+0, -0x1.2f9a99f19a243p-3 }, + { 0x1.1a7b3e66cdd4fp+0, -0x1.22d747344446p-3 }, + { 0x1.1944e11dc56cdp+0, -0x1.1622020d4f7f5p-3 }, + { 0x1.18112aebb1a6ep+0, -0x1.097aabb3553f3p-3 }, + { 0x1.16e013231b7e9p+0, -0x1.f9c24b48014c5p-4 }, + { 0x1.15b1913f156cfp+0, -0x1.e0aaa3bdc858ap-4 }, + { 0x1.14859cdedde13p+0, -0x1.c7ae257c952d6p-4 }, + { 0x1.135c2dc68cfa4p+0, -0x1.aecc960a03e58p-4 }, + { 0x1.12353bdb01684p+0, -0x1.9605bb724d541p-4 }, + { 0x1.1110bf25b85b4p+0, -0x1.7d595ca7147cep-4 }, + { 0x1.0feeafd2f8577p+0, -0x1.64c74165002d9p-4 }, + { 0x1.0ecf062c51c3bp+0, -0x1.4c4f31c86d344p-4 }, + { 0x1.0db1baa076c8bp+0, -0x1.33f0f70388258p-4 }, + { 0x1.0c96c5bb3048ep+0, -0x1.1bac5abb3037dp-4 }, + { 0x1.0b7e20263e070p+0, -0x1.0381272495f21p-4 }, + { 0x1.0a67c2acd0ce3p+0, -0x1.d6de4eba2de2ap-5 }, + { 0x1.0953a6391e982p+0, -0x1.a6ec4e8156898p-5 }, + { 0x1.0841c3caea380p+0, -0x1.772be542e3e1bp-5 }, + { 0x1.07321489b13eap+0, -0x1.479cadcde852dp-5 }, + { 0x1.062491aee9904p+0, -0x1.183e4265faa5p-5 }, + { 0x1.05193497a7cc5p+0, -0x1.d2207fdaa1b85p-6 }, + { 0x1.040ff6b5f5e9fp+0, -0x1.742486cb4a6a2p-6 }, + { 0x1.0308d19aa6127p+0, -0x1.1687d77cfc299p-6 }, + { 0x1.0203beedb0c67p+0, -0x1.7293623a6b5dep-7 }, + { 0x1.010037d38bcc2p+0, -0x1.70ec80ec8f25dp-8 }, + { 1.0, 0.0 }, + { 0x1.fc06d493cca10p-1, 0x1.704c1ca6b6bc9p-7 }, + { 0x1.f81e6ac3b918fp-1, 0x1.6eac8ba664beap-6 }, + { 0x1.f44546ef18996p-1, 0x1.11e67d040772dp-5 }, + { 0x1.f07b10382c84bp-1, 0x1.6bc665e2105dep-5 }, + { 0x1.ecbf7070e59d4p-1, 0x1.c4f8a9772bf1dp-5 }, + { 0x1.e91213f715939p-1, 0x1.0ebff10fbb951p-4 }, + { 0x1.e572a9a75f7b7p-1, 0x1.3aaf4d7805d11p-4 }, + { 0x1.e1e0e2c530207p-1, 0x1.664ba81a4d717p-4 }, + { 0x1.de5c72d8a8be3p-1, 0x1.9196387da6de4p-4 }, + { 0x1.dae50fa5658ccp-1, 0x1.bc902f2b7796p-4 }, + { 0x1.d77a71145a2dap-1, 0x1.e73ab5f584f28p-4 }, + { 0x1.d41c51166623ep-1, 0x1.08cb78510d232p-3 }, + { 0x1.d0ca6ba0bb29fp-1, 0x1.1dd2fe2f0dcb5p-3 }, + { 0x1.cd847e8e59681p-1, 0x1.32b4784400df4p-3 }, + { 0x1.ca4a499693e00p-1, 0x1.47706f3d49942p-3 }, + { 0x1.c71b8e399e821p-1, 0x1.5c0768ee4a4dcp-3 }, + { 0x1.c3f80faf19077p-1, 0x1.7079e86fc7c6dp-3 }, + { 0x1.c0df92dc2b0ecp-1, 0x1.84c86e1183467p-3 }, + { 0x1.bdd1de3cbb542p-1, 0x1.98f377a34b499p-3 }, + { 0x1.baceb9e1007a3p-1, 0x1.acfb803bc924bp-3 }, + { 0x1.b7d5ef543e55ep-1, 0x1.c0e10098b025fp-3 }, + { 0x1.b4e749977d953p-1, 0x1.d4a46efe103efp-3 }, + { 0x1.b20295155478ep-1, 0x1.e8463f45b8d0bp-3 }, + { 0x1.af279f8e82be2p-1, 0x1.fbc6e3228997fp-3 }, + { 0x1.ac5638197fdf3p-1, 0x1.079364f2e5aa8p-2 }, + { 0x1.a98e2f102e087p-1, 0x1.1133306010a63p-2 }, + { 0x1.a6cf5606d05c1p-1, 0x1.1ac309631bd17p-2 }, + { 0x1.a4197fc04d746p-1, 0x1.24432485370c1p-2 }, + { 0x1.a16c80293dc01p-1, 0x1.2db3b5449132fp-2 }, + { 0x1.9ec82c4dc5bc9p-1, 0x1.3714ee1d7a32p-2 }, + { 0x1.9c2c5a491f534p-1, 0x1.406700ab52c94p-2 }, + { 0x1.9998e1480b618p-1, 0x1.49aa1d87522b2p-2 }, + { 0x1.970d9977c6c2dp-1, 0x1.52de746d7ecb2p-2 }, + { 0x1.948a5c023d212p-1, 0x1.5c0434336b343p-2 }, + { 0x1.920f0303d6809p-1, 0x1.651b8ad6c90d1p-2 }, + { 0x1.8f9b698a98b45p-1, 0x1.6e24a56ab5831p-2 }, + { 0x1.8d2f6b81726f6p-1, 0x1.771fb04ec29b1p-2 }, + { 0x1.8acae5bb55badp-1, 0x1.800cd6f19c25ep-2 }, + { 0x1.886db5d9275b8p-1, 0x1.88ec441df11dfp-2 }, + { 0x1.8617ba567c13cp-1, 0x1.91be21b7c93f5p-2 }, + { 0x1.83c8d27487800p-1, 0x1.9a8298f8c7454p-2 }, + { 0x1.8180de3c5dbe7p-1, 0x1.a339d255c04ddp-2 }, + { 0x1.7f3fbe71cdb71p-1, 0x1.abe3f59f43db7p-2 }, + { 0x1.7d055498071c1p-1, 0x1.b48129deca9efp-2 }, + { 0x1.7ad182e54f65ap-1, 0x1.bd119575364c1p-2 }, + { 0x1.78a42c3c90125p-1, 0x1.c5955e23ebcbcp-2 }, + { 0x1.767d342f76944p-1, 0x1.ce0ca8f4e1557p-2 }, + { 0x1.745c7ef26b00ap-1, 0x1.d6779a5a75774p-2 }, + { 0x1.7241f15769d0fp-1, 0x1.ded6563550d27p-2 }, + { 0x1.702d70d396e41p-1, 0x1.e728ffafd840ep-2 }, + { 0x1.6e1ee3700cd11p-1, 0x1.ef6fb96c8d739p-2 }, + { 0x1.6c162fc9cbe02p-1, 0x1.f7aaa57907219p-2 } } +}; diff --git a/sysdeps/aarch64/fpu/vecmath_config.h b/sysdeps/aarch64/fpu/vecmath_config.h index 0abfd8b701..3aa6c280aa 100644 --- a/sysdeps/aarch64/fpu/vecmath_config.h +++ b/sysdeps/aarch64/fpu/vecmath_config.h @@ -50,4 +50,16 @@ extern const struct v_log_data #define V_EXP_TABLE_BITS 7 extern const uint64_t __v_exp_data[1 << V_EXP_TABLE_BITS] attribute_hidden; + +#define V_LOG2_TABLE_BITS 7 +extern const struct v_log2_data +{ + double poly[5]; + double invln2; + struct + { + double invc, log2c; + } table[1 << V_LOG2_TABLE_BITS]; +} __v_log2_data attribute_hidden; + #endif diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index a1e5651c87..7a5af571e2 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1220,11 +1220,19 @@ double: 1 float: 1 ldouble: 3 +Function: "log2_advsimd": +double: 1 +float: 2 + Function: "log2_downward": double: 3 float: 3 ldouble: 3 +Function: "log2_sve": +double: 1 +float: 2 + Function: "log2_towardzero": double: 2 float: 2 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 6046c3d046..657edab7ae 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -15,10 +15,14 @@ GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F GLIBC_2.39 _ZGVnN2v_exp2 F +GLIBC_2.39 _ZGVnN2v_log2 F GLIBC_2.39 _ZGVnN2v_tan F GLIBC_2.39 _ZGVnN4v_exp2f F +GLIBC_2.39 _ZGVnN4v_log2f F GLIBC_2.39 _ZGVnN4v_tanf F GLIBC_2.39 _ZGVsMxv_exp2 F GLIBC_2.39 _ZGVsMxv_exp2f F +GLIBC_2.39 _ZGVsMxv_log2 F +GLIBC_2.39 _ZGVsMxv_log2f F GLIBC_2.39 _ZGVsMxv_tan F GLIBC_2.39 _ZGVsMxv_tanf F From patchwork Thu Oct 5 16:10:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 77184 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 37ED23887F6A for ; Thu, 5 Oct 2023 16:12:49 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2042.outbound.protection.outlook.com [40.107.104.42]) by sourceware.org (Postfix) with ESMTPS id C77783857036 for ; Thu, 5 Oct 2023 16:11:10 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C77783857036 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tXLFPddzgu4W1ehf2U3LpdwwNdBfFYIX0LaZ9HN53Cs=; b=NZoik+7Nuwr1KPNt3P+6wIW+bokTy7KJ2xY9FKo8cE5LMd2WJSyj4VM4b//zN3/hDtMUjXoPqWpQB3+PbdrE4M78+RwcHy3NK1R6NNA2RMDSEuC96rAAbP0o9fOniI6Ya5J1WmVY/YISjwQ77cSzsO+xisMqnFsgloQiZrzX4JE= Received: from DU2P250CA0005.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:231::10) by DB5PR08MB10309.eurprd08.prod.outlook.com (2603:10a6:10:4a9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.35; Thu, 5 Oct 2023 16:11:05 +0000 Received: from DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:231:cafe::9f) by DU2P250CA0005.outlook.office365.com (2603:10a6:10:231::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.37 via Frontend Transport; Thu, 5 Oct 2023 16:11:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT047.mail.protection.outlook.com (100.127.143.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.26 via Frontend Transport; Thu, 5 Oct 2023 16:11:05 +0000 Received: ("Tessian outbound 0ae75d4034ba:v211"); Thu, 05 Oct 2023 16:11:05 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 9aa58c137a416fba X-CR-MTA-TID: 64aa7808 Received: from 7d97566441f4.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 449C8CD6-7C3C-424B-9163-79D8709C8A15.1; Thu, 05 Oct 2023 16:10:59 +0000 Received: from EUR03-DBA-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7d97566441f4.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 05 Oct 2023 16:10:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iVWtujp2g46L1GL1cWFbS0Kfwn9p3H7HRU0f/3E3BswrwHQnzSU32ksVapPGaOD0+Wmb/pPCfqNa8djDZSmIv/RVCCB9DZQkWoWqR/HTLHgIAAoolsoJjxr+RKeK6dabqZqO+j7I4bd+pAvY0TBd2C25+Ri1xxTrObiRjoDogXgmI9b5qLqIDzyUPTdV/cZ0nuRtikybBAVz/wuff2vcNayGpEUMgNg+QlXUnH9RoOpXx3lH4pJdjZEGDRR9yTSMkiAEgDxjTuSDl1Z+yUCQlgZLC0+kAgMeRRfYUcwHyrzZr/4NGk4gzTBuoFS7i2/ANLfVzg4kcajxU+qoKrMUPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tXLFPddzgu4W1ehf2U3LpdwwNdBfFYIX0LaZ9HN53Cs=; b=h/ukteRFvHcPdYjchEUndm2Qi6ubikPyhQhBS6JPmGh5pVBIK+cLLuVJF7hLyniP9kUiiFLZam10GkYh3hi1mQZ8SxcNTbYmGAoesuO2lqHR04+rcQxEEwqBIS3Cr021w81rketE86AsJazRDHF5i+8xgMplxXyW71PXqFWikb0AWeVEI4k9w4WXndRH5njwmAzXr9mVJbCL2QRrJQHtfdgIcW0GtseG+kmsHjUhq75AJ2Nup/AYDf71xtEggVRfPygP9uhDTiI7mJ3QFDO2U7H3503K2xm/iU6TD3DBtxAFmAGiRpzvBkXnIw+6WdzHYGfOE3mJAVxKz4j+vJpv0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tXLFPddzgu4W1ehf2U3LpdwwNdBfFYIX0LaZ9HN53Cs=; b=NZoik+7Nuwr1KPNt3P+6wIW+bokTy7KJ2xY9FKo8cE5LMd2WJSyj4VM4b//zN3/hDtMUjXoPqWpQB3+PbdrE4M78+RwcHy3NK1R6NNA2RMDSEuC96rAAbP0o9fOniI6Ya5J1WmVY/YISjwQ77cSzsO+xisMqnFsgloQiZrzX4JE= Received: from AS9PR06CA0626.eurprd06.prod.outlook.com (2603:10a6:20b:46e::35) by VI1PR08MB10176.eurprd08.prod.outlook.com (2603:10a6:800:1ce::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.28; Thu, 5 Oct 2023 16:10:56 +0000 Received: from AM7EUR03FT043.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:46e:cafe::d7) by AS9PR06CA0626.outlook.office365.com (2603:10a6:20b:46e::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33 via Frontend Transport; Thu, 5 Oct 2023 16:10:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT043.mail.protection.outlook.com (100.127.140.160) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6863.29 via Frontend Transport; Thu, 5 Oct 2023 16:10:56 +0000 Received: from AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 16:10:54 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 16:10:54 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 From: Joe Ramsay To: CC: Joe Ramsay Subject: [PATCH 4/5] aarch64: Add vector implementations of log10 routines Date: Thu, 5 Oct 2023 17:10:51 +0100 Message-ID: <20231005161052.11878-4-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231005161052.11878-1-Joe.Ramsay@arm.com> References: <20231005161052.11878-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT043:EE_|VI1PR08MB10176:EE_|DBAEUR03FT047:EE_|DB5PR08MB10309:EE_ X-MS-Office365-Filtering-Correlation-Id: 72e02d3c-0caf-4b6c-b0ae-08dbc5bdadd3 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: YtqzEI/IqV7Wu5tVSPvl4x+DzV0ylzZaqI5IuiuSOu3HEOxXn0bkdXQyYaOCsEtthSqWhh9s8Vp4ohZ/FGUguH9U9ugvwY+REL1yGxRUEjAlmqzZnjZ9HbTVl00iOVDjckisJV2PR/8YDou/uq/wHz6wH8jdXFWuHjbyIwAyuP+RY02Wpl6wZCdJnWbd+rsEa9I+i5I26rgQB0FcxiVwP9PYYFo2AtuENwH6+fDBtp3AGcrL4YIkXXi02J0VHxGvz6fPbQRJAKVVCpzR+kc8yKiO+ZukluGgcjHrrduIejjsXSWhx8Z+c/jKI8rkWiwvYaTfYBheihAx+IGk51uXsPwXgEoEBV43Ok2tdY4qlVY75X4GZe1mTcyP8kJSle2zEitfv5uvCSbRfIJiLr+OM3nUnuY33YyJvkkCOF0e1nB9H57u3ZgdC6EnmSgo8eOaY5Id8gwG8587632IJ8nBqocHmAo1Rlzv/c8wLRjlYfxXLLw0YRFub6jQkcAnrNNhPatJL9uUWvN2vxCsxnzw++r3Ld9Muf90niWgpiGkMoTQtz+Slmf6AvowI2bQJOc99cSTTHmIGuiZhKzyXAaiTkxiUL5hE69LHb4W98B1Fjb+f2UmA5x24Ii2QHi/xwGNfUFNw6Nys9/sKbXD0wVFHJeGKz6OB5MiJsRxgOVIrepbNa/D+cu4ijOAVGUXweVknPKa5pGkJI6+Qml4TnlF1creF5ta3LIGbNInn8rP7omb1PSLjGlSor8Rie5QKlPFyhdM0HVVVjsbMD7uHW/dNQ== X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(346002)(376002)(136003)(39850400004)(396003)(230922051799003)(82310400011)(186009)(64100799003)(451199024)(1800799009)(46966006)(36840700001)(316002)(26005)(2616005)(1076003)(2906002)(81166007)(6916009)(8936002)(4326008)(82740400003)(8676002)(356005)(41300700001)(478600001)(86362001)(426003)(30864003)(70586007)(5660300002)(336012)(70206006)(7696005)(36860700001)(36756003)(40480700001)(83380400001)(47076005)(6666004)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB10176 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: fdc225e0-8c2f-4bd1-ddd5-08dbc5bda813 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lyoKeWxzcrIshn1tZiExMKN69Uk6dk/qtqQgeAcidv6vC4wQIP4NiILxsnDJooiqW1P2EqQycye3O/IRlCv33h+dxVFnUYXYJkCq7qnKjM3VjaDZhvpOjGSwvd298N4FB+M8TqABfO1ni0mfcdPvG2NRNy9J0j8iayCQ7FZXQLvMeNiz4qTIu2EU8Q5tlJuGogoKef+QLUc8yeQVBEDM9/ZpGBOj5KopdLecGFl4vRZTyckXvINSbVIr44IqRNxszj1mspn9N+XabxgzKAIBaeIoPkJJrSRuFSJVMeMGY23WFdbVbu5rq5wC/5hRrUd8YnjxsEmIL1H8ccWPkH0Hd7CyVfpYZLnx/zql+nBrycMPFUWuvek/89U0h73WoWpATCTBpzrJwAwvohDBm8a0PX4zXhCJy3fpq2v57IRmKLTzORTslrfcncW4S3j+PFPtcQMF/xPjJ2bBZLuw/u3ZAsWk6+nd6bJJL3Ua7yEt1NNplo1EhR0VgTKK9z0ypkrGeTzXbaADPdh8Y7tmBcZhbV2J7u9Ea7g0zt9O0maZf64iHkXxQGlFnjC3iw2g3eImIz4JSFAwpgCKXu7+sJ/AqlRAwwAigjh8epTfUctzq2BbnG9b+LR2KNaZO8oaL+joQ8tXTXkEeIhGDewSD0hTAOr1GodQ+UZ7qkQ3vZzYyoWgK8anxIpQyrMD6ldMA+WUYjmEYuVLp2b1wa8JL3jTZcjk4uPzIHo0Ol25voqmMgyQZWwsWOvbPHZKTNZ/NW65exXsREKp0gm1Qgf8sOAeHw== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(39850400004)(136003)(396003)(346002)(376002)(230922051799003)(186009)(64100799003)(451199024)(1800799009)(82310400011)(40470700004)(36840700001)(46966006)(83380400001)(6666004)(336012)(426003)(41300700001)(4326008)(7696005)(2616005)(8936002)(8676002)(478600001)(26005)(1076003)(47076005)(36860700001)(70586007)(70206006)(316002)(6916009)(40480700001)(82740400003)(36756003)(2906002)(30864003)(86362001)(81166007)(40460700003)(5660300002)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 16:11:05.8411 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 72e02d3c-0caf-4b6c-b0ae-08dbc5bdadd3 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR08MB10309 X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org A table is also added, which is shared between AdvSIMD and SVE log10. --- Changes from v1: * Transpose table layout for zipwise access * Use half-vectors for AdvSIMD special-case comparison * Optimise return values Thanks, Joe sysdeps/aarch64/fpu/Makefile | 4 +- sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/log10_advsimd.c | 119 ++++++++++++ sysdeps/aarch64/fpu/log10_sve.c | 76 ++++++++ sysdeps/aarch64/fpu/log10f_advsimd.c | 82 ++++++++ sysdeps/aarch64/fpu/log10f_sve.c | 94 ++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/fpu/v_log10_data.c | 175 ++++++++++++++++++ sysdeps/aarch64/fpu/vecmath_config.h | 11 ++ sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 15 files changed, 584 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/fpu/log10_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log10_sve.c create mode 100644 sysdeps/aarch64/fpu/log10f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log10f_sve.c create mode 100644 sysdeps/aarch64/fpu/v_log10_data.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index c3f204ff0d..0a047a150d 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -2,6 +2,7 @@ libmvec-supported-funcs = cos \ exp \ exp2 \ log \ + log10 \ log2 \ sin \ tan @@ -18,7 +19,8 @@ libmvec-support = $(addsuffix f_advsimd,$(float-advsimd-funcs)) \ $(addsuffix _sve,$(double-sve-funcs)) \ v_log_data \ v_exp_data \ - v_log2_data + v_log2_data \ + v_log10_data endif sve-cflags = -march=armv8-a+sve diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index ffe62a6f65..358efed5ee 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -22,6 +22,10 @@ libmvec { _ZGVnN2v_exp2; _ZGVsMxv_exp2f; _ZGVsMxv_exp2; + _ZGVnN4v_log10f; + _ZGVnN2v_log10; + _ZGVsMxv_log10f; + _ZGVsMxv_log10; _ZGVnN4v_log2f; _ZGVnN2v_log2; _ZGVsMxv_log2f; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 92f214b194..59f2efa6d7 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,6 +53,7 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_log10f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_log2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); @@ -61,6 +62,7 @@ __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_log10 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -74,6 +76,7 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_log10f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_log2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); @@ -82,6 +85,7 @@ __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_log10 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/log10_advsimd.c b/sysdeps/aarch64/fpu/log10_advsimd.c new file mode 100644 index 0000000000..05b509f134 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10_advsimd.c @@ -0,0 +1,119 @@ +/* Double-precision vector (AdvSIMD) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +#define N (1 << V_LOG10_TABLE_BITS) + +static const struct data +{ + uint64x2_t min_norm; + uint32x4_t special_bound; + float64x2_t poly[5]; + float64x2_t invln10, log10_2, ln2; + uint64x2_t sign_exp_mask; +} data = { + /* Computed from log coefficients divided by log(10) then rounded to double + precision. */ + .poly = { V2 (-0x1.bcb7b1526e506p-3), V2 (0x1.287a7636be1d1p-3), + V2 (-0x1.bcb7b158af938p-4), V2 (0x1.63c78734e6d07p-4), + V2 (-0x1.287461742fee4p-4) }, + .ln2 = V2 (0x1.62e42fefa39efp-1), + .invln10 = V2 (0x1.bcb7b1526e50ep-2), + .log10_2 = V2 (0x1.34413509f79ffp-2), + .min_norm = V2 (0x0010000000000000), /* asuint64(0x1p-1022). */ + .special_bound = V4 (0x7fe00000), /* asuint64(inf) - min_norm. */ + .sign_exp_mask = V2 (0xfff0000000000000), +}; + +#define Off v_u64 (0x3fe6900900000000) +#define IndexMask (N - 1) + +#define T(s, i) __v_log10_data.s[i] + +struct entry +{ + float64x2_t invc; + float64x2_t log10c; +}; + +static inline struct entry +lookup (uint64x2_t i) +{ + struct entry e; + uint64_t i0 = (i[0] >> (52 - V_LOG10_TABLE_BITS)) & IndexMask; + uint64_t i1 = (i[1] >> (52 - V_LOG10_TABLE_BITS)) & IndexMask; + float64x2_t e0 = vld1q_f64 (&__v_log10_data.table[i0].invc); + float64x2_t e1 = vld1q_f64 (&__v_log10_data.table[i1].invc); + e.invc = vuzp1q_f64 (e0, e1); + e.log10c = vuzp2q_f64 (e0, e1); + return e; +} + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x, float64x2_t y, float64x2_t hi, float64x2_t r2, + uint32x2_t special) +{ + return v_call_f64 (log10, x, vfmaq_f64 (hi, r2, y), vmovl_u32 (special)); +} + +/* Fast implementation of double-precision vector log10 + is a slight modification of double-precision vector log. + Max ULP error: < 2.5 ulp (nearest rounding.) + Maximum measured at 2.46 ulp for x in [0.96, 0.97] + _ZGVnN2v_log10(0x1.13192407fcb46p+0) got 0x1.fff6be3cae4bbp-6 + want 0x1.fff6be3cae4b9p-6. */ +float64x2_t VPCS_ATTR V_NAME_D1 (log10) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t ix = vreinterpretq_u64_f64 (x); + uint32x2_t special = vcge_u32 (vsubhn_u64 (ix, d->min_norm), + vget_low_u32 (d->special_bound)); + + /* x = 2^k z; where z is in range [OFF,2*OFF) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + uint64x2_t tmp = vsubq_u64 (ix, Off); + int64x2_t k = vshrq_n_s64 (vreinterpretq_s64_u64 (tmp), 52); + uint64x2_t iz = vsubq_u64 (ix, vandq_u64 (tmp, d->sign_exp_mask)); + float64x2_t z = vreinterpretq_f64_u64 (iz); + + struct entry e = lookup (tmp); + + /* log10(x) = log1p(z/c-1)/log(10) + log10(c) + k*log10(2). */ + float64x2_t r = vfmaq_f64 (v_f64 (-1.0), z, e.invc); + float64x2_t kd = vcvtq_f64_s64 (k); + + /* hi = r / log(10) + log10(c) + k*log10(2). + Constants in v_log10_data.c are computed (in extended precision) as + e.log10c := e.logc * ivln10. */ + float64x2_t w = vfmaq_f64 (e.log10c, r, d->invln10); + + /* y = log10(1+r) + n * log10(2). */ + float64x2_t hi = vfmaq_f64 (w, kd, d->log10_2); + + /* y = r2*(A0 + r*A1 + r2*(A2 + r*A3 + r2*A4)) + hi. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t y = v_pw_horner_4_f64 (r, r2, d->poly); + + if (__glibc_unlikely (v_any_u32h (special))) + return special_case (x, y, hi, r2, special); + return vfmaq_f64 (hi, r2, y); +} diff --git a/sysdeps/aarch64/fpu/log10_sve.c b/sysdeps/aarch64/fpu/log10_sve.c new file mode 100644 index 0000000000..91060ab4a2 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10_sve.c @@ -0,0 +1,76 @@ +/* Double-precision vector (SVE) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define Min 0x0010000000000000 +#define Max 0x7ff0000000000000 +#define Thres 0x7fe0000000000000 /* Max - Min. */ +#define Off 0x3fe6900900000000 +#define N (1 << V_LOG10_TABLE_BITS) + +static svfloat64_t NOINLINE +special_case (svfloat64_t x, svfloat64_t y, svbool_t special) +{ + return sv_call_f64 (log10, x, y, special); +} + +/* SVE log10 algorithm. + Maximum measured error is 2.46 ulps. + SV_NAME_D1 (log10)(0x1.131956cd4b627p+0) got 0x1.fffbdf6eaa669p-6 + want 0x1.fffbdf6eaa667p-6. */ +svfloat64_t SV_NAME_D1 (log10) (svfloat64_t x, const svbool_t pg) +{ + svuint64_t ix = svreinterpret_u64 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thres); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + svuint64_t tmp = svsub_x (pg, ix, Off); + svuint64_t i = svlsr_x (pg, tmp, 51 - V_LOG10_TABLE_BITS); + i = svand_x (pg, i, (N - 1) << 1); + svfloat64_t k = svcvt_f64_x (pg, svasr_x (pg, svreinterpret_s64 (tmp), 52)); + svfloat64_t z = svreinterpret_f64 ( + svsub_x (pg, ix, svand_x (pg, tmp, 0xfffULL << 52))); + + /* log(x) = k*log(2) + log(c) + log(z/c). */ + svfloat64_t invc = svld1_gather_index (pg, &__v_log10_data.table[0].invc, i); + svfloat64_t logc + = svld1_gather_index (pg, &__v_log10_data.table[0].log10c, i); + + /* We approximate log(z/c) with a polynomial P(x) ~= log(x + 1): + r = z/c - 1 (we look up precomputed 1/c) + log(z/c) ~= P(r). */ + svfloat64_t r = svmad_x (pg, invc, z, -1.0); + + /* hi = log(c) + k*log(2). */ + svfloat64_t w = svmla_x (pg, logc, r, __v_log10_data.invln10); + svfloat64_t hi = svmla_x (pg, w, k, __v_log10_data.log10_2); + + /* y = r2*(A0 + r*A1 + r2*(A2 + r*A3 + r2*A4)) + hi. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = sv_pw_horner_4_f64_x (pg, r, r2, __v_log10_data.poly); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (svnot_z (pg, special), hi, r2, y), + special); + return svmla_x (pg, hi, r2, y); +} diff --git a/sysdeps/aarch64/fpu/log10f_advsimd.c b/sysdeps/aarch64/fpu/log10f_advsimd.c new file mode 100644 index 0000000000..ba02060bbe --- /dev/null +++ b/sysdeps/aarch64/fpu/log10f_advsimd.c @@ -0,0 +1,82 @@ +/* Single-precision vector (AdvSIMD) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + uint32x4_t min_norm; + uint16x8_t special_bound; + float32x4_t poly[8]; + float32x4_t inv_ln10, ln2; + uint32x4_t off, mantissa_mask; +} data = { + /* Use order 9 for log10(1+x), i.e. order 8 for log10(1+x)/x, with x in + [-1/3, 1/3] (offset=2/3). Max. relative error: 0x1.068ee468p-25. */ + .poly = { V4 (-0x1.bcb79cp-3f), V4 (0x1.2879c8p-3f), V4 (-0x1.bcd472p-4f), + V4 (0x1.6408f8p-4f), V4 (-0x1.246f8p-4f), V4 (0x1.f0e514p-5f), + V4 (-0x1.0fc92cp-4f), V4 (0x1.f5f76ap-5f) }, + .ln2 = V4 (0x1.62e43p-1f), + .inv_ln10 = V4 (0x1.bcb7b2p-2f), + .min_norm = V4 (0x00800000), + .special_bound = V8 (0x7f00), /* asuint32(inf) - min_norm. */ + .off = V4 (0x3f2aaaab), /* 0.666667. */ + .mantissa_mask = V4 (0x007fffff), +}; + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, float32x4_t p, float32x4_t r2, + uint16x4_t cmp) +{ + /* Fall back to scalar code. */ + return v_call_f32 (log10f, x, vfmaq_f32 (y, p, r2), vmovl_u16 (cmp)); +} + +/* Fast implementation of AdvSIMD log10f, + uses a similar approach as AdvSIMD logf with the same offset (i.e., 2/3) and + an order 9 polynomial. + Maximum error: 3.305ulps (nearest rounding.) + _ZGVnN4v_log10f(0x1.555c16p+0) got 0x1.ffe2fap-4 + want 0x1.ffe2f4p-4. */ +float32x4_t VPCS_ATTR V_NAME_F1 (log10) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + uint32x4_t u = vreinterpretq_u32_f32 (x); + uint16x4_t special = vcge_u16 (vsubhn_u32 (u, d->min_norm), + vget_low_u16 (d->special_bound)); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = vsubq_u32 (u, d->off); + float32x4_t n = vcvtq_f32_s32 ( + vshrq_n_s32 (vreinterpretq_s32_u32 (u), 23)); /* signextend. */ + u = vaddq_u32 (vandq_u32 (u, d->mantissa_mask), d->off); + float32x4_t r = vsubq_f32 (vreinterpretq_f32_u32 (u), v_f32 (1.0f)); + + /* y = log10(1+r) + n * log10(2). */ + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t poly = v_pw_horner_7_f32 (r, r2, d->poly); + /* y = Log10(2) * n + poly * InvLn(10). */ + float32x4_t y = vfmaq_f32 (r, d->ln2, n); + y = vmulq_f32 (y, d->inv_ln10); + + if (__glibc_unlikely (v_any_u16h (special))) + return special_case (x, y, poly, r2, special); + return vfmaq_f32 (y, poly, r2); +} diff --git a/sysdeps/aarch64/fpu/log10f_sve.c b/sysdeps/aarch64/fpu/log10f_sve.c new file mode 100644 index 0000000000..8729b17ef9 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10f_sve.c @@ -0,0 +1,94 @@ +/* Single-precision vector (SVE) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float poly_0246[4]; + float poly_1357[4]; + float ln2, inv_ln10; +} data = { + .poly_1357 = { + /* Coefficients copied from the AdvSIMD routine, then rearranged so that coeffs + 1, 3, 5 and 7 can be loaded as a single quad-word, hence used with _lane + variant of MLA intrinsic. */ + 0x1.2879c8p-3f, 0x1.6408f8p-4f, 0x1.f0e514p-5f, 0x1.f5f76ap-5f + }, + .poly_0246 = { -0x1.bcb79cp-3f, -0x1.bcd472p-4f, -0x1.246f8p-4f, + -0x1.0fc92cp-4f }, + .ln2 = 0x1.62e43p-1f, + .inv_ln10 = 0x1.bcb7b2p-2f, +}; + +#define Min 0x00800000 +#define Max 0x7f800000 +#define Thres 0x7f000000 /* Max - Min. */ +#define Offset 0x3f2aaaab /* 0.666667. */ +#define MantissaMask 0x007fffff + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (log10f, x, y, special); +} + +/* Optimised implementation of SVE log10f using the same algorithm and + polynomial as AdvSIMD log10f. + Maximum error is 3.31ulps: + SV_NAME_F1 (log10)(0x1.555c16p+0) got 0x1.ffe2fap-4 + want 0x1.ffe2f4p-4. */ +svfloat32_t SV_NAME_F1 (log10) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svuint32_t ix = svreinterpret_u32 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thres); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + ix = svsub_x (pg, ix, Offset); + svfloat32_t n = svcvt_f32_x ( + pg, svasr_x (pg, svreinterpret_s32 (ix), 23)); /* signextend. */ + ix = svand_x (pg, ix, MantissaMask); + ix = svadd_x (pg, ix, Offset); + svfloat32_t r = svsub_x (pg, svreinterpret_f32 (ix), 1.0f); + + /* y = log10(1+r) + n*log10(2) + log10(1+r) ~ r * InvLn(10) + P(r) + where P(r) is a polynomial. Use order 9 for log10(1+x), i.e. order 8 for + log10(1+x)/x, with x in [-1/3, 1/3] (offset=2/3). */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t r4 = svmul_x (pg, r2, r2); + svfloat32_t p_1357 = svld1rq (svptrue_b32 (), &d->poly_1357[0]); + svfloat32_t q_01 = svmla_lane (sv_f32 (d->poly_0246[0]), r, p_1357, 0); + svfloat32_t q_23 = svmla_lane (sv_f32 (d->poly_0246[1]), r, p_1357, 1); + svfloat32_t q_45 = svmla_lane (sv_f32 (d->poly_0246[2]), r, p_1357, 2); + svfloat32_t q_67 = svmla_lane (sv_f32 (d->poly_0246[3]), r, p_1357, 3); + svfloat32_t q_47 = svmla_x (pg, q_45, r2, q_67); + svfloat32_t q_03 = svmla_x (pg, q_01, r2, q_23); + svfloat32_t y = svmla_x (pg, q_03, r4, q_47); + + /* Using hi = Log10(2)*n + r*InvLn(10) is faster but less accurate. */ + svfloat32_t hi = svmla_x (pg, r, n, d->ln2); + hi = svmul_x (pg, hi, d->inv_ln10); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (svnot_z (pg, special), hi, r2, y), + special); + return svmla_x (pg, hi, r2, y); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index d30dcd6f95..8d05498ec9 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,6 +27,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) +VPCS_VECTOR_WRAPPER (log10_advsimd, _ZGVnN2v_log10) VPCS_VECTOR_WRAPPER (log2_advsimd, _ZGVnN2v_log2) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 22a8479100..b65bc6f1e6 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,6 +36,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) +SVE_VECTOR_WRAPPER (log10_sve, _ZGVsMxv_log10) SVE_VECTOR_WRAPPER (log2_sve, _ZGVsMxv_log2) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index e8f7f47c67..6ced0d4488 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,6 +27,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) +VPCS_VECTOR_WRAPPER (log10f_advsimd, _ZGVnN4v_log10f) VPCS_VECTOR_WRAPPER (log2f_advsimd, _ZGVnN4v_log2f) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index f5e9584265..2ed8d0659a 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,6 +36,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) +SVE_VECTOR_WRAPPER (log10f_sve, _ZGVsMxv_log10f) SVE_VECTOR_WRAPPER (log2f_sve, _ZGVsMxv_log2f) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/fpu/v_log10_data.c b/sysdeps/aarch64/fpu/v_log10_data.c new file mode 100644 index 0000000000..564be8eebe --- /dev/null +++ b/sysdeps/aarch64/fpu/v_log10_data.c @@ -0,0 +1,175 @@ +/* Lookup table for double-precision log10(x) vector function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "vecmath_config.h" + +const struct v_log10_data __v_log10_data = { + /* Computed from log's coefficients div by log(10) then rounded to double + precision. */ + .poly = { -0x1.bcb7b1526e506p-3, 0x1.287a7636be1d1p-3, -0x1.bcb7b158af938p-4, + 0x1.63c78734e6d07p-4, -0x1.287461742fee4p-4 }, + .invln10 = 0x1.bcb7b1526e50ep-2, + .log10_2 = 0x1.34413509f79ffp-2, + /* Algorithm: + + x = 2^k z + log10(x) = k log10(2) + log10(c) + poly(z/c - 1) / log(10) + + where z is in [a;2a) which is split into N subintervals (a=0x1.69009p-1, + N=128) and log(c) and 1/c for the ith subinterval comes from lookup + tables: + + table[i].invc = 1/c + table[i].log10c = (double)log10(c) + + where c is near the center of the subinterval and is chosen by trying + several floating point invc candidates around 1/center and selecting one + for which the error in (double)log(c) is minimized (< 0x1p-74), except the + subinterval that contains 1 and the previous one got tweaked to avoid + cancellation. NB: invc should be optimized to minimize error in + (double)log10(c) instead. */ + .table = { { 0x1.6a133d0dec120p+0, -0x1.345825f221684p-3 }, + { 0x1.6815f2f3e42edp+0, -0x1.2f71a1f0c554ep-3 }, + { 0x1.661e39be1ac9ep+0, -0x1.2a91fdb30b1f4p-3 }, + { 0x1.642bfa30ac371p+0, -0x1.25b9260981a04p-3 }, + { 0x1.623f1d916f323p+0, -0x1.20e7081762193p-3 }, + { 0x1.60578da220f65p+0, -0x1.1c1b914aeefacp-3 }, + { 0x1.5e75349dea571p+0, -0x1.1756af5de404dp-3 }, + { 0x1.5c97fd387a75ap+0, -0x1.12985059c90bfp-3 }, + { 0x1.5abfd2981f200p+0, -0x1.0de0628f63df4p-3 }, + { 0x1.58eca051dc99cp+0, -0x1.092ed492e08eep-3 }, + { 0x1.571e526d9df12p+0, -0x1.0483954caf1dfp-3 }, + { 0x1.5554d555b3fcbp+0, -0x1.ffbd27a9adbcp-4 }, + { 0x1.539015e2a20cdp+0, -0x1.f67f7f2e3d1ap-4 }, + { 0x1.51d0014ee0164p+0, -0x1.ed4e1071ceebep-4 }, + { 0x1.50148538cd9eep+0, -0x1.e428bb47413c4p-4 }, + { 0x1.4e5d8f9f698a1p+0, -0x1.db0f6003028d6p-4 }, + { 0x1.4cab0edca66bep+0, -0x1.d201df6749831p-4 }, + { 0x1.4afcf1a9db874p+0, -0x1.c9001ac5c9672p-4 }, + { 0x1.495327136e16fp+0, -0x1.c009f3c78c79p-4 }, + { 0x1.47ad9e84af28fp+0, -0x1.b71f4cb642e53p-4 }, + { 0x1.460c47b39ae15p+0, -0x1.ae400818526b2p-4 }, + { 0x1.446f12b278001p+0, -0x1.a56c091954f87p-4 }, + { 0x1.42d5efdd720ecp+0, -0x1.9ca3332f096eep-4 }, + { 0x1.4140cfe001a0fp+0, -0x1.93e56a3f23e55p-4 }, + { 0x1.3fafa3b421f69p+0, -0x1.8b3292a3903bp-4 }, + { 0x1.3e225c9c8ece5p+0, -0x1.828a9112d9618p-4 }, + { 0x1.3c98ec29a211ap+0, -0x1.79ed4ac35f5acp-4 }, + { 0x1.3b13442a413fep+0, -0x1.715aa51ed28c4p-4 }, + { 0x1.399156baa3c54p+0, -0x1.68d2861c999e9p-4 }, + { 0x1.38131639b4cdbp+0, -0x1.6054d40ded21p-4 }, + { 0x1.36987540fbf53p+0, -0x1.57e17576bc9a2p-4 }, + { 0x1.352166b648f61p+0, -0x1.4f7851798bb0bp-4 }, + { 0x1.33adddb3eb575p+0, -0x1.47194f5690ae3p-4 }, + { 0x1.323dcd99fc1d3p+0, -0x1.3ec456d58ec47p-4 }, + { 0x1.30d129fefc7d2p+0, -0x1.36794ff3e5f55p-4 }, + { 0x1.2f67e6b72fe7dp+0, -0x1.2e382315725e4p-4 }, + { 0x1.2e01f7cf8b187p+0, -0x1.2600b8ed82e91p-4 }, + { 0x1.2c9f518ddc86ep+0, -0x1.1dd2fa85efc12p-4 }, + { 0x1.2b3fe86e5f413p+0, -0x1.15aed136e3961p-4 }, + { 0x1.29e3b1211b25cp+0, -0x1.0d94269d1a30dp-4 }, + { 0x1.288aa08b373cfp+0, -0x1.0582e4a7659f5p-4 }, + { 0x1.2734abcaa8467p+0, -0x1.faf5eb655742dp-5 }, + { 0x1.25e1c82459b81p+0, -0x1.eaf888487e8eep-5 }, + { 0x1.2491eb1ad59c5p+0, -0x1.db0d75ef25a82p-5 }, + { 0x1.23450a54048b5p+0, -0x1.cb348a49e6431p-5 }, + { 0x1.21fb1bb09e578p+0, -0x1.bb6d9c69acdd8p-5 }, + { 0x1.20b415346d8f7p+0, -0x1.abb88368aa7ap-5 }, + { 0x1.1f6fed179a1acp+0, -0x1.9c1517476af14p-5 }, + { 0x1.1e2e99b93c7b3p+0, -0x1.8c833051bfa4dp-5 }, + { 0x1.1cf011a7a882ap+0, -0x1.7d02a78e7fb31p-5 }, + { 0x1.1bb44b97dba5ap+0, -0x1.6d93565e97c5fp-5 }, + { 0x1.1a7b3e66cdd4fp+0, -0x1.5e351695db0c5p-5 }, + { 0x1.1944e11dc56cdp+0, -0x1.4ee7c2ba67adcp-5 }, + { 0x1.18112aebb1a6ep+0, -0x1.3fab35ba16c01p-5 }, + { 0x1.16e013231b7e9p+0, -0x1.307f4ad854bc9p-5 }, + { 0x1.15b1913f156cfp+0, -0x1.2163ddf4f988cp-5 }, + { 0x1.14859cdedde13p+0, -0x1.1258cb5d19e22p-5 }, + { 0x1.135c2dc68cfa4p+0, -0x1.035defdba3188p-5 }, + { 0x1.12353bdb01684p+0, -0x1.e8e651191bce4p-6 }, + { 0x1.1110bf25b85b4p+0, -0x1.cb30a62be444cp-6 }, + { 0x1.0feeafd2f8577p+0, -0x1.ad9a9b3043823p-6 }, + { 0x1.0ecf062c51c3bp+0, -0x1.9023ecda1ccdep-6 }, + { 0x1.0db1baa076c8bp+0, -0x1.72cc592bd82dp-6 }, + { 0x1.0c96c5bb3048ep+0, -0x1.55939eb1f9c6ep-6 }, + { 0x1.0b7e20263e070p+0, -0x1.38797ca6cc5ap-6 }, + { 0x1.0a67c2acd0ce3p+0, -0x1.1b7db35c2c072p-6 }, + { 0x1.0953a6391e982p+0, -0x1.fd400812ee9a2p-7 }, + { 0x1.0841c3caea380p+0, -0x1.c3c05fb4620f1p-7 }, + { 0x1.07321489b13eap+0, -0x1.8a7bf3c40e2e3p-7 }, + { 0x1.062491aee9904p+0, -0x1.517249c15a75cp-7 }, + { 0x1.05193497a7cc5p+0, -0x1.18a2ea5330c91p-7 }, + { 0x1.040ff6b5f5e9fp+0, -0x1.c01abc8cdc4e2p-8 }, + { 0x1.0308d19aa6127p+0, -0x1.4f6261750dec9p-8 }, + { 0x1.0203beedb0c67p+0, -0x1.be37b6612afa7p-9 }, + { 0x1.010037d38bcc2p+0, -0x1.bc3a8398ac26p-10 }, + { 1.0, 0.0 }, + { 0x1.fc06d493cca10p-1, 0x1.bb796219f30a5p-9 }, + { 0x1.f81e6ac3b918fp-1, 0x1.b984fdcba61cep-8 }, + { 0x1.f44546ef18996p-1, 0x1.49cf12adf8e8cp-7 }, + { 0x1.f07b10382c84bp-1, 0x1.b6075b5217083p-7 }, + { 0x1.ecbf7070e59d4p-1, 0x1.10b7466fc30ddp-6 }, + { 0x1.e91213f715939p-1, 0x1.4603e4db6a3a1p-6 }, + { 0x1.e572a9a75f7b7p-1, 0x1.7aeb10e99e105p-6 }, + { 0x1.e1e0e2c530207p-1, 0x1.af6e49b0f0e36p-6 }, + { 0x1.de5c72d8a8be3p-1, 0x1.e38f064f41179p-6 }, + { 0x1.dae50fa5658ccp-1, 0x1.0ba75abbb7623p-5 }, + { 0x1.d77a71145a2dap-1, 0x1.25575ee2dba86p-5 }, + { 0x1.d41c51166623ep-1, 0x1.3ed83f477f946p-5 }, + { 0x1.d0ca6ba0bb29fp-1, 0x1.582aa79af60efp-5 }, + { 0x1.cd847e8e59681p-1, 0x1.714f400fa83aep-5 }, + { 0x1.ca4a499693e00p-1, 0x1.8a46ad3901cb9p-5 }, + { 0x1.c71b8e399e821p-1, 0x1.a311903b6b87p-5 }, + { 0x1.c3f80faf19077p-1, 0x1.bbb086f216911p-5 }, + { 0x1.c0df92dc2b0ecp-1, 0x1.d4242bdda648ep-5 }, + { 0x1.bdd1de3cbb542p-1, 0x1.ec6d167c2af1p-5 }, + { 0x1.baceb9e1007a3p-1, 0x1.0245ed8221426p-4 }, + { 0x1.b7d5ef543e55ep-1, 0x1.0e40856c74f64p-4 }, + { 0x1.b4e749977d953p-1, 0x1.1a269a31120fep-4 }, + { 0x1.b20295155478ep-1, 0x1.25f8718fc076cp-4 }, + { 0x1.af279f8e82be2p-1, 0x1.31b64ffc95bfp-4 }, + { 0x1.ac5638197fdf3p-1, 0x1.3d60787ca5063p-4 }, + { 0x1.a98e2f102e087p-1, 0x1.48f72ccd187fdp-4 }, + { 0x1.a6cf5606d05c1p-1, 0x1.547aad6602f1cp-4 }, + { 0x1.a4197fc04d746p-1, 0x1.5feb3989d3acbp-4 }, + { 0x1.a16c80293dc01p-1, 0x1.6b490f3978c79p-4 }, + { 0x1.9ec82c4dc5bc9p-1, 0x1.76946b3f5e703p-4 }, + { 0x1.9c2c5a491f534p-1, 0x1.81cd895717c83p-4 }, + { 0x1.9998e1480b618p-1, 0x1.8cf4a4055c30ep-4 }, + { 0x1.970d9977c6c2dp-1, 0x1.9809f4c48c0ebp-4 }, + { 0x1.948a5c023d212p-1, 0x1.a30db3f9899efp-4 }, + { 0x1.920f0303d6809p-1, 0x1.ae001905458fcp-4 }, + { 0x1.8f9b698a98b45p-1, 0x1.b8e15a2e3a2cdp-4 }, + { 0x1.8d2f6b81726f6p-1, 0x1.c3b1ace2b0996p-4 }, + { 0x1.8acae5bb55badp-1, 0x1.ce71456edfa62p-4 }, + { 0x1.886db5d9275b8p-1, 0x1.d9205759882c4p-4 }, + { 0x1.8617ba567c13cp-1, 0x1.e3bf1513af0dfp-4 }, + { 0x1.83c8d27487800p-1, 0x1.ee4db0412c414p-4 }, + { 0x1.8180de3c5dbe7p-1, 0x1.f8cc5998de3a5p-4 }, + { 0x1.7f3fbe71cdb71p-1, 0x1.019da085eaeb1p-3 }, + { 0x1.7d055498071c1p-1, 0x1.06cd4acdb4e3dp-3 }, + { 0x1.7ad182e54f65ap-1, 0x1.0bf542bef813fp-3 }, + { 0x1.78a42c3c90125p-1, 0x1.11159f14da262p-3 }, + { 0x1.767d342f76944p-1, 0x1.162e761c10d1cp-3 }, + { 0x1.745c7ef26b00ap-1, 0x1.1b3fddc60d43ep-3 }, + { 0x1.7241f15769d0fp-1, 0x1.2049ebac86aa6p-3 }, + { 0x1.702d70d396e41p-1, 0x1.254cb4fb7836ap-3 }, + { 0x1.6e1ee3700cd11p-1, 0x1.2a484e8d0d252p-3 }, + { 0x1.6c162fc9cbe02p-1, 0x1.2f3ccce1c860bp-3 } } +}; diff --git a/sysdeps/aarch64/fpu/vecmath_config.h b/sysdeps/aarch64/fpu/vecmath_config.h index 3aa6c280aa..2c8e243236 100644 --- a/sysdeps/aarch64/fpu/vecmath_config.h +++ b/sysdeps/aarch64/fpu/vecmath_config.h @@ -62,4 +62,15 @@ extern const struct v_log2_data } table[1 << V_LOG2_TABLE_BITS]; } __v_log2_data attribute_hidden; +#define V_LOG10_TABLE_BITS 7 +extern const struct v_log10_data +{ + double poly[5]; + double invln10, log10_2; + struct + { + double invc, log10c; + } table[1 << V_LOG10_TABLE_BITS]; +} __v_log10_data attribute_hidden; + #endif diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 7a5af571e2..6641c7fa0b 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1180,11 +1180,19 @@ double: 2 float: 2 ldouble: 2 +Function: "log10_advsimd": +double: 1 +float: 2 + Function: "log10_downward": double: 2 float: 3 ldouble: 1 +Function: "log10_sve": +double: 1 +float: 2 + Function: "log10_towardzero": double: 2 float: 2 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 657edab7ae..f8776a6bea 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -15,13 +15,17 @@ GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F GLIBC_2.39 _ZGVnN2v_exp2 F +GLIBC_2.39 _ZGVnN2v_log10 F GLIBC_2.39 _ZGVnN2v_log2 F GLIBC_2.39 _ZGVnN2v_tan F GLIBC_2.39 _ZGVnN4v_exp2f F +GLIBC_2.39 _ZGVnN4v_log10f F GLIBC_2.39 _ZGVnN4v_log2f F GLIBC_2.39 _ZGVnN4v_tanf F GLIBC_2.39 _ZGVsMxv_exp2 F GLIBC_2.39 _ZGVsMxv_exp2f F +GLIBC_2.39 _ZGVsMxv_log10 F +GLIBC_2.39 _ZGVsMxv_log10f F GLIBC_2.39 _ZGVsMxv_log2 F GLIBC_2.39 _ZGVsMxv_log2f F GLIBC_2.39 _ZGVsMxv_tan F From patchwork Thu Oct 5 16:10:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 77181 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id AB316382DC7D for ; Thu, 5 Oct 2023 16:11:29 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2077.outbound.protection.outlook.com [40.107.22.77]) by sourceware.org (Postfix) with ESMTPS id 61D8F3857701 for ; Thu, 5 Oct 2023 16:11:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 61D8F3857701 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=teixP/RKsGniaVlKykIrxTijpXjXG4Nlvtif+O5Io4M=; b=fM//YBhmWyS/EtSmEuZez038kDcYnOUeCsa6L0rHraV8DPIh0PQRdqOlqyLuboQlykr3ALF+XDzLMwwrI4iUzeVn5DT3wRicFu6AsnIs0zJw1wPFKiumzwpyNYvU8fviCDKZVGoQrXwpUpd1TsJFbI2ApXuysH0Mx1JCrNhiagk= Received: from DU2P250CA0001.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:231::6) by AS8PR08MB6008.eurprd08.prod.outlook.com (2603:10a6:20b:29d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.21; Thu, 5 Oct 2023 16:11:04 +0000 Received: from DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:231:cafe::64) by DU2P250CA0001.outlook.office365.com (2603:10a6:10:231::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.37 via Frontend Transport; Thu, 5 Oct 2023 16:11:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT047.mail.protection.outlook.com (100.127.143.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.26 via Frontend Transport; Thu, 5 Oct 2023 16:11:04 +0000 Received: ("Tessian outbound 0ae75d4034ba:v211"); Thu, 05 Oct 2023 16:11:04 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 52854eb5eadc013c X-CR-MTA-TID: 64aa7808 Received: from 385ce8890467.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id BAA2D029-2839-4289-A1B9-E3B3BC11217D.1; Thu, 05 Oct 2023 16:10:58 +0000 Received: from EUR01-VE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 385ce8890467.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 05 Oct 2023 16:10:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fk23jKjtAmYMnTrGsAbsaMWi25MoHVyHNRYTvEZYbDm14zELS1bssamZaIeUXaCvNiInWIT2DwlCXcrN49WORoxrB9rX6poUuvjmnItlW4FDOSKfjGQc5ckNgjohZPanIVclFwo3J1GoRQGAIptil3V9i+P6UE5XERHPApg6T/PMxnzW2ibXvCy4YWC8JRdzr41jhkZgnPQx6sJ/0M9Isf3RiSf8UhiElzwolGZ8CKWOtxc6+wwqJslfMCgKrMdUMHJm3W/Zxon4bePmqJ6oPzdmIqEw71sZ9JbYL4NVeicTuVPwKqz3hfYaw8vimIfcbqIdFr0xQxceFetAJtvg9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=teixP/RKsGniaVlKykIrxTijpXjXG4Nlvtif+O5Io4M=; b=JgkuTZ20fPF2MYvULza2hyl+PUraFJJGkmsB6WpdYh4RfYdIF6fa+RrlHul7V6q2A1RbN0E16xpsPlgFuzeip6ySNYliz2jqD5yEEh/BQ+Wq6YnF1GOuuUZRB3eZabuSaA3xsRNX3U77y2MIEq6Ly2QTmd0ooEYZNttBuZKDIC3RhG+yGz8AdjnCfgzyzOfBuWdYy71Fly6/hXwEEjCWkk0nTefOg/Iz1GTrGnQqsk9o1XTzjrPolY9CsZXDPR7zcTX6vW3D4CPGcn/Hw3oLsarW6oWrDjy7kiJQ+VXS+lLNkulxqdjhKXcOf+30KSRfNCkNPLf7nydVyO/2FLFwLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=teixP/RKsGniaVlKykIrxTijpXjXG4Nlvtif+O5Io4M=; b=fM//YBhmWyS/EtSmEuZez038kDcYnOUeCsa6L0rHraV8DPIh0PQRdqOlqyLuboQlykr3ALF+XDzLMwwrI4iUzeVn5DT3wRicFu6AsnIs0zJw1wPFKiumzwpyNYvU8fviCDKZVGoQrXwpUpd1TsJFbI2ApXuysH0Mx1JCrNhiagk= Received: from AS9PR06CA0619.eurprd06.prod.outlook.com (2603:10a6:20b:46e::33) by AS8PR08MB6535.eurprd08.prod.outlook.com (2603:10a6:20b:336::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.35; Thu, 5 Oct 2023 16:10:56 +0000 Received: from AM7EUR03FT043.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:46e:cafe::be) by AS9PR06CA0619.outlook.office365.com (2603:10a6:20b:46e::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.29 via Frontend Transport; Thu, 5 Oct 2023 16:10:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT043.mail.protection.outlook.com (100.127.140.160) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6863.29 via Frontend Transport; Thu, 5 Oct 2023 16:10:56 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 16:10:54 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 5 Oct 2023 16:10:54 +0000 From: Joe Ramsay To: CC: Joe Ramsay Subject: [PATCH 5/5] aarch64: Add vector implementations of exp10 routines Date: Thu, 5 Oct 2023 17:10:52 +0100 Message-ID: <20231005161052.11878-5-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231005161052.11878-1-Joe.Ramsay@arm.com> References: <20231005161052.11878-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT043:EE_|AS8PR08MB6535:EE_|DBAEUR03FT047:EE_|AS8PR08MB6008:EE_ X-MS-Office365-Filtering-Correlation-Id: b9930668-ddfa-4f6d-0a91-08dbc5bdad28 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: Pz3T+WRTuBcLWnZ5uB/vvF4od+BQPAAPd69c+2qwqzLo/KEI1wnX2YCtcxOZgJ081pnW79eQHa5up4Au+fVWuRqgZ0pT0e/qTEEs0f4TNkw5vrL8R4ew+K8LGHJDly0BMp9ZAg9lGmg8E7IFOSd9mVAJ8jtPLhWHEo9gVZLc9rfaJk65JE/qRcyzh1XIjM6mvdODwCZ4r70Sdx6AOfw3fU2gHDPu4lKGxRfKnLsvhne4tcyKPSoTFJcOJ/xnu4PJjzfCM3zDH9kKl+ON3b08eVWCEQjfEhx6KvN+W4JK8h4F2qAuTDKISjJ6T+/lSvdcZ0GhRmTa/RspAI1LyjxhLOZmMA4TPYnh8ZhGwm+VsEbnJmlfBiKWS11LurhsnpEjHcCVmFu5fQeXshUqeuzATu9p2SivoLN4mBw+VVmIvvpF0MVcVMs+g4cwaOvVwniupLDrfhvchJ4P/uNCbT5fykMtbOol8AfxqnM3ZzlQgONbNpjRcMgK+ugLYCpXQZmUa/1JygudcsBe03ZTfJbVhWf9dMajIU+KdFi1UYgjVb3wnZJpH+05qAMPfKW52phfq2hsAwbzFglEUj6JAFwNajXQyKNb3G1iMGKMUTRLExj8zn16t7eTr8dQl88vqGWESDjvWD2fhsfG97Aq1L/KBmnvcIvZWPHaynikcobp0DSvw2Lqyzdf2UGECWaXsat/tpOV02J+7Ht8zKxe6UM2RYupqtUfP28+iE3nb9pUNyzZk+/5UH7EfDvniD7k/QaIOcswMS3zvL73XkbLg0Jzzs2LXUJSCWz3ANv5DvuXsfNIicS6CpTs9jIleuq2VJv4 X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(346002)(136003)(396003)(376002)(39860400002)(230922051799003)(186009)(64100799003)(451199024)(1800799009)(82310400011)(46966006)(40470700004)(36840700001)(40460700003)(36860700001)(47076005)(2906002)(30864003)(36756003)(40480700001)(82740400003)(86362001)(7696005)(356005)(81166007)(26005)(1076003)(2616005)(41300700001)(8676002)(8936002)(4326008)(316002)(70206006)(6916009)(70586007)(5660300002)(83380400001)(426003)(336012)(478600001)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6535 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: bb9e8d5b-4e08-47b3-4221-08dbc5bda7f7 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ssWBZlqqxmXGYgiwT8UQI+xcmMhJu6G/5IFDwbRsEKvqphjTEhftPQpqEfLuPbxrWLFaRou4ydDSYBBh3qzelLNXMPWaj8D1aI60Ad/d/pFIpoe05J6ki/EDbbmG3b+k3w8+G9Zou5jYvtChAF9rebodI6beb+KAv/aHZJyJnjl8mwV984negv8ZsjW7xSHSKq4jRADNuNehX7iiklP9UP1QHhTySWty/z8WdEDi085wklLiEoj916LkFdTK+p+PJcMjVGTn7Au3ILzzjn5CFpGPe5VK8L/IbOkd/nd3QzsbRjiS5ZzrUZSqOHMkkqsOGeWhWYXu8jnzY+lp5iE+AEma4I7RkIf4+3My8drGnOfhGcMM4u+/2u3ejryJn309TJOusSbsp7mvxU/El+8rqQ02LbBcMQ12nOYKTkgnBP+p1YK61iy9nSyRUeqgnQxLkDjLwrth+bNoAV1NtV7ib0LpeWO6CTBVKSqApG0RskcDojyNS07k3vcyLWWATp+WITxOddRt93rErU3GszFuhfRHGIhSfZeoqI4hwDh2vBmLZW5baH1x5FqNULIDUF9md4IH9/V+qi7XED3vzNAIrr/RdbSJm9JNxtlAHu3BjbXmKNGvl9wBiho3Vkpta+S9PIhkweEqoyb6qtcbgwTGr/YJu8NGulxOtKrPGQrcZxfwRfDQNGq0+6WBelYIflwp22YqBxr62HWBH2bB7OHsY67JkW6V3mYNwBshywPRqApfNT3dbO7D2pzDfwwrMRMH1NguXNVjPtmWsJZCwqBCAQ== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(396003)(376002)(39860400002)(136003)(346002)(230922051799003)(64100799003)(82310400011)(451199024)(1800799009)(186009)(46966006)(40470700004)(36840700001)(47076005)(36860700001)(8936002)(8676002)(83380400001)(4326008)(1076003)(7696005)(70206006)(41300700001)(40480700001)(30864003)(2616005)(70586007)(426003)(316002)(26005)(82740400003)(2906002)(86362001)(478600001)(336012)(6916009)(81166007)(40460700003)(36756003)(5660300002)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 16:11:04.7161 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b9930668-ddfa-4f6d-0a91-08dbc5bdad28 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT047.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6008 X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Double-precision routines either reuse the exp table (AdvSIMD) or use SVE FEXPA intruction. --- Changes from v1: * Use overloaded intrinsics, reflects recent updates in AOR Thanks, Joe sysdeps/aarch64/fpu/Makefile | 1 + sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/exp10_advsimd.c | 145 ++++++++++++++++++ sysdeps/aarch64/fpu/exp10_sve.c | 127 +++++++++++++++ sysdeps/aarch64/fpu/exp10f_advsimd.c | 140 +++++++++++++++++ sysdeps/aarch64/fpu/exp10f_sve.c | 91 +++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 13 files changed, 528 insertions(+) create mode 100644 sysdeps/aarch64/fpu/exp10_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp10_sve.c create mode 100644 sysdeps/aarch64/fpu/exp10f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp10f_sve.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 0a047a150d..1f1ac2a2b8 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,5 +1,6 @@ libmvec-supported-funcs = cos \ exp \ + exp10 \ exp2 \ log \ log10 \ diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index 358efed5ee..eb5ad50017 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -18,6 +18,10 @@ libmvec { _ZGVsMxv_sinf; } GLIBC_2.39 { + _ZGVnN4v_exp10f; + _ZGVnN2v_exp10; + _ZGVsMxv_exp10f; + _ZGVsMxv_exp10; _ZGVnN4v_exp2f; _ZGVnN2v_exp2; _ZGVsMxv_exp2f; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 59f2efa6d7..06587ffa91 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -51,6 +51,7 @@ typedef __SVBool_t __sv_bool_t; __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_exp10f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_log10f (__f32x4_t); @@ -60,6 +61,7 @@ __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_exp10 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log10 (__f64x2_t); @@ -74,6 +76,7 @@ __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_exp10f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_log10f (__sv_f32_t, __sv_bool_t); @@ -83,6 +86,7 @@ __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_exp10 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log10 (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/exp10_advsimd.c b/sysdeps/aarch64/fpu/exp10_advsimd.c new file mode 100644 index 0000000000..c584e2a270 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10_advsimd.c @@ -0,0 +1,145 @@ +/* Double-precision vector (AdvSIMD) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" + +/* Value of |x| above which scale overflows without special treatment. */ +#define SpecialBound 306.0 /* floor (log10 (2^1023)) - 1. */ +/* Value of n above which scale overflows even with special treatment. */ +#define ScaleBound 163840.0 /* 1280.0 * N. */ + +const static struct data +{ + float64x2_t poly[4]; + float64x2_t log10_2, log2_10_hi, log2_10_lo, shift; +#if !WANT_SIMD_EXCEPT + float64x2_t special_bound, scale_thresh; +#endif +} data = { + /* Coefficients generated using Remez algorithm. + rel error: 0x1.5ddf8f28p-54 + abs error: 0x1.5ed266c8p-54 in [ -log10(2)/256, log10(2)/256 ] + maxerr: 1.14432 +0.5 ulp. */ + .poly = { V2 (0x1.26bb1bbb5524p1), V2 (0x1.53524c73cecdap1), + V2 (0x1.047060efb781cp1), V2 (0x1.2bd76040f0d16p0) }, + .log10_2 = V2 (0x1.a934f0979a371p8), /* N/log2(10). */ + .log2_10_hi = V2 (0x1.34413509f79ffp-9), /* log2(10)/N. */ + .log2_10_lo = V2 (-0x1.9dc1da994fd21p-66), + .shift = V2 (0x1.8p+52), +#if !WANT_SIMD_EXCEPT + .scale_thresh = V2 (ScaleBound), + .special_bound = V2 (SpecialBound), +#endif +}; + +#define N (1 << V_EXP_TABLE_BITS) +#define IndexMask v_u64 (N - 1) + +#if WANT_SIMD_EXCEPT + +# define TinyBound v_u64 (0x2000000000000000) /* asuint64 (0x1p-511). */ +# define BigBound v_u64 (0x4070000000000000) /* asuint64 (0x1p8). */ +# define Thres v_u64 (0x2070000000000000) /* BigBound - TinyBound. */ + +static inline float64x2_t VPCS_ATTR +special_case (float64x2_t x, float64x2_t y, uint64x2_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine for special lanes. */ + return v_call_f64 (exp10, x, y, cmp); +} + +#else + +# define SpecialOffset v_u64 (0x6000000000000000) /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +# define SpecialBias1 v_u64 (0x7000000000000000) /* 0x1p769. */ +# define SpecialBias2 v_u64 (0x3010000000000000) /* 0x1p-254. */ + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t s, float64x2_t y, float64x2_t n, + const struct data *d) +{ + /* 2^(n/N) may overflow, break it up into s1*s2. */ + uint64x2_t b = vandq_u64 (vcltzq_f64 (n), SpecialOffset); + float64x2_t s1 = vreinterpretq_f64_u64 (vsubq_u64 (SpecialBias1, b)); + float64x2_t s2 = vreinterpretq_f64_u64 ( + vaddq_u64 (vsubq_u64 (vreinterpretq_u64_f64 (s), SpecialBias2), b)); + uint64x2_t cmp = vcagtq_f64 (n, d->scale_thresh); + float64x2_t r1 = vmulq_f64 (s1, s1); + float64x2_t r0 = vmulq_f64 (vfmaq_f64 (s2, y, s2), s1); + return vbslq_f64 (cmp, r1, r0); +} + +#endif + +/* Fast vector implementation of exp10. + Maximum measured error is 1.64 ulp. + _ZGVnN2v_exp10(0x1.ccd1c9d82cc8cp+0) got 0x1.f8dab6d7fed0cp+5 + want 0x1.f8dab6d7fed0ap+5. */ +float64x2_t VPCS_ATTR V_NAME_D1 (exp10) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t cmp; +#if WANT_SIMD_EXCEPT + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special_case to fix special lanes later. This is only necessary if fenv + exceptions are to be triggered correctly. */ + float64x2_t xm = x; + uint64x2_t iax = vreinterpretq_u64_f64 (vabsq_f64 (x)); + cmp = vcgeq_u64 (vsubq_u64 (iax, TinyBound), Thres); + if (__glibc_unlikely (v_any_u64 (cmp))) + x = vbslq_f64 (cmp, v_f64 (1), x); +#else + cmp = vcageq_f64 (x, d->special_bound); +#endif + + /* n = round(x/(log10(2)/N)). */ + float64x2_t z = vfmaq_f64 (d->shift, x, d->log10_2); + uint64x2_t u = vreinterpretq_u64_f64 (z); + float64x2_t n = vsubq_f64 (z, d->shift); + + /* r = x - n*log10(2)/N. */ + float64x2_t r = x; + r = vfmsq_f64 (r, d->log2_10_hi, n); + r = vfmsq_f64 (r, d->log2_10_lo, n); + + uint64x2_t e = vshlq_n_u64 (u, 52 - V_EXP_TABLE_BITS); + uint64x2_t i = vandq_u64 (u, IndexMask); + + /* y = exp10(r) - 1 ~= C0 r + C1 r^2 + C2 r^3 + C3 r^4. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t p = vfmaq_f64 (d->poly[0], r, d->poly[1]); + float64x2_t y = vfmaq_f64 (d->poly[2], r, d->poly[3]); + p = vfmaq_f64 (p, y, r2); + y = vmulq_f64 (r, p); + + /* s = 2^(n/N). */ + u = v_lookup_u64 (__v_exp_data, i); + float64x2_t s = vreinterpretq_f64_u64 (vaddq_u64 (u, e)); + + if (__glibc_unlikely (v_any_u64 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f64 (s, y, s), cmp); +#else + return special_case (s, y, n, d); +#endif + + return vfmaq_f64 (s, y, s); +} diff --git a/sysdeps/aarch64/fpu/exp10_sve.c b/sysdeps/aarch64/fpu/exp10_sve.c new file mode 100644 index 0000000000..a8cef7b692 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10_sve.c @@ -0,0 +1,127 @@ +/* Double-precision vector (SVE) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define SpecialBound 307.0 /* floor (log10 (2^1023)). */ + +static const struct data +{ + double poly[5]; + double shift, log10_2, log2_10_hi, log2_10_lo, scale_thres, special_bound; +} data = { + /* Coefficients generated using Remez algorithm. + rel error: 0x1.9fcb9b3p-60 + abs error: 0x1.a20d9598p-60 in [ -log10(2)/128, log10(2)/128 ] + max ulp err 0.52 +0.5. */ + .poly = { 0x1.26bb1bbb55516p1, 0x1.53524c73cd32ap1, 0x1.0470591daeafbp1, + 0x1.2bd77b1361ef6p0, 0x1.142b5d54e9621p-1 }, + /* 1.5*2^46+1023. This value is further explained below. */ + .shift = 0x1.800000000ffc0p+46, + .log10_2 = 0x1.a934f0979a371p1, /* 1/log2(10). */ + .log2_10_hi = 0x1.34413509f79ffp-2, /* log2(10). */ + .log2_10_lo = -0x1.9dc1da994fd21p-59, + .scale_thres = 1280.0, + .special_bound = SpecialBound, +}; + +#define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +#define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +#define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +/* Update of both special and non-special cases, if any special case is + detected. */ +static inline svfloat64_t +special_case (svbool_t pg, svfloat64_t s, svfloat64_t y, svfloat64_t n, + const struct data *d) +{ + /* s=2^n may overflow, break it up into s=s1*s2, + such that exp = s + s*y can be computed as s1*(s2+s2*y) + and s1*s1 overflows only if n>0. */ + + /* If n<=0 then set b to 0x6, 0 otherwise. */ + svbool_t p_sign = svcmple (pg, n, 0.0); /* n <= 0. */ + svuint64_t b = svdup_u64_z (p_sign, SpecialOffset); + + /* Set s1 to generate overflow depending on sign of exponent n. */ + svfloat64_t s1 = svreinterpret_f64 (svsubr_x (pg, b, SpecialBias1)); + /* Offset s to avoid overflow in final result if n is below threshold. */ + svfloat64_t s2 = svreinterpret_f64 ( + svadd_x (pg, svsub_x (pg, svreinterpret_u64 (s), SpecialBias2), b)); + + /* |n| > 1280 => 2^(n) overflows. */ + svbool_t p_cmp = svacgt (pg, n, d->scale_thres); + + svfloat64_t r1 = svmul_x (pg, s1, s1); + svfloat64_t r2 = svmla_x (pg, s2, s2, y); + svfloat64_t r0 = svmul_x (pg, r2, s1); + + return svsel (p_cmp, r1, r0); +} + +/* Fast vector implementation of exp10 using FEXPA instruction. + Maximum measured error is 1.02 ulp. + SV_NAME_D1 (exp10)(-0x1.2862fec805e58p+2) got 0x1.885a89551d782p-16 + want 0x1.885a89551d781p-16. */ +svfloat64_t SV_NAME_D1 (exp10) (svfloat64_t x, svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svbool_t no_big_scale = svacle (pg, x, d->special_bound); + svbool_t special = svnot_z (pg, no_big_scale); + + /* n = round(x/(log10(2)/N)). */ + svfloat64_t shift = sv_f64 (d->shift); + svfloat64_t z = svmla_x (pg, shift, x, d->log10_2); + svfloat64_t n = svsub_x (pg, z, shift); + + /* r = x - n*log10(2)/N. */ + svfloat64_t log2_10 = svld1rq (svptrue_b64 (), &d->log2_10_hi); + svfloat64_t r = x; + r = svmls_lane (r, n, log2_10, 0); + r = svmls_lane (r, n, log2_10, 1); + + /* scale = 2^(n/N), computed using FEXPA. FEXPA does not propagate NaNs, so + for consistent NaN handling we have to manually propagate them. This + comes at significant performance cost. */ + svuint64_t u = svreinterpret_u64 (z); + svfloat64_t scale = svexpa (u); + + /* Approximate exp10(r) using polynomial. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = svmla_x (pg, svmul_x (pg, r, d->poly[0]), r2, + sv_pairwise_poly_3_f64_x (pg, r, r2, d->poly + 1)); + + /* Assemble result as exp10(x) = 2^n * exp10(r). If |x| > SpecialBound + multiplication may overflow, so use special case routine. */ + if (__glibc_unlikely (svptest_any (pg, special))) + { + /* FEXPA zeroes the sign bit, however the sign is meaningful to the + special case function so needs to be copied. + e = sign bit of u << 46. */ + svuint64_t e = svand_x (pg, svlsl_x (pg, u, 46), 0x8000000000000000); + /* Copy sign to scale. */ + scale = svreinterpret_f64 (svadd_x (pg, e, svreinterpret_u64 (scale))); + return special_case (pg, scale, y, n, d); + } + + /* No special case. */ + return svmla_x (pg, scale, scale, y); +} diff --git a/sysdeps/aarch64/fpu/exp10f_advsimd.c b/sysdeps/aarch64/fpu/exp10f_advsimd.c new file mode 100644 index 0000000000..9e754c46fa --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10f_advsimd.c @@ -0,0 +1,140 @@ +/* Single-precision vector (AdvSIMD) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +#define ScaleBound 192.0f + +static const struct data +{ + float32x4_t poly[5]; + float32x4_t shift, log10_2, log2_10_hi, log2_10_lo; +#if !WANT_SIMD_EXCEPT + float32x4_t scale_thresh; +#endif +} data = { + /* Coefficients generated using Remez algorithm with minimisation of relative + error. + rel error: 0x1.89dafa3p-24 + abs error: 0x1.167d55p-23 in [-log10(2)/2, log10(2)/2] + maxerr: 1.85943 +0.5 ulp. */ + .poly = { V4 (0x1.26bb16p+1f), V4 (0x1.5350d2p+1f), V4 (0x1.04744ap+1f), + V4 (0x1.2d8176p+0f), V4 (0x1.12b41ap-1f) }, + .shift = V4 (0x1.8p23f), + .log10_2 = V4 (0x1.a934fp+1), + .log2_10_hi = V4 (0x1.344136p-2), + .log2_10_lo = V4 (-0x1.ec10cp-27), +#if !WANT_SIMD_EXCEPT + .scale_thresh = V4 (ScaleBound) +#endif +}; + +#define ExponentBias v_u32 (0x3f800000) + +#if WANT_SIMD_EXCEPT + +# define SpecialBound 38.0f /* rint(log10(2^127)). */ +# define TinyBound v_u32 (0x20000000) /* asuint (0x1p-63). */ +# define BigBound v_u32 (0x42180000) /* asuint (SpecialBound). */ +# define Thres v_u32 (0x22180000) /* BigBound - TinyBound. */ + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine to special lanes. */ + return v_call_f32 (exp10f, x, y, cmp); +} + +#else + +# define SpecialBound 126.0f /* rint (log2 (2^127 / (1 + sqrt (2)))). */ +# define SpecialOffset v_u32 (0x82000000) +# define SpecialBias v_u32 (0x7f000000) + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t poly, float32x4_t n, uint32x4_t e, uint32x4_t cmp1, + float32x4_t scale, const struct data *d) +{ + /* 2^n may overflow, break it up into s1*s2. */ + uint32x4_t b = vandq_u32 (vclezq_f32 (n), SpecialOffset); + float32x4_t s1 = vreinterpretq_f32_u32 (vaddq_u32 (b, SpecialBias)); + float32x4_t s2 = vreinterpretq_f32_u32 (vsubq_u32 (e, b)); + uint32x4_t cmp2 = vcagtq_f32 (n, d->scale_thresh); + float32x4_t r2 = vmulq_f32 (s1, s1); + float32x4_t r1 = vmulq_f32 (vfmaq_f32 (s2, poly, s2), s1); + /* Similar to r1 but avoids double rounding in the subnormal range. */ + float32x4_t r0 = vfmaq_f32 (scale, poly, scale); + float32x4_t r = vbslq_f32 (cmp1, r1, r0); + return vbslq_f32 (cmp2, r2, r); +} + +#endif + +/* Fast vector implementation of single-precision exp10. + Algorithm is accurate to 2.36 ULP. + _ZGVnN4v_exp10f(0x1.be2b36p+1) got 0x1.7e79c4p+11 + want 0x1.7e79cp+11. */ +float32x4_t VPCS_ATTR V_NAME_F1 (exp10) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); +#if WANT_SIMD_EXCEPT + /* asuint(x) - TinyBound >= BigBound - TinyBound. */ + uint32x4_t cmp = vcgeq_u32 ( + vsubq_u32 (vandq_u32 (vreinterpretq_u32_f32 (x), v_u32 (0x7fffffff)), + TinyBound), + Thres); + float32x4_t xm = x; + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special case handler to fix special lanes later. This is only necessary if + fenv exceptions are to be triggered correctly. */ + if (__glibc_unlikely (v_any_u32 (cmp))) + x = vbslq_f32 (cmp, v_f32 (1), x); +#endif + + /* exp10(x) = 2^n * 10^r = 2^n * (1 + poly (r)), + with poly(r) in [1/sqrt(2), sqrt(2)] and + x = r + n * log10 (2), with r in [-log10(2)/2, log10(2)/2]. */ + float32x4_t z = vfmaq_f32 (d->shift, x, d->log10_2); + float32x4_t n = vsubq_f32 (z, d->shift); + float32x4_t r = vfmsq_f32 (x, n, d->log2_10_hi); + r = vfmsq_f32 (r, n, d->log2_10_lo); + uint32x4_t e = vshlq_n_u32 (vreinterpretq_u32_f32 (z), 23); + + float32x4_t scale = vreinterpretq_f32_u32 (vaddq_u32 (e, ExponentBias)); + +#if !WANT_SIMD_EXCEPT + uint32x4_t cmp = vcagtq_f32 (n, v_f32 (SpecialBound)); +#endif + + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t poly + = vfmaq_f32 (vmulq_f32 (r, d->poly[0]), + v_pairwise_poly_3_f32 (r, r2, d->poly + 1), r2); + + if (__glibc_unlikely (v_any_u32 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f32 (scale, poly, scale), cmp); +#else + return special_case (poly, n, e, cmp, scale, d); +#endif + + return vfmaq_f32 (scale, poly, scale); +} diff --git a/sysdeps/aarch64/fpu/exp10f_sve.c b/sysdeps/aarch64/fpu/exp10f_sve.c new file mode 100644 index 0000000000..3df3246e05 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10f_sve.c @@ -0,0 +1,91 @@ +/* Single-precision vector (SVE) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f32.h" + +/* For x < -SpecialBound, the result is subnormal and not handled correctly by + FEXPA. */ +#define SpecialBound 37.9 + +static const struct data +{ + float poly[5]; + float shift, log10_2, log2_10_hi, log2_10_lo, special_bound; +} data = { + /* Coefficients generated using Remez algorithm with minimisation of relative + error. + rel error: 0x1.89dafa3p-24 + abs error: 0x1.167d55p-23 in [-log10(2)/2, log10(2)/2] + maxerr: 0.52 +0.5 ulp. */ + .poly = { 0x1.26bb16p+1f, 0x1.5350d2p+1f, 0x1.04744ap+1f, 0x1.2d8176p+0f, + 0x1.12b41ap-1f }, + /* 1.5*2^17 + 127, a shift value suitable for FEXPA. */ + .shift = 0x1.903f8p17f, + .log10_2 = 0x1.a934fp+1, + .log2_10_hi = 0x1.344136p-2, + .log2_10_lo = -0x1.ec10cp-27, + .special_bound = SpecialBound, +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (exp10f, x, y, special); +} + +/* Single-precision SVE exp10f routine. Implements the same algorithm + as AdvSIMD exp10f. + Worst case error is 1.02 ULPs. + _ZGVsMxv_exp10f(-0x1.040488p-4) got 0x1.ba5f9ep-1 + want 0x1.ba5f9cp-1. */ +svfloat32_t SV_NAME_F1 (exp10) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + /* exp10(x) = 2^(n/N) * 10^r = 2^n * (1 + poly (r)), + with poly(r) in [1/sqrt(2), sqrt(2)] and + x = r + n * log10(2) / N, with r in [-log10(2)/2N, log10(2)/2N]. */ + + /* Load some constants in quad-word chunks to minimise memory access (last + lane is wasted). */ + svfloat32_t log10_2_and_inv = svld1rq (svptrue_b32 (), &d->log10_2); + + /* n = round(x/(log10(2)/N)). */ + svfloat32_t shift = sv_f32 (d->shift); + svfloat32_t z = svmla_lane (shift, x, log10_2_and_inv, 0); + svfloat32_t n = svsub_x (pg, z, shift); + + /* r = x - n*log10(2)/N. */ + svfloat32_t r = svmls_lane (x, n, log10_2_and_inv, 1); + r = svmls_lane (r, n, log10_2_and_inv, 2); + + svbool_t special = svacgt (pg, x, d->special_bound); + svfloat32_t scale = svexpa (svreinterpret_u32 (z)); + + /* Polynomial evaluation: poly(r) ~ exp10(r)-1. */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t poly + = svmla_x (pg, svmul_x (pg, r, d->poly[0]), + sv_pairwise_poly_3_f32_x (pg, r, r2, d->poly + 1), r2); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (pg, scale, scale, poly), special); + + return svmla_x (pg, scale, scale, poly); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index 8d05498ec9..26d5ecf66f 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) +VPCS_VECTOR_WRAPPER (exp10_advsimd, _ZGVnN2v_exp10) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (log10_advsimd, _ZGVnN2v_log10) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index b65bc6f1e6..86efd60779 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) +SVE_VECTOR_WRAPPER (exp10_sve, _ZGVsMxv_exp10) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (log10_sve, _ZGVsMxv_log10) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 6ced0d4488..8f7ebea1ac 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) +VPCS_VECTOR_WRAPPER (exp10f_advsimd, _ZGVnN4v_exp10f) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (log10f_advsimd, _ZGVnN4v_log10f) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index 2ed8d0659a..885e58ac39 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) +SVE_VECTOR_WRAPPER (exp10f_sve, _ZGVsMxv_exp10f) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (log10f_sve, _ZGVsMxv_log10f) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 6641c7fa0b..d117209c06 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -970,11 +970,19 @@ double: 2 float: 1 ldouble: 2 +Function: "exp10_advsimd": +double: 1 +float: 2 + Function: "exp10_downward": double: 2 float: 1 ldouble: 3 +Function: "exp10_sve": +double: 1 +float: 1 + Function: "exp10_towardzero": double: 2 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index f8776a6bea..cad774521a 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,14 +14,18 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_exp10 F GLIBC_2.39 _ZGVnN2v_exp2 F GLIBC_2.39 _ZGVnN2v_log10 F GLIBC_2.39 _ZGVnN2v_log2 F GLIBC_2.39 _ZGVnN2v_tan F +GLIBC_2.39 _ZGVnN4v_exp10f F GLIBC_2.39 _ZGVnN4v_exp2f F GLIBC_2.39 _ZGVnN4v_log10f F GLIBC_2.39 _ZGVnN4v_log2f F GLIBC_2.39 _ZGVnN4v_tanf F +GLIBC_2.39 _ZGVsMxv_exp10 F +GLIBC_2.39 _ZGVsMxv_exp10f F GLIBC_2.39 _ZGVsMxv_exp2 F GLIBC_2.39 _ZGVsMxv_exp2f F GLIBC_2.39 _ZGVsMxv_log10 F