From patchwork Tue Dec 7 19:03:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 48598 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 40A563858401 for ; Tue, 7 Dec 2021 19:06:33 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 40A563858401 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1638903993; bh=GnRH8idW43+aZwTJUlRlnDVl0ZbtruzkWZOdP6wT/qA=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=xAxWt0pRQDEE2LU49htDvRQx101tB6qz+YFL3xbfpnTBzastfSIv1If7HB6Mb/GeK GZwEcho2rD3JSHR72UViryVFqRyNIXERm2hoKmSk8GvmsdCM0kwD0mcx+EFzvHjJs0 Gc9eKW4kAa37y7rV9WNaWRfIaBuqBgE5SenS/SPU= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-ua1-x92c.google.com (mail-ua1-x92c.google.com [IPv6:2607:f8b0:4864:20::92c]) by sourceware.org (Postfix) with ESMTPS id 173633858001 for ; Tue, 7 Dec 2021 19:04:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 173633858001 Received: by mail-ua1-x92c.google.com with SMTP id r15so356819uao.3 for ; Tue, 07 Dec 2021 11:04:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GnRH8idW43+aZwTJUlRlnDVl0ZbtruzkWZOdP6wT/qA=; b=xYYUWfeB0eOGJafJ2/b1xhwpt9Dc0uFnT6M0T9QdzGlqIojXe7effb2KjN/vzex3u7 bdHcXvKsPlWhf5mUGjVGviQvd1Ggrt/GDNkFPPnxKIPyGiGmeiiZXw8P373WS5/PY9B7 ohI2SF5mhaFotkzxB8vtxvntmbo/9IHUZs2Vx4VRRioRvdVQ20hiagA/YOfRwZ9zrhGl r0tOl0WUMNH2/rGwLkuL2e/yrPCkdQ96Z/8o5T6uIw/ks+zyC1WVM5h19FY0/T929842 KqgQIUerSSDc5jW4CqZsShrI46m6sCDQZNlXChEbQM3j7qtad5+SH62YepSEUQmiSFPz b13g== X-Gm-Message-State: AOAM533/s0G6NSgLDS99DMlUqu+zm0js8krFbeXAuABMCGLYbUvVaSDo 41h3VIgf/buo3IBbhpWaORGz7u2+X3h50w== X-Google-Smtp-Source: ABdhPJwKP/VrwVTa3LQdWWq/XwDAJOocJ1riIey3nOCod08/uW4MAdpbg8YSYWn+dtU8J5UizdcUoQ== X-Received: by 2002:a05:6102:94b:: with SMTP id a11mr47518805vsi.39.1638903869559; Tue, 07 Dec 2021 11:04:29 -0800 (PST) Received: from birita.. ([2804:431:c7ca:a776:246c:70fd:1377:eec7]) by smtp.gmail.com with ESMTPSA id l190sm312904vsc.26.2021.12.07.11.04.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Dec 2021 11:04:29 -0800 (PST) To: libc-alpha@sourceware.org, Paul Zimmermann , Wilco Dijkstra Subject: [PATCH v5 03/12] math: Improve hypot performance with FMA Date: Tue, 7 Dec 2021 16:03:44 -0300 Message-Id: <20211207190353.3282666-4-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211207190353.3282666-1-adhemerval.zanella@linaro.org> References: <20211207190353.3282666-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" From: Wilco Dijkstra Improve hypot performance significantly by using fma when available. The fma version has twice the throughput of the previous version and 70% of the latency. The non-fma version has 30% higher throughput and 10% higher latency. Max ULP error is 0.949 with fma and 0.792 without fma. Passes GLIBC testsuite. --- sysdeps/ieee754/dbl-64/e_hypot.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/sysdeps/ieee754/dbl-64/e_hypot.c b/sysdeps/ieee754/dbl-64/e_hypot.c index 75bce2df4e..6fedf0d61f 100644 --- a/sysdeps/ieee754/dbl-64/e_hypot.c +++ b/sysdeps/ieee754/dbl-64/e_hypot.c @@ -26,7 +26,11 @@ rounding mode. - Handle required underflow exception for subnormal results. - The expected ULP is ~0.792. + The expected ULP is ~0.792 or ~0.948 if FMA is used. For FMA, the + correction is not used and the error of sqrt (x^2 + y^2) is below 1 ULP + if x^2 + y^2 is computed with less than 0.707 ULP error. If |x| >= |2y|, + fma (x, x, y^2) has ~0.625 ULP. If |x| < |2y|, fma (|2x|, |y|, (x - y)^2) + has ~0.625 ULP. [1] https://arxiv.org/pdf/1904.09481.pdf */ @@ -48,6 +52,16 @@ static inline double kernel (double ax, double ay) { double t1, t2; +#ifdef __FP_FAST_FMA + t1 = ay + ay; + t2 = ax - ay; + + if (t1 >= ax) + return sqrt (fma (t1, ax, t2 * t2)); + else + return sqrt (fma (ax, ax, ay * ay)); + +#else double h = sqrt (ax * ax + ay * ay); if (h <= 2.0 * ay) { @@ -64,6 +78,7 @@ kernel (double ax, double ay) h -= (t1 + t2) / (2.0 * h); return h; +#endif } double