From patchwork Fri Dec 3 00:00:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 48440 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 26E5B3857C59 for ; Fri, 3 Dec 2021 00:03:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 26E5B3857C59 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1638489823; bh=JeRGAiF/AZkTIssuXcQM8q6M0tLNbw1qPOwQrq6SBOY=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=SmvvNRDpRS4Uq9HrwvDvS4oAcZkzGFBdI6wcm9UE/t2dWkPvhKFYCB1YcbjYNPFzK lFYoeQt5ge3HyF2Ub4DxJyoGjgLcPwR1ynm6a1rCJXxyZsDmCZcxi4WjP9ql8V1Mnt j+dp/tHC/wuXsvQ8gbMYn0RWfJf/h2iSLfQaebVE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com [IPv6:2607:f8b0:4864:20::f34]) by sourceware.org (Postfix) with ESMTPS id 51E53385AC3E for ; Fri, 3 Dec 2021 00:01:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 51E53385AC3E Received: by mail-qv1-xf34.google.com with SMTP id i12so1123026qvh.11 for ; Thu, 02 Dec 2021 16:01:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JeRGAiF/AZkTIssuXcQM8q6M0tLNbw1qPOwQrq6SBOY=; b=K90urPp2JA9n35dwaFLhH8LB1jf6T2jDvdob53Gsmk7qC9mQZ6dIKwszGHGFpKXY0c ft+LMpk13Uq5wzXjEBqA4JeI/J99nzUMzbVGGLPHb9Kraw4j22PZFa8sIYaGFBspxQ6Z p0qVBteO7Qq0umUxJmoHO/OxfvIEyBNsM9cuAhUcUJ4LEAbl0hDitxhukDox5pN6dpOb OAM367PTFf1BXkas0xiuGAbIzqGp+Tr8J5rO74Te4VCCJVJSQVu8BoSFeUHBWQnT9HVZ +w3resWiGiz6Zj0ZOQI6HezsPBXCgc7MAVyXcSxwhTuautlyTJK0pO+CKEIebNplNILR Vy3g== X-Gm-Message-State: AOAM530xPQMI0B3VRBzE6waJsjXOnPvjNtBUoXIppd3laxqgkZCipPFg SpYcXP0UTFDLiYZmoR776vxVHh4oFWqipQ== X-Google-Smtp-Source: ABdhPJy2R0SIzqRYeHI49BQ+SuNZ8AingcVmJgPWGZMzXUJKw9lRVAlvEOcHpt0TGYcynjo64I0ieQ== X-Received: by 2002:a05:6214:238e:: with SMTP id fw14mr16512623qvb.86.1638489673717; Thu, 02 Dec 2021 16:01:13 -0800 (PST) Received: from birita.. ([2804:431:c7cb:30f8:3030:59d3:d31c:ed39]) by smtp.gmail.com with ESMTPSA id m9sm938714qkn.59.2021.12.02.16.01.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Dec 2021 16:01:13 -0800 (PST) To: libc-alpha@sourceware.org, Paul Zimmermann , Wilco Dijkstra Subject: [PATCH v4 03/12] math: Improve hypot performance with FMA Date: Thu, 2 Dec 2021 21:00:54 -0300 Message-Id: <20211203000103.737833-4-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211203000103.737833-1-adhemerval.zanella@linaro.org> References: <20211203000103.737833-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" From: Wilco Dijkstra Improve hypot performance significantly by using fma when available. The fma version has twice the throughput of the previous version and 70% of the latency. The non-fma version has 30% higher throughput and 10% higher latency. Max ULP error is 0.949 with fma and 0.792 without fma. Passes GLIBC testsuite. --- sysdeps/ieee754/dbl-64/e_hypot.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/sysdeps/ieee754/dbl-64/e_hypot.c b/sysdeps/ieee754/dbl-64/e_hypot.c index 274b14b57e..f53061badc 100644 --- a/sysdeps/ieee754/dbl-64/e_hypot.c +++ b/sysdeps/ieee754/dbl-64/e_hypot.c @@ -26,7 +26,11 @@ rounding mode. - Handle required underflow exception for subnormal results. - The expected ULP is ~0.792. + The expected ULP is ~0.792 or ~0.948 if FMA is used. For FMA, the + correction is not used and the error of sqrt (x^2 + y^2) is below 1 ULP + if x^2 + y^2 is computed with less than 0.707 ULP error. If |x| >= |2y|, + fma (x, x, y^2) has ~0.625 ULP. If |x| < |2y|, fma (|2x|, |y|, (x - y)^2) + has ~0.625 ULP. [1] https://arxiv.org/pdf/1904.09481.pdf */ @@ -48,6 +52,16 @@ static inline double kernel (double ax, double ay) { double t1, t2; +#ifdef __FP_FAST_FMA + t1 = ay + ay; + t2 = ax - ay; + + if (t1 >= ax) + return sqrt (fma (t1, ax, t2 * t2)); + else + return sqrt (fma (ax, ax, ay * ay)); + +#else double h = sqrt (ax * ax + ay * ay); if (h <= 2.0 * ay) { @@ -64,6 +78,7 @@ kernel (double ax, double ay) h -= (t1 + t2) / (2.0 * h); return h; +#endif } double