From patchwork Wed Oct 6 18:05:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 45935 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D83523858425 for ; Wed, 6 Oct 2021 18:15:54 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D83523858425 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1633544154; bh=2SDYG2kqLihPPrTxRYAo33e6WKOI9PW3aFMqRFMXWRc=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=b4U6Q4HneoM11oSsbrbcnJYamQMD2AFX1XiyNqH9YV1Oni99pgH1b5PV58uEgYQV2 nAVTbJYZYvtjc4cYsa7z8ZlzgO4xeIrzCR7Lu0gKo+8JbcB1EjM/Ouik9Q8t4XjpBR bLgT4P9WZGtSW+pWYrJ8KBgeqXYSV0XX1N+ON4fo= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) by sourceware.org (Postfix) with ESMTPS id 6D7E53858C39 for ; Wed, 6 Oct 2021 18:15:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6D7E53858C39 Received: by mail-qk1-x730.google.com with SMTP id p4so3471228qki.3 for ; Wed, 06 Oct 2021 11:15:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2SDYG2kqLihPPrTxRYAo33e6WKOI9PW3aFMqRFMXWRc=; b=g5u0pgJGiad11Sr91KYuzPecFJdqmbe/RbOWuA+qgHla694LkGOvEZ1xSPbvOs4TFs 98hnCKUPoBIy+qE7HKNUZAPv+ApKvblCXsPYZYHIqzKfxrLUFF3EWGsuc12mMBUSsjTO IbOyRk5hg4CvGonWKbbJKfzFqesVwIba3D5byOAu4RC1GURQ8BlIB4+bGnTJdXrMvxS2 uYqsqpbtp5C5c9HuSCevcrxQMMyUM7a+GzMTIEtmt9nO9GQdVGvW4BBZ6d2sZ6dFMC0d KzxIxXN41unY8znTb2bxVTM7ZaD1ugfveyS0pztddk35hyNJ5sfl/jOxcEjbDGEYZjes mTgg== X-Gm-Message-State: AOAM5314BUBSj4BePX9A5YajRnEfbYaW/HMCMjANOEyxfuQ54qCIfLNr 0dgw0KR8OjFQ2mE3AfX9x9Hwj123Jkzy9Q== X-Google-Smtp-Source: ABdhPJypdBR0KDCH1ffxM28qXiFAm0Hm7PR9aVci8aWDNWkvpBCuEEh82yd9N0J3LZ1VYfFY2oa3lA== X-Received: by 2002:ae9:ef97:: with SMTP id d145mr134125qkg.293.1633544131636; Wed, 06 Oct 2021 11:15:31 -0700 (PDT) Received: from birita.. ([2804:431:c7cb:807a:2ebe:4b13:27bd:f11d]) by smtp.gmail.com with ESMTPSA id 18sm14161168qtz.49.2021.10.06.11.15.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Oct 2021 11:15:31 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH 4/9] math: Use an improved algorithm for hypot (dbl-64) Date: Wed, 6 Oct 2021 15:05:52 -0300 Message-Id: <20211006180557.933826-5-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211006180557.933826-1-adhemerval.zanella@linaro.org> References: <20211006180557.933826-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This implementation is based on the 'An Improved Algorithm for hypot(a,b)' by Carlos F. Borges [1] using the MyHypot3 with the following changes: - Handle qNaN and sNaN. - Tune the 'widely varying operands' to avoid spurious underflow due the multiplication and fix the return value for upwards rounding mode. - Handle required underflow exception for denormal results. The main advantage of the new algorithm is its precision: with a random 1e9 input pairs in the range of [DBL_MIN, DBL_MAX], glibc current implementation shows around 0.34% results with an error of 1 ulp (3424869 results) while the new implementation only shows 0.002% of total (18851). The performance result are also only slight worse than current implementation. On x86_64 (Ryzen 5900X) with gcc 10.3.1: Before: "hypot": { "workload-random": { "duration": 3.71687e+09, "iterations": 2.08e+08, "reciprocal-throughput": 15.0308, "latency": 20.7083, "max-throughput": 6.65299e+07, "min-throughput": 4.82897e+07 } After: "hypot": { "workload-random": { "duration": 3.70911e+09, "iterations": 2.44e+08, "reciprocal-throughput": 15.0929, "latency": 15.3097, "max-throughput": 6.62565e+07, "min-throughput": 6.53181e+07 } } Co-Authored-By: Paul Zimmermann Checked on x86_64-linux-gnu and aarch64-linux-gnu. [1] https://arxiv.org/pdf/1904.09481.pdf --- sysdeps/ieee754/dbl-64/e_hypot.c | 225 ++++++++++++------------------- 1 file changed, 86 insertions(+), 139 deletions(-) diff --git a/sysdeps/ieee754/dbl-64/e_hypot.c b/sysdeps/ieee754/dbl-64/e_hypot.c index 9ec4c1ced0..231fb0d70f 100644 --- a/sysdeps/ieee754/dbl-64/e_hypot.c +++ b/sysdeps/ieee754/dbl-64/e_hypot.c @@ -1,164 +1,111 @@ -/* @(#)e_hypot.c 5.1 93/09/24 */ -/* - * ==================================================== - * Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved. - * - * Developed at SunPro, a Sun Microsystems, Inc. business. - * Permission to use, copy, modify, and distribute this - * software is freely granted, provided that this notice - * is preserved. - * ==================================================== - */ +/* Euclidean distance function. Double/Binary64 version. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. -/* __ieee754_hypot(x,y) - * - * Method : - * If (assume round-to-nearest) z=x*x+y*y - * has error less than sqrt(2)/2 ulp, than - * sqrt(z) has error less than 1 ulp (exercise). - * - * So, compute sqrt(x*x+y*y) with some care as - * follows to get the error below 1 ulp: - * - * Assume x>y>0; - * (if possible, set rounding to round-to-nearest) - * 1. if x > 2y use - * x1*x1+(y*y+(x2*(x+x1))) for x*x+y*y - * where x1 = x with lower 32 bits cleared, x2 = x-x1; else - * 2. if x <= 2y use - * t1*y1+((x-y)*(x-y)+(t1*y2+t2*y)) - * where t1 = 2x with lower 32 bits cleared, t2 = 2x-t1, - * y1= y with lower 32 bits chopped, y2 = y-y1. - * - * NOTE: scaling may be necessary if some argument is too - * large or too tiny - * - * Special cases: - * hypot(x,y) is INF if x or y is +INF or -INF; else - * hypot(x,y) is NAN if x or y is NAN. - * - * Accuracy: - * hypot(x,y) returns sqrt(x^2+y^2) with error less - * than 1 ulps (units in the last place) - */ + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* This implementation is based on 'An Improved Algorithm for hypot(a,b)' by + Carlos F. Borges [1] using the MyHypot3 with the following changes: + + - Handle qNaN and sNaN. + - Tune the 'widely varying operands' to avoid spurious underflow + due the multiplication and fix the return value for upwards + rounding mode. + - Handle required underflow exception for subnormal results. + + [1] https://arxiv.org/pdf/1904.09481.pdf */ #include #include #include +#include #include +#include + +/* sqrt (DBL_EPSILON / 2.0) */ +#define SQRT_EPS_DIV_2 0x1.6a09e667f3bcdp-27 +/* DBL_MIN / (sqrt (DBL_EPSILON / 2.0)) */ +#define DBL_MIN_THRESHOLD 0x1.6a09e667f3bcdp-996 +/* eps (double) * sqrt (DBL_MIN)) */ +#define SCALE 0x1p-563 +/* 1 / eps (sqrt (DBL_MIN) */ +#define INV_SCALE 0x1p+563 +/* sqrt (DBL_MAX) */ +#define SQRT_DBL_MAX 0x1.6a09e667f3bccp+511 +/* sqrt (DBL_MIN) */ +#define SQRT_DBL_MIN 0x1p-511 double __ieee754_hypot (double x, double y) { - double a, b, t1, t2, y1, y2, w; - int32_t j, k, ha, hb; + if ((isinf (x) || isinf (y)) + && !issignaling (x) && !issignaling (y)) + return INFINITY; + if (isnan (x) || isnan (y)) + return x + y; - GET_HIGH_WORD (ha, x); - ha &= 0x7fffffff; - GET_HIGH_WORD (hb, y); - hb &= 0x7fffffff; - if (hb > ha) - { - a = y; b = x; j = ha; ha = hb; hb = j; - } - else + double ax = fabs (x); + double ay = fabs (y); + if (ay > ax) { - a = x; b = y; + double tmp = ax; + ax = ay; + ay = tmp; } - SET_HIGH_WORD (a, ha); /* a <- |a| */ - SET_HIGH_WORD (b, hb); /* b <- |b| */ - if ((ha - hb) > 0x3c00000) - { - return a + b; - } /* x/y > 2**60 */ - k = 0; - if (__glibc_unlikely (ha > 0x5f300000)) /* a>2**500 */ - { - if (ha >= 0x7ff00000) /* Inf or NaN */ - { - uint32_t low; - w = a + b; /* for sNaN */ - if (issignaling (a) || issignaling (b)) - return w; - GET_LOW_WORD (low, a); - if (((ha & 0xfffff) | low) == 0) - w = a; - GET_LOW_WORD (low, b); - if (((hb ^ 0x7ff00000) | low) == 0) - w = b; - return w; - } - /* scale a and b by 2**-600 */ - ha -= 0x25800000; hb -= 0x25800000; k += 600; - SET_HIGH_WORD (a, ha); - SET_HIGH_WORD (b, hb); - } - if (__builtin_expect (hb < 0x23d00000, 0)) /* b < 2**-450 */ + + /* Widely varying operands. The DBL_MIN_THRESHOLD check is used to avoid + a spurious underflow from the multiplication. */ + if (ax >= DBL_MIN_THRESHOLD && ay <= ax * SQRT_EPS_DIV_2) + return (ay == 0.0) ? ax : math_narrow_eval (ax + DBL_TRUE_MIN); + + double scale = SCALE; + if (ax > SQRT_DBL_MAX) { - if (hb <= 0x000fffff) /* subnormal b or 0 */ - { - uint32_t low; - GET_LOW_WORD (low, b); - if ((hb | low) == 0) - return a; - t1 = 0; - SET_HIGH_WORD (t1, 0x7fd00000); /* t1=2^1022 */ - b *= t1; - a *= t1; - k -= 1022; - GET_HIGH_WORD (ha, a); - GET_HIGH_WORD (hb, b); - if (hb > ha) - { - t1 = a; - a = b; - b = t1; - j = ha; - ha = hb; - hb = j; - } - } - else /* scale a and b by 2^600 */ - { - ha += 0x25800000; /* a *= 2^600 */ - hb += 0x25800000; /* b *= 2^600 */ - k -= 600; - SET_HIGH_WORD (a, ha); - SET_HIGH_WORD (b, hb); - } + ax *= scale; + ay *= scale; + scale = INV_SCALE; } - /* medium size a and b */ - w = a - b; - if (w > b) + else if (ay < SQRT_DBL_MIN) { - t1 = 0; - SET_HIGH_WORD (t1, ha); - t2 = a - t1; - w = sqrt (t1 * t1 - (b * (-b) - t2 * (a + t1))); + ax /= scale; + ay /= scale; } else + scale = 1.0; + + double h = sqrt (ax * ax + ay * ay); + + double t1, t2; + if (h == 0.0) + return h; + else if (h <= 2.0 * ay) { - a = a + a; - y1 = 0; - SET_HIGH_WORD (y1, hb); - y2 = b - y1; - t1 = 0; - SET_HIGH_WORD (t1, ha + 0x00100000); - t2 = a - t1; - w = sqrt (t1 * y1 - (w * (-w) - (t1 * y2 + t2 * b))); + double delta = h - ay; + t1 = ax * (2.0 * delta - ax); + t2 = (delta - 2.0 * (ax - ay)) * delta; } - if (k != 0) + else { - uint32_t high; - t1 = 1.0; - GET_HIGH_WORD (high, t1); - SET_HIGH_WORD (t1, high + (k << 20)); - w *= t1; - math_check_force_underflow_nonneg (w); - return w; + double delta = h - ax; + t1 = 2.0 * delta * (ax - 2 * ay); + t2 = (4.0 * delta - ay) * ay + delta * delta; } - else - return w; + h -= (t1 + t2) / (2.0 * h); + h = math_narrow_eval (h * scale); + math_check_force_underflow_nonneg (h); + return h; } #ifndef __ieee754_hypot libm_alias_finite (__ieee754_hypot, __hypot)