From patchwork Tue Jul 12 18:48:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 55977 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 27C1A385AC3E for ; Tue, 12 Jul 2022 18:48:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 27C1A385AC3E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1657651709; bh=C2zmTwCI5dRpU9KtcOTyXVdFP2y5NFu6dNGmVCwVO8U=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=qjptUYNMrZsUUImvZOCd2ehrKbqDW9y9J9WQ29Xypj7n/SwbX5cdK0Hb0JnkltPIh /q4n/PJskuuJa99oZixYCNa07DQ5FyBOt9S+JZHBau0Cr/ZgNMPvgEwIh31t4YELcH kNGX76HP8B1hChePuq3H3HNPiyXWUSlkmq9bh2aU= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pg1-x535.google.com (mail-pg1-x535.google.com [IPv6:2607:f8b0:4864:20::535]) by sourceware.org (Postfix) with ESMTPS id 8DD5D3858C50 for ; Tue, 12 Jul 2022 18:48:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8DD5D3858C50 Received: by mail-pg1-x535.google.com with SMTP id p11so1619300pgr.12 for ; Tue, 12 Jul 2022 11:48:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C2zmTwCI5dRpU9KtcOTyXVdFP2y5NFu6dNGmVCwVO8U=; b=m4fDWq7RU37uGnEe+0O2gOmYSe+4xeL2r79cE1TbivAl2V5N6ZKpGtCBMay1M6E8KK 0k8+eID+d6HoxoNj4vsdGPF0Zhuy5c+oqxI0sBCLx11s0g11jiIt39TqGTepuWrZM/vq BTDZJ4a0JheFa5fYouIxHbhGjBspEu2Rop6R+2udBzNC+F3TLV57J0LbxG72cUCosoJk gHkz70xV2yT9EqzRjNEtMlUFEknwGJpBfCP/M3sFrD4xpSNETbv525VW9EOu4uwxKfj+ zV3AsvpFTaiscsCV7m8b4kt+lehftHN7xV7IF8Rqu9TEZIAD98vpKVhkUDJ2e3TlarvE jpNg== X-Gm-Message-State: AJIora84ydBGUk9s4XMnTJPg1pS9/VLL4fuXM+MEfI73iCMVHrNIlYD8 WT8lKO2V0l1La5B0chXoYXGt8UE/MbY= X-Google-Smtp-Source: AGRyM1s6h9y/DWoovq7XeCKbu55a0j2/WQ9rfO9icgCUqT057vHBHLOh+UzCz6Vkudg7mYYYef3Yow== X-Received: by 2002:aa7:8ec2:0:b0:52a:d4f5:20b with SMTP id b2-20020aa78ec2000000b0052ad4f5020bmr10015676pfr.71.1657651687464; Tue, 12 Jul 2022 11:48:07 -0700 (PDT) Received: from noah-tgl.. ([192.55.60.37]) by smtp.gmail.com with ESMTPSA id u7-20020a170903124700b0016bee668a5asm7230886plh.161.2022.07.12.11.48.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Jul 2022 11:48:07 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v3] x86: Remove __mmask intrinsics in strstr-avx512.c Date: Tue, 12 Jul 2022 11:48:04 -0700 Message-Id: <20220712184804.2374822-1-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220711220730.1968923-1-goldstein.w.n@gmail.com> References: <20220711220730.1968923-1-goldstein.w.n@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" The intrinsics are not available before GCC7 and using standard operators generates code of equivilent or better quality. Removed: _cvtmask64_u64 _kshiftri_mask64 _kand_mask64 Geometric Mean of 5 Runs of Full Benchmark Suite New / Old: 0.958 --- sysdeps/x86_64/multiarch/strstr-avx512.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/sysdeps/x86_64/multiarch/strstr-avx512.c b/sysdeps/x86_64/multiarch/strstr-avx512.c index 2ab9e96db8..e44c1a05dc 100644 --- a/sysdeps/x86_64/multiarch/strstr-avx512.c +++ b/sysdeps/x86_64/multiarch/strstr-avx512.c @@ -26,6 +26,10 @@ #define ZMM_SIZE_IN_BYTES 64 #define PAGESIZE 4096 +#define cvtmask64_u64(...) (uint64_t) (__VA_ARGS__) +#define kshiftri_mask64(x, y) ((x) >> (y)) +#define kand_mask64(x, y) ((x) & (y)) + /* Returns the index of the first edge within the needle, returns 0 if no edge is found. Example: 'ab' is the first edge in 'aaaaaaaaaabaarddg' @@ -133,15 +137,15 @@ __strstr_avx512 (const char *haystack, const char *ned) __m512i hay0 = _mm512_maskz_loadu_epi8 (loadmask, haystack + hay_index); /* Search for NULL and compare only till null char */ uint64_t nullmask - = _cvtmask64_u64 (_mm512_mask_testn_epi8_mask (loadmask, hay0, hay0)); + = cvtmask64_u64 (_mm512_mask_testn_epi8_mask (loadmask, hay0, hay0)); uint64_t cmpmask = nullmask ^ (nullmask - ONE_64BIT); - cmpmask = cmpmask & _cvtmask64_u64 (loadmask); + cmpmask = cmpmask & cvtmask64_u64 (loadmask); /* Search for the 2 charaters of needle */ __mmask64 k0 = _mm512_cmpeq_epi8_mask (hay0, ned0); __mmask64 k1 = _mm512_cmpeq_epi8_mask (hay0, ned1); - k1 = _kshiftri_mask64 (k1, 1); + k1 = kshiftri_mask64 (k1, 1); /* k2 masks tell us if both chars from needle match */ - uint64_t k2 = _cvtmask64_u64 (_kand_mask64 (k0, k1)) & cmpmask; + uint64_t k2 = cvtmask64_u64 (kand_mask64 (k0, k1)) & cmpmask; /* For every match, search for the entire needle for a full match */ while (k2) { @@ -178,13 +182,13 @@ __strstr_avx512 (const char *haystack, const char *ned) hay0 = _mm512_loadu_si512 (haystack + hay_index); hay1 = _mm512_load_si512 (haystack + hay_index + 1); // Always 64 byte aligned - nullmask = _cvtmask64_u64 (_mm512_testn_epi8_mask (hay1, hay1)); + nullmask = cvtmask64_u64 (_mm512_testn_epi8_mask (hay1, hay1)); /* Compare only till null char */ cmpmask = nullmask ^ (nullmask - ONE_64BIT); k0 = _mm512_cmpeq_epi8_mask (hay0, ned0); k1 = _mm512_cmpeq_epi8_mask (hay1, ned1); /* k2 masks tell us if both chars from needle match */ - k2 = _cvtmask64_u64 (_kand_mask64 (k0, k1)) & cmpmask; + k2 = cvtmask64_u64 (kand_mask64 (k0, k1)) & cmpmask; /* For every match, compare full strings for potential match */ while (k2) {