From patchwork Wed Jun 5 15:31:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Wakely X-Patchwork-Id: 91570 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8F59E398A7EF for ; Wed, 5 Jun 2024 15:33:19 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id AE18F398A7CA for ; Wed, 5 Jun 2024 15:32:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org AE18F398A7CA Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org AE18F398A7CA Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717601571; cv=none; b=C1bRqI5p7bV5w4DJvkiJ0bpflhMSR4vp7k4BfJ1SWvOMBYL4vQ6pXGEGFDuvGVC2wG8VYlqbMLHLc2IDt9ZmZksmy0u9CBORqOfK7Dyb55GYvVst/OpLIh4RSIhFYi7UV4FcLgocTzX2vaJb8myZLSTRVM0TROJiSDEtDxIqYGM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717601571; c=relaxed/simple; bh=TTRCsVl/51u0up0sVjP4PLvMM/rwVAanIFbl+zX+Bz0=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=pYFXrCpI8Ulp1X4dfHh+WPB5e9YZteIaoKkfIokRRYlz1xU3UmywYf6PTKctfIBY2rIiIpBGlNOYz0iGm6djHjaDtYwKUDEvi+7IJ+XkeHNsjKdQXyhmWE+BkXuyg4DEHtoZwTedotngUtEpxB/L6K5cdrGlqFru3KmaYKDqg18= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1717601569; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/R0X1jLWglW+Ig0pMLTM60dB75D7qYcQmrlhqeiGnyA=; b=UMG+HvK8SZKY+NkJ6z3ujg571rKv4MH5maRkbDrk1Mx5RDPF0VBy0BWMHfGEyV5nE4l06s LjCqYDnM8qGCqswxLHbT+O6TtKZk6SNfF9cLDLLmPi3hiwlXntOAhwSPRHV/FkddtOpnCa KPJx/ftAPf4d8kUv0hvJWzDEgMOKhVg= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-683-NltZApiTNSSeswQr4uPTUA-1; Wed, 05 Jun 2024 11:32:35 -0400 X-MC-Unique: NltZApiTNSSeswQr4uPTUA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 618CB3806700; Wed, 5 Jun 2024 15:32:35 +0000 (UTC) Received: from localhost (unknown [10.39.192.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 11310111F3C6; Wed, 5 Jun 2024 15:32:34 +0000 (UTC) From: Jonathan Wakely To: libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org Subject: [PATCH] libstdc++: Use memchr to optimize std::find [PR88545] Date: Wed, 5 Jun 2024 16:31:46 +0100 Message-ID: <20240605153233.119881-1-jwakely@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org I plan to push this after testing finishes. -- >8 -- This optimizes std::find to use memchr when searching for an integer in a range of bytes. libstdc++-v3/ChangeLog: PR libstdc++/88545 PR libstdc++/115040 * include/bits/cpp_type_traits.h (__can_use_memchr_for_find): New variable template. * include/bits/ranges_util.h (__find_fn): Use memchr when possible. * include/bits/stl_algo.h (find): Likewise. * testsuite/25_algorithms/find/bytes.cc: New test. --- libstdc++-v3/include/bits/cpp_type_traits.h | 13 ++ libstdc++-v3/include/bits/ranges_util.h | 17 +++ libstdc++-v3/include/bits/stl_algo.h | 35 ++++++ .../testsuite/25_algorithms/find/bytes.cc | 112 ++++++++++++++++++ 4 files changed, 177 insertions(+) create mode 100644 libstdc++-v3/testsuite/25_algorithms/find/bytes.cc diff --git a/libstdc++-v3/include/bits/cpp_type_traits.h b/libstdc++-v3/include/bits/cpp_type_traits.h index 59f1a1875eb..466e6792a11 100644 --- a/libstdc++-v3/include/bits/cpp_type_traits.h +++ b/libstdc++-v3/include/bits/cpp_type_traits.h @@ -35,6 +35,10 @@ #pragma GCC system_header #include +#include +#if __glibcxx_type_trait_variable_templates +# include // is_same_v, is_integral_v +#endif // // This file provides some compile-time information about various types. @@ -589,6 +593,15 @@ __INT_N(__GLIBCXX_TYPE_INT_N_3) { static constexpr bool __value = false; }; #endif +#if __glibcxx_type_trait_variable_templates + template + constexpr bool __can_use_memchr_for_find + // Can only use memchr to search for narrow characters and std::byte. + = __is_byte<_ValT>::__value + // And only if the value to find is an integer (or is also std::byte). + && (is_same_v<_Tp, _ValT> || is_integral_v<_Tp>); +#endif + // // Move iterator type // diff --git a/libstdc++-v3/include/bits/ranges_util.h b/libstdc++-v3/include/bits/ranges_util.h index 9b79c3a229d..7247e89a79d 100644 --- a/libstdc++-v3/include/bits/ranges_util.h +++ b/libstdc++-v3/include/bits/ranges_util.h @@ -34,6 +34,7 @@ # include # include # include +# include // __can_use_memchr_for_find #ifdef __glibcxx_ranges namespace std _GLIBCXX_VISIBILITY(default) @@ -494,6 +495,22 @@ namespace ranges operator()(_Iter __first, _Sent __last, const _Tp& __value, _Proj __proj = {}) const { + if constexpr (is_same_v<_Proj, identity>) + if constexpr(__can_use_memchr_for_find, _Tp>) + if constexpr (sized_sentinel_for<_Sent, _Iter>) + if constexpr (contiguous_iterator<_Iter>) + { + auto __n = __last - __first; + if (__n > 0) + { + const int __ival = static_cast(__value); + const void* __p0 = std::to_address(__first); + if (auto __p1 = __builtin_memchr(__p0, __ival, __n)) + __n = (const char*)__p1 - (const char*)__p0; + } + return __first + __n; + } + while (__first != __last && !(std::__invoke(__proj, *__first) == __value)) ++__first; diff --git a/libstdc++-v3/include/bits/stl_algo.h b/libstdc++-v3/include/bits/stl_algo.h index 1a996aa61da..eba3157a480 100644 --- a/libstdc++-v3/include/bits/stl_algo.h +++ b/libstdc++-v3/include/bits/stl_algo.h @@ -3836,6 +3836,7 @@ _GLIBCXX_BEGIN_NAMESPACE_ALGO * such that @c *i == @p __val, or @p __last if no such iterator exists. */ template + _GLIBCXX_NODISCARD _GLIBCXX20_CONSTEXPR inline _InputIterator find(_InputIterator __first, _InputIterator __last, @@ -3846,6 +3847,40 @@ _GLIBCXX_BEGIN_NAMESPACE_ALGO __glibcxx_function_requires(_EqualOpConcept< typename iterator_traits<_InputIterator>::value_type, _Tp>) __glibcxx_requires_valid_range(__first, __last); + +#if __cpp_if_constexpr && __glibcxx_type_trait_variable_templates + using _ValT = typename iterator_traits<_InputIterator>::value_type; + if constexpr (__can_use_memchr_for_find<_ValT, _Tp>) + { + // If converting the value to the 1-byte value_type alters its value, + // then it would not be found by std::find using equality comparison. + // We need to check this here, because otherwise something like + // memchr("a", 'a'+256, 1) would give a false positive match. + if (static_cast<_ValT>(__val) != __val) + return __last; + + const void* __p0 = nullptr; + if constexpr (is_pointer_v) + __p0 = std::__niter_base(__first); +#if __cpp_lib_concepts + else if constexpr (contiguous_iterator<_InputIterator>) + __p0 = std::to_address(__first); +#endif + + if (__p0) + { + auto __n = std::distance(__first, __last); + if (__n > 0) + { + const int __ival = static_cast(__val); + if (auto __p1 = __builtin_memchr(__p0, __ival, __n)) + return __first + ((const char*)__p1 - (const char*)__p0); + } + return __last; + } + } +#endif + return std::__find_if(__first, __last, __gnu_cxx::__ops::__iter_equals_val(__val)); } diff --git a/libstdc++-v3/testsuite/25_algorithms/find/bytes.cc b/libstdc++-v3/testsuite/25_algorithms/find/bytes.cc new file mode 100644 index 00000000000..ac189dac65f --- /dev/null +++ b/libstdc++-v3/testsuite/25_algorithms/find/bytes.cc @@ -0,0 +1,112 @@ +// { dg-do run } + +#include +#include // std::byte +#include + +// PR libstdc++/88545 made std::find use memchr as an optimization. +// This test verifies that it didn't change any semantics. + +template +void +test_char() +{ + const C a[] = { (C)'a', (C)'b', (C)'c', (C)'d' }; + const C* end = a + sizeof(a); + const C* res = std::find(a, end, a[0]); + VERIFY( res == a ); + res = std::find(a, end, a[2]); + VERIFY( res == a+2 ); + res = std::find(a, end, a[0] + 256); + VERIFY( res == end ); + res = std::find(a, end, a[0] - 256); + VERIFY( res == end ); + res = std::find(a, end, 256); + VERIFY( res == end ); + +#ifdef __cpp_lib_ranges + res = std::ranges::find(a, a[0]); + VERIFY( res == a ); + res = std::ranges::find(a, a[2]); + VERIFY( res == a+2 ); + res = std::ranges::find(a, a[0] + 256); + VERIFY( res == end ); + res = std::ranges::find(a, a[0] - 256); + VERIFY( res == end ); + res = std::ranges::find(a, 256); + VERIFY( res == end ); +#endif +} + +// Trivial type of size 1, with custom equality. +struct S { + bool operator==(const S&) const { return true; }; + char c; +}; + +// Trivial type of size 1, with custom equality. +enum E +#if __cplusplus >= 201103L +: unsigned char +#endif +{ e1 = 1, e255 = 255 }; + +bool operator==(E l, E r) { return (l % 3) == (r % 3); } + +struct X { char c; }; +bool operator==(X, char) { return false; } +bool operator==(char, X) { return false; } + +void +test_non_characters() +{ + S s[3] = { {'a'}, {'b'}, {'c'} }; + S sx = {'x'}; + S* sres = std::find(s, s+3, sx); + VERIFY( sres == s ); // memchr optimization would not find a match + + E e[3] = { E(1), E(2), E(3) }; + E* eres = std::find(e, e+3, E(4)); + VERIFY( eres == e ); // memchr optimization would not find a match + + char x[1] = { 'x' }; + X xx = { 'x' }; + char* xres = std::find(x, x+1, xx); + VERIFY( xres == x+1 ); // memchr optimization would find a match + +#ifdef __cpp_lib_byte + std::byte b[] = { std::byte{0}, std::byte{1}, std::byte{2}, std::byte{3} }; + std::byte* bres = std::find(b, b+4, std::byte{4}); + VERIFY( bres == b+4 ); + bres = std::find(b, b+2, std::byte{3}); + VERIFY( bres == b+2 ); + bres = std::find(b, b+3, std::byte{3}); + VERIFY( bres == b+3 ); +#endif + +#ifdef __cpp_lib_ranges + sres = std::ranges::find(s, sx); + VERIFY( sres == s ); + + eres = std::ranges::find(e, e+3, E(4)); + VERIFY( eres == e ); + + xres = std::ranges::find(x, xx); + VERIFY( xres == std::ranges::end(x) ); + + bres = std::ranges::find(b, std::byte{4}); + VERIFY( bres == b+4 ); + bres = std::ranges::find(b, b+2, std::byte{3}); + VERIFY( bres == b+2 ); + bres = std::ranges::find(b, b+3, std::byte{3}); + VERIFY( bres == b+3 ); +#endif +} + +int main() +{ + test_char(); + test_char(); + test_char(); + test_non_characters(); +}