From patchwork Fri Jan 10 21:23:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Wakely X-Patchwork-Id: 104537 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2A7BB385828E for ; Fri, 10 Jan 2025 21:36:37 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2A7BB385828E Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D3eysbgM X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTP id 6C4533858294 for ; Fri, 10 Jan 2025 21:28:22 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6C4533858294 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 6C4533858294 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1736544502; cv=none; b=LjOuNi0vy85l32P/TXTpBaBLtyyZSfJfDqcq14urrZxUWCSiWPS8TR5Hc0nAsT+Mzf6pX4VzNxAyRTPt3kA2CvCq1e98ljqdf9KENMqwpZk+03yMk5Ml/SvT84CBw2Myng/g8oOExC88JqFG9KEomITtA8vSIgF69sMKa+wQZQM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1736544502; c=relaxed/simple; bh=GRykXzW9Dg1dH3puH66mHby/45RogDJPTo1Vh7ZGCos=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=eF9Pk7iPKUlkvRcAN+nlzrbXuuvBiuojKqI+00KDUj40SA/GsEQilnfQ/gLn68M6wTeZpQrkb2QL3LgRU3O1ly72nCLXB4I041qUAhRqm+oBeFIOUe/VvHACLWg8F6ciYRy/Z6vte1lzcbwY8xX8tHQoeCHyVhBns9ncd/3qYJo= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6C4533858294 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1736544502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mg6QvtvV0KtXwZPV1okS6yPdDJ0J3AWvwsjeDcv4F8o=; b=D3eysbgM4iF9K/ajvQgYx6l1UJtjk4qmEuzAFam0gYgt3WSsoteLF2Flknm/hDRRttzslo i2xkLuErKqLfve1bnEdoo5Te3l4T6LcEMqFpg3rZLWGFVqpD77DD3bLOGEDKqVot/yJRBa RrghmuCUDq7MTmA3PxbdBR2T3xgCMWc= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-569-QnOsY_34MPCRTM1e702Cpg-1; Fri, 10 Jan 2025 16:28:18 -0500 X-MC-Unique: QnOsY_34MPCRTM1e702Cpg-1 X-Mimecast-MFC-AGG-ID: QnOsY_34MPCRTM1e702Cpg Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EB8191956064; Fri, 10 Jan 2025 21:28:17 +0000 (UTC) Received: from localhost (unknown [10.42.28.9]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 50A5A30001BE; Fri, 10 Jan 2025 21:28:17 +0000 (UTC) From: Jonathan Wakely To: libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org Subject: [PATCH 02/10] libstdc++: Pass __wait_args to internal API by const pointer Date: Fri, 10 Jan 2025 21:23:05 +0000 Message-ID: <20250110212810.832494-3-jwakely@redhat.com> In-Reply-To: <20250110212810.832494-1-jwakely@redhat.com> References: <20250110212810.832494-1-jwakely@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: CBBdGCAdGzuPLMxdz-B6Ort367MM2R5mm54x7nwwT8M_1736544498 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org From: Thomas Rodgers This change splits the __wait_args data members to a new struct __wait_args_base and then passes that type by const pointer to the low level implementation functions. libstdc++-v3/ChangeLog: * include/bits/atomic_timed_wait.h (__spin_until_impl): Accept __wait_args as const __wait_args_base*. (__wait_until_impl): Likewise. (__wait_until): Likewise. (__wait_for): Likewise. (__atomic_wait_address_until): Pass __wait_args by address. (__atomic_wait_address_for): Likewise. * include/bits/atomic_wait.h (__wait_args_base): New struct. (__wait_args): Derive from __wait_args_base. (__wait_args::__wait_args()): Adjust ctors to call call base ctor. (__wait_args::__wait_args(const __wait_args_base&)): New ctor. (__wait_args::operator|=): New method. (__wait_args::_S_flags_for): Change return type to __wait_flags. (__spin_impl): Accept __wait_args as const __wait_args_base*. (__wait_impl): Likewise. (__notify_impl): Likewise. (__atomic_wait_address): Pass __wait_args by address. (__atomic_wait_address_v): Likewise. (__atomic_notify_address): Likewise. --- libstdc++-v3/include/bits/atomic_timed_wait.h | 35 +++++++---- libstdc++-v3/include/bits/atomic_wait.h | 62 +++++++++++++------ 2 files changed, 64 insertions(+), 33 deletions(-) diff --git a/libstdc++-v3/include/bits/atomic_timed_wait.h b/libstdc++-v3/include/bits/atomic_timed_wait.h index 196548484024..4504b1b84bb8 100644 --- a/libstdc++-v3/include/bits/atomic_timed_wait.h +++ b/libstdc++-v3/include/bits/atomic_timed_wait.h @@ -135,9 +135,11 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION #endif // _GLIBCXX_HAS_GTHREADS inline __wait_result_type - __spin_until_impl(const __platform_wait_t* __addr, __wait_args __args, + __spin_until_impl(const __platform_wait_t* __addr, + const __wait_args_base* __a, const __wait_clock_t::time_point& __deadline) { + __wait_args __args{ *__a }; auto __t0 = __wait_clock_t::now(); using namespace literals::chrono_literals; @@ -163,7 +165,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION } else { - auto __res = __detail::__spin_impl(__addr, __args); + auto __res = __detail::__spin_impl(__addr, __a); if (__res.first) return __res; } @@ -176,9 +178,11 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION } inline __wait_result_type - __wait_until_impl(const __platform_wait_t* __addr, __wait_args __args, + __wait_until_impl(const __platform_wait_t* __addr, + const __wait_args_base* __a, const __wait_clock_t::time_point& __atime) { + __wait_args __args{ *__a }; #ifdef _GLIBCXX_HAVE_PLATFORM_TIMED_WAIT __waiter_pool_impl* __pool = nullptr; #else @@ -200,7 +204,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION if (__args & __wait_flags::__do_spin) { - auto __res = __detail::__spin_until_impl(__wait_addr, __args, __atime); + auto __res = __detail::__spin_until_impl(__wait_addr, __a, __atime); if (__res.first) return __res; if (__args & __wait_flags::__spin_only) @@ -246,7 +250,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION template __wait_result_type - __wait_until(const __platform_wait_t* __addr, __wait_args __args, + __wait_until(const __platform_wait_t* __addr, const __wait_args* __args, const chrono::time_point<_Clock, _Dur>& __atime) noexcept { if constexpr (is_same_v<__wait_clock_t, _Clock>) @@ -269,15 +273,20 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION template __wait_result_type - __wait_for(const __platform_wait_t* __addr, __wait_args __args, + __wait_for(const __platform_wait_t* __addr, const __wait_args_base* __a, const chrono::duration<_Rep, _Period>& __rtime) noexcept { + __wait_args __args{ *__a }; if (!__rtime.count()) - // no rtime supplied, just spin a bit - return __detail::__wait_impl(__addr, __args | __wait_flags::__spin_only); + { + // no rtime supplied, just spin a bit + __args |= __wait_flags::__spin_only; + return __detail::__wait_impl(__addr, &__args); + } + auto const __reltime = chrono::ceil<__wait_clock_t::duration>(__rtime); auto const __atime = chrono::steady_clock::now() + __reltime; - return __detail::__wait_until(__addr, __args, __atime); + return __detail::__wait_until(__addr, &__args, __atime); } } // namespace __detail @@ -297,7 +306,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION _Tp __val = __vfn(); while (!__pred(__val)) { - auto __res = __detail::__wait_until(__wait_addr, __args, __atime); + auto __res = __detail::__wait_until(__wait_addr, &__args, __atime); if (!__res.first) // timed out return __res.first; // C++26 will also return last observed __val @@ -315,7 +324,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION bool __bare_wait = false) noexcept { __detail::__wait_args __args{ __addr, __old, __order, __bare_wait }; - auto __res = __detail::__wait_until(__addr, __args, __atime); + auto __res = __detail::__wait_until(__addr, &__args, __atime); return __res.first; // C++26 will also return last observed __val } @@ -347,7 +356,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION _Tp __val = __vfn(); while (!__pred(__val)) { - auto __res = __detail::__wait_for(__wait_addr, __args, __rtime); + auto __res = __detail::__wait_for(__wait_addr, &__args, __rtime); if (!__res.first) // timed out return __res.first; // C++26 will also return last observed __val @@ -365,7 +374,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION bool __bare_wait = false) noexcept { __detail::__wait_args __args{ __addr, __old, __order, __bare_wait }; - auto __res = __detail::__wait_for(__addr, __args, __rtime); + auto __res = __detail::__wait_for(__addr, &__args, __rtime); return __res.first; // C++26 will also return last observed __Val } diff --git a/libstdc++-v3/include/bits/atomic_wait.h b/libstdc++-v3/include/bits/atomic_wait.h index 18cfc2ef7bd2..cb246ed616d8 100644 --- a/libstdc++-v3/include/bits/atomic_wait.h +++ b/libstdc++-v3/include/bits/atomic_wait.h @@ -215,23 +215,28 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION __abi_version_mask = 0xffff0000, }; - struct __wait_args + struct __wait_args_base { - __platform_wait_t _M_old; - int _M_order = __ATOMIC_ACQUIRE; __wait_flags _M_flags; + int _M_order = __ATOMIC_ACQUIRE; + __platform_wait_t _M_old = 0; + }; + struct __wait_args : __wait_args_base + { template explicit __wait_args(const _Tp* __addr, bool __bare_wait = false) noexcept - : _M_flags{ _S_flags_for(__addr, __bare_wait) } + : __wait_args_base{ _S_flags_for(__addr, __bare_wait) } { } __wait_args(const __platform_wait_t* __addr, __platform_wait_t __old, int __order, bool __bare_wait = false) noexcept - : _M_old{ __old } - , _M_order{ __order } - , _M_flags{ _S_flags_for(__addr, __bare_wait) } + : __wait_args_base{ _S_flags_for(__addr, __bare_wait), __order, __old } + { } + + explicit __wait_args(const __wait_args_base& __base) + : __wait_args_base{ __base } { } __wait_args(const __wait_args&) noexcept = default; @@ -257,6 +262,16 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION return __res; } + __wait_args& + operator|=(__wait_flags __flag) noexcept + { + using __t = underlying_type_t<__wait_flags>; + const auto __flags = static_cast<__t>(_M_flags) + | static_cast<__t>(__flag); + _M_flags = __wait_flags{ __flags }; + return *this; + } + private: static int constexpr _S_default_flags() noexcept @@ -267,7 +282,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION } template - static int + static __wait_flags constexpr _S_flags_for(const _Tp*, bool __bare_wait) noexcept { auto __res = _S_default_flags(); @@ -275,7 +290,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION __res |= static_cast(__wait_flags::__track_contention); if constexpr (!__platform_wait_uses_type<_Tp>) __res |= static_cast(__wait_flags::__proxy_wait); - return __res; + return static_cast<__wait_flags>(__res); } template @@ -290,13 +305,13 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION using __wait_result_type = pair; inline __wait_result_type - __spin_impl(const __platform_wait_t* __addr, __wait_args __args) + __spin_impl(const __platform_wait_t* __addr, const __wait_args_base* __args) { __platform_wait_t __val; for (auto __i = 0; __i < __atomic_spin_count; ++__i) { - __atomic_load(__addr, &__val, __args._M_order); - if (__val != __args._M_old) + __atomic_load(__addr, &__val, __args->_M_order); + if (__val != __args->_M_old) return make_pair(true, __val); if (__i < __atomic_spin_count_relax) __detail::__thread_relax(); @@ -307,8 +322,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION } inline __wait_result_type - __wait_impl(const __platform_wait_t* __addr, __wait_args __args) + __wait_impl(const __platform_wait_t* __addr, const __wait_args_base* __a) { + __wait_args __args{ *__a }; #ifdef _GLIBCXX_HAVE_PLATFORM_WAIT __waiter_pool_impl* __pool = nullptr; #else @@ -317,20 +333,25 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION #endif __platform_wait_t* __wait_addr; + __platform_wait_t __old; if (__args & __wait_flags::__proxy_wait) { #ifdef _GLIBCXX_HAVE_PLATFORM_WAIT __pool = &__waiter_pool_impl::_S_impl_for(__addr); #endif __wait_addr = &__pool->_M_ver; - __atomic_load(__wait_addr, &__args._M_old, __args._M_order); + __atomic_load(__wait_addr, &__old, __args._M_order); } else - __wait_addr = const_cast<__platform_wait_t*>(__addr); + { + __wait_addr = const_cast<__platform_wait_t*>(__addr); + __old = __args._M_old; + } + if (__args & __wait_flags::__do_spin) { - auto __res = __detail::__spin_impl(__wait_addr, __args); + auto __res = __detail::__spin_impl(__wait_addr, __a); if (__res.first) return __res; if (__args & __wait_flags::__spin_only) @@ -372,8 +393,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION inline void __notify_impl(const __platform_wait_t* __addr, [[maybe_unused]] bool __all, - __wait_args __args) + const __wait_args_base* __a) { + __wait_args __args{ __a }; #ifdef _GLIBCXX_HAVE_PLATFORM_WAIT __waiter_pool_impl* __pool = nullptr; #else @@ -424,7 +446,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION _Tp __val = __vfn(); while (!__pred(__val)) { - __detail::__wait_impl(__wait_addr, __args); + __detail::__wait_impl(__wait_addr, &__args); __val = __vfn(); } // C++26 will return __val @@ -437,7 +459,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION { __detail::__wait_args __args{ __addr, __old, __order }; // C++26 will not ignore the return value here - __detail::__wait_impl(__addr, __args); + __detail::__wait_impl(__addr, &__args); } template @@ -458,7 +480,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION const auto __wait_addr = reinterpret_cast(__addr); __detail::__wait_args __args{ __addr, __bare_wait }; - __detail::__notify_impl(__wait_addr, __all, __args); + __detail::__notify_impl(__wait_addr, __all, &__args); } _GLIBCXX_END_NAMESPACE_VERSION } // namespace std