From patchwork Wed Dec 1 15:08:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Wakely X-Patchwork-Id: 48364 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0491C3857C66 for ; Wed, 1 Dec 2021 15:11:17 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0491C3857C66 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1638371477; bh=ydtC6OvDDHAYtmWSU9hmrq6GfrRatRNOuUyHlcVjZew=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=QjRG4yBgEgp0gJNnVdHqdB7mmNy+/rLLl4cbnOuEeShgrenoh2NNQwEpCcVyDEQih oQmmRgNLAtAJK35QEsUaX05nLozJ2xnHKJfUqipAI9DxrBDTCvdoJGGu1eQ+64r/XP ROAAUoVlw7elkfPWDMaCOFct6dLgFMWxi8IN5wM4= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 4030C3857C42 for ; Wed, 1 Dec 2021 15:08:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4030C3857C42 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-47-z-vVzIM4OXyAty2Nh3zhsg-1; Wed, 01 Dec 2021 10:08:26 -0500 X-MC-Unique: z-vVzIM4OXyAty2Nh3zhsg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4744D1023F72; Wed, 1 Dec 2021 15:08:17 +0000 (UTC) Received: from localhost (unknown [10.33.36.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id E29345DF21; Wed, 1 Dec 2021 15:08:16 +0000 (UTC) To: libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org Subject: [committed] libstdc++: Optimize ref-count updates in COW std::string Date: Wed, 1 Dec 2021 15:08:16 +0000 Message-Id: <20211201150816.217497-1-jwakely@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-14.0 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=unavailable autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jonathan Wakely via Gcc-patches From: Jonathan Wakely Reply-To: Jonathan Wakely Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Tested x86_64-linux, pushed to trunk. Most ref-count updates in the COW string are done via the functions in , which will use non-atomic ops when the program is known to be single-threaded. The _M_is_leaked() and _M_is_shared() functions use __atomic_load_n directly, because doesn't provide a load operation. Those functions can check the __is_single_threaded() predicate to avoid using __atomic_load_n when not needed. The move constructor for the fully-dynamic-string increments the ref-count by either 2 or 1, for leaked or non-leaked strings respectively. That can be changed to use a non-atomic store of 1 for all non-shared strings. It can be non-atomic because even if the program is multi-threaded, conflicting access to the rvalue object while it's being moved from would be data race anyway. It can store 1 directly for all non-shared strings because it doesn't matter whether the initial refcount was -1 or 0, it should be 1 after the move constructor creates a second owner. libstdc++-v3/ChangeLog: * include/bits/cow_string.h (basic_string::_M_is_leaked): Use non-atomic load when __is_single_threaded() is true. (basic_string::_M_is_shared): Likewise. (basic_string::(basic_string&&)) [_GLIBCXX_FULLY_DYNAMIC_STRING]: Use non-atomic store when rvalue is not shared. --- libstdc++-v3/include/bits/cow_string.h | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/libstdc++-v3/include/bits/cow_string.h b/libstdc++-v3/include/bits/cow_string.h index ced395b80b8..4fae1d02981 100644 --- a/libstdc++-v3/include/bits/cow_string.h +++ b/libstdc++-v3/include/bits/cow_string.h @@ -105,7 +105,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION * destroy the empty-string _Rep object. * * All but the last paragraph is considered pretty conventional - * for a C++ string implementation. + * for a Copy-On-Write C++ string implementation. */ // 21.3 Template class basic_string template @@ -207,10 +207,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION // so we need to use an atomic load. However, _M_is_leaked // predicate does not change concurrently (i.e. the string is either // leaked or not), so a relaxed load is enough. - return __atomic_load_n(&this->_M_refcount, __ATOMIC_RELAXED) < 0; -#else - return this->_M_refcount < 0; + if (!__gnu_cxx::__is_single_threaded()) + return __atomic_load_n(&this->_M_refcount, __ATOMIC_RELAXED) < 0; #endif + return this->_M_refcount < 0; } bool @@ -222,10 +222,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION // but one reference concurrently with this check, so we need this // load to be acquire to synchronize with release fetch_and_add in // _M_dispose. - return __atomic_load_n(&this->_M_refcount, __ATOMIC_ACQUIRE) > 0; -#else - return this->_M_refcount > 0; + if (!__gnu_cxx::__is_single_threaded()) + return __atomic_load_n(&this->_M_refcount, __ATOMIC_ACQUIRE) > 0; #endif + return this->_M_refcount > 0; } void @@ -629,12 +629,12 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION #else // Rather than allocate an empty string for the rvalue string, // just share ownership with it by incrementing the reference count. - // If the rvalue string was "leaked" then it was the unique owner, - // so need an extra increment to indicate shared ownership. - if (_M_rep()->_M_is_leaked()) - __gnu_cxx::__atomic_add_dispatch(&_M_rep()->_M_refcount, 2); - else + // If the rvalue string was the unique owner then there are exactly + // two owners now. + if (_M_rep()->_M_is_shared()) __gnu_cxx::__atomic_add_dispatch(&_M_rep()->_M_refcount, 1); + else + _M_rep()->_M_refcount = 1; #endif }