libstdc++: Clear padding bits in atomic compare_exchange
Commit Message
From: Thomas Rodgers <rodgert@twrodgers.com>
This change implements P0528 which requires that padding bits not
participate in atomic compare exchange operations. All arguments to the
generic template are 'sanitized' by the __builtin_clear_padding intrinsic
before they are used in atomic compare_exchange. This alrequires that any
stores also sanitize the incoming value.
Signed-off-by: Thomas Rodgers <trodgers@redhat.com>
libstdc++=v3/ChangeLog:
* include/std/atomic (atomic<T>::atomic(_Tp) clear padding for
__cplusplus > 201703L.
(atomic<T>::store()) Clear padding.
(atomic<T>::exchange()) Likewise.
(atomic<T>::compare_exchange_weak()) Likewise.
(atomic<T>::compare_exchange_strong()) Likewise.
* testsuite/29_atomics/atomic/compare_exchange_padding.cc: New
test.
---
libstdc++-v3/include/std/atomic | 23 +++++++++-
.../atomic/compare_exchange_padding.cc | 42 +++++++++++++++++++
2 files changed, 63 insertions(+), 2 deletions(-)
create mode 100644 libstdc++-v3/testsuite/29_atomics/atomic/compare_exchange_padding.cc
Comments
On Thu, Sep 23, 2021 at 11:08:37AM -0700, Thomas Rodgers wrote:
> From: Thomas Rodgers <rodgert@twrodgers.com>
>
> This change implements P0528 which requires that padding bits not
> participate in atomic compare exchange operations. All arguments to the
Thanks for working on this.
> generic template are 'sanitized' by the __builtin_clear_padding intrinsic
> before they are used in atomic compare_exchange. This alrequires that any
> stores also sanitize the incoming value.
Not a review, just random nit.
Shouldn't the __builtin_clear_padding calls be guarded with
#if __has_builtin(__builtin_clear_padding)
or #ifdef _GLIBCXX_HAVE_BUILTIN_CLEAR_PADDING defined based on that?
I think clang doesn't support it (yet?), and it doesn't support the MSVC
__builtin_zero_non_value_bits (with very similar, but slightly different,
behavior).
> Signed-off-by: Thomas Rodgers <trodgers@redhat.com>
>
> libstdc++=v3/ChangeLog:
>
> * include/std/atomic (atomic<T>::atomic(_Tp) clear padding for
> __cplusplus > 201703L.
> (atomic<T>::store()) Clear padding.
> (atomic<T>::exchange()) Likewise.
> (atomic<T>::compare_exchange_weak()) Likewise.
> (atomic<T>::compare_exchange_strong()) Likewise.
> * testsuite/29_atomics/atomic/compare_exchange_padding.cc: New
> test.
Jakub
Agreed, I'll revise the patch to do so.
On Thu, Sep 23, 2021 at 12:07 PM Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, Sep 23, 2021 at 11:08:37AM -0700, Thomas Rodgers wrote:
> > From: Thomas Rodgers <rodgert@twrodgers.com>
> >
> > This change implements P0528 which requires that padding bits not
> > participate in atomic compare exchange operations. All arguments to the
>
> Thanks for working on this.
>
> > generic template are 'sanitized' by the __builtin_clear_padding intrinsic
> > before they are used in atomic compare_exchange. This alrequires that any
> > stores also sanitize the incoming value.
>
> Not a review, just random nit.
> Shouldn't the __builtin_clear_padding calls be guarded with
> #if __has_builtin(__builtin_clear_padding)
> or #ifdef _GLIBCXX_HAVE_BUILTIN_CLEAR_PADDING defined based on that?
> I think clang doesn't support it (yet?), and it doesn't support the MSVC
> __builtin_zero_non_value_bits (with very similar, but slightly different,
> behavior).
>
> > Signed-off-by: Thomas Rodgers <trodgers@redhat.com>
> >
> > libstdc++=v3/ChangeLog:
> >
> > * include/std/atomic (atomic<T>::atomic(_Tp) clear padding for
> > __cplusplus > 201703L.
> > (atomic<T>::store()) Clear padding.
> > (atomic<T>::exchange()) Likewise.
> > (atomic<T>::compare_exchange_weak()) Likewise.
> > (atomic<T>::compare_exchange_strong()) Likewise.
> > * testsuite/29_atomics/atomic/compare_exchange_padding.cc: New
> > test.
>
> Jakub
>
>
On Thu, 23 Sep 2021, 20:07 Jakub Jelinek via Libstdc++, <
libstdc++@gcc.gnu.org> wrote:
> On Thu, Sep 23, 2021 at 11:08:37AM -0700, Thomas Rodgers wrote:
> > From: Thomas Rodgers <rodgert@twrodgers.com>
> >
> > This change implements P0528 which requires that padding bits not
> > participate in atomic compare exchange operations. All arguments to the
>
> Thanks for working on this.
>
> > generic template are 'sanitized' by the __builtin_clear_padding intrinsic
> > before they are used in atomic compare_exchange. This alrequires that any
> > stores also sanitize the incoming value.
>
> Not a review, just random nit.
> Shouldn't the __builtin_clear_padding calls be guarded with
> #if __has_builtin(__builtin_clear_padding)
> or #ifdef _GLIBCXX_HAVE_BUILTIN_CLEAR_PADDING defined based on that?
>
Yes. It can just use __has_builtin directly. All the compilers we care
about support that now.
I think clang doesn't support it (yet?), and it doesn't support the MSVC
> __builtin_zero_non_value_bits (with very similar, but slightly different,
> behavior).
>
> > Signed-off-by: Thomas Rodgers <trodgers@redhat.com>
> >
> > libstdc++=v3/ChangeLog:
> >
> > * include/std/atomic (atomic<T>::atomic(_Tp) clear padding for
> > __cplusplus > 201703L.
> > (atomic<T>::store()) Clear padding.
> > (atomic<T>::exchange()) Likewise.
> > (atomic<T>::compare_exchange_weak()) Likewise.
> > (atomic<T>::compare_exchange_strong()) Likewise.
> > * testsuite/29_atomics/atomic/compare_exchange_padding.cc: New
> > test.
>
> Jakub
>
>
@@ -228,7 +228,13 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
atomic& operator=(const atomic&) = delete;
atomic& operator=(const atomic&) volatile = delete;
- constexpr atomic(_Tp __i) noexcept : _M_i(__i) { }
+#if __cplusplus > 201703L
+ constexpr atomic(_Tp __i) noexcept : _M_i(__i)
+ { __builtin_clear_padding(std::__addressof(_M_i)); }
+#else
+ atomic(_Tp __i) noexcept : _M_i(__i)
+ { }
+#endif
operator _Tp() const noexcept
{ return load(); }
@@ -268,12 +274,14 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
void
store(_Tp __i, memory_order __m = memory_order_seq_cst) noexcept
{
+ __builtin_clear_padding(std::__addressof(__i));
__atomic_store(std::__addressof(_M_i), std::__addressof(__i), int(__m));
}
void
store(_Tp __i, memory_order __m = memory_order_seq_cst) volatile noexcept
{
+ __builtin_clear_padding(std::__addressof(__i));
__atomic_store(std::__addressof(_M_i), std::__addressof(__i), int(__m));
}
@@ -300,6 +308,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
alignas(_Tp) unsigned char __buf[sizeof(_Tp)];
_Tp* __ptr = reinterpret_cast<_Tp*>(__buf);
+ __builtin_clear_padding(std::__addressof(__i));
__atomic_exchange(std::__addressof(_M_i), std::__addressof(__i),
__ptr, int(__m));
return *__ptr;
@@ -311,6 +320,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
alignas(_Tp) unsigned char __buf[sizeof(_Tp)];
_Tp* __ptr = reinterpret_cast<_Tp*>(__buf);
+ __builtin_clear_padding(std::__addressof(__i));
__atomic_exchange(std::__addressof(_M_i), std::__addressof(__i),
__ptr, int(__m));
return *__ptr;
@@ -322,6 +332,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
__glibcxx_assert(__is_valid_cmpexch_failure_order(__f));
+ __builtin_clear_padding(std::__addressof(__e));
+ __builtin_clear_padding(std::__addressof(__i));
return __atomic_compare_exchange(std::__addressof(_M_i),
std::__addressof(__e),
std::__addressof(__i),
@@ -334,6 +346,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
__glibcxx_assert(__is_valid_cmpexch_failure_order(__f));
+ __builtin_clear_padding(std::__addressof(__e));
+ __builtin_clear_padding(std::__addressof(__i));
return __atomic_compare_exchange(std::__addressof(_M_i),
std::__addressof(__e),
std::__addressof(__i),
@@ -358,6 +372,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
__glibcxx_assert(__is_valid_cmpexch_failure_order(__f));
+ __builtin_clear_padding(std::__addressof(__e));
+ __builtin_clear_padding(std::__addressof(__i));
return __atomic_compare_exchange(std::__addressof(_M_i),
std::__addressof(__e),
std::__addressof(__i),
@@ -370,6 +386,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
__glibcxx_assert(__is_valid_cmpexch_failure_order(__f));
+ __builtin_clear_padding(std::__addressof(__e));
+ __builtin_clear_padding(std::__addressof(__i));
return __atomic_compare_exchange(std::__addressof(_M_i),
std::__addressof(__e),
std::__addressof(__i),
@@ -392,6 +410,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
void
wait(_Tp __old, memory_order __m = memory_order_seq_cst) const noexcept
{
+ __builtin_clear_padding(std::__addressof(__old));
std::__atomic_wait_address_v(&_M_i, __old,
[__m, this] { return this->load(__m); });
}
@@ -407,7 +426,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{ std::__atomic_notify_address(&_M_i, true); }
#endif // __cpp_lib_atomic_wait
- };
+ };
#undef _GLIBCXX20_INIT
/// Partial specialization for pointer types.
new file mode 100644
@@ -0,0 +1,42 @@
+// { dg-options "-std=gnu++2a" }
+// { dg-do run { target c++2a } }
+// { dg-add-options libatomic }
+
+#include <atomic>
+
+#include <testsuite_hooks.h>
+
+struct S { char c; short s; };
+
+void __attribute__((noinline,noipa))
+fill_struct(S& s)
+{ __builtin_memset(&s, 0xff, sizeof(S)); }
+
+bool
+compare_struct(const S& a, const S& b)
+{ return __builtin_memcmp(&a, &b, sizeof(S)) == 0; }
+
+int
+main ()
+{
+ S s;
+ fill_struct(s);
+ s.c = 'a';
+ s.s = 42;
+
+ std::atomic<S> as{ s };
+ auto ts = as.load();
+ VERIFY( !compare_struct(s, ts) ); // padding cleared on construction
+ as.exchange(s);
+ auto es = as.load();
+ VERIFY( compare_struct(ts, es) ); // padding cleared on exchange
+
+ S n;
+ fill_struct(n);
+ n.c = 'b';
+ n.s = 71;
+ // padding cleared on compexchg
+ VERIFY( as.compare_exchange_weak(s, n) );
+ VERIFY( as.compare_exchange_strong(n, s) );
+ return 0;
+}