From patchwork Thu Nov 21 19:08:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Jeanson X-Patchwork-Id: 101695 X-Patchwork-Delegate: fweimer@redhat.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6B0D53857BBA for ; Thu, 21 Nov 2024 19:22:34 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6B0D53857BBA Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=efficios.com header.i=@efficios.com header.a=rsa-sha256 header.s=smtpout1 header.b=X1m8OQeH X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from smtpout.efficios.com (smtpout.efficios.com [167.114.26.122]) by sourceware.org (Postfix) with ESMTPS id 94D193857BA0 for ; Thu, 21 Nov 2024 19:09:47 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 94D193857BA0 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=efficios.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=efficios.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 94D193857BA0 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=167.114.26.122 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1732216187; cv=none; b=F09uOVtm5MREKNfqqJhtnQhhTjMS+gz0VUohNwtqz9bBeW3oL/b/SdbbVTowa/fCy+nHTC4r6x04MnJRFWwMB9Rhh4s13QsGHxNMBufiTMfBxkf/2Y/w9jeDnLsHqCPnYfd6M/KksKcuYNkIjrs/NKBpAcjCO0cznfjtONSbsao= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1732216187; c=relaxed/simple; bh=5Y/pQYgurNSxVPLw5/I2FbdfbGCi+Z0KpoX1/zcFSak=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=R9ECwC9fDHm1TEPMxMuAae7rR5mYGAqLolgqg8Bj+1fABaHsLCCqQVwf5zNSOq26owF90g31rbBNaBzGP5n45GPkVhUZOzAc1wOxYEWzKDTZIRHvtLPjCiHeNR0V+Ps+4NMCHqECsMy4l0+zTqjLRdK2t6Mr7QiMEthNiBmPGWo= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 94D193857BA0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1732216180; bh=5Y/pQYgurNSxVPLw5/I2FbdfbGCi+Z0KpoX1/zcFSak=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X1m8OQeHQOxnngcPxB/pkoXItG2ntzCYwXQ8PYEPeXxTxmdafWG7L0+RHN0gfYVrk vpbGJD3SOVO5+M5xT0mVea1MJKf/S5eOgAI4sGzuSPGrsgyCb4kqEEhrVHLrKhGRcb nbXNGzH6sxXbmfduZ272xUVgTqPb4rJ4J/qi17t9d0izpxayIDxTjlj3mmaHM/aJTL ElIlw9jWJShTO/cu4AfmE2ymROyxFrnOIIyTsNBQ6nKBFxPRx2fKEokoH+f4Owf8sf L5Pd2i8CLie6urmqa4Oc6CQ1jhAtECnQx2JAHmHKgXx6GABcFpPsb6kJNvbBd39CF9 l0TeA92kWzZoQ== Received: from laptop-mjeanson.internal.efficios.com (96-127-217-162.qc.cable.ebox.net [96.127.217.162]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4XvSTc3Wy1z141Y; Thu, 21 Nov 2024 14:09:40 -0500 (EST) From: Michael Jeanson To: libc-alpha@sourceware.org Cc: Michael Jeanson , Florian Weimer , Carlos O'Donell , DJ Delorie , Mathieu Desnoyers Subject: [PATCH v14 6/9] nptl: Introduce for RSEQ_* accessors Date: Thu, 21 Nov 2024 14:08:56 -0500 Message-ID: <20241121190924.837446-7-mjeanson@efficios.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241121190924.837446-1-mjeanson@efficios.com> References: <20241121190924.837446-1-mjeanson@efficios.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org In preparation to move the rseq area to the 'extra TLS' block, we need accessors based on the thread pointer and the rseq offset. The VOLATILE variant of the accessors ensures single-copy atomicity for loads and stores which is required for all fields once the registration is active. A separate header is required to allow including which results in an include loop when added to . Signed-off-by: Michael Jeanson Reviewed-by: Mathieu Desnoyers --- Changes since v13: - Ensure that the VOLATILE variant static assert on 64bit types on 32bit architectures - Split to separate header to allow including 'atomic.h' without an include loop - Move rtld_hidden_proto rseq symbols to a separate patch Changes since v12: - Split RSEQ_SET/GETMEM from THREAD_SET/GETMEM - Rename rseq_get_area() to RSEQ_SELF() - Add rtld_hidden_proto to __rseq_size and __rseq_offset --- sysdeps/i386/nptl/rseq-access.h | 98 +++++++++++++++++++++++++ sysdeps/nptl/rseq-access.h | 58 +++++++++++++++ sysdeps/unix/sysv/linux/rseq-internal.h | 8 ++ sysdeps/x86_64/nptl/rseq-access.h | 79 ++++++++++++++++++++ 4 files changed, 243 insertions(+) create mode 100644 sysdeps/i386/nptl/rseq-access.h create mode 100644 sysdeps/nptl/rseq-access.h create mode 100644 sysdeps/x86_64/nptl/rseq-access.h diff --git a/sysdeps/i386/nptl/rseq-access.h b/sysdeps/i386/nptl/rseq-access.h new file mode 100644 index 0000000000..1ddb0219c0 --- /dev/null +++ b/sysdeps/i386/nptl/rseq-access.h @@ -0,0 +1,98 @@ +/* RSEQ_* accessors. i386 version. + Copyright (C) 2002-2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define __RSEQ_GETMEM(member) \ + ({ __typeof (RSEQ_SELF()->member) __value; \ + if (sizeof (__value) == 1) \ + asm volatile ("movb %%gs:%P2(%3),%b0" \ + : "=q" (__value) \ + : "0" (0), "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + else if (sizeof (__value) == 4) \ + asm volatile ("movl %%gs:%P1(%2),%0" \ + : "=r" (__value) \ + : "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + else /* 8 */ \ + { \ + asm volatile ("movl %%gs:%P1(%2),%%eax\n\t" \ + "movl %%gs:4+%P1(%2),%%edx" \ + : "=&A" (__value) \ + : "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + } \ + __value; }) + +/* Read member of the RSEQ area directly. */ +#define RSEQ_GETMEM(member) \ + ({ \ + _Static_assert (sizeof (RSEQ_SELF()->member) == 1 \ + || sizeof (RSEQ_SELF()->member) == 4 \ + || sizeof (RSEQ_SELF()->member) == 8, \ + "size of rseq data"); \ + __RSEQ_GETMEM(member); }) + +/* Read member of the RSEQ area directly, with single-copy atomicity semantics, + hence the 'volatile'. Static assert for types >= 64 bits since they can't be + loaded atomically on x86-32. */ +#define RSEQ_GETMEM_VOLATILE(member) \ + ({ \ + _Static_assert (sizeof (RSEQ_SELF()->member) == 1 \ + || sizeof (RSEQ_SELF()->member) == 4, \ + "size of rseq data"); \ + __RSEQ_GETMEM(member); }) + +#define __RSEQ_SETMEM(member, value) \ + ({ \ + if (sizeof (RSEQ_SELF()->member) == 1) \ + asm volatile ("movb %b0,%%gs:%P1(%2)" : \ + : "iq" (value), \ + "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + else if (sizeof (RSEQ_SELF()->member) == 4) \ + asm volatile ("movl %0,%%gs:%P1(%2)" : \ + : "ir" (value), \ + "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + else /* 8 */ \ + { \ + asm volatile ("movl %%eax,%%gs:%P1(%2)\n\t" \ + "movl %%edx,%%gs:4+%P1(%2)" : \ + : "A" ((uint64_t) cast_to_integer (value)), \ + "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + }}) + +/* Set member of the RSEQ area directly. */ +#define RSEQ_SETMEM(member, value) \ + ({ \ + _Static_assert (sizeof (RSEQ_SELF()->member) == 1 \ + || sizeof (RSEQ_SELF()->member) == 4 \ + || sizeof (RSEQ_SELF()->member) == 8, \ + "size of rseq data"); \ + __RSEQ_SETMEM(member, value); }) + +/* Set member of the RSEQ area directly, with single-copy atomicity semantics, + hence the 'volatile'. Static assert for types >= 64 bits since they can't be + stored atomically on x86-32. */ +#define RSEQ_SETMEM_VOLATILE(member, value) \ + ({ \ + _Static_assert (sizeof (RSEQ_SELF()->member) == 1 \ + || sizeof (RSEQ_SELF()->member) == 4, \ + "size of rseq data"); \ + __RSEQ_SETMEM(member, value); }) diff --git a/sysdeps/nptl/rseq-access.h b/sysdeps/nptl/rseq-access.h new file mode 100644 index 0000000000..1681a53ebf --- /dev/null +++ b/sysdeps/nptl/rseq-access.h @@ -0,0 +1,58 @@ +/* RSEQ_* accessors. Generic version. + Copyright (C) 2002-2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +/* Read member of the RSEQ area directly. */ +#define RSEQ_GETMEM(member) \ + RSEQ_SELF()->member + +/* Set member of the RSEQ area directly. */ +#define RSEQ_SETMEM(member, value) \ + RSEQ_SELF()->member = (value) + +/* Static assert for types that can't be loaded/stored atomically on the + current architecture. */ +#if __HAVE_64B_ATOMICS +#define __RSEQ_ASSERT_ATOMIC(member) \ + _Static_assert (sizeof (RSEQ_SELF()->member) == 1 \ + || sizeof (RSEQ_SELF()->member) == 4 \ + || sizeof (RSEQ_SELF()->member) == 8, \ + "size of rseq data") +#else +#define __RSEQ_ASSERT_ATOMIC(member) \ + _Static_assert (sizeof (RSEQ_SELF()->member) == 1 \ + || sizeof (RSEQ_SELF()->member) == 4, \ + "size of rseq data") +#endif + +/* Read member of the RSEQ area directly, with single-copy atomicity semantics, + hence the 'volatile'. */ +#define RSEQ_GETMEM_VOLATILE(member) \ + ({ \ + __RSEQ_ASSERT_ATOMIC(member); \ + (*(volatile __typeof (RSEQ_SELF()->member) *)&RSEQ_SELF()->member); \ + }) + +/* Set member of the RSEQ area directly, with single-copy atomicity semantics, + hence the 'volatile'. */ +#define RSEQ_SETMEM_VOLATILE(member, value) \ + ({ \ + __RSEQ_ASSERT_ATOMIC(member); \ + (*(volatile __typeof (RSEQ_SELF()->member) *)&RSEQ_SELF()->member = (value)); \ + }) diff --git a/sysdeps/unix/sysv/linux/rseq-internal.h b/sysdeps/unix/sysv/linux/rseq-internal.h index 30b3d5bb19..8e4a292ffc 100644 --- a/sysdeps/unix/sysv/linux/rseq-internal.h +++ b/sysdeps/unix/sysv/linux/rseq-internal.h @@ -25,6 +25,7 @@ #include #include #include +#include /* Minimum size of the rseq area allocation required by the syscall. The actually used rseq feature size may be less (20 bytes initially). */ @@ -59,6 +60,13 @@ extern ptrdiff_t _rseq_offset attribute_hidden; rtld_hidden_proto (__rseq_size) rtld_hidden_proto (__rseq_offset) +/* Returns a pointer to the current thread rseq area. */ +static inline struct rseq_area * +RSEQ_SELF (void) +{ + return (struct rseq_area *) ((char *) __thread_pointer () + __rseq_offset); +} + #ifdef RSEQ_SIG static inline bool rseq_register_current_thread (struct pthread *self, bool do_rseq) diff --git a/sysdeps/x86_64/nptl/rseq-access.h b/sysdeps/x86_64/nptl/rseq-access.h new file mode 100644 index 0000000000..fad215bccf --- /dev/null +++ b/sysdeps/x86_64/nptl/rseq-access.h @@ -0,0 +1,79 @@ +/* RSEQ_* accessors. x86_64 version. + Copyright (C) 2002-2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Read member of the RSEQ area directly, with single-copy atomicity semantics, + hence the 'volatile'. */ +#define RSEQ_GETMEM_VOLATILE(member) \ + ({ __typeof (RSEQ_SELF()->member) __value; \ + _Static_assert (sizeof (__value) == 1 \ + || sizeof (__value) == 4 \ + || sizeof (__value) == 8, \ + "size of rseq data"); \ + if (sizeof (__value) == 1) \ + asm volatile ("movb %%fs:%P2(%q3),%b0" \ + : "=q" (__value) \ + : "0" (0), "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + else if (sizeof (__value) == 4) \ + asm volatile ("movl %%fs:%P1(%q2),%0" \ + : "=r" (__value) \ + : "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + else /* 8 */ \ + { \ + asm volatile ("movq %%fs:%P1(%q2),%q0" \ + : "=r" (__value) \ + : "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + } \ + __value; }) + +/* Read member of the RSEQ area directly. */ +#define RSEQ_GETMEM(member) RSEQ_GETMEM_VOLATILE(member) + +/* Set member of the RSEQ area directly, with single-copy atomicity semantics, + hence the 'volatile'. */ +#define RSEQ_SETMEM_VOLATILE(member, value) \ + ({ \ + _Static_assert (sizeof (RSEQ_SELF()->member) == 1 \ + || sizeof (RSEQ_SELF()->member) == 4 \ + || sizeof (RSEQ_SELF()->member) == 8, \ + "size of rseq data"); \ + if (sizeof (RSEQ_SELF()->member) == 1) \ + asm volatile ("movb %b0,%%fs:%P1(%q2)" : \ + : "iq" (value), \ + "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + else if (sizeof (RSEQ_SELF()->member) == 4) \ + asm volatile ("movl %0,%%fs:%P1(%q2)" : \ + : IMM_MODE (value), \ + "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + else /* 8 */ \ + { \ + /* Since movq takes a signed 32-bit immediate or a register source \ + operand, use "er" constraint for 32-bit signed integer constant \ + or register. */ \ + asm volatile ("movq %q0,%%fs:%P1(%q2)" : \ + : "er" ((uint64_t) cast_to_integer (value)), \ + "i" (offsetof (struct rseq_area, member)), \ + "r" (__rseq_offset)); \ + }}) + +/* Set member of the RSEQ area directly. */ +#define RSEQ_SETMEM(member, value) RSEQ_SETMEM_VOLATILE(member, value)