From patchwork Wed Oct 23 08:38:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99360 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 716A23858417 for ; Wed, 23 Oct 2024 08:40:47 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 322DB3858D21 for ; Wed, 23 Oct 2024 08:40:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 322DB3858D21 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 322DB3858D21 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672817; cv=none; b=WW1FOjOJVBFm4PH5hc7oE679H4t6AF5zNiCPX1nrG9DFFC6KxUetKgv+ImLJ3ltQ8aoUlpk2QzeP7ARsRSl6VL9iRMIjFlMMRTxF14aWGAluhZQfZ6fV9YWASh6p+opSEnI0ZuZg7cZ+9yy19aCNif2BesloeaxblMGPr96c3vs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672817; c=relaxed/simple; bh=D7kcmnnzny2hSXoLS33QBll/LR5PvMsXossNFquFaVk=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=hqDQEIwlEU0ygS+KORSQzR9jyURRZU0Ul+0BZVyoVQglQGWaUVzf+IA3wQAw00spmOHbq3MCr7eAn8iABXVCPOM8E0vIf7Sj67DowbmDmd5PcOITy1ewf7gUhDdaWBn5SrsgthXM38Z/7cgLAllz4Xmsh1zzeDCCPhNThk3f/HE= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A3EF0497; Wed, 23 Oct 2024 01:40:38 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4BAA83F528; Wed, 23 Oct 2024 01:40:08 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 01/23] aarch64: Add HWCAP_GCS Date: Wed, 23 Oct 2024 09:38:58 +0100 Message-Id: <20241023083920.466015-2-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-10.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Use upper 32 bits of HWCAP. --- sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h | 1 + 1 file changed, 1 insertion(+) diff --git a/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h b/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h index 8dceaa1a52..9c4f4f46d1 100644 --- a/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h +++ b/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h @@ -54,6 +54,7 @@ #define HWCAP_SB (1 << 29) #define HWCAP_PACA (1 << 30) #define HWCAP_PACG (1UL << 31) +#define HWCAP_GCS (1UL << 32) #define HWCAP2_DCPODP (1 << 0) #define HWCAP2_SVE2 (1 << 1) From patchwork Wed Oct 23 08:38:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99361 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6F7323858C53 for ; Wed, 23 Oct 2024 08:41:07 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id EB17E3858403 for ; Wed, 23 Oct 2024 08:40:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org EB17E3858403 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org EB17E3858403 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672835; cv=none; b=QwO3ETPD2niGUNTjeOMElP7gOwQFj+Iq3rvUvKPvzECOKAQUDBOYazaRKUgZQnvkkhHT7GUncrWID5xm9scDeUZfAD7aM+5aoLLaC1IFVcbXSLV74KInquugLG50yed5dJzF65pWe5OX0UJ06ErFZYOkzSQwi0sRlmHaDFduujk= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672835; c=relaxed/simple; bh=XmgfFpWY/RBQCwbEzagDnN6jiag+6zt5EChv6hSEumk=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=VUXEDCR7TogJYAy/QseXjVSsxzW8YYtWUpxgwIcCsL475rTAf8grzkpsAUiDZcjCbrNpdw3a1zK7hY+eAOgKQmbnV60cmgtnhXOCK5CdfTyoVv7DZ2HJjjcUXXkzExHTPdRJ17hnwoqzHmcnZP4w9ch97+HhK8DZNDZXgAJEJAU= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6BDCC152B; Wed, 23 Oct 2024 01:41:02 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 13E363F528; Wed, 23 Oct 2024 01:40:31 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 02/23] aarch64: Add asm helpers for GCS Date: Wed, 23 Oct 2024 09:38:59 +0100 Message-Id: <20241023083920.466015-3-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy The Guarded Control Stack instructions can be present even if the hardware does not support the extension (runtime checked feature), so the asm code should be backward compatible with old assemblers. --- sysdeps/aarch64/sysdep.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/sysdeps/aarch64/sysdep.h b/sysdeps/aarch64/sysdep.h index 464674e89c..661d9fe8c1 100644 --- a/sysdeps/aarch64/sysdep.h +++ b/sysdeps/aarch64/sysdep.h @@ -74,6 +74,13 @@ strip_pac (void *p) #define PACIASP hint 25 #define AUTIASP hint 29 +/* Guarded Control Stack support. */ +#define CHKFEAT_X16 hint 40 +#define MRS_GCSPR(x) mrs x, s3_3_c2_c5_1 +#define GCSPOPM(x) sysl x, #3, c7, c7, #1 +#define GCSSS1(x) sys #3, c7, c7, #2, x +#define GCSSS2(x) sysl x, #3, c7, c7, #3 + /* GNU_PROPERTY_AARCH64_* macros from elf.h for use in asm code. */ #define FEATURE_1_AND 0xc0000000 #define FEATURE_1_BTI 1 From patchwork Wed Oct 23 08:39:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99362 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6B49C3858C56 for ; Wed, 23 Oct 2024 08:41:27 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id D6F663858C39 for ; Wed, 23 Oct 2024 08:40:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D6F663858C39 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D6F663858C39 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672856; cv=none; b=JYwK72/ZppkO7D3bIAlxrfl3rbeE/XrCdYyCOGXiYLUUq6TIT0nz5z6ZLtMLJHIQGbo9hUsvtEniaa/RLVPTof/dz4/0la1GpPY9KL1aJpqVanwSgmgDmQeSf0KwzlCF61FudZ3YI3R78WnL2TT52as7FiafAf+JRb2esicvcdw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672856; c=relaxed/simple; bh=almvxLNCLNePzePUIYPbiJ4IJxi8uURzggirX9JF8dM=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=SuD7ntdfokaqFa8bdvRLojGNeKrMGX2zrC8sQcRARy9Nx1ugi8Akv1LlH6qGxNkG6CseZLrIqHOOahZZ3eTwVMaU5e8LKyEocvAQUU+f+4AcArzEt8ExTE+r64Rd12rG2bBrD9iOJL8KPemPedGQ89PBkTmaptD4b3X62toNdvQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5F0701570; Wed, 23 Oct 2024 01:41:20 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 065E03F528; Wed, 23 Oct 2024 01:40:49 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 03/23] elf.h: Define GNU_PROPERTY_AARCH64_FEATURE_1_GCS Date: Wed, 23 Oct 2024 09:39:00 +0100 Message-Id: <20241023083920.466015-4-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy --- elf/elf.h | 1 + 1 file changed, 1 insertion(+) diff --git a/elf/elf.h b/elf/elf.h index 33aea7f743..970c3daaec 100644 --- a/elf/elf.h +++ b/elf/elf.h @@ -1389,6 +1389,7 @@ typedef struct #define GNU_PROPERTY_AARCH64_FEATURE_1_BTI (1U << 0) #define GNU_PROPERTY_AARCH64_FEATURE_1_PAC (1U << 1) +#define GNU_PROPERTY_AARCH64_FEATURE_1_GCS (1U << 2) /* The x86 instruction sets indicated by the corresponding bits are used in program. Their support in the hardware is optional. */ From patchwork Wed Oct 23 08:39:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99363 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C2F9B3858C56 for ; Wed, 23 Oct 2024 08:42:02 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id F21383858410 for ; Wed, 23 Oct 2024 08:41:02 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org F21383858410 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org F21383858410 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672866; cv=none; b=a56+q7AMjb47MsCqP0W53hyRI6tl+KY2e7dlVWhHjh0tFaZMOXmIIcOEfonYVE5FsQMybz2Vr3IzwDKB0Pa2myP3YoLSBprf0s8Kny7+CAyCoDXYHShbLT5EIO6wHFc23zj/zK4lQ218YzcuzEl6Wb2A6SceYFgiR74OPfvLHvo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672866; c=relaxed/simple; bh=rcZgcUEx2HP55xMYHTg1ZE/3c/0PaA6vDeZ6t6iGtWQ=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=Vj+qeHWLgkdrYAakXcheQHFejjIRRPRXwfuBB0noEmwZUhMDsTzNe7f1FHgVNCctIf0g9axY8/VC670cwZemP6FQkxRgid/M0Pm7RdUz34rQsCcq9kGuu2p3fcH/E9rP0SGndG0uBmBMFTB3juxGP5FhIUj+mBo824suWXfPKGc= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6960B1575; Wed, 23 Oct 2024 01:41:32 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0F69F3F528; Wed, 23 Oct 2024 01:41:01 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 04/23] aarch64: Define jmp_buf offset for GCS Date: Wed, 23 Oct 2024 09:39:01 +0100 Message-Id: <20241023083920.466015-5-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy The target specific internal __longjmp is called with a __jmp_buf argument which has its size exposed in the ABI. On aarch64 this has no space left, so GCSPR cannot be restored in longjmp in the usual way, which is needed for the Guarded Control Stack (GCS) extension. setjmp is implemented via __sigsetjmp which has a jmp_buf argument however it is also called with __pthread_unwind_buf_t argument cast to jmp_buf (in cancellation cleanup code built with -fno-exception). The two types, jmp_buf and __pthread_unwind_buf_t, have common bits beyond the __jmp_buf field and there is unused space there which we can use for saving GCSPR. For this to work some bits of those two generic types have to be reserved for target specific use and the generic code in glibc has to ensure that __longjmp is always called with a __jmp_buf that is embedded into one of those two types. Morally __longjmp should be changed to take jmp_buf as argument, but that is an intrusive change across targets. Note: longjmp is never called with __pthread_unwind_buf_t from user code, only the internal __libc_longjmp is called with that type and thus the two types could have separate longjmp implementations on a target. We don't rely on this now (but migh in the future given that cancellation unwind does not need to restore GCSPR). Given the above this patch finds an unused slot for GCSPR. This placement is not exposed in the ABI so it may change in the future. This is also very target ABI specific so the generic types cannot be easily changed to clearly mark the reserved fields. --- sysdeps/aarch64/jmpbuf-offsets.h | 63 ++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/sysdeps/aarch64/jmpbuf-offsets.h b/sysdeps/aarch64/jmpbuf-offsets.h index 632328c7e2..ec047cf6b1 100644 --- a/sysdeps/aarch64/jmpbuf-offsets.h +++ b/sysdeps/aarch64/jmpbuf-offsets.h @@ -39,6 +39,69 @@ #define JB_D14 20 #define JB_D15 21 +/* The target specific part of jmp_buf has no space for expansion but + the public jmp_buf ABI type has. Unfortunately there is another type + that is used with setjmp APIs and exposed by thread cancellation (in + binaries built with -fno-exceptions) which complicates the situation. + + // Internal layout of the public jmp_buf type on AArch64. + // This is passed to setjmp, longjmp, sigsetjmp, siglongjmp. + struct + { + uint64_t jmpbuf[22]; // Target specific part. + uint32_t mask_was_saved; // savemask bool used by sigsetjmp/siglongjmp. + uint32_t pad; + uint64_t saved_mask; // sigset_t bits used on linux. + uint64_t unused[15]; // sigset_t bits not used on linux. + }; + + // Internal layout of the public __pthread_unwind_buf_t type. + // This is passed to sigsetjmp with !savemask and to the internal + // __libc_longjmp (currently alias of longjmp on AArch64). + struct + { + uint64_t jmpbuf[22]; // Must match jmp_buf. + uint32_t mask_was_saved; // Must match jmp_buf, always 0. + uint32_t pad; + void *prev; // List for unwinding. + void *cleanup; // Cleanup handlers. + uint32_t canceltype; // 1 bit cancellation type. + uint32_t pad2; + void *pad3; + }; + + Ideally only the target specific part of jmp_buf (A) is accessed by + __setjmp and __longjmp. But that is always embedded into one of the + two types above so the bits that are unused in those types (B) may be + reused for target specific purposes. Setjmp can't distinguish between + jmp_buf and __pthread_unwind_buf_t, but longjmp can: only an internal + longjmp call uses the latter, so state that is not needed for cancel + cleanups can go to fields (C). If generic code is refactored then the + usage of additional fields can be optimized (D). And some fields are + only accessible in the savedmask case (E). Reusability of jmp_buf + fields on AArch64 for target purposes: + + struct + { + uint64_t A[22]; // 0 .. 176 + uint32_t D; // 176 .. 180 + uint32_t B; // 180 .. 184 + uint64_t D; // 184 .. 192 + uint64_t C; // 192 .. 200 + uint32_t C; // 200 .. 204 + uint32_t B; // 204 .. 208 + uint64_t B; // 208 .. 216 + uint64_t E[12]; // 216 .. 312 + } + + The B fields can be used with minimal glibc code changes. We need a + 64 bit field for the Guarded Control Stack pointer (GCSPR_EL0) which + can use a C field too as cancellation cleanup does not execute RET + for a previous BL of the cancelled thread, but that would require a + custom __libc_longjmp. This layout can change in the future. +*/ +#define JB_GCSPR 208 + #ifndef __ASSEMBLER__ #include #include From patchwork Wed Oct 23 08:39:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99365 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7ED7F3858408 for ; Wed, 23 Oct 2024 08:42:23 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 897A73858D28 for ; Wed, 23 Oct 2024 08:41:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 897A73858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 897A73858D28 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672885; cv=none; b=s5nxmeyMum0nKbcd5BcIsAAPyi9PzxRGfEQvPX2b//UmPOwI3C2zNibeRr/ogTAdp+iopw6aaA4jkSAky+p4WU/wXX/OsJpHObVYs0oXmw8bfFLjsvS11gDZ6BYEHfGmPZThW71ntoSexmeDvlCdGvEpeb8osmiWlbpwx04nICI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672885; c=relaxed/simple; bh=We0/sUf5uqWPoMZSKgn9OZUQ1ngusMLmCjQClHsXsBA=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=MLuyt6PyORdsbxcB9+UyeKUDSPpMMnyvs6EiLVQJC//pVtkA2XbFM0LxT2OdrjeE3rKBXi0TRqeKWfkzkmG9szFJY1IXQ5kfxz6AcFbupQw5sr20BHoyjueqxt1Xq9cgk0DPyShoh0lhAEcx6ihssEFjBExNJke+F6gv9jw8YK4= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0E0391575; Wed, 23 Oct 2024 01:41:44 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A92E63F528; Wed, 23 Oct 2024 01:41:13 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 05/23] aarch64: Add GCS support to longjmp Date: Wed, 23 Oct 2024 09:39:02 +0100 Message-Id: <20241023083920.466015-6-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy This implementations ensures that longjmp across different stacks works: it scans for GCS cap token and switches GCS if necessary then the target GCSPR is restored with a GCSPOPM loop once the current GCSPR is on the same GCS. This makes longjmp linear time in the number of jumped over stack frames when GCS is enabled. --- sysdeps/aarch64/__longjmp.S | 30 ++++++++++++++++++++++++++++++ sysdeps/aarch64/setjmp.S | 10 ++++++++++ 2 files changed, 40 insertions(+) diff --git a/sysdeps/aarch64/__longjmp.S b/sysdeps/aarch64/__longjmp.S index 7b6add751e..d8670d8995 100644 --- a/sysdeps/aarch64/__longjmp.S +++ b/sysdeps/aarch64/__longjmp.S @@ -91,6 +91,36 @@ ENTRY (__longjmp) ldp d12, d13, [x0, #JB_D12<<3] ldp d14, d15, [x0, #JB_D14<<3] + /* GCS support. */ + mov x16, 1 + CHKFEAT_X16 + tbnz x16, 0, L(gcs_done) + MRS_GCSPR (x2) + ldr x3, [x0, #JB_GCSPR] + mov x4, x3 + /* x2: GCSPR now. x3, x4: target GCSPR. x5, x6: tmp regs. */ +L(gcs_scan): + cmp x2, x4 + b.eq L(gcs_pop) + sub x4, x4, 8 + /* Check for a cap token. */ + ldr x5, [x4] + and x6, x4, 0xfffffffffffff000 + orr x6, x6, 1 + cmp x5, x6 + b.ne L(gcs_scan) +L(gcs_switch): + add x2, x4, 8 + GCSSS1 (x4) + GCSSS2 (xzr) +L(gcs_pop): + cmp x2, x3 + b.eq L(gcs_done) + GCSPOPM (xzr) + add x2, x2, 8 + b L(gcs_pop) +L(gcs_done): + /* Originally this was implemented with a series of .cfi_restore() directives. diff --git a/sysdeps/aarch64/setjmp.S b/sysdeps/aarch64/setjmp.S index 43fdb1b2fb..f7ffccfaba 100644 --- a/sysdeps/aarch64/setjmp.S +++ b/sysdeps/aarch64/setjmp.S @@ -57,6 +57,16 @@ ENTRY (__sigsetjmp) stp d10, d11, [x0, #JB_D10<<3] stp d12, d13, [x0, #JB_D12<<3] stp d14, d15, [x0, #JB_D14<<3] + + /* GCS support. */ + mov x16, 1 + CHKFEAT_X16 + tbnz x16, 0, L(gcs_done) + MRS_GCSPR (x2) + add x2, x2, 8 /* GCS state right after setjmp returns. */ + str x2, [x0, #JB_GCSPR] +L(gcs_done): + #ifdef PTR_MANGLE mov x4, sp PTR_MANGLE (5, 4, 3, 2) From patchwork Wed Oct 23 08:39:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99364 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3255F3858CDB for ; Wed, 23 Oct 2024 08:42:04 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 0AA50385840A for ; Wed, 23 Oct 2024 08:41:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0AA50385840A Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0AA50385840A Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672899; cv=none; b=R6VKqaeZTRkjr2oun3+p7MPZfDorSn6i3a+zPZu4YD+8cC6i/VjL0NRf/zY/sEygfFehaMQiwsiUJA///UumQZK7Wx0pjWTSZBRrmVqFH/qasyD9rXvvR6js3IbnH0DfAFHx+ThgBSDfYYdah0lkzNaO4y8ttmBKy3xsiBOelh0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672899; c=relaxed/simple; bh=oWXhLGXNw74nmW+2QMEMJteXb00LobTKqr1NJH8bRwE=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=TTf7GhbM0m6SvbQi4nRSZshp6VlwNG0NpXq3fI/7GiEaNxZYqAKR96Id/XUIWQ9kcuGmD17AdNg/W0Lvhj499GAGJaLKHjZTkQStzDT+HCEOi15v0qqodmACT/xh16+Rryk9b/vgl6Mp4ZnwZn/P3Km6moHsodbfVSjM1HD3M04= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 837441575; Wed, 23 Oct 2024 01:42:06 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2A8F03F528; Wed, 23 Oct 2024 01:41:36 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 06/23] aarch64: Add GCS support to vfork Date: Wed, 23 Oct 2024 09:39:03 +0100 Message-Id: <20241023083920.466015-7-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy --- sysdeps/unix/sysv/linux/aarch64/vfork.S | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/sysdeps/unix/sysv/linux/aarch64/vfork.S b/sysdeps/unix/sysv/linux/aarch64/vfork.S index e71e492da3..cfaf4a1ffb 100644 --- a/sysdeps/unix/sysv/linux/aarch64/vfork.S +++ b/sysdeps/unix/sysv/linux/aarch64/vfork.S @@ -33,8 +33,14 @@ ENTRY (__vfork) cmn x0, #4095 b.cs .Lsyscall_error + cbz x0, L(child) RET - +L(child): + /* Return with indirect branch in the child to support GCS. + Clear x30 to crash early if the child tries to ret. */ + mov x1, x30 + mov x30, 0 + br x1 PSEUDO_END (__vfork) libc_hidden_def (__vfork) From patchwork Wed Oct 23 08:39:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99366 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id AA44C385841D for ; Wed, 23 Oct 2024 08:42:44 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 1E8013858C48 for ; Wed, 23 Oct 2024 08:42:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 1E8013858C48 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 1E8013858C48 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672930; cv=none; b=ObKQhJhyrRukqU9ZMYMnSN5F0U4oaOgA2IlGTVrnvOaakuQb8WQTSOTKol41wiYKT96n56tU70Q37nO/7teItkk8NUyAzKov4BWK/w0IjExAHfjUCHNuKLWRhU4IIaFoWcXjJUNQDOTdWsqLuVZDO5Te/ApiQ+3FPYuCTIzVaMs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672930; c=relaxed/simple; bh=sSiJ2As3jnFiz+mw19mes0h5j+xMWP2O2uyo4+3NoDs=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=EsMhGVs5WYB8Cr/t3DtDLitaDMU3jngV19tup5/+nDSH7Gl1Jg1agWOxCq/VO9ErXjTzIB1eHdDwUedXcjlrsdthu6cf8eBnpQg8z0MY8x2r1fm7SvYGs2cjPu51/2nwVIcWfYip2MfkSTs+AxyswUdAzFJLZvlYFS8fwaQre2g= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 972931575; Wed, 23 Oct 2024 01:42:34 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3E54C3F528; Wed, 23 Oct 2024 01:42:04 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 07/23] aarch64: Add GCS support for setcontext Date: Wed, 23 Oct 2024 09:39:04 +0100 Message-Id: <20241023083920.466015-8-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_STOCKGEN, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Userspace ucontext needs to store GCSPR, it does not have to be compatible with the kernel ucontext. For now we use the linux struct gcs_context layout but only use the gcspr field from it. Similar implementation to the longjmp code, supports switching GCS if the target GCS is capped, and unwinding a continous GCS to a previous state. --- sysdeps/unix/sysv/linux/aarch64/getcontext.S | 17 ++++++++- sysdeps/unix/sysv/linux/aarch64/setcontext.S | 38 +++++++++++++++++++ sysdeps/unix/sysv/linux/aarch64/swapcontext.S | 32 ++++++++++++---- .../sysv/linux/aarch64/ucontext-internal.h | 5 +++ 4 files changed, 83 insertions(+), 9 deletions(-) diff --git a/sysdeps/unix/sysv/linux/aarch64/getcontext.S b/sysdeps/unix/sysv/linux/aarch64/getcontext.S index e5b69c9a82..30e2b39399 100644 --- a/sysdeps/unix/sysv/linux/aarch64/getcontext.S +++ b/sysdeps/unix/sysv/linux/aarch64/getcontext.S @@ -83,9 +83,24 @@ ENTRY(__getcontext) mrs x4, fpcr str w4, [x3, oFPCR - oFPSR] - /* Write the termination context extension header. */ add x2, x2, #FPSIMD_CONTEXT_SIZE + /* Save the GCSPR. */ + mov x16, 1 + CHKFEAT_X16 + tbnz x16, 0, L(gcs_done) + mov w3, #(GCS_MAGIC & 0xffff) + movk w3, #(GCS_MAGIC >> 16), lsl #16 + str w3, [x2, #oHEAD + oMAGIC] + mov w3, #GCS_CONTEXT_SIZE + str w3, [x2, #oHEAD + oSIZE] + MRS_GCSPR (x4) + add x4, x4, 8 /* GCS state right after getcontext returns. */ + str x4, [x2, #oGCSPR] + add x2, x2, #GCS_CONTEXT_SIZE +L(gcs_done): + + /* Write the termination context extension header. */ str wzr, [x2, #oHEAD + oMAGIC] str wzr, [x2, #oHEAD + oSIZE] diff --git a/sysdeps/unix/sysv/linux/aarch64/setcontext.S b/sysdeps/unix/sysv/linux/aarch64/setcontext.S index ba659438c5..bdfd4580ec 100644 --- a/sysdeps/unix/sysv/linux/aarch64/setcontext.S +++ b/sysdeps/unix/sysv/linux/aarch64/setcontext.S @@ -130,6 +130,44 @@ ENTRY (__setcontext) ldr w4, [x3, oFPCR - oFPSR] msr fpcr, x4 + /* Restore the GCS. */ + mov x16, 1 + CHKFEAT_X16 + tbnz x16, 0, L(gcs_done) + /* Get target GCS from GCS context. */ + ldr w1, [x2, #oHEAD + oSIZE] + add x2, x2, x1 + mov w3, #(GCS_MAGIC & 0xffff) + movk w3, #(GCS_MAGIC >> 16), lsl #16 + ldr w1, [x2, #oHEAD + oMAGIC] + cmp w1, w3 + b.ne L(gcs_done) + ldr x3, [x2, #oGCSPR] + MRS_GCSPR (x2) + mov x4, x3 + /* x2: GCSPR now. x3, x4: target GCSPR. x5, x6: tmp regs. */ +L(gcs_scan): + cmp x2, x4 + b.eq L(gcs_pop) + sub x4, x4, 8 + /* Check for a cap token. */ + ldr x5, [x4] + and x6, x4, 0xfffffffffffff000 + orr x6, x6, 1 + cmp x5, x6 + b.ne L(gcs_scan) +L(gcs_switch): + add x2, x4, 8 + GCSSS1 (x4) + GCSSS2 (xzr) +L(gcs_pop): + cmp x2, x3 + b.eq L(gcs_done) + GCSPOPM (xzr) + add x2, x2, 8 + b L(gcs_pop) +L(gcs_done): + 2: ldr x16, [x0, oPC] /* Restore arg registers. */ diff --git a/sysdeps/unix/sysv/linux/aarch64/swapcontext.S b/sysdeps/unix/sysv/linux/aarch64/swapcontext.S index f049140d35..45b1277c74 100644 --- a/sysdeps/unix/sysv/linux/aarch64/swapcontext.S +++ b/sysdeps/unix/sysv/linux/aarch64/swapcontext.S @@ -32,8 +32,15 @@ ENTRY(__swapcontext) And set up x1 to become the return address of the caller, so we can return there with a normal RET instead of an indirect jump. */ stp xzr, x30, [x0, oX0 + 0 * SZREG] + + /* With GCS, swapcontext calls are followed by BTI J, otherwise + we have to be compatible with old BTI enabled binaries. */ + mov x16, 1 + CHKFEAT_X16 + tbz x16, 0, L(skip_x30_redirect) /* Arrange the oucp context to return to 2f. */ adr x30, 2f +L(skip_x30_redirect): stp x18, x19, [x0, oX0 + 18 * SZREG] stp x20, x21, [x0, oX0 + 20 * SZREG] @@ -72,14 +79,27 @@ ENTRY(__swapcontext) mrs x4, fpcr str w4, [x3, #oFPCR - oFPSR] - /* Write the termination context extension header. */ add x2, x2, #FPSIMD_CONTEXT_SIZE + /* Save the GCSPR. */ + tbnz x16, 0, L(gcs_done) + mov w3, #(GCS_MAGIC & 0xffff) + movk w3, #(GCS_MAGIC >> 16), lsl #16 + str w3, [x2, #oHEAD + oMAGIC] + mov w3, #GCS_CONTEXT_SIZE + str w3, [x2, #oHEAD + oSIZE] + MRS_GCSPR (x4) + add x4, x4, 8 /* GCSPR of the caller. */ + str x4, [x2, #oGCSPR] + add x2, x2, #GCS_CONTEXT_SIZE +L(gcs_done): + + /* Write the termination context extension header. */ str wzr, [x2, #oHEAD + oMAGIC] str wzr, [x2, #oHEAD + oSIZE] /* Preserve ucp. */ - mov x21, x1 + mov x9, x1 /* rt_sigprocmask (SIG_SETMASK, &ucp->uc_sigmask, &oucp->uc_sigmask, _NSIG8) */ @@ -93,12 +113,8 @@ ENTRY(__swapcontext) svc 0 cbnz x0, 1f - mov x22, x30 - mov x0, x21 - bl JUMPTARGET (__setcontext) - mov x30, x22 - RET - + mov x0, x9 + b JUMPTARGET (__setcontext) 1: b C_SYMBOL_NAME(__syscall_error) 2: diff --git a/sysdeps/unix/sysv/linux/aarch64/ucontext-internal.h b/sysdeps/unix/sysv/linux/aarch64/ucontext-internal.h index 096d5fb7c7..84f5365c0e 100644 --- a/sysdeps/unix/sysv/linux/aarch64/ucontext-internal.h +++ b/sysdeps/unix/sysv/linux/aarch64/ucontext-internal.h @@ -43,3 +43,8 @@ #define oX21 (oX0 + 21*8) #define oFP (oX0 + 29*8) #define oLR (oX0 + 30*8) + +/* Use kernel layout for saving GCSPR in ucontext. */ +#define GCS_MAGIC 0x47435300 +#define GCS_CONTEXT_SIZE 32 +#define oGCSPR 8 From patchwork Wed Oct 23 08:39:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99367 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 12AD1385842C for ; Wed, 23 Oct 2024 08:43:14 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 45E2B3858C35 for ; Wed, 23 Oct 2024 08:42:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 45E2B3858C35 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 45E2B3858C35 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672948; cv=none; b=PACXlyFIZ6k1ejkyZLJPtfofAodZ9EKwiuhAsgnxgEMx5SXqtVctHrvfcpfrQLaJWnr0/59PDdQ4w1QJytPsIuNQmOoDGRHTlacxhY5xsSjXHBX4WXKeZYyeWoxLPbmGUxXr6yPvMT9JqJTQC3Y51X2SZznfyFf/AkQ2WAfI+Qo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672948; c=relaxed/simple; bh=drjrkgcPtoTucmHisp6+M/HTlfkjgOi9EKNnAeY2aIg=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=VB6P2UFP/YtY7hutJFFrqKYutSHZcx32h3/ZmVauqQpujYRWHDnyBCvr7zoBSjT6UAdq6UUQsTgNDa2J+3QFflpSt8v96wlQeaFGWiz+lHQFuMTCkErYrqH/0J7x1LyUIMYNw7sstC9fkLyFyH+35BL2JzvkCLJADBuWyaJ0trw= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C1D0C1575; Wed, 23 Oct 2024 01:42:52 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 66C363F528; Wed, 23 Oct 2024 01:42:22 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 08/23] aarch64: Mark swapcontext with indirect_return Date: Wed, 23 Oct 2024 09:39:05 +0100 Message-Id: <20241023083920.466015-9-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.0 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy --- sysdeps/aarch64/bits/indirect-return.h | 36 ++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 sysdeps/aarch64/bits/indirect-return.h diff --git a/sysdeps/aarch64/bits/indirect-return.h b/sysdeps/aarch64/bits/indirect-return.h new file mode 100644 index 0000000000..99905f21fa --- /dev/null +++ b/sysdeps/aarch64/bits/indirect-return.h @@ -0,0 +1,36 @@ +/* Definition of __INDIRECT_RETURN. AArch64 version. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _UCONTEXT_H +# error "Never include directly; use instead." +#endif + +/* __INDIRECT_RETURN indicates that swapcontext may return via + an indirect branch. This happens when GCS is enabled, so + add the attribute if available, otherwise returns_twice has + a similar effect, but it prevents some code transformations + that can cause build failures in some rare cases so it is + only used when GCS is enabled. */ +#if __glibc_has_attribute (__indirect_return__) +# define __INDIRECT_RETURN __attribute__ ((__indirect_return__)) +#elif __glibc_has_attribute (__returns_twice__) \ + && defined __ARM_FEATURE_GCS_DEFAULT +# define __INDIRECT_RETURN __attribute__ ((__returns_twice__)) +#else +# define __INDIRECT_RETURN +#endif From patchwork Wed Oct 23 08:39:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99368 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D2B923858CDB for ; Wed, 23 Oct 2024 08:43:22 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 5CE153858D33 for ; Wed, 23 Oct 2024 08:42:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5CE153858D33 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5CE153858D33 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672963; cv=none; b=uZBARL+JiKpFSupjQthZElHktqIsS3wL79/5bwh22vEQVomesTDL7b8++Lu7Up+4g2mf56NxFH5wdI5k2VVrWvkHEW8pXvSLpPF97WDm9qW8Y039lGHgeyWV01tNosG0E9VZ8Gc9BTKW3ZB1TjGYPUZ8SL5R2d8RTI4xt32Gnaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672963; c=relaxed/simple; bh=h0+6ZoJAcZf89DCarO6jLLzUW6MqqkA+fFTK16tAZRc=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=bxSxyRy5bYGWepsqd0WYoEobdC0HgRo0k9aQpO16jalQYTr2ktv28b4nNb/AGWZRkF/fZYdzn3PJtyEYs8dGWoNrnjv02cnkaU0rMSbZmNlmTocY41kcKKeewdwb7ZpSsoVnGkm8E3SnhkaZaYeZYsFAjvyorYywRqY4YkLSHKI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CF6AC1575; Wed, 23 Oct 2024 01:43:08 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 74DB53F528; Wed, 23 Oct 2024 01:42:38 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 09/23] aarch64: Add GCS support for makecontext Date: Wed, 23 Oct 2024 09:39:06 +0100 Message-Id: <20241023083920.466015-10-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Changed the makecontext logic: previously the first setcontext jumped straight to the user callback function and the return address is set to __startcontext. This does not work when GCS is enabled as the integrity of the return address is protected, so instead the context is setup such that setcontext jumps to __startcontext which calls the user callback (passed in x20). The map_shadow_stack syscall is used to allocate a suitably sized GCS (which includes some reserved area to account for altstack signal handlers and otherwise supports maximum number of 16 byte aligned stack frames on the given stack) however the GCS is never freed as the lifetime of ucontext and related stack is user managed. --- sysdeps/unix/sysv/linux/aarch64/makecontext.c | 61 ++++++++++++++++++- sysdeps/unix/sysv/linux/aarch64/setcontext.S | 4 ++ 2 files changed, 63 insertions(+), 2 deletions(-) diff --git a/sysdeps/unix/sysv/linux/aarch64/makecontext.c b/sysdeps/unix/sysv/linux/aarch64/makecontext.c index 5aec182013..9e66b6761c 100644 --- a/sysdeps/unix/sysv/linux/aarch64/makecontext.c +++ b/sysdeps/unix/sysv/linux/aarch64/makecontext.c @@ -22,6 +22,52 @@ #include #include +#define GCS_MAGIC 0x47435300 + +static struct _aarch64_ctx *extension (void *p) +{ + return p; +} + +#ifndef __NR_map_shadow_stack +# define __NR_map_shadow_stack 453 +#endif +#ifndef SHADOW_STACK_SET_TOKEN +# define SHADOW_STACK_SET_TOKEN (1UL << 0) +# define SHADOW_STACK_SET_MARKER (1UL << 1) +#endif + +static void * +map_shadow_stack (void *addr, size_t size, unsigned long flags) +{ + return (void *) INLINE_SYSCALL_CALL (map_shadow_stack, addr, size, flags); +} + +#define GCS_MAX_SIZE (1UL << 31) +#define GCS_ALTSTACK_RESERVE 160 + +static void * +alloc_makecontext_gcs (size_t stack_size) +{ + size_t size = (stack_size / 2 + GCS_ALTSTACK_RESERVE) & -8UL; + if (size > GCS_MAX_SIZE) + size = GCS_MAX_SIZE; + + unsigned long flags = SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN; + void *base = map_shadow_stack (NULL, size, flags); + if (base == (void *) -1) + /* ENOSYS, bad size or OOM. */ + abort (); + uint64_t *gcsp = (uint64_t *) ((char *) base + size); + /* Skip end of GCS token. */ + gcsp--; + /* Verify GCS cap token. */ + gcsp--; + if (((uint64_t)gcsp & 0xfffffffffffff000) + 1 != *gcsp) + abort (); + /* Return the target GCS pointer for context switch. */ + return gcsp + 1; +} /* makecontext sets up a stack and the registers for the user context. The stack looks like this: @@ -56,10 +102,21 @@ __makecontext (ucontext_t *ucp, void (*func) (void), int argc, ...) sp = (uint64_t *) (((uintptr_t) sp) & -16L); ucp->uc_mcontext.regs[19] = (uintptr_t) ucp->uc_link; + ucp->uc_mcontext.regs[20] = (uintptr_t) func; ucp->uc_mcontext.sp = (uintptr_t) sp; - ucp->uc_mcontext.pc = (uintptr_t) func; + ucp->uc_mcontext.pc = (uintptr_t) __startcontext; ucp->uc_mcontext.regs[29] = (uintptr_t) 0; - ucp->uc_mcontext.regs[30] = (uintptr_t) &__startcontext; + ucp->uc_mcontext.regs[30] = (uintptr_t) 0; + + void *p = ucp->uc_mcontext.__reserved; + if (extension (p)->magic == FPSIMD_MAGIC) + p = (char *)p + extension (p)->size; + if (extension (p)->magic == GCS_MAGIC) + { + /* Using the kernel struct gcs_context layout. */ + struct { uint64_t x, gcspr, y, z; } *q = p; + q->gcspr = (uint64_t) alloc_makecontext_gcs (ucp->uc_stack.ss_size); + } va_start (ap, argc); for (i = 0; i < argc; ++i) diff --git a/sysdeps/unix/sysv/linux/aarch64/setcontext.S b/sysdeps/unix/sysv/linux/aarch64/setcontext.S index bdfd4580ec..17b0a48ce1 100644 --- a/sysdeps/unix/sysv/linux/aarch64/setcontext.S +++ b/sysdeps/unix/sysv/linux/aarch64/setcontext.S @@ -180,7 +180,11 @@ L(gcs_done): PSEUDO_END (__setcontext) weak_alias (__setcontext, setcontext) +/* makecontext start function: receives uc_link in x19 and func in x20. + Arguments of func, x29, x30 and sp are set up by the caller. */ ENTRY (__startcontext) + cfi_undefined (x30) + blr x20 mov x0, x19 cbnz x0, __setcontext 1: b HIDDEN_JUMPTARGET (exit) From patchwork Wed Oct 23 08:39:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99370 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B595D385840E for ; Wed, 23 Oct 2024 08:44:29 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 877603858404 for ; Wed, 23 Oct 2024 08:42:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 877603858404 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 877603858404 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672977; cv=none; b=XF/ww4fYc5BbinNa39O3Ik4lP5AD0ah2mg/cKUFG38IGsXn7MriVdauWHMwik1s6TAX1QXGwyKu0s5HN0pECtIviBhzTXDkfY5LtiC1orTsBXwAtFXCkosZvsiKm9RLtHHIhCxkoSNmMbil/Mjooj9IObVfSUBDY/rd1nZXqXs8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672977; c=relaxed/simple; bh=uPmk9Otp4kOQ19bCTmHJS3sAyZyI84rJLaqPZBIRam8=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=LQw9ysY+ch+VyyFZ4NG4pmEU5iIZ3EWDPH8bIjOHqpFLEGL2rX+LpVg+RfyYWkedZn9lp4jbeLDVRaHSs/VfdVrPHr27Vum4VlUhyQoUzFo+qsq680U+dHd4b8pDfdMiGYKhVQXc9/JvwYfjb9UpfPbJ6e/3Uh1qi/twNVL2W64= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0BC3C1575; Wed, 23 Oct 2024 01:43:24 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A56A93F528; Wed, 23 Oct 2024 01:42:53 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 10/23] aarch64: Try to free the GCS of makecontext Date: Wed, 23 Oct 2024 09:39:07 +0100 Message-Id: <20241023083920.466015-11-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Free GCS after a makecontext start func returns and at thread exit, so assume makecontext cannot outlive the thread where it was created. This is an attempt to bound the lifetime of the GCS allocated for makecontext, but it is still possible to have significant GCS leaks, new GCS aware APIs could solve that, but that would not allow using GCS with existing code transparently. --- include/set-freeres.h | 4 ++ malloc/thread-freeres.c | 3 + sysdeps/unix/sysv/linux/aarch64/makecontext.c | 65 +++++++++++++++++++ sysdeps/unix/sysv/linux/aarch64/setcontext.S | 19 +++++- sysdeps/unix/sysv/linux/aarch64/sysdep.h | 6 +- 5 files changed, 93 insertions(+), 4 deletions(-) diff --git a/include/set-freeres.h b/include/set-freeres.h index 4177b453fa..c3d64b4f41 100644 --- a/include/set-freeres.h +++ b/include/set-freeres.h @@ -78,6 +78,10 @@ extern void __nss_database_freeres (void) attribute_hidden; extern int _IO_cleanup (void) attribute_hidden;; /* From dlfcn/dlerror.c */ extern void __libc_dlerror_result_free (void) attribute_hidden; +/* From libc.so, arch specific. */ +#ifdef ARCH_THREAD_FREERES +extern void ARCH_THREAD_FREERES (void) attribute_hidden; +#endif /* From either libc.so or libpthread.so */ extern void __libpthread_freeres (void) attribute_hidden; diff --git a/malloc/thread-freeres.c b/malloc/thread-freeres.c index 55ba4e7b83..69867f3a3b 100644 --- a/malloc/thread-freeres.c +++ b/malloc/thread-freeres.c @@ -29,6 +29,9 @@ void __libc_thread_freeres (void) { +#ifdef ARCH_THREAD_FREERES + call_function_static_weak (ARCH_THREAD_FREERES); +#endif #if SHLIB_COMPAT (libc, GLIBC_2_0, GLIBC_2_32) __rpc_thread_destroy (); #endif diff --git a/sysdeps/unix/sysv/linux/aarch64/makecontext.c b/sysdeps/unix/sysv/linux/aarch64/makecontext.c index 9e66b6761c..779f7e55aa 100644 --- a/sysdeps/unix/sysv/linux/aarch64/makecontext.c +++ b/sysdeps/unix/sysv/linux/aarch64/makecontext.c @@ -20,7 +20,9 @@ #include #include #include +#include #include +#include #define GCS_MAGIC 0x47435300 @@ -29,6 +31,47 @@ static struct _aarch64_ctx *extension (void *p) return p; } +struct gcs_list { + struct gcs_list *next; + void *base; + size_t size; +}; + +static __thread struct gcs_list *gcs_list_head = NULL; + +static void +record_gcs (void *base, size_t size) +{ + struct gcs_list *p = malloc (sizeof *p); + if (p == NULL) + abort (); + p->base = base; + p->size = size; + p->next = gcs_list_head; + gcs_list_head = p; +} + +static void +free_gcs_list (void) +{ + for (;;) + { + struct gcs_list *p = gcs_list_head; + if (p == NULL) + break; + gcs_list_head = p->next; + __munmap (p->base, p->size); + free (p); + } +} + +/* Called during thread shutdown to free resources. */ +void +__libc_aarch64_thread_freeres (void) +{ + free_gcs_list (); +} + #ifndef __NR_map_shadow_stack # define __NR_map_shadow_stack 453 #endif @@ -58,6 +101,9 @@ alloc_makecontext_gcs (size_t stack_size) if (base == (void *) -1) /* ENOSYS, bad size or OOM. */ abort (); + + record_gcs (base, size); + uint64_t *gcsp = (uint64_t *) ((char *) base + size); /* Skip end of GCS token. */ gcsp--; @@ -69,6 +115,25 @@ alloc_makecontext_gcs (size_t stack_size) return gcsp + 1; } +void +__free_makecontext_gcs (void *gcs) +{ + struct gcs_list *p = gcs_list_head; + struct gcs_list **q = &gcs_list_head; + for (;;) + { + if (p == NULL) + abort (); + if (gcs == p->base + p->size - 8) + break; + q = &p->next; + p = p->next; + } + *q = p->next; + __munmap (p->base, p->size); + free (p); +} + /* makecontext sets up a stack and the registers for the user context. The stack looks like this: diff --git a/sysdeps/unix/sysv/linux/aarch64/setcontext.S b/sysdeps/unix/sysv/linux/aarch64/setcontext.S index 17b0a48ce1..853dcb7a73 100644 --- a/sysdeps/unix/sysv/linux/aarch64/setcontext.S +++ b/sysdeps/unix/sysv/linux/aarch64/setcontext.S @@ -34,6 +34,9 @@ .text ENTRY (__setcontext) + /* If x10 is set then old GCS is freed. */ + mov x10, 0 +__setcontext_internal: PTR_ARG (0) /* Save a copy of UCP. */ mov x9, x0 @@ -145,7 +148,8 @@ ENTRY (__setcontext) ldr x3, [x2, #oGCSPR] MRS_GCSPR (x2) mov x4, x3 - /* x2: GCSPR now. x3, x4: target GCSPR. x5, x6: tmp regs. */ + mov x1, x2 + /* x1, x2: GCSPR now. x3, x4: target GCSPR. x5, x6: tmp regs. */ L(gcs_scan): cmp x2, x4 b.eq L(gcs_pop) @@ -162,10 +166,18 @@ L(gcs_switch): GCSSS2 (xzr) L(gcs_pop): cmp x2, x3 - b.eq L(gcs_done) + b.eq L(gcs_free_old) GCSPOPM (xzr) add x2, x2, 8 b L(gcs_pop) +L(gcs_free_old): + cbz x10, L(gcs_done) + mov x28, x0 + mov x0, x1 + bl __free_makecontext_gcs + mov x0, x28 + ldp x28, x29, [x0, oX0 + 28 * SZREG] + ldr x30, [x0, oX0 + 30 * SZREG] L(gcs_done): 2: @@ -186,6 +198,7 @@ ENTRY (__startcontext) cfi_undefined (x30) blr x20 mov x0, x19 - cbnz x0, __setcontext + mov x10, 1 + cbnz x0, __setcontext_internal 1: b HIDDEN_JUMPTARGET (exit) END (__startcontext) diff --git a/sysdeps/unix/sysv/linux/aarch64/sysdep.h b/sysdeps/unix/sysv/linux/aarch64/sysdep.h index bbbe35723c..590318dee8 100644 --- a/sysdeps/unix/sysv/linux/aarch64/sysdep.h +++ b/sysdeps/unix/sysv/linux/aarch64/sysdep.h @@ -29,8 +29,12 @@ #include -/* In order to get __set_errno() definition in INLINE_SYSCALL. */ #ifndef __ASSEMBLER__ +/* Thread cleanup function. */ +#define ARCH_THREAD_FREERES __libc_aarch64_thread_freeres +void __libc_aarch64_thread_freeres (void) attribute_hidden; + +/* In order to get __set_errno() definition in INLINE_SYSCALL. */ #include #endif From patchwork Wed Oct 23 08:39:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99369 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 073953858423 for ; Wed, 23 Oct 2024 08:44:00 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 7826C3858405 for ; Wed, 23 Oct 2024 08:43:02 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 7826C3858405 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 7826C3858405 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672992; cv=none; b=ap7r/6zGL2hV4n/x6EaOw5YzU5CLB9en+9U+gA7j8JG5c/89SqBimBw1ZeAUciI1CuBDbhOgX98Jvry0IT8tpmQoAUueUv7b9XUFhZemzCw6JIyPpfV1LwWQ2hQ9QV4iYRVAbPzIYeGg8UTQYmz9gNzuEoIyUGxIJEF83ZdFJ8c= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672992; c=relaxed/simple; bh=DZJU4qeu3+nxbXM7KoddZBEl5SPqS5o/ZO9oJOMCf5A=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=dfKlo4c5yf4f9pivpc3pAQT0ChiIoa74T6jKcVY47KeTayiRGIwfbD/JYqIdbOvb524uS2r80v5EcWvuGxj24CEFyJYSED8CVCaoaRmyam+xMvbUp//lz1ALMD/yQSDGlQN6X5EuNhfFieY43ec5SMT6QrnQCOR05MWfPRxryQw= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F27DC1575; Wed, 23 Oct 2024 01:43:31 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9A37E3F528; Wed, 23 Oct 2024 01:43:01 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 11/23] aarch64: Add glibc.cpu.aarch64_gcs tunable Date: Wed, 23 Oct 2024 09:39:08 +0100 Message-Id: <20241023083920.466015-12-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy This tunable is for controlling the GCS status. It is the argument to the PR_SET_SHADOW_STACK_STATUS prctl, by default 0, so GCS is disabled. The status is stored into GL(dl_aarch64_gcs) early and only applied later, since enabling GCS is tricky: it must happen on a top level stack frame. (Using GL instead of GLRO because it may need updates depending on loaded libraries that happen after readonly protection is applied, however library marking based GCS setting is not yet implemented.) --- sysdeps/aarch64/dl-tunables.list | 5 +++ .../unix/sysv/linux/aarch64/cpu-features.c | 4 ++ .../unix/sysv/linux/aarch64/dl-procruntime.c | 37 +++++++++++++++++++ 3 files changed, 46 insertions(+) create mode 100644 sysdeps/unix/sysv/linux/aarch64/dl-procruntime.c diff --git a/sysdeps/aarch64/dl-tunables.list b/sysdeps/aarch64/dl-tunables.list index 92c6cbfa92..c08be86932 100644 --- a/sysdeps/aarch64/dl-tunables.list +++ b/sysdeps/aarch64/dl-tunables.list @@ -21,5 +21,10 @@ glibc { name { type: STRING } + aarch64_gcs { + type: UINT_64 + minval: 0 + default: 0 + } } } diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c index c0b047bc0d..4a63875b19 100644 --- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c +++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c @@ -176,4 +176,8 @@ init_cpu_features (struct cpu_features *cpu_features) /* Check if MOPS is supported. */ cpu_features->mops = GLRO (dl_hwcap2) & HWCAP2_MOPS; + + if (GLRO (dl_hwcap) & HWCAP_GCS) + /* GCS status may be updated later by binary compatibility checks. */ + GL (dl_aarch64_gcs) = TUNABLE_GET (glibc, cpu, aarch64_gcs, uint64_t, 0); } diff --git a/sysdeps/unix/sysv/linux/aarch64/dl-procruntime.c b/sysdeps/unix/sysv/linux/aarch64/dl-procruntime.c new file mode 100644 index 0000000000..23c61da6c4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/aarch64/dl-procruntime.c @@ -0,0 +1,37 @@ +/* Data for processor runtime information. AArch64 version. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef PROCINFO_CLASS +# define PROCINFO_CLASS +#endif + +#if !IS_IN (ldconfig) +# if !defined PROCINFO_DECL && defined SHARED + ._dl_aarch64_gcs +# else +PROCINFO_CLASS unsigned long _dl_aarch64_gcs +# endif +# ifndef PROCINFO_DECL += 0 +# endif +# if !defined SHARED || defined PROCINFO_DECL +; +# else +, +# endif +#endif From patchwork Wed Oct 23 08:39:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99372 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 482CE3858CDB for ; Wed, 23 Oct 2024 08:44:45 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 297EF3858CD1 for ; Wed, 23 Oct 2024 08:43:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 297EF3858CD1 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 297EF3858CD1 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672994; cv=none; b=IIY49cmpGiRE1j2Le7b55oiJuEOn5KO6GtK1O4pZOZub5dNupsY7yKXvgiK0Dp3xAM5L8RieXY/72YIXDXgbQIVdKjt6T72az1rgJ+FHhDtr56QdLUxqyQgXnXHcwopLpGw3+MwCoYPq3barmRSbxu0ifslvE6+PaNCs1cOSxSk= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729672994; c=relaxed/simple; bh=wVqyhE49Q6gGKWVmKiHoIrGXzJMcgNoL7IIthNw/93k=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=PtQxmWbAsGKJjvp8xNRn5UTvp3C3fNT4MmBZT+ghwwmj7FEWrgP4qHKi2lvE2y5sEcz63QJ+w8tUXzmcEpswYUOC2TqUCh7wynalnTh/hJWJNFREXWpYkV+2I/QeYZ2Wnalg3GuqNPHof7tl6iZl5wWjSIy3wuqDKpgiz0nXjIU= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A113C1576; Wed, 23 Oct 2024 01:43:41 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 48F8E3F528; Wed, 23 Oct 2024 01:43:11 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 12/23] aarch64: Enable GCS in static linked exe Date: Wed, 23 Oct 2024 09:39:09 +0100 Message-Id: <20241023083920.466015-13-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Use the ARCH_SETUP_TLS hook to enable GCS in the static linked case. The system call must be inlined and then GCS is enabled on a top level stack frame that does not return and has no exception handlers above it. --- sysdeps/unix/sysv/linux/aarch64/libc-start.h | 49 ++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 sysdeps/unix/sysv/linux/aarch64/libc-start.h diff --git a/sysdeps/unix/sysv/linux/aarch64/libc-start.h b/sysdeps/unix/sysv/linux/aarch64/libc-start.h new file mode 100644 index 0000000000..0dd102c374 --- /dev/null +++ b/sysdeps/unix/sysv/linux/aarch64/libc-start.h @@ -0,0 +1,49 @@ +/* AArch64 definitions for libc main startup. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _LIBC_START_H +#define _LIBC_START_H + +#ifndef SHARED + +# ifndef PR_SET_SHADOW_STACK_STATUS +# define PR_GET_SHADOW_STACK_STATUS 74 +# define PR_SET_SHADOW_STACK_STATUS 75 +# define PR_LOCK_SHADOW_STACK_STATUS 76 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) +# define PR_SHADOW_STACK_WRITE (1UL << 1) +# define PR_SHADOW_STACK_PUSH (1UL << 2) +# endif + +/* Must be on a top-level stack frame that does not return. */ +static inline void __attribute__((always_inline)) +aarch64_libc_setup_tls (void) +{ + __libc_setup_tls (); + + uint64_t s = GL(dl_aarch64_gcs); + if (s != 0) + INLINE_SYSCALL_CALL (prctl, PR_SET_SHADOW_STACK_STATUS, s, 0, 0, 0); +} + +# define ARCH_SETUP_IREL() apply_irel () +# define ARCH_SETUP_TLS() aarch64_libc_setup_tls () +# define ARCH_APPLY_IREL() +#endif /* ! SHARED */ + +#endif /* _LIBC_START_H */ From patchwork Wed Oct 23 08:39:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99373 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 56E1C3857349 for ; Wed, 23 Oct 2024 08:45:18 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 71CD33858C42 for ; Wed, 23 Oct 2024 08:43:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 71CD33858C42 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 71CD33858C42 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673014; cv=none; b=AqEm3mmjOEp8XP/uw94eS88VcMYlJMZbXZFq1Eimz1jQULUm0p5eHMsdu4X8YbXNcHRlo0s+/9nRnOwrJcJ8KNsDbdMGsn3J5TlShl7ShWVEnN0gtTo8gKCRamcdLgb5/B7qCgZkx+uR386aI0k7pl2aeiCemwB7fzc6SI/XEoQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673014; c=relaxed/simple; bh=n+Lg1JKA1uD4aWJ+JZKderDl5ueKdZ4/dF4CZhFd1Pk=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=cFXKP8YRE3vDqQ2aQGBs07U4Ij08C1qnW773j9h9zVixybUAU7o5YmIIA8mO2C4hy+wEp5OjnRH9JxECuFHyZAMjWYvHtI5C5N12jMCqKcVBlmE5EwBAMXYt2x8eiGnZL3egHu8XsoEzUnvPki/B9ssP7CSXgVl8qVzondS43iQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E053D1575; Wed, 23 Oct 2024 01:43:52 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 84A123F528; Wed, 23 Oct 2024 01:43:22 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 13/23] aarch64: Enable GCS in dynamic linked exe Date: Wed, 23 Oct 2024 09:39:10 +0100 Message-Id: <20241023083920.466015-14-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Use the dynamic linker start code to enable GCS in the dynamic linked case after _dl_start returns and before _dl_start_user which marks the point after which user code may run. Like in the static linked case this ensures that GCS is enabled on a top level stack frame. --- sysdeps/aarch64/Makefile | 4 +++- sysdeps/aarch64/dl-start.S | 23 +++++++++++++++++++++-- sysdeps/aarch64/rtld-global-offsets.sym | 5 +++++ 3 files changed, 29 insertions(+), 3 deletions(-) diff --git a/sysdeps/aarch64/Makefile b/sysdeps/aarch64/Makefile index 141d7d9cc2..ca8b96f550 100644 --- a/sysdeps/aarch64/Makefile +++ b/sysdeps/aarch64/Makefile @@ -35,7 +35,9 @@ endif ifeq ($(subdir),elf) sysdep-rtld-routines += dl-start sysdep-dl-routines += tlsdesc dl-tlsdesc -gen-as-const-headers += dl-link.sym +gen-as-const-headers += \ + dl-link.sym \ + rtld-global-offsets.sym tests-internal += tst-ifunc-arg-1 tst-ifunc-arg-2 diff --git a/sysdeps/aarch64/dl-start.S b/sysdeps/aarch64/dl-start.S index d645484e79..bca5c408af 100644 --- a/sysdeps/aarch64/dl-start.S +++ b/sysdeps/aarch64/dl-start.S @@ -18,6 +18,7 @@ . */ #include +#include ENTRY (_start) /* Create an initial frame with 0 LR and FP */ @@ -25,11 +26,30 @@ ENTRY (_start) mov x29, #0 mov x30, #0 + /* Load and relocate all library dependencies. */ mov x0, sp PTR_ARG (0) bl _dl_start /* Returns user entry point in x0. */ mov PTR_REG (21), PTR_REG (0) + + /* Use GL(dl_aarch64_gcs) to set the shadow stack status. */ + adrp x16, _rtld_local + add PTR_REG (16), PTR_REG (16), :lo12:_rtld_local + ldr x1, [x16, GL_DL_AARCH64_GCS_OFFSET] + cbz x1, L(skip_gcs_enable) + + /* Enable GCS before user code runs. Note that IFUNC resolvers and + LD_AUDIT hooks may run before, but should not create threads. */ +#define PR_SET_SHADOW_STACK_STATUS 75 + mov x0, PR_SET_SHADOW_STACK_STATUS + mov x2, 0 + mov x3, 0 + mov x4, 0 + mov x8, #SYS_ify(prctl) + svc 0x0 +L(skip_gcs_enable): + .globl _dl_start_user .type _dl_start_user, %function _dl_start_user: @@ -40,8 +60,7 @@ _dl_start_user: /* Compute envp. */ add PTR_REG (3), PTR_REG (2), PTR_REG (1), lsl PTR_LOG_SIZE add PTR_REG (3), PTR_REG (3), PTR_SIZE - adrp x16, _rtld_local - add PTR_REG (16), PTR_REG (16), :lo12:_rtld_local + /* Run the init functions of the loaded modules. */ ldr PTR_REG (0), [x16] bl _dl_init /* Load the finalizer function. */ diff --git a/sysdeps/aarch64/rtld-global-offsets.sym b/sysdeps/aarch64/rtld-global-offsets.sym index 23cdaf7d9e..6c0690bb95 100644 --- a/sysdeps/aarch64/rtld-global-offsets.sym +++ b/sysdeps/aarch64/rtld-global-offsets.sym @@ -3,8 +3,13 @@ #include #define GLRO_offsetof(name) offsetof (struct rtld_global_ro, _##name) +#define GL_offsetof(name) offsetof (struct rtld_global, _##name) -- Offsets of _rtld_global_ro in libc.so GLRO_DL_HWCAP_OFFSET GLRO_offsetof (dl_hwcap) GLRO_DL_HWCAP2_OFFSET GLRO_offsetof (dl_hwcap2) + +-- Offsets of _rtld_global in libc.so + +GL_DL_AARCH64_GCS_OFFSET GL_offsetof (dl_aarch64_gcs) From patchwork Wed Oct 23 08:39:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99375 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 468F23857015 for ; Wed, 23 Oct 2024 08:45:37 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id BEB753858CDB for ; Wed, 23 Oct 2024 08:43:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BEB753858CDB Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BEB753858CDB Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673013; cv=none; b=jrjxYugJeCkP1Bt8PDY4gi2TT5kBuQTc+6gH1wDI5ahUEA2cyGcxpSIcMpXVhqUjI4vXlX9pLz2f1XmvaYlALu4RapJmWkOljo23SYdoGejYSSlW3La7JAQvDOUUhh2qm+oaCwP1SLRV/KLx9IUvvIAtJLiIHVAWRQjESLVQ9hk= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673013; c=relaxed/simple; bh=MvUQNpxFf+Ipd473sWA65unaBFkAgqKEs3RV+kflTi8=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=K3heZwVnT++ruzr96/CvnxwS0eiHG2MDUhmyvpY+tO3Tq5gMZK3ZMSeXMrFdFltP58n2uApHt9COFs8V159o322fGKlMMQPoE9fa8IvBG3N22BkIxpc5TyDfbL30JX6BHlhRe0v0mmGYeljuogF3JjR/3ifHQsc9UXUayJ+U780= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3FDF91576; Wed, 23 Oct 2024 01:44:00 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DBD6A3F528; Wed, 23 Oct 2024 01:43:29 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 14/23] aarch64: Mark objects with GCS property note Date: Wed, 23 Oct 2024 09:39:11 +0100 Message-Id: <20241023083920.466015-15-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy --- sysdeps/aarch64/sysdep.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/sysdeps/aarch64/sysdep.h b/sysdeps/aarch64/sysdep.h index 661d9fe8c1..55e0230b5e 100644 --- a/sysdeps/aarch64/sysdep.h +++ b/sysdeps/aarch64/sysdep.h @@ -85,6 +85,7 @@ strip_pac (void *p) #define FEATURE_1_AND 0xc0000000 #define FEATURE_1_BTI 1 #define FEATURE_1_PAC 2 +#define FEATURE_1_GCS 4 /* Add a NT_GNU_PROPERTY_TYPE_0 note. */ #define GNU_PROPERTY(type, value) \ @@ -103,9 +104,9 @@ strip_pac (void *p) /* Add GNU property note with the supported features to all asm code where sysdep.h is included. */ #if HAVE_AARCH64_BTI && HAVE_AARCH64_PAC_RET -GNU_PROPERTY (FEATURE_1_AND, FEATURE_1_BTI|FEATURE_1_PAC) +GNU_PROPERTY (FEATURE_1_AND, FEATURE_1_BTI|FEATURE_1_PAC|FEATURE_1_GCS) #elif HAVE_AARCH64_BTI -GNU_PROPERTY (FEATURE_1_AND, FEATURE_1_BTI) +GNU_PROPERTY (FEATURE_1_AND, FEATURE_1_BTI|FEATURE_1_GCS) #endif /* Define an entry point visible from C. */ From patchwork Wed Oct 23 08:39:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99377 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6D8293858C52 for ; Wed, 23 Oct 2024 08:46:04 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id A111D3858429 for ; Wed, 23 Oct 2024 08:43:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A111D3858429 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A111D3858429 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673023; cv=none; b=pQOibNHWvHO16Kc/j0xpDd3xXKVtWzjHkUv+7w8hNXli5ILWcYxqNtkUzEG0q1LQw2i9nrVeBCjF9XhkTgArM1AUK0tZwspT7IkhJwrbS9w/yblb/854iziop3LE9LimPsTmXQSehYgLFmdalEHsOx6xM8MCdtxXciuTlIVdtUM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673023; c=relaxed/simple; bh=t50onkiO5G04+3GwchV7etD3tmRVbbguqiS/urUe2yk=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=OsMBrZ67UAy9jX2UEce0ghNMRkSo33xwNBFkIqEjrpTOkOG108mpkuOlo0GFP4FdVhmgQ9TZzlzlxYNklWomIxLUlt48lhEtWxApVcqCMR9RRYJYK+5eBUE6UEAYgJ3bpGrte9qE2ZR9sUysQw7jzMKiMslHRlRgQ5y4GySqKbg= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 26D841575; Wed, 23 Oct 2024 01:44:07 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C1D1B3F528; Wed, 23 Oct 2024 01:43:36 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 15/23] aarch64: Add glibc.cpu.aarch64_gcs_policy Date: Wed, 23 Oct 2024 09:39:12 +0100 Message-Id: <20241023083920.466015-16-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP, T_FILL_THIS_FORM_SHORT autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy policy sets how gcs tunable and gcs marking turns into gcs state: 0: state = tunable 1: state = marking ? tunable : (tunable && dlopen ? err : 0) 2: state = marking ? tunable : (tunable ? err : 0) --- sysdeps/aarch64/dl-tunables.list | 5 +++++ sysdeps/unix/sysv/linux/aarch64/cpu-features.c | 9 +++++++-- sysdeps/unix/sysv/linux/aarch64/dl-procinfo.c | 13 +++++++++++++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/sysdeps/aarch64/dl-tunables.list b/sysdeps/aarch64/dl-tunables.list index c08be86932..2a07a6216b 100644 --- a/sysdeps/aarch64/dl-tunables.list +++ b/sysdeps/aarch64/dl-tunables.list @@ -26,5 +26,10 @@ glibc { minval: 0 default: 0 } + aarch64_gcs_policy { + type: UINT_64 + minval: 0 + default: 0 + } } } diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c index 4a63875b19..925756d79e 100644 --- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c +++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c @@ -178,6 +178,11 @@ init_cpu_features (struct cpu_features *cpu_features) cpu_features->mops = GLRO (dl_hwcap2) & HWCAP2_MOPS; if (GLRO (dl_hwcap) & HWCAP_GCS) - /* GCS status may be updated later by binary compatibility checks. */ - GL (dl_aarch64_gcs) = TUNABLE_GET (glibc, cpu, aarch64_gcs, uint64_t, 0); + { + /* GCS status may be updated later by binary compatibility checks. */ + GL (dl_aarch64_gcs) = TUNABLE_GET (glibc, cpu, aarch64_gcs, uint64_t, 0); + /* Fixed GCS policy. */ + GLRO (dl_aarch64_gcs_policy) = + TUNABLE_GET (glibc, cpu, aarch64_gcs_policy, uint64_t, 0); + } } diff --git a/sysdeps/unix/sysv/linux/aarch64/dl-procinfo.c b/sysdeps/unix/sysv/linux/aarch64/dl-procinfo.c index 7af232de52..a9d5ee9df5 100644 --- a/sysdeps/unix/sysv/linux/aarch64/dl-procinfo.c +++ b/sysdeps/unix/sysv/linux/aarch64/dl-procinfo.c @@ -54,6 +54,19 @@ PROCINFO_CLASS struct cpu_features _dl_aarch64_cpu_features # else , # endif +# if !defined PROCINFO_DECL && defined SHARED + ._dl_aarch64_gcs_policy +# else +PROCINFO_CLASS uint64_t _dl_aarch64_gcs_policy +# endif +# ifndef PROCINFO_DECL += 0 +# endif +# if !defined SHARED || defined PROCINFO_DECL +; +# else +, +# endif #endif /* Number of HWCAP bits set. */ From patchwork Wed Oct 23 08:39:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99379 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 46B793858C39 for ; Wed, 23 Oct 2024 08:46:44 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id E5F483858D3C for ; Wed, 23 Oct 2024 08:43:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E5F483858D3C Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E5F483858D3C Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673043; cv=none; b=KsiYo4VuP0svFaQ9Aa1ZPrFpDtCBfDdXuptSfAphhURDmU8X6dWGMkVlyAxqSp/k2BIeBhuqYJre3gl+meePH5xuWbCOFP/hFuQqNLT90ggiuz2y7spbJP0X+dt5pacnsoJYFYvWeh7aN9jt25IN7bT+LFdsxyEU21hIHAmtd58= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673043; c=relaxed/simple; bh=W686nIykdpeyzK86pRT4fnxf241+RINhSGBgURxW60Y=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=nHUpNAlYxvgNRaclge+BpbFGzo2E6oFauC+X+YO8ZHPx7B2z/xaxM4jv+WaFPOjTf9VNyZNDdstlHBoRGCD/y4GX+aSBld8L8bM/DmwasYGi6XP8nof+3VTeBxFDTMfZGEoVJfgEjae6hoXUTKVGmmBNHE/7k60zShsmwJWOFns= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 673551575; Wed, 23 Oct 2024 01:44:23 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0F51D3F528; Wed, 23 Oct 2024 01:43:52 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 16/23] aarch64: Use l_searchlist.r_list for bti Date: Wed, 23 Oct 2024 09:39:13 +0100 Message-Id: <20241023083920.466015-17-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Allows using the same function for static exe. --- sysdeps/aarch64/dl-bti.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/sysdeps/aarch64/dl-bti.c b/sysdeps/aarch64/dl-bti.c index fd0d308a39..e03bfc2acb 100644 --- a/sysdeps/aarch64/dl-bti.c +++ b/sysdeps/aarch64/dl-bti.c @@ -84,10 +84,9 @@ _dl_bti_check (struct link_map *l, const char *program) if (l->l_mach.bti_fail) bti_failed (l, program); - unsigned int i = l->l_searchlist.r_nlist; - while (i-- > 0) + for (unsigned int i = 0; i < l->l_searchlist.r_nlist; i++) { - struct link_map *dep = l->l_initfini[i]; + struct link_map *dep = l->l_searchlist.r_list[i]; if (dep->l_mach.bti_fail) bti_failed (dep, program); } From patchwork Wed Oct 23 08:39:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99371 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2AFD43858424 for ; Wed, 23 Oct 2024 08:44:43 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 58EAE385841D for ; Wed, 23 Oct 2024 08:44:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 58EAE385841D Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 58EAE385841D Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673043; cv=none; b=rHXaO0flltVKP81GshGXgceREwonsSOTKo4Alz2y2dPSHDial21LvPSQ1xh8uUsbUtYbjCetIFkLk5NBNlSdmMsOEYlaVGD7cOei/LvVxww2wjzGhbjYgNfktI7oAdzDlRKgzM4kpZRMPYwBseGT58rqtQ1IeZyiBX3/eb1PEEs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673043; c=relaxed/simple; bh=smDODWu4h5JVw+3jea+ek5IafU08BQrRM282Ijz/m5A=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=bU5nT2fa5D3KYBHoiUV/oEFIlH3BxuBoPjXbltD/hK3bRFZkJrYa5vqngrdcS3g/5rk42IolzbSvSvm7r5k+4Jo+iz+pHuU3pB1DaYYSTK7XmeKWJJKB53P565vgP0LVn913iC95/IyQ3MNsByGqLo8/wgdbGCeubxFz5SAoEng= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D04BF1576; Wed, 23 Oct 2024 01:44:29 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 76DF83F528; Wed, 23 Oct 2024 01:43:59 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 17/23] aarch64: Handle gcs marking Date: Wed, 23 Oct 2024 09:39:14 +0100 Message-Id: <20241023083920.466015-18-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy --- sysdeps/aarch64/Makefile | 4 ++- sysdeps/aarch64/dl-gcs.c | 59 +++++++++++++++++++++++++++++++++++++++ sysdeps/aarch64/dl-prop.h | 15 ++++++---- sysdeps/aarch64/linkmap.h | 1 + 4 files changed, 73 insertions(+), 6 deletions(-) create mode 100644 sysdeps/aarch64/dl-gcs.c diff --git a/sysdeps/aarch64/Makefile b/sysdeps/aarch64/Makefile index ca8b96f550..74479604f2 100644 --- a/sysdeps/aarch64/Makefile +++ b/sysdeps/aarch64/Makefile @@ -9,7 +9,9 @@ LDFLAGS-rtld += -Wl,-z,force-bti,--fatal-warnings endif ifeq ($(subdir),elf) -sysdep-dl-routines += dl-bti +sysdep-dl-routines += \ + dl-bti \ + dl-gcs tests += tst-audit26 \ tst-audit27 diff --git a/sysdeps/aarch64/dl-gcs.c b/sysdeps/aarch64/dl-gcs.c new file mode 100644 index 0000000000..a92deb54b5 --- /dev/null +++ b/sysdeps/aarch64/dl-gcs.c @@ -0,0 +1,59 @@ +/* AArch64 GCS functions. + Copyright (C) 2024 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include + +static void +fail (struct link_map *l, const char *program) +{ + if (program) + _dl_fatal_printf ("%s: %s: not GCS compatible\n", program, l->l_name); + else + _dl_signal_error (0, l->l_name, "dlopen", N_("not GCS compatible")); +} + +static void +check_gcs (struct link_map *l, const char *program) +{ + if (!l->l_mach.gcs) + { + if (GLRO(dl_aarch64_gcs_policy) == 2 || !program) + fail (l, program); + if (GLRO(dl_aarch64_gcs_policy) == 1 && program) + GL(dl_aarch64_gcs) = 0; + } +} + +/* Apply GCS policy for L and its dependencies. */ + +void +_dl_gcs_check (struct link_map *l, const char *program) +{ + /* GCS is disabled. */ + if (GL(dl_aarch64_gcs) == 0) + return; + /* GCS marking is ignored. */ + if (GLRO(dl_aarch64_gcs_policy) == 0) + return; + + check_gcs (l, program); + for (unsigned int i = 0; i < l->l_searchlist.r_nlist; i++) + check_gcs (l->l_initfini[i], program); +} diff --git a/sysdeps/aarch64/dl-prop.h b/sysdeps/aarch64/dl-prop.h index df05c0211d..72ac11aec0 100644 --- a/sysdeps/aarch64/dl-prop.h +++ b/sysdeps/aarch64/dl-prop.h @@ -24,16 +24,21 @@ extern void _dl_bti_protect (struct link_map *, int) attribute_hidden; extern void _dl_bti_check (struct link_map *, const char *) attribute_hidden; +extern void _dl_gcs_check (struct link_map *, const char *) + attribute_hidden; + static inline void __attribute__ ((always_inline)) _rtld_main_check (struct link_map *m, const char *program) { _dl_bti_check (m, program); + _dl_gcs_check (m, program); } static inline void __attribute__ ((always_inline)) _dl_open_check (struct link_map *m) { _dl_bti_check (m, NULL); + _dl_gcs_check (m, NULL); } static inline void __attribute__ ((always_inline)) @@ -45,10 +50,6 @@ static inline int _dl_process_gnu_property (struct link_map *l, int fd, uint32_t type, uint32_t datasz, void *data) { - if (!GLRO(dl_aarch64_cpu_features).bti) - /* Skip note processing. */ - return 0; - if (type == GNU_PROPERTY_AARCH64_FEATURE_1_AND) { /* Stop if the property note is ill-formed. */ @@ -57,7 +58,11 @@ _dl_process_gnu_property (struct link_map *l, int fd, uint32_t type, unsigned int feature_1 = *(unsigned int *) data; if (feature_1 & GNU_PROPERTY_AARCH64_FEATURE_1_BTI) - _dl_bti_protect (l, fd); + if (GLRO(dl_aarch64_cpu_features).bti) + _dl_bti_protect (l, fd); + + if (feature_1 & GNU_PROPERTY_AARCH64_FEATURE_1_GCS) + l->l_mach.gcs = 1; /* Stop if we processed the property note. */ return 0; diff --git a/sysdeps/aarch64/linkmap.h b/sysdeps/aarch64/linkmap.h index 56a63fc3dd..423fc0bd8e 100644 --- a/sysdeps/aarch64/linkmap.h +++ b/sysdeps/aarch64/linkmap.h @@ -23,4 +23,5 @@ struct link_map_machine ElfW(Addr) plt; /* Address of .plt */ void *tlsdesc_table; /* Address of TLS descriptor hash table. */ bool bti_fail; /* Failed to enable Branch Target Identification. */ + bool gcs; /* Guarded Control Stack marking. */ }; From patchwork Wed Oct 23 08:39:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99376 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A3D19385AC23 for ; Wed, 23 Oct 2024 08:45:51 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id EDB563858CD1 for ; Wed, 23 Oct 2024 08:44:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org EDB563858CD1 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org EDB563858CD1 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673054; cv=none; b=H7ud6HAAHexS2tI8BQp8Q9JZkFbVyYHxx/Ym/nnUqbDejEimgNFKn2drIojYlBjRb59PimQb0u5Yk/PksdJnXLHwU065hwEInxFXE24nFHIeTz7BrggXZ6pfglO1hw5Rk7iS8Dj4T+vwAITbv/26fMMJshdTmG055zAxGJkmvV8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673054; c=relaxed/simple; bh=tY/loOEvatCICl8jTyWuqZuTV2AGBIAJoRW2JVTJnZI=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=KU2uPs9/2FxHnEglc4vh285rZ8E8YcCs+ObE/Hl93WANn+K1n2ErGr75O8DRz3QV/CKhkoknKkOAkdNdEyXRyCGQECVmxmeKvSdq8Gg3aBufXrZ15QSTTEAcR8KTanTJn/T/AFS9Ac084FpQr+OaA91K8UmhqEeTKHS47/Bn13Q= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4B0FA1575; Wed, 23 Oct 2024 01:44:37 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E62843F528; Wed, 23 Oct 2024 01:44:06 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 18/23] aarch64: Use l_searchlist.r_list for gcs Date: Wed, 23 Oct 2024 09:39:15 +0100 Message-Id: <20241023083920.466015-19-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.8 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Allows using the same function for static exe. --- sysdeps/aarch64/dl-gcs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sysdeps/aarch64/dl-gcs.c b/sysdeps/aarch64/dl-gcs.c index a92deb54b5..764b8a56e9 100644 --- a/sysdeps/aarch64/dl-gcs.c +++ b/sysdeps/aarch64/dl-gcs.c @@ -55,5 +55,5 @@ _dl_gcs_check (struct link_map *l, const char *program) check_gcs (l, program); for (unsigned int i = 0; i < l->l_searchlist.r_nlist; i++) - check_gcs (l->l_initfini[i], program); + check_gcs (l->l_searchlist.r_list[i], program); } From patchwork Wed Oct 23 08:39:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99378 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id ACCCD3858CD1 for ; Wed, 23 Oct 2024 08:46:20 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 2FECB385842D for ; Wed, 23 Oct 2024 08:44:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2FECB385842D Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2FECB385842D Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673064; cv=none; b=hdfDutMn617xp7QTvvia7NZL+HxxuwwCFtQzpOvC1uYcNXrzO+ClQY5fkmnzDVA4h1FmAO94Urn57L3hqwJZ1yh6DEiL4xDyvWyU61rmuhn4f7CyLXIcBDixG4oJm/TJRAd6CV3hpa8TPyfu/cGBMtT8hydWq1JQU5+C4gMnz7M= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673064; c=relaxed/simple; bh=pgGt1V0e6j9TXRuUGoKKPsFBocHqTbLU7Z2kO0A2P1g=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=tAtLwCncslISFwN28Y6AgYBMgb9JZ40a/osEQ3XQP7xx8gwHv0DlHgUUjsyxP5Lbgore56w4ihKB+P/V0wch1b+Pzs6CBJV9zZR+ByvIC5KFYOwL4UJgNa8SLvH7bOd+eFEm1HSBUbPiTi7i2bWYZUex8muSXgxv0XMdPgyJzFk= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9F0A81576; Wed, 23 Oct 2024 01:44:43 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 46EA33F528; Wed, 23 Oct 2024 01:44:13 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 19/23] aarch64: Ignore GCS property of ld.so Date: Wed, 23 Oct 2024 09:39:16 +0100 Message-Id: <20241023083920.466015-20-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy check_gcs is called for each dependency of a DSO, but the GNU property of the ld.so is not processed so ldso->l_mach.gcs may not be correct. Just assume ld.so is GCS compatible independently of the ELF marking. --- sysdeps/aarch64/dl-gcs.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sysdeps/aarch64/dl-gcs.c b/sysdeps/aarch64/dl-gcs.c index 764b8a56e9..b81aa30787 100644 --- a/sysdeps/aarch64/dl-gcs.c +++ b/sysdeps/aarch64/dl-gcs.c @@ -32,6 +32,11 @@ fail (struct link_map *l, const char *program) static void check_gcs (struct link_map *l, const char *program) { +#ifdef SHARED + /* Ignore GCS marking on ld.so: its properties are not processed. */ + if (l->l_real == &GL(dl_rtld_map)) + return; +#endif if (!l->l_mach.gcs) { if (GLRO(dl_aarch64_gcs_policy) == 2 || !program) From patchwork Wed Oct 23 08:39:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99381 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A32293858D28 for ; Wed, 23 Oct 2024 08:47:31 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 945063858C50 for ; Wed, 23 Oct 2024 08:44:20 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 945063858C50 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 945063858C50 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673064; cv=none; b=vPyBxMGHABEd+sPakBT219Cvc0EwWOWHV4A6j7c+XbA3LQeQ/2XfDGwXLrU0NVufUt8hupXNUYPu+h+0Z7jz75iSFce9n4K0EtFLctWBf+GYsR+05b1awiDnhCxYAAkqpZeBUtP+x6SFtW850fKSk16lYCi78N5pGgzpmYLhMK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673064; c=relaxed/simple; bh=Q8HslczlT64puxqYjZNDWc4qXOvaRIUoFjvqx8w5sSY=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=bAZE3BY0AiRNmbKfeVDda3UOPF05t7lJS+mB74oS/XeB/c3a5ii8VuGkKblPtGVen7jOhb3bW435e5mKtX7A8+ayvuPgWrCsjuUcVHih97wwbL4M+BuMRg2cid8AfjjVyaVphmzw+L7EzeXXkWa4C7Q0lY52K6lvzA5dIVDJwGY= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 079601575; Wed, 23 Oct 2024 01:44:50 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 956BC3F528; Wed, 23 Oct 2024 01:44:19 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 20/23] aarch64: Process gnu properties in static exe Date: Wed, 23 Oct 2024 09:39:17 +0100 Message-Id: <20241023083920.466015-21-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-14.0 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Unlike for BTI, the kernel does not process GCS properties so update GL(dl_aarch64_gcs) before the GCS status is set. --- sysdeps/unix/sysv/linux/aarch64/libc-start.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/sysdeps/unix/sysv/linux/aarch64/libc-start.h b/sysdeps/unix/sysv/linux/aarch64/libc-start.h index 0dd102c374..1cc4dab818 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libc-start.h +++ b/sysdeps/unix/sysv/linux/aarch64/libc-start.h @@ -21,6 +21,8 @@ #ifndef SHARED +# include + # ifndef PR_SET_SHADOW_STACK_STATUS # define PR_GET_SHADOW_STACK_STATUS 74 # define PR_SET_SHADOW_STACK_STATUS 75 @@ -36,6 +38,16 @@ aarch64_libc_setup_tls (void) { __libc_setup_tls (); + struct link_map *main_map = _dl_get_dl_main_map (); + const ElfW(Phdr) *phdr = GL(dl_phdr); + const ElfW(Phdr) *ph; + for (ph = phdr; ph < phdr + GL(dl_phnum); ph++) + if (ph->p_type == PT_GNU_PROPERTY) + { + _dl_process_pt_gnu_property (main_map, -1, ph); + _rtld_main_check (main_map, _dl_argv[0]); + break; + } uint64_t s = GL(dl_aarch64_gcs); if (s != 0) INLINE_SYSCALL_CALL (prctl, PR_SET_SHADOW_STACK_STATUS, s, 0, 0, 0); From patchwork Wed Oct 23 08:39:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99380 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2CB2C3858C41 for ; Wed, 23 Oct 2024 08:47:00 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id A34BC3858D28 for ; Wed, 23 Oct 2024 08:44:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A34BC3858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A34BC3858D28 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673083; cv=none; b=d7MbJiPdnPCM6U3HPYvAcl7jphTPn7lrxDpaLZ3z5ZcMudSdwqzWKSCDAJ2beeIIf44Xc27kIyqi23QM7bt4wxXq0fyxdWJx6hVPDRwy6xpncYdq92HnhB0dONRW3IF6n75mkBhVV8rpwAs4jfxIGsx+0j8f0OU8naGCRNshVyU= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673083; c=relaxed/simple; bh=8Iw5rQsiZbL+jLZO63amAwGFryK7ll/KvJhJauRYqYs=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=YriajVBt6DMnRIBILKgcjWf0dWKIuUluPoTObKsDGOmw6Sp2dUrFQvfQd4uhEcTH4jLuTttOZSFckjXP/Qo2iJYwJFgLYl2sVDihggwg3gB7MaGi1MeSYwovaW+DQ8dDNie0qvGL24x+l9+7Ef4DgwiB003Fl9jNX4ygQjDFEUI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 276791575; Wed, 23 Oct 2024 01:45:06 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C22653F528; Wed, 23 Oct 2024 01:44:35 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 21/23] aarch64: Add GCS user-space allocation logic Date: Wed, 23 Oct 2024 09:39:18 +0100 Message-Id: <20241023083920.466015-22-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy Allocate GCS based on the stack size, this can be used for coroutines (makecontext) and thread creation (if the kernel allows user allocated GCS). --- sysdeps/aarch64/Makefile | 3 +- sysdeps/aarch64/__alloc_gcs.c | 66 +++++++++++++++++++++++++++++++++++ sysdeps/aarch64/aarch64-gcs.h | 36 +++++++++++++++++++ 3 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/__alloc_gcs.c create mode 100644 sysdeps/aarch64/aarch64-gcs.h diff --git a/sysdeps/aarch64/Makefile b/sysdeps/aarch64/Makefile index 74479604f2..4b7f8a5c07 100644 --- a/sysdeps/aarch64/Makefile +++ b/sysdeps/aarch64/Makefile @@ -71,7 +71,8 @@ sysdep_headers += sys/ifunc.h sysdep_routines += \ __mtag_tag_zero_region \ __mtag_tag_region \ - __arm_za_disable + __arm_za_disable \ + __alloc_gcs tests += \ tst-sme-jmp diff --git a/sysdeps/aarch64/__alloc_gcs.c b/sysdeps/aarch64/__alloc_gcs.c new file mode 100644 index 0000000000..62b7b6f396 --- /dev/null +++ b/sysdeps/aarch64/__alloc_gcs.c @@ -0,0 +1,66 @@ +/* AArch64 GCS allocation. + Copyright (C) 2024 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include + +#ifndef __NR_map_shadow_stack +# define __NR_map_shadow_stack 453 +#endif +#ifndef SHADOW_STACK_SET_TOKEN +# define SHADOW_STACK_SET_TOKEN (1UL << 0) +# define SHADOW_STACK_SET_MARKER (1UL << 1) +#endif + +static void * +map_shadow_stack (void *addr, size_t size, unsigned long flags) +{ + return (void *) INLINE_SYSCALL_CALL (map_shadow_stack, addr, size, flags); +} + +#define GCS_MAX_SIZE (1UL << 31) +#define GCS_ALTSTACK_RESERVE 160 + +void * +__alloc_gcs (size_t stack_size, void **ss_base, size_t *ss_size) +{ + size_t size = (stack_size / 2 + GCS_ALTSTACK_RESERVE) & -8UL; + if (size > GCS_MAX_SIZE) + size = GCS_MAX_SIZE; + + unsigned long flags = SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN; + void *base = map_shadow_stack (NULL, size, flags); + if (base == (void *) -1) + return NULL; + + *ss_base = base; + *ss_size = size; + + uint64_t *gcsp = (uint64_t *) ((char *) base + size); + /* Skip end of GCS token. */ + gcsp--; + /* Verify GCS cap token. */ + gcsp--; + if (((uint64_t)gcsp & 0xfffffffffffff000) + 1 != *gcsp) + { + __munmap (base, size); + return NULL; + } + /* Return the target GCS pointer for context switch. */ + return gcsp + 1; +} diff --git a/sysdeps/aarch64/aarch64-gcs.h b/sysdeps/aarch64/aarch64-gcs.h new file mode 100644 index 0000000000..cbd1065a5d --- /dev/null +++ b/sysdeps/aarch64/aarch64-gcs.h @@ -0,0 +1,36 @@ +/* AArch64 GCS (Guarded Control Stack) declarations. + This file is part of the GNU C Library. + Copyright (C) 2024 Free Software Foundation, Inc. + Copyright The GNU Toolchain Authors. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _AARCH64_GCS_H +#define _AARCH64_GCS_H + +#include +#include + +void *__alloc_gcs (size_t, void **, size_t *) attribute_hidden; + +static inline bool +has_gcs (void) +{ + register unsigned long x16 asm ("x16") = 1; + asm ("hint 40" /* chkfeat x16 */ : "+r" (x16)); + return x16 == 0; +} + +#endif From patchwork Wed Oct 23 08:39:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99374 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1AAEC3858427 for ; Wed, 23 Oct 2024 08:45:29 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 14D50385842A for ; Wed, 23 Oct 2024 08:44:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 14D50385842A Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 14D50385842A Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673094; cv=none; b=tp/PjIEWYetGPZ1lW2S9y1DhxK0IwgZ45RYQgfJW6VVVK8LGq9xM1vc0ggDpXEd9P9HcFcbqI6bgWivttdVjYV3QoOdVYbX/FdVcAoM4phuAwf6qVd5HH+sNaB93fzMQnC5bG/nUTocuhlHdYnTiPbHkTNCb4FolUScg9SP83Q4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673094; c=relaxed/simple; bh=jbcaNgigrOYvU0NtWMUQs+c8RQF/78bBfNqzudtjWp0=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=azUO2/ZJjSP1B/FwVvGvocQRW56RLEeSF7PCHrdCkKssjfyIc6R7PG8DKR64q0B2In/mnsavgezlmDTzT8NrWvCHMnGDNqlblTKtSPjpraxea0FPGlTYecp02VDOEh8yGPpAHM7m0S4rapN6RnTC3rpwW8nKda2zZrKS1RdiNU4= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 909E71575; Wed, 23 Oct 2024 01:45:13 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 384A03F528; Wed, 23 Oct 2024 01:44:43 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 22/23] aarch64: use __alloc_gcs in makecontext Date: Wed, 23 Oct 2024 09:39:19 +0100 Message-Id: <20241023083920.466015-23-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-14.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy --- sysdeps/unix/sysv/linux/aarch64/makecontext.c | 41 +++---------------- 1 file changed, 6 insertions(+), 35 deletions(-) diff --git a/sysdeps/unix/sysv/linux/aarch64/makecontext.c b/sysdeps/unix/sysv/linux/aarch64/makecontext.c index 779f7e55aa..33824ae614 100644 --- a/sysdeps/unix/sysv/linux/aarch64/makecontext.c +++ b/sysdeps/unix/sysv/linux/aarch64/makecontext.c @@ -23,6 +23,7 @@ #include #include #include +#include "aarch64-gcs.h" #define GCS_MAGIC 0x47435300 @@ -72,47 +73,17 @@ __libc_aarch64_thread_freeres (void) free_gcs_list (); } -#ifndef __NR_map_shadow_stack -# define __NR_map_shadow_stack 453 -#endif -#ifndef SHADOW_STACK_SET_TOKEN -# define SHADOW_STACK_SET_TOKEN (1UL << 0) -# define SHADOW_STACK_SET_MARKER (1UL << 1) -#endif - -static void * -map_shadow_stack (void *addr, size_t size, unsigned long flags) -{ - return (void *) INLINE_SYSCALL_CALL (map_shadow_stack, addr, size, flags); -} - -#define GCS_MAX_SIZE (1UL << 31) -#define GCS_ALTSTACK_RESERVE 160 - static void * alloc_makecontext_gcs (size_t stack_size) { - size_t size = (stack_size / 2 + GCS_ALTSTACK_RESERVE) & -8UL; - if (size > GCS_MAX_SIZE) - size = GCS_MAX_SIZE; - - unsigned long flags = SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN; - void *base = map_shadow_stack (NULL, size, flags); - if (base == (void *) -1) + void *base; + size_t size; + void *gcsp = __alloc_gcs (stack_size, &base, &size); + if (gcsp == NULL) /* ENOSYS, bad size or OOM. */ abort (); - record_gcs (base, size); - - uint64_t *gcsp = (uint64_t *) ((char *) base + size); - /* Skip end of GCS token. */ - gcsp--; - /* Verify GCS cap token. */ - gcsp--; - if (((uint64_t)gcsp & 0xfffffffffffff000) + 1 != *gcsp) - abort (); - /* Return the target GCS pointer for context switch. */ - return gcsp + 1; + return gcsp; } void From patchwork Wed Oct 23 08:39:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Khrustalev X-Patchwork-Id: 99382 X-Patchwork-Delegate: Wilco.Dijkstra@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 87D363858404 for ; Wed, 23 Oct 2024 08:47:39 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id E724F3858C5F for ; Wed, 23 Oct 2024 08:44:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E724F3858C5F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E724F3858C5F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673101; cv=none; b=fLEg4DKW1iA0hxMjd7H8fFejsNPouLeAeDfX6Q0Kh8PVM3/TEVqGgfr/3NDtwGxts7tMXbeoHTO30n71hm2bNZzO9AH/V3qZlAbKNaiCCku6jqQhoQsyRsvpyCrZju1MPEcqe/lw9urKU0MCFgR4goFNTTloXUIH7KMN7m0lV20= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729673101; c=relaxed/simple; bh=0mjzYyBexG6r8PXXHlNRTZOmc4MiuC8/4jnwyMI1TNE=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=kP1pwYIuYvWtTIN5BRO9u62WaI//kQ9fU6kCX1skDwhTMKrOIhBqdeAvxt9gdjiik1rCj8KnMswslzNCAu3r5ZpYxqKAORGZKnPk3ghbqAzEoWzxIy2i+DVfTMqHRBbm6+WSrSNGtdSaj5NKhSRHV5NUJUtI88jw7454sitYuLA= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 666DE1576; Wed, 23 Oct 2024 01:45:20 -0700 (PDT) Received: from udebian.localdomain (unknown [10.1.39.30]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0ACC13F528; Wed, 23 Oct 2024 01:44:49 -0700 (PDT) From: Yury Khrustalev To: libc-alpha@sourceware.org Cc: fweimer@redhat.com, adhemerval.zanella@linaro.org, codonell@redhat.com, nsz@gcc.gnu.org Subject: [PATCH v3 23/23] doc: Add plain text readme for using GCS Date: Wed, 23 Oct 2024 09:39:20 +0100 Message-Id: <20241023083920.466015-24-yury.khrustalev@arm.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241023083920.466015-1-yury.khrustalev@arm.com> References: <20241023083920.466015-1-yury.khrustalev@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-14.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: Szabolcs Nagy TODO: this is just for the arm/gcs branch while it is being developed --- README | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/README b/README index 2e360eb70a..24a8ac4adf 100644 --- a/README +++ b/README @@ -1,3 +1,71 @@ +this branch contains experimental GCS support (not ABI stable) + +source and branches +------------------- + +binutils-gdb: upstream-git users/ARM/gcs-binutils-gdb-master +gcc (trunk): upstream-git vendors/ARM/gcs + note: gcc vendor branches need setup https://gcc.gnu.org/gitwrite.html#vendor +glibc: upstream-git arm/gcs +linux: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/gcs +fvp fast model can be used for testing + +toolchain build +--------------- + +two options: + +(1) branch-protect by default + configure gcc with --enable-standard-branch-protection + and build glibc normally + +(2) do not branch-protect by default, require explicit cflags + configure gcc with + CFLAGS_FOR_TARGET='-O2 -mbranch-protection=standard' + CXXFLAGS_FOR_TARGET='-O2 -mbranch-protection=standard' + and configure glibc with + CFLAGS='-g -O2 -mbranch-protection=standard' + build user code with + CFLAGS+=-mbranch-protection=standard + (equivalent to -mbranch-protection=bti+pac+gcs) + +linking +------- + +use ldflags: + +-z gcs={always,never,implicit} + always: force GCS marking on + never: force GCS marking off + implicit: mark output if all inputs are marked (default) + +-z gcs-report={none,warning,error} + none: silent (default) + warning: when output is marked, unmarked input is a warning + error: when output is marked, unmarked input is an error + +runtime +------- + +run with environment var + + GLIBC_TUNABLES=glibc.cpu.aarch64_gcs=1:glibc.cpu.aarch64_gcs_policy=2 + +by default both tunables are 0, the meaning is + +glibc.cpu.aarch64_gcs_policy=0: + GCS is enabled if glibc.cpu.aarch64_gcs is set +glibc.cpu.aarch64_gcs_policy=1: + GCS is enabled if glibc.cpu.aarch64_gcs is set and binary is marked + if GCS is enabled an incompatible dlopen is an error +glibc.cpu.aarch64_gcs_policy=2: + GCS is enabled if glibc.cpu.aarch64_gcs is set + if GCS is enabled any incompatible binary is an error + + +original readme +--------------- + This directory contains the sources of the GNU C Library. See the file "version.h" for what release version you have.