From patchwork Sat Nov 13 09:45:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Blinov X-Patchwork-Id: 47602 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5BEC73857C60 for ; Sat, 13 Nov 2021 09:46:47 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from tamago.local (unknown [212.69.42.53]) by sourceware.org (Postfix) with ESMTP id C81753858410 for ; Sat, 13 Nov 2021 09:45:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C81753858410 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=embecosm.com Authentication-Results: sourceware.org; spf=none smtp.mailfrom=tamago.local Received: by tamago.local (Postfix, from userid 506) id 6DEEF1A84B80; Sat, 13 Nov 2021 09:45:31 +0000 (GMT) From: Maxim Blinov To: gcc-patches@gcc.gnu.org Subject: [PATCH 1/4] Generate off-stack nested function trampolines Date: Sat, 13 Nov 2021 09:45:22 +0000 Message-Id: <20211113094525.96299-1-maxim.blinov@embecosm.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) MIME-Version: 1.0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, NO_DNS_FOR_FROM, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: iain@sandoe.co.uk, Andrew Burgess Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Add support for allocating nested function trampolines on an executable heap rather than on the stack. This is motivated by targets such as AArch64 Darwin, which globally prohibit executing code on the stack. The target-specific routines for allocating and writing trampolines is to be provided in libgcc, and is by-default _not_ compiled in unless the target specifically requires it, or you manually provide --enable-off-stack-trampolines when configuring gcc/libgcc. The gcc flag -foff-stack-trampolines controls whether to generate code that instantiates trampolines on the stack, or to emit calls to __builtin_nested_func_ptr_created and __builtin_nested_func_ptr_deleted. Note that this flag is completely independent of libgcc: If libgcc is for any reason missing those symbols, you will get a link failure. This implementation imposes some implicit restrictions as compared to stack trampolines. longjmp'ing back to a state before a trampoline was created will cause us to skip over the corresponding __builtin_nested_func_ptr_deleted, which will leak trampolines starting from the beginning of the linked list of allocated trampolines. There may be scope for instrumenting longjmp/setjmp to trigger cleanups of trampolines. Co-authored-by: Andrew Burgess gcc/ChangeLog: * builtins.def (BUILT_IN_NESTED_PTR_CREATED): Define. (BUILT_IN_NESTED_PTR_DELETED): Ditto. * common.opt (foff-stack-trampolines): Add flag to control generation of heap-based trampoline instantiation. * tree-nested.c (convert_tramp_reference_op): Don't bother calling __builtin_adjust_trampoline for the off-stack case. (finalize_nesting_tree_1): Emit calls to __builtin_nested_...{created,deleted} if we're generating with -foff-stack-trampolines. * tree.c (build_common_builtin_nodes): Build __builtin_nested_...{created,deleted}. * dov/invoke.texi (-foff-stack-trampolines): Document. libgcc/ChangeLog: * configure.ac: Add configure parameter --enable-off-stack-trampolines, and do error checking if we've trying to enable off-stack trampolines for a platform that doesn't provide any such implementation. * configure: Regenerate. * libgcc-std.ver.in: Ditto. * libgcc2.h (__builtin_nested_func_ptr_created): Declare. (__builtin_nested_func_ptr_deleted): Ditto. --- gcc/builtins.def | 2 + gcc/common.opt | 4 ++ gcc/config.gcc | 7 +++ gcc/doc/invoke.texi | 14 +++++ gcc/tree-nested.c | 121 +++++++++++++++++++++++++++++++++------ gcc/tree.c | 17 ++++++ libgcc/configure | 26 +++++++++ libgcc/configure.ac | 17 ++++++ libgcc/libgcc-std.ver.in | 3 + libgcc/libgcc2.h | 3 + 10 files changed, 197 insertions(+), 17 deletions(-) diff --git a/gcc/builtins.def b/gcc/builtins.def index 45a09b4d42d..90a94a6dd0f 100644 --- a/gcc/builtins.def +++ b/gcc/builtins.def @@ -950,6 +950,8 @@ DEF_BUILTIN_STUB (BUILT_IN_ADJUST_TRAMPOLINE, "__builtin_adjust_trampoline") DEF_BUILTIN_STUB (BUILT_IN_INIT_DESCRIPTOR, "__builtin_init_descriptor") DEF_BUILTIN_STUB (BUILT_IN_ADJUST_DESCRIPTOR, "__builtin_adjust_descriptor") DEF_BUILTIN_STUB (BUILT_IN_NONLOCAL_GOTO, "__builtin_nonlocal_goto") +DEF_BUILTIN_STUB (BUILT_IN_NESTED_PTR_CREATED, "__builtin_nested_func_ptr_created") +DEF_BUILTIN_STUB (BUILT_IN_NESTED_PTR_DELETED, "__builtin_nested_func_ptr_deleted") /* Implementing __builtin_setjmp. */ DEF_BUILTIN_STUB (BUILT_IN_SETJMP_SETUP, "__builtin_setjmp_setup") diff --git a/gcc/common.opt b/gcc/common.opt index de9b848eda5..a97aeeb2165 100644 --- a/gcc/common.opt +++ b/gcc/common.opt @@ -2149,6 +2149,10 @@ foffload-abi= Common Joined RejectNegative Enum(offload_abi) -foffload-abi=[lp64|ilp32] Set the ABI to use in an offload compiler. +foff-stack-trampolines +Common RejectNegative Var(flag_off_stack_trampolines) Init(OFF_STACK_TRAMPOLINES_INIT) +Generate trampolines in executable memory rather than executable stack. + Enum Name(offload_abi) Type(enum offload_abi) UnknownError(unknown offload ABI %qs) diff --git a/gcc/config.gcc b/gcc/config.gcc index edd12655c4a..c479aa4cc44 100644 --- a/gcc/config.gcc +++ b/gcc/config.gcc @@ -1070,6 +1070,13 @@ case ${target} in ;; esac +# Figure out if we need to enable -foff-stack-trampolines by default. +case ${target} in +*) + tm_defines="$tm_defines OFF_STACK_TRAMPOLINES_INIT=0" + ;; +esac + case ${target} in aarch64*-*-elf | aarch64*-*-fuchsia* | aarch64*-*-rtems*) tm_file="${tm_file} dbxelf.h elfos.h newlib-stdint.h" diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index 2aba4c70b44..a5db65f8721 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -660,6 +660,7 @@ Objective-C and Objective-C++ Dialects}. @gccoptlist{-fcall-saved-@var{reg} -fcall-used-@var{reg} @gol -ffixed-@var{reg} -fexceptions @gol -fnon-call-exceptions -fdelete-dead-exceptions -funwind-tables @gol +-foff-stack-trampolines @gol -fasynchronous-unwind-tables @gol -fno-gnu-unique @gol -finhibit-size-directive -fcommon -fno-ident @gol @@ -16683,6 +16684,19 @@ instructions. It does not allow exceptions to be thrown from arbitrary signal handlers such as @code{SIGALRM}. This enables @option{-fexceptions}. +@item -foff-stack-trampolines +@opindex foff-stack-trampolines +Certain platforms (such as the Apple M1) do not permit an executable +stack. Generate calls to @code{__builtin_nested_func_ptr_created} and +@code{__builtin_nested_func_ptr_deleted} in order to allocate and +deallocate trampoline space on the executable heap. Please note that +these functions are implemented in libgcc, and will not be compiled in +unless you provide @option{--enable-off-stack-trampolines} when +building gcc. @emph{PLEASE NOTE}: The trampolines are @emph{not} +guaranteed to be correctly deallocated if you @code{setjmp}, +instantiate nested functions, and then @code{longjmp} back to a state +prior to having allocated those nested functions. + @item -fdelete-dead-exceptions @opindex fdelete-dead-exceptions Consider that instructions that may throw exceptions but don't otherwise diff --git a/gcc/tree-nested.c b/gcc/tree-nested.c index c7f50ebd21c..a405c905e1d 100644 --- a/gcc/tree-nested.c +++ b/gcc/tree-nested.c @@ -611,6 +611,14 @@ get_trampoline_type (struct nesting_info *info) if (trampoline_type) return trampoline_type; + /* When trampolines are created off-stack then the only thing we need in the + local frame is a single pointer. */ + if (flag_off_stack_trampolines) + { + trampoline_type = build_pointer_type (void_type_node); + return trampoline_type; + } + align = TRAMPOLINE_ALIGNMENT; size = TRAMPOLINE_SIZE; @@ -2784,17 +2792,27 @@ convert_tramp_reference_op (tree *tp, int *walk_subtrees, void *data) /* Compute the address of the field holding the trampoline. */ x = get_frame_field (info, target_context, x, &wi->gsi); - x = build_addr (x); - x = gsi_gimplify_val (info, x, &wi->gsi); - /* Do machine-specific ugliness. Normally this will involve - computing extra alignment, but it can really be anything. */ - if (descr) - builtin = builtin_decl_implicit (BUILT_IN_ADJUST_DESCRIPTOR); + /* APB: We don't need to do the adjustment calls when using off-stack + trampolines, any such adjustment will be done when the off-stack + trampoline is created. */ + if (flag_off_stack_trampolines) + x = gsi_gimplify_val (info, x, &wi->gsi); else - builtin = builtin_decl_implicit (BUILT_IN_ADJUST_TRAMPOLINE); - call = gimple_build_call (builtin, 1, x); - x = init_tmp_var_with_call (info, &wi->gsi, call); + { + x = build_addr (x); + + x = gsi_gimplify_val (info, x, &wi->gsi); + + /* Do machine-specific ugliness. Normally this will involve + computing extra alignment, but it can really be anything. */ + if (descr) + builtin = builtin_decl_implicit (BUILT_IN_ADJUST_DESCRIPTOR); + else + builtin = builtin_decl_implicit (BUILT_IN_ADJUST_TRAMPOLINE); + call = gimple_build_call (builtin, 1, x); + x = init_tmp_var_with_call (info, &wi->gsi, call); + } /* Cast back to the proper function type. */ x = build1 (NOP_EXPR, TREE_TYPE (t), x); @@ -3373,6 +3391,7 @@ build_init_call_stmt (struct nesting_info *info, tree decl, tree field, static void finalize_nesting_tree_1 (struct nesting_info *root) { + gimple_seq cleanup_list = NULL; gimple_seq stmt_list = NULL; gimple *stmt; tree context = root->context; @@ -3504,9 +3523,48 @@ finalize_nesting_tree_1 (struct nesting_info *root) if (!field) continue; - x = builtin_decl_implicit (BUILT_IN_INIT_TRAMPOLINE); - stmt = build_init_call_stmt (root, i->context, field, x); - gimple_seq_add_stmt (&stmt_list, stmt); + if (flag_off_stack_trampolines) + { + /* We pass a whole bunch of arguments to the builtin function that + creates the off-stack trampoline, these are + 1. The nested function chain value (that must be passed to the + nested function so it can find the function arguments). + 2. A pointer to the nested function implementation, + 3. The address in the local stack frame where we should write + the address of the trampoline. + + When this code was originally written I just kind of threw + everything at the builtin, figuring I'd work out what was + actually needed later, I think, the stack pointer could + certainly be dropped, arguments #2 and #4 are based off the + stack pointer anyway, so #1 doesn't seem to add much value. */ + tree arg1, arg2, arg3; + + gcc_assert (DECL_STATIC_CHAIN (i->context)); + arg1 = build_addr (root->frame_decl); + arg2 = build_addr (i->context); + + x = build3 (COMPONENT_REF, TREE_TYPE (field), + root->frame_decl, field, NULL_TREE); + arg3 = build_addr (x); + + x = builtin_decl_implicit (BUILT_IN_NESTED_PTR_CREATED); + stmt = gimple_build_call (x, 3, arg1, arg2, arg3); + gimple_seq_add_stmt (&stmt_list, stmt); + + /* This call to delete the nested function trampoline is added to + the cleanup list, and called when we exit the current scope. */ + x = builtin_decl_implicit (BUILT_IN_NESTED_PTR_DELETED); + stmt = gimple_build_call (x, 0); + gimple_seq_add_stmt (&cleanup_list, stmt); + } + else + { + /* Original code to initialise the on stack trampoline. */ + x = builtin_decl_implicit (BUILT_IN_INIT_TRAMPOLINE); + stmt = build_init_call_stmt (root, i->context, field, x); + gimple_seq_add_stmt (&stmt_list, stmt); + } } } @@ -3531,11 +3589,40 @@ finalize_nesting_tree_1 (struct nesting_info *root) /* If we created initialization statements, insert them. */ if (stmt_list) { - gbind *bind; - annotate_all_with_location (stmt_list, DECL_SOURCE_LOCATION (context)); - bind = gimple_seq_first_stmt_as_a_bind (gimple_body (context)); - gimple_seq_add_seq (&stmt_list, gimple_bind_body (bind)); - gimple_bind_set_body (bind, stmt_list); + if (flag_off_stack_trampolines) + { + /* Handle the new, off stack trampolines. */ + gbind *bind; + annotate_all_with_location (stmt_list, DECL_SOURCE_LOCATION (context)); + annotate_all_with_location (cleanup_list, DECL_SOURCE_LOCATION (context)); + bind = gimple_seq_first_stmt_as_a_bind (gimple_body (context)); + gimple_seq_add_seq (&stmt_list, gimple_bind_body (bind)); + + gimple_seq xxx_list = NULL; + + if (cleanup_list != NULL) + { + /* We Maybe shouldn't be creating this try/finally if -fno-exceptions is + in use. If this is the case, then maybe we should, instead, be + inserting the cleanup code onto every path out of this function? Not + yet figured out how we would do this. */ + gtry *t = gimple_build_try (stmt_list, cleanup_list, GIMPLE_TRY_FINALLY); + gimple_seq_add_stmt (&xxx_list, t); + } + else + xxx_list = stmt_list; + + gimple_bind_set_body (bind, xxx_list); + } + else + { + /* The traditional, on stack trampolines. */ + gbind *bind; + annotate_all_with_location (stmt_list, DECL_SOURCE_LOCATION (context)); + bind = gimple_seq_first_stmt_as_a_bind (gimple_body (context)); + gimple_seq_add_seq (&stmt_list, gimple_bind_body (bind)); + gimple_bind_set_body (bind, stmt_list); + } } /* If a chain_decl was created, then it needs to be registered with diff --git a/gcc/tree.c b/gcc/tree.c index f2c829fa4c6..968963a3127 100644 --- a/gcc/tree.c +++ b/gcc/tree.c @@ -9648,6 +9648,23 @@ build_common_builtin_nodes (void) "__builtin_nonlocal_goto", ECF_NORETURN | ECF_NOTHROW); + tree ptr_ptr_type_node = build_pointer_type (ptr_type_node); + + ftype = build_function_type_list (void_type_node, + ptr_type_node, // void *chain + ptr_type_node, // void *func + ptr_ptr_type_node, // void **dst + NULL_TREE); + local_define_builtin ("__builtin_nested_func_ptr_created", ftype, + BUILT_IN_NESTED_PTR_CREATED, + "__builtin_nested_func_ptr_created", ECF_NOTHROW); + + ftype = build_function_type_list (void_type_node, + NULL_TREE); + local_define_builtin ("__builtin_nested_func_ptr_deleted", ftype, + BUILT_IN_NESTED_PTR_DELETED, + "__builtin_nested_func_ptr_deleted", ECF_NOTHROW); + ftype = build_function_type_list (void_type_node, ptr_type_node, ptr_type_node, NULL_TREE); local_define_builtin ("__builtin_setjmp_setup", ftype, diff --git a/libgcc/configure b/libgcc/configure index 4919a56f518..2f469219e07 100755 --- a/libgcc/configure +++ b/libgcc/configure @@ -654,6 +654,7 @@ build with_aix_soname enable_vtable_verify enable_gcov +off_stack_trampolines enable_shared libgcc_topdir target_alias @@ -701,6 +702,7 @@ with_target_subdir with_cross_host with_ld enable_shared +enable_off_stack_trampolines enable_gcov enable_vtable_verify with_aix_soname @@ -1342,6 +1344,9 @@ Optional Features: --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] --disable-shared don't provide a shared libgcc + --enable-off-stack-trampolines + Specify whether to support generating off-stack trampolines + --disable-gcov don't provide libgcov and related host tools --enable-vtable-verify Enable vtable verification feature --enable-version-specific-runtime-libs Specify that runtime libraries should be installed in a compiler-specific directory @@ -2252,6 +2257,27 @@ fi +# Check whether --enable-off-stack-trampolines was given. +if test "${enable_off_stack_trampolines+set}" = set; then : + enableval=$enable_off_stack_trampolines; +case "$target" in + *) + as_fn_error $? "Configure option --enable-off-stack-trampolines is not supported \ +for this platform" "$LINENO" 5 + off_stack_trampolines=no + ;; +esac +else + +case "$target" in + *) + off_stack_trampolines=no + ;; +esac +fi + + + # Check whether --enable-gcov was given. if test "${enable_gcov+set}" = set; then : enableval=$enable_gcov; diff --git a/libgcc/configure.ac b/libgcc/configure.ac index 13a80b2551b..97bbd4bd35c 100644 --- a/libgcc/configure.ac +++ b/libgcc/configure.ac @@ -68,6 +68,23 @@ AC_ARG_ENABLE(shared, ], [enable_shared=yes]) AC_SUBST(enable_shared) +AC_ARG_ENABLE([off-stack-trampolines], + [AS_HELP_STRING([--enable-off-stack-trampolines] + [Specify whether to support generating off-stack trampolines])],[ +case "$target" in + *) + AC_MSG_ERROR([Configure option --enable-off-stack-trampolines is not supported \ +for this platform]) + off_stack_trampolines=no + ;; +esac],[ +case "$target" in + *) + off_stack_trampolines=no + ;; +esac]) +AC_SUBST(off_stack_trampolines) + AC_ARG_ENABLE(gcov, [ --disable-gcov don't provide libgcov and related host tools], [], [enable_gcov=yes]) diff --git a/libgcc/libgcc-std.ver.in b/libgcc/libgcc-std.ver.in index cea33267e53..f26ad3cdf5d 100644 --- a/libgcc/libgcc-std.ver.in +++ b/libgcc/libgcc-std.ver.in @@ -1943,4 +1943,7 @@ GCC_4.8.0 { GCC_7.0.0 { __PFX__divmoddi4 __PFX__divmodti4 + + __builtin_nested_func_ptr_created + __builtin_nested_func_ptr_deleted } diff --git a/libgcc/libgcc2.h b/libgcc/libgcc2.h index 1819ff3ac3d..1a448c02c04 100644 --- a/libgcc/libgcc2.h +++ b/libgcc/libgcc2.h @@ -29,6 +29,9 @@ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see #pragma GCC visibility push(default) #endif +extern void __builtin_nested_func_ptr_created (void *, void *, void **); +extern void __builtin_nested_func_ptr_deleted (void); + extern int __gcc_bcmp (const unsigned char *, const unsigned char *, size_t); extern void __clear_cache (void *, void *); extern void __eprintf (const char *, const char *, unsigned int, const char *) From patchwork Sat Nov 13 09:45:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Blinov X-Patchwork-Id: 47601 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 376283857C73 for ; Sat, 13 Nov 2021 09:46:18 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from tamago.local (unknown [212.69.42.53]) by sourceware.org (Postfix) with ESMTP id C3650385840E for ; Sat, 13 Nov 2021 09:45:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C3650385840E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=embecosm.com Authentication-Results: sourceware.org; spf=none smtp.mailfrom=tamago.local Received: by tamago.local (Postfix, from userid 506) id 6B9BA1A84B7F; Sat, 13 Nov 2021 09:45:31 +0000 (GMT) From: Maxim Blinov To: gcc-patches@gcc.gnu.org Subject: [PATCH 2/4] Add x86_64-linux support for off-stack trampolines Date: Sat, 13 Nov 2021 09:45:23 +0000 Message-Id: <20211113094525.96299-2-maxim.blinov@embecosm.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20211113094525.96299-1-maxim.blinov@embecosm.com> References: <20211113094525.96299-1-maxim.blinov@embecosm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, NO_DNS_FOR_FROM, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: iain@sandoe.co.uk, Andrew Burgess Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Implement the __builtin_nested_func_ptr_{created,deleted} functions for the x86_64-linux platform. This serves to exercise the infrastructure added in libgcc (--enable-off-stack-trampolines) and gcc (-foff-stack-trampolines) in supporting off-stack trampoline generation, and is intended primarily for demonstration and debugging purposes. Co-authored-by: Andrew Burgess libgcc/ChangeLog: * config/i386/heap-trampoline.c: New file: Implement off-stack trampolines for x86_64. * config/i386/t-heap-trampoline: Add rule to build config/i386/heap-trampoline.c * config.host (x86_64-*-linux*): Handle --enable-off-stack-trampolines. * configure.ac (--enable-off-stack-trampolines): Permit setting for target x86_64-*-linux*. * configure: Regenerate. --- libgcc/config.host | 4 + libgcc/config/i386/heap-trampoline.c | 143 +++++++++++++++++++++++++++ libgcc/config/i386/t-heap-trampoline | 21 ++++ libgcc/configure | 3 + libgcc/configure.ac | 3 + 5 files changed, 174 insertions(+) create mode 100644 libgcc/config/i386/heap-trampoline.c create mode 100644 libgcc/config/i386/t-heap-trampoline diff --git a/libgcc/config.host b/libgcc/config.host index 168535b1780..163cd4c4161 100644 --- a/libgcc/config.host +++ b/libgcc/config.host @@ -753,6 +753,10 @@ x86_64-*-linux*) tmake_file="${tmake_file} i386/t-crtpc t-crtfm i386/t-crtstuff t-dfprules" tm_file="${tm_file} i386/elf-lib.h" md_unwind_header=i386/linux-unwind.h + if test x$off_stack_trampolines = xyes; then + extra_parts="${extra_parts} heap-trampoline.o" + tmake_file="${tmake_file} i386/t-heap-trampoline" + fi ;; x86_64-*-kfreebsd*-gnu) extra_parts="$extra_parts crtprec32.o crtprec64.o crtprec80.o crtfastmath.o" diff --git a/libgcc/config/i386/heap-trampoline.c b/libgcc/config/i386/heap-trampoline.c new file mode 100644 index 00000000000..6c202660c35 --- /dev/null +++ b/libgcc/config/i386/heap-trampoline.c @@ -0,0 +1,143 @@ +#include +#include +#include +#include +#include +#include + +void *allocate_trampoline_page (void); +int get_trampolines_per_page (void); +struct tramp_ctrl_data *allocate_tramp_ctrl (struct tramp_ctrl_data *parent); +void *allocate_trampoline_page (void); + +void __builtin_nested_func_ptr_created (void *chain, void *func, void **dst); +void __builtin_nested_func_ptr_deleted (void); + +struct tramp_ctrl_data; +struct tramp_ctrl_data +{ + struct tramp_ctrl_data *prev; + + int free_trampolines; + + /* This will be pointing to an executable mmap'ed page. */ + union ix86_trampoline *trampolines; +}; + +static const uint8_t trampoline_insns[] = { + /* movabs $,%r11 */ + 0x49, 0xbb, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + + /* movabs $,%r10 */ + 0x49, 0xba, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + + /* rex.WB jmpq *%r11 */ + 0x41, 0xff, 0xe3 +}; + +union ix86_trampoline { + uint8_t insns[sizeof(trampoline_insns)]; + + struct __attribute__((packed)) fields { + uint8_t insn_0[2]; + void *func_ptr; + uint8_t insn_1[2]; + void *chain_ptr; + uint8_t insn_2[3]; + } fields; +}; + +int +get_trampolines_per_page (void) +{ + return getpagesize() / sizeof(union ix86_trampoline); +} + +static _Thread_local struct tramp_ctrl_data *tramp_ctrl_curr = NULL; + +void * +allocate_trampoline_page (void) +{ + void *page; + + page = mmap (0, getpagesize (), PROT_WRITE | PROT_EXEC, + MAP_ANON | MAP_PRIVATE, 0, 0); + + return page; +} + +struct tramp_ctrl_data * +allocate_tramp_ctrl (struct tramp_ctrl_data *parent) +{ + struct tramp_ctrl_data *p = malloc (sizeof (struct tramp_ctrl_data)); + if (p == NULL) + return NULL; + + p->trampolines = allocate_trampoline_page (); + + if (p->trampolines == MAP_FAILED) + return NULL; + + p->prev = parent; + p->free_trampolines = get_trampolines_per_page(); + + return p; +} + +void +__builtin_nested_func_ptr_created (void *chain, void *func, void **dst) +{ + if (tramp_ctrl_curr == NULL) + { + tramp_ctrl_curr = allocate_tramp_ctrl (NULL); + if (tramp_ctrl_curr == NULL) + abort (); + } + + if (tramp_ctrl_curr->free_trampolines == 0) + { + void *tramp_ctrl = allocate_tramp_ctrl (tramp_ctrl_curr); + if (!tramp_ctrl) + abort (); + + tramp_ctrl_curr = tramp_ctrl; + } + + union ix86_trampoline *trampoline + = &tramp_ctrl_curr->trampolines[get_trampolines_per_page () + - tramp_ctrl_curr->free_trampolines]; + + memcpy (trampoline->insns, trampoline_insns, + sizeof(trampoline_insns)); + trampoline->fields.func_ptr = func; + trampoline->fields.chain_ptr = chain; + + tramp_ctrl_curr->free_trampolines -= 1; + + __builtin___clear_cache ((void *)trampoline->insns, + ((void *)trampoline->insns + sizeof(trampoline->insns))); + + *dst = &trampoline->insns; +} + +void +__builtin_nested_func_ptr_deleted (void) +{ + if (tramp_ctrl_curr == NULL) + abort (); + + tramp_ctrl_curr->free_trampolines += 1; + + if (tramp_ctrl_curr->free_trampolines == get_trampolines_per_page ()) + { + if (tramp_ctrl_curr->prev == NULL) + return; + + munmap (tramp_ctrl_curr->trampolines, getpagesize()); + struct tramp_ctrl_data *prev = tramp_ctrl_curr->prev; + free (tramp_ctrl_curr); + tramp_ctrl_curr = prev; + } +} diff --git a/libgcc/config/i386/t-heap-trampoline b/libgcc/config/i386/t-heap-trampoline new file mode 100644 index 00000000000..d9dc9cfeac4 --- /dev/null +++ b/libgcc/config/i386/t-heap-trampoline @@ -0,0 +1,21 @@ +# Machine description for AArch64 architecture. +# Copyright (C) 2012-2021 Free Software Foundation, Inc. +# Contributed by ARM Ltd. +# +# This file is part of GCC. +# +# GCC is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GCC is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GCC; see the file COPYING3. If not see +# . + +LIB2ADD += $(srcdir)/config/i386/heap-trampoline.c diff --git a/libgcc/configure b/libgcc/configure index 2f469219e07..d9a74e3cc7c 100755 --- a/libgcc/configure +++ b/libgcc/configure @@ -2261,6 +2261,9 @@ fi if test "${enable_off_stack_trampolines+set}" = set; then : enableval=$enable_off_stack_trampolines; case "$target" in + x86_64-*-linux* ) + off_stack_trampolines=$enableval + ;; *) as_fn_error $? "Configure option --enable-off-stack-trampolines is not supported \ for this platform" "$LINENO" 5 diff --git a/libgcc/configure.ac b/libgcc/configure.ac index 97bbd4bd35c..4bec0a54493 100644 --- a/libgcc/configure.ac +++ b/libgcc/configure.ac @@ -72,6 +72,9 @@ AC_ARG_ENABLE([off-stack-trampolines], [AS_HELP_STRING([--enable-off-stack-trampolines] [Specify whether to support generating off-stack trampolines])],[ case "$target" in + x86_64-*-linux* ) + off_stack_trampolines=$enableval + ;; *) AC_MSG_ERROR([Configure option --enable-off-stack-trampolines is not supported \ for this platform]) From patchwork Sat Nov 13 09:45:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Blinov X-Patchwork-Id: 47600 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2462C3858015 for ; Sat, 13 Nov 2021 09:45:49 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from tamago.local (unknown [212.69.42.53]) by sourceware.org (Postfix) with ESMTP id D0576385841A for ; Sat, 13 Nov 2021 09:45:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D0576385841A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=embecosm.com Authentication-Results: sourceware.org; spf=none smtp.mailfrom=tamago.local Received: by tamago.local (Postfix, from userid 506) id 6E6131A84B81; Sat, 13 Nov 2021 09:45:31 +0000 (GMT) From: Maxim Blinov To: gcc-patches@gcc.gnu.org Subject: [PATCH 3/4] Add aarch64-linux support for off-stack trampolines Date: Sat, 13 Nov 2021 09:45:24 +0000 Message-Id: <20211113094525.96299-3-maxim.blinov@embecosm.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20211113094525.96299-1-maxim.blinov@embecosm.com> References: <20211113094525.96299-1-maxim.blinov@embecosm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, NO_DNS_FOR_FROM, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: iain@sandoe.co.uk, Andrew Burgess Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Implement the __builtin_nested_func_ptr_{created,deleted} functions for the aarch64-linux platform. This serves to exercise the infrastructure added in libgcc (--enable-off-stack-trampolines) and gcc (-foff-stack-trampolines) in supporting off-stack trampoline generation, and is intended primarily for demonstration and debugging purposes. Co-authored-by: Andrew Burgess libgcc/ChangeLog: * config/aarch64/heap-trampoline.c: New file: Implement off-stack trampolines for aarch64. * config/aarch64/t-heap-trampoline: Add rule to build config/aarch64/heap-trampoline.c * config.host (aarch64-*-linux*): Handle --enable-off-stack-trampolines. * configure.ac (--enable-off-stack-trampolines): Permit setting for target aarch64-*-linux*. * configure: Regenerate. --- libgcc/config.host | 4 + libgcc/config/aarch64/heap-trampoline.c | 133 ++++++++++++++++++++++++ libgcc/config/aarch64/t-heap-trampoline | 21 ++++ libgcc/configure | 3 + libgcc/configure.ac | 3 + 5 files changed, 164 insertions(+) create mode 100644 libgcc/config/aarch64/heap-trampoline.c create mode 100644 libgcc/config/aarch64/t-heap-trampoline diff --git a/libgcc/config.host b/libgcc/config.host index 163cd4c4161..912477db7d9 100644 --- a/libgcc/config.host +++ b/libgcc/config.host @@ -388,6 +388,10 @@ aarch64*-*-linux*) tmake_file="${tmake_file} ${cpu_type}/t-aarch64" tmake_file="${tmake_file} ${cpu_type}/t-lse t-slibgcc-libgcc" tmake_file="${tmake_file} ${cpu_type}/t-softfp t-softfp t-crtfm" + if test x$off_stack_trampolines = xyes; then + extra_parts="$extra_parts heap-trampoline.o" + tmake_file="${tmake_file} ${cpu_type}/t-heap-trampoline" + fi ;; aarch64*-*-vxworks7*) extra_parts="$extra_parts crtfastmath.o" diff --git a/libgcc/config/aarch64/heap-trampoline.c b/libgcc/config/aarch64/heap-trampoline.c new file mode 100644 index 00000000000..721a2bed400 --- /dev/null +++ b/libgcc/config/aarch64/heap-trampoline.c @@ -0,0 +1,133 @@ +#include +#include +#include +#include +#include +#include + +void *allocate_trampoline_page (void); +int get_trampolines_per_page (void); +struct tramp_ctrl_data *allocate_tramp_ctrl (struct tramp_ctrl_data *parent); +void *allocate_trampoline_page (void); + +void __builtin_nested_func_ptr_created (void *chain, void *func, void **dst); +void __builtin_nested_func_ptr_deleted (void); + +struct tramp_ctrl_data; +struct tramp_ctrl_data +{ + struct tramp_ctrl_data *prev; + + int free_trampolines; + + /* This will be pointing to an executable mmap'ed page. */ + struct aarch64_trampoline *trampolines; +}; + +struct aarch64_trampoline { + uint32_t insns[6]; + void *func_ptr; + void *chain_ptr; +}; + +int +get_trampolines_per_page (void) +{ + return getpagesize() / sizeof(struct aarch64_trampoline); +} + +static _Thread_local struct tramp_ctrl_data *tramp_ctrl_curr = NULL; + +void * +allocate_trampoline_page (void) +{ + void *page; + + page = mmap (0, getpagesize (), PROT_WRITE | PROT_EXEC, + MAP_ANON | MAP_PRIVATE, 0, 0); + + return page; +} + +struct tramp_ctrl_data * +allocate_tramp_ctrl (struct tramp_ctrl_data *parent) +{ + struct tramp_ctrl_data *p = malloc (sizeof (struct tramp_ctrl_data)); + if (p == NULL) + return NULL; + + p->trampolines = allocate_trampoline_page (); + + if (p->trampolines == MAP_FAILED) + return NULL; + + p->prev = parent; + p->free_trampolines = get_trampolines_per_page(); + + return p; +} + +static const uint32_t aarch64_trampoline_insns[] = { + 0xd503245f, /* hint 34 */ + 0x580000b1, /* ldr x17, .+20 */ + 0x580000d2, /* ldr x18, .+24 */ + 0xd61f0220, /* br x17 */ + 0xd5033f9f, /* dsb sy */ + 0xd5033fdf /* isb */ +}; + +void +__builtin_nested_func_ptr_created (void *chain, void *func, void **dst) +{ + if (tramp_ctrl_curr == NULL) + { + tramp_ctrl_curr = allocate_tramp_ctrl (NULL); + if (tramp_ctrl_curr == NULL) + abort (); + } + + if (tramp_ctrl_curr->free_trampolines == 0) + { + void *tramp_ctrl = allocate_tramp_ctrl (tramp_ctrl_curr); + if (!tramp_ctrl) + abort (); + + tramp_ctrl_curr = tramp_ctrl; + } + + struct aarch64_trampoline *trampoline + = &tramp_ctrl_curr->trampolines[get_trampolines_per_page () + - tramp_ctrl_curr->free_trampolines]; + + memcpy (trampoline->insns, aarch64_trampoline_insns, + sizeof(aarch64_trampoline_insns)); + trampoline->func_ptr = func; + trampoline->chain_ptr = chain; + + tramp_ctrl_curr->free_trampolines -= 1; + + __builtin___clear_cache ((void *)trampoline->insns, + ((void *)trampoline->insns + sizeof(trampoline->insns))); + + *dst = &trampoline->insns; +} + +void +__builtin_nested_func_ptr_deleted (void) +{ + if (tramp_ctrl_curr == NULL) + abort (); + + tramp_ctrl_curr->free_trampolines += 1; + + if (tramp_ctrl_curr->free_trampolines == get_trampolines_per_page ()) + { + if (tramp_ctrl_curr->prev == NULL) + return; + + munmap (tramp_ctrl_curr->trampolines, getpagesize()); + struct tramp_ctrl_data *prev = tramp_ctrl_curr->prev; + free (tramp_ctrl_curr); + tramp_ctrl_curr = prev; + } +} diff --git a/libgcc/config/aarch64/t-heap-trampoline b/libgcc/config/aarch64/t-heap-trampoline new file mode 100644 index 00000000000..26b7756265e --- /dev/null +++ b/libgcc/config/aarch64/t-heap-trampoline @@ -0,0 +1,21 @@ +# Machine description for AArch64 architecture. +# Copyright (C) 2012-2021 Free Software Foundation, Inc. +# Contributed by ARM Ltd. +# +# This file is part of GCC. +# +# GCC is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GCC is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GCC; see the file COPYING3. If not see +# . + +LIB2ADD += $(srcdir)/config/aarch64/heap-trampoline.c diff --git a/libgcc/configure b/libgcc/configure index d9a74e3cc7c..5abcea8bed3 100755 --- a/libgcc/configure +++ b/libgcc/configure @@ -2264,6 +2264,9 @@ case "$target" in x86_64-*-linux* ) off_stack_trampolines=$enableval ;; + aarch64*-*-linux* ) + off_stack_trampolines=$enableval + ;; *) as_fn_error $? "Configure option --enable-off-stack-trampolines is not supported \ for this platform" "$LINENO" 5 diff --git a/libgcc/configure.ac b/libgcc/configure.ac index 4bec0a54493..c6eaceec957 100644 --- a/libgcc/configure.ac +++ b/libgcc/configure.ac @@ -75,6 +75,9 @@ case "$target" in x86_64-*-linux* ) off_stack_trampolines=$enableval ;; + aarch64*-*-linux* ) + off_stack_trampolines=$enableval + ;; *) AC_MSG_ERROR([Configure option --enable-off-stack-trampolines is not supported \ for this platform]) From patchwork Sat Nov 13 09:45:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Blinov X-Patchwork-Id: 47603 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3203F3858410 for ; Sat, 13 Nov 2021 09:47:16 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from tamago.local (unknown [212.69.42.53]) by sourceware.org (Postfix) with ESMTP id D49603858427 for ; Sat, 13 Nov 2021 09:45:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D49603858427 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=embecosm.com Authentication-Results: sourceware.org; spf=none smtp.mailfrom=tamago.local Received: by tamago.local (Postfix, from userid 506) id 6ED2C1A84B82; Sat, 13 Nov 2021 09:45:31 +0000 (GMT) From: Maxim Blinov To: gcc-patches@gcc.gnu.org Subject: [PATCH 4/4] Add aarch64-darwin support for off-stack trampolines Date: Sat, 13 Nov 2021 09:45:25 +0000 Message-Id: <20211113094525.96299-4-maxim.blinov@embecosm.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20211113094525.96299-1-maxim.blinov@embecosm.com> References: <20211113094525.96299-1-maxim.blinov@embecosm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, NO_DNS_FOR_FROM, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: iain@sandoe.co.uk, Andrew Burgess Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Note: This patch is not yet ready for trunk as its dependent on some patches that are not-yet-upstream, however it serves as motivation for the previous patch(es) which are independent. ---- Implement the __builtin_nested_func_ptr_{created,deleted} functions for the aarch64-darwin platform. For this platform --enable-off-stack-trampolines is enabled by default, and -foff-stack-trampolines is enabled by default if the host MacOS operating system version is 11.x or greater. Co-authored-by: Andrew Burgess libgcc/ChangeLog: * config/aarch64/heap-trampoline.c (allocate_trampoline_page): Request for MAP_JIT in the case of __APPLE__. Provide __APPLE__ variant of aarch64_trampoline_insns that uses x16 as the chain pointer. (__builtin_nested_func_ptr_created): Call pthread_jit_write_protect_np() to toggle read/write permission on page. * config.host (aarch64*-*darwin* | arm64*-*darwin*): Handle --enable-off-stack-trampolines. * configure.ac (--enable-off-stack-trampolines): Permit setting for target aarch64*-*darwin* | arm64*-*darwin*, and set default to enabled. * configure: Regenerate. --- gcc/config.gcc | 7 +++++ libgcc/config.host | 4 +++ libgcc/config/aarch64/heap-trampoline.c | 36 +++++++++++++++++++++++++ libgcc/configure | 6 +++++ libgcc/configure.ac | 6 +++++ 5 files changed, 59 insertions(+) diff --git a/gcc/config.gcc b/gcc/config.gcc index 031be563c5d..c13f7629d44 100644 --- a/gcc/config.gcc +++ b/gcc/config.gcc @@ -1072,6 +1072,13 @@ esac # Figure out if we need to enable -foff-stack-trampolines by default. case ${target} in +aarch64*-*darwin* | arm64*-*darwin*) + if test ${macos_maj} = 11 || test ${macos_maj} = 12; then + tm_defines="$tm_defines OFF_STACK_TRAMPOLINES_INIT=1" + else + tm_defines="$tm_defines OFF_STACK_TRAMPOLINES_INIT=0" + fi + ;; *) tm_defines="$tm_defines OFF_STACK_TRAMPOLINES_INIT=0" ;; diff --git a/libgcc/config.host b/libgcc/config.host index d1a491d27e7..3c536b0928a 100644 --- a/libgcc/config.host +++ b/libgcc/config.host @@ -414,6 +414,10 @@ aarch64*-*darwin* | arm64*-*darwin* ) tmake_file="${tmake_file} t-crtfm" # No soft float for now because our long double is DF not TF. md_unwind_header=aarch64/aarch64-unwind.h + if test x$off_stack_trampolines = xyes; then + extra_parts="$extra_parts heap-trampoline.o" + tmake_file="${tmake_file} ${cpu_type}/t-heap-trampoline" + fi ;; aarch64*-*-freebsd*) extra_parts="$extra_parts crtfastmath.o" diff --git a/libgcc/config/aarch64/heap-trampoline.c b/libgcc/config/aarch64/heap-trampoline.c index 721a2bed400..6994602beaf 100644 --- a/libgcc/config/aarch64/heap-trampoline.c +++ b/libgcc/config/aarch64/heap-trampoline.c @@ -5,6 +5,9 @@ #include #include +/* For pthread_jit_write_protect_np */ +#include + void *allocate_trampoline_page (void); int get_trampolines_per_page (void); struct tramp_ctrl_data *allocate_tramp_ctrl (struct tramp_ctrl_data *parent); @@ -43,8 +46,15 @@ allocate_trampoline_page (void) { void *page; +#if defined(__gnu_linux__) page = mmap (0, getpagesize (), PROT_WRITE | PROT_EXEC, MAP_ANON | MAP_PRIVATE, 0, 0); +#elif defined(__APPLE__) + page = mmap (0, getpagesize (), PROT_WRITE | PROT_EXEC, + MAP_ANON | MAP_PRIVATE | MAP_JIT, 0, 0); +#else + page = MAP_FAILED; +#endif return page; } @@ -67,6 +77,7 @@ allocate_tramp_ctrl (struct tramp_ctrl_data *parent) return p; } +#if defined(__gnu_linux__) static const uint32_t aarch64_trampoline_insns[] = { 0xd503245f, /* hint 34 */ 0x580000b1, /* ldr x17, .+20 */ @@ -76,6 +87,20 @@ static const uint32_t aarch64_trampoline_insns[] = { 0xd5033fdf /* isb */ }; +#elif defined(__APPLE__) +static const uint32_t aarch64_trampoline_insns[] = { + 0xd503245f, /* hint 34 */ + 0x580000b1, /* ldr x17, .+20 */ + 0x580000d0, /* ldr x16, .+24 */ + 0xd61f0220, /* br x17 */ + 0xd5033f9f, /* dsb sy */ + 0xd5033fdf /* isb */ +}; + +#else +#error "Unsupported AArch64 platform for heap trampolines" +#endif + void __builtin_nested_func_ptr_created (void *chain, void *func, void **dst) { @@ -99,11 +124,22 @@ __builtin_nested_func_ptr_created (void *chain, void *func, void **dst) = &tramp_ctrl_curr->trampolines[get_trampolines_per_page () - tramp_ctrl_curr->free_trampolines]; +#if defined(__APPLE__) + /* Disable write protection for the MAP_JIT regions in this thread (see + https://developer.apple.com/documentation/apple-silicon/porting-just-in-time-compilers-to-apple-silicon) */ + pthread_jit_write_protect_np (0); +#endif + memcpy (trampoline->insns, aarch64_trampoline_insns, sizeof(aarch64_trampoline_insns)); trampoline->func_ptr = func; trampoline->chain_ptr = chain; +#if defined(__APPLE__) + /* Re-enable write protection. */ + pthread_jit_write_protect_np (1); +#endif + tramp_ctrl_curr->free_trampolines -= 1; __builtin___clear_cache ((void *)trampoline->insns, diff --git a/libgcc/configure b/libgcc/configure index 5abcea8bed3..35cdf8f8c05 100755 --- a/libgcc/configure +++ b/libgcc/configure @@ -2267,6 +2267,9 @@ case "$target" in aarch64*-*-linux* ) off_stack_trampolines=$enableval ;; + aarch64*-*darwin* | arm64*-*darwin* ) + off_stack_trampolines=$enableval + ;; *) as_fn_error $? "Configure option --enable-off-stack-trampolines is not supported \ for this platform" "$LINENO" 5 @@ -2276,6 +2279,9 @@ esac else case "$target" in + aarch64*-*darwin* | arm64*-*darwin* ) + off_stack_trampolines=yes + ;; *) off_stack_trampolines=no ;; diff --git a/libgcc/configure.ac b/libgcc/configure.ac index c6eaceec957..3b129f1a4b8 100644 --- a/libgcc/configure.ac +++ b/libgcc/configure.ac @@ -78,6 +78,9 @@ case "$target" in aarch64*-*-linux* ) off_stack_trampolines=$enableval ;; + aarch64*-*darwin* | arm64*-*darwin* ) + off_stack_trampolines=$enableval + ;; *) AC_MSG_ERROR([Configure option --enable-off-stack-trampolines is not supported \ for this platform]) @@ -85,6 +88,9 @@ for this platform]) ;; esac],[ case "$target" in + aarch64*-*darwin* | arm64*-*darwin* ) + off_stack_trampolines=yes + ;; *) off_stack_trampolines=no ;;