From patchwork Fri Jun 10 16:35:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 55023 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7FAA5385AE7D for ; Fri, 10 Jun 2022 16:37:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 7FAA5385AE7D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1654879063; bh=sBtvDfPL2GyBY6Ot9Jds/K/8+rkxeeZ3UL+Y8bCTEbc=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=yosYkXmVR9k9nhbTaOEnr7RsVPFfn65z3VPJAoqYMkmXfdrpUDwWTrZsiM9CRuzIx GEJz9j6sIsKIfMBl/f57+VSeO6ftZ3IcTKbEKsr2IOlwAlEORazaOh/4I5fJcUObDM pbk0c6EUqPns5BZwGmW4SZ3xHogemz0fbMbXDPmE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by sourceware.org (Postfix) with ESMTPS id 6166938376E9 for ; Fri, 10 Jun 2022 16:35:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6166938376E9 Received: by mail-oi1-x234.google.com with SMTP id w16so26330191oie.5 for ; Fri, 10 Jun 2022 09:35:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sBtvDfPL2GyBY6Ot9Jds/K/8+rkxeeZ3UL+Y8bCTEbc=; b=wiRObC/+zOO6+XLU7IJTPBRCatCWF+XRQL4NPgDe8OZ3MnlXHJltm5gcaP35Nkg4Da C8IxYdGTkqCss9BV9m8RcbmLbnC+uEmGrGUL2LK30U1Zwxv4AuG2MqgEnia++BS9XBgm 9XqA1kzRpl9XnhMoqfcSEcPvDwHrPsxgZRNoJlFwRv3Wd3bNBfmApVt4+/sX/ROuOc21 J5HZYFXBbq0gulnT5HX2JLw7zVJivf6k2SR1thzdIXwnJfdCR3AeOncti+cD7vv+VVai waf00OiPjqz7yQuUQMlzQXNbMbaECjUnAJXQhlVf6Syp8TFCJ790AF08uCodMoimXXrw At9Q== X-Gm-Message-State: AOAM533ywNr9SG+2QgiMlX21I6r7fIW5bZA0vSzJAww70356Fjz/rAFk 5gWpc7v0SODucXyyB8T8Lbb5wiL+HYgK9Q== X-Google-Smtp-Source: ABdhPJzz+R3KYu3RJnrQagOdZq7+xxpZgtiBEc8zjN5uvDzSpcvwEvLpRobkBxm/RKEnXkohg7uIHQ== X-Received: by 2002:a05:6808:171a:b0:2f9:ac95:fc5b with SMTP id bc26-20020a056808171a00b002f9ac95fc5bmr350400oib.191.1654878958409; Fri, 10 Jun 2022 09:35:58 -0700 (PDT) Received: from birita.. ([2804:431:c7cb:a613:818b:b86c:a3f8:d455]) by smtp.gmail.com with ESMTPSA id p203-20020aca42d4000000b0032efe5871b0sm1262424oia.45.2022.06.10.09.35.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 09:35:58 -0700 (PDT) To: libc-alpha@sourceware.org, Wilco Dijkstra Subject: [PATCH v2 1/4] misc: Optimize internal usage of __libc_single_threaded Date: Fri, 10 Jun 2022 13:35:49 -0300 Message-Id: <20220610163552.3587064-2-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220610163552.3587064-1-adhemerval.zanella@linaro.org> References: <20220610163552.3587064-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" By adding an internal hidden_def alias to avoid the GOT indirection. On some architecture, __libc_single_thread may be accessed through copy relocations and thus it requires to update both copies. To obtain the correct address of the __libc_single_thread, __libc_dlsym is extended to support RTLD_DEFAULT. It searches through all scope instead of default local ones. Checked on x86_64-linux-gnu and i686-linux-gnu. --- elf/dl-libc.c | 20 ++++++++++++++++++-- elf/libc_early_init.c | 9 +++++++++ include/sys/single_threaded.h | 11 +++++++++++ misc/single_threaded.c | 2 ++ nptl/pthread_create.c | 6 +++++- 5 files changed, 45 insertions(+), 3 deletions(-) diff --git a/elf/dl-libc.c b/elf/dl-libc.c index 266e068da6..e64f4b9910 100644 --- a/elf/dl-libc.c +++ b/elf/dl-libc.c @@ -16,6 +16,7 @@ License along with the GNU C Library; if not, see . */ +#include #include #include #include @@ -72,6 +73,7 @@ struct do_dlsym_args /* Arguments to do_dlsym. */ struct link_map *map; const char *name; + const void *caller_dlsym; /* Return values of do_dlsym. */ lookup_t loadbase; @@ -102,8 +104,21 @@ do_dlsym (void *ptr) { struct do_dlsym_args *args = (struct do_dlsym_args *) ptr; args->ref = NULL; - args->loadbase = GLRO(dl_lookup_symbol_x) (args->name, args->map, &args->ref, - args->map->l_local_scope, NULL, 0, + struct link_map *match = args->map; + struct r_scope_elem **scope; + if (args->map == RTLD_DEFAULT) + { + ElfW(Addr) caller = (ElfW(Addr)) args->caller_dlsym; + match = _dl_find_dso_for_object (caller); + /* It is only used internally, so caller should be always recognized. */ + assert (match != NULL); + scope = match->l_scope; + } + else + scope = args->map->l_local_scope; + + args->loadbase = GLRO(dl_lookup_symbol_x) (args->name, match, &args->ref, + scope, NULL, 0, DL_LOOKUP_RETURN_NEWEST, NULL); } @@ -182,6 +197,7 @@ __libc_dlsym (void *map, const char *name) struct do_dlsym_args args; args.map = map; args.name = name; + args.caller_dlsym = RETURN_ADDRESS (0); #ifdef SHARED if (GLRO (dl_dlfcn_hook) != NULL) diff --git a/elf/libc_early_init.c b/elf/libc_early_init.c index 3c4a19cf6b..7cc2997122 100644 --- a/elf/libc_early_init.c +++ b/elf/libc_early_init.c @@ -16,7 +16,9 @@ License along with the GNU C Library; if not, see . */ +#include #include +#include #include #include #include @@ -38,6 +40,13 @@ __libc_early_init (_Bool initial) __libc_single_threaded = initial; #ifdef SHARED + /* __libc_single_threaded can be accessed through copy relocations, so it + requires to update the external copy. */ + __libc_external_single_threaded = __libc_dlsym (RTLD_DEFAULT, + "__libc_single_threaded"); + assert (__libc_external_single_threaded != NULL); + *__libc_external_single_threaded = initial; + __libc_initial = initial; #endif diff --git a/include/sys/single_threaded.h b/include/sys/single_threaded.h index 18f6972482..258b01e0b2 100644 --- a/include/sys/single_threaded.h +++ b/include/sys/single_threaded.h @@ -1 +1,12 @@ #include + +#ifndef _ISOMAC + +libc_hidden_proto (__libc_single_threaded); + +# ifdef SHARED +extern __typeof (__libc_single_threaded) *__libc_external_single_threaded + attribute_hidden; +# endif + +#endif diff --git a/misc/single_threaded.c b/misc/single_threaded.c index 96ada9137b..201d86a273 100644 --- a/misc/single_threaded.c +++ b/misc/single_threaded.c @@ -22,6 +22,8 @@ __libc_early_init (as false for inner libcs). */ #ifdef SHARED char __libc_single_threaded; +__typeof (__libc_single_threaded) *__libc_external_single_threaded; #else char __libc_single_threaded = 1; #endif +libc_hidden_data_def (__libc_single_threaded) diff --git a/nptl/pthread_create.c b/nptl/pthread_create.c index e7a099acb7..5633d01c62 100644 --- a/nptl/pthread_create.c +++ b/nptl/pthread_create.c @@ -627,7 +627,11 @@ __pthread_create_2_1 (pthread_t *newthread, const pthread_attr_t *attr, if (__libc_single_threaded) { late_init (); - __libc_single_threaded = 0; + __libc_single_threaded = +#ifdef SHARED + *__libc_external_single_threaded = +#endif + 0; } const struct pthread_attr *iattr = (struct pthread_attr *) attr; From patchwork Fri Jun 10 16:35:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 55024 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6377F392AC26 for ; Fri, 10 Jun 2022 16:38:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6377F392AC26 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1654879109; bh=6HoW3AnniE80AXIZ/XsiUUjSoZax5afOL+v6PBJkLSw=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=GCA5kjDTOc/QsOk+S5pslgArLhHFMEeKrsSX8zoe9mxaSU7xPXi2YfIeP2HxCwfBN SGaqXVgEnPV2eFixWCFpjZJktGXUvim/tEg8DfAoGykyxtYKsqQo0y5hTjRoLzqrOC IuRDx/QCSq9ekbvzFylPS7PUpxCNE4bytZi3MqvI= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oi1-x22b.google.com (mail-oi1-x22b.google.com [IPv6:2607:f8b0:4864:20::22b]) by sourceware.org (Postfix) with ESMTPS id 30A5638376E9 for ; Fri, 10 Jun 2022 16:36:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 30A5638376E9 Received: by mail-oi1-x22b.google.com with SMTP id p129so31552947oig.3 for ; Fri, 10 Jun 2022 09:36:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6HoW3AnniE80AXIZ/XsiUUjSoZax5afOL+v6PBJkLSw=; b=yIQaMeyUd2M7rr9Te4aKMCSxObfzNPJVnXIBb7jYoP1A6+ucFcFlAK2KWIRKB89bPi bAEuPwvFQvLP0ctf08yCRg58c281ii9vsG71r+UgQF6IGB0Y/itbRzq/+3sQBUcpKCrF HeUksp9+2TTpGdy0WMSnfNsvkVGhZbrkqU5MpwMELeegnuBecHqQRqyoBlq7wnSC/WLs F7QDdb89CIe2dxJkX9YrLdADJnvDYXU0sS6PMYKMADTpx7Hi1khhyN4OiiOyBiWvhbfD 5bYfXyeCovtEd0M9+nuPojiadcB55N8G5wOORMqt4obTks2pH+ge3BjdqYc8DRm9iAZH F4NA== X-Gm-Message-State: AOAM532FryfR3cbE+6lzHMIX2s1RDrIE/xUECPy+aPd0qzHCLR/orKrM EFavR6krqcLFrqZf4uyQWQR2wST/P7nG3g== X-Google-Smtp-Source: ABdhPJygDFBJwrAdqxwn6gq4QmT9tHYt3g0aNiNck0LKiEwjsje2qhy++x5cSSyeWz6fiae3FXBUxw== X-Received: by 2002:a05:6808:f09:b0:328:b61f:1b52 with SMTP id m9-20020a0568080f0900b00328b61f1b52mr364957oiw.150.1654878960159; Fri, 10 Jun 2022 09:36:00 -0700 (PDT) Received: from birita.. ([2804:431:c7cb:a613:818b:b86c:a3f8:d455]) by smtp.gmail.com with ESMTPSA id p203-20020aca42d4000000b0032efe5871b0sm1262424oia.45.2022.06.10.09.35.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 09:35:59 -0700 (PDT) To: libc-alpha@sourceware.org, Wilco Dijkstra Subject: [PATCH v2 2/4] Replace __libc_multiple_threads with __libc_single_threaded Date: Fri, 10 Jun 2022 13:35:50 -0300 Message-Id: <20220610163552.3587064-3-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220610163552.3587064-1-adhemerval.zanella@linaro.org> References: <20220610163552.3587064-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" And also fixes the SINGLE_THREAD_P macro for SINGLE_THREAD_BY_GLOBAL, since header inclusion single-thread.h is in the wrong order, the define needs to come before including sysdeps/unix/sysdep.h. The macro is now moved to a per-arch single-threade.h header. --- nptl/Makefile | 1 - nptl/allocatestack.c | 6 ---- nptl/libc_multiple_threads.c | 28 ------------------- nptl/pthread_cancel.c | 2 +- .../unix/sysv/linux/aarch64/single-thread.h | 2 ++ sysdeps/unix/sysv/linux/aarch64/sysdep.h | 2 -- sysdeps/unix/sysv/linux/alpha/sysdep.h | 2 -- sysdeps/unix/sysv/linux/arc/single-thread.h | 2 ++ sysdeps/unix/sysv/linux/arc/sysdep.h | 2 -- sysdeps/unix/sysv/linux/arm/single-thread.h | 2 ++ sysdeps/unix/sysv/linux/arm/sysdep.h | 2 -- sysdeps/unix/sysv/linux/hppa/single-thread.h | 2 ++ sysdeps/unix/sysv/linux/hppa/sysdep.h | 2 -- .../sysv/linux/microblaze/single-thread.h | 2 ++ sysdeps/unix/sysv/linux/microblaze/sysdep.h | 2 -- sysdeps/unix/sysv/linux/s390/single-thread.h | 2 ++ sysdeps/unix/sysv/linux/s390/sysdep.h | 3 -- sysdeps/unix/sysv/linux/single-thread.h | 11 ++++---- .../unix/sysv/linux/x86_64/single-thread.h | 2 ++ sysdeps/unix/sysv/linux/x86_64/sysdep.h | 2 -- 20 files changed, 20 insertions(+), 59 deletions(-) delete mode 100644 nptl/libc_multiple_threads.c create mode 100644 sysdeps/unix/sysv/linux/aarch64/single-thread.h create mode 100644 sysdeps/unix/sysv/linux/arc/single-thread.h create mode 100644 sysdeps/unix/sysv/linux/arm/single-thread.h create mode 100644 sysdeps/unix/sysv/linux/hppa/single-thread.h create mode 100644 sysdeps/unix/sysv/linux/microblaze/single-thread.h create mode 100644 sysdeps/unix/sysv/linux/s390/single-thread.h create mode 100644 sysdeps/unix/sysv/linux/x86_64/single-thread.h diff --git a/nptl/Makefile b/nptl/Makefile index b585663974..3d2ce8af8a 100644 --- a/nptl/Makefile +++ b/nptl/Makefile @@ -50,7 +50,6 @@ routines = \ events \ futex-internal \ libc-cleanup \ - libc_multiple_threads \ lowlevellock \ nptl-stack \ nptl_deallocate_tsd \ diff --git a/nptl/allocatestack.c b/nptl/allocatestack.c index 01a282f3f6..98f5f6dd85 100644 --- a/nptl/allocatestack.c +++ b/nptl/allocatestack.c @@ -292,9 +292,6 @@ allocate_stack (const struct pthread_attr *attr, struct pthread **pdp, /* This is at least the second thread. */ pd->header.multiple_threads = 1; -#ifndef TLS_MULTIPLE_THREADS_IN_TCB - __libc_multiple_threads = 1; -#endif #ifdef NEED_DL_SYSINFO SETUP_THREAD_SYSINFO (pd); @@ -413,9 +410,6 @@ allocate_stack (const struct pthread_attr *attr, struct pthread **pdp, /* This is at least the second thread. */ pd->header.multiple_threads = 1; -#ifndef TLS_MULTIPLE_THREADS_IN_TCB - __libc_multiple_threads = 1; -#endif #ifdef NEED_DL_SYSINFO SETUP_THREAD_SYSINFO (pd); diff --git a/nptl/libc_multiple_threads.c b/nptl/libc_multiple_threads.c deleted file mode 100644 index 0c2dc33d0d..0000000000 --- a/nptl/libc_multiple_threads.c +++ /dev/null @@ -1,28 +0,0 @@ -/* Copyright (C) 2002-2022 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#include - -#if IS_IN (libc) -# ifndef TLS_MULTIPLE_THREADS_IN_TCB -/* Variable set to a nonzero value either if more than one thread runs or ran, - or if a single-threaded process is trying to cancel itself. See - nptl/descr.h for more context on the single-threaded process case. */ -int __libc_multiple_threads; -libc_hidden_data_def (__libc_multiple_threads) -# endif -#endif diff --git a/nptl/pthread_cancel.c b/nptl/pthread_cancel.c index e67b2df5cc..e1735279f2 100644 --- a/nptl/pthread_cancel.c +++ b/nptl/pthread_cancel.c @@ -161,7 +161,7 @@ __pthread_cancel (pthread_t th) points get executed. */ THREAD_SETMEM (THREAD_SELF, header.multiple_threads, 1); #ifndef TLS_MULTIPLE_THREADS_IN_TCB - __libc_multiple_threads = 1; + __libc_single_threaded = 0; #endif } while (!atomic_compare_exchange_weak_acquire (&pd->cancelhandling, &oldval, diff --git a/sysdeps/unix/sysv/linux/aarch64/single-thread.h b/sysdeps/unix/sysv/linux/aarch64/single-thread.h new file mode 100644 index 0000000000..a5d3a2aaf4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/aarch64/single-thread.h @@ -0,0 +1,2 @@ +#define SINGLE_THREAD_BY_GLOBAL +#include_next diff --git a/sysdeps/unix/sysv/linux/aarch64/sysdep.h b/sysdeps/unix/sysv/linux/aarch64/sysdep.h index 3b230dccf1..f1853e012f 100644 --- a/sysdeps/unix/sysv/linux/aarch64/sysdep.h +++ b/sysdeps/unix/sysv/linux/aarch64/sysdep.h @@ -164,8 +164,6 @@ # define HAVE_CLOCK_GETTIME64_VSYSCALL "__kernel_clock_gettime" # define HAVE_GETTIMEOFDAY_VSYSCALL "__kernel_gettimeofday" -# define SINGLE_THREAD_BY_GLOBAL 1 - # undef INTERNAL_SYSCALL_RAW # define INTERNAL_SYSCALL_RAW(name, nr, args...) \ ({ long _sys_result; \ diff --git a/sysdeps/unix/sysv/linux/alpha/sysdep.h b/sysdeps/unix/sysv/linux/alpha/sysdep.h index 3051a744b4..77ec2b5400 100644 --- a/sysdeps/unix/sysv/linux/alpha/sysdep.h +++ b/sysdeps/unix/sysv/linux/alpha/sysdep.h @@ -32,8 +32,6 @@ #undef SYS_ify #define SYS_ify(syscall_name) __NR_##syscall_name -#define SINGLE_THREAD_BY_GLOBAL 1 - #ifdef __ASSEMBLER__ #include #include diff --git a/sysdeps/unix/sysv/linux/arc/single-thread.h b/sysdeps/unix/sysv/linux/arc/single-thread.h new file mode 100644 index 0000000000..a5d3a2aaf4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/arc/single-thread.h @@ -0,0 +1,2 @@ +#define SINGLE_THREAD_BY_GLOBAL +#include_next diff --git a/sysdeps/unix/sysv/linux/arc/sysdep.h b/sysdeps/unix/sysv/linux/arc/sysdep.h index 29b0e0161c..d0c1a78381 100644 --- a/sysdeps/unix/sysv/linux/arc/sysdep.h +++ b/sysdeps/unix/sysv/linux/arc/sysdep.h @@ -132,8 +132,6 @@ L (call_syscall_err): ASM_LINE_SEP \ #else /* !__ASSEMBLER__ */ -# define SINGLE_THREAD_BY_GLOBAL 1 - # if IS_IN (libc) extern long int __syscall_error (long int); hidden_proto (__syscall_error) diff --git a/sysdeps/unix/sysv/linux/arm/single-thread.h b/sysdeps/unix/sysv/linux/arm/single-thread.h new file mode 100644 index 0000000000..a5d3a2aaf4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/arm/single-thread.h @@ -0,0 +1,2 @@ +#define SINGLE_THREAD_BY_GLOBAL +#include_next diff --git a/sysdeps/unix/sysv/linux/arm/sysdep.h b/sysdeps/unix/sysv/linux/arm/sysdep.h index 7bdd218063..1f270b961e 100644 --- a/sysdeps/unix/sysv/linux/arm/sysdep.h +++ b/sysdeps/unix/sysv/linux/arm/sysdep.h @@ -408,8 +408,6 @@ __local_syscall_error: \ #define INTERNAL_SYSCALL_NCS(number, nr, args...) \ INTERNAL_SYSCALL_RAW (number, nr, args) -#define SINGLE_THREAD_BY_GLOBAL 1 - #endif /* __ASSEMBLER__ */ #endif /* linux/arm/sysdep.h */ diff --git a/sysdeps/unix/sysv/linux/hppa/single-thread.h b/sysdeps/unix/sysv/linux/hppa/single-thread.h new file mode 100644 index 0000000000..a5d3a2aaf4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/hppa/single-thread.h @@ -0,0 +1,2 @@ +#define SINGLE_THREAD_BY_GLOBAL +#include_next diff --git a/sysdeps/unix/sysv/linux/hppa/sysdep.h b/sysdeps/unix/sysv/linux/hppa/sysdep.h index 42f7705852..2f339a4bd6 100644 --- a/sysdeps/unix/sysv/linux/hppa/sysdep.h +++ b/sysdeps/unix/sysv/linux/hppa/sysdep.h @@ -474,6 +474,4 @@ L(pre_end): ASM_LINE_SEP \ #define PTR_MANGLE(var) (void) (var) #define PTR_DEMANGLE(var) (void) (var) -#define SINGLE_THREAD_BY_GLOBAL 1 - #endif /* _LINUX_HPPA_SYSDEP_H */ diff --git a/sysdeps/unix/sysv/linux/microblaze/single-thread.h b/sysdeps/unix/sysv/linux/microblaze/single-thread.h new file mode 100644 index 0000000000..a5d3a2aaf4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/microblaze/single-thread.h @@ -0,0 +1,2 @@ +#define SINGLE_THREAD_BY_GLOBAL +#include_next diff --git a/sysdeps/unix/sysv/linux/microblaze/sysdep.h b/sysdeps/unix/sysv/linux/microblaze/sysdep.h index dfd6312506..fda78f6467 100644 --- a/sysdeps/unix/sysv/linux/microblaze/sysdep.h +++ b/sysdeps/unix/sysv/linux/microblaze/sysdep.h @@ -308,8 +308,6 @@ SYSCALL_ERROR_LABEL_DCL: \ # define PTR_MANGLE(var) (void) (var) # define PTR_DEMANGLE(var) (void) (var) -# define SINGLE_THREAD_BY_GLOBAL 1 - #undef HAVE_INTERNAL_BRK_ADDR_SYMBOL #define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1 diff --git a/sysdeps/unix/sysv/linux/s390/single-thread.h b/sysdeps/unix/sysv/linux/s390/single-thread.h new file mode 100644 index 0000000000..a5d3a2aaf4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/s390/single-thread.h @@ -0,0 +1,2 @@ +#define SINGLE_THREAD_BY_GLOBAL +#include_next diff --git a/sysdeps/unix/sysv/linux/s390/sysdep.h b/sysdeps/unix/sysv/linux/s390/sysdep.h index 78c7e8c7e2..2d0a26779c 100644 --- a/sysdeps/unix/sysv/linux/s390/sysdep.h +++ b/sysdeps/unix/sysv/linux/s390/sysdep.h @@ -93,9 +93,6 @@ #define ASMFMT_5 , "0" (gpr2), "d" (gpr3), "d" (gpr4), "d" (gpr5), "d" (gpr6) #define ASMFMT_6 , "0" (gpr2), "d" (gpr3), "d" (gpr4), "d" (gpr5), "d" (gpr6), "d" (gpr7) -#define SINGLE_THREAD_BY_GLOBAL 1 - - #define VDSO_NAME "LINUX_2.6.29" #define VDSO_HASH 123718585 diff --git a/sysdeps/unix/sysv/linux/single-thread.h b/sysdeps/unix/sysv/linux/single-thread.h index 4529a906d2..208edccce6 100644 --- a/sysdeps/unix/sysv/linux/single-thread.h +++ b/sysdeps/unix/sysv/linux/single-thread.h @@ -19,6 +19,10 @@ #ifndef _SINGLE_THREAD_H #define _SINGLE_THREAD_H +#ifndef __ASSEMBLER__ +# include +#endif + /* The default way to check if the process is single thread is by using the pthread_t 'multiple_threads' field. However, for some architectures it is faster to either use an extra field on TCB or global variables (the TCB @@ -27,16 +31,11 @@ The ABI might define SINGLE_THREAD_BY_GLOBAL to enable the single thread check to use global variables instead of the pthread_t field. */ -#ifndef __ASSEMBLER__ -extern int __libc_multiple_threads; -libc_hidden_proto (__libc_multiple_threads) -#endif - #if !defined SINGLE_THREAD_BY_GLOBAL || IS_IN (rtld) # define SINGLE_THREAD_P \ (THREAD_GETMEM (THREAD_SELF, header.multiple_threads) == 0) #else -# define SINGLE_THREAD_P (__libc_multiple_threads == 0) +# define SINGLE_THREAD_P (__libc_single_threaded != 0) #endif #define RTLD_SINGLE_THREAD_P SINGLE_THREAD_P diff --git a/sysdeps/unix/sysv/linux/x86_64/single-thread.h b/sysdeps/unix/sysv/linux/x86_64/single-thread.h new file mode 100644 index 0000000000..a5d3a2aaf4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/x86_64/single-thread.h @@ -0,0 +1,2 @@ +#define SINGLE_THREAD_BY_GLOBAL +#include_next diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep.h b/sysdeps/unix/sysv/linux/x86_64/sysdep.h index e1ce3b62eb..740abefcfd 100644 --- a/sysdeps/unix/sysv/linux/x86_64/sysdep.h +++ b/sysdeps/unix/sysv/linux/x86_64/sysdep.h @@ -379,8 +379,6 @@ # define HAVE_CLONE3_WRAPPER 1 -# define SINGLE_THREAD_BY_GLOBAL 1 - #endif /* __ASSEMBLER__ */ From patchwork Fri Jun 10 16:35:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 55026 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2EFFF3835684 for ; Fri, 10 Jun 2022 16:40:00 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2EFFF3835684 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1654879200; bh=eg0ZTcCDRZalX0q11b4nQ4uCtHRVE15jNlFjf0cLsVU=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=uIEUF81cZng9903SPI0eYR7CirRZVyNTzmcUWPxUFwinTe11JuSG8xmuA2EogzHGK 8iO+Phqc9BYRLXtlpgSIxRGh/oq4Ber4MgrmyJauUxJ6FUsf46lr2wF0lE7mfB4ct6 2/DgRJvcdxFThJQzuvM4I1J4U7P1DyxBQrEJPyuM= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oi1-x22b.google.com (mail-oi1-x22b.google.com [IPv6:2607:f8b0:4864:20::22b]) by sourceware.org (Postfix) with ESMTPS id 35A37383E693 for ; Fri, 10 Jun 2022 16:36:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 35A37383E693 Received: by mail-oi1-x22b.google.com with SMTP id q11so20874206oih.10 for ; Fri, 10 Jun 2022 09:36:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eg0ZTcCDRZalX0q11b4nQ4uCtHRVE15jNlFjf0cLsVU=; b=rTy+7wdmYjyD5cxjeJQTZ3KVYkgPF9dUpRkhzU1Sai9h0JRCwDTsXAH3yemeQmz3vm ja7QsWn583xqoJGDAagVmaaRpoyyaWmfsjttJLT3pUqjXcyyin0umaHifsz0UUndE8pY CLF6Q/XXtPhKTYaHorCXuw0YcHVlviDGdVBv2lx1EGntbxvY7oKR2WiRY5gfglyaoA8N sNfuErxpalJZv7avhMrEDa3J4IP6HNrNCV/rtkSvHrSUSkV1D9M0+B+xZhRFWXC1aL33 GxhGQzK8XDGMcEUBbS7I76hIb+mxzkapb61OtTmA22WP1Tp+3dde26nTpul52YqwbNML NhBg== X-Gm-Message-State: AOAM530+TL2RS068x13cRWPAdqe0lH+JFaQPP7BSNGk8B5wfTfn1aWx9 +owLIN7756a27z5kp0LMxpmNn4EwfSlvbA== X-Google-Smtp-Source: ABdhPJzIDpCk5FCIL3CiZZz0RBWmphdAdImZq/0dmkHFIzCqjGxWr0GS4LxntBfjW698VW0XlGaTWA== X-Received: by 2002:a05:6808:1153:b0:32e:b45d:bd74 with SMTP id u19-20020a056808115300b0032eb45dbd74mr345105oiu.259.1654878961963; Fri, 10 Jun 2022 09:36:01 -0700 (PDT) Received: from birita.. ([2804:431:c7cb:a613:818b:b86c:a3f8:d455]) by smtp.gmail.com with ESMTPSA id p203-20020aca42d4000000b0032efe5871b0sm1262424oia.45.2022.06.10.09.36.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 09:36:01 -0700 (PDT) To: libc-alpha@sourceware.org, Wilco Dijkstra Subject: [PATCH v2 3/4] Remove usage of TLS_MULTIPLE_THREADS_IN_TCB Date: Fri, 10 Jun 2022 13:35:51 -0300 Message-Id: <20220610163552.3587064-4-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220610163552.3587064-1-adhemerval.zanella@linaro.org> References: <20220610163552.3587064-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Instead use __libc_single_threaded on all architectures. The TCB field is renamed to avoid change the struct layout. The x86 atomic need some adjustments since it has single-thread optimizationi builtin within the inline assemblye. It now uses SINGLE_THREAD_P and atomic optimizations are removed (since they are not used). Checked on x86_64-linux-gnu and i686-linux-gnu. --- misc/tst-atomic.c | 1 + nptl/allocatestack.c | 6 - nptl/descr.h | 17 +- nptl/pthread_cancel.c | 7 +- nptl/pthread_create.c | 5 - sysdeps/i386/htl/tcb-offsets.sym | 1 - sysdeps/i386/nptl/tcb-offsets.sym | 1 - sysdeps/i386/nptl/tls.h | 4 +- sysdeps/ia64/nptl/tcb-offsets.sym | 1 - sysdeps/ia64/nptl/tls.h | 2 - sysdeps/mach/hurd/i386/tls.h | 4 +- sysdeps/nios2/nptl/tcb-offsets.sym | 1 - sysdeps/or1k/nptl/tls.h | 2 - sysdeps/powerpc/nptl/tcb-offsets.sym | 3 - sysdeps/powerpc/nptl/tls.h | 3 - sysdeps/s390/nptl/tcb-offsets.sym | 1 - sysdeps/s390/nptl/tls.h | 6 +- sysdeps/sh/nptl/tcb-offsets.sym | 1 - sysdeps/sh/nptl/tls.h | 2 - sysdeps/sparc/nptl/tcb-offsets.sym | 1 - sysdeps/sparc/nptl/tls.h | 2 +- sysdeps/unix/sysv/linux/single-thread.h | 15 +- sysdeps/x86/atomic-machine.h | 484 +++++++----------------- sysdeps/x86_64/nptl/tcb-offsets.sym | 1 - 24 files changed, 145 insertions(+), 426 deletions(-) diff --git a/misc/tst-atomic.c b/misc/tst-atomic.c index 6d681a7bfd..ddbc618e25 100644 --- a/misc/tst-atomic.c +++ b/misc/tst-atomic.c @@ -18,6 +18,7 @@ #include #include +#include #ifndef atomic_t # define atomic_t int diff --git a/nptl/allocatestack.c b/nptl/allocatestack.c index 98f5f6dd85..3e0d01cb52 100644 --- a/nptl/allocatestack.c +++ b/nptl/allocatestack.c @@ -290,9 +290,6 @@ allocate_stack (const struct pthread_attr *attr, struct pthread **pdp, stack cache nor will the memory (except the TLS memory) be freed. */ pd->user_stack = true; - /* This is at least the second thread. */ - pd->header.multiple_threads = 1; - #ifdef NEED_DL_SYSINFO SETUP_THREAD_SYSINFO (pd); #endif @@ -408,9 +405,6 @@ allocate_stack (const struct pthread_attr *attr, struct pthread **pdp, descriptor. */ pd->specific[0] = pd->specific_1stblock; - /* This is at least the second thread. */ - pd->header.multiple_threads = 1; - #ifdef NEED_DL_SYSINFO SETUP_THREAD_SYSINFO (pd); #endif diff --git a/nptl/descr.h b/nptl/descr.h index bb46b5958e..77b25d8267 100644 --- a/nptl/descr.h +++ b/nptl/descr.h @@ -137,22 +137,7 @@ struct pthread #else struct { - /* multiple_threads is enabled either when the process has spawned at - least one thread or when a single-threaded process cancels itself. - This enables additional code to introduce locking before doing some - compare_and_exchange operations and also enable cancellation points. - The concepts of multiple threads and cancellation points ideally - should be separate, since it is not necessary for multiple threads to - have been created for cancellation points to be enabled, as is the - case is when single-threaded process cancels itself. - - Since enabling multiple_threads enables additional code in - cancellation points and compare_and_exchange operations, there is a - potential for an unneeded performance hit when it is enabled in a - single-threaded, self-canceling process. This is OK though, since a - single-threaded process will enable async cancellation only when it - looks to cancel itself and is hence going to end anyway. */ - int multiple_threads; + int unused_multiple_threads; int gscope_flag; } header; #endif diff --git a/nptl/pthread_cancel.c b/nptl/pthread_cancel.c index e1735279f2..6d26a15d0e 100644 --- a/nptl/pthread_cancel.c +++ b/nptl/pthread_cancel.c @@ -157,12 +157,9 @@ __pthread_cancel (pthread_t th) /* A single-threaded process should be able to kill itself, since there is nothing in the POSIX specification that says that it - cannot. So we set multiple_threads to true so that cancellation - points get executed. */ - THREAD_SETMEM (THREAD_SELF, header.multiple_threads, 1); -#ifndef TLS_MULTIPLE_THREADS_IN_TCB + cannot. So we set __libc_single_threaded to true so that + cancellation points get executed. */ __libc_single_threaded = 0; -#endif } while (!atomic_compare_exchange_weak_acquire (&pd->cancelhandling, &oldval, newval)); diff --git a/nptl/pthread_create.c b/nptl/pthread_create.c index 5633d01c62..d43865352f 100644 --- a/nptl/pthread_create.c +++ b/nptl/pthread_create.c @@ -882,11 +882,6 @@ __pthread_create_2_1 (pthread_t *newthread, const pthread_attr_t *attr, other reason that create_thread chose. Now let it run free. */ lll_unlock (pd->lock, LLL_PRIVATE); - - /* We now have for sure more than one thread. The main thread might - not yet have the flag set. No need to set the global variable - again if this is what we use. */ - THREAD_SETMEM (THREAD_SELF, header.multiple_threads, 1); } out: diff --git a/sysdeps/i386/htl/tcb-offsets.sym b/sysdeps/i386/htl/tcb-offsets.sym index 7b7c719369..f3f7df6c06 100644 --- a/sysdeps/i386/htl/tcb-offsets.sym +++ b/sysdeps/i386/htl/tcb-offsets.sym @@ -2,7 +2,6 @@ #include #include -MULTIPLE_THREADS_OFFSET offsetof (tcbhead_t, multiple_threads) SYSINFO_OFFSET offsetof (tcbhead_t, sysinfo) POINTER_GUARD offsetof (tcbhead_t, pointer_guard) SIGSTATE_OFFSET offsetof (tcbhead_t, _hurd_sigstate) diff --git a/sysdeps/i386/nptl/tcb-offsets.sym b/sysdeps/i386/nptl/tcb-offsets.sym index 2ec9e787c1..1efd1469d8 100644 --- a/sysdeps/i386/nptl/tcb-offsets.sym +++ b/sysdeps/i386/nptl/tcb-offsets.sym @@ -6,7 +6,6 @@ RESULT offsetof (struct pthread, result) TID offsetof (struct pthread, tid) CANCELHANDLING offsetof (struct pthread, cancelhandling) CLEANUP_JMP_BUF offsetof (struct pthread, cleanup_jmp_buf) -MULTIPLE_THREADS_OFFSET offsetof (tcbhead_t, multiple_threads) SYSINFO_OFFSET offsetof (tcbhead_t, sysinfo) CLEANUP offsetof (struct pthread, cleanup) CLEANUP_PREV offsetof (struct _pthread_cleanup_buffer, __prev) diff --git a/sysdeps/i386/nptl/tls.h b/sysdeps/i386/nptl/tls.h index 91090bf287..48940a9f44 100644 --- a/sysdeps/i386/nptl/tls.h +++ b/sysdeps/i386/nptl/tls.h @@ -36,7 +36,7 @@ typedef struct thread descriptor used by libpthread. */ dtv_t *dtv; void *self; /* Pointer to the thread descriptor. */ - int multiple_threads; + int unused_multiple_threads; uintptr_t sysinfo; uintptr_t stack_guard; uintptr_t pointer_guard; @@ -57,8 +57,6 @@ typedef struct _Static_assert (offsetof (tcbhead_t, __private_ss) == 0x30, "offset of __private_ss != 0x30"); -# define TLS_MULTIPLE_THREADS_IN_TCB 1 - #else /* __ASSEMBLER__ */ # include #endif diff --git a/sysdeps/ia64/nptl/tcb-offsets.sym b/sysdeps/ia64/nptl/tcb-offsets.sym index b01f712be2..ab2cb180f9 100644 --- a/sysdeps/ia64/nptl/tcb-offsets.sym +++ b/sysdeps/ia64/nptl/tcb-offsets.sym @@ -2,5 +2,4 @@ #include TID offsetof (struct pthread, tid) - TLS_PRE_TCB_SIZE -MULTIPLE_THREADS_OFFSET offsetof (struct pthread, header.multiple_threads) - TLS_PRE_TCB_SIZE SYSINFO_OFFSET offsetof (tcbhead_t, __private) diff --git a/sysdeps/ia64/nptl/tls.h b/sysdeps/ia64/nptl/tls.h index 8ccedb73e6..008e080fc4 100644 --- a/sysdeps/ia64/nptl/tls.h +++ b/sysdeps/ia64/nptl/tls.h @@ -36,8 +36,6 @@ typedef struct register struct pthread *__thread_self __asm__("r13"); -# define TLS_MULTIPLE_THREADS_IN_TCB 1 - #else /* __ASSEMBLER__ */ # include #endif diff --git a/sysdeps/mach/hurd/i386/tls.h b/sysdeps/mach/hurd/i386/tls.h index 264ed9a9c5..d33e91c922 100644 --- a/sysdeps/mach/hurd/i386/tls.h +++ b/sysdeps/mach/hurd/i386/tls.h @@ -33,7 +33,7 @@ typedef struct void *tcb; /* Points to this structure. */ dtv_t *dtv; /* Vector of pointers to TLS data. */ thread_t self; /* This thread's control port. */ - int multiple_threads; + int unused_multiple_threads; uintptr_t sysinfo; uintptr_t stack_guard; uintptr_t pointer_guard; @@ -117,8 +117,6 @@ _hurd_tls_init (tcbhead_t *tcb) /* This field is used by TLS accesses to get our "thread pointer" from the TLS point of view. */ tcb->tcb = tcb; - /* We always at least start the sigthread anyway. */ - tcb->multiple_threads = 1; /* Get the first available selector. */ int sel = -1; diff --git a/sysdeps/nios2/nptl/tcb-offsets.sym b/sysdeps/nios2/nptl/tcb-offsets.sym index 3cd8d984ac..93a695ac7f 100644 --- a/sysdeps/nios2/nptl/tcb-offsets.sym +++ b/sysdeps/nios2/nptl/tcb-offsets.sym @@ -8,6 +8,5 @@ # define __thread_self ((void *) 0) # define thread_offsetof(mem) ((ptrdiff_t) THREAD_SELF + offsetof (struct pthread, mem)) -MULTIPLE_THREADS_OFFSET thread_offsetof (header.multiple_threads) TID_OFFSET thread_offsetof (tid) POINTER_GUARD (offsetof (tcbhead_t, pointer_guard) - TLS_TCB_OFFSET - sizeof (tcbhead_t)) diff --git a/sysdeps/or1k/nptl/tls.h b/sysdeps/or1k/nptl/tls.h index c6ffe62c3f..3bb07beef8 100644 --- a/sysdeps/or1k/nptl/tls.h +++ b/sysdeps/or1k/nptl/tls.h @@ -35,8 +35,6 @@ typedef struct register tcbhead_t *__thread_self __asm__("r10"); -# define TLS_MULTIPLE_THREADS_IN_TCB 1 - /* Get system call information. */ # include diff --git a/sysdeps/powerpc/nptl/tcb-offsets.sym b/sysdeps/powerpc/nptl/tcb-offsets.sym index 4c01615ad0..a0ee95f94d 100644 --- a/sysdeps/powerpc/nptl/tcb-offsets.sym +++ b/sysdeps/powerpc/nptl/tcb-offsets.sym @@ -10,9 +10,6 @@ # define thread_offsetof(mem) ((ptrdiff_t) THREAD_SELF + offsetof (struct pthread, mem)) -#if TLS_MULTIPLE_THREADS_IN_TCB -MULTIPLE_THREADS_OFFSET thread_offsetof (header.multiple_threads) -#endif TID thread_offsetof (tid) POINTER_GUARD (offsetof (tcbhead_t, pointer_guard) - TLS_TCB_OFFSET - sizeof (tcbhead_t)) TAR_SAVE (offsetof (tcbhead_t, tar_save) - TLS_TCB_OFFSET - sizeof (tcbhead_t)) diff --git a/sysdeps/powerpc/nptl/tls.h b/sysdeps/powerpc/nptl/tls.h index 22b0075235..fd5ee51981 100644 --- a/sysdeps/powerpc/nptl/tls.h +++ b/sysdeps/powerpc/nptl/tls.h @@ -52,9 +52,6 @@ # define TLS_DTV_AT_TP 1 # define TLS_TCB_AT_TP 0 -/* We use the multiple_threads field in the pthread struct */ -#define TLS_MULTIPLE_THREADS_IN_TCB 1 - /* Get the thread descriptor definition. */ # include diff --git a/sysdeps/s390/nptl/tcb-offsets.sym b/sysdeps/s390/nptl/tcb-offsets.sym index 9c1c01f353..bc7b267463 100644 --- a/sysdeps/s390/nptl/tcb-offsets.sym +++ b/sysdeps/s390/nptl/tcb-offsets.sym @@ -1,6 +1,5 @@ #include #include -MULTIPLE_THREADS_OFFSET offsetof (tcbhead_t, multiple_threads) STACK_GUARD offsetof (tcbhead_t, stack_guard) TID offsetof (struct pthread, tid) diff --git a/sysdeps/s390/nptl/tls.h b/sysdeps/s390/nptl/tls.h index ff210ffeb2..d69ed539f7 100644 --- a/sysdeps/s390/nptl/tls.h +++ b/sysdeps/s390/nptl/tls.h @@ -35,7 +35,7 @@ typedef struct thread descriptor used by libpthread. */ dtv_t *dtv; void *self; /* Pointer to the thread descriptor. */ - int multiple_threads; + int unused_multiple_threads; uintptr_t sysinfo; uintptr_t stack_guard; int gscope_flag; @@ -44,10 +44,6 @@ typedef struct void *__private_ss; } tcbhead_t; -# ifndef __s390x__ -# define TLS_MULTIPLE_THREADS_IN_TCB 1 -# endif - #else /* __ASSEMBLER__ */ # include #endif diff --git a/sysdeps/sh/nptl/tcb-offsets.sym b/sysdeps/sh/nptl/tcb-offsets.sym index 234207779d..4e452d9c6c 100644 --- a/sysdeps/sh/nptl/tcb-offsets.sym +++ b/sysdeps/sh/nptl/tcb-offsets.sym @@ -6,7 +6,6 @@ RESULT offsetof (struct pthread, result) TID offsetof (struct pthread, tid) CANCELHANDLING offsetof (struct pthread, cancelhandling) CLEANUP_JMP_BUF offsetof (struct pthread, cleanup_jmp_buf) -MULTIPLE_THREADS_OFFSET offsetof (struct pthread, header.multiple_threads) TLS_PRE_TCB_SIZE sizeof (struct pthread) MUTEX_FUTEX offsetof (pthread_mutex_t, __data.__lock) POINTER_GUARD offsetof (tcbhead_t, pointer_guard) diff --git a/sysdeps/sh/nptl/tls.h b/sysdeps/sh/nptl/tls.h index 76591ab6ef..8778cb4ac0 100644 --- a/sysdeps/sh/nptl/tls.h +++ b/sysdeps/sh/nptl/tls.h @@ -36,8 +36,6 @@ typedef struct uintptr_t pointer_guard; } tcbhead_t; -# define TLS_MULTIPLE_THREADS_IN_TCB 1 - #else /* __ASSEMBLER__ */ # include #endif /* __ASSEMBLER__ */ diff --git a/sysdeps/sparc/nptl/tcb-offsets.sym b/sysdeps/sparc/nptl/tcb-offsets.sym index f75d02065e..e4a7e4720f 100644 --- a/sysdeps/sparc/nptl/tcb-offsets.sym +++ b/sysdeps/sparc/nptl/tcb-offsets.sym @@ -1,6 +1,5 @@ #include #include -MULTIPLE_THREADS_OFFSET offsetof (tcbhead_t, multiple_threads) POINTER_GUARD offsetof (tcbhead_t, pointer_guard) TID offsetof (struct pthread, tid) diff --git a/sysdeps/sparc/nptl/tls.h b/sysdeps/sparc/nptl/tls.h index d1e2bb4ad1..b78cf0d6b4 100644 --- a/sysdeps/sparc/nptl/tls.h +++ b/sysdeps/sparc/nptl/tls.h @@ -35,7 +35,7 @@ typedef struct thread descriptor used by libpthread. */ dtv_t *dtv; void *self; - int multiple_threads; + int unused_multiple_threads; #if __WORDSIZE == 64 int gscope_flag; #endif diff --git a/sysdeps/unix/sysv/linux/single-thread.h b/sysdeps/unix/sysv/linux/single-thread.h index 208edccce6..dd80e82c82 100644 --- a/sysdeps/unix/sysv/linux/single-thread.h +++ b/sysdeps/unix/sysv/linux/single-thread.h @@ -23,20 +23,7 @@ # include #endif -/* The default way to check if the process is single thread is by using the - pthread_t 'multiple_threads' field. However, for some architectures it is - faster to either use an extra field on TCB or global variables (the TCB - field is also used on x86 for some single-thread atomic optimizations). - - The ABI might define SINGLE_THREAD_BY_GLOBAL to enable the single thread - check to use global variables instead of the pthread_t field. */ - -#if !defined SINGLE_THREAD_BY_GLOBAL || IS_IN (rtld) -# define SINGLE_THREAD_P \ - (THREAD_GETMEM (THREAD_SELF, header.multiple_threads) == 0) -#else -# define SINGLE_THREAD_P (__libc_single_threaded != 0) -#endif +#define SINGLE_THREAD_P (__libc_single_threaded != 0) #define RTLD_SINGLE_THREAD_P SINGLE_THREAD_P diff --git a/sysdeps/x86/atomic-machine.h b/sysdeps/x86/atomic-machine.h index f24f1c71ed..23e087e7e0 100644 --- a/sysdeps/x86/atomic-machine.h +++ b/sysdeps/x86/atomic-machine.h @@ -51,292 +51,145 @@ #define atomic_compare_and_exchange_bool_acq(mem, newval, oldval) \ (! __sync_bool_compare_and_swap (mem, oldval, newval)) +#define __cmpxchg_op(lock, mem, newval, oldval) \ + ({ __typeof (*mem) __ret; \ + if (sizeof (*mem) == 1) \ + asm volatile (lock "cmpxchgb %2, %1" \ + : "=a" (ret), "+m" (*mem) \ + : BR_CONSTRAINT (newval), "0" (oldval) \ + : "memory"); \ + else if (sizeof (*mem) == 2) \ + asm volatile (lock "cmpxchgw %2, %1" \ + : "=a" (ret), "+m" (*mem) \ + : BR_CONSTRAINT (newval), "0" (oldval) \ + : "memory"); \ + else if (sizeof (*mem) == 4) \ + asm volatile (lock "cmpxchgl %2, %1" \ + : "=a" (ret), "+m" (*mem) \ + : BR_CONSTRAINT (newval), "0" (oldval) \ + : "memory"); \ + else if (__HAVE_64B_ATOMICS) \ + asm volatile (lock "cmpxchgq %2, %1" \ + : "=a" (ret), "+m" (*mem) \ + : "q" ((int64_t) cast_to_integer (newval)), \ + "0" ((int64_t) cast_to_integer (oldval)) \ + : "memory"); \ + else \ + __atomic_link_error (); \ + __ret; }) -#define __arch_c_compare_and_exchange_val_8_acq(mem, newval, oldval) \ +#define __arch_c_compare_and_exchange_val_8_acq(mem, newval, oldval) \ ({ __typeof (*mem) ret; \ - __asm __volatile ("cmpl $0, %%" SEG_REG ":%P5\n\t" \ - "je 0f\n\t" \ - "lock\n" \ - "0:\tcmpxchgb %b2, %1" \ - : "=a" (ret), "=m" (*mem) \ - : BR_CONSTRAINT (newval), "m" (*mem), "0" (oldval), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ + if (SINGLE_THREAD_P) \ + __cmpxchg_op ("", (mem), (newval), (oldval)); \ + else \ + __cmpxchg_op (LOCK_PREFIX, (mem), (newval), (oldval)); \ ret; }) -#define __arch_c_compare_and_exchange_val_16_acq(mem, newval, oldval) \ +#define __arch_c_compare_and_exchange_val_16_acq(mem, newval, oldval) \ ({ __typeof (*mem) ret; \ - __asm __volatile ("cmpl $0, %%" SEG_REG ":%P5\n\t" \ - "je 0f\n\t" \ - "lock\n" \ - "0:\tcmpxchgw %w2, %1" \ - : "=a" (ret), "=m" (*mem) \ - : BR_CONSTRAINT (newval), "m" (*mem), "0" (oldval), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ + if (SINGLE_THREAD_P) \ + __cmpxchg_op ("", (mem), (newval), (oldval)); \ + else \ + __cmpxchg_op (LOCK_PREFIX, (mem), (newval), (oldval)); \ ret; }) -#define __arch_c_compare_and_exchange_val_32_acq(mem, newval, oldval) \ +#define __arch_c_compare_and_exchange_val_32_acq(mem, newval, oldval) \ ({ __typeof (*mem) ret; \ - __asm __volatile ("cmpl $0, %%" SEG_REG ":%P5\n\t" \ - "je 0f\n\t" \ - "lock\n" \ - "0:\tcmpxchgl %2, %1" \ - : "=a" (ret), "=m" (*mem) \ - : BR_CONSTRAINT (newval), "m" (*mem), "0" (oldval), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ + if (SINGLE_THREAD_P) \ + __cmpxchg_op ("", (mem), (newval), (oldval)); \ + else \ + __cmpxchg_op (LOCK_PREFIX, (mem), (newval), (oldval)); \ ret; }) -#ifdef __x86_64__ -# define __arch_c_compare_and_exchange_val_64_acq(mem, newval, oldval) \ +#define __arch_c_compare_and_exchange_val_64_acq(mem, newval, oldval) \ ({ __typeof (*mem) ret; \ - __asm __volatile ("cmpl $0, %%fs:%P5\n\t" \ - "je 0f\n\t" \ - "lock\n" \ - "0:\tcmpxchgq %q2, %1" \ - : "=a" (ret), "=m" (*mem) \ - : "q" ((int64_t) cast_to_integer (newval)), \ - "m" (*mem), \ - "0" ((int64_t) cast_to_integer (oldval)), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - ret; }) -# define do_exchange_and_add_val_64_acq(pfx, mem, value) 0 -# define do_add_val_64_acq(pfx, mem, value) do { } while (0) -#else -/* XXX We do not really need 64-bit compare-and-exchange. At least - not in the moment. Using it would mean causing portability - problems since not many other 32-bit architectures have support for - such an operation. So don't define any code for now. If it is - really going to be used the code below can be used on Intel Pentium - and later, but NOT on i486. */ -# define __arch_c_compare_and_exchange_val_64_acq(mem, newval, oldval) \ - ({ __typeof (*mem) ret = *(mem); \ - __atomic_link_error (); \ - ret = (newval); \ - ret = (oldval); \ - ret; }) - -# define __arch_compare_and_exchange_val_64_acq(mem, newval, oldval) \ - ({ __typeof (*mem) ret = *(mem); \ - __atomic_link_error (); \ - ret = (newval); \ - ret = (oldval); \ - ret; }) - -# define do_exchange_and_add_val_64_acq(pfx, mem, value) \ - ({ __typeof (value) __addval = (value); \ - __typeof (*mem) __result; \ - __typeof (mem) __memp = (mem); \ - __typeof (*mem) __tmpval; \ - __result = *__memp; \ - do \ - __tmpval = __result; \ - while ((__result = pfx##_compare_and_exchange_val_64_acq \ - (__memp, __result + __addval, __result)) == __tmpval); \ - __result; }) - -# define do_add_val_64_acq(pfx, mem, value) \ - { \ - __typeof (value) __addval = (value); \ - __typeof (mem) __memp = (mem); \ - __typeof (*mem) __oldval = *__memp; \ - __typeof (*mem) __tmpval; \ - do \ - __tmpval = __oldval; \ - while ((__oldval = pfx##_compare_and_exchange_val_64_acq \ - (__memp, __oldval + __addval, __oldval)) == __tmpval); \ - } -#endif - - -/* Note that we need no lock prefix. */ -#define atomic_exchange_acq(mem, newvalue) \ - ({ __typeof (*mem) result; \ - if (sizeof (*mem) == 1) \ - __asm __volatile ("xchgb %b0, %1" \ - : "=q" (result), "=m" (*mem) \ - : "0" (newvalue), "m" (*mem)); \ - else if (sizeof (*mem) == 2) \ - __asm __volatile ("xchgw %w0, %1" \ - : "=r" (result), "=m" (*mem) \ - : "0" (newvalue), "m" (*mem)); \ - else if (sizeof (*mem) == 4) \ - __asm __volatile ("xchgl %0, %1" \ - : "=r" (result), "=m" (*mem) \ - : "0" (newvalue), "m" (*mem)); \ - else if (__HAVE_64B_ATOMICS) \ - __asm __volatile ("xchgq %q0, %1" \ - : "=r" (result), "=m" (*mem) \ - : "0" ((int64_t) cast_to_integer (newvalue)), \ - "m" (*mem)); \ - else \ - { \ - result = 0; \ - __atomic_link_error (); \ - } \ - result; }) - - -#define __arch_exchange_and_add_body(lock, pfx, mem, value) \ - ({ __typeof (*mem) __result; \ - __typeof (value) __addval = (value); \ - if (sizeof (*mem) == 1) \ - __asm __volatile (lock "xaddb %b0, %1" \ - : "=q" (__result), "=m" (*mem) \ - : "0" (__addval), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 2) \ - __asm __volatile (lock "xaddw %w0, %1" \ - : "=r" (__result), "=m" (*mem) \ - : "0" (__addval), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 4) \ - __asm __volatile (lock "xaddl %0, %1" \ - : "=r" (__result), "=m" (*mem) \ - : "0" (__addval), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (__HAVE_64B_ATOMICS) \ - __asm __volatile (lock "xaddq %q0, %1" \ - : "=r" (__result), "=m" (*mem) \ - : "0" ((int64_t) cast_to_integer (__addval)), \ - "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ + if (SINGLE_THREAD_P) \ + __cmpxchg_op ("", (mem), (newval), (oldval)); \ else \ - __result = do_exchange_and_add_val_64_acq (pfx, (mem), __addval); \ - __result; }) - -#define atomic_exchange_and_add(mem, value) \ - __sync_fetch_and_add (mem, value) - -#define __arch_exchange_and_add_cprefix \ - "cmpl $0, %%" SEG_REG ":%P4\n\tje 0f\n\tlock\n0:\t" - -#define catomic_exchange_and_add(mem, value) \ - __arch_exchange_and_add_body (__arch_exchange_and_add_cprefix, __arch_c, \ - mem, value) - - -#define __arch_add_body(lock, pfx, apfx, mem, value) \ - do { \ - if (__builtin_constant_p (value) && (value) == 1) \ - pfx##_increment (mem); \ - else if (__builtin_constant_p (value) && (value) == -1) \ - pfx##_decrement (mem); \ - else if (sizeof (*mem) == 1) \ - __asm __volatile (lock "addb %b1, %0" \ - : "=m" (*mem) \ - : IBR_CONSTRAINT (value), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 2) \ - __asm __volatile (lock "addw %w1, %0" \ - : "=m" (*mem) \ - : "ir" (value), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 4) \ - __asm __volatile (lock "addl %1, %0" \ - : "=m" (*mem) \ - : "ir" (value), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (__HAVE_64B_ATOMICS) \ - __asm __volatile (lock "addq %q1, %0" \ - : "=m" (*mem) \ - : "ir" ((int64_t) cast_to_integer (value)), \ - "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else \ - do_add_val_64_acq (apfx, (mem), (value)); \ - } while (0) - -# define atomic_add(mem, value) \ - __arch_add_body (LOCK_PREFIX, atomic, __arch, mem, value) - -#define __arch_add_cprefix \ - "cmpl $0, %%" SEG_REG ":%P3\n\tje 0f\n\tlock\n0:\t" - -#define catomic_add(mem, value) \ - __arch_add_body (__arch_add_cprefix, atomic, __arch_c, mem, value) + __cmpxchg_op (LOCK_PREFIX, (mem), (newval), (oldval)); \ + ret; }) -#define atomic_add_negative(mem, value) \ - ({ unsigned char __result; \ +#define __xchg_op(lock, mem, arg, op) \ + ({ __typeof (*mem) __ret = (arg); \ if (sizeof (*mem) == 1) \ - __asm __volatile (LOCK_PREFIX "addb %b2, %0; sets %1" \ - : "=m" (*mem), "=qm" (__result) \ - : IBR_CONSTRAINT (value), "m" (*mem)); \ + __asm __volatile (lock #op "b %b0, %1" \ + : "=q" (__ret), "=m" (*mem) \ + : "0" (arg), "m" (*mem) \ + : "memory", "cc"); \ else if (sizeof (*mem) == 2) \ - __asm __volatile (LOCK_PREFIX "addw %w2, %0; sets %1" \ - : "=m" (*mem), "=qm" (__result) \ - : "ir" (value), "m" (*mem)); \ + __asm __volatile (lock #op "w %w0, %1" \ + : "=r" (__ret), "=m" (*mem) \ + : "0" (arg), "m" (*mem) \ + : "memory", "cc"); \ else if (sizeof (*mem) == 4) \ - __asm __volatile (LOCK_PREFIX "addl %2, %0; sets %1" \ - : "=m" (*mem), "=qm" (__result) \ - : "ir" (value), "m" (*mem)); \ + __asm __volatile (lock #op "l %0, %1" \ + : "=r" (__ret), "=m" (*mem) \ + : "0" (arg), "m" (*mem) \ + : "memory", "cc"); \ else if (__HAVE_64B_ATOMICS) \ - __asm __volatile (LOCK_PREFIX "addq %q2, %0; sets %1" \ - : "=m" (*mem), "=qm" (__result) \ - : "ir" ((int64_t) cast_to_integer (value)), \ - "m" (*mem)); \ + __asm __volatile (lock #op "q %q0, %1" \ + : "=r" (__ret), "=m" (*mem) \ + : "0" ((int64_t) cast_to_integer (arg)), \ + "m" (*mem) \ + : "memory", "cc"); \ else \ __atomic_link_error (); \ - __result; }) - + __ret; }) -#define atomic_add_zero(mem, value) \ - ({ unsigned char __result; \ +#define __single_op(lock, mem, op) \ + ({ \ if (sizeof (*mem) == 1) \ - __asm __volatile (LOCK_PREFIX "addb %b2, %0; setz %1" \ - : "=m" (*mem), "=qm" (__result) \ - : IBR_CONSTRAINT (value), "m" (*mem)); \ + __asm __volatile (lock #op "b %b0" \ + : "=m" (*mem) \ + : "m" (*mem) \ + : "memory", "cc"); \ else if (sizeof (*mem) == 2) \ - __asm __volatile (LOCK_PREFIX "addw %w2, %0; setz %1" \ - : "=m" (*mem), "=qm" (__result) \ - : "ir" (value), "m" (*mem)); \ + __asm __volatile (lock #op "w %b0" \ + : "=m" (*mem) \ + : "m" (*mem) \ + : "memory", "cc"); \ else if (sizeof (*mem) == 4) \ - __asm __volatile (LOCK_PREFIX "addl %2, %0; setz %1" \ - : "=m" (*mem), "=qm" (__result) \ - : "ir" (value), "m" (*mem)); \ + __asm __volatile (lock #op "l %b0" \ + : "=m" (*mem) \ + : "m" (*mem) \ + : "memory", "cc"); \ else if (__HAVE_64B_ATOMICS) \ - __asm __volatile (LOCK_PREFIX "addq %q2, %0; setz %1" \ - : "=m" (*mem), "=qm" (__result) \ - : "ir" ((int64_t) cast_to_integer (value)), \ - "m" (*mem)); \ + __asm __volatile (lock #op "q %b0" \ + : "=m" (*mem) \ + : "m" (*mem) \ + : "memory", "cc"); \ else \ - __atomic_link_error (); \ - __result; }) + __atomic_link_error (); \ + }) +/* Note that we need no lock prefix. */ +#define atomic_exchange_acq(mem, newvalue) \ + __xchg_op ("", (mem), (newvalue), xchg) -#define __arch_increment_body(lock, pfx, mem) \ - do { \ - if (sizeof (*mem) == 1) \ - __asm __volatile (lock "incb %b0" \ - : "=m" (*mem) \ - : "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 2) \ - __asm __volatile (lock "incw %w0" \ - : "=m" (*mem) \ - : "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 4) \ - __asm __volatile (lock "incl %0" \ - : "=m" (*mem) \ - : "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (__HAVE_64B_ATOMICS) \ - __asm __volatile (lock "incq %q0" \ - : "=m" (*mem) \ - : "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else \ - do_add_val_64_acq (pfx, mem, 1); \ - } while (0) +#define atomic_add(mem, value) \ + __xchg_op (LOCK_PREFIX, (mem), (value), add); \ -#define atomic_increment(mem) __arch_increment_body (LOCK_PREFIX, __arch, mem) +#define catomic_add(mem, value) \ + ({ \ + if (SINGLE_THREAD_P) \ + __xchg_op ("", (mem), (value), add); \ + else \ + atomic_add (mem, value); \ + }) -#define __arch_increment_cprefix \ - "cmpl $0, %%" SEG_REG ":%P2\n\tje 0f\n\tlock\n0:\t" -#define catomic_increment(mem) \ - __arch_increment_body (__arch_increment_cprefix, __arch_c, mem) +#define atomic_increment(mem) \ + __single_op (LOCK_PREFIX, (mem), inc) +#define catomic_increment(mem) \ + ({ \ + if (SINGLE_THREAD_P) \ + __single_op ("", (mem), inc); \ + else \ + atomic_increment (mem); \ + }) #define atomic_increment_and_test(mem) \ ({ unsigned char __result; \ @@ -357,43 +210,20 @@ : "=m" (*mem), "=qm" (__result) \ : "m" (*mem)); \ else \ - __atomic_link_error (); \ + __atomic_link_error (); \ __result; }) -#define __arch_decrement_body(lock, pfx, mem) \ - do { \ - if (sizeof (*mem) == 1) \ - __asm __volatile (lock "decb %b0" \ - : "=m" (*mem) \ - : "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 2) \ - __asm __volatile (lock "decw %w0" \ - : "=m" (*mem) \ - : "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 4) \ - __asm __volatile (lock "decl %0" \ - : "=m" (*mem) \ - : "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (__HAVE_64B_ATOMICS) \ - __asm __volatile (lock "decq %q0" \ - : "=m" (*mem) \ - : "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else \ - do_add_val_64_acq (pfx, mem, -1); \ - } while (0) - -#define atomic_decrement(mem) __arch_decrement_body (LOCK_PREFIX, __arch, mem) +#define atomic_decrement(mem) \ + __single_op (LOCK_PREFIX, (mem), dec) -#define __arch_decrement_cprefix \ - "cmpl $0, %%" SEG_REG ":%P2\n\tje 0f\n\tlock\n0:\t" - -#define catomic_decrement(mem) \ - __arch_decrement_body (__arch_decrement_cprefix, __arch_c, mem) +#define catomic_decrement(mem) \ + ({ \ + if (SINGLE_THREAD_P) \ + __single_op ("", (mem), dec); \ + else \ + atomic_decrement (mem); \ + }) #define atomic_decrement_and_test(mem) \ @@ -463,73 +293,31 @@ : "=q" (__result), "=m" (*mem) \ : "m" (*mem), "ir" (bit)); \ else \ - __atomic_link_error (); \ + __atomic_link_error (); \ __result; }) -#define __arch_and_body(lock, mem, mask) \ - do { \ - if (sizeof (*mem) == 1) \ - __asm __volatile (lock "andb %b1, %0" \ - : "=m" (*mem) \ - : IBR_CONSTRAINT (mask), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 2) \ - __asm __volatile (lock "andw %w1, %0" \ - : "=m" (*mem) \ - : "ir" (mask), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 4) \ - __asm __volatile (lock "andl %1, %0" \ - : "=m" (*mem) \ - : "ir" (mask), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (__HAVE_64B_ATOMICS) \ - __asm __volatile (lock "andq %q1, %0" \ - : "=m" (*mem) \ - : "ir" (mask), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else \ - __atomic_link_error (); \ - } while (0) - -#define __arch_cprefix \ - "cmpl $0, %%" SEG_REG ":%P3\n\tje 0f\n\tlock\n0:\t" - -#define atomic_and(mem, mask) __arch_and_body (LOCK_PREFIX, mem, mask) - -#define catomic_and(mem, mask) __arch_and_body (__arch_cprefix, mem, mask) +#define atomic_and(mem, mask) \ + __xchg_op (LOCK_PREFIX, (mem), (mask), and) +#define catomic_and(mem, mask) \ + ({ \ + if (SINGLE_THREAD_P) \ + __xchg_op ("", (mem), (mask), and); \ + else \ + atomic_and (mem, mask); \ + }) -#define __arch_or_body(lock, mem, mask) \ - do { \ - if (sizeof (*mem) == 1) \ - __asm __volatile (lock "orb %b1, %0" \ - : "=m" (*mem) \ - : IBR_CONSTRAINT (mask), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 2) \ - __asm __volatile (lock "orw %w1, %0" \ - : "=m" (*mem) \ - : "ir" (mask), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (sizeof (*mem) == 4) \ - __asm __volatile (lock "orl %1, %0" \ - : "=m" (*mem) \ - : "ir" (mask), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else if (__HAVE_64B_ATOMICS) \ - __asm __volatile (lock "orq %q1, %0" \ - : "=m" (*mem) \ - : "ir" (mask), "m" (*mem), \ - "i" (offsetof (tcbhead_t, multiple_threads))); \ - else \ - __atomic_link_error (); \ - } while (0) - -#define atomic_or(mem, mask) __arch_or_body (LOCK_PREFIX, mem, mask) +#define atomic_or(mem, mask) \ + __xchg_op (LOCK_PREFIX, (mem), (mask), or) -#define catomic_or(mem, mask) __arch_or_body (__arch_cprefix, mem, mask) +#define catomic_or(mem, mask) \ + ({ \ + if (SINGLE_THREAD_P) \ + __xchg_op ("", (mem), (mask), or); \ + else \ + atomic_or (mem, mask); \ + }) /* We don't use mfence because it is supposedly slower due to having to provide stronger guarantees (e.g., regarding self-modifying code). */ diff --git a/sysdeps/x86_64/nptl/tcb-offsets.sym b/sysdeps/x86_64/nptl/tcb-offsets.sym index 2bbd563a6c..8ec55a7ea8 100644 --- a/sysdeps/x86_64/nptl/tcb-offsets.sym +++ b/sysdeps/x86_64/nptl/tcb-offsets.sym @@ -9,7 +9,6 @@ CLEANUP_JMP_BUF offsetof (struct pthread, cleanup_jmp_buf) CLEANUP offsetof (struct pthread, cleanup) CLEANUP_PREV offsetof (struct _pthread_cleanup_buffer, __prev) MUTEX_FUTEX offsetof (pthread_mutex_t, __data.__lock) -MULTIPLE_THREADS_OFFSET offsetof (tcbhead_t, multiple_threads) POINTER_GUARD offsetof (tcbhead_t, pointer_guard) FEATURE_1_OFFSET offsetof (tcbhead_t, feature_1) SSP_BASE_OFFSET offsetof (tcbhead_t, ssp_base) From patchwork Fri Jun 10 16:35:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 55025 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3D832382D46E for ; Fri, 10 Jun 2022 16:39:17 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3D832382D46E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1654879157; bh=PKpyTKcdx9tEukaQbE7GhEKl0y9FxcY22LMoVYZmow0=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=vmV63btYnDuUr4jpl8snQ6DmBFJvsLhOwB5x83xIZAQNRal3lOhslIsSYQc2xFK38 RgBTvN9wNBH/rDrE+bF/JEakvnQHHLeAv63JFmh7NLEdeWiGNviCTbJ3ghdLVJgL2A PZ5uHoC1g06dhwA1q849kg9tmjQTfgFPObfiZ/M0= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oi1-x229.google.com (mail-oi1-x229.google.com [IPv6:2607:f8b0:4864:20::229]) by sourceware.org (Postfix) with ESMTPS id 732D5392AC29 for ; Fri, 10 Jun 2022 16:36:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 732D5392AC29 Received: by mail-oi1-x229.google.com with SMTP id k24so9479614oij.2 for ; Fri, 10 Jun 2022 09:36:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PKpyTKcdx9tEukaQbE7GhEKl0y9FxcY22LMoVYZmow0=; b=At4c0vHKXEYTwS2LxiHuAdVcYdDd8KhzUPzbqCurG/smE3FxS5E0dOIjMP2WdcM2Fm ZqJm4u9smFB1Nyg5s+NAW89VN8Y0S+Swx/bDiuMpgyizb/oJhzHxBd9Ih2SVr0pzU52f 4ocnQSJGXz+heznd3kcLTlUl9orjGXnSySLU7Pw1FL4ie28nbQzAdRyqk5IwCmg3hR1Y 3tfgKcMTswh8cA9DnvoZ77zjDcka2PCeDaQfsMIhEZB3g44dXL7dYtzq0JQDkEnObn8l qgvqAoemWsXs5tEehH8yB9nDwG78PZom8I5iXuEKgWXZGNlmE+oXmChvDAbQyaI3PrP/ HqXA== X-Gm-Message-State: AOAM5316namSgXnzSZ+7EOzpgpCPN34mNaY6Ch5T6u6rWA0PZhd5/enN oZc2p/kWUq6FL10DGFrr6M2vHbL2QjbafQ== X-Google-Smtp-Source: ABdhPJy2QmBaDqaxtx+u3pytSXDffKxTYv8mvYL8uAdh4qT2MMRHq9msQ+jDmIk7JF5bk4GScpplHg== X-Received: by 2002:a05:6808:2383:b0:32e:72a1:bd6e with SMTP id bp3-20020a056808238300b0032e72a1bd6emr383475oib.240.1654878963535; Fri, 10 Jun 2022 09:36:03 -0700 (PDT) Received: from birita.. ([2804:431:c7cb:a613:818b:b86c:a3f8:d455]) by smtp.gmail.com with ESMTPSA id p203-20020aca42d4000000b0032efe5871b0sm1262424oia.45.2022.06.10.09.36.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 09:36:03 -0700 (PDT) To: libc-alpha@sourceware.org, Wilco Dijkstra Subject: [PATCH v2 4/4] Remove single-thread.h Date: Fri, 10 Jun 2022 13:35:52 -0300 Message-Id: <20220610163552.3587064-5-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220610163552.3587064-1-adhemerval.zanella@linaro.org> References: <20220610163552.3587064-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" And move SINGLE_THREAD_P macro to sys/single_threaded.h. --- include/sys/single_threaded.h | 15 +++++++--- sysdeps/generic/single-thread.h | 25 ---------------- sysdeps/mach/hurd/sysdep-cancel.h | 5 ---- sysdeps/unix/sysdep.h | 2 +- .../unix/sysv/linux/aarch64/single-thread.h | 2 -- sysdeps/unix/sysv/linux/arc/single-thread.h | 2 -- sysdeps/unix/sysv/linux/arm/single-thread.h | 2 -- sysdeps/unix/sysv/linux/hppa/single-thread.h | 2 -- .../sysv/linux/microblaze/single-thread.h | 2 -- sysdeps/unix/sysv/linux/s390/single-thread.h | 2 -- sysdeps/unix/sysv/linux/single-thread.h | 30 ------------------- .../unix/sysv/linux/x86_64/single-thread.h | 2 -- 12 files changed, 12 insertions(+), 79 deletions(-) delete mode 100644 sysdeps/generic/single-thread.h delete mode 100644 sysdeps/unix/sysv/linux/aarch64/single-thread.h delete mode 100644 sysdeps/unix/sysv/linux/arc/single-thread.h delete mode 100644 sysdeps/unix/sysv/linux/arm/single-thread.h delete mode 100644 sysdeps/unix/sysv/linux/hppa/single-thread.h delete mode 100644 sysdeps/unix/sysv/linux/microblaze/single-thread.h delete mode 100644 sysdeps/unix/sysv/linux/s390/single-thread.h delete mode 100644 sysdeps/unix/sysv/linux/single-thread.h delete mode 100644 sysdeps/unix/sysv/linux/x86_64/single-thread.h diff --git a/include/sys/single_threaded.h b/include/sys/single_threaded.h index 258b01e0b2..c08bd52ab8 100644 --- a/include/sys/single_threaded.h +++ b/include/sys/single_threaded.h @@ -1,12 +1,19 @@ -#include +#ifndef __ASSEMBLER__ +# include -#ifndef _ISOMAC +# ifndef _ISOMAC libc_hidden_proto (__libc_single_threaded); -# ifdef SHARED +# ifdef SHARED extern __typeof (__libc_single_threaded) *__libc_external_single_threaded attribute_hidden; +# endif + +# define SINGLE_THREAD_P (__libc_single_threaded != 0) + +# define RTLD_SINGLE_THREAD_P SINGLE_THREAD_P + # endif -#endif +#endif /* __ASSEMBLER__ */ diff --git a/sysdeps/generic/single-thread.h b/sysdeps/generic/single-thread.h deleted file mode 100644 index 7f8222b38a..0000000000 --- a/sysdeps/generic/single-thread.h +++ /dev/null @@ -1,25 +0,0 @@ -/* Single thread optimization, generic version. - Copyright (C) 2019-2022 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#ifndef _SINGLE_THREAD_H -#define _SINGLE_THREAD_H - -#define SINGLE_THREAD_P (0) -#define RTLD_SINGLE_THREAD_P (0) - -#endif /* _SINGLE_THREAD_H */ diff --git a/sysdeps/mach/hurd/sysdep-cancel.h b/sysdeps/mach/hurd/sysdep-cancel.h index 669c17151a..9311367ab9 100644 --- a/sysdeps/mach/hurd/sysdep-cancel.h +++ b/sysdeps/mach/hurd/sysdep-cancel.h @@ -6,11 +6,6 @@ void __pthread_disable_asynccancel (int oldtype); #pragma weak __pthread_enable_asynccancel #pragma weak __pthread_disable_asynccancel -/* Always multi-thread (since there's at least the sig handler), but no - handling enabled. */ -#define SINGLE_THREAD_P (0) -#define RTLD_SINGLE_THREAD_P (0) - #define LIBC_CANCEL_ASYNC() ({ \ int __cancel_oldtype = 0; \ if (__pthread_enable_asynccancel) \ diff --git a/sysdeps/unix/sysdep.h b/sysdeps/unix/sysdep.h index a1d9df4c73..a8abecb92b 100644 --- a/sysdeps/unix/sysdep.h +++ b/sysdeps/unix/sysdep.h @@ -16,7 +16,7 @@ . */ #include -#include +#include #include #define HAVE_SYSCALLS diff --git a/sysdeps/unix/sysv/linux/aarch64/single-thread.h b/sysdeps/unix/sysv/linux/aarch64/single-thread.h deleted file mode 100644 index a5d3a2aaf4..0000000000 --- a/sysdeps/unix/sysv/linux/aarch64/single-thread.h +++ /dev/null @@ -1,2 +0,0 @@ -#define SINGLE_THREAD_BY_GLOBAL -#include_next diff --git a/sysdeps/unix/sysv/linux/arc/single-thread.h b/sysdeps/unix/sysv/linux/arc/single-thread.h deleted file mode 100644 index a5d3a2aaf4..0000000000 --- a/sysdeps/unix/sysv/linux/arc/single-thread.h +++ /dev/null @@ -1,2 +0,0 @@ -#define SINGLE_THREAD_BY_GLOBAL -#include_next diff --git a/sysdeps/unix/sysv/linux/arm/single-thread.h b/sysdeps/unix/sysv/linux/arm/single-thread.h deleted file mode 100644 index a5d3a2aaf4..0000000000 --- a/sysdeps/unix/sysv/linux/arm/single-thread.h +++ /dev/null @@ -1,2 +0,0 @@ -#define SINGLE_THREAD_BY_GLOBAL -#include_next diff --git a/sysdeps/unix/sysv/linux/hppa/single-thread.h b/sysdeps/unix/sysv/linux/hppa/single-thread.h deleted file mode 100644 index a5d3a2aaf4..0000000000 --- a/sysdeps/unix/sysv/linux/hppa/single-thread.h +++ /dev/null @@ -1,2 +0,0 @@ -#define SINGLE_THREAD_BY_GLOBAL -#include_next diff --git a/sysdeps/unix/sysv/linux/microblaze/single-thread.h b/sysdeps/unix/sysv/linux/microblaze/single-thread.h deleted file mode 100644 index a5d3a2aaf4..0000000000 --- a/sysdeps/unix/sysv/linux/microblaze/single-thread.h +++ /dev/null @@ -1,2 +0,0 @@ -#define SINGLE_THREAD_BY_GLOBAL -#include_next diff --git a/sysdeps/unix/sysv/linux/s390/single-thread.h b/sysdeps/unix/sysv/linux/s390/single-thread.h deleted file mode 100644 index a5d3a2aaf4..0000000000 --- a/sysdeps/unix/sysv/linux/s390/single-thread.h +++ /dev/null @@ -1,2 +0,0 @@ -#define SINGLE_THREAD_BY_GLOBAL -#include_next diff --git a/sysdeps/unix/sysv/linux/single-thread.h b/sysdeps/unix/sysv/linux/single-thread.h deleted file mode 100644 index dd80e82c82..0000000000 --- a/sysdeps/unix/sysv/linux/single-thread.h +++ /dev/null @@ -1,30 +0,0 @@ -/* Single thread optimization, Linux version. - Copyright (C) 2019-2022 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#ifndef _SINGLE_THREAD_H -#define _SINGLE_THREAD_H - -#ifndef __ASSEMBLER__ -# include -#endif - -#define SINGLE_THREAD_P (__libc_single_threaded != 0) - -#define RTLD_SINGLE_THREAD_P SINGLE_THREAD_P - -#endif /* _SINGLE_THREAD_H */ diff --git a/sysdeps/unix/sysv/linux/x86_64/single-thread.h b/sysdeps/unix/sysv/linux/x86_64/single-thread.h deleted file mode 100644 index a5d3a2aaf4..0000000000 --- a/sysdeps/unix/sysv/linux/x86_64/single-thread.h +++ /dev/null @@ -1,2 +0,0 @@ -#define SINGLE_THREAD_BY_GLOBAL -#include_next