From patchwork Wed Jun 23 18:43:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 43977 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DAB9F398B43C for ; Wed, 23 Jun 2021 18:45:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org DAB9F398B43C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1624473949; bh=5IJJqU1N0o6UJfqdVdfU6iuTzZ6HTY+F6Lr2H9mT/kk=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=H2FEaDyt29GeiYwrb82otGl+4DS+vKESYsSrty3PLZ7wNIIrcPGD8ftxId44adIbA CJ8NB8fdLYziKEp3dAIsm2dtn87m0gg83VUTHAHTkcPqJagmXRopz+6+/V0T1C2ZGE +Z+UakVc8WVkex/iJ82TDeKLm2U9W8H2BpmIkzg4= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) by sourceware.org (Postfix) with ESMTPS id 3F67C386187B for ; Wed, 23 Jun 2021 18:44:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 3F67C386187B Received: by mail-qk1-x72c.google.com with SMTP id bj15so7699977qkb.11 for ; Wed, 23 Jun 2021 11:44:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5IJJqU1N0o6UJfqdVdfU6iuTzZ6HTY+F6Lr2H9mT/kk=; b=FQ4j8BPW2AVH+tqcAFxyO6q7+0Expt1do8bmVs9eO1BYYPNS9n8gumVpbOTZAWrAV1 zdBH4/fzqYHlqhRgjuIQHpM94jQN2CGVByScAu0aVr2vRW2VKaRsL/wHKqVJfUV43TNQ b46lbeEI8QxG6nCV2YG8YWiZGSImVGbBt2xRGAfnt0yN2z2j2OywddiotHsNTqpKll81 x+iYp3YOGNcJUQR6cwM79M7BEuAToh7rOG7BWklChQDLkNgMCymarh32NTbmhfwemm7B k50UQKOE+ZrYxbkdvgv8y2pGcCRXoAGhJgQ1GD9IeluzGoqu3bjew86qivNI1mhURd9l vEDg== X-Gm-Message-State: AOAM530A6RAlHK0oMDuTDT+HCzrke2byz9er60YQ7bcjeNnf4QU0Vpv1 Ou6ByvkZBMqb105RBw8gcs0gO/3S8DHYfw== X-Google-Smtp-Source: ABdhPJzH8SVJ5nCxGR+u4eMPEykyJXDdQZ1ZxIJ24S1TgEVqudlriloSb7+lplVOjfhpfUiEZv51ow== X-Received: by 2002:ae9:e407:: with SMTP id q7mr1514095qkc.410.1624473839430; Wed, 23 Jun 2021 11:43:59 -0700 (PDT) Received: from birita.. ([177.194.59.218]) by smtp.gmail.com with ESMTPSA id j4sm500664qtr.72.2021.06.23.11.43.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 11:43:59 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v4 1/4] posix: Consolidate fork implementation Date: Wed, 23 Jun 2021 15:43:51 -0300 Message-Id: <20210623184354.395316-2-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210623184354.395316-1-adhemerval.zanella@linaro.org> References: <20210623184354.395316-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" The Linux nptl implementation is used as base for generic fork implementation to handle the internal locks and mutexes. The system specific bits are moved a new internal _Fork symbol. (This new implementation will be used to provide a async-signal-safe _Fork now that POSIX has clarified that fork might not be async-signal-safe [1]). For Hurd it means that the __nss_database_fork_prepare_parent and __nss_database_fork_subprocess will be run in a slight different order. [1] https://austingroupbugs.net/view.php?id=62 --- include/unistd.h | 4 + posix/Makefile | 3 +- posix/_Fork.c | 34 ++++ posix/fork.c | 116 +++++++++-- sysdeps/generic/fork.h | 32 ++++ sysdeps/mach/hurd/{fork.c => _Fork.c} | 21 +- sysdeps/nptl/_Fork.c | 52 +++++ sysdeps/nptl/fork.c | 264 -------------------------- sysdeps/nptl/fork.h | 148 +++++++++++++++ sysdeps/unix/sysv/linux/arch-fork.h | 3 + 10 files changed, 380 insertions(+), 297 deletions(-) create mode 100644 posix/_Fork.c create mode 100644 sysdeps/generic/fork.h rename sysdeps/mach/hurd/{fork.c => _Fork.c} (98%) create mode 100644 sysdeps/nptl/_Fork.c delete mode 100644 sysdeps/nptl/fork.c create mode 100644 sysdeps/nptl/fork.h diff --git a/include/unistd.h b/include/unistd.h index 34872d8b41..d554d44edb 100644 --- a/include/unistd.h +++ b/include/unistd.h @@ -140,6 +140,10 @@ extern __pid_t __vfork (void); libc_hidden_proto (__vfork) extern int __ttyname_r (int __fd, char *__buf, size_t __buflen); libc_hidden_proto (__ttyname_r) +extern int __ttyname_r (int __fd, char *__buf, size_t __buflen) + attribute_hidden; +extern __pid_t _Fork (void); +libc_hidden_proto (_Fork); extern int __isatty (int __fd) attribute_hidden; extern int __link (const char *__from, const char *__to); extern int __symlink (const char *__from, const char *__to); diff --git a/posix/Makefile b/posix/Makefile index 8d139e54f6..3bd7d605df 100644 --- a/posix/Makefile +++ b/posix/Makefile @@ -39,7 +39,7 @@ routines := \ times \ wait waitpid wait3 wait4 waitid \ alarm sleep pause nanosleep \ - fork vfork _exit register-atfork \ + fork _Fork vfork _exit register-atfork \ execve fexecve execv execle execl execvp execlp execvpe \ getpid getppid \ getuid geteuid getgid getegid getgroups setuid setgid group_member \ @@ -266,6 +266,7 @@ CFLAGS-execl.os = -fomit-frame-pointer CFLAGS-execvp.os = -fomit-frame-pointer CFLAGS-execlp.os = -fomit-frame-pointer CFLAGS-nanosleep.c += -fexceptions -fasynchronous-unwind-tables +CFLAGS-fork.c = $(libio-mtsafe) tstgetopt-ARGS = -a -b -cfoobar --required foobar --optional=bazbug \ --none random --col --color --colour diff --git a/posix/_Fork.c b/posix/_Fork.c new file mode 100644 index 0000000000..4a998c04f1 --- /dev/null +++ b/posix/_Fork.c @@ -0,0 +1,34 @@ +/* _Fork implementation. Generic version. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +/* Clone the calling process, creating an exact copy. Return -1 for errors, + 0 to the new process, and the process ID of the new process to the + old process. + Different than fork, this functions is marked as async-signal-safe by + POSIX (by Austin Group issue 62). */ +pid_t +_Fork (void) +{ + __set_errno (ENOSYS); + return -1; +} +libc_hidden_def (_Fork) +stub_warning (_Fork) diff --git a/posix/fork.c b/posix/fork.c index 05bda04ac5..44caf8d166 100644 --- a/posix/fork.c +++ b/posix/fork.c @@ -1,4 +1,5 @@ -/* Copyright (C) 1991-2021 Free Software Foundation, Inc. +/* fork - create a child process. + Copyright (C) 1991-2021 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -15,20 +16,109 @@ License along with the GNU C Library; if not, see . */ -#include -#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +static void +fresetlockfiles (void) +{ + _IO_ITER i; + + for (i = _IO_iter_begin(); i != _IO_iter_end(); i = _IO_iter_next(i)) + if ((_IO_iter_file (i)->_flags & _IO_USER_LOCK) == 0) + _IO_lock_init (*((_IO_lock_t *) _IO_iter_file(i)->_lock)); +} -/* Clone the calling process, creating an exact copy. - Return -1 for errors, 0 to the new process, - and the process ID of the new process to the old process. */ -int -__fork (void) +pid_t +__libc_fork (void) { - __set_errno (ENOSYS); - return -1; + /* Determine if we are running multiple threads. We skip some fork + handlers in the single-thread case, to make fork safer to use in + signal handlers. */ + bool multiple_threads = __libc_single_threaded == 0; + + __run_fork_handlers (atfork_run_prepare, multiple_threads); + + struct nss_database_data nss_database_data; + + /* If we are not running multiple threads, we do not have to + preserve lock state. If fork runs from a signal handler, only + async-signal-safe functions can be used in the child. These data + structures are only used by unsafe functions, so their state does + not matter if fork was called from a signal handler. */ + if (multiple_threads) + { + call_function_static_weak (__nss_database_fork_prepare_parent, + &nss_database_data); + + _IO_list_lock (); + + /* Acquire malloc locks. This needs to come last because fork + handlers may use malloc, and the libio list lock has an + indirect malloc dependency as well (via the getdelim + function). */ + call_function_static_weak (__malloc_fork_lock_parent); + } + + pid_t pid = _Fork (); + + if (pid == 0) + { + fork_system_setup (); + + /* Reset the lock state in the multi-threaded case. */ + if (multiple_threads) + { + __libc_unwind_link_after_fork (); + + fork_system_setup_after_fork (); + + /* Release malloc locks. */ + call_function_static_weak (__malloc_fork_unlock_child); + + /* Reset the file list. These are recursive mutexes. */ + fresetlockfiles (); + + /* Reset locks in the I/O code. */ + _IO_list_resetlock (); + + call_function_static_weak (__nss_database_fork_subprocess, + &nss_database_data); + } + + /* Reset the lock the dynamic loader uses to protect its data. */ + __rtld_lock_initialize (GL(dl_load_lock)); + + reclaim_stacks (); + + /* Run the handlers registered for the child. */ + __run_fork_handlers (atfork_run_child, multiple_threads); + } + else + { + /* Release acquired locks in the multi-threaded case. */ + if (multiple_threads) + { + /* Release malloc locks, parent process variant. */ + call_function_static_weak (__malloc_fork_unlock_parent); + + /* We execute this even if the 'fork' call failed. */ + _IO_list_unlock (); + } + + /* Run the handlers registered for the parent. */ + __run_fork_handlers (atfork_run_parent, multiple_threads); + } + + return pid; } +weak_alias (__libc_fork, __fork) libc_hidden_def (__fork) -stub_warning (fork) - -weak_alias (__fork, fork) +weak_alias (__libc_fork, fork) diff --git a/sysdeps/generic/fork.h b/sysdeps/generic/fork.h new file mode 100644 index 0000000000..be42cf2481 --- /dev/null +++ b/sysdeps/generic/fork.h @@ -0,0 +1,32 @@ +/* System specific fork hooks. Generic version. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +static inline void +fork_system_setup (void) +{ +} + +static inline void +fork_system_setup_after_fork (void) +{ +} + +static inline void +reclaim_stacks (void) +{ +} diff --git a/sysdeps/mach/hurd/fork.c b/sysdeps/mach/hurd/_Fork.c similarity index 98% rename from sysdeps/mach/hurd/fork.c rename to sysdeps/mach/hurd/_Fork.c index 92a51693e0..df4ee05faa 100644 --- a/sysdeps/mach/hurd/fork.c +++ b/sysdeps/mach/hurd/_Fork.c @@ -58,16 +58,13 @@ DEFINE_HOOK (_hurd_fork_parent_hook, (void)); Return -1 for errors, 0 to the new process, and the process ID of the new process to the old process. */ pid_t -__fork (void) +_Fork (void) { jmp_buf env; pid_t pid; size_t i; error_t err; struct hurd_sigstate *volatile ss; - struct nss_database_data nss_database_data; - - __run_fork_handlers (atfork_run_prepare, true); ss = _hurd_self_sigstate (); retry: @@ -108,9 +105,6 @@ retry: /* Run things that prepare for forking before we create the task. */ RUN_HOOK (_hurd_fork_prepare_hook, ()); - call_function_static_weak (__nss_database_fork_prepare_parent, - &nss_database_data); - /* Lock things that want to be locked before we fork. */ { void *const *p; @@ -670,9 +664,6 @@ retry: _hurd_malloc_fork_child (); call_function_static_weak (__malloc_fork_unlock_child); - call_function_static_weak (__nss_database_fork_subprocess, - &nss_database_data); - /* Run things that want to run in the child task to set up. */ RUN_HOOK (_hurd_fork_child_hook, ()); @@ -723,14 +714,6 @@ retry: /* Got a signal while inside an RPC of the critical section, retry again */ goto retry; - if (!err) - { - __run_fork_handlers (pid == 0 ? atfork_run_child : atfork_run_parent, - true); - } - return err ? __hurd_fail (err) : pid; } -libc_hidden_def (__fork) - -weak_alias (__fork, fork) +libc_hidden_def (_Fork) diff --git a/sysdeps/nptl/_Fork.c b/sysdeps/nptl/_Fork.c new file mode 100644 index 0000000000..6eae146b4d --- /dev/null +++ b/sysdeps/nptl/_Fork.c @@ -0,0 +1,52 @@ +/* _Fork implementation. Linux version. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +/* Pointer to the fork generation counter in the thread library. */ +extern unsigned long int *__fork_generation_pointer attribute_hidden; + +pid_t +_Fork (void) +{ + pid_t pid = arch_fork (&THREAD_SELF->tid); + if (pid == 0) + { + struct pthread *self = THREAD_SELF; + + /* Initialize the robust mutex list setting in the kernel which has + been reset during the fork. We do not check for errors because if + it fails here, it must have failed at process startup as well and + nobody could have used robust mutexes. + Before we do that, we have to clear the list of robust mutexes + because we do not inherit ownership of mutexes from the parent. + We do not have to set self->robust_head.futex_offset since we do + inherit the correct value from the parent. We do not need to clear + the pending operation because it must have been zero when fork was + called. */ +#if __PTHREAD_MUTEX_HAVE_PREV + self->robust_prev = &self->robust_head; +#endif + self->robust_head.list = &self->robust_head; + INTERNAL_SYSCALL_CALL (set_robust_list, &self->robust_head, + sizeof (struct robust_list_head)); + } + return pid; +} +libc_hidden_def (_Fork) diff --git a/sysdeps/nptl/fork.c b/sysdeps/nptl/fork.c deleted file mode 100644 index 39ab797612..0000000000 --- a/sysdeps/nptl/fork.c +++ /dev/null @@ -1,264 +0,0 @@ -/* Copyright (C) 2002-2021 Free Software Foundation, Inc. - This file is part of the GNU C Library. - Contributed by Ulrich Drepper , 2002. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -static void -fresetlockfiles (void) -{ - _IO_ITER i; - - for (i = _IO_iter_begin(); i != _IO_iter_end(); i = _IO_iter_next(i)) - if ((_IO_iter_file (i)->_flags & _IO_USER_LOCK) == 0) - _IO_lock_init (*((_IO_lock_t *) _IO_iter_file(i)->_lock)); -} - -/* In case of a fork() call the memory allocation in the child will be - the same but only one thread is running. All stacks except that of - the one running thread are not used anymore. We have to recycle - them. */ -static void -reclaim_stacks (void) -{ - struct pthread *self = (struct pthread *) THREAD_SELF; - - /* No locking necessary. The caller is the only stack in use. But - we have to be aware that we might have interrupted a list - operation. */ - - if (GL (dl_in_flight_stack) != 0) - { - bool add_p = GL (dl_in_flight_stack) & 1; - list_t *elem = (list_t *) (GL (dl_in_flight_stack) & ~(uintptr_t) 1); - - if (add_p) - { - /* We always add at the beginning of the list. So in this case we - only need to check the beginning of these lists to see if the - pointers at the head of the list are inconsistent. */ - list_t *l = NULL; - - if (GL (dl_stack_used).next->prev != &GL (dl_stack_used)) - l = &GL (dl_stack_used); - else if (GL (dl_stack_cache).next->prev != &GL (dl_stack_cache)) - l = &GL (dl_stack_cache); - - if (l != NULL) - { - assert (l->next->prev == elem); - elem->next = l->next; - elem->prev = l; - l->next = elem; - } - } - else - { - /* We can simply always replay the delete operation. */ - elem->next->prev = elem->prev; - elem->prev->next = elem->next; - } - - GL (dl_in_flight_stack) = 0; - } - - /* Mark all stacks except the still running one as free. */ - list_t *runp; - list_for_each (runp, &GL (dl_stack_used)) - { - struct pthread *curp = list_entry (runp, struct pthread, list); - if (curp != self) - { - /* This marks the stack as free. */ - curp->tid = 0; - - /* Account for the size of the stack. */ - GL (dl_stack_cache_actsize) += curp->stackblock_size; - - if (curp->specific_used) - { - /* Clear the thread-specific data. */ - memset (curp->specific_1stblock, '\0', - sizeof (curp->specific_1stblock)); - - curp->specific_used = false; - - for (size_t cnt = 1; cnt < PTHREAD_KEY_1STLEVEL_SIZE; ++cnt) - if (curp->specific[cnt] != NULL) - { - memset (curp->specific[cnt], '\0', - sizeof (curp->specific_1stblock)); - - /* We have allocated the block which we do not - free here so re-set the bit. */ - curp->specific_used = true; - } - } - } - } - - /* Add the stack of all running threads to the cache. */ - list_splice (&GL (dl_stack_used), &GL (dl_stack_cache)); - - /* Remove the entry for the current thread to from the cache list - and add it to the list of running threads. Which of the two - lists is decided by the user_stack flag. */ - list_del (&self->list); - - /* Re-initialize the lists for all the threads. */ - INIT_LIST_HEAD (&GL (dl_stack_used)); - INIT_LIST_HEAD (&GL (dl_stack_user)); - - if (__glibc_unlikely (THREAD_GETMEM (self, user_stack))) - list_add (&self->list, &GL (dl_stack_user)); - else - list_add (&self->list, &GL (dl_stack_used)); -} - -pid_t -__libc_fork (void) -{ - pid_t pid; - - /* Determine if we are running multiple threads. We skip some fork - handlers in the single-thread case, to make fork safer to use in - signal handlers. POSIX requires that fork is async-signal-safe, - but our current fork implementation is not. */ - bool multiple_threads = __libc_single_threaded == 0; - - __run_fork_handlers (atfork_run_prepare, multiple_threads); - - struct nss_database_data nss_database_data; - - /* If we are not running multiple threads, we do not have to - preserve lock state. If fork runs from a signal handler, only - async-signal-safe functions can be used in the child. These data - structures are only used by unsafe functions, so their state does - not matter if fork was called from a signal handler. */ - if (multiple_threads) - { - call_function_static_weak (__nss_database_fork_prepare_parent, - &nss_database_data); - - _IO_list_lock (); - - /* Acquire malloc locks. This needs to come last because fork - handlers may use malloc, and the libio list lock has an - indirect malloc dependency as well (via the getdelim - function). */ - call_function_static_weak (__malloc_fork_lock_parent); - } - - pid = arch_fork (&THREAD_SELF->tid); - - if (pid == 0) - { - struct pthread *self = THREAD_SELF; - - /* See __pthread_once. */ - __fork_generation += __PTHREAD_ONCE_FORK_GEN_INCR; - - /* Initialize the robust mutex list setting in the kernel which has - been reset during the fork. We do not check for errors because if - it fails here, it must have failed at process startup as well and - nobody could have used robust mutexes. - Before we do that, we have to clear the list of robust mutexes - because we do not inherit ownership of mutexes from the parent. - We do not have to set self->robust_head.futex_offset since we do - inherit the correct value from the parent. We do not need to clear - the pending operation because it must have been zero when fork was - called. */ -#if __PTHREAD_MUTEX_HAVE_PREV - self->robust_prev = &self->robust_head; -#endif - self->robust_head.list = &self->robust_head; - INTERNAL_SYSCALL_CALL (set_robust_list, &self->robust_head, - sizeof (struct robust_list_head)); - - /* Reset the lock state in the multi-threaded case. */ - if (multiple_threads) - { - __libc_unwind_link_after_fork (); - - /* There is one thread running. */ - __nptl_nthreads = 1; - - /* Initialize thread library locks. */ - GL (dl_stack_cache_lock) = LLL_LOCK_INITIALIZER; - __default_pthread_attr_lock = LLL_LOCK_INITIALIZER; - - /* Release malloc locks. */ - call_function_static_weak (__malloc_fork_unlock_child); - - /* Reset the file list. These are recursive mutexes. */ - fresetlockfiles (); - - /* Reset locks in the I/O code. */ - _IO_list_resetlock (); - - call_function_static_weak (__nss_database_fork_subprocess, - &nss_database_data); - } - - /* Reset the lock the dynamic loader uses to protect its data. */ - __rtld_lock_initialize (GL(dl_load_lock)); - - reclaim_stacks (); - - /* Run the handlers registered for the child. */ - __run_fork_handlers (atfork_run_child, multiple_threads); - } - else - { - /* Release acquired locks in the multi-threaded case. */ - if (multiple_threads) - { - /* Release malloc locks, parent process variant. */ - call_function_static_weak (__malloc_fork_unlock_parent); - - /* We execute this even if the 'fork' call failed. */ - _IO_list_unlock (); - } - - /* Run the handlers registered for the parent. */ - __run_fork_handlers (atfork_run_parent, multiple_threads); - } - - return pid; -} -weak_alias (__libc_fork, __fork) -libc_hidden_def (__fork) -weak_alias (__libc_fork, fork) diff --git a/sysdeps/nptl/fork.h b/sysdeps/nptl/fork.h new file mode 100644 index 0000000000..3134d7ab94 --- /dev/null +++ b/sysdeps/nptl/fork.h @@ -0,0 +1,148 @@ +/* System specific fork hooks. Linux version. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _FORK_H +#define _FORK_H + +#include +#include +#include +#include +#include + +static inline void +fork_system_setup (void) +{ + /* See __pthread_once. */ + __fork_generation += __PTHREAD_ONCE_FORK_GEN_INCR; +} + +static void +fork_system_setup_after_fork (void) +{ + /* There is one thread running. */ + __nptl_nthreads = 1; + + /* Initialize thread library locks. */ + GL (dl_stack_cache_lock) = LLL_LOCK_INITIALIZER; + __default_pthread_attr_lock = LLL_LOCK_INITIALIZER; +} + +/* In case of a fork() call the memory allocation in the child will be + the same but only one thread is running. All stacks except that of + the one running thread are not used anymore. We have to recycle + them. */ +static void +reclaim_stacks (void) +{ + struct pthread *self = (struct pthread *) THREAD_SELF; + + /* No locking necessary. The caller is the only stack in use. But + we have to be aware that we might have interrupted a list + operation. */ + + if (GL (dl_in_flight_stack) != 0) + { + bool add_p = GL (dl_in_flight_stack) & 1; + list_t *elem = (list_t *) (GL (dl_in_flight_stack) & ~(uintptr_t) 1); + + if (add_p) + { + /* We always add at the beginning of the list. So in this case we + only need to check the beginning of these lists to see if the + pointers at the head of the list are inconsistent. */ + list_t *l = NULL; + + if (GL (dl_stack_used).next->prev != &GL (dl_stack_used)) + l = &GL (dl_stack_used); + else if (GL (dl_stack_cache).next->prev != &GL (dl_stack_cache)) + l = &GL (dl_stack_cache); + + if (l != NULL) + { + assert (l->next->prev == elem); + elem->next = l->next; + elem->prev = l; + l->next = elem; + } + } + else + { + /* We can simply always replay the delete operation. */ + elem->next->prev = elem->prev; + elem->prev->next = elem->next; + } + + GL (dl_in_flight_stack) = 0; + } + + /* Mark all stacks except the still running one as free. */ + list_t *runp; + list_for_each (runp, &GL (dl_stack_used)) + { + struct pthread *curp = list_entry (runp, struct pthread, list); + if (curp != self) + { + /* This marks the stack as free. */ + curp->tid = 0; + + /* Account for the size of the stack. */ + GL (dl_stack_cache_actsize) += curp->stackblock_size; + + if (curp->specific_used) + { + /* Clear the thread-specific data. */ + memset (curp->specific_1stblock, '\0', + sizeof (curp->specific_1stblock)); + + curp->specific_used = false; + + for (size_t cnt = 1; cnt < PTHREAD_KEY_1STLEVEL_SIZE; ++cnt) + if (curp->specific[cnt] != NULL) + { + memset (curp->specific[cnt], '\0', + sizeof (curp->specific_1stblock)); + + /* We have allocated the block which we do not + free here so re-set the bit. */ + curp->specific_used = true; + } + } + } + } + + /* Add the stack of all running threads to the cache. */ + list_splice (&GL (dl_stack_used), &GL (dl_stack_cache)); + + /* Remove the entry for the current thread to from the cache list + and add it to the list of running threads. Which of the two + lists is decided by the user_stack flag. */ + list_del (&self->list); + + /* Re-initialize the lists for all the threads. */ + INIT_LIST_HEAD (&GL (dl_stack_used)); + INIT_LIST_HEAD (&GL (dl_stack_user)); + + if (__glibc_unlikely (THREAD_GETMEM (self, user_stack))) + list_add (&self->list, &GL (dl_stack_user)); + else + list_add (&self->list, &GL (dl_stack_used)); +} + + +#endif diff --git a/sysdeps/unix/sysv/linux/arch-fork.h b/sysdeps/unix/sysv/linux/arch-fork.h index 35d9397ff8..b846da08f9 100644 --- a/sysdeps/unix/sysv/linux/arch-fork.h +++ b/sysdeps/unix/sysv/linux/arch-fork.h @@ -19,6 +19,9 @@ #ifndef __ARCH_FORK_H #define __ARCH_FORK_H +#include +#include +#include #include /* Call the clone syscall with fork semantic. The CTID address is used