From patchwork Mon Jun 21 11:16:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kurt Kanzenbach X-Patchwork-Id: 43930 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 64BC53898528 for ; Mon, 21 Jun 2021 11:21:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 64BC53898528 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1624274508; bh=fJCVXMASq2girTD1YGIxd060Hnj1iMvzPvXz6S3CUAE=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=maZIVv4ZSlL2GZ64M0Jk6KxPS4EdoQmCNffzotfEqP1HI8uPFAhi8yMyLgG+ysQfl fcfXgKmgAnNIzrV+TDiy1T7Ng+hA1rGcZ6E56NXK94vrwrgMPCkyIxYxAa8kxD989u DQ4509uL9b5Pgn73Gf2o9EDhkJ3D5T0/mn3BfAmQ= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by sourceware.org (Postfix) with ESMTPS id 5286C393A41B for ; Mon, 21 Jun 2021 11:17:20 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 5286C393A41B To: libc-alpha@sourceware.org Subject: [PATCH RFC 1/3] nptl: Introduce futex_lock_pi2() Date: Mon, 21 Jun 2021 13:16:48 +0200 Message-Id: <20210621111650.1164689-2-kurt@linutronix.de> In-Reply-To: <20210621111650.1164689-1-kurt@linutronix.de> References: <20210621111650.1164689-1-kurt@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-10.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kurt Kanzenbach via Libc-alpha From: Kurt Kanzenbach Reply-To: Kurt Kanzenbach Cc: Florian Weimer , Sebastian Andrzej Siewior , Kurt Kanzenbach , Thomas Gleixner Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This variant of futex_lock() has support for selectable clocks and priority inheritance. The underlying FUTEX_LOCK_PI2 operation has been recently introduced into the Linux kernel. It can be used for implementing pthread_mutex_clocklock(MONOTONIC)/PI. Signed-off-by: Kurt Kanzenbach --- sysdeps/nptl/futex-internal.h | 77 +++++++++++++++++++++++++++++++ sysdeps/nptl/lowlevellock-futex.h | 1 + 2 files changed, 78 insertions(+) diff --git a/sysdeps/nptl/futex-internal.h b/sysdeps/nptl/futex-internal.h index 969ab2bf4bf8..60bdae84405c 100644 --- a/sysdeps/nptl/futex-internal.h +++ b/sysdeps/nptl/futex-internal.h @@ -295,6 +295,83 @@ futex_lock_pi64 (int *futex_word, const struct __timespec64 *abstime, } } +/* The operation checks the value of the futex, if the value is 0, then + it is atomically set to the caller's thread ID. If the futex value is + nonzero, it is atomically sets the FUTEX_WAITERS bit, which signals wrt + other futex owner that it cannot unlock the futex in user space by + atomically by setting its value to 0. + + If more than one wait operations is issued, the enqueueing of the waiters + are done in descending priority order. + + The ABSTIME arguments provides an absolute timeout (measured against the + CLOCK_REALTIME or CLOCK_MONOTONIC clock). If TIMEOUT is NULL, the operation + will block indefinitely. + + Returns: + + - 0 if woken by a PI unlock operation or spuriously. + - EAGAIN if the futex owner thread ID is about to exit, but has not yet + handled the state cleanup. + - EDEADLK if the futex is already locked by the caller. + - ESRCH if the thread ID int he futex does not exist. + - EINVAL is the state is corrupted or if there is a waiter on the + futex or if the clockid is invalid. + - ETIMEDOUT if the ABSTIME expires. + - ENOSYS if FUTEX_LOCK_PI2 is not available. +*/ +static __always_inline int +futex_lock_pi2_64 (int *futex_word, clockid_t clockid, + const struct __timespec64 *abstime, + int private) +{ + unsigned int clockbit; + + if (! lll_futex_supported_clockid (clockid)) + return EINVAL; + + clockbit = (clockid == CLOCK_REALTIME) ? FUTEX_CLOCK_REALTIME : 0; + int op = __lll_private_flag (FUTEX_LOCK_PI2 | clockbit, private); + + int err = INTERNAL_SYSCALL_CALL (futex_time64, futex_word, op, + 0, abstime); +#ifndef __ASSUME_TIME64_SYSCALLS + if (err == -ENOSYS) + { + if (abstime != NULL && ! in_time_t_range (abstime->tv_sec)) + return EOVERFLOW; + + struct timespec ts32; + if (abstime != NULL) + ts32 = valid_timespec64_to_timespec (*abstime); + + err = INTERNAL_SYSCALL_CALL (futex, futex_word, op, 0, + abstime != NULL ? &ts32 : NULL); + } +#endif + switch (err) + { + case 0: + case -EAGAIN: + case -EINTR: + case -ETIMEDOUT: + case -ESRCH: + case -EDEADLK: + case -ENOSYS: /* Futex lock pi 2 is not available on this kernel. */ + case -EINVAL: /* This indicates either state corruption or that the kernel + found a waiter on futex address which is waiting via + FUTEX_WAIT or FUTEX_WAIT_BITSET. This is reported on + some futex_lock_pi usage (pthread_mutex_timedlock for + instance). */ + return -err; + + case -EFAULT: /* Must have been caused by a glibc or application bug. */ + /* No other errors are documented at this time. */ + default: + futex_fatal_error (); + } +} + /* Wakes the top priority waiter that called a futex_lock_pi operation on the futex. diff --git a/sysdeps/nptl/lowlevellock-futex.h b/sysdeps/nptl/lowlevellock-futex.h index 66ebfe50f4c1..abda179e0de2 100644 --- a/sysdeps/nptl/lowlevellock-futex.h +++ b/sysdeps/nptl/lowlevellock-futex.h @@ -38,6 +38,7 @@ #define FUTEX_WAKE_BITSET 10 #define FUTEX_WAIT_REQUEUE_PI 11 #define FUTEX_CMP_REQUEUE_PI 12 +#define FUTEX_LOCK_PI2 13 #define FUTEX_PRIVATE_FLAG 128 #define FUTEX_CLOCK_REALTIME 256 From patchwork Mon Jun 21 11:16:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kurt Kanzenbach X-Patchwork-Id: 43929 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A4B7B389683C for ; Mon, 21 Jun 2021 11:21:04 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A4B7B389683C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1624274464; bh=SMwqXTZ4uLqd9fi30wR6RF8ajsp6+eRpzhW9EvbhUhM=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=lNZqcrJR1pL8gO79bQn1LZh17l5qfhXPWODU1OOt461cfwHmdQUmxHY6MGI7lsizo C0fcwoIeRs9NVACVR8JtdbrKszTebPCVZGLb/P4ZVtbjwA89jQ+ZpN9YnR1ZaWVSP1 n9TUfRrhIlpBCNbG1evMZUpLy/XjsQwp1AVgd8Lo= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by sourceware.org (Postfix) with ESMTPS id 75953393A426 for ; Mon, 21 Jun 2021 11:17:20 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 75953393A426 To: libc-alpha@sourceware.org Subject: [PATCH RFC 2/3] nptl: Use futex_lock_pi2() Date: Mon, 21 Jun 2021 13:16:49 +0200 Message-Id: <20210621111650.1164689-3-kurt@linutronix.de> In-Reply-To: <20210621111650.1164689-1-kurt@linutronix.de> References: <20210621111650.1164689-1-kurt@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-11.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kurt Kanzenbach via Libc-alpha From: Kurt Kanzenbach Reply-To: Kurt Kanzenbach Cc: Florian Weimer , Sebastian Andrzej Siewior , Kurt Kanzenbach , Thomas Gleixner Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Use futex_lock_pi2() by default, because it supports selectable clocks. On older kernels futex_lock_pi2() will return ENOSYS. Fallback to futex_lock_pi() and don't support CLOCK_MONOTONIC then. Signed-off-by: Kurt Kanzenbach --- nptl/pthread_mutex_timedlock.c | 24 ++++++++++++++++++------ sysdeps/nptl/futex-internal.h | 17 +++++++++++++++++ 2 files changed, 35 insertions(+), 6 deletions(-) diff --git a/nptl/pthread_mutex_timedlock.c b/nptl/pthread_mutex_timedlock.c index 5afd6222d61e..e7e001d2c17a 100644 --- a/nptl/pthread_mutex_timedlock.c +++ b/nptl/pthread_mutex_timedlock.c @@ -300,11 +300,15 @@ __pthread_mutex_clocklock_common (pthread_mutex_t *mutex, case PTHREAD_MUTEX_PI_ROBUST_NORMAL_NP: case PTHREAD_MUTEX_PI_ROBUST_ADAPTIVE_NP: { - /* Currently futex FUTEX_LOCK_PI operation only provides support for - CLOCK_REALTIME and trying to emulate by converting a - CLOCK_MONOTONIC to CLOCK_REALTIME will take in account possible - changes to the wall clock. */ - if (__glibc_unlikely (clockid != CLOCK_REALTIME)) + /* Currently futex FUTEX_LOCK_PI operation only provides support for + CLOCK_REALTIME and trying to emulate by converting a + CLOCK_MONOTONIC to CLOCK_REALTIME will take in account possible + changes to the wall clock. + + However, newer Linux kernels support FUTEX_LOCK_PI2 with selectable + clock support. */ + if (__glibc_unlikely (! futex_lockpi2_supported () && + clockid != CLOCK_REALTIME)) return EINVAL; int kind, robust; @@ -370,7 +374,15 @@ __pthread_mutex_clocklock_common (pthread_mutex_t *mutex, int private = (robust ? PTHREAD_ROBUST_MUTEX_PSHARED (mutex) : PTHREAD_MUTEX_PSHARED (mutex)); - int e = futex_lock_pi64 (&mutex->__data.__lock, abstime, private); + int e = futex_lock_pi2_64 (&mutex->__data.__lock, clockid, abstime, + private); + + /* Use futex futex_lock_pi2() by default. This doesn't work for + older kernels. So fallback to futex_lock_pi() then. The clockid + is checked above already. */ + if (__glibc_unlikely (e == ENOSYS)) + e = futex_lock_pi64 (&mutex->__data.__lock, abstime, private); + if (e == ETIMEDOUT) return ETIMEDOUT; else if (e == ESRCH || e == EDEADLK) diff --git a/sysdeps/nptl/futex-internal.h b/sysdeps/nptl/futex-internal.h index 60bdae84405c..b5f101b484c2 100644 --- a/sysdeps/nptl/futex-internal.h +++ b/sysdeps/nptl/futex-internal.h @@ -372,6 +372,23 @@ futex_lock_pi2_64 (int *futex_word, clockid_t clockid, } } +/* Checks whether FUTEX_LOCK_PI2 operation is available on the running Linux + kernel. FUTEX_LOCK_PI2 has been introduced on newer kernel versions. + + Returns true if FUTEX_LOCK_PI2 is available, false otherwise. */ +static __always_inline bool +futex_lockpi2_supported (void) +{ + int err = INTERNAL_SYSCALL_CALL (futex_time64, NULL, FUTEX_LOCK_PI2, 0, NULL); + +#ifndef __ASSUME_TIME64_SYSCALLS + if (err == -ENOSYS) + err = INTERNAL_SYSCALL_CALL (futex, NULL, FUTEX_LOCK_PI2, 0, NULL); +#endif + + return err != -ENOSYS; +} + /* Wakes the top priority waiter that called a futex_lock_pi operation on the futex. From patchwork Mon Jun 21 11:16:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kurt Kanzenbach X-Patchwork-Id: 43931 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5FF073898516 for ; Mon, 21 Jun 2021 11:22:32 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5FF073898516 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1624274552; bh=dXLn4q07B5AjJKQbo/WqnAY5L9DKAHzerKc17b+8LZE=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=avQKXd9iMW5cQg3euDjneuOrMNAe010zPAS95OpnLnYD9JpQHu4MtcOmHvKGjlT26 lHLugZr2z6dMCu3Qfb8YzxC3bKshTBMsZuT26uhHwROZvrZ+Y/I+ba3YNydtwG2kSg Fg8GxbENNNI4gqDDeSRImdqSMm8vKsLrHjsJYsBk= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by sourceware.org (Postfix) with ESMTPS id 6CEAC393A42C for ; Mon, 21 Jun 2021 11:17:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6CEAC393A42C To: libc-alpha@sourceware.org Subject: [PATCH RFC 3/3] nptl: Include CLOCK_MONOTONIC in mutex tests Date: Mon, 21 Jun 2021 13:16:50 +0200 Message-Id: <20210621111650.1164689-4-kurt@linutronix.de> In-Reply-To: <20210621111650.1164689-1-kurt@linutronix.de> References: <20210621111650.1164689-1-kurt@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-11.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kurt Kanzenbach via Libc-alpha From: Kurt Kanzenbach Reply-To: Kurt Kanzenbach Cc: Florian Weimer , Sebastian Andrzej Siewior , Kurt Kanzenbach , Thomas Gleixner Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Include pthread_mutex_clocklock(MONOTONIC)/PI in the testcases if FUTEX_LOCK_PI2 is available. Add the check at compile time to keep the test implementation simple. Signed-off-by: Kurt Kanzenbach --- nptl/tst-mutexpi10.c | 8 ++++++++ sysdeps/pthread/tst-mutex5.c | 3 ++- sysdeps/pthread/tst-mutex9.c | 3 ++- sysdeps/unix/sysv/linux/kernel-features.h | 8 ++++++++ 4 files changed, 20 insertions(+), 2 deletions(-) diff --git a/nptl/tst-mutexpi10.c b/nptl/tst-mutexpi10.c index da781d0d7a93..19644b6f06a9 100644 --- a/nptl/tst-mutexpi10.c +++ b/nptl/tst-mutexpi10.c @@ -25,6 +25,8 @@ #include #include +#include + static int do_test (void) { @@ -56,8 +58,14 @@ do_test (void) struct timespec tmo = timespec_add (xclock_now (CLOCK_MONOTONIC), make_timespec (0, 100000000)); +#if __ASSUME_FUTEX_LOCK_PI2 + TEST_COMPARE (pthread_mutex_clocklock (&mtx, CLOCK_MONOTONIC, &tmo), + 0); + xpthread_mutex_unlock (&mtx); +#else TEST_COMPARE (pthread_mutex_clocklock (&mtx, CLOCK_MONOTONIC, &tmo), EINVAL); +#endif xpthread_mutex_destroy (&mtx); } diff --git a/sysdeps/pthread/tst-mutex5.c b/sysdeps/pthread/tst-mutex5.c index 7dc5331cfc08..82235ae479b0 100644 --- a/sysdeps/pthread/tst-mutex5.c +++ b/sysdeps/pthread/tst-mutex5.c @@ -26,6 +26,7 @@ #include #include #include +#include #ifdef ENABLE_PP #include "tst-tpp.h" @@ -122,7 +123,7 @@ static int do_test (void) do_test_clock (CLOCK_USE_TIMEDLOCK, "timedlock"); do_test_clock (CLOCK_REALTIME, "clocklock(realtime)"); -#ifndef ENABLE_PI +#if ! defined(ENABLE_PI) || __ASSUME_FUTEX_LOCK_PI2 do_test_clock (CLOCK_MONOTONIC, "clocklock(monotonic)"); #endif return 0; diff --git a/sysdeps/pthread/tst-mutex9.c b/sysdeps/pthread/tst-mutex9.c index 58c3a1aec263..e43efb80bada 100644 --- a/sysdeps/pthread/tst-mutex9.c +++ b/sysdeps/pthread/tst-mutex9.c @@ -29,6 +29,7 @@ #include #include #include +#include #ifdef ENABLE_PP #include "tst-tpp.h" @@ -144,7 +145,7 @@ do_test (void) do_test_clock (CLOCK_USE_TIMEDLOCK); do_test_clock (CLOCK_REALTIME); -#ifndef ENABLE_PI +#if ! defined(ENABLE_PI) || __ASSUME_FUTEX_LOCK_PI2 do_test_clock (CLOCK_MONOTONIC); #endif return 0; diff --git a/sysdeps/unix/sysv/linux/kernel-features.h b/sysdeps/unix/sysv/linux/kernel-features.h index 1680b10ca1b6..eb5c4d5a04bd 100644 --- a/sysdeps/unix/sysv/linux/kernel-features.h +++ b/sysdeps/unix/sysv/linux/kernel-features.h @@ -218,4 +218,12 @@ # define __ASSUME_FACCESSAT2 0 #endif +/* The FUTEX_LOCK_PI2 operation was introduced across all architectures in Linux + 5.14. */ +#if __LINUX_KERNEL_VERSION >= 0x051400 +# define __ASSUME_FUTEX_LOCK_PI2 1 +#else +# define __ASSUME_FUTEX_LOCK_PI2 0 +#endif + #endif /* kernel-features.h */