From patchwork Fri Jun 25 08:11:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kurt Kanzenbach X-Patchwork-Id: 44012 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 012BF39BF4B3 for ; Fri, 25 Jun 2021 08:12:35 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 012BF39BF4B3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1624608755; bh=FYTOmX+g/Uw3po98kn9GWUvUfReDXVQxsz+NEpkcv9o=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=xFNmLRIWVoTYVzhOuKQg4AYfXxRuac67rTLHikCWMU4iPxDSDgw50q6M2tLiQtE4P nXtNrOnmNDbya67LLiYiHbg2/BuReOPTY97UNiSS1IxTB9qynWejmlYGqGxI7eANQ/ BB52a9XYcshf9hCrYfazv/rqBmEEKUp6QJA2jsDs= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by sourceware.org (Postfix) with ESMTPS id C305C385AC22 for ; Fri, 25 Jun 2021 08:11:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C305C385AC22 To: libc-alpha@sourceware.org Subject: [PATCH v1 2/6] nptl: Introduce futex_lock_pi2() Date: Fri, 25 Jun 2021 10:11:00 +0200 Message-Id: <20210625081104.1134598-3-kurt@linutronix.de> In-Reply-To: <20210625081104.1134598-1-kurt@linutronix.de> References: <20210625081104.1134598-1-kurt@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-10.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kurt Kanzenbach via Libc-alpha From: Kurt Kanzenbach Reply-To: Kurt Kanzenbach Cc: Florian Weimer , Sebastian Andrzej Siewior , Kurt Kanzenbach , Peter Zijlstra , Thomas Gleixner , Joseph Myers Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This variant of futex_lock() has support for selectable clocks and priority inheritance. The underlying FUTEX_LOCK_PI2 operation has been recently introduced into the Linux kernel. It can be used for implementing pthread_mutex_clocklock(MONOTONIC)/PI. The code works like this: * If kernel support is assumed, then use FUTEX_LOCK_PI2 * If not, distuingish between clockid: * For realtime use FUTEX_LOCK_PI * For monotonic try to use FUTEX_LOCK_PI2 which might not be available Signed-off-by: Kurt Kanzenbach --- sysdeps/nptl/futex-internal.h | 131 ++++++++++++++++++++++++++++++ sysdeps/nptl/lowlevellock-futex.h | 1 + 2 files changed, 132 insertions(+) diff --git a/sysdeps/nptl/futex-internal.h b/sysdeps/nptl/futex-internal.h index 79a366604d9e..38c969831276 100644 --- a/sysdeps/nptl/futex-internal.h +++ b/sysdeps/nptl/futex-internal.h @@ -303,6 +303,137 @@ futex_lock_pi64 (int *futex_word, const struct __timespec64 *abstime, } } +/* The operation checks the value of the futex, if the value is 0, then + it is atomically set to the caller's thread ID. If the futex value is + nonzero, it is atomically sets the FUTEX_WAITERS bit, which signals wrt + other futex owner that it cannot unlock the futex in user space by + atomically by setting its value to 0. + + If more than one wait operations is issued, the enqueueing of the waiters + are done in descending priority order. + + The ABSTIME arguments provides an absolute timeout (measured against the + CLOCK_REALTIME or CLOCK_MONOTONIC clock). If TIMEOUT is NULL, the operation + will block indefinitely. + + Returns: + + - 0 if woken by a PI unlock operation or spuriously. + - EAGAIN if the futex owner thread ID is about to exit, but has not yet + handled the state cleanup. + - EDEADLK if the futex is already locked by the caller. + - ESRCH if the thread ID int he futex does not exist. + - EINVAL is the state is corrupted or if there is a waiter on the + futex or if the clockid is invalid. + - ETIMEDOUT if the ABSTIME expires. +*/ +static __always_inline int +futex_lock_pi2_64 (int *futex_word, clockid_t clockid, + const struct __timespec64 *abstime, int private) +{ + unsigned int clockbit; + int err; + + if (! lll_futex_supported_clockid (clockid)) + return EINVAL; + + clockbit = (clockid == CLOCK_REALTIME) ? FUTEX_CLOCK_REALTIME : 0; + int op = __lll_private_flag (FUTEX_LOCK_PI2 | clockbit, private); + + /* FUTEX_LOCK_PI2 is a new futex operation. It supports selectable clocks + whereas the old FUTEX_LOCK_PI does only support CLOCK_REALTIME. + + Therefore, the code works like this: + + - If kernel support is available, then use FUTEX_LOCK_PI2 + - If not, distuingish between clockid: For realtime use FUTEX_LOCK_PI and + for monotonic try to use FUTEX_LOCK_PI2 which might not be available. */ + +#if __ASSUME_FUTEX_LOCK_PI2 + +# ifdef __ASSUME_TIME64_SYSCALLS + err = INTERNAL_SYSCALL_CALL (futex_time64, futex_word, op, 0, abstime); +# else + + bool need_time64 = abstime != NULL && !in_time_t_range (abstime->tv_sec); + if (need_time64) + { + err = INTERNAL_SYSCALL_CALL (futex_time64, futex_word, op, 0, abstime); + if (err == -ENOSYS) + err = -EOVERFLOW; + } + else + { + struct timespec ts32; + + if (abstime != NULL) + ts32 = valid_timespec64_to_timespec (*abstime); + + err = INTERNAL_SYSCALL_CALL (futex, futex_word, op, 0, + abstime != NULL ? &ts32 : NULL); + } +# endif /* __ASSUME_TIME64_SYSCALLS */ + +#else + + /* For CLOCK_MONOTONIC the only option is to use FUTEX_LOCK_PI2 */ + if (abstime != NULL && clockid != CLOCK_REALTIME) + { +# ifdef __ASSUME_TIME64_SYSCALLS + err = INTERNAL_SYSCALL_CALL (futex_time64, futex_word, op, 0, abstime); +# else + bool need_time64 = abstime != NULL && !in_time_t_range (abstime->tv_sec); + if (need_time64) + { + err = INTERNAL_SYSCALL_CALL (futex_time64, futex_word, op, 0, + abstime); + } + else + { + struct timespec ts32; + + if (abstime != NULL) + ts32 = valid_timespec64_to_timespec (*abstime); + + err = INTERNAL_SYSCALL_CALL (futex, futex_word, op, 0, + abstime != NULL ? &ts32 : NULL); + } +# endif /* __ASSUME_TIME64_SYSCALLS */ + + /* FUTEX_LOCK_PI2 is not available on this kernel */ + if (err == -ENOSYS) + return EINVAL; + } + else + { + /* Otherwise use CLOCK_REALTIME and FUTEX_LOCK_PI */ + return futex_lock_pi64 (futex_word, abstime, private); + } +#endif /* __ASSUME_FUTEX_LOCK_PI2 */ + + switch (err) + { + case 0: + case -EAGAIN: + case -EINTR: + case -ETIMEDOUT: + case -ESRCH: + case -EDEADLK: + case -EINVAL: /* This indicates either state corruption or that the kernel + found a waiter on futex address which is waiting via + FUTEX_WAIT or FUTEX_WAIT_BITSET. This is reported on + some futex_lock_pi usage (pthread_mutex_timedlock for + instance). */ + return -err; + + case -EFAULT: /* Must have been caused by a glibc or application bug. */ + case -ENOSYS: /* Must have been caused by a glibc bug. */ + /* No other errors are documented at this time. */ + default: + futex_fatal_error (); + } +} + /* Wakes the top priority waiter that called a futex_lock_pi operation on the futex. diff --git a/sysdeps/nptl/lowlevellock-futex.h b/sysdeps/nptl/lowlevellock-futex.h index 66ebfe50f4c1..abda179e0de2 100644 --- a/sysdeps/nptl/lowlevellock-futex.h +++ b/sysdeps/nptl/lowlevellock-futex.h @@ -38,6 +38,7 @@ #define FUTEX_WAKE_BITSET 10 #define FUTEX_WAIT_REQUEUE_PI 11 #define FUTEX_CMP_REQUEUE_PI 12 +#define FUTEX_LOCK_PI2 13 #define FUTEX_PRIVATE_FLAG 128 #define FUTEX_CLOCK_REALTIME 256