From patchwork Tue Feb 27 22:56:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 86492 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A80863858427 for ; Tue, 27 Feb 2024 22:58:41 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by sourceware.org (Postfix) with ESMTPS id 48C223858C62 for ; Tue, 27 Feb 2024 22:56:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 48C223858C62 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 48C223858C62 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::636 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074620; cv=none; b=wGeBsi52LAmRMyyiiUQpCo2El6BIyFRvwjH8lj09olLDTqWxty1MQSH1//f77m3Kb1uo0M2pG3i5xOM20pEbxXR7BWggHClUyy3dZYNkibwueoWMLYSv4MNDuf4R6iRHG5IB1nd0Ra7GsGb731uBBpCYH/JV2zFwohoYTS7IJuQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074620; c=relaxed/simple; bh=pg+j8ziAC36Xq0WYFbJ7JVZVlHJ99Y0uF+htEzmsScQ=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=wKWaxQ9S17+PJX2Idd40gqRDH88qsdGHbhCwaaJNmnV1vhb9XgjS2zTkmO4hgoGua9idv/6QhLXgmBcPbOF/LnHRZqYRAyWpuEL1XxpU5hD/dSwJimUHfq1o1nEpSPqXjGm3b9oP0mMLS/kkM77MgEQAsg7VwvHjnwg4j6a1A9I= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1d944e8f367so35591005ad.0 for ; Tue, 27 Feb 2024 14:56:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709074615; x=1709679415; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9tQdci6eo46FdgeE282njQbvkkZcix77Voudga3zlGs=; b=Iar+LvHdimVzJmNAFlxyNeIIx7t0l5JAtf2S4WzAFmAzZfM/kfurhwOTQl35gzL68i GEUKhpfkfTe3zlV6iDwfQJaY2VTvIrbsRX0qoguDcd2jLtGiqqo6S5VMbjmQc1HO/g/d h7r8eR2UCdhRiP3C3YQMFRcv+ED3Tq/5wV1ACpcBVI+3P8JvtZTRWnzyNCpC61a24Dch FV1/TVVKi+wtMq8/yE6SnxlcTd2iOkTR7Xq9nKU3c8/4NWPa6dSjZrmBQS3QhsQQxktj rvcCBqUONS4jEwA07tw0wYQyjyNigwm1ruzgGY/Spm9ASGK31Azo6JF6fy5ulxuJmejy 7L4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709074615; x=1709679415; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9tQdci6eo46FdgeE282njQbvkkZcix77Voudga3zlGs=; b=V0SpPgSrjsdlL2Zxs/OTbu6FCXSBkXHPsWOciNRByNDxx2hBOZJib0q/0qA+7FJPpB wPk8G8/f3Ir0J/V2v4Ry8h6XHkzO0CjisAeURRxoaAFMLEqzMP07JYeRkOKjiBe5sG/J 1LR2IaMlzdSqt52BUWkmQCS2QKhIWSVCtOdpTlNJtT6fvOzhdK3VwT3x0hzKfs9POmI5 oacqOyd8zNBAhyCDM6zUxEPQPrXAYrhNkM1mVPSvVMYZ5c0/zh1eTqYCiZUEn6Csq1dg 1qWmVQt+lbyyfGLtmAzI8Kk7TIYSBeOuLudLn17WQlirwFUnvxvtymGNyOqJhTSkjJnB MGeA== X-Gm-Message-State: AOJu0Yy1j1FZcQd15NenlgefKkwUgi6fuftkRCLlf3VqX1Qv4qrBUALE c6UWySugcuPoSDp+gBmKefIVkjq1rmU8CDqDixURiNLZ6ISVdPRUXxWv/264XkwS7ff/Ct8Uc0x a X-Google-Smtp-Source: AGHT+IFhSd0oC7mwCSUj1hnqU7d3mQSbUj0gFY7Al4vL5mdFeK7AfyMnNUnqxr/qRExxC7QMvCtVAw== X-Received: by 2002:a17:902:cf0d:b0:1dc:affd:6510 with SMTP id i13-20020a170902cf0d00b001dcaffd6510mr6350232plg.49.1709074615341; Tue, 27 Feb 2024 14:56:55 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id li3-20020a170903294300b001dc90ac1cecsm2029225plb.284.2024.02.27.14.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 14:56:54 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, Florian Weimer , palmer@rivosinc.com, Evan Green Subject: [PATCH v13 1/7] riscv: Add Linux hwprobe syscall support Date: Tue, 27 Feb 2024 14:56:37 -0800 Message-Id: <20240227225644.724901-2-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227225644.724901-1-evan@rivosinc.com> References: <20240227225644.724901-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add awareness and a thin wrapper function around a new Linux system call that allows callers to get architecture and microarchitecture information about the CPUs from the kernel. This can be used to do things like dynamically choose a memcpy implementation. Signed-off-by: Evan Green --- (no changes since v12) Changes in v12: - Updated version to 2.40 Changes in v11: - Update copyright year (Adhemerval) Changes in v10: - Remove spurious 5 from syscall patch (Adhemerval) - Use one item per line in Makefile (Adhemerval) - Remove double underscores from __riscv_hwprobe definition (Adhemerval) - Use only spaces in macro definitions of hwprobe.h (Adhemerval) Changes in v9: - Alphabetize new entries in libc.abilist (to pass checks) Changes in v8: - Fix missed 2.39 in abilists (Joseph) - Just return -r (Florian) Changes in v7: - Bumped Versions up to 2.39 (Joseph) - Used INTERNAL_SYSCALL_CALL, and return positive errno to match pthreads API (Florian). - Remove __THROW since it creates a warning in combination with the fortified access decorators. Changes in v6: - Prefixed __riscv_hwprobe() parameters names with __ to avoid user macro namespace pollution (Joseph) Changes in v4: - Remove __USE_GNU (Florian) - __nonnull, __wur, __THROW, and __fortified_attr_access decorations (Florian) - change long to long int (Florian) - Fix comment formatting (Florian) - Update backup kernel header content copy. - Fix function declaration formatting (Florian) - Changed export versions to 2.38 Changes in v3: - Update argument types to match v4 kernel interface Changes in v2: - hwprobe.h: Use __has_include and duplicate Linux content to make compilation work when Linux headers are absent (Adhemerval) - hwprobe.h: Put declaration under __USE_GNU (Adhemerval) - Use INLINE_SYSCALL_CALL (Adhemerval) - Update versions - Update UNALIGNED_MASK to match kernel v3 series. sysdeps/unix/sysv/linux/riscv/Makefile | 12 ++- sysdeps/unix/sysv/linux/riscv/Versions | 3 + sysdeps/unix/sysv/linux/riscv/hwprobe.c | 36 +++++++++ .../unix/sysv/linux/riscv/rv32/libc.abilist | 1 + .../unix/sysv/linux/riscv/rv64/libc.abilist | 1 + sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 74 +++++++++++++++++++ 6 files changed, 125 insertions(+), 2 deletions(-) create mode 100644 sysdeps/unix/sysv/linux/riscv/hwprobe.c create mode 100644 sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h diff --git a/sysdeps/unix/sysv/linux/riscv/Makefile b/sysdeps/unix/sysv/linux/riscv/Makefile index 4b6eacb32f..04abf226ad 100644 --- a/sysdeps/unix/sysv/linux/riscv/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/Makefile @@ -1,6 +1,14 @@ ifeq ($(subdir),misc) -sysdep_headers += sys/cachectl.h -sysdep_routines += flush-icache +sysdep_headers += \ + sys/cachectl.h \ + sys/hwprobe.h \ + # sysdep_headers + +sysdep_routines += \ + flush-icache \ + hwprobe \ + # sysdep_routines + endif ifeq ($(subdir),stdlib) diff --git a/sysdeps/unix/sysv/linux/riscv/Versions b/sysdeps/unix/sysv/linux/riscv/Versions index 5625d2a0b8..8183e9feb6 100644 --- a/sysdeps/unix/sysv/linux/riscv/Versions +++ b/sysdeps/unix/sysv/linux/riscv/Versions @@ -8,4 +8,7 @@ libc { GLIBC_2.27 { __riscv_flush_icache; } + GLIBC_2.40 { + __riscv_hwprobe; + } } diff --git a/sysdeps/unix/sysv/linux/riscv/hwprobe.c b/sysdeps/unix/sysv/linux/riscv/hwprobe.c new file mode 100644 index 0000000000..28a448175b --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/hwprobe.c @@ -0,0 +1,36 @@ +/* RISC-V hardware feature probing support on Linux + Copyright (C) 2024 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include + +int __riscv_hwprobe (struct riscv_hwprobe *pairs, size_t pair_count, + size_t cpu_count, unsigned long int *cpus, + unsigned int flags) +{ + int r; + + r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, pairs, pair_count, + cpu_count, cpus, flags); + + /* Negate negative errno values to match pthreads API. */ + return -r; +} diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist index f90c94bc35..6397a9cb91 100644 --- a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist +++ b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist @@ -2511,3 +2511,4 @@ GLIBC_2.39 stdc_trailing_zeros_ui F GLIBC_2.39 stdc_trailing_zeros_ul F GLIBC_2.39 stdc_trailing_zeros_ull F GLIBC_2.39 stdc_trailing_zeros_us F +GLIBC_2.40 __riscv_hwprobe F diff --git a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist index e04ff93bd2..71bbf94f66 100644 --- a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist +++ b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist @@ -2711,3 +2711,4 @@ GLIBC_2.39 stdc_trailing_zeros_ui F GLIBC_2.39 stdc_trailing_zeros_ul F GLIBC_2.39 stdc_trailing_zeros_ull F GLIBC_2.39 stdc_trailing_zeros_us F +GLIBC_2.40 __riscv_hwprobe F diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h new file mode 100644 index 0000000000..5592b9e100 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -0,0 +1,74 @@ +/* RISC-V architecture probe interface + Copyright (C) 2024 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#ifndef _SYS_HWPROBE_H +#define _SYS_HWPROBE_H 1 + +#include +#include +#ifdef __has_include +# if __has_include () +# include +# endif +#endif + +/* Define a (probably stale) version of the interface if the Linux headers + aren't present. */ +#ifndef RISCV_HWPROBE_KEY_MVENDORID +struct riscv_hwprobe { + signed long long int key; + unsigned long long int value; +}; + +#define RISCV_HWPROBE_KEY_MVENDORID 0 +#define RISCV_HWPROBE_KEY_MARCHID 1 +#define RISCV_HWPROBE_KEY_MIMPID 2 +#define RISCV_HWPROBE_KEY_BASE_BEHAVIOR 3 +#define RISCV_HWPROBE_BASE_BEHAVIOR_IMA (1 << 0) +#define RISCV_HWPROBE_KEY_IMA_EXT_0 4 +#define RISCV_HWPROBE_IMA_FD (1 << 0) +#define RISCV_HWPROBE_IMA_C (1 << 1) +#define RISCV_HWPROBE_IMA_V (1 << 2) +#define RISCV_HWPROBE_EXT_ZBA (1 << 3) +#define RISCV_HWPROBE_EXT_ZBB (1 << 4) +#define RISCV_HWPROBE_EXT_ZBS (1 << 5) +#define RISCV_HWPROBE_EXT_ZICBOZ (1 << 6) +#define RISCV_HWPROBE_KEY_CPUPERF_0 5 +#define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) +#define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) +#define RISCV_HWPROBE_MISALIGNED_SLOW (2 << 0) +#define RISCV_HWPROBE_MISALIGNED_FAST (3 << 0) +#define RISCV_HWPROBE_MISALIGNED_UNSUPPORTED (4 << 0) +#define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) +#define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 + +#endif /* RISCV_HWPROBE_KEY_MVENDORID */ + +__BEGIN_DECLS + +extern int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) + __nonnull ((1)) __wur + __fortified_attr_access (__read_write__, 1, 2) + __fortified_attr_access (__read_only__, 4, 3); + +__END_DECLS + +#endif /* sys/hwprobe.h */ From patchwork Tue Feb 27 22:56:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 86495 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9C4373858C32 for ; Tue, 27 Feb 2024 22:59:31 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by sourceware.org (Postfix) with ESMTPS id D1E163858C31 for ; Tue, 27 Feb 2024 22:56:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D1E163858C31 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D1E163858C31 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::631 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074621; cv=none; b=la2pENQV02L/ckKPRA4lXx1rDhye5QIpb1a8yGdVmrEN9WJZgQRY5flJln2JV8urRpDCwS7jbnAQywY88JxXRpjFgTOhYNZDLuc8E0yLP7MUyl101L4j0lq/+Fw4whwAOoqjq0vfmZvBPJNVXFzZOnb4icy9anQmyBPpa58ugQs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074621; c=relaxed/simple; bh=lFhTdBTh/bHcRa5gvCPseMIeoQab7VvAwqqDEuifkBM=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=IEqC3035doBulqDq43qBBJRygCCVfgEsloa/aHytkWW8nvMLrcaWIGK+h2jaNcm3YgnKcAh4uqZ0T1o68l30O+7+So3cN0jYuvIvrjZyDdwDrkHMJxqTPBemZ2KyPOfNNxvNMDf5RQQkO+zlqQq3+3Un+m+JpJ+bxxcw0RH0inY= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1dc1ff58fe4so43211685ad.1 for ; Tue, 27 Feb 2024 14:56:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709074618; x=1709679418; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+d7OlNenA73M5nIUk0dJy0TaP9BcEP0MKokwJIb+yRA=; b=Ls+UDPwcJrqO6mg8QyqndmrBhA0Dqn6cYQNfb6ZmXPHq6UUQ6YaaM7q5+u26YMjqzY j2xBimL/GmxoI7AoIPQayMmbiDCCd3OAAL+JHVSpnAmGortzNWbOu7/iwTvym05cPwA1 rA6JPqxohhlN1YzZk7qipu/UJICD0WX8LTfWj7PPz23ahyzK8kOg89dSfvDFElOjBaPo VDQOlZ6HO6L25gsBRFITNzbd1O699dwiJTE6uzFi8XhMubv7bf2S2pOfGFXwvBCSoz4l M6MgHjkaoeQIaFMpiLAIUqrFa/knyPs3vyjoSycldeouczkWXad37pLwbXdz2dBtQRFO Jllg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709074618; x=1709679418; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+d7OlNenA73M5nIUk0dJy0TaP9BcEP0MKokwJIb+yRA=; b=axOWtyCNxobHzZ4ZISHiHnA4AvORY86GDWOIJvIiZVZwNIoN2ycDfxwRE87fZ+fIn+ 8Inot0+x1i5Qj5OBQnkpiUFOfELPgUuiJmS4cbaE42kO+pvB896r/LohONjFn63TvteM P3EpkCLzqI7KRuCzr/OFbYmT0dwaIq07x+N9UaWSAVvwUbdtHGjTh/i75ZOkPB09kaDo 4VJCueSBvcxVk2FC2dsequ3JLFg3ydK8c6/EZrizPxjoFZCMw7NHmLJzW4TyvIL4oDrs oJfc2iioObg/a3IGmfm7KxixDlUu2fL2JHRrFkIy3EXQbSH+sfT8QM4vtC36hqsoP1FF VL9w== X-Gm-Message-State: AOJu0YwLvpm1GgufxL0OOZ2Fp6aqLADJ6+i/VMD5k50QGYe0cqjPbIEf dLK0sYPzQ+XgBfTHw1IxTE2DY0oLm/EaJrcGEVXlNT8DFISqAL9rTiI2aNeDvfE2AGbMWBoxPUa u X-Google-Smtp-Source: AGHT+IHab3AnHozMQgcZHQ1q5K8au9pZ7V+y9ezxYNxG41adsKsSE2x8VdSz4orVrkVTeaZlVhxqjQ== X-Received: by 2002:a17:902:b198:b0:1dc:a837:7e1b with SMTP id s24-20020a170902b19800b001dca8377e1bmr6235102plr.63.1709074617995; Tue, 27 Feb 2024 14:56:57 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id li3-20020a170903294300b001dc90ac1cecsm2029225plb.284.2024.02.27.14.56.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 14:56:57 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, Florian Weimer , palmer@rivosinc.com, Evan Green Subject: [PATCH v13 2/7] linux: Introduce INTERNAL_VSYSCALL Date: Tue, 27 Feb 2024 14:56:38 -0800 Message-Id: <20240227225644.724901-3-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227225644.724901-1-evan@rivosinc.com> References: <20240227225644.724901-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add an INTERNAL_VSYSCALL() macro that makes a vDSO call, falling back to a regular syscall, but without setting errno. Instead, the return value is plumbed straight out of the macro. Signed-off-by: Evan Green --- Changes in v13: - Remove label, use -ENOSYS and return value directly (Florian) Changes in v10: - Introduced INTERNAL_VSYSCALL patch sysdeps/unix/sysv/linux/sysdep-vdso.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/sysdeps/unix/sysv/linux/sysdep-vdso.h b/sysdeps/unix/sysv/linux/sysdep-vdso.h index 189319ad98..2f53ada6e5 100644 --- a/sysdeps/unix/sysv/linux/sysdep-vdso.h +++ b/sysdeps/unix/sysv/linux/sysdep-vdso.h @@ -53,4 +53,16 @@ sc_ret; \ }) +#define INTERNAL_VSYSCALL(name, nr, args...) \ + ({ \ + long int sc_ret = -ENOSYS; \ + \ + __typeof (GLRO(dl_vdso_##name)) vdsop = GLRO(dl_vdso_##name); \ + if (vdsop != NULL) \ + sc_ret = INTERNAL_VSYSCALL_CALL (vdsop, nr, ##args); \ + if (sc_ret == -ENOSYS) \ + sc_ret = INTERNAL_SYSCALL_CALL (name, ##args); \ + sc_ret; \ + }) + #endif /* SYSDEP_VDSO_LINUX_H */ From patchwork Tue Feb 27 22:56:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 86489 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 4F0433858293 for ; Tue, 27 Feb 2024 22:58:06 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by sourceware.org (Postfix) with ESMTPS id DC0643858402 for ; Tue, 27 Feb 2024 22:57:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org DC0643858402 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org DC0643858402 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::635 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074623; cv=none; b=GVHDeyB6PFbn/5Aq3CmOrDxKMoybi+PDRZe8ozNWo4zCX0lLCwzgZV3eF1lkwxr+wulecMtDlGn8ob/N69CRE+nJywv+G+ZmmKFW7NB7c/UVKQ4yJq76MvLhuc/9byS+80DSPWYfY8PxRJtkNcK6bqYZ3pYidiRYQZ4f2371XN4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074623; c=relaxed/simple; bh=mB2uCA7GFf8IcHf8n94d1wmGmYoiXZHMQQ84vEGbFaw=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=rOrQmDnO0VuWp8tUuNiV6Ku5Jp0j174Px6OrDbTSxowRdynsoiusoFQXJ4RyfSXlmkK2mH1Tk2jIciRVaUqgBFYaZFNCcMhak5rZNUjzPYD0GKpJt+QXIjYKKMtmM7tuotr2gHPmldLt+6urJQzOIG3cTyeDc/dbXNCsFuEsH7E= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1d7232dcb3eso35157385ad.2 for ; Tue, 27 Feb 2024 14:57:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709074620; x=1709679420; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N3idKZzOv0gJ2EL0bSki58RkN41BRlAo5gY4kORw7kQ=; b=NALCA1jSCFaoSDhB6/NA7kiQ+kKAun/UnyCcUT2+PoseHuoFn4iKp2S3Ou6ysR6v3U 7WHKMAr0LNzL5oudgJqV6bvxPxoV8oaE7mzKwADnWnA/g0E9x+BsBQFDfhNGDlbsRrFn Qja4EApUPXi/db3MITCcAGBixli4Kpk41Bgp6jE1rxxos9VOdpr/GjumMS3MFZx05Z/u jFeWGew/RKEYn70mCdMvJCTEsrBrJCarDYbdgNj5tTrEzQRCbNpJsATJSTKcKvV9QXHW eI6YL/Z+8swigqaz/Enpr0sZfT4x8e5WOhgs1SEpiRi+1CY7SDzfQLa5QkDvgcrlw/gN yhew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709074620; x=1709679420; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N3idKZzOv0gJ2EL0bSki58RkN41BRlAo5gY4kORw7kQ=; b=ouM10hXpFZy0jAu3pPHClfwb9kZ5bNGUqLQW589ALoAkfSQCig4CLye7EjsxgIr07f 7mkgjPCSo6WrAULb8EX8DQHCkIjJvrj+dXSEaIoQ6ckxDJlGeKJNNLkUJRLXkjq+h2Xt A3Y4gI9BX7nnKX8mjo5K51pZstbcUMPTQbTC6DA6HIZrT7Da2tX010K5Ch1SJj+ILsji mLzKvJAcswU4Cn2b6mtE9PO2hV1wRcCNN3VpL8TIN/wzYfk70BHsfH4EsEQPOObHet+9 0FrJ77gWmkuwXMYR/uUkwj5zzAQd3i9kynsCmuLioh/kHVDrHzzc8c2n2qJ+Y4jlOSQw UmEg== X-Gm-Message-State: AOJu0YyWZxmkUppNiL0Dved95Wis2GcRfCTg0ED8W8lkYAIrSIDcYF6k 9s3EzvOoY002rGTl197Mt3siNBzHJPI6s5L0fRmx0RltvBRZjFD8YRR5drSse45unZ5tc6joAXq x X-Google-Smtp-Source: AGHT+IHvj6ig/+mstIgH7vuuQ5qLRAum+4g5nNMQ76w4saDpwbQ4YnVW/eMY1L0fs92/JrEqoLzU5A== X-Received: by 2002:a17:903:22cb:b0:1dc:4a90:b87d with SMTP id y11-20020a17090322cb00b001dc4a90b87dmr12431954plg.66.1709074620006; Tue, 27 Feb 2024 14:57:00 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id li3-20020a170903294300b001dc90ac1cecsm2029225plb.284.2024.02.27.14.56.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 14:56:59 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, Florian Weimer , palmer@rivosinc.com, Evan Green Subject: [PATCH v13 3/7] riscv: Add hwprobe vdso call support Date: Tue, 27 Feb 2024 14:56:39 -0800 Message-Id: <20240227225644.724901-4-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227225644.724901-1-evan@rivosinc.com> References: <20240227225644.724901-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The new riscv_hwprobe syscall also comes with a vDSO for faster answers to your most common questions. Call in today to speak with a kernel representative near you! Signed-off-by: Evan Green --- (no changes since v10) Changes in v10: - Remove leading underscores in definition (Adhemerval) - Remove spurious 5 from INTERNAL_SYSCALL_CALL (Adhemerval) - Use new INTERNAL_VSYSCALL macro instead (Adhemerval) Changes in v7: - Use INTERNAL_VSYSCALL_CALL (Florian) Changes in v3: - Add the "return" to the vsyscall - Fix up vdso arg types to match kernel v4 version - Remove ifdef around INLINE_VSYSCALL (Adhemerval) Changes in v2: - Add vDSO interface sysdeps/unix/sysv/linux/dl-vdso-setup.c | 10 ++++++++++ sysdeps/unix/sysv/linux/dl-vdso-setup.h | 3 +++ sysdeps/unix/sysv/linux/riscv/hwprobe.c | 4 ++-- sysdeps/unix/sysv/linux/riscv/sysdep.h | 1 + 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/sysdeps/unix/sysv/linux/dl-vdso-setup.c b/sysdeps/unix/sysv/linux/dl-vdso-setup.c index 5dd7ed9d12..3a44944dbb 100644 --- a/sysdeps/unix/sysv/linux/dl-vdso-setup.c +++ b/sysdeps/unix/sysv/linux/dl-vdso-setup.c @@ -71,6 +71,16 @@ PROCINFO_CLASS int (*_dl_vdso_clock_getres_time64) (clockid_t, # ifdef HAVE_GET_TBFREQ PROCINFO_CLASS uint64_t (*_dl_vdso_get_tbfreq)(void) RELRO; # endif + +/* RISC-V specific ones. */ +# ifdef HAVE_RISCV_HWPROBE +PROCINFO_CLASS int (*_dl_vdso_riscv_hwprobe)(void *, + size_t, + size_t, + unsigned long *, + unsigned int) RELRO; +# endif + #endif #undef RELRO diff --git a/sysdeps/unix/sysv/linux/dl-vdso-setup.h b/sysdeps/unix/sysv/linux/dl-vdso-setup.h index e87d886940..8aee5a8212 100644 --- a/sysdeps/unix/sysv/linux/dl-vdso-setup.h +++ b/sysdeps/unix/sysv/linux/dl-vdso-setup.h @@ -47,6 +47,9 @@ setup_vdso_pointers (void) #ifdef HAVE_GET_TBFREQ GLRO(dl_vdso_get_tbfreq) = dl_vdso_vsym (HAVE_GET_TBFREQ); #endif +#ifdef HAVE_RISCV_HWPROBE + GLRO(dl_vdso_riscv_hwprobe) = dl_vdso_vsym (HAVE_RISCV_HWPROBE); +#endif } #endif diff --git a/sysdeps/unix/sysv/linux/riscv/hwprobe.c b/sysdeps/unix/sysv/linux/riscv/hwprobe.c index 28a448175b..e64c159eb3 100644 --- a/sysdeps/unix/sysv/linux/riscv/hwprobe.c +++ b/sysdeps/unix/sysv/linux/riscv/hwprobe.c @@ -28,8 +28,8 @@ int __riscv_hwprobe (struct riscv_hwprobe *pairs, size_t pair_count, { int r; - r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, pairs, pair_count, - cpu_count, cpus, flags); + r = INTERNAL_VSYSCALL (riscv_hwprobe, 5, pairs, pair_count, + cpu_count, cpus, flags); /* Negate negative errno values to match pthreads API. */ return -r; diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h index 5583b96d23..ee015dfeb6 100644 --- a/sysdeps/unix/sysv/linux/riscv/sysdep.h +++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h @@ -156,6 +156,7 @@ /* List of system calls which are supported as vsyscalls (for RV32 and RV64). */ # define HAVE_GETCPU_VSYSCALL "__vdso_getcpu" +# define HAVE_RISCV_HWPROBE "__vdso_riscv_hwprobe" # undef HAVE_INTERNAL_BRK_ADDR_SYMBOL # define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1 From patchwork Tue Feb 27 22:56:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 86493 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6CE153858412 for ; Tue, 27 Feb 2024 22:58:56 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by sourceware.org (Postfix) with ESMTPS id 0FC19385840A for ; Tue, 27 Feb 2024 22:57:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0FC19385840A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0FC19385840A Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::431 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074625; cv=none; b=QE+4TEHLNXWGOHzBe5+DhSuqKOobyS46UxjRPP4kcxJ7ODJNoBWP8mWw+jJ67DbYZnp6TGZzkZ8+wnv6W/1xf1whCJSOqwRUO8DaWW9ACQ4Cxnd08N462M2auxq6hOKe7UTYL2SZ+cheZO4+8o5msLh//8+K13XJcAcaId8X1Io= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074625; c=relaxed/simple; bh=3KEb2aY1qvkZ9DfZSgGgFnctwp76uRNqePMZFCzhuPg=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=M+Laf2eTFbAjmCVkTzmLu+/MFSyuwh5W2wIesbJOXipSC0whrDVg/JHFDFlhlkTBEFcHPMvA57OGxOCJLnw3asN9pG7wiIAfLb1170zZtsLu8CqlpfB45rLKhMSnjsp8BLksiBRWxS0aygSp58GZUSewRU+XjiukTum0r9+naiQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-6e459b39e2cso212998b3a.1 for ; Tue, 27 Feb 2024 14:57:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709074622; x=1709679422; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wVONlSWH7rFsPXRa1lJYOvQx3bPsSIZvkL8AKiDA2+0=; b=oIyzeU6MEQa4tMs/JkfkEQy/x/1fLPkap7KTj4eI5nrl/PUZdrLqx5MH1h81N9rkYH MdHRHDsPK8cfW5j3qse+AbUCDX+6/EcCD0ZoPAuv1db6g81hj5KCenxlt/QieHyot/C3 v+f3g+yFnicE9zf3WuSMjUZKbaubzz1ifQ4lFnQoWbRvFGajlyQEC7EqBc3nv5zVHk0p AKiwV3ujMT2GrCIdHw+uiGyzN2x0Mpt5Oo0hZURlKuSB8JBQQmCuk6e61Wp9aOsh/f5n SxSiS1mU+22SPLjY9nneRHj0bT2hoREDm/zKX/QVZXpRrMbohE56N13lsc84w6JYo60e v7wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709074622; x=1709679422; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wVONlSWH7rFsPXRa1lJYOvQx3bPsSIZvkL8AKiDA2+0=; b=YAn5gtt2NKfYJlZaV82I9Q2JOhTkSUTM1kZeIFKcTa6RK39f4CoWxVfoplAOT/jrSb r7TwYtVxIAAJgwawt8OjVm8Rhnlo/aUIYIVX5yN6OwAraq6nCbU+fLI3ezkhopLV7wA8 HG1jhTnNFzeIEuYnq/gbq5IiF8sPdi3Hx7NoIQt3u5U8WLiUGpuzsjpdG+faRj+RyEoA MQ4IR0Mc0oJE5yVlR1i9+FpcIdvqGFHhFxzI9odg0WB8qC/v1qJ26ZOBWX6Le0lSarXD AYnerGhahIMTJZ5CNwAjw+GVrdezKAwrixzgW1GHYAFx75hcRC+7RiHi+Avs0FaGpsPA gJQg== X-Gm-Message-State: AOJu0YyZd90w0RpYfd9iHF8ughHpiCt4ykX4FOmrlA88/e3qooXbx54F MLh/lG6rN8vmXBai8wygJEHJUC1CItHfxhWYOkAufisQMzl7r53zWgFSDRMTH/JFptGct9n6oYp g X-Google-Smtp-Source: AGHT+IGw8/nH1GeXOnlfrmo5Q0Re3CzcwFQfz2HD4H6zZrRjT7pHzkwPRKGaoMKLwFrO19Kg94E/5w== X-Received: by 2002:a17:902:6544:b0:1dc:abe9:8270 with SMTP id d4-20020a170902654400b001dcabe98270mr708369pln.26.1709074621892; Tue, 27 Feb 2024 14:57:01 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id li3-20020a170903294300b001dc90ac1cecsm2029225plb.284.2024.02.27.14.57.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 14:57:01 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, Florian Weimer , palmer@rivosinc.com, Evan Green Subject: [PATCH v13 4/7] riscv: Add __riscv_hwprobe pointer to ifunc calls Date: Tue, 27 Feb 2024 14:56:40 -0800 Message-Id: <20240227225644.724901-5-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227225644.724901-1-evan@rivosinc.com> References: <20240227225644.724901-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The new __riscv_hwprobe() function is designed to be used by ifunc selector functions. This presents a challenge for applications and libraries, as ifunc selectors are invoked before all relocations have been performed, so an external call to __riscv_hwprobe() from an ifunc selector won't work. To address this, pass a pointer to the __riscv_hwprobe() function into ifunc selectors as the second argument (alongside dl_hwcap, which was already being passed). Include a typedef as well for convenience, so that ifunc users don't have to go through contortions to call this routine. Users will need to remember to check the second argument for NULL, to account for older glibcs that don't pass the function. Signed-off-by: Evan Green --- (no changes since v10) Changes in v10: - Hand in pointer to __riscv_hwprobe(), not vDSO function Changes in v7: - Remove __THROW from function pointer type, as it creates warnings together with __fortified_attr_access. sysdeps/riscv/dl-irel.h | 9 +++++---- sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 10 ++++++++++ 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/sysdeps/riscv/dl-irel.h b/sysdeps/riscv/dl-irel.h index e6ab51ccd4..61b3511c96 100644 --- a/sysdeps/riscv/dl-irel.h +++ b/sysdeps/riscv/dl-irel.h @@ -24,6 +24,7 @@ #include #include #include +#include #define ELF_MACHINE_IRELA 1 @@ -31,10 +32,10 @@ static inline ElfW(Addr) __attribute ((always_inline)) elf_ifunc_invoke (ElfW(Addr) addr) { - /* The second argument is a void pointer to preserve the extension - fexibility. */ - return ((ElfW(Addr) (*) (uint64_t, void *)) (addr)) - (GLRO(dl_hwcap), NULL); + /* The third argument is a void pointer to preserve the extension + flexibility. */ + return ((ElfW(Addr) (*) (uint64_t, void *, void *)) (addr)) + (GLRO(dl_hwcap), __riscv_hwprobe, NULL); } static inline void diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h index 5592b9e100..34a2e3dbc2 100644 --- a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -69,6 +69,16 @@ extern int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, __fortified_attr_access (__read_write__, 1, 2) __fortified_attr_access (__read_only__, 4, 3); +/* A pointer to the __riscv_hwprobe vDSO function is passed as the second + argument to ifunc selector routines. Include a function pointer type for + convenience in calling the function in those settings. */ +typedef int (*__riscv_hwprobe_t) (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) + __nonnull ((1)) __wur + __fortified_attr_access (__read_write__, 1, 2) + __fortified_attr_access (__read_only__, 4, 3); + __END_DECLS #endif /* sys/hwprobe.h */ From patchwork Tue Feb 27 22:56:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 86490 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5295138582A5 for ; Tue, 27 Feb 2024 22:58:19 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by sourceware.org (Postfix) with ESMTPS id D6E903858C5E for ; Tue, 27 Feb 2024 22:57:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D6E903858C5E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D6E903858C5E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::529 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074628; cv=none; b=kb4hOG92do1kSEaHVGk+n7p4gdeRhnr6olZVXUXCXCfKx0PBcXzbqfz7VDWi4J/FgdRLeNQdi2S3Mmz6GC4m3cGR2ewUvvqolnUDSR7a3FK4GLXrYUuh/IAUv5+n/oA/RHSnBmgWXdNWkknjEc272++beYZx5SqaevbbxpVKB4A= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074628; c=relaxed/simple; bh=He0coedeXxVwBEBhThbFqG4Wfscxa1R8DG1zTdGQgSw=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=W2Ll34OPjsWVFGJPBp/uLaMLENMmo4IjK3xwXkvvaCjSxWUihp8/B5aevKdrEoB9imD4zanmhTzmZthhgAsQWxuEv49rYSVAJD55igdlx/zzvcE2r7OFrmoY6uf1szmtdqnM4UiVNaInXyQLRHDP7Nn+1z1oaukVztb5u+nFWQY= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-5ce07cf1e5dso3904005a12.2 for ; Tue, 27 Feb 2024 14:57:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709074624; x=1709679424; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t5YFWj+0XZdGB3LIInvIM7u+IB5PiOZRs0l/f1Qnpes=; b=WN5o5Muv2zxRUgb0g2v3BLFaQu/BBzgC/6V7BKFQW8FJhdjYPFh9WiXiuoCqJkfbHL SIeb8e6/yH/YGzH3YohgRtSmcwr7ifgh2NVGhata5BeSg3MA7Wt0qdnhR1clAL4oFy1b a1DtKWHVIwe6HjnIqo3MOcCDUxGtm34Pk6j3IqM/8hKWFBQT1ALWV6uQ1ce7bioXNXNW UJyLn8EexGf+kx1jtPOd5SvhNc+2sEFLbCH6TPmbLmHe2K/ArTtJi1ZSf3dxHytXBoi/ GxGCsRTd7YBq9j418cS+Zm1tlrMSj2d2vomHlw21YY+/aBFGyGNg8tGzUz6vG/u0c913 smrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709074624; x=1709679424; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t5YFWj+0XZdGB3LIInvIM7u+IB5PiOZRs0l/f1Qnpes=; b=I/f0dD0x/1rrLZCnRXQ9VQQoDP0ol5gJWYIOhKTaYcZL/HnFO0hOyjT8+CGV634DZp tpdJxE0Shp1HNy2tpo4xJaHW3+acwEJ5fHdjRuhvINRsyynZhQizs1YiH9tLCfwA20f5 +Ce1mnIRtRhIVDevvAkNgQt1CwR3XmJ8OFrFyS6m8HDFljT2p2yokPVikfYlx3gkixCe mMOS06vNS/o17Fg6jktsicPyViSmjmN73iRxJS9GV4tqw9T4RboovNs/bUmBTFHgvWMe glJPGmvJwrqTicQaTswF2DNQrljedIeRnL39XyXY27yKqCpsB/Rc+tQC6foR5ThSobEU pwMg== X-Gm-Message-State: AOJu0YxBOKQTNW8T1AGY3aBQ88TShsGK/hsHUAWfqZ+mwa5E6Is9329M LrIYplY85i/zcvzwfpwc2Nu2zQrjjVQleybjtgCRVy7YxvQxCYZzgwn1Idkx9dEVPbgFjhnT+HW 2 X-Google-Smtp-Source: AGHT+IE7wxqPsIEBldZDE3R954rKaCALbzqKe8BGDUZ2rvAVV69YYTwp6p/635PClQU5iQf4p9mlCw== X-Received: by 2002:a05:6a21:31c7:b0:1a0:e4a3:582c with SMTP id zb7-20020a056a2131c700b001a0e4a3582cmr3801386pzb.2.1709074623900; Tue, 27 Feb 2024 14:57:03 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id li3-20020a170903294300b001dc90ac1cecsm2029225plb.284.2024.02.27.14.57.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 14:57:03 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, Florian Weimer , palmer@rivosinc.com, Evan Green Subject: [PATCH v13 5/7] riscv: Enable multi-arg ifunc resolvers Date: Tue, 27 Feb 2024 14:56:41 -0800 Message-Id: <20240227225644.724901-6-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227225644.724901-1-evan@rivosinc.com> References: <20240227225644.724901-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, URIBL_BLACK autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org RISC-V is apparently the first architecture to pass more than one argument to ifunc resolvers. The helper macros in libc-symbols.h, __ifunc_resolver(), __ifunc(), and __ifunc_hidden(), are incompatible with this. These macros have an "arg" (non-final) parameter that represents the parameter signature of the ifunc resolver. The result is an inability to pass the required comma through in a single preprocessor argument. Rearrange the __ifunc_resolver() macro to be variadic, and pass the types as those variable parameters. Move the guts of __ifunc() and __ifunc_hidden() into new macros, __ifunc_args(), and __ifunc_args_hidden(), that pass the variable arguments down through to __ifunc_resolver(). Then redefine __ifunc() and __ifunc_hidden(), which are used in a bunch of places, to simply shuffle the arguments down into __ifunc_args[_hidden]. Finally, define a riscv-ifunc.h header, which provides convenience macros to those looking to write ifunc selectors that use both arguments. Signed-off-by: Evan Green Reviewed-by: Florian Weimer --- (no changes since v11) Changes in v11: - Update copyright year (Adhemerval) Changes in v9: - Fix a couple of typos causing powerpc not to build (build-many-glibcs) Changes in v6: - Introduced riscv-ifunc.h for multi-arg ifunc selectors. --- include/libc-symbols.h | 28 +++++++++++++++++----------- sysdeps/riscv/riscv-ifunc.h | 27 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 11 deletions(-) create mode 100644 sysdeps/riscv/riscv-ifunc.h diff --git a/include/libc-symbols.h b/include/libc-symbols.h index e21bb599b3..4367aa6740 100644 --- a/include/libc-symbols.h +++ b/include/libc-symbols.h @@ -667,9 +667,9 @@ for linking") #endif /* Helper / base macros for indirect function symbols. */ -#define __ifunc_resolver(type_name, name, expr, arg, init, classifier) \ +#define __ifunc_resolver(type_name, name, expr, init, classifier, ...) \ classifier inhibit_stack_protector \ - __typeof (type_name) *name##_ifunc (arg) \ + __typeof (type_name) *name##_ifunc (__VA_ARGS__) \ { \ init (); \ __typeof (type_name) *res = expr; \ @@ -677,13 +677,13 @@ for linking") } #ifdef HAVE_GCC_IFUNC -# define __ifunc(type_name, name, expr, arg, init) \ +# define __ifunc_args(type_name, name, expr, init, ...) \ extern __typeof (type_name) name __attribute__ \ ((ifunc (#name "_ifunc"))); \ - __ifunc_resolver (type_name, name, expr, arg, init, static) + __ifunc_resolver (type_name, name, expr, init, static, __VA_ARGS__) -# define __ifunc_hidden(type_name, name, expr, arg, init) \ - __ifunc (type_name, name, expr, arg, init) +# define __ifunc_args_hidden(type_name, name, expr, init, ...) \ + __ifunc_args (type_name, name, expr, init, __VA_ARGS__) #else /* Gcc does not support __attribute__ ((ifunc (...))). Use the old behaviour as fallback. But keep in mind that the debug information for the ifunc @@ -694,18 +694,24 @@ for linking") different signatures. (Gcc support is disabled at least on a ppc64le Ubuntu 14.04 system.) */ -# define __ifunc(type_name, name, expr, arg, init) \ +# define __ifunc_args(type_name, name, expr, init, ...) \ extern __typeof (type_name) name; \ - __typeof (type_name) *name##_ifunc (arg) __asm__ (#name); \ - __ifunc_resolver (type_name, name, expr, arg, init,) \ + __typeof (type_name) *name##_ifunc (__VA_ARGS__) __asm__ (#name); \ + __ifunc_resolver (type_name, name, expr, init, , __VA_ARGS__) \ __asm__ (".type " #name ", %gnu_indirect_function"); -# define __ifunc_hidden(type_name, name, expr, arg, init) \ +# define __ifunc_args_hidden(type_name, name, expr, init, ...) \ extern __typeof (type_name) __libc_##name; \ - __ifunc (type_name, __libc_##name, expr, arg, init) \ + __ifunc (type_name, __libc_##name, expr, __VA_ARGS__, init) \ strong_alias (__libc_##name, name); #endif /* !HAVE_GCC_IFUNC */ +#define __ifunc(type_name, name, expr, arg, init) \ + __ifunc_args (type_name, name, expr, init, arg) + +#define __ifunc_hidden(type_name, name, expr, arg, init) \ + __ifunc_args_hidden (type_name, name, expr, init, arg) + /* The following macros are used for indirect function symbols in libc.so. First of all, you need to have the function prototyped somewhere, say in foo.h: diff --git a/sysdeps/riscv/riscv-ifunc.h b/sysdeps/riscv/riscv-ifunc.h new file mode 100644 index 0000000000..c77ab51548 --- /dev/null +++ b/sysdeps/riscv/riscv-ifunc.h @@ -0,0 +1,27 @@ +/* Common definition for ifunc resolvers. Linux/RISC-V version. + This file is part of the GNU C Library. + Copyright (C) 2024 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include + +#define INIT_ARCH() + +#define riscv_libc_ifunc(name, expr) \ + __ifunc_args (name, name, expr(hwcap, hwprobe), INIT_ARCH, \ + uint64_t hwcap, __riscv_hwprobe_t hwprobe) From patchwork Tue Feb 27 22:56:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 86494 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A1499385840D for ; Tue, 27 Feb 2024 22:59:07 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by sourceware.org (Postfix) with ESMTPS id BC3033858C35 for ; Tue, 27 Feb 2024 22:57:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BC3033858C35 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BC3033858C35 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::62f ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074633; cv=none; b=Knw8rNMnt5faMrdvMDptCOYCNTj0IpzBMDcOz+ji+0lNiLS0YboqitopONmSVoviJBfcZT31NkcqUSXZ81w1fglWjVZ2m11veJF428y2ozSfoMzKzZx3YLgzbdJyjS+CBUWzWUNCwp/3swbuBxXXgZ7LJcpKVI9MnAVYSyF7eXw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074633; c=relaxed/simple; bh=9HKwofNDtM3mQbdw5HQr4IwgKqiNwFrv7q8+/y8NToQ=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=Fue3S0VGESGNRcmJFRIkc0coaRFXcw+mq7jXvB4cD6W6KaNZnyKLISWzbYIK/dXnoF3iHdewtIPKEaYZ80EYc2sCzjghY2OIGti7Ivh8s/76ylZuiJ1aN63QZJBC2YbQWjNt9Lybj0DEuc3/o8xdPBKwol1e2w5kKtvCY/UyFOs= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1dc9222b337so29819235ad.2 for ; Tue, 27 Feb 2024 14:57:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709074626; x=1709679426; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PmGUIHJh6VHAV6X1wPidNHCpvUG1fN4EMb5F0/4NlFc=; b=DR4vv+/W20monVKtcuBQT8xRKkj2XSiagKVhuYlwhQNZsSJcXVU9Opv5uQ6n3yufHD cjIdBPgkEbgt7JmDal2RrQsl5oT5v1k5pl7CdCDMxmozGKv/FrfqpuHf4G45RuM4mgPm awuMscDLENBL+dtaveW1Xkws7ijMiuuxj172U7Qplq83feNJjJSNWJowGupoP5yEsFfI cbMdzp4Jd/xN9o0hhVu1HPaHbP3y20jWUdNB7LySuWRLe8O6QaS8qXCvM7lcX7FENkQH qPj5o6Sf+gsQ5u+qMTSFkOXndJboRvnXRQ5GfwvdNSR+gNc+SJneenuHPKFEdsJGbeUF bSGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709074626; x=1709679426; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PmGUIHJh6VHAV6X1wPidNHCpvUG1fN4EMb5F0/4NlFc=; b=FHTmuua3W8USQqzl7OO/U/9VCCf1GnoLKITaLDeUH0kQouqyTM6rO7wGGnP3uCXAs4 8RKd6krkatsJ0l7Zp0P0doKrkSLh5E7/2PHYXqohxZET6mgPD4S3dbBEq+Josomooeg3 7Zp49KVlVEs/dWqcu5Xz8qiRnVEmFhZDj3Ot6H3/1nbah7upewXbiamWG+/GL+RvDHg3 DDlDpTWOm1s95io+xBDVAkxklpfpsv+oP94VpGwxc1XQqbXWGOEp9m+aOlYoIl/0h98B TqRSSYI69Xy+1nS68yeY54dWDkBei/a++edVp6xTCGrAPk6zNlilG2ZVa7zpeFkxaIJz uwvg== X-Gm-Message-State: AOJu0YwGZuabZpJ46sgU3bGSqpus2zHnQTvN8VTD14sAYZIhM4Guk1PJ lwv2hnnTtbXj2DHA5lopGpSTs36R7kMEqbfJLkbzrTk1czsJKcYz9yKLo185iU+Tu0vr7N+UKcU h X-Google-Smtp-Source: AGHT+IFT01LZA2Id3OaRZUgrKPCEh5n1xHVwY9T4yU9QWdF8mNEeAeppqNXvHsU5/EnE/nGQ8ePTnw== X-Received: by 2002:a17:903:1246:b0:1db:f830:c381 with SMTP id u6-20020a170903124600b001dbf830c381mr13821748plh.44.1709074625886; Tue, 27 Feb 2024 14:57:05 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id li3-20020a170903294300b001dc90ac1cecsm2029225plb.284.2024.02.27.14.57.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 14:57:05 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, Florian Weimer , palmer@rivosinc.com, Evan Green Subject: [PATCH v13 6/7] riscv: Add ifunc helper method to hwprobe.h Date: Tue, 27 Feb 2024 14:56:42 -0800 Message-Id: <20240227225644.724901-7-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227225644.724901-1-evan@rivosinc.com> References: <20240227225644.724901-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add a little helper method so it's easier to fetch a single value from the hwprobe function when used within an ifunc selector. Signed-off-by: Evan Green --- (no changes since v11) Changes in v11: - Remove superfluous 'signed' (Adhemerval) - Put helper before __END_DECLS (Adhemerval) - Add comment about +1 on non-Linux systems (Stefan) Changes in v10: - Avoid implicit comparisons (Adhemerval) Changes in v9: - Use __inline rather than inline so c89 compiles (build-many-glibcs) Changes in v7: - Introduced static inline helper (Richard) sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 29 +++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h index 34a2e3dbc2..8ecb43bb69 100644 --- a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -22,6 +22,7 @@ #include #include +#include #ifdef __has_include # if __has_include () # include @@ -79,6 +80,34 @@ typedef int (*__riscv_hwprobe_t) (struct riscv_hwprobe *__pairs, size_t __pair_c __fortified_attr_access (__read_write__, 1, 2) __fortified_attr_access (__read_only__, 4, 3); +/* Helper function usable from ifunc selectors that probes a single key. */ +static __inline int +__riscv_hwprobe_one(__riscv_hwprobe_t hwprobe_func, + long long int key, + unsigned long long int *value) +{ + struct riscv_hwprobe pair; + int rc; + + /* Earlier versions of glibc pass NULL as the second ifunc parameter. Other C + libraries on non-Linux systems may pass +1 as this function pointer to + indicate no support. Users copying this function to exotic worlds + (non-Linux non-glibc) may want to do additional validity checks here. */ + if (hwprobe_func == NULL) + return ENOSYS; + + pair.key = key; + rc = hwprobe_func (&pair, 1, 0, NULL, 0); + if (rc != 0) + return rc; + + if (pair.key < 0) + return ENOENT; + + *value = pair.value; + return 0; +} + __END_DECLS #endif /* sys/hwprobe.h */ From patchwork Tue Feb 27 22:56:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 86491 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 730D03858299 for ; Tue, 27 Feb 2024 22:58:26 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by sourceware.org (Postfix) with ESMTPS id D215E3858C32 for ; Tue, 27 Feb 2024 22:57:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D215E3858C32 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D215E3858C32 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::532 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074634; cv=none; b=AmQtKPVmgUYUal2X0JLDmDt7zqyI7wTs/F7/qsiVHofLlpx6oaVsDRpOqM9JZGYkTh977sHCWwO9cSXQvGcdBBYfe9jUa80/NCICn2wV6NhG000g2e4qdZSQ02V06gl7wpEZUUCuAoPYW5EIIqqvCCsZO+3S+t7lvPv//dosbX8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709074634; c=relaxed/simple; bh=RQeCsTdLzL95jS0ANE2Z4YVo2AodGkOejdkJeicqBls=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=JUSx6sMQ6zalBeQ2ipSHBuFT/ZDA7+gPHJq2A7j5R5TCtCOYzVFaEsx6qN3iGAiloKE94RrjaR35lNEUSAs8FgWmuS89XXAOwzIB7XerzTt7wZCx0AsUVWOYy8Y47Ox7CchPYTyMTuU1a4NBhCI/r3C+4Rg5dEmn5mle7URrp88= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-5cdbc4334edso3556859a12.3 for ; Tue, 27 Feb 2024 14:57:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709074628; x=1709679428; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d0YVHUk/lddbP97jJ8T2SQlEKI7riLRj+3PwAqGif+0=; b=P1+rMMUVfKJT1xxB+Jm/8Pksbb4Ib7AL8uGqyYywx/OsFTbd5nsFo/tdB+Dp8ve8GZ 9zi+gi86BnxsY93ppjGVf2hZlMYi48AX8Qz/05xizjn7wRDy7zMeenJIyXIiYI4bF6eW 0uUeeAxcAlwb0oM0ozKAQIDtQ31XijP4wnEAxn/hsjlGV3d/qtmgA8mHDVOIAyseNb8h eg7GOOfAqvOEgmjAh/RnVeGau54Ujiwm7Mxc8PRiDMEH2aOThn5t0c/umOpR2mNMPYDS BG1tvg2uLKUwqIv3fmNcEY/6T8VJRkCjuxJLJNlTMx/qHVLvOWE6ANXngA/eLN3tdvlv 1WZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709074628; x=1709679428; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d0YVHUk/lddbP97jJ8T2SQlEKI7riLRj+3PwAqGif+0=; b=oUZM9tT1gGxxLw1qhPs1LUa8lQ7QjK7hkm2e0Yhh5lliWPv/AwnUqtr00l6A6SBOUO GMAfZY2RxjaHNS0bocvdao9/eELRMU0NvPmPwBmlsvRD2Kbw1+cgAySXbAoqcWShtwXz B3xGly4siQJsqn2dVlnOxDtGKORzbzIaxm6EEJqYAYiFRM/v86GKwyqUIuh0wl/trs5u yX7vYJPiC8KAxwVzI7EdSng4zBAPii6yKPc1hPcYIN4HZ9ONOFskI/akFN7N1LqGD9Ih EwdURLr09jujEvzvxglVJZu8lkDc/j6/jMlOz/1tnPmpqM2tg+FYe+sWhib6dli7Lm4B xTBg== X-Gm-Message-State: AOJu0YyCDRJHRKehXyfjEV9SBvh4Ew/GeWI4Pp9WX5fq1PE1IwUn2N4x thEnYmNiQFv33KcIC54nX8n1/U6brE3r2mZGZs3xMgV0tEYYIpJ3Yxou4uaAn5tm6SpMwXH3Lum N X-Google-Smtp-Source: AGHT+IHfOy7a8KA1yM4XLI23oCo7kdntgbCBkbOvv6TjeG/qyO56K+8ei10uRXH1Xq12rERFHjHpMQ== X-Received: by 2002:a05:6a20:94c9:b0:1a0:9115:48c8 with SMTP id ht9-20020a056a2094c900b001a0911548c8mr4200980pzb.51.1709074627944; Tue, 27 Feb 2024 14:57:07 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id li3-20020a170903294300b001dc90ac1cecsm2029225plb.284.2024.02.27.14.57.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 14:57:07 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, Florian Weimer , palmer@rivosinc.com, Evan Green Subject: [PATCH v13 7/7] riscv: Add and use alignment-ignorant memcpy Date: Tue, 27 Feb 2024 14:56:43 -0800 Message-Id: <20240227225644.724901-8-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240227225644.724901-1-evan@rivosinc.com> References: <20240227225644.724901-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org For CPU implementations that can perform unaligned accesses with little or no performance penalty, create a memcpy implementation that does not bother aligning buffers. It will use a block of integer registers, a single integer register, and fall back to bytewise copy for the remainder. Signed-off-by: Evan Green Reviewed-by: Palmer Dabbelt --- (no changes since v11) Changes in v11: - Update copyrights to 2024 (Adhemerval) - Avoid implicit check in select_memcpy_ifunc (Adhemerval) - Remove hidden_def (__memcpy_noalignment) (Adhemerval) Changes in v10: - One line per function in Makefile for memcpy (Adhemerval) - Space before argument-like things (Adhemerval) Changes in v7: - Use new helper function in memcpy ifunc selector (Richard) Changes in v6: - Fix a couple regressions in the assembly from v5 :/ - Use passed hwprobe pointer in memcpy ifunc selector. Changes in v5: - Do unaligned word access for final trailing bytes (Richard) Changes in v4: - Fixed comment style (Florian) Changes in v3: - Word align dest for large memcpy()s. - Add tags - Remove spurious blank line from sysdeps/riscv/memcpy.c Changes in v2: - Used _MASK instead of _FAST value itself. --- sysdeps/riscv/memcopy.h | 26 ++++ sysdeps/riscv/memcpy.c | 63 ++++++++ sysdeps/riscv/memcpy_noalignment.S | 136 ++++++++++++++++++ sysdeps/unix/sysv/linux/riscv/Makefile | 9 ++ .../unix/sysv/linux/riscv/memcpy-generic.c | 24 ++++ 5 files changed, 258 insertions(+) create mode 100644 sysdeps/riscv/memcopy.h create mode 100644 sysdeps/riscv/memcpy.c create mode 100644 sysdeps/riscv/memcpy_noalignment.S create mode 100644 sysdeps/unix/sysv/linux/riscv/memcpy-generic.c diff --git a/sysdeps/riscv/memcopy.h b/sysdeps/riscv/memcopy.h new file mode 100644 index 0000000000..27675964b0 --- /dev/null +++ b/sysdeps/riscv/memcopy.h @@ -0,0 +1,26 @@ +/* memcopy.h -- definitions for memory copy functions. RISC-V version. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +/* Redefine the generic memcpy implementation to __memcpy_generic, so + the memcpy ifunc can select between generic and special versions. + In rtld, don't bother with all the ifunciness. */ +#if IS_IN (libc) +#define MEMCPY __memcpy_generic +#endif diff --git a/sysdeps/riscv/memcpy.c b/sysdeps/riscv/memcpy.c new file mode 100644 index 0000000000..20f9548c44 --- /dev/null +++ b/sysdeps/riscv/memcpy.c @@ -0,0 +1,63 @@ +/* Multiple versions of memcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017-2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memcpy so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memcpy +# define memcpy __redirect_memcpy +# include +# include +# include +# include +# include + +# define INIT_ARCH() + +extern __typeof (__redirect_memcpy) __libc_memcpy; + +extern __typeof (__redirect_memcpy) __memcpy_generic attribute_hidden; +extern __typeof (__redirect_memcpy) __memcpy_noalignment attribute_hidden; + +static inline __typeof (__redirect_memcpy) * +select_memcpy_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long int value; + + INIT_ARCH (); + + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_CPUPERF_0, &value) != 0) + return __memcpy_generic; + + if ((value & RISCV_HWPROBE_MISALIGNED_MASK) == RISCV_HWPROBE_MISALIGNED_FAST) + return __memcpy_noalignment; + + return __memcpy_generic; +} + +riscv_libc_ifunc (__libc_memcpy, select_memcpy_ifunc); + +# undef memcpy +strong_alias (__libc_memcpy, memcpy); +# ifdef SHARED +__hidden_ver1 (memcpy, __GI_memcpy, __redirect_memcpy) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memcpy); +# endif + +#endif diff --git a/sysdeps/riscv/memcpy_noalignment.S b/sysdeps/riscv/memcpy_noalignment.S new file mode 100644 index 0000000000..621f8d028f --- /dev/null +++ b/sysdeps/riscv/memcpy_noalignment.S @@ -0,0 +1,136 @@ +/* memcpy for RISC-V, ignoring buffer alignment + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#include +#include + +/* void *memcpy(void *, const void *, size_t) */ +ENTRY (__memcpy_noalignment) + move t6, a0 /* Preserve return value */ + + /* Bail if 0 */ + beqz a2, 7f + + /* Jump to byte copy if size < SZREG */ + li a4, SZREG + bltu a2, a4, 5f + + /* Round down to the nearest "page" size */ + andi a4, a2, ~((16*SZREG)-1) + beqz a4, 2f + add a3, a1, a4 + + /* Copy the first word to get dest word aligned */ + andi a5, t6, SZREG-1 + beqz a5, 1f + REG_L a6, (a1) + REG_S a6, (t6) + + /* Align dst up to a word, move src and size as well. */ + addi t6, t6, SZREG-1 + andi t6, t6, ~(SZREG-1) + sub a5, t6, a0 + add a1, a1, a5 + sub a2, a2, a5 + + /* Recompute page count */ + andi a4, a2, ~((16*SZREG)-1) + beqz a4, 2f + +1: + /* Copy "pages" (chunks of 16 registers) */ + REG_L a4, 0(a1) + REG_L a5, SZREG(a1) + REG_L a6, 2*SZREG(a1) + REG_L a7, 3*SZREG(a1) + REG_L t0, 4*SZREG(a1) + REG_L t1, 5*SZREG(a1) + REG_L t2, 6*SZREG(a1) + REG_L t3, 7*SZREG(a1) + REG_L t4, 8*SZREG(a1) + REG_L t5, 9*SZREG(a1) + REG_S a4, 0(t6) + REG_S a5, SZREG(t6) + REG_S a6, 2*SZREG(t6) + REG_S a7, 3*SZREG(t6) + REG_S t0, 4*SZREG(t6) + REG_S t1, 5*SZREG(t6) + REG_S t2, 6*SZREG(t6) + REG_S t3, 7*SZREG(t6) + REG_S t4, 8*SZREG(t6) + REG_S t5, 9*SZREG(t6) + REG_L a4, 10*SZREG(a1) + REG_L a5, 11*SZREG(a1) + REG_L a6, 12*SZREG(a1) + REG_L a7, 13*SZREG(a1) + REG_L t0, 14*SZREG(a1) + REG_L t1, 15*SZREG(a1) + addi a1, a1, 16*SZREG + REG_S a4, 10*SZREG(t6) + REG_S a5, 11*SZREG(t6) + REG_S a6, 12*SZREG(t6) + REG_S a7, 13*SZREG(t6) + REG_S t0, 14*SZREG(t6) + REG_S t1, 15*SZREG(t6) + addi t6, t6, 16*SZREG + bltu a1, a3, 1b + andi a2, a2, (16*SZREG)-1 /* Update count */ + +2: + /* Remainder is smaller than a page, compute native word count */ + beqz a2, 7f + andi a5, a2, ~(SZREG-1) + andi a2, a2, (SZREG-1) + add a3, a1, a5 + /* Jump directly to last word if no words. */ + beqz a5, 4f + +3: + /* Use single native register copy */ + REG_L a4, 0(a1) + addi a1, a1, SZREG + REG_S a4, 0(t6) + addi t6, t6, SZREG + bltu a1, a3, 3b + + /* Jump directly out if no more bytes */ + beqz a2, 7f + +4: + /* Copy the last word unaligned */ + add a3, a1, a2 + add a4, t6, a2 + REG_L a5, -SZREG(a3) + REG_S a5, -SZREG(a4) + ret + +5: + /* Copy bytes when the total copy is . */ + +#include + +extern __typeof (memcpy) __memcpy_generic; +hidden_proto (__memcpy_generic) + +#include