From patchwork Wed Feb 14 14:31:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 85726 X-Patchwork-Delegate: palmer@dabbelt.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C96CA385E448 for ; Wed, 14 Feb 2024 14:33:13 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by sourceware.org (Postfix) with ESMTPS id 3600D385E036 for ; Wed, 14 Feb 2024 14:32:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3600D385E036 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 3600D385E036 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::42f ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921164; cv=none; b=ehy2EJq82o/W2VRF1IEPAgo585dkZwZI2cUNAUdPWaygcXhznNMz5B2yCa5uPHs5Amjw4L/RiTJ08XX9SE6sfUg62CfOcojT2HV3+lhM5AKAksQ6s/21RCQqgOyykRY/TmLbbZjZDlClzzke3Ht/X9o4Zofx7+AKzPo1qZXVyvI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921164; c=relaxed/simple; bh=OQU9YIOXJvatDB3qSLwGciF83TDvxGgbxzQjZA7+Djg=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=HKuD/NCpBO8cjv6ibdeXZRstcNS0CVbRM7G3pw8Sd5f/aaWi1DiC0afwNF7+Zb0Wx+mies0t48A9IUpLD3H6b7kPuQm8Ruc+jZa8zmwECxZGgY4oiX6f+eChSHlwAh09NaqVpsCDX9e547Q1GQA2QgOChtw6cqhsJgTfNksf4O8= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-6e08dd0c7eeso1440688b3a.1 for ; Wed, 14 Feb 2024 06:32:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1707921160; x=1708525960; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GDQ4QURRkri2MAwsFFwoNg4fpoAYNGAtyYfanV2h104=; b=DhuhSz+WdCJWzPt9p7eSVMeYfyKiIFT8nc3OD2lmBDd2q6qwgGG/uUXOsYHqoRUHva 42APjqboTKgt9KQqnN3/GVYUW8x6FFu4qzJtsXFXbyoX5i+KveareJIJ5S5iOpazvsQ6 sWOm/uAJVx0wmqHQv1t6mCnNcdxGLNJvTF3RJ61UrsXz9VZgOu/qQQm93zdVgUGx5ky5 3ayAeRNsdGMq/ZY4eZi3hsAFT9g2Agvw6AujuX9EwscMDv+m5Koimu/Dy160jDZh7PcS nKUBSF+/PdbuRpBylOocJW7TaNCgIE/Hi47+8QfXMbu2I2WFc5SiRTXnEVR7Z483mW37 +FyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707921160; x=1708525960; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GDQ4QURRkri2MAwsFFwoNg4fpoAYNGAtyYfanV2h104=; b=PuyWd1FXGVrV1XK2nW3jfwbmXuEjtRSMyYN1FbsUPTkS0GD373osIgq0pX9Q2cUuYx n9jZgSvcLu3rw5wOvIB7U8f5tzoDwSk80PIs49+IXeVNmb6aiu6HSHkjCk6vB2IuoiUE UVh4etEeyKQwRKoHLRhTptU8I2HTi/54s1TpKXlDCeRPGaAQmiH9DPTO2whHefWjX17D AW20LPmsQpqI1eYxnf6IzaWkahJlVcOi6KEFU4OesmAFL4Wcog8Wg+RfYWzMBbU5C7rM bjYGvbks4+pjHzFU1NzRQqIihZ/UMUYssXrzvtbd0KqqH/juk/QeqqBJdde0SahHY8T7 7l5w== X-Gm-Message-State: AOJu0Yxn1Nc5/kvi6ddg10z2+HmLWoXKvSOa4UOKFvWBDwNNYDxqYOZs 9N9sn+duN7zZM4eqRwaP1Yox3mFHEEDuyIFeT0IlkEIPXs7GeQHCmqo7PylIBKMftMk3kcl2zr2 8 X-Google-Smtp-Source: AGHT+IFGkpOcTI12NmcbYM1aQBi3Bupc3iVXqcvxgc3MT8QbqRRPlC2sszMc0a6hKUQJGEJ/lsx5/w== X-Received: by 2002:a05:6a20:d908:b0:19a:47f2:5766 with SMTP id jd8-20020a056a20d90800b0019a47f25766mr3607284pzb.56.1707921159834; Wed, 14 Feb 2024 06:32:39 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVP6UeYfCgN+tV7pjdRBkK0jkassCiPjOGDlCkrnZLwRNFa73OVEr1wvzVYeHmrm7rEeBoSsdJMIF85MCAbYX0Fu+pMtioVJghMsGQi0sfndYTmVMtTwmqMsxdzi/R2FcbaslLBX0fDBtBdNKHUgTZjkC1tWvoAtyiw3/txJ89L Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id k21-20020a635a55000000b005dc98d9114bsm1140255pgm.43.2024.02.14.06.32.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Feb 2024 06:32:39 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, Florian Weimer , slewis@rivosinc.com, palmer@rivosinc.com, Evan Green Subject: [PATCH v12 1/7] riscv: Add Linux hwprobe syscall support Date: Wed, 14 Feb 2024 06:31:52 -0800 Message-Id: <20240214143159.2951158-2-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214143159.2951158-1-evan@rivosinc.com> References: <20240214143159.2951158-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add awareness and a thin wrapper function around a new Linux system call that allows callers to get architecture and microarchitecture information about the CPUs from the kernel. This can be used to do things like dynamically choose a memcpy implementation. Signed-off-by: Evan Green --- Changes in v12: - Updated version to 2.40 Changes in v11: - Update copyright year (Adhemerval) Changes in v10: - Remove spurious 5 from syscall patch (Adhemerval) - Use one item per line in Makefile (Adhemerval) - Remove double underscores from __riscv_hwprobe definition (Adhemerval) - Use only spaces in macro definitions of hwprobe.h (Adhemerval) Changes in v9: - Alphabetize new entries in libc.abilist (to pass checks) Changes in v8: - Fix missed 2.39 in abilists (Joseph) - Just return -r (Florian) Changes in v7: - Bumped Versions up to 2.39 (Joseph) - Used INTERNAL_SYSCALL_CALL, and return positive errno to match pthreads API (Florian). - Remove __THROW since it creates a warning in combination with the fortified access decorators. Changes in v6: - Prefixed __riscv_hwprobe() parameters names with __ to avoid user macro namespace pollution (Joseph) Changes in v4: - Remove __USE_GNU (Florian) - __nonnull, __wur, __THROW, and __fortified_attr_access decorations (Florian) - change long to long int (Florian) - Fix comment formatting (Florian) - Update backup kernel header content copy. - Fix function declaration formatting (Florian) - Changed export versions to 2.38 Changes in v3: - Update argument types to match v4 kernel interface Changes in v2: - hwprobe.h: Use __has_include and duplicate Linux content to make compilation work when Linux headers are absent (Adhemerval) - hwprobe.h: Put declaration under __USE_GNU (Adhemerval) - Use INLINE_SYSCALL_CALL (Adhemerval) - Update versions - Update UNALIGNED_MASK to match kernel v3 series. sysdeps/unix/sysv/linux/riscv/Makefile | 12 ++- sysdeps/unix/sysv/linux/riscv/Versions | 3 + sysdeps/unix/sysv/linux/riscv/hwprobe.c | 36 +++++++++ .../unix/sysv/linux/riscv/rv32/libc.abilist | 1 + .../unix/sysv/linux/riscv/rv64/libc.abilist | 1 + sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 74 +++++++++++++++++++ 6 files changed, 125 insertions(+), 2 deletions(-) create mode 100644 sysdeps/unix/sysv/linux/riscv/hwprobe.c create mode 100644 sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h diff --git a/sysdeps/unix/sysv/linux/riscv/Makefile b/sysdeps/unix/sysv/linux/riscv/Makefile index 4b6eacb32f..04abf226ad 100644 --- a/sysdeps/unix/sysv/linux/riscv/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/Makefile @@ -1,6 +1,14 @@ ifeq ($(subdir),misc) -sysdep_headers += sys/cachectl.h -sysdep_routines += flush-icache +sysdep_headers += \ + sys/cachectl.h \ + sys/hwprobe.h \ + # sysdep_headers + +sysdep_routines += \ + flush-icache \ + hwprobe \ + # sysdep_routines + endif ifeq ($(subdir),stdlib) diff --git a/sysdeps/unix/sysv/linux/riscv/Versions b/sysdeps/unix/sysv/linux/riscv/Versions index 5625d2a0b8..8183e9feb6 100644 --- a/sysdeps/unix/sysv/linux/riscv/Versions +++ b/sysdeps/unix/sysv/linux/riscv/Versions @@ -8,4 +8,7 @@ libc { GLIBC_2.27 { __riscv_flush_icache; } + GLIBC_2.40 { + __riscv_hwprobe; + } } diff --git a/sysdeps/unix/sysv/linux/riscv/hwprobe.c b/sysdeps/unix/sysv/linux/riscv/hwprobe.c new file mode 100644 index 0000000000..28a448175b --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/hwprobe.c @@ -0,0 +1,36 @@ +/* RISC-V hardware feature probing support on Linux + Copyright (C) 2024 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include + +int __riscv_hwprobe (struct riscv_hwprobe *pairs, size_t pair_count, + size_t cpu_count, unsigned long int *cpus, + unsigned int flags) +{ + int r; + + r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, pairs, pair_count, + cpu_count, cpus, flags); + + /* Negate negative errno values to match pthreads API. */ + return -r; +} diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist index f90c94bc35..6397a9cb91 100644 --- a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist +++ b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist @@ -2511,3 +2511,4 @@ GLIBC_2.39 stdc_trailing_zeros_ui F GLIBC_2.39 stdc_trailing_zeros_ul F GLIBC_2.39 stdc_trailing_zeros_ull F GLIBC_2.39 stdc_trailing_zeros_us F +GLIBC_2.40 __riscv_hwprobe F diff --git a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist index e04ff93bd2..71bbf94f66 100644 --- a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist +++ b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist @@ -2711,3 +2711,4 @@ GLIBC_2.39 stdc_trailing_zeros_ui F GLIBC_2.39 stdc_trailing_zeros_ul F GLIBC_2.39 stdc_trailing_zeros_ull F GLIBC_2.39 stdc_trailing_zeros_us F +GLIBC_2.40 __riscv_hwprobe F diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h new file mode 100644 index 0000000000..5592b9e100 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -0,0 +1,74 @@ +/* RISC-V architecture probe interface + Copyright (C) 2024 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#ifndef _SYS_HWPROBE_H +#define _SYS_HWPROBE_H 1 + +#include +#include +#ifdef __has_include +# if __has_include () +# include +# endif +#endif + +/* Define a (probably stale) version of the interface if the Linux headers + aren't present. */ +#ifndef RISCV_HWPROBE_KEY_MVENDORID +struct riscv_hwprobe { + signed long long int key; + unsigned long long int value; +}; + +#define RISCV_HWPROBE_KEY_MVENDORID 0 +#define RISCV_HWPROBE_KEY_MARCHID 1 +#define RISCV_HWPROBE_KEY_MIMPID 2 +#define RISCV_HWPROBE_KEY_BASE_BEHAVIOR 3 +#define RISCV_HWPROBE_BASE_BEHAVIOR_IMA (1 << 0) +#define RISCV_HWPROBE_KEY_IMA_EXT_0 4 +#define RISCV_HWPROBE_IMA_FD (1 << 0) +#define RISCV_HWPROBE_IMA_C (1 << 1) +#define RISCV_HWPROBE_IMA_V (1 << 2) +#define RISCV_HWPROBE_EXT_ZBA (1 << 3) +#define RISCV_HWPROBE_EXT_ZBB (1 << 4) +#define RISCV_HWPROBE_EXT_ZBS (1 << 5) +#define RISCV_HWPROBE_EXT_ZICBOZ (1 << 6) +#define RISCV_HWPROBE_KEY_CPUPERF_0 5 +#define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) +#define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) +#define RISCV_HWPROBE_MISALIGNED_SLOW (2 << 0) +#define RISCV_HWPROBE_MISALIGNED_FAST (3 << 0) +#define RISCV_HWPROBE_MISALIGNED_UNSUPPORTED (4 << 0) +#define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) +#define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 + +#endif /* RISCV_HWPROBE_KEY_MVENDORID */ + +__BEGIN_DECLS + +extern int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) + __nonnull ((1)) __wur + __fortified_attr_access (__read_write__, 1, 2) + __fortified_attr_access (__read_only__, 4, 3); + +__END_DECLS + +#endif /* sys/hwprobe.h */ From patchwork Wed Feb 14 14:31:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 85727 X-Patchwork-Delegate: palmer@dabbelt.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 706073860015 for ; Wed, 14 Feb 2024 14:33:16 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by sourceware.org (Postfix) with ESMTPS id 6FC50385E007 for ; Wed, 14 Feb 2024 14:32:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6FC50385E007 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 6FC50385E007 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::429 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921168; cv=none; b=uN8GqASdXb87vmYBWMD4PJsoZPftPvZmjPOT/radQxnJMWIo41RgsN1Fzbr88VTEZ06e+SSpROUGAA59w8iYIBJspyp499vcNs5vXL/ZM5JcdBON0OZ6uOrStmmEX3pEZfEc9ygMdynT+vwSPDYj864if2DPj6GoOmS3P2mgAmM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921168; c=relaxed/simple; bh=1wuQLv8CmSDMOXjtH1MNMaPK7zKggX9VA0d9GfH+IqM=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=KOCaVFYGI+pn9mbFUw8aCCwbx8XFmxYd+D6N4J6fHO0KPaovxcZDPjAzLbPZjrxEqw1Lr//GNLfYzi1rt2+Sj5kpRyHDQksAQDAm7FnBU0PmcNwfIedHzXCe+FiFc3l/sTnzbGROCT+o9W1pPtdv6NFFm0blV5nlCcCbjnkTmlQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-6e0f4e3bc59so1601971b3a.0 for ; Wed, 14 Feb 2024 06:32:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1707921164; x=1708525964; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p5uSuckPc0H6i5FwW8PtNnyo41Zsuwt/Ty+Wr6YA8Jc=; b=RF206U1HydYXiglZriEUWnIx+REw1/sGUkn4PyYSNSzYq5x9x4w/j7QhIx/K+CPzCb OqnOkoETI64ayn4p3DPP6UQRwXKrrXDq/pFOzFSAuOIvT+24/1JDajVstvRCSOAiNtke o0snl9Sakc7YEjU/ne7MaMrHucNyS4UBoWtyt+D4/0peH2h2D+D90irVNPe/0IFkpMjW gBKr1qhdgkU1EuDqZyvjZ5UbmWWvITy84JMuX4IoSWWIKguDvO++2do/z1CS517qKQlR aYZg1JMRGnQciWS93pntbq1O6WBGQaVIXaKNHYwNLpwUDQwYdbwjtIogjypB7E3Zl2wK DB+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707921164; x=1708525964; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p5uSuckPc0H6i5FwW8PtNnyo41Zsuwt/Ty+Wr6YA8Jc=; b=gGKTvfzvbvXnc2BCpgZy5Qx01SsXQpz7z5u4Z5qfol71pQsAqowpVkFliH77amYswl ZPZeJgbMNmqANEMlyow3/+4IyzpZ5+y2wDFJgm+mPf4FtQGi+Ee96+2W1Je1v3nwH7Gt UhsjAorXOVrvgXrfzmQTY8QXISCZY67rKRt2iDs0f2miMH7Y7QQzGYyhygtIF7HsIRo5 5cw27hzRc+ZzRwLr19QowPCE1R3kceSO+UIsQ6D+g90gIM68MGxSOfSKQD2Dh8Y1/LEl gl/258kIG+W1yLekDO0ytL2wsBBwQKC60YpCDMME5UCcj+3FLPiDA7MzTOhTeeVbwibq UP0Q== X-Gm-Message-State: AOJu0YwbqfEmK5Ggh25g5PEhYZJK50thdWq8kkKMyzBoqR2Ypegr//Ir hgWQk8+T3N5CvKa4PEHcY+Cc4T5RvdNZ0Z3COCfwhGbI2G5oULdCQfoap9MAb/9jct7pNONaoxr / X-Google-Smtp-Source: AGHT+IHj1PMUC6w+mmhWc8OmJGMa4ZmmWVFt6DHOrRvE5FbtndoCzAcvradD5eM1Ea26FJw0+2+ZoQ== X-Received: by 2002:a05:6a21:1645:b0:19f:2df0:13fd with SMTP id no5-20020a056a21164500b0019f2df013fdmr3469627pzb.29.1707921164587; Wed, 14 Feb 2024 06:32:44 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXkLOmZj0FJSodzm/AUoihDwkpPJhS8TYyXsUzVurlC1nUC/SRIllTe8vV13NldzlnnOdF3UHYUOLfV69Xnyy5sYM+GzSSQW9PzNChWvbZ5U3bzZYy/HckQWu5lE4UJoAn+3NL6/zIgecepA/mh8Dam2lLI4uuix2uF5mt5kLHv Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id k21-20020a635a55000000b005dc98d9114bsm1140255pgm.43.2024.02.14.06.32.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Feb 2024 06:32:44 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, Florian Weimer , slewis@rivosinc.com, palmer@rivosinc.com, Evan Green Subject: [PATCH v12 2/7] linux: Introduce INTERNAL_VSYSCALL Date: Wed, 14 Feb 2024 06:31:53 -0800 Message-Id: <20240214143159.2951158-3-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214143159.2951158-1-evan@rivosinc.com> References: <20240214143159.2951158-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add an INTERNAL_VSYSCALL() macro that makes a vDSO call, falling back to a regular syscall, but without setting errno. Instead, the return value is plumbed straight out of the macro. Signed-off-by: Evan Green --- (no changes since v10) Changes in v10: - Introduced INTERNAL_VSYSCALL patch sysdeps/unix/sysv/linux/sysdep-vdso.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/sysdeps/unix/sysv/linux/sysdep-vdso.h b/sysdeps/unix/sysv/linux/sysdep-vdso.h index 189319ad98..f7d4b29b25 100644 --- a/sysdeps/unix/sysv/linux/sysdep-vdso.h +++ b/sysdeps/unix/sysv/linux/sysdep-vdso.h @@ -53,4 +53,23 @@ sc_ret; \ }) +#define INTERNAL_VSYSCALL(name, nr, args...) \ + ({ \ + __label__ out; \ + long int sc_ret; \ + \ + __typeof (GLRO(dl_vdso_##name)) vdsop = GLRO(dl_vdso_##name); \ + if (vdsop != NULL) \ + { \ + sc_ret = INTERNAL_VSYSCALL_CALL (vdsop, nr, ##args); \ + if ((!INTERNAL_SYSCALL_ERROR_P (sc_ret)) || \ + (INTERNAL_SYSCALL_ERRNO (sc_ret) != ENOSYS)) \ + goto out; \ + } \ + \ + sc_ret = INTERNAL_SYSCALL_CALL (name, ##args); \ + out: \ + sc_ret; \ + }) + #endif /* SYSDEP_VDSO_LINUX_H */ From patchwork Wed Feb 14 14:31:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 85728 X-Patchwork-Delegate: palmer@dabbelt.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A87C7385E83D for ; Wed, 14 Feb 2024 14:33:40 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by sourceware.org (Postfix) with ESMTPS id 6386D385E457 for ; Wed, 14 Feb 2024 14:32:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6386D385E457 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 6386D385E457 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::436 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921172; cv=none; b=WK3L3YGsaSBaU/aPzZ0hq9HZJw++O/n4SoiahCXpdyhqUo+94EqVrEisDUJId2rH8ryLtWGZ0Lf+rUf9LjePHBXCLtNZFauTpV+nemUNwL+535DTPuN9bNCKZ6uuuLMg9r80Gs/Axo7ljSziu6Ep1YwiwK5yWnvEtjMXK1sK6qc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921172; c=relaxed/simple; bh=mB2uCA7GFf8IcHf8n94d1wmGmYoiXZHMQQ84vEGbFaw=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=hl4qZRbNCQcNc8p0+SbHmaW5Z4geM402ubcoQCUWok6UC4NJw/lEGNtlmryAGQHprgQRDo47lVrou8baB6TNxNJLwJuMaTM+JnZsLOaT285L5u4mpAcYTzrFaS16B8zcPypnXyRtIHHR+v50hHx4lTbbgpkm0tVQzXP1if/WDEM= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6e09ea155c5so3168725b3a.2 for ; Wed, 14 Feb 2024 06:32:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1707921168; x=1708525968; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N3idKZzOv0gJ2EL0bSki58RkN41BRlAo5gY4kORw7kQ=; b=W9X01W48t0p+zZWLEXu4bglEoR8+zGAbcT5xoGqdjznLhnAb8WBv/b0KQ98ZaAIIs4 zBzDMI/Y3OQfqFsv3EHB+sxyegEeWM2xpIXNZwgesFXIRpjOOhtJ+VFyGpSNY+60C6og n/43KC1oD10rNuS3qnYMbkjNPf5tJxyNZAxSY5oTUmKH+HuYwXxiyyYp+XMsJFWdkOEc d0A02sePPcOnUpwqPpwlDsXMtavu0KTCPnvbL7TeGff+OyMQ5vPVqXMCvwsFrIulAFUn xMB2nWJugMpXsJXmMMwW0MnI/sLeVe6dERzTzFkOLIlFAigA92tnw3/NbABXcLeKYUiC qlxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707921168; x=1708525968; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N3idKZzOv0gJ2EL0bSki58RkN41BRlAo5gY4kORw7kQ=; b=UocbnzxW/WJneuN07oPuWmlQlDmB/561302xVOrhNJPpAR9vF8vr5s4wqD/UopVGPr B75Cz2Q2VY8dFprksYrjWk6nBqUoQcxHI9QbwLt9WvlE2YfW51Kn8/xJuJMjl7M1PRZo EIE/QNju07U9H65j8E/jzff/B+/mV593zOSZ9Cj/UfhSQ31hvws5hU0P7nl0A39Y/mth 7yN6VjxvgToLbVB6ekMUUAYob+B1601YVMKaz2z2DAHnPm42kwXg7jazbX2BAzwb4CwK Xc6wLDtEBE0E+e0vCE5Pf5cghdINy1wLfWSYPcJaFeYi+X/4NsKsgBRtJ8iDiF+sZQrE 1izw== X-Gm-Message-State: AOJu0Yy6ql8vudjylVmajDbWVvKoi6wKBeMN8VSyn5bu3WUAblBWTiDM 2HKdAaIqhKVPOpgY9meL0JKjYwvM0nj737HHjZe3CiZjMy5XZ4aLuR20Z1rRzLxe/yaLB9rvtHN L X-Google-Smtp-Source: AGHT+IHU6rrsIGYd99HnaAnpuRpCsC/Vt0/hBA1GijcibnbqNv+p9tOkpR2JGzMH2KNXa2eeedyIrQ== X-Received: by 2002:a05:6a20:9f45:b0:19e:a1b5:e765 with SMTP id ml5-20020a056a209f4500b0019ea1b5e765mr3336116pzb.9.1707921168476; Wed, 14 Feb 2024 06:32:48 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXi062y4h3p8j/LPpb3BpJzFxgWi1BX2WpouLlzGDobMtY4nxYLNLn7gFFtMWbMFYQPT+ZpVLsQTHuCC3VPqPJRKzQMjazN1lWyeQXiFFF8/otES0C2ILGsnMXrPUJGMC9S1FOMPwTekyL4IVYmWLZBwjizVRT3siUqDSoCJ70V Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id k21-20020a635a55000000b005dc98d9114bsm1140255pgm.43.2024.02.14.06.32.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Feb 2024 06:32:48 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, Florian Weimer , slewis@rivosinc.com, palmer@rivosinc.com, Evan Green Subject: [PATCH v12 3/7] riscv: Add hwprobe vdso call support Date: Wed, 14 Feb 2024 06:31:54 -0800 Message-Id: <20240214143159.2951158-4-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214143159.2951158-1-evan@rivosinc.com> References: <20240214143159.2951158-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The new riscv_hwprobe syscall also comes with a vDSO for faster answers to your most common questions. Call in today to speak with a kernel representative near you! Signed-off-by: Evan Green --- (no changes since v10) Changes in v10: - Remove leading underscores in definition (Adhemerval) - Remove spurious 5 from INTERNAL_SYSCALL_CALL (Adhemerval) - Use new INTERNAL_VSYSCALL macro instead (Adhemerval) Changes in v7: - Use INTERNAL_VSYSCALL_CALL (Florian) Changes in v3: - Add the "return" to the vsyscall - Fix up vdso arg types to match kernel v4 version - Remove ifdef around INLINE_VSYSCALL (Adhemerval) Changes in v2: - Add vDSO interface sysdeps/unix/sysv/linux/dl-vdso-setup.c | 10 ++++++++++ sysdeps/unix/sysv/linux/dl-vdso-setup.h | 3 +++ sysdeps/unix/sysv/linux/riscv/hwprobe.c | 4 ++-- sysdeps/unix/sysv/linux/riscv/sysdep.h | 1 + 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/sysdeps/unix/sysv/linux/dl-vdso-setup.c b/sysdeps/unix/sysv/linux/dl-vdso-setup.c index 5dd7ed9d12..3a44944dbb 100644 --- a/sysdeps/unix/sysv/linux/dl-vdso-setup.c +++ b/sysdeps/unix/sysv/linux/dl-vdso-setup.c @@ -71,6 +71,16 @@ PROCINFO_CLASS int (*_dl_vdso_clock_getres_time64) (clockid_t, # ifdef HAVE_GET_TBFREQ PROCINFO_CLASS uint64_t (*_dl_vdso_get_tbfreq)(void) RELRO; # endif + +/* RISC-V specific ones. */ +# ifdef HAVE_RISCV_HWPROBE +PROCINFO_CLASS int (*_dl_vdso_riscv_hwprobe)(void *, + size_t, + size_t, + unsigned long *, + unsigned int) RELRO; +# endif + #endif #undef RELRO diff --git a/sysdeps/unix/sysv/linux/dl-vdso-setup.h b/sysdeps/unix/sysv/linux/dl-vdso-setup.h index e87d886940..8aee5a8212 100644 --- a/sysdeps/unix/sysv/linux/dl-vdso-setup.h +++ b/sysdeps/unix/sysv/linux/dl-vdso-setup.h @@ -47,6 +47,9 @@ setup_vdso_pointers (void) #ifdef HAVE_GET_TBFREQ GLRO(dl_vdso_get_tbfreq) = dl_vdso_vsym (HAVE_GET_TBFREQ); #endif +#ifdef HAVE_RISCV_HWPROBE + GLRO(dl_vdso_riscv_hwprobe) = dl_vdso_vsym (HAVE_RISCV_HWPROBE); +#endif } #endif diff --git a/sysdeps/unix/sysv/linux/riscv/hwprobe.c b/sysdeps/unix/sysv/linux/riscv/hwprobe.c index 28a448175b..e64c159eb3 100644 --- a/sysdeps/unix/sysv/linux/riscv/hwprobe.c +++ b/sysdeps/unix/sysv/linux/riscv/hwprobe.c @@ -28,8 +28,8 @@ int __riscv_hwprobe (struct riscv_hwprobe *pairs, size_t pair_count, { int r; - r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, pairs, pair_count, - cpu_count, cpus, flags); + r = INTERNAL_VSYSCALL (riscv_hwprobe, 5, pairs, pair_count, + cpu_count, cpus, flags); /* Negate negative errno values to match pthreads API. */ return -r; diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h index 5583b96d23..ee015dfeb6 100644 --- a/sysdeps/unix/sysv/linux/riscv/sysdep.h +++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h @@ -156,6 +156,7 @@ /* List of system calls which are supported as vsyscalls (for RV32 and RV64). */ # define HAVE_GETCPU_VSYSCALL "__vdso_getcpu" +# define HAVE_RISCV_HWPROBE "__vdso_riscv_hwprobe" # undef HAVE_INTERNAL_BRK_ADDR_SYMBOL # define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1 From patchwork Wed Feb 14 14:31:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 85730 X-Patchwork-Delegate: palmer@dabbelt.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DB6A63860002 for ; Wed, 14 Feb 2024 14:34:15 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by sourceware.org (Postfix) with ESMTPS id 5752E385E02E for ; Wed, 14 Feb 2024 14:32:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5752E385E02E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5752E385E02E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::431 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921175; cv=none; b=Jd1i6QSoW34VdMW3T27cE/U8dJfBbZn9rF+W1p4KKgTlglGNfC2OFKrh6YTpUzX8jD8Kp9gpNpIiqx0Hh7VfGEypgltYBt1rMNLnmcpj/TlBJwTZ50mwFsnxATM4BZBmeiKTp6BfR8WCfCOrtuxd6ljds8nlmLjeU2umY3tpHuQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921175; c=relaxed/simple; bh=3KEb2aY1qvkZ9DfZSgGgFnctwp76uRNqePMZFCzhuPg=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=JGbIVT70E/0K692OrjUXVpwDVw5FpPsrUuC4AQ7l1Ron0VdhEl89cR0XXQdhdhumYUmzpMV0YDv+56mwFO6f/g8DfizKfOyu0HHRy0yuInJ0Ffwe7P8PSHvzvez8nFK9nXHrWh7xgYOqj9Q82zp/Gd2dIKsu+4YPK8K9elOOEtI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-6e0f4e3bc59so1602128b3a.0 for ; Wed, 14 Feb 2024 06:32:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1707921171; x=1708525971; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wVONlSWH7rFsPXRa1lJYOvQx3bPsSIZvkL8AKiDA2+0=; b=BdDZNNgrKiWqKYVBwaif8QGL6MTdDX4P6oIlbmtMioAG2i0uwPjtHNzRmlTfPe4J+4 Zgtol9eOFBhDwtIi5jERYCG00j3cqyPpcVHMK0gw9K/4cTTPHbUz2B2UaThrSkEY8KNf jpWJ+utV/vjLCOAq60w8ajbE8lymfnngRWfNL/aCtm1GlwrbBxve+AHJTlUCtBz1I42h 3zNpoiAF6Mdci1OAf71taS7YQHOFyebgqMq3ycF+xco48VCX3M3lrBPzbXZ9fFEz9IcC cdeabvq83SUwQ4J1tZiyiBcmzv3cDys3NRqTDjjmtcE8H7EyTSPkrmI2O987UkjQXZQH 8E9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707921171; x=1708525971; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wVONlSWH7rFsPXRa1lJYOvQx3bPsSIZvkL8AKiDA2+0=; b=V8UceSuc+0wMJIh83UqjFebNyO8B9lv9QP7u9OL32Hs2wF9JatrAC8YSTmmnLDVFwi sWrWuYQo8phtI00nYHHQNHlScM1J6Z+CBm7Y3BaC6sllcanuK6defrFlsni8nNTQbepu W5kTVQ2Z7txBAxobqUkdyi3xa3e2Y3s4G3p1YZwXzBVx4Py4zK02umXfIBKZtXndgtJF p6or8r8mhaINEHIS8cohzv8eirdkjZtw2Qi3rv1zjIt90uDm2k6ahkJlFpv3bHmXt7Z5 WCfC5jWhBxH7aAC3n6YS40idpKDJN4uOOGhH8b0JAZnB2Gg6HF15wqSCIa+PXAwwIu6O YkyQ== X-Gm-Message-State: AOJu0YyJo59lJ52JqAnR+RsP3c62199QufRmywXXDwKgLutbBg1goZWg AvG6PQ1o5lEnmG8jYWZDCF/SlyFi6I1A8IKpopUriiy1zYihnAyofjM8zHr9XDuLZKR9CaIgDYt u X-Google-Smtp-Source: AGHT+IE24hI+x1icZ8FasjJYDDvmvSJt1idvSKA6956Tbvyef1T9tTXZkQzp3BdCb1s01q2vkrADDg== X-Received: by 2002:a05:6a20:6f90:b0:19e:304c:5b57 with SMTP id gv16-20020a056a206f9000b0019e304c5b57mr2851313pzb.60.1707921171505; Wed, 14 Feb 2024 06:32:51 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCWfjwduNLixa8i1udwRoMBjPon/ZIuAzxKJXh1f7WsctmGtH5F9vORGw7e/sg5KAZ9cnMXAGs0YSKMoWURrec0r29XyCCWQKMpLf6Fc27scsWakB0VraSqM+I6KfNgpLa3v0TnFo6TzZETU4SvPz8KLuKMdHrmQm++wVHnOfH2v Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id k21-20020a635a55000000b005dc98d9114bsm1140255pgm.43.2024.02.14.06.32.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Feb 2024 06:32:51 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, Florian Weimer , slewis@rivosinc.com, palmer@rivosinc.com, Evan Green Subject: [PATCH v12 4/7] riscv: Add __riscv_hwprobe pointer to ifunc calls Date: Wed, 14 Feb 2024 06:31:55 -0800 Message-Id: <20240214143159.2951158-5-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214143159.2951158-1-evan@rivosinc.com> References: <20240214143159.2951158-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The new __riscv_hwprobe() function is designed to be used by ifunc selector functions. This presents a challenge for applications and libraries, as ifunc selectors are invoked before all relocations have been performed, so an external call to __riscv_hwprobe() from an ifunc selector won't work. To address this, pass a pointer to the __riscv_hwprobe() function into ifunc selectors as the second argument (alongside dl_hwcap, which was already being passed). Include a typedef as well for convenience, so that ifunc users don't have to go through contortions to call this routine. Users will need to remember to check the second argument for NULL, to account for older glibcs that don't pass the function. Signed-off-by: Evan Green --- (no changes since v10) Changes in v10: - Hand in pointer to __riscv_hwprobe(), not vDSO function Changes in v7: - Remove __THROW from function pointer type, as it creates warnings together with __fortified_attr_access. sysdeps/riscv/dl-irel.h | 9 +++++---- sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 10 ++++++++++ 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/sysdeps/riscv/dl-irel.h b/sysdeps/riscv/dl-irel.h index e6ab51ccd4..61b3511c96 100644 --- a/sysdeps/riscv/dl-irel.h +++ b/sysdeps/riscv/dl-irel.h @@ -24,6 +24,7 @@ #include #include #include +#include #define ELF_MACHINE_IRELA 1 @@ -31,10 +32,10 @@ static inline ElfW(Addr) __attribute ((always_inline)) elf_ifunc_invoke (ElfW(Addr) addr) { - /* The second argument is a void pointer to preserve the extension - fexibility. */ - return ((ElfW(Addr) (*) (uint64_t, void *)) (addr)) - (GLRO(dl_hwcap), NULL); + /* The third argument is a void pointer to preserve the extension + flexibility. */ + return ((ElfW(Addr) (*) (uint64_t, void *, void *)) (addr)) + (GLRO(dl_hwcap), __riscv_hwprobe, NULL); } static inline void diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h index 5592b9e100..34a2e3dbc2 100644 --- a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -69,6 +69,16 @@ extern int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, __fortified_attr_access (__read_write__, 1, 2) __fortified_attr_access (__read_only__, 4, 3); +/* A pointer to the __riscv_hwprobe vDSO function is passed as the second + argument to ifunc selector routines. Include a function pointer type for + convenience in calling the function in those settings. */ +typedef int (*__riscv_hwprobe_t) (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) + __nonnull ((1)) __wur + __fortified_attr_access (__read_write__, 1, 2) + __fortified_attr_access (__read_only__, 4, 3); + __END_DECLS #endif /* sys/hwprobe.h */ From patchwork Wed Feb 14 14:31:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 85729 X-Patchwork-Delegate: palmer@dabbelt.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A7CEF385E445 for ; Wed, 14 Feb 2024 14:33:53 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-il1-x130.google.com (mail-il1-x130.google.com [IPv6:2607:f8b0:4864:20::130]) by sourceware.org (Postfix) with ESMTPS id 66994385E02F for ; Wed, 14 Feb 2024 14:32:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 66994385E02F Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 66994385E02F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::130 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921179; cv=none; b=VQ18miTpw5GxVVuwrpdIx1c9R3YCOxdJ3C8apO715uJepIs3vm7VPX28RI6uP0ipeGSiLa6+rujAb4JrnQJMkgFfAVd90S/CBvfLDQlGahozURRpR+g51ns5SvXrgOADpY+LAfnb5LxYEIUqF7c6v//Xbvzn/na0Uoaqhp0BCss= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921179; c=relaxed/simple; bh=UnKyy1ukjSKYqVd90v+CtuJef8i2fEiXmdUVIwiOHko=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=bLqVs8VMfw6XpgKHMt8GCgtRi+p0eq61Yg92yRB+NdWyN1vdoiC8H+NfVvmc7wLX8RbiI6GHFUVZW9P02hmsPJ8C/AFzmZ1XLbbzoSROmKYh0qnhBrew28aZ5Vvih9dUNslM36k2I4sMD5saTyMtV8CF7Gu6WJBdc6KtqgUnsEI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-il1-x130.google.com with SMTP id e9e14a558f8ab-363d85fda93so18267305ab.1 for ; Wed, 14 Feb 2024 06:32:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1707921175; x=1708525975; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gojGF3ao4MuuQ17fKclkOOLHiBX3fJLJqwJjkB2z6QU=; b=FeAjgLvjKwKvVfN1xElTz2mVqK1K/hiLPfxJ2Lwwkzh0vL/yZPW1mFt0Li48Iv+5YB 4kjjEofZLqKKlSOLihTRi2sVjFnU5h4gxA471l0D7yJ7jqNRws2v3ciomZcbHHkg3mmR C2YOlA6VuiqOqiK8SeCM9YCCk1j5UvM0z9Cwvm+k1ydr0rwrC10MRItjMcirZxKipysL b0FPX8VuRILgAarxD5L5FdzTb3eX5jb1uCTKZ4T3HUEPWEq/ZSqhPGjb7jIvc55pZeUl EnGALzRI6MQFas7RO2TF0ISqunhA2ljJmvMplLsu92gfGtgsDMZekdhT0a8DWsZN3+u9 73ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707921175; x=1708525975; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gojGF3ao4MuuQ17fKclkOOLHiBX3fJLJqwJjkB2z6QU=; b=i81ozNzrfScCJk1SHrEZpIp9YDW4wE8GpIfZnUWITvFh/uNnfPBYl2WurhQslc184K iC9JbVMDJA/J4DwLWKlOg+8BcfjdmXzipNU2MhZ0D387PS8t4H86cZAJp/h9q7vfHRwK dB1UXSwkKf1OhZLy7oQsCxHtBXhsQvRtWFMAOHqdWhCNv4dkDixfW6uFbNewAC3FC/oU x02yRY8/tsFOswzkc/Uvtiq84a1cnasKuEoyHF6f2kT7p/8lVmUPDD5bZUNsFR13KuOq wJDrVsGv1JY6w4nfoO8KoVElj3EHUVWfZnIiTEXZ2YJvrnuau9fRocLeysXsKAAL6UpE 8WWA== X-Gm-Message-State: AOJu0YxA1soOUQ+uwDt0I7pD9KYDvfllwVtF0HBSZBIM+LW2ak0w02n1 jYUsvxRCZ/iof/eD+HI/IrKLaOmGBU916iHPiOVFvzPemMKoJ3Ori/S2ObIF94l5aljHei5gsnH 2 X-Google-Smtp-Source: AGHT+IHUKd89IcbC3jb4yQSGSQI10Ij2heD3Enz0uDgkS/1ns4zHRDo9MlWEGp8RAGcCm6lJg/6g0A== X-Received: by 2002:a05:6e02:80e:b0:363:bc58:1ebd with SMTP id u14-20020a056e02080e00b00363bc581ebdmr3089326ilm.10.1707921175058; Wed, 14 Feb 2024 06:32:55 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCWAe7tG2kjKtOLj/vhaEo0I6gP+MsqR+45IHA91kTTDzNbM317Lio0yykSiiA48BWDJu9HVtFZCSV6B/14R20OiTOFtLZn+Sg1HcXPNE5D0RmCp9by9iuhqk4OVQEaXkQ8CajXcaEISOFnPyFPPiZ25lMLy8sRQydAeyuDGXJel Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id k21-20020a635a55000000b005dc98d9114bsm1140255pgm.43.2024.02.14.06.32.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Feb 2024 06:32:54 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, Florian Weimer , slewis@rivosinc.com, palmer@rivosinc.com, Evan Green Subject: [PATCH v12 5/7] riscv: Enable multi-arg ifunc resolvers Date: Wed, 14 Feb 2024 06:31:56 -0800 Message-Id: <20240214143159.2951158-6-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214143159.2951158-1-evan@rivosinc.com> References: <20240214143159.2951158-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, URIBL_BLACK autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org RISC-V is apparently the first architecture to pass more than one argument to ifunc resolvers. The helper macros in libc-symbols.h, __ifunc_resolver(), __ifunc(), and __ifunc_hidden(), are incompatible with this. These macros have an "arg" (non-final) parameter that represents the parameter signature of the ifunc resolver. The result is an inability to pass the required comma through in a single preprocessor argument. Rearrange the __ifunc_resolver() macro to be variadic, and pass the types as those variable parameters. Move the guts of __ifunc() and __ifunc_hidden() into new macros, __ifunc_args(), and __ifunc_args_hidden(), that pass the variable arguments down through to __ifunc_resolver(). Then redefine __ifunc() and __ifunc_hidden(), which are used in a bunch of places, to simply shuffle the arguments down into __ifunc_args[_hidden]. Finally, define a riscv-ifunc.h header, which provides convenience macros to those looking to write ifunc selectors that use both arguments. Signed-off-by: Evan Green Reviewed-by: Florian Weimer --- (no changes since v11) Changes in v11: - Update copyright year (Adhemerval) Changes in v9: - Fix a couple of typos causing powerpc not to build (build-many-glibcs) Changes in v6: - Introduced riscv-ifunc.h for multi-arg ifunc selectors. --- include/libc-symbols.h | 28 +++++++++++++++++----------- sysdeps/riscv/riscv-ifunc.h | 27 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 11 deletions(-) create mode 100644 sysdeps/riscv/riscv-ifunc.h diff --git a/include/libc-symbols.h b/include/libc-symbols.h index e21bb599b3..4367aa6740 100644 --- a/include/libc-symbols.h +++ b/include/libc-symbols.h @@ -667,9 +667,9 @@ for linking") #endif /* Helper / base macros for indirect function symbols. */ -#define __ifunc_resolver(type_name, name, expr, arg, init, classifier) \ +#define __ifunc_resolver(type_name, name, expr, init, classifier, ...) \ classifier inhibit_stack_protector \ - __typeof (type_name) *name##_ifunc (arg) \ + __typeof (type_name) *name##_ifunc (__VA_ARGS__) \ { \ init (); \ __typeof (type_name) *res = expr; \ @@ -677,13 +677,13 @@ for linking") } #ifdef HAVE_GCC_IFUNC -# define __ifunc(type_name, name, expr, arg, init) \ +# define __ifunc_args(type_name, name, expr, init, ...) \ extern __typeof (type_name) name __attribute__ \ ((ifunc (#name "_ifunc"))); \ - __ifunc_resolver (type_name, name, expr, arg, init, static) + __ifunc_resolver (type_name, name, expr, init, static, __VA_ARGS__) -# define __ifunc_hidden(type_name, name, expr, arg, init) \ - __ifunc (type_name, name, expr, arg, init) +# define __ifunc_args_hidden(type_name, name, expr, init, ...) \ + __ifunc_args (type_name, name, expr, init, __VA_ARGS__) #else /* Gcc does not support __attribute__ ((ifunc (...))). Use the old behaviour as fallback. But keep in mind that the debug information for the ifunc @@ -694,18 +694,24 @@ for linking") different signatures. (Gcc support is disabled at least on a ppc64le Ubuntu 14.04 system.) */ -# define __ifunc(type_name, name, expr, arg, init) \ +# define __ifunc_args(type_name, name, expr, init, ...) \ extern __typeof (type_name) name; \ - __typeof (type_name) *name##_ifunc (arg) __asm__ (#name); \ - __ifunc_resolver (type_name, name, expr, arg, init,) \ + __typeof (type_name) *name##_ifunc (__VA_ARGS__) __asm__ (#name); \ + __ifunc_resolver (type_name, name, expr, init, , __VA_ARGS__) \ __asm__ (".type " #name ", %gnu_indirect_function"); -# define __ifunc_hidden(type_name, name, expr, arg, init) \ +# define __ifunc_args_hidden(type_name, name, expr, init, ...) \ extern __typeof (type_name) __libc_##name; \ - __ifunc (type_name, __libc_##name, expr, arg, init) \ + __ifunc (type_name, __libc_##name, expr, __VA_ARGS__, init) \ strong_alias (__libc_##name, name); #endif /* !HAVE_GCC_IFUNC */ +#define __ifunc(type_name, name, expr, arg, init) \ + __ifunc_args (type_name, name, expr, init, arg) + +#define __ifunc_hidden(type_name, name, expr, arg, init) \ + __ifunc_args_hidden (type_name, name, expr, init, arg) + /* The following macros are used for indirect function symbols in libc.so. First of all, you need to have the function prototyped somewhere, say in foo.h: diff --git a/sysdeps/riscv/riscv-ifunc.h b/sysdeps/riscv/riscv-ifunc.h new file mode 100644 index 0000000000..c77ab51548 --- /dev/null +++ b/sysdeps/riscv/riscv-ifunc.h @@ -0,0 +1,27 @@ +/* Common definition for ifunc resolvers. Linux/RISC-V version. + This file is part of the GNU C Library. + Copyright (C) 2024 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include + +#define INIT_ARCH() + +#define riscv_libc_ifunc(name, expr) \ + __ifunc_args (name, name, expr(hwcap, hwprobe), INIT_ARCH, \ + uint64_t hwcap, __riscv_hwprobe_t hwprobe) From patchwork Wed Feb 14 14:31:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 85732 X-Patchwork-Delegate: palmer@dabbelt.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7664E385E02E for ; Wed, 14 Feb 2024 14:34:57 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by sourceware.org (Postfix) with ESMTPS id 82A87385E006 for ; Wed, 14 Feb 2024 14:32:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 82A87385E006 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 82A87385E006 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::529 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921182; cv=none; b=rYy5xQzEwza6h17KK1dTujYeBEDCtcjm2DLJvM8xMrknSDzDFyS/0Io6eYM01PqX6DR5j6y9+E0FTSrcaDBWVp1cCoaVEwLizET9L55mBbOEfFvGrh4H+PiqPrm9gME3788tcZcY7l7eyI8YEPyLldzK2g5pfI+IwHjuLLR/fRM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921182; c=relaxed/simple; bh=9HKwofNDtM3mQbdw5HQr4IwgKqiNwFrv7q8+/y8NToQ=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=ghEzOX50hs+psByJahUe22BOL+R7zohr1qPGWEhUcRrfbeU81Ks5m733SfPIdxNM2T1LFXcUvKXfkH+p/urg/GxQq0a71N/zRsWcUfc0T1WkmXX/JuIbnh57NRYyp4zKdkuOVtu+fBtQJyZrsM8DdOaRJTlq4a1aFu5n224izFg= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-5d8ddbac4fbso4781568a12.0 for ; Wed, 14 Feb 2024 06:32:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1707921178; x=1708525978; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PmGUIHJh6VHAV6X1wPidNHCpvUG1fN4EMb5F0/4NlFc=; b=MeBWesYKaP+WlKDJ1GoZDmmMdv1xDy57UYh00sfs3HYCYkGq/7do1f0qHMQKbM6I6G 0+L1jr7F9D3m9mtUd34RwJXieKuJtzbmS9BUQXj3hmxt2qU9H97zD58/n5nmfmxr8hB9 fCX78ii83nlJrHwMKzFbr0pilsxqqWNhiwKZFCpW3ZRdUrB14RUh2wDkSd5UbEIjDV4T 9IUZJY0L9JTYj0loKDA+eEuBf8FqA1O8rXJa7bsniTCFUkI5NWKzxCeXsTblSougeKWy eVN5NZbdxX2npDsbbSm3GcuAqvOfqkZYDDVLVBxwxjilho+q+JfZ92IJvB3AxpSEvjgA aeGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707921178; x=1708525978; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PmGUIHJh6VHAV6X1wPidNHCpvUG1fN4EMb5F0/4NlFc=; b=Ffvgxbjb9gp11iBiNX5kKnf1Zvoerw5Vy9TBh9MjR3b1IA13BDitiU/7W+hx67Hf2G GtfMHnEzw3LhPQ2rrEJh+I7UCRl9X8BhXMIfPoqfGNc44CkHV7DLFMPSJbjsjLflcxPu 7H3KJ5FEi3DG6PRVP2PAc2h5AekEm42R7InOeT3jcpKFmQrZ/DQh9iK11KE2Iui8ACt9 9btU1su9aLNCt2FZLrKVJOZiF00o2Qf4MMWBPKi6o060ZWinIAJShbwepFVSZ23cXcwK NEyOC8P9u3Wis5/DQXBgv+L1sOyua2eKopsNb/pUws+rJ57k8uS1+W5bTon7SIIUoVNt RsOQ== X-Gm-Message-State: AOJu0Yyn5wcagbakSCoXPLq9ku9Y1HXENn2LoParda3nRf4v5BhDobw6 tfdFDfnHa6MwhBRwHvrxF/aw56D69IqyDYMYAnXp4jTpeudH4wVy8Xi2zJyHkMCRQmgLpr3thrt P X-Google-Smtp-Source: AGHT+IHktMeO2BzPH7/ZPuTxVKQsGWN7To653ISkTJMioAGpyMcjmixM85oxU140VNvz+qAJfQDhgg== X-Received: by 2002:a17:90a:c585:b0:298:b784:c72a with SMTP id l5-20020a17090ac58500b00298b784c72amr2786823pjt.23.1707921178083; Wed, 14 Feb 2024 06:32:58 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCWX+vMbYV+aiB+S1RiH1OJaujJ+qre7+Ma3dYil90Uk8LK1MvnQnaz8eQ4ZA3jzeScaolIHgAzZNWavTXN6cfWE21+4FuP1/ojq6gOo7XLCwE7SshP8Q53XJHS9Pits7QheiQFp8gAAINAlmc6hw8WsLtgPb20Bf1jAYAH/tdeU Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id k21-20020a635a55000000b005dc98d9114bsm1140255pgm.43.2024.02.14.06.32.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Feb 2024 06:32:57 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, Florian Weimer , slewis@rivosinc.com, palmer@rivosinc.com, Evan Green Subject: [PATCH v12 6/7] riscv: Add ifunc helper method to hwprobe.h Date: Wed, 14 Feb 2024 06:31:57 -0800 Message-Id: <20240214143159.2951158-7-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214143159.2951158-1-evan@rivosinc.com> References: <20240214143159.2951158-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add a little helper method so it's easier to fetch a single value from the hwprobe function when used within an ifunc selector. Signed-off-by: Evan Green --- (no changes since v11) Changes in v11: - Remove superfluous 'signed' (Adhemerval) - Put helper before __END_DECLS (Adhemerval) - Add comment about +1 on non-Linux systems (Stefan) Changes in v10: - Avoid implicit comparisons (Adhemerval) Changes in v9: - Use __inline rather than inline so c89 compiles (build-many-glibcs) Changes in v7: - Introduced static inline helper (Richard) sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 29 +++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h index 34a2e3dbc2..8ecb43bb69 100644 --- a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -22,6 +22,7 @@ #include #include +#include #ifdef __has_include # if __has_include () # include @@ -79,6 +80,34 @@ typedef int (*__riscv_hwprobe_t) (struct riscv_hwprobe *__pairs, size_t __pair_c __fortified_attr_access (__read_write__, 1, 2) __fortified_attr_access (__read_only__, 4, 3); +/* Helper function usable from ifunc selectors that probes a single key. */ +static __inline int +__riscv_hwprobe_one(__riscv_hwprobe_t hwprobe_func, + long long int key, + unsigned long long int *value) +{ + struct riscv_hwprobe pair; + int rc; + + /* Earlier versions of glibc pass NULL as the second ifunc parameter. Other C + libraries on non-Linux systems may pass +1 as this function pointer to + indicate no support. Users copying this function to exotic worlds + (non-Linux non-glibc) may want to do additional validity checks here. */ + if (hwprobe_func == NULL) + return ENOSYS; + + pair.key = key; + rc = hwprobe_func (&pair, 1, 0, NULL, 0); + if (rc != 0) + return rc; + + if (pair.key < 0) + return ENOENT; + + *value = pair.value; + return 0; +} + __END_DECLS #endif /* sys/hwprobe.h */ From patchwork Wed Feb 14 14:31:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 85731 X-Patchwork-Delegate: palmer@dabbelt.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2BCF2385E446 for ; Wed, 14 Feb 2024 14:34:34 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by sourceware.org (Postfix) with ESMTPS id 1F1793860006 for ; Wed, 14 Feb 2024 14:33:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 1F1793860006 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 1F1793860006 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::52c ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921186; cv=none; b=n4DO5Y6QxOtYs1wJ8JpZmRX+KY7MGlyY+SA/gMrpg2Cr70+0wIlYljaS/io2ypUT0uE5lnsWSH6u0D/5QePuAnk7mofnVLRRP+4H51WbZF4m/cXZrzVle++EmmYeSQtXJ2nt5rYQX9G0IhJnemdL9uFmnXl95Zxuila3ghgwPrU= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707921186; c=relaxed/simple; bh=RQeCsTdLzL95jS0ANE2Z4YVo2AodGkOejdkJeicqBls=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=sRmi1YtWFc3FXeGIhwWRn2i3C2qRCqxDPD0pBkusPqrtqEFStsLLEsphbcsK4+KWlb0vrVwKEIyiVKAOCvfvN4KdfyPWy3zr2FD00NffH7iaAn1PmjMwSqoXTOSt63PwZPcRhn3BfNJXL11ajRuQpPhuK9p7gjaSIQHjJdOgagk= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-5d8ddbac4fbso4781630a12.0 for ; Wed, 14 Feb 2024 06:33:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1707921181; x=1708525981; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d0YVHUk/lddbP97jJ8T2SQlEKI7riLRj+3PwAqGif+0=; b=mSDcxn9Pr5TerLMEqM/8Rd2/LSr9YpQlDAiq4PsUWiAMB5gp4mfLtc1s4ZPGG2FOFD 3KlkN2PvJUro2khXs2NTam41h63OEpDQ6j5KRv6dlkNneZVR2ZsM3/fjau7lY9AtcfWa MNOF69amM29RiatWEAv6QvPl1Yfq2hiPQOdiuHnD3QnpOuR4qUXI4CzKgP5V9sg7fmip FZrHMGTFbx5ebd3kPVt6KpRx6d9F53D0yfCAMkVJ4rBtI5CMBXv7fnOFEvKM/yBB2mLj CuRhA9RMRlGU7VHcjHNT7lDO1EStW62PFxPRSJQOSKxTy9UuNXETEg4ullEIlj4MfT7A P1Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707921181; x=1708525981; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d0YVHUk/lddbP97jJ8T2SQlEKI7riLRj+3PwAqGif+0=; b=K24Wj6wcN8Sis33+DzNM+IcBkb/NyGftaZ4dS07b5EZSx4imYapVAiMIBTGHQCb2H5 ankM0MIAP57mF+eWr6OKX1ywdeDS8IMPpOeCItTX7/rWsEjhmHR4we08dhweX1RASQcT wxOumUrI7YtiZn72eHFYp1uLaC+Gi1ECr1ShKb8y8DuviVk8w+jIA2Z3/ZsDl4MQpcQg cO7+j1Uk0nuU1K/72UhJ6uVav9LhzwcFemBqvsxGkfLBpDPSE89QMsQe1iDNakJsRU1Y 1lt+wa+fXQjEsxZ6Mcyo9iCrxfbOOxrvKfPJ4MKX+LPxWrm3UuZNHoZzctYnXvFL+g2v VJaA== X-Gm-Message-State: AOJu0Yx4UmAjpvkb8y/xp7HdUd2T7dBhoz2jrY/tSd04PLepyAhZqoVg 6uJGKByQD9KAnib0czWuATXibzqfNbesyfkhr9+PBOO0Mw/M+mOwnRBpEdJe+j21UGQ1SCV1jRb E X-Google-Smtp-Source: AGHT+IFzzNZL1XPE9/kAZMz4/GPSFCyGsUMYMfYcAE57C3t8E7z48d2+4YZkKMcuQEv0IKTpLqJX8g== X-Received: by 2002:a17:90b:450b:b0:298:bca7:c530 with SMTP id iu11-20020a17090b450b00b00298bca7c530mr2622850pjb.42.1707921181157; Wed, 14 Feb 2024 06:33:01 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVmCbKrBganTHqPp3CdP5eN0uiq8UZQLdFfg6rqSWesgabaM3qJo8WhsLwpxkVsPKoBOGsnZbcAt+RNwnALQUb5fF/EbGOJUT4QVMdDD6r+9nuUsNolPqiTnQwceRqYjHGaFIvKeYYntMTIodg36/lQuFM03GT6wqzn/VwQB8xs Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id k21-20020a635a55000000b005dc98d9114bsm1140255pgm.43.2024.02.14.06.33.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Feb 2024 06:33:00 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, Florian Weimer , slewis@rivosinc.com, palmer@rivosinc.com, Evan Green Subject: [PATCH v12 7/7] riscv: Add and use alignment-ignorant memcpy Date: Wed, 14 Feb 2024 06:31:58 -0800 Message-Id: <20240214143159.2951158-8-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214143159.2951158-1-evan@rivosinc.com> References: <20240214143159.2951158-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org For CPU implementations that can perform unaligned accesses with little or no performance penalty, create a memcpy implementation that does not bother aligning buffers. It will use a block of integer registers, a single integer register, and fall back to bytewise copy for the remainder. Signed-off-by: Evan Green Reviewed-by: Palmer Dabbelt --- (no changes since v11) Changes in v11: - Update copyrights to 2024 (Adhemerval) - Avoid implicit check in select_memcpy_ifunc (Adhemerval) - Remove hidden_def (__memcpy_noalignment) (Adhemerval) Changes in v10: - One line per function in Makefile for memcpy (Adhemerval) - Space before argument-like things (Adhemerval) Changes in v7: - Use new helper function in memcpy ifunc selector (Richard) Changes in v6: - Fix a couple regressions in the assembly from v5 :/ - Use passed hwprobe pointer in memcpy ifunc selector. Changes in v5: - Do unaligned word access for final trailing bytes (Richard) Changes in v4: - Fixed comment style (Florian) Changes in v3: - Word align dest for large memcpy()s. - Add tags - Remove spurious blank line from sysdeps/riscv/memcpy.c Changes in v2: - Used _MASK instead of _FAST value itself. --- sysdeps/riscv/memcopy.h | 26 ++++ sysdeps/riscv/memcpy.c | 63 ++++++++ sysdeps/riscv/memcpy_noalignment.S | 136 ++++++++++++++++++ sysdeps/unix/sysv/linux/riscv/Makefile | 9 ++ .../unix/sysv/linux/riscv/memcpy-generic.c | 24 ++++ 5 files changed, 258 insertions(+) create mode 100644 sysdeps/riscv/memcopy.h create mode 100644 sysdeps/riscv/memcpy.c create mode 100644 sysdeps/riscv/memcpy_noalignment.S create mode 100644 sysdeps/unix/sysv/linux/riscv/memcpy-generic.c diff --git a/sysdeps/riscv/memcopy.h b/sysdeps/riscv/memcopy.h new file mode 100644 index 0000000000..27675964b0 --- /dev/null +++ b/sysdeps/riscv/memcopy.h @@ -0,0 +1,26 @@ +/* memcopy.h -- definitions for memory copy functions. RISC-V version. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +/* Redefine the generic memcpy implementation to __memcpy_generic, so + the memcpy ifunc can select between generic and special versions. + In rtld, don't bother with all the ifunciness. */ +#if IS_IN (libc) +#define MEMCPY __memcpy_generic +#endif diff --git a/sysdeps/riscv/memcpy.c b/sysdeps/riscv/memcpy.c new file mode 100644 index 0000000000..20f9548c44 --- /dev/null +++ b/sysdeps/riscv/memcpy.c @@ -0,0 +1,63 @@ +/* Multiple versions of memcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017-2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memcpy so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memcpy +# define memcpy __redirect_memcpy +# include +# include +# include +# include +# include + +# define INIT_ARCH() + +extern __typeof (__redirect_memcpy) __libc_memcpy; + +extern __typeof (__redirect_memcpy) __memcpy_generic attribute_hidden; +extern __typeof (__redirect_memcpy) __memcpy_noalignment attribute_hidden; + +static inline __typeof (__redirect_memcpy) * +select_memcpy_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long int value; + + INIT_ARCH (); + + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_CPUPERF_0, &value) != 0) + return __memcpy_generic; + + if ((value & RISCV_HWPROBE_MISALIGNED_MASK) == RISCV_HWPROBE_MISALIGNED_FAST) + return __memcpy_noalignment; + + return __memcpy_generic; +} + +riscv_libc_ifunc (__libc_memcpy, select_memcpy_ifunc); + +# undef memcpy +strong_alias (__libc_memcpy, memcpy); +# ifdef SHARED +__hidden_ver1 (memcpy, __GI_memcpy, __redirect_memcpy) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memcpy); +# endif + +#endif diff --git a/sysdeps/riscv/memcpy_noalignment.S b/sysdeps/riscv/memcpy_noalignment.S new file mode 100644 index 0000000000..621f8d028f --- /dev/null +++ b/sysdeps/riscv/memcpy_noalignment.S @@ -0,0 +1,136 @@ +/* memcpy for RISC-V, ignoring buffer alignment + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#include +#include + +/* void *memcpy(void *, const void *, size_t) */ +ENTRY (__memcpy_noalignment) + move t6, a0 /* Preserve return value */ + + /* Bail if 0 */ + beqz a2, 7f + + /* Jump to byte copy if size < SZREG */ + li a4, SZREG + bltu a2, a4, 5f + + /* Round down to the nearest "page" size */ + andi a4, a2, ~((16*SZREG)-1) + beqz a4, 2f + add a3, a1, a4 + + /* Copy the first word to get dest word aligned */ + andi a5, t6, SZREG-1 + beqz a5, 1f + REG_L a6, (a1) + REG_S a6, (t6) + + /* Align dst up to a word, move src and size as well. */ + addi t6, t6, SZREG-1 + andi t6, t6, ~(SZREG-1) + sub a5, t6, a0 + add a1, a1, a5 + sub a2, a2, a5 + + /* Recompute page count */ + andi a4, a2, ~((16*SZREG)-1) + beqz a4, 2f + +1: + /* Copy "pages" (chunks of 16 registers) */ + REG_L a4, 0(a1) + REG_L a5, SZREG(a1) + REG_L a6, 2*SZREG(a1) + REG_L a7, 3*SZREG(a1) + REG_L t0, 4*SZREG(a1) + REG_L t1, 5*SZREG(a1) + REG_L t2, 6*SZREG(a1) + REG_L t3, 7*SZREG(a1) + REG_L t4, 8*SZREG(a1) + REG_L t5, 9*SZREG(a1) + REG_S a4, 0(t6) + REG_S a5, SZREG(t6) + REG_S a6, 2*SZREG(t6) + REG_S a7, 3*SZREG(t6) + REG_S t0, 4*SZREG(t6) + REG_S t1, 5*SZREG(t6) + REG_S t2, 6*SZREG(t6) + REG_S t3, 7*SZREG(t6) + REG_S t4, 8*SZREG(t6) + REG_S t5, 9*SZREG(t6) + REG_L a4, 10*SZREG(a1) + REG_L a5, 11*SZREG(a1) + REG_L a6, 12*SZREG(a1) + REG_L a7, 13*SZREG(a1) + REG_L t0, 14*SZREG(a1) + REG_L t1, 15*SZREG(a1) + addi a1, a1, 16*SZREG + REG_S a4, 10*SZREG(t6) + REG_S a5, 11*SZREG(t6) + REG_S a6, 12*SZREG(t6) + REG_S a7, 13*SZREG(t6) + REG_S t0, 14*SZREG(t6) + REG_S t1, 15*SZREG(t6) + addi t6, t6, 16*SZREG + bltu a1, a3, 1b + andi a2, a2, (16*SZREG)-1 /* Update count */ + +2: + /* Remainder is smaller than a page, compute native word count */ + beqz a2, 7f + andi a5, a2, ~(SZREG-1) + andi a2, a2, (SZREG-1) + add a3, a1, a5 + /* Jump directly to last word if no words. */ + beqz a5, 4f + +3: + /* Use single native register copy */ + REG_L a4, 0(a1) + addi a1, a1, SZREG + REG_S a4, 0(t6) + addi t6, t6, SZREG + bltu a1, a3, 3b + + /* Jump directly out if no more bytes */ + beqz a2, 7f + +4: + /* Copy the last word unaligned */ + add a3, a1, a2 + add a4, t6, a2 + REG_L a5, -SZREG(a3) + REG_S a5, -SZREG(a4) + ret + +5: + /* Copy bytes when the total copy is . */ + +#include + +extern __typeof (memcpy) __memcpy_generic; +hidden_proto (__memcpy_generic) + +#include