From patchwork Wed Dec 13 21:11:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 82088 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E1C8F385C40E for ; Wed, 13 Dec 2023 21:12:09 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oo1-xc29.google.com (mail-oo1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) by sourceware.org (Postfix) with ESMTPS id 48C923858003 for ; Wed, 13 Dec 2023 21:11:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 48C923858003 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 48C923858003 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::c29 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501918; cv=none; b=bo3nAujBLqET29DoOmWIA69Pp4PYVW/gV6PXDMrX+Izdph1yvl2S/Dca+fKzB6SpNYBZEVoDtqBrdT47TZ4WLOvm6diIeup2CAjFXs2hty46guctaxW4ds8uGrp7Mhc8Bonxp3Nm0dGszyFaVPHG5xlSMq8Drya2uoanOU1ylqU= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501918; c=relaxed/simple; bh=zPQvXavsIFa8FToIA2nPa63fToftTKbPRW7psJgb0rQ=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=rqDMbR8vBu17OSyv7M7v9C6cjcTlCsYpKl/Ca/lpb0XorRLx03mjyCoMBzAmz7gjATtnVe6b0KFBzMnzhw5T1Yo2byt05JuJGW5IiacedylNXL6vwShYjUihyArp/Am0Rwg6MvuqzEybuuOJg8ctoaG8Ssl+ptLYIE9T4ZK3sFQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oo1-xc29.google.com with SMTP id 006d021491bc7-58dd3528497so4618529eaf.3 for ; Wed, 13 Dec 2023 13:11:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1702501914; x=1703106714; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=H6tR/CIoxSHbSfthxRkAfVpq0Z0KZVFHSV/sgop7Now=; b=JTupxXgZlSZIWBTpcvQrg1URBKs+Skgn56H0u5vB3o7jVspzo+negjbPkKleprIVss kempudufBiV081HNZEezcXqCef9mxFGR8Slc/g6BvhQrpSYeKLmGwSWqfAicfufxg9H3 vGHFc7JnqKF+s/oza1uxOCTNbZsgt9W4DCqJjpc4bxDW43tRhB57VzeZb2ax6bBXMwM8 eCKbsWQy1184nsUBMEGvhV5j33W7I94yCXOAF+xlH4Jq7TcZcbkocu25GrhbROnXXwI8 NwiGI/KRtJm9Vop44dH4jeE5bq4NhWLkNj2rAHsZkZzVzLcI6W4IkGgC2/KclF7rUtap Jydw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702501914; x=1703106714; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H6tR/CIoxSHbSfthxRkAfVpq0Z0KZVFHSV/sgop7Now=; b=NfEazvzZMVDNok33pV1SgPC+93m+vKiDN3nlUWLKaMgD6zTdVAcPkbqIz8XS8FRpAq sYSeThomz7L/WifuTURNIGHn+G4zlNJccf+HZBD4OmsQuAxMZQIJMmUcZ3JzSSlkpm6m DSfZcYYU4e6mNq+H/5tigowBFDxrUyz1ASCFnwXbDbvQPriBlpIzpGvOA3oKWDeN+Ffm gDmvc3WcQ28sfYk1BSBqb9Ygv7QDV7es5gIxaDOFb836q3cuMa4zka4ff6MlKVl+e1+u 5oweOU8EQby4ny04iyuXBOBi7/3TTN6jnuytosOExu5aQHvGeVET8t1BmGN/nxrjJBar tBYA== X-Gm-Message-State: AOJu0YxwtDQ8GOhVDADlGOPZ46loPXj0VGsRZL1vpqSXIjOmp3dZTbYX PCCzLHxmpjE0UBhnr4O3rZzTlOze1Iu0nxrbsIY= X-Google-Smtp-Source: AGHT+IEPSRVn1k7dPnuEiLWM7nIjW+DFFUVSZiH2iPGcJeUw4QhsYi8IPEdQoE11dodrYFohThUU1w== X-Received: by 2002:a4a:8514:0:b0:58e:1fa2:49c with SMTP id k20-20020a4a8514000000b0058e1fa2049cmr6489935ooh.8.1702501914610; Wed, 13 Dec 2023 13:11:54 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id c15-20020a4a380f000000b0058d1de21893sm3223109ooa.24.2023.12.13.13.11.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 13:11:54 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, palmer@rivosinc.com, Florian Weimer , Evan Green Subject: [PATCH v10 1/7] riscv: Add Linux hwprobe syscall support Date: Wed, 13 Dec 2023 13:11:36 -0800 Message-Id: <20231213211142.1543025-2-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231213211142.1543025-1-evan@rivosinc.com> References: <20231213211142.1543025-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add awareness and a thin wrapper function around a new Linux system call that allows callers to get architecture and microarchitecture information about the CPUs from the kernel. This can be used to do things like dynamically choose a memcpy implementation. Signed-off-by: Evan Green --- Changes in v10: - Remove spurious 5 from syscall patch (Adhemerval) - Use one item per line in Makefile (Adhemerval) - Remove double underscores from __riscv_hwprobe definition (Adhemerval) - Use only spaces in macro definitions of hwprobe.h (Adhemerval) Changes in v9: - Alphabetize new entries in libc.abilist (to pass checks) Changes in v8: - Fix missed 2.39 in abilists (Joseph) - Just return -r (Florian) Changes in v7: - Bumped Versions up to 2.39 (Joseph) - Used INTERNAL_SYSCALL_CALL, and return positive errno to match pthreads API (Florian). - Remove __THROW since it creates a warning in combination with the fortified access decorators. Changes in v6: - Prefixed __riscv_hwprobe() parameters names with __ to avoid user macro namespace pollution (Joseph) Changes in v4: - Remove __USE_GNU (Florian) - __nonnull, __wur, __THROW, and __fortified_attr_access decorations (Florian) - change long to long int (Florian) - Fix comment formatting (Florian) - Update backup kernel header content copy. - Fix function declaration formatting (Florian) - Changed export versions to 2.38 Changes in v3: - Update argument types to match v4 kernel interface Changes in v2: - hwprobe.h: Use __has_include and duplicate Linux content to make compilation work when Linux headers are absent (Adhemerval) - hwprobe.h: Put declaration under __USE_GNU (Adhemerval) - Use INLINE_SYSCALL_CALL (Adhemerval) - Update versions - Update UNALIGNED_MASK to match kernel v3 series. sysdeps/unix/sysv/linux/riscv/Makefile | 12 ++- sysdeps/unix/sysv/linux/riscv/Versions | 3 + sysdeps/unix/sysv/linux/riscv/hwprobe.c | 36 +++++++++ .../unix/sysv/linux/riscv/rv32/libc.abilist | 1 + .../unix/sysv/linux/riscv/rv64/libc.abilist | 1 + sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 74 +++++++++++++++++++ 6 files changed, 125 insertions(+), 2 deletions(-) create mode 100644 sysdeps/unix/sysv/linux/riscv/hwprobe.c create mode 100644 sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h diff --git a/sysdeps/unix/sysv/linux/riscv/Makefile b/sysdeps/unix/sysv/linux/riscv/Makefile index 4b6eacb32f..04abf226ad 100644 --- a/sysdeps/unix/sysv/linux/riscv/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/Makefile @@ -1,6 +1,14 @@ ifeq ($(subdir),misc) -sysdep_headers += sys/cachectl.h -sysdep_routines += flush-icache +sysdep_headers += \ + sys/cachectl.h \ + sys/hwprobe.h \ + # sysdep_headers + +sysdep_routines += \ + flush-icache \ + hwprobe \ + # sysdep_routines + endif ifeq ($(subdir),stdlib) diff --git a/sysdeps/unix/sysv/linux/riscv/Versions b/sysdeps/unix/sysv/linux/riscv/Versions index 5625d2a0b8..8717b62a4a 100644 --- a/sysdeps/unix/sysv/linux/riscv/Versions +++ b/sysdeps/unix/sysv/linux/riscv/Versions @@ -8,4 +8,7 @@ libc { GLIBC_2.27 { __riscv_flush_icache; } + GLIBC_2.39 { + __riscv_hwprobe; + } } diff --git a/sysdeps/unix/sysv/linux/riscv/hwprobe.c b/sysdeps/unix/sysv/linux/riscv/hwprobe.c new file mode 100644 index 0000000000..af84eb4a7f --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/hwprobe.c @@ -0,0 +1,36 @@ +/* RISC-V hardware feature probing support on Linux + Copyright (C) 2023 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include + +int __riscv_hwprobe (struct riscv_hwprobe *pairs, size_t pair_count, + size_t cpu_count, unsigned long int *cpus, + unsigned int flags) +{ + int r; + + r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, pairs, pair_count, + cpu_count, cpus, flags); + + /* Negate negative errno values to match pthreads API. */ + return -r; +} diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist index a13c484582..6c96e25552 100644 --- a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist +++ b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist @@ -2436,6 +2436,7 @@ GLIBC_2.38 strlcat F GLIBC_2.38 strlcpy F GLIBC_2.38 wcslcat F GLIBC_2.38 wcslcpy F +GLIBC_2.39 __riscv_hwprobe F GLIBC_2.39 pidfd_getpid F GLIBC_2.39 pidfd_spawn F GLIBC_2.39 pidfd_spawnp F diff --git a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist index cf65d8d6d4..225278807b 100644 --- a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist +++ b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist @@ -2636,6 +2636,7 @@ GLIBC_2.38 strlcat F GLIBC_2.38 strlcpy F GLIBC_2.38 wcslcat F GLIBC_2.38 wcslcpy F +GLIBC_2.39 __riscv_hwprobe F GLIBC_2.39 pidfd_getpid F GLIBC_2.39 pidfd_spawn F GLIBC_2.39 pidfd_spawnp F diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h new file mode 100644 index 0000000000..c52832f7d5 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -0,0 +1,74 @@ +/* RISC-V architecture probe interface + Copyright (C) 2023 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#ifndef _SYS_HWPROBE_H +#define _SYS_HWPROBE_H 1 + +#include +#include +#ifdef __has_include +# if __has_include () +# include +# endif +#endif + +/* Define a (probably stale) version of the interface if the Linux headers + aren't present. */ +#ifndef RISCV_HWPROBE_KEY_MVENDORID +struct riscv_hwprobe { + signed long long int key; + unsigned long long int value; +}; + +#define RISCV_HWPROBE_KEY_MVENDORID 0 +#define RISCV_HWPROBE_KEY_MARCHID 1 +#define RISCV_HWPROBE_KEY_MIMPID 2 +#define RISCV_HWPROBE_KEY_BASE_BEHAVIOR 3 +#define RISCV_HWPROBE_BASE_BEHAVIOR_IMA (1 << 0) +#define RISCV_HWPROBE_KEY_IMA_EXT_0 4 +#define RISCV_HWPROBE_IMA_FD (1 << 0) +#define RISCV_HWPROBE_IMA_C (1 << 1) +#define RISCV_HWPROBE_IMA_V (1 << 2) +#define RISCV_HWPROBE_EXT_ZBA (1 << 3) +#define RISCV_HWPROBE_EXT_ZBB (1 << 4) +#define RISCV_HWPROBE_EXT_ZBS (1 << 5) +#define RISCV_HWPROBE_EXT_ZICBOZ (1 << 6) +#define RISCV_HWPROBE_KEY_CPUPERF_0 5 +#define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) +#define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) +#define RISCV_HWPROBE_MISALIGNED_SLOW (2 << 0) +#define RISCV_HWPROBE_MISALIGNED_FAST (3 << 0) +#define RISCV_HWPROBE_MISALIGNED_UNSUPPORTED (4 << 0) +#define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) +#define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 + +#endif /* RISCV_HWPROBE_KEY_MVENDORID */ + +__BEGIN_DECLS + +extern int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) + __nonnull ((1)) __wur + __fortified_attr_access (__read_write__, 1, 2) + __fortified_attr_access (__read_only__, 4, 3); + +__END_DECLS + +#endif /* sys/hwprobe.h */ From patchwork Wed Dec 13 21:11:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 82089 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9D7E03861813 for ; Wed, 13 Dec 2023 21:12:18 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oo1-xc2b.google.com (mail-oo1-xc2b.google.com [IPv6:2607:f8b0:4864:20::c2b]) by sourceware.org (Postfix) with ESMTPS id 306BA385B516 for ; Wed, 13 Dec 2023 21:11:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 306BA385B516 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 306BA385B516 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::c2b ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501925; cv=none; b=E4yevUananvf/qDrA4xoKaZPhrnJrfZqX9VQ76I8ECsm7kp/oMC4hl8iO3vMBDaMJ6vDJx2xICZD+qpkDcczD/YjrAsiWjL8cRFrXNAEL09kmYOVkplXEhJKVgUV8leKL/4dcd/xQQfKJ5Htwb0HuvG8rto7IkK2xn+qmajPSgI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501925; c=relaxed/simple; bh=RrgBRwsIXn0t+zBqq842fnWckQNWet6kKCk5X+hbfzs=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=DBsBWA7SXCR4yOyazroEvMRG7bfiRxswy0WxytN16NvouZMxG/rePPkmVrztyrU9DJblsZ8Sy+3Hyth1zHgY8LVbGedNjs9gbbZgLaLiuPU/HijwcO9S5YL76RqN0ald69Rd0UYF6qyd3N/yQVHGI66MSOK19gTFTanH4ipG+/U= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oo1-xc2b.google.com with SMTP id 006d021491bc7-59067ccb090so4749823eaf.1 for ; Wed, 13 Dec 2023 13:11:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1702501917; x=1703106717; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+2qQ1SUsKwL6OUNgVYMmwF/9joyr4Le9FFVTYHSWq6w=; b=qrF/Y5E5Tihx9XNtvxLVaQkk7V7/TgCVz0/FeNFfSMexJD7yT3muVojwfPE+wjYxq3 OBpZERt5u28TyuzWVN+DcQxSN2AVnOp7S3jXBlaQ44gmHRsaGik4od268l0mSbxyf+2m KTsKrAZ1QwIfPnIKXbPGK5sLNvThPGvjuzqalsZONcB6/nVhaNNphACC+qhcZU2mzSBD yZauwupuIND05ZGgH9cyS45EwmzFPEdFLDGpG5qgP6l7WsK7tIznMlMQqTxALiAvAUjH c0LHBgu8fbZVSCy/lmdSNnABQ+WWbhel0SjUxN5KW6AXrHTwVQ1h1a/77g1RCvdpfKFK 1qTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702501917; x=1703106717; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+2qQ1SUsKwL6OUNgVYMmwF/9joyr4Le9FFVTYHSWq6w=; b=FlnE7/n+be8BXKlX4PRlUDtxuW3OnB4nesAtmDv1d1DOoG8uqXpp+fttxyO5n8bg5o uKeSzVf+a65F8Ug+Id+14MZFVA7fZGFePadQm8Zeo3EZWttJGygy5JPlDu8zBvKE6gyr 3yz0aOGs6uw09OsMgdPoDhnS2iKSLU0W1ZZkafsYeWbbc+5qEvMKb2DTordUOVgcs4ym XNfqOBdp3TFo7P/HB1Dxq39qJsMSU2GgCZMWFNycqY9RzAZKks1RH04al/j4D6iiZSqb ywbTsyLGB6dGsi6Wkc1gUVjta1U1Ja4yzEm+qmk6xzxE4RkEXOcJLnxSZUQvBDvSV5Gz gK6g== X-Gm-Message-State: AOJu0Yxd99s+8Qds1C6n5jt99yopoxUhVkN2d2duYqpuLag90eulXV9I SgixUkuBdZCdCxt0BE9FbnrgIHtrjyC87woOd5o= X-Google-Smtp-Source: AGHT+IGgm6AkoJnfdRxoFTlPZ1lf9h+egZfHR23L/F096ertwvUIUeB3HJM202TQDBpxUgOxKKfOpA== X-Received: by 2002:a4a:98e1:0:b0:590:d3ab:c9e2 with SMTP id b30-20020a4a98e1000000b00590d3abc9e2mr2733332ooj.19.1702501917642; Wed, 13 Dec 2023 13:11:57 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id c15-20020a4a380f000000b0058d1de21893sm3223109ooa.24.2023.12.13.13.11.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 13:11:57 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, palmer@rivosinc.com, Florian Weimer , Evan Green Subject: [PATCH v10 2/7] linux: Introduce INTERNAL_VSYSCALL Date: Wed, 13 Dec 2023 13:11:37 -0800 Message-Id: <20231213211142.1543025-3-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231213211142.1543025-1-evan@rivosinc.com> References: <20231213211142.1543025-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add an INTERNAL_VSYSCALL() macro that makes a vDSO call, falling back to a regular syscall, but without setting errno. Instead, the return value is plumbed straight out of the macro. Signed-off-by: Evan Green --- Changes in v10: - Introduced INTERNAL_VSYSCALL patch sysdeps/unix/sysv/linux/sysdep-vdso.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/sysdeps/unix/sysv/linux/sysdep-vdso.h b/sysdeps/unix/sysv/linux/sysdep-vdso.h index a51b5deb37..2a1fc17e23 100644 --- a/sysdeps/unix/sysv/linux/sysdep-vdso.h +++ b/sysdeps/unix/sysv/linux/sysdep-vdso.h @@ -53,4 +53,23 @@ sc_ret; \ }) +#define INTERNAL_VSYSCALL(name, nr, args...) \ + ({ \ + __label__ out; \ + long int sc_ret; \ + \ + __typeof (GLRO(dl_vdso_##name)) vdsop = GLRO(dl_vdso_##name); \ + if (vdsop != NULL) \ + { \ + sc_ret = INTERNAL_VSYSCALL_CALL (vdsop, nr, ##args); \ + if ((!INTERNAL_SYSCALL_ERROR_P (sc_ret)) || \ + (INTERNAL_SYSCALL_ERRNO (sc_ret) != ENOSYS)) \ + goto out; \ + } \ + \ + sc_ret = INTERNAL_SYSCALL_CALL (name, ##args); \ + out: \ + sc_ret; \ + }) + #endif /* SYSDEP_VDSO_LINUX_H */ From patchwork Wed Dec 13 21:11:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 82090 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 02AFD3861838 for ; Wed, 13 Dec 2023 21:12:26 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oo1-xc2c.google.com (mail-oo1-xc2c.google.com [IPv6:2607:f8b0:4864:20::c2c]) by sourceware.org (Postfix) with ESMTPS id E00A8385DC29 for ; Wed, 13 Dec 2023 21:12:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E00A8385DC29 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E00A8385DC29 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::c2c ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501925; cv=none; b=d/vHbpDA8TWpyrZyfF4cwEi6EjC9Tz1d7tjh0LR8mx8Ne3ciJ5XVinBq41sas8v+rYTiDeJxMX6yqjfZy4IjZtg4upNTreMxORETOyK6h0LpuKMmF82QYVLlNOmD99o4nevpMqzy84v0xvSvTd0Jbq7le87B0W6uH6vPeF7stKQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501925; c=relaxed/simple; bh=si3nlXVmqsJIVSt66LWuQDsPtKZt7WHlxnFipwumVcU=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=fV89sPnlJnOFtsg67+n2/HC3XzDYN9JTgHO2hRbzVafdTkHRnZuo64/CN10DaFbUj9C9VtGc0bfBmoRLDEeC2fLTP25pwAOELzbOcjSuLWtxL17pYznD3Tz8ZTQ0R74qjBMK0A6ylhdByscCIKhCU7ffazoMzCuLKgWjZ+W4Nes= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oo1-xc2c.google.com with SMTP id 006d021491bc7-59067ccb090so4749849eaf.1 for ; Wed, 13 Dec 2023 13:12:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1702501920; x=1703106720; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6R2nKdRh8JjcImBZ02Cmdtc7yAtNLzNCryFyVjwVl5A=; b=fXzJAsCCArKLc3ZUXbPx7tebb4IoWg7/DBDAz1mLP9rh0glKgU91ysIS1DACWL5JQq 4+owC/k7ZX1IbAqiY1qHsRHZFY/0l/8zpk4evNTC4/F2rJbwbpFmJ5XIy8IGUR++nwQh zYuvTXg4Hj+GXqW45/0B9bhyzj8ioFFA0pkIh6DpLmfuMnsmc69LOrD//RxKWhtvCvCd NhcOMWSsLwF18f1bx7fB77w93JSkogPlbi5JsM1nez/drJyjlI27MSI/PNUV2DbV4+qm Hf0loVE8V9Xx0x25bsy6QU51KNgTmud2kxNQ0sj9i4C8XhKbg0UXf70YhdARf6SKaqG8 RtjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702501920; x=1703106720; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6R2nKdRh8JjcImBZ02Cmdtc7yAtNLzNCryFyVjwVl5A=; b=UzS4wLlzfeEllzF2gTnrYtNMJSqrqDORJLNjNOtsiU2QNxZ07TCN1wEFZYPJzMlgZK 1mAfkFkVREEIsZdih9O7VCXd6bfJjfShX2aRic8OQUKE98llhp0uv+cBYb71sedxms5b LSPNzJTYIC80KqQHktVcnWHwX1E3IOKO/bX1bAmzOGe81hEWWkcfhUrnideHcC8COJV/ bEShu7rXXZNW9G7eVRZt+sSysYNiEyplOxXa+DtU+nu9IU8Z0eDG0tfTsCn1LBpyME67 xc5MjqVKDs8XZRXUNTy3Wxghw+lzAuWjHi4xnpUhNDIYOOej6Qqur1aLoFBqqSQGehEL n3Iw== X-Gm-Message-State: AOJu0YyypMrvVV3uS7VmVV4AoiRye7NNue+j7BB19tt2nwZYDhgCix0K sAqIGDUVNRj1F8UJtndfYxONGlrmqMDztteOw9o= X-Google-Smtp-Source: AGHT+IEJ5oELA25zqUNKTBco+SyWKO8pILiIxZw19MWvZDZB4HMZvQnU5PsvMuz3XdY92JghQ+VFqA== X-Received: by 2002:a4a:a9cd:0:b0:590:7041:ba13 with SMTP id h13-20020a4aa9cd000000b005907041ba13mr5305029oon.0.1702501920326; Wed, 13 Dec 2023 13:12:00 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id c15-20020a4a380f000000b0058d1de21893sm3223109ooa.24.2023.12.13.13.11.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 13:12:00 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, palmer@rivosinc.com, Florian Weimer , Evan Green Subject: [PATCH v10 3/7] riscv: Add hwprobe vdso call support Date: Wed, 13 Dec 2023 13:11:38 -0800 Message-Id: <20231213211142.1543025-4-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231213211142.1543025-1-evan@rivosinc.com> References: <20231213211142.1543025-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The new riscv_hwprobe syscall also comes with a vDSO for faster answers to your most common questions. Call in today to speak with a kernel representative near you! Signed-off-by: Evan Green --- Changes in v10: - Remove leading underscores in definition (Adhemerval) - Remove spurious 5 from INTERNAL_SYSCALL_CALL (Adhemerval) - Use new INTERNAL_VSYSCALL macro instead (Adhemerval) Changes in v7: - Use INTERNAL_VSYSCALL_CALL (Florian) Changes in v3: - Add the "return" to the vsyscall - Fix up vdso arg types to match kernel v4 version - Remove ifdef around INLINE_VSYSCALL (Adhemerval) Changes in v2: - Add vDSO interface sysdeps/unix/sysv/linux/dl-vdso-setup.c | 10 ++++++++++ sysdeps/unix/sysv/linux/dl-vdso-setup.h | 3 +++ sysdeps/unix/sysv/linux/riscv/hwprobe.c | 4 ++-- sysdeps/unix/sysv/linux/riscv/sysdep.h | 1 + 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/sysdeps/unix/sysv/linux/dl-vdso-setup.c b/sysdeps/unix/sysv/linux/dl-vdso-setup.c index 97eaaeac37..ed8b1ef426 100644 --- a/sysdeps/unix/sysv/linux/dl-vdso-setup.c +++ b/sysdeps/unix/sysv/linux/dl-vdso-setup.c @@ -71,6 +71,16 @@ PROCINFO_CLASS int (*_dl_vdso_clock_getres_time64) (clockid_t, # ifdef HAVE_GET_TBFREQ PROCINFO_CLASS uint64_t (*_dl_vdso_get_tbfreq)(void) RELRO; # endif + +/* RISC-V specific ones. */ +# ifdef HAVE_RISCV_HWPROBE +PROCINFO_CLASS int (*_dl_vdso_riscv_hwprobe)(void *, + size_t, + size_t, + unsigned long *, + unsigned int) RELRO; +# endif + #endif #undef RELRO diff --git a/sysdeps/unix/sysv/linux/dl-vdso-setup.h b/sysdeps/unix/sysv/linux/dl-vdso-setup.h index 867072b897..39eafd5316 100644 --- a/sysdeps/unix/sysv/linux/dl-vdso-setup.h +++ b/sysdeps/unix/sysv/linux/dl-vdso-setup.h @@ -47,6 +47,9 @@ setup_vdso_pointers (void) #ifdef HAVE_GET_TBFREQ GLRO(dl_vdso_get_tbfreq) = dl_vdso_vsym (HAVE_GET_TBFREQ); #endif +#ifdef HAVE_RISCV_HWPROBE + GLRO(dl_vdso_riscv_hwprobe) = dl_vdso_vsym (HAVE_RISCV_HWPROBE); +#endif } #endif diff --git a/sysdeps/unix/sysv/linux/riscv/hwprobe.c b/sysdeps/unix/sysv/linux/riscv/hwprobe.c index af84eb4a7f..441a44eafc 100644 --- a/sysdeps/unix/sysv/linux/riscv/hwprobe.c +++ b/sysdeps/unix/sysv/linux/riscv/hwprobe.c @@ -28,8 +28,8 @@ int __riscv_hwprobe (struct riscv_hwprobe *pairs, size_t pair_count, { int r; - r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, pairs, pair_count, - cpu_count, cpus, flags); + r = INTERNAL_VSYSCALL (riscv_hwprobe, 5, pairs, pair_count, + cpu_count, cpus, flags); /* Negate negative errno values to match pthreads API. */ return -r; diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h index 5583b96d23..ee015dfeb6 100644 --- a/sysdeps/unix/sysv/linux/riscv/sysdep.h +++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h @@ -156,6 +156,7 @@ /* List of system calls which are supported as vsyscalls (for RV32 and RV64). */ # define HAVE_GETCPU_VSYSCALL "__vdso_getcpu" +# define HAVE_RISCV_HWPROBE "__vdso_riscv_hwprobe" # undef HAVE_INTERNAL_BRK_ADDR_SYMBOL # define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1 From patchwork Wed Dec 13 21:11:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 82091 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 043453857024 for ; Wed, 13 Dec 2023 21:12:31 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oo1-xc35.google.com (mail-oo1-xc35.google.com [IPv6:2607:f8b0:4864:20::c35]) by sourceware.org (Postfix) with ESMTPS id 98A95385DC2E for ; Wed, 13 Dec 2023 21:12:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 98A95385DC2E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 98A95385DC2E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::c35 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501926; cv=none; b=qA1qR9paIoRvo/GtKvMm927YbX95UKazOR8MJhV0hD9ish7+J09vJDPzdT6cmgsrzzwt0Vjhd0CXBItNScEy1BxjEZ7QJmKDdGwqfvZMeda4+cxvPSc70dq9BlOawSgiLEIHWZ/dXrSX/qqoMQA/RLwPjqY2HgAMrydsBCgFJ9E= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501926; c=relaxed/simple; bh=ORfTWsc8kx707LRg8MJDpQ217M4k9tXAmoRmx3gKC/k=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=q4DwoJmRtQrmVnH58HMCcW1cZVULAK9n+ddxnmspg3w6sEVCi2dvnzJL8poS2fT//R+sO2gDKpgfA9oSquY0dRrpO5Ar/fQCj6+8NQe4MD7/q3T8iB38yUSSOpOSqfn8XjiOKk2xwTcgk0pUmEx9jPIzxn89b3EHqyhD6gom25k= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oo1-xc35.google.com with SMTP id 006d021491bc7-58ceab7daddso3225548eaf.3 for ; Wed, 13 Dec 2023 13:12:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1702501923; x=1703106723; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kjV2BNQi1zBoDRaNWrQRya0fQ9bui+5W780LGc/OuFA=; b=mqFgyRvFOz9tX8bV3pjfRc1FcoAHQwonvO15ER6rlntYYMxkzLHe4DeOo0IDQYO634 +yszDDxgPr95kNCR4kUSyOheU01Xlz8rsdetSDIWVh1NF6jjxAsk25h51zYk/Litb6f9 slJakR0yGqRTd2EgdVfdVku6qnxQHQQNFIeBzrLKEx3Chh2Xp+5O9LzMfIy2dUi3euv/ KnMvg6pj/maIlLC5jJGJwJTx85VlmBBWHkJhseUCCLi76bB24W2LrgdH8qvHCUJLyNUl sfN1bW89Zr2V1fqazoJUMiGItV8bMEj5VuFr2ozRbyqLqpS/HFJ3uOLArg1X8IXbQlsc 3aCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702501923; x=1703106723; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kjV2BNQi1zBoDRaNWrQRya0fQ9bui+5W780LGc/OuFA=; b=Y/XlF4iDQu+HfsbzVLBic6T15QPlTCfJTgiY7ZOzN6gGS8ulRGulJmHrcKmsy2x7aJ VAmRMK4aQ/4NwbGXY7Up6dOD2u5aoqQoF8mNvnJR9++SC/vEkOSe2zBYj2HJuwgMfthL xbs2toh7ShMWyVoebCa3m7zRISzLcbNL+MIONyF2VLQgfd7o4Ulg2t4dWCmQCPn1fBSH DhS3gWOf+KX1sgXeANeGLZ5KlQQzT0CEQs/W41gZlXWP3+R25b+SscCPN8lmQVASOXvi 50eAx1xUQbV8HdygfVOwqvDDckJ9yUnuUmrainjoKrZjKamE6u+BMABeyy7+6WQ+KH29 sDFg== X-Gm-Message-State: AOJu0YwXXb1f3ne0wD743jv1axgvINfUG4yYjlzB7iitg7VgNsUWQ0Jn 6rTXcFbhSFd3PT7EpJkbmCi4ci1jnnHRhKfFO5U= X-Google-Smtp-Source: AGHT+IFIlEm1xwvX/5Q5ffKiWDsbgEDItwVprTrNLDWVvu+9xunqiQu/EsXKdXO+iyzDeowDabI09w== X-Received: by 2002:a4a:a9cd:0:b0:590:78ca:4963 with SMTP id h13-20020a4aa9cd000000b0059078ca4963mr4958694oon.6.1702501923075; Wed, 13 Dec 2023 13:12:03 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id c15-20020a4a380f000000b0058d1de21893sm3223109ooa.24.2023.12.13.13.12.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 13:12:02 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, palmer@rivosinc.com, Florian Weimer , Evan Green Subject: [PATCH v10 4/7] riscv: Add __riscv_hwprobe pointer to ifunc calls Date: Wed, 13 Dec 2023 13:11:39 -0800 Message-Id: <20231213211142.1543025-5-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231213211142.1543025-1-evan@rivosinc.com> References: <20231213211142.1543025-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The new __riscv_hwprobe() function is designed to be used by ifunc selector functions. This presents a challenge for applications and libraries, as ifunc selectors are invoked before all relocations have been performed, so an external call to __riscv_hwprobe() from an ifunc selector won't work. To address this, pass a pointer to the __riscv_hwprobe() function into ifunc selectors as the second argument (alongside dl_hwcap, which was already being passed). Include a typedef as well for convenience, so that ifunc users don't have to go through contortions to call this routine. Users will need to remember to check the second argument for NULL, to account for older glibcs that don't pass the function. Signed-off-by: Evan Green --- Changes in v10: - Hand in pointer to __riscv_hwprobe(), not vDSO function Changes in v7: - Remove __THROW from function pointer type, as it creates warnings together with __fortified_attr_access. sysdeps/riscv/dl-irel.h | 9 +++++---- sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 10 ++++++++++ 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/sysdeps/riscv/dl-irel.h b/sysdeps/riscv/dl-irel.h index eaeec5467c..791f8169fa 100644 --- a/sysdeps/riscv/dl-irel.h +++ b/sysdeps/riscv/dl-irel.h @@ -24,6 +24,7 @@ #include #include #include +#include #define ELF_MACHINE_IRELA 1 @@ -31,10 +32,10 @@ static inline ElfW(Addr) __attribute ((always_inline)) elf_ifunc_invoke (ElfW(Addr) addr) { - /* The second argument is a void pointer to preserve the extension - fexibility. */ - return ((ElfW(Addr) (*) (uint64_t, void *)) (addr)) - (GLRO(dl_hwcap), NULL); + /* The third argument is a void pointer to preserve the extension + flexibility. */ + return ((ElfW(Addr) (*) (uint64_t, void *, void *)) (addr)) + (GLRO(dl_hwcap), __riscv_hwprobe, NULL); } static inline void diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h index c52832f7d5..c4e3927667 100644 --- a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -69,6 +69,16 @@ extern int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, __fortified_attr_access (__read_write__, 1, 2) __fortified_attr_access (__read_only__, 4, 3); +/* A pointer to the __riscv_hwprobe vDSO function is passed as the second + argument to ifunc selector routines. Include a function pointer type for + convenience in calling the function in those settings. */ +typedef int (*__riscv_hwprobe_t) (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) + __nonnull ((1)) __wur + __fortified_attr_access (__read_write__, 1, 2) + __fortified_attr_access (__read_only__, 4, 3); + __END_DECLS #endif /* sys/hwprobe.h */ From patchwork Wed Dec 13 21:11:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 82094 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9823D384DEE1 for ; Wed, 13 Dec 2023 21:12:57 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oo1-xc31.google.com (mail-oo1-xc31.google.com [IPv6:2607:f8b0:4864:20::c31]) by sourceware.org (Postfix) with ESMTPS id 4C869385DC11 for ; Wed, 13 Dec 2023 21:12:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4C869385DC11 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4C869385DC11 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::c31 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501929; cv=none; b=s658/Qh4YwD9FahZd7HHzJojxVqboVZEatES5Z2kt1P1W8P8r4614ks7e1TcpMiWMxCeirq+tWMUfyBRNmrSDWqi75Xreg1N5jFdHmQzEP55kayi7ZlZfL/IQ5T5wrUK/3ujRC4D4uu+KQq88kIS21Xeoou9NudXjF1wUDpbRPA= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501929; c=relaxed/simple; bh=OmmaHEABqlfURONSAAIufLVjUUd2psZiBVxAsCEj8dk=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=P57SI2nRbVfl9XvhEN+V+Yt+tbUyTY6EtndC+MSy/Yi9ALlWTXcY6y7FfpTcz0gUyyQvbbQH2Yp60Rw30Unu4pJK8xkQWGTq+el31eyUYmQ9zKBM0cMTk95NwuGhdqhgLg49nEf7KgPHCYGJ3LTWmS3oDkhJmLjjj6BB/RR4ePQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oo1-xc31.google.com with SMTP id 006d021491bc7-58d1b767b2bso4746096eaf.2 for ; Wed, 13 Dec 2023 13:12:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1702501925; x=1703106725; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IKe4xa9VzbKEWsnhQ0HsyHNmoz/T30Gb29Jw2h1jDMk=; b=j2YgTOc52alQxPPzA0fktjT7Sx2CrWx85E730w8/wbPhOUZT6uTkNE25PshdbO3SJz gNm1yLhKN1b5syzQWy7PDKPKQbYQFK6XRXMTI9+HZo4+t3mivCR+/FrvKitejcf6oZ99 KkD8IzLy7dLA8I6338GeJFKpi7JQywTxfSL2nNve7ezPg20q9A8KHqnSPebwkzzK5+UT 12KU4+8OaHt5RbEE08HU7CNPaGchJtUixgPmC7D/WnK8DvHd/8t49vk95SHazlR1eK+v KskpyS1nm/ePEjFIiBvAlBtg5mobZljSgD2MTYVw5FlykdZH0rSh9GyKwqcbxItk1NkQ ObfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702501925; x=1703106725; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IKe4xa9VzbKEWsnhQ0HsyHNmoz/T30Gb29Jw2h1jDMk=; b=lVoW90QeB2DYB0FFlzCVqPrp8SYwTs2+ZxAHgAwU9Qvah5ljjriRkpoUWFns9oS+2c 70jH0XugcbGxy8AihVpiBKP5K3vBwG/vxvmmx7iRXOgEoaReUc+mYcYWYfaiKAmohQ0k ntgHL06/L90c5L/X9Ay5z9Ni8ovGc+G3KXYksclhIv1EjfBjSPMPi/zmh4SNqPbzQ83C hy3rYNxV+ySePz6Xm5UZalvrxDnJy3nArjnx0wArkddb0j5Wiug4W4Cb1Rq9385WeOfV 5wQVqTbpLJDackMeRPMhQqK/mhcD5v5a7rYRdmhukMLlg6J3mPq6HpDkTEJ848/2aCnF oFnA== X-Gm-Message-State: AOJu0YweX2ZBwIkQGYPOqjSZAx1SW5JMlUwo2d8yWbqwXFmeTNL5m0lr GoY2km2deWXQhaMlAnJWuSVZhvLXBJMac2ZK6p4= X-Google-Smtp-Source: AGHT+IHgFaXQ6IIBIjlKB4887KgC6QOsrxXBSSvDJOrS+Hn28DO2B+ttYhg1rAu11DGPpLNOPkCUiQ== X-Received: by 2002:a05:6820:826:b0:581:e089:9e5c with SMTP id bg38-20020a056820082600b00581e0899e5cmr7771847oob.6.1702501925657; Wed, 13 Dec 2023 13:12:05 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id c15-20020a4a380f000000b0058d1de21893sm3223109ooa.24.2023.12.13.13.12.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 13:12:05 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, palmer@rivosinc.com, Florian Weimer , Evan Green Subject: [PATCH v10 5/7] riscv: Enable multi-arg ifunc resolvers Date: Wed, 13 Dec 2023 13:11:40 -0800 Message-Id: <20231213211142.1543025-6-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231213211142.1543025-1-evan@rivosinc.com> References: <20231213211142.1543025-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, URIBL_BLACK autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org RISC-V is apparently the first architecture to pass more than one argument to ifunc resolvers. The helper macros in libc-symbols.h, __ifunc_resolver(), __ifunc(), and __ifunc_hidden(), are incompatible with this. These macros have an "arg" (non-final) parameter that represents the parameter signature of the ifunc resolver. The result is an inability to pass the required comma through in a single preprocessor argument. Rearrange the __ifunc_resolver() macro to be variadic, and pass the types as those variable parameters. Move the guts of __ifunc() and __ifunc_hidden() into new macros, __ifunc_args(), and __ifunc_args_hidden(), that pass the variable arguments down through to __ifunc_resolver(). Then redefine __ifunc() and __ifunc_hidden(), which are used in a bunch of places, to simply shuffle the arguments down into __ifunc_args[_hidden]. Finally, define a riscv-ifunc.h header, which provides convenience macros to those looking to write ifunc selectors that use both arguments. Signed-off-by: Evan Green --- (no changes since v9) Changes in v9: - Fix a couple of typos causing powerpc not to build (build-many-glibcs) Changes in v6: - Introduced riscv-ifunc.h for multi-arg ifunc selectors. --- include/libc-symbols.h | 28 +++++++++++++++++----------- sysdeps/riscv/riscv-ifunc.h | 27 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 11 deletions(-) create mode 100644 sysdeps/riscv/riscv-ifunc.h diff --git a/include/libc-symbols.h b/include/libc-symbols.h index 5794614488..b45fad72b3 100644 --- a/include/libc-symbols.h +++ b/include/libc-symbols.h @@ -665,9 +665,9 @@ for linking") #endif /* Helper / base macros for indirect function symbols. */ -#define __ifunc_resolver(type_name, name, expr, arg, init, classifier) \ +#define __ifunc_resolver(type_name, name, expr, init, classifier, ...) \ classifier inhibit_stack_protector \ - __typeof (type_name) *name##_ifunc (arg) \ + __typeof (type_name) *name##_ifunc (__VA_ARGS__) \ { \ init (); \ __typeof (type_name) *res = expr; \ @@ -675,13 +675,13 @@ for linking") } #ifdef HAVE_GCC_IFUNC -# define __ifunc(type_name, name, expr, arg, init) \ +# define __ifunc_args(type_name, name, expr, init, ...) \ extern __typeof (type_name) name __attribute__ \ ((ifunc (#name "_ifunc"))); \ - __ifunc_resolver (type_name, name, expr, arg, init, static) + __ifunc_resolver (type_name, name, expr, init, static, __VA_ARGS__) -# define __ifunc_hidden(type_name, name, expr, arg, init) \ - __ifunc (type_name, name, expr, arg, init) +# define __ifunc_args_hidden(type_name, name, expr, init, ...) \ + __ifunc_args (type_name, name, expr, init, __VA_ARGS__) #else /* Gcc does not support __attribute__ ((ifunc (...))). Use the old behaviour as fallback. But keep in mind that the debug information for the ifunc @@ -692,18 +692,24 @@ for linking") different signatures. (Gcc support is disabled at least on a ppc64le Ubuntu 14.04 system.) */ -# define __ifunc(type_name, name, expr, arg, init) \ +# define __ifunc_args(type_name, name, expr, init, ...) \ extern __typeof (type_name) name; \ - __typeof (type_name) *name##_ifunc (arg) __asm__ (#name); \ - __ifunc_resolver (type_name, name, expr, arg, init,) \ + __typeof (type_name) *name##_ifunc (__VA_ARGS__) __asm__ (#name); \ + __ifunc_resolver (type_name, name, expr, init, , __VA_ARGS__) \ __asm__ (".type " #name ", %gnu_indirect_function"); -# define __ifunc_hidden(type_name, name, expr, arg, init) \ +# define __ifunc_args_hidden(type_name, name, expr, init, ...) \ extern __typeof (type_name) __libc_##name; \ - __ifunc (type_name, __libc_##name, expr, arg, init) \ + __ifunc (type_name, __libc_##name, expr, __VA_ARGS__, init) \ strong_alias (__libc_##name, name); #endif /* !HAVE_GCC_IFUNC */ +#define __ifunc(type_name, name, expr, arg, init) \ + __ifunc_args (type_name, name, expr, init, arg) + +#define __ifunc_hidden(type_name, name, expr, arg, init) \ + __ifunc_args_hidden (type_name, name, expr, init, arg) + /* The following macros are used for indirect function symbols in libc.so. First of all, you need to have the function prototyped somewhere, say in foo.h: diff --git a/sysdeps/riscv/riscv-ifunc.h b/sysdeps/riscv/riscv-ifunc.h new file mode 100644 index 0000000000..7bff591d1e --- /dev/null +++ b/sysdeps/riscv/riscv-ifunc.h @@ -0,0 +1,27 @@ +/* Common definition for ifunc resolvers. Linux/RISC-V version. + This file is part of the GNU C Library. + Copyright (C) 2023 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include + +#define INIT_ARCH() + +#define riscv_libc_ifunc(name, expr) \ + __ifunc_args (name, name, expr(hwcap, hwprobe), INIT_ARCH, \ + uint64_t hwcap, __riscv_hwprobe_t hwprobe) From patchwork Wed Dec 13 21:11:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 82092 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DD9F03861866 for ; Wed, 13 Dec 2023 21:12:46 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oo1-xc2c.google.com (mail-oo1-xc2c.google.com [IPv6:2607:f8b0:4864:20::c2c]) by sourceware.org (Postfix) with ESMTPS id A043B3861003 for ; Wed, 13 Dec 2023 21:12:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A043B3861003 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A043B3861003 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::c2c ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501931; cv=none; b=dww1Qewmhty65c7R1hQ8rnuMjLfnMLNYplF/sahzRPMTtdotjFv+FnLNYK9ZxZ4jfzaMZCMPklnxtcN0vmzNMvdQ1IORFsguSa80toDYbQ6ktVwK2TKXSKNcsaMLjaQAmwGcacH36tKSO02YXG/fnnOgzduo94dX32/inzFwUDc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501931; c=relaxed/simple; bh=z1vJHZCylR7gWswaX1YedLgXlnv8wJeHQ6njXQLNraQ=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=V7xgjymTF762LwtvvrFxUag85CieU3CLyi5tRdRio4BEagfeKpW06cMqmc+GGiW3+tLr4ekPvdogi2/XDIRhvfMcz8me22OiLkeZCxwH09UMDEfHm4Oe8dnYAq471XIzWuhygO+yGAFDvznTO4G93dg9I8SXC6CHcEyHXy3zFlY= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oo1-xc2c.google.com with SMTP id 006d021491bc7-59148c1ad35so1066871eaf.2 for ; Wed, 13 Dec 2023 13:12:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1702501928; x=1703106728; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a55kv/01G2tHxlqkOGuuQIyeV/sMfhNPjRYqXB5gRis=; b=qr7qHVoA4q7Q/MpX9vgttbe0INZCesDYXgRmM223yIWgb4sEsBeY0t4TYZxC79G6NU h9Jwk5TZ4UnffHst+SngHx5em9a18/eVUyzzPFPAgc/Wh1q/jnXV+HfDT9ysSVGeo2Xi RDeI/eFbeNqa5pFZGMvESbnvcXqek61pqVa7HCKb5j+taFq10DrwJ1aLg288hNtrlC0F 1FiVJRdoXgbeKZJBvX2R4IPynmrlZlPUU8ibGGLlxpSorKoodNZhY/H9NUJf3opr+RWS aU3Lo0b1jbGQUW9ZDMYXDh2V1jrrWMQlNHI3GVK8yIoDYC+Zd6/z4CUh/tEYFH+SdYyv IfGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702501928; x=1703106728; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a55kv/01G2tHxlqkOGuuQIyeV/sMfhNPjRYqXB5gRis=; b=TNbfx4i538jMZM1osucbxrqchZuscZXQJV9u+W+6AKKC1NYhHahRyDQiXI1aV/0+dO NJWIihjc243nFIDhpFs4vne2vrIpNkPXvvakCSGYiKpOub42hIQyKs+eSHMDbg6r952c iGqBzLxuKvZtg3MBr4C2ev3QyQulj4KRQHDR1UMd7dOEE0Y+j0SiKA2E0fs4K8ZopcKn 0YvMvr8flB9Vv0CtAy2noTrg59KH51BTj0bOXZLqLTEAmKi0xckqMf1vJop1b25q/8Mu vkqsm9BG0IeOLPBxPioT0CYOhfARQX0dhuwCPOZ3LBAj89/nsyhRZlYnnGrP1RUuyXjI sqhA== X-Gm-Message-State: AOJu0YwTvfD0UMHLzORjYpuXoxMTolvNBryfL50W9lPjIP170DsPSmeo H8ITjYgcSkERtvYA5FtW9mSrw6huLOGJZgIgpjM= X-Google-Smtp-Source: AGHT+IHU2UnV+dchdSkIDr2peFQ4aJt6rri3MKVWADDkRbqFd0dGRwsToqJjH8lJUUOHLjR0hr7Img== X-Received: by 2002:a4a:9891:0:b0:590:f911:42ef with SMTP id a17-20020a4a9891000000b00590f91142efmr3590756ooj.18.1702501928232; Wed, 13 Dec 2023 13:12:08 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id c15-20020a4a380f000000b0058d1de21893sm3223109ooa.24.2023.12.13.13.12.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 13:12:07 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, palmer@rivosinc.com, Florian Weimer , Evan Green Subject: [PATCH v10 6/7] riscv: Add ifunc helper method to hwprobe.h Date: Wed, 13 Dec 2023 13:11:41 -0800 Message-Id: <20231213211142.1543025-7-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231213211142.1543025-1-evan@rivosinc.com> References: <20231213211142.1543025-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add a little helper method so it's easier to fetch a single value from the hwprobe function when used within an ifunc selector. Signed-off-by: Evan Green --- Changes in v10: - Avoid implicit comparisons (Adhemerval) Changes in v9: - Use __inline rather than inline so c89 compiles (build-many-glibcs) Changes in v7: - Introduced static inline helper (Richard) sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 25 +++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h index c4e3927667..d85d49cd73 100644 --- a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -22,6 +22,7 @@ #include #include +#include #ifdef __has_include # if __has_include () # include @@ -81,4 +82,28 @@ typedef int (*__riscv_hwprobe_t) (struct riscv_hwprobe *__pairs, size_t __pair_c __END_DECLS +/* Helper function usable from ifunc selectors that probes a single key. */ +static __inline int +__riscv_hwprobe_one(__riscv_hwprobe_t hwprobe_func, + signed long long int key, + unsigned long long int *value) +{ + struct riscv_hwprobe pair; + int rc; + + if (hwprobe_func == NULL) + return ENOSYS; + + pair.key = key; + rc = hwprobe_func (&pair, 1, 0, NULL, 0); + if (rc != 0) + return rc; + + if (pair.key < 0) + return ENOENT; + + *value = pair.value; + return 0; +} + #endif /* sys/hwprobe.h */ From patchwork Wed Dec 13 21:11:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 82093 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 119D038618E8 for ; Wed, 13 Dec 2023 21:12:50 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oo1-xc32.google.com (mail-oo1-xc32.google.com [IPv6:2607:f8b0:4864:20::c32]) by sourceware.org (Postfix) with ESMTPS id CBD35385DC19 for ; Wed, 13 Dec 2023 21:12:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CBD35385DC19 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org CBD35385DC19 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::c32 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501934; cv=none; b=XaXL8ixy44nghh42CXuCj5gwFoyC38Bfsgz36rKA75G3OcAcleST44zkHp7/lx894gsanEWCmZyJ9V/nXWpbLkBv6X3nJBoq3JFlAwIsUEtFRsPosn70A0+YCkkPwtqKTOu0pqcjYBamN9AXRa2ZF0BpJS3OG9gzIVCEL3BZMhc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702501934; c=relaxed/simple; bh=Et4TyqAqiFYhHfWfIFrTZT3trg0g2fE1W2Al3tTlo7w=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=v/7nwz/e5aAP1GPZxd1dAG6h9HaJN+R89njRiuwXyy6YYPtesR47u6rI/x0WIABVm+krf9OLdS3f8cpybp3AehxP7DiOUaASR2So4xKM7V7/G6w8zb+/cQgOdq1+MVcmjrSecDPyM07MteOHjZdArWhUACn+t5x+7SxE5cZnF7o= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oo1-xc32.google.com with SMTP id 006d021491bc7-59064bca27dso4298337eaf.0 for ; Wed, 13 Dec 2023 13:12:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1702501931; x=1703106731; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UqBeA8pR1wdCDstBQynp0KBXk/iPot1SB5zy7cg0jWI=; b=kWzEwLbVDPa9XrbbC/t2E78vp9HNmfYYHl129JxvUTe/8plQP177A/Nlto9DBAT1tl PJUWf/WuJy7uXnzuvJiyn074t3CC9GMNrL9B7Np3rQIxOK+2GIic0kTSvpehPMbWAenk hy6AWL2tiWLD5UrzmCDUmhbQpeoozU0pcZniuN0f2gNoRHmvBgFmZj52pT9Vby3/Izzu iHBZTUaAQlUlrc039BKLiS8D936pC1zDo3Vn/LWlw7BlLFoBYIACGAEcfgVhfKUcKsHp s4EQyxNoWt3f96XAx3LMQ92F1kdvCzo1iHycSe9UtBJyggJChREXWVp6nzEga2H1GSNq ZIgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702501931; x=1703106731; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UqBeA8pR1wdCDstBQynp0KBXk/iPot1SB5zy7cg0jWI=; b=an5U1jNaABA4jf5MB/iEJTAB4SQ2PfSXyYqrEhqR1G+5rpxT0R6VR/cVIImX9792JS LhyiFwJKfbBLAhoaPok5165Yvc2zIluw15VncVJBqbJ80nGJ5mA/3g2Ng4yv5jvtqzrM rkfoV1GApfwsJoODLwnpN34xlkYbkTsPB13nqca5X4AbIF9+u1I7LdHjS8d0gKNzCaZN KHSCcDdUYPkxqdFSc7MaJ815HEipX6y08TEiJXmQhpzFXsAsn8L97Cuew9hS9aCKROaj eQsLnD/qWMuSsf0sj8A0/i/SYQsuIxPO6ZJqndD6tkXKdn+EWLd/yQlTa+U1eDieyG36 EkEg== X-Gm-Message-State: AOJu0Yy55i8ftX/l1tJiK9oIJVugBv54Tzpvqt3DPL8gao8ZBP9b6w65 yh2MTt8XI0D23fHV58np92wniEbpe1MX+vPjq28= X-Google-Smtp-Source: AGHT+IFotlZh+TguTFxjq0Me9WDwOBLTcNS5GohiH63FoWqhWDS2MlTNUeBbc9qz2I4G1M4y5sNnLg== X-Received: by 2002:a4a:3815:0:b0:58e:1c48:39ac with SMTP id c21-20020a4a3815000000b0058e1c4839acmr6234287ooa.18.1702501930674; Wed, 13 Dec 2023 13:12:10 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id c15-20020a4a380f000000b0058d1de21893sm3223109ooa.24.2023.12.13.13.12.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 13:12:10 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: vineetg@rivosinc.com, slewis@rivosinc.com, palmer@rivosinc.com, Florian Weimer , Evan Green Subject: [PATCH v10 7/7] riscv: Add and use alignment-ignorant memcpy Date: Wed, 13 Dec 2023 13:11:42 -0800 Message-Id: <20231213211142.1543025-8-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231213211142.1543025-1-evan@rivosinc.com> References: <20231213211142.1543025-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org For CPU implementations that can perform unaligned accesses with little or no performance penalty, create a memcpy implementation that does not bother aligning buffers. It will use a block of integer registers, a single integer register, and fall back to bytewise copy for the remainder. Signed-off-by: Evan Green Reviewed-by: Palmer Dabbelt --- Changes in v10: - One line per function in Makefile for memcpy (Adhemerval) - Space before argument-like things (Adhemerval) Changes in v7: - Use new helper function in memcpy ifunc selector (Richard) Changes in v6: - Fix a couple regressions in the assembly from v5 :/ - Use passed hwprobe pointer in memcpy ifunc selector. Changes in v5: - Do unaligned word access for final trailing bytes (Richard) Changes in v4: - Fixed comment style (Florian) Changes in v3: - Word align dest for large memcpy()s. - Add tags - Remove spurious blank line from sysdeps/riscv/memcpy.c Changes in v2: - Used _MASK instead of _FAST value itself. --- sysdeps/riscv/memcopy.h | 26 ++++ sysdeps/riscv/memcpy.c | 63 ++++++++ sysdeps/riscv/memcpy_noalignment.S | 138 ++++++++++++++++++ sysdeps/unix/sysv/linux/riscv/Makefile | 9 ++ .../unix/sysv/linux/riscv/memcpy-generic.c | 24 +++ 5 files changed, 260 insertions(+) create mode 100644 sysdeps/riscv/memcopy.h create mode 100644 sysdeps/riscv/memcpy.c create mode 100644 sysdeps/riscv/memcpy_noalignment.S create mode 100644 sysdeps/unix/sysv/linux/riscv/memcpy-generic.c diff --git a/sysdeps/riscv/memcopy.h b/sysdeps/riscv/memcopy.h new file mode 100644 index 0000000000..2b685c8aa0 --- /dev/null +++ b/sysdeps/riscv/memcopy.h @@ -0,0 +1,26 @@ +/* memcopy.h -- definitions for memory copy functions. RISC-V version. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +/* Redefine the generic memcpy implementation to __memcpy_generic, so + the memcpy ifunc can select between generic and special versions. + In rtld, don't bother with all the ifunciness. */ +#if IS_IN (libc) +#define MEMCPY __memcpy_generic +#endif diff --git a/sysdeps/riscv/memcpy.c b/sysdeps/riscv/memcpy.c new file mode 100644 index 0000000000..df6b7db1f4 --- /dev/null +++ b/sysdeps/riscv/memcpy.c @@ -0,0 +1,63 @@ +/* Multiple versions of memcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017-2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memcpy so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memcpy +# define memcpy __redirect_memcpy +# include +# include +# include +# include +# include + +# define INIT_ARCH() + +extern __typeof (__redirect_memcpy) __libc_memcpy; + +extern __typeof (__redirect_memcpy) __memcpy_generic attribute_hidden; +extern __typeof (__redirect_memcpy) __memcpy_noalignment attribute_hidden; + +static inline __typeof (__redirect_memcpy) * +select_memcpy_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long int value; + + INIT_ARCH (); + + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_CPUPERF_0, &value)) + return __memcpy_generic; + + if ((value & RISCV_HWPROBE_MISALIGNED_MASK) == RISCV_HWPROBE_MISALIGNED_FAST) + return __memcpy_noalignment; + + return __memcpy_generic; +} + +riscv_libc_ifunc (__libc_memcpy, select_memcpy_ifunc); + +# undef memcpy +strong_alias (__libc_memcpy, memcpy); +# ifdef SHARED +__hidden_ver1 (memcpy, __GI_memcpy, __redirect_memcpy) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memcpy); +# endif + +#endif diff --git a/sysdeps/riscv/memcpy_noalignment.S b/sysdeps/riscv/memcpy_noalignment.S new file mode 100644 index 0000000000..f3bf8e5867 --- /dev/null +++ b/sysdeps/riscv/memcpy_noalignment.S @@ -0,0 +1,138 @@ +/* memcpy for RISC-V, ignoring buffer alignment + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#include +#include + +/* void *memcpy(void *, const void *, size_t) */ +ENTRY (__memcpy_noalignment) + move t6, a0 /* Preserve return value */ + + /* Bail if 0 */ + beqz a2, 7f + + /* Jump to byte copy if size < SZREG */ + li a4, SZREG + bltu a2, a4, 5f + + /* Round down to the nearest "page" size */ + andi a4, a2, ~((16*SZREG)-1) + beqz a4, 2f + add a3, a1, a4 + + /* Copy the first word to get dest word aligned */ + andi a5, t6, SZREG-1 + beqz a5, 1f + REG_L a6, (a1) + REG_S a6, (t6) + + /* Align dst up to a word, move src and size as well. */ + addi t6, t6, SZREG-1 + andi t6, t6, ~(SZREG-1) + sub a5, t6, a0 + add a1, a1, a5 + sub a2, a2, a5 + + /* Recompute page count */ + andi a4, a2, ~((16*SZREG)-1) + beqz a4, 2f + +1: + /* Copy "pages" (chunks of 16 registers) */ + REG_L a4, 0(a1) + REG_L a5, SZREG(a1) + REG_L a6, 2*SZREG(a1) + REG_L a7, 3*SZREG(a1) + REG_L t0, 4*SZREG(a1) + REG_L t1, 5*SZREG(a1) + REG_L t2, 6*SZREG(a1) + REG_L t3, 7*SZREG(a1) + REG_L t4, 8*SZREG(a1) + REG_L t5, 9*SZREG(a1) + REG_S a4, 0(t6) + REG_S a5, SZREG(t6) + REG_S a6, 2*SZREG(t6) + REG_S a7, 3*SZREG(t6) + REG_S t0, 4*SZREG(t6) + REG_S t1, 5*SZREG(t6) + REG_S t2, 6*SZREG(t6) + REG_S t3, 7*SZREG(t6) + REG_S t4, 8*SZREG(t6) + REG_S t5, 9*SZREG(t6) + REG_L a4, 10*SZREG(a1) + REG_L a5, 11*SZREG(a1) + REG_L a6, 12*SZREG(a1) + REG_L a7, 13*SZREG(a1) + REG_L t0, 14*SZREG(a1) + REG_L t1, 15*SZREG(a1) + addi a1, a1, 16*SZREG + REG_S a4, 10*SZREG(t6) + REG_S a5, 11*SZREG(t6) + REG_S a6, 12*SZREG(t6) + REG_S a7, 13*SZREG(t6) + REG_S t0, 14*SZREG(t6) + REG_S t1, 15*SZREG(t6) + addi t6, t6, 16*SZREG + bltu a1, a3, 1b + andi a2, a2, (16*SZREG)-1 /* Update count */ + +2: + /* Remainder is smaller than a page, compute native word count */ + beqz a2, 7f + andi a5, a2, ~(SZREG-1) + andi a2, a2, (SZREG-1) + add a3, a1, a5 + /* Jump directly to last word if no words. */ + beqz a5, 4f + +3: + /* Use single native register copy */ + REG_L a4, 0(a1) + addi a1, a1, SZREG + REG_S a4, 0(t6) + addi t6, t6, SZREG + bltu a1, a3, 3b + + /* Jump directly out if no more bytes */ + beqz a2, 7f + +4: + /* Copy the last word unaligned */ + add a3, a1, a2 + add a4, t6, a2 + REG_L a5, -SZREG(a3) + REG_S a5, -SZREG(a4) + ret + +5: + /* Copy bytes when the total copy is . */ + +#include + +extern __typeof (memcpy) __memcpy_generic; +hidden_proto (__memcpy_generic) + +#include