From patchwork Thu Nov 30 18:32:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 81045 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 190933858429 for ; Thu, 30 Nov 2023 18:33:04 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by sourceware.org (Postfix) with ESMTPS id A9992385840F for ; Thu, 30 Nov 2023 18:32:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A9992385840F Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A9992385840F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::431 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369171; cv=none; b=wq21F+qx8M7ERjx9mRipM91F1THA2Du46MnpIJfZuq6PhHfWv6Npp9g4jHp5AY0dIWkRrCwlSsBGXVT4ipNpKnN/Gp0NeohNjCJEAIAIfiTNnOosqNDr9IKSGDpxEtRC5MCqVAMmVLdV2nl90YbTHKVT+ZPOI+hiN4q/wLA/soM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369171; c=relaxed/simple; bh=3oYD465zr4RGAxWGzlktISxi4N4g+BZtAa/TAu5xdT0=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=vSEwxLO1lsd78c654M1wj7ps7VoW+P3mvXQ/v4FEVtwYpbdPJQtoP2jADvX9E0oZRegmMYGzYD5C/nmtEUPDGBSACKL1AuueTdePsLfptE5U0NPrxCJYE6HLz/ww/PaQ82wTSighoPwE10XoihshYROKgqeQrgJsXGw9ql7BYt4= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-6cdcef787ffso1274207b3a.0 for ; Thu, 30 Nov 2023 10:32:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1701369168; x=1701973968; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6Zx45RzJbxceNnEsxeYJuktuapS+y4xNuSKbuBCP5d0=; b=w+qML+YsJhKNWUi60o7ZzSPOTrgfTPXjjy5mEGsaTyX/Vh8FdeUhgzu0SXw+bsxzJg FfA0ZrFpHajNm7qPM960J86unwuAl/ipHk6CYM2h2mQ9DEGEAa4Qi8M+k1TscalHwlrz O1yQYhZKfkBeaFhdrEGbFHZKKPuXPr6B0/692fnaMyTTwcFQKDs0bqX1ZbAo7PZpaCtg f4c82BJmkNcx0QY4iewxhRlRxRln0Otj0ukfdUU9td/qi4YOnE84EDJ/tuIeK9QFfEq8 InS+A/vuvG/KxPZxINyNooSlBBsS3P5MBOCsnOszGzUgQ9NEU57DwEq/nHz5t4sp9jGf D9Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701369168; x=1701973968; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6Zx45RzJbxceNnEsxeYJuktuapS+y4xNuSKbuBCP5d0=; b=YwkqkuCMvf6dskN/dy6jGEXXJg1KIYv2SWWavuUvUEY01tEtQkhyLN8Cdh+xZCfyJF 31pR2td3I5wxmyynHIJitd5oA+5u8CruaBxJKs+5oAYgxdGnupCSojr292p+AbdsJMX3 Ig48nFz37BIUIG9HFy6AogPuDNnbGUCBrI1csZwL+ybU6UP99PZtLjppXZDA6d4NZAPf 19FcoBIoFGBrFE8IJhD/shuReptOwnSQZwnU6C0W52LptyVqEzOseX1bAfJLGo/pu+JH hKkqX9o++SSe1bg47zxpWpCVchZtW10YVIvvPPfQeq427L9BMdJxDgtUGV0Z+t+a2ha1 gGAg== X-Gm-Message-State: AOJu0YwRXkSmK4jSOWwyJ/AD3/4mlta0cbmQqoUi03zHL+dDEObrpBLL aRe9ploVKkMLhwPd6+ruA2zcUSGmwbCSsrjLoIE= X-Google-Smtp-Source: AGHT+IEZaF3ZSBEk3/RUEUzU5foSqCr7hMdLT45Dd+PRXrbeiLIY1r8XkBCr5WR9AjQR4xQrSBCldg== X-Received: by 2002:a05:6a21:789e:b0:18c:3a58:fcd3 with SMTP id bf30-20020a056a21789e00b0018c3a58fcd3mr24086162pzc.23.1701369168249; Thu, 30 Nov 2023 10:32:48 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id p3-20020a62ab03000000b0068fd026b496sm1512939pff.46.2023.11.30.10.32.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 10:32:47 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: Florian Weimer , palmer@rivosinc.com, vineetg@rivosinc.com, slewis@rivosinc.com, Evan Green Subject: [PATCH v9 1/6] riscv: Add Linux hwprobe syscall support Date: Thu, 30 Nov 2023 10:32:34 -0800 Message-Id: <20231130183239.598100-2-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231130183239.598100-1-evan@rivosinc.com> References: <20231130183239.598100-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add awareness and a thin wrapper function around a new Linux system call that allows callers to get architecture and microarchitecture information about the CPUs from the kernel. This can be used to do things like dynamically choose a memcpy implementation. Signed-off-by: Evan Green Reviewed-by: Palmer Dabbelt --- Changes in v9: - Alphabetize new entries in libc.abilist (to pass checks) Changes in v8: - Fix missed 2.39 in abilists (Joseph) - Just return -r (Florian) Changes in v7: - Bumped Versions up to 2.39 (Joseph) - Used INTERNAL_SYSCALL_CALL, and return positive errno to match pthreads API (Florian). - Remove __THROW since it creates a warning in combination with the fortified access decorators. Changes in v6: - Prefixed __riscv_hwprobe() parameters names with __ to avoid user macro namespace pollution (Joseph) Changes in v4: - Remove __USE_GNU (Florian) - __nonnull, __wur, __THROW, and __fortified_attr_access decorations (Florian) - change long to long int (Florian) - Fix comment formatting (Florian) - Update backup kernel header content copy. - Fix function declaration formatting (Florian) - Changed export versions to 2.38 Changes in v3: - Update argument types to match v4 kernel interface Changes in v2: - hwprobe.h: Use __has_include and duplicate Linux content to make compilation work when Linux headers are absent (Adhemerval) - hwprobe.h: Put declaration under __USE_GNU (Adhemerval) - Use INLINE_SYSCALL_CALL (Adhemerval) - Update versions - Update UNALIGNED_MASK to match kernel v3 series. sysdeps/unix/sysv/linux/riscv/Makefile | 4 +- sysdeps/unix/sysv/linux/riscv/Versions | 3 + sysdeps/unix/sysv/linux/riscv/hwprobe.c | 36 ++++++++++ .../unix/sysv/linux/riscv/rv32/libc.abilist | 1 + .../unix/sysv/linux/riscv/rv64/libc.abilist | 1 + sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 72 +++++++++++++++++++ 6 files changed, 115 insertions(+), 2 deletions(-) create mode 100644 sysdeps/unix/sysv/linux/riscv/hwprobe.c create mode 100644 sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h diff --git a/sysdeps/unix/sysv/linux/riscv/Makefile b/sysdeps/unix/sysv/linux/riscv/Makefile index 4b6eacb32f..45cc29e40d 100644 --- a/sysdeps/unix/sysv/linux/riscv/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/Makefile @@ -1,6 +1,6 @@ ifeq ($(subdir),misc) -sysdep_headers += sys/cachectl.h -sysdep_routines += flush-icache +sysdep_headers += sys/cachectl.h sys/hwprobe.h +sysdep_routines += flush-icache hwprobe endif ifeq ($(subdir),stdlib) diff --git a/sysdeps/unix/sysv/linux/riscv/Versions b/sysdeps/unix/sysv/linux/riscv/Versions index 5625d2a0b8..8717b62a4a 100644 --- a/sysdeps/unix/sysv/linux/riscv/Versions +++ b/sysdeps/unix/sysv/linux/riscv/Versions @@ -8,4 +8,7 @@ libc { GLIBC_2.27 { __riscv_flush_icache; } + GLIBC_2.39 { + __riscv_hwprobe; + } } diff --git a/sysdeps/unix/sysv/linux/riscv/hwprobe.c b/sysdeps/unix/sysv/linux/riscv/hwprobe.c new file mode 100644 index 0000000000..e28194e344 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/hwprobe.c @@ -0,0 +1,36 @@ +/* RISC-V hardware feature probing support on Linux + Copyright (C) 2023 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include + +int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) +{ + int r; + + r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, 5, __pairs, __pair_count, + __cpu_count, __cpus, __flags); + + /* Negate negative errno values to match pthreads API. */ + return -r; +} diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist index a13c484582..6c96e25552 100644 --- a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist +++ b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist @@ -2436,6 +2436,7 @@ GLIBC_2.38 strlcat F GLIBC_2.38 strlcpy F GLIBC_2.38 wcslcat F GLIBC_2.38 wcslcpy F +GLIBC_2.39 __riscv_hwprobe F GLIBC_2.39 pidfd_getpid F GLIBC_2.39 pidfd_spawn F GLIBC_2.39 pidfd_spawnp F diff --git a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist index cf65d8d6d4..225278807b 100644 --- a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist +++ b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist @@ -2636,6 +2636,7 @@ GLIBC_2.38 strlcat F GLIBC_2.38 strlcpy F GLIBC_2.38 wcslcat F GLIBC_2.38 wcslcpy F +GLIBC_2.39 __riscv_hwprobe F GLIBC_2.39 pidfd_getpid F GLIBC_2.39 pidfd_spawn F GLIBC_2.39 pidfd_spawnp F diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h new file mode 100644 index 0000000000..aa189a4818 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -0,0 +1,72 @@ +/* RISC-V architecture probe interface + Copyright (C) 2023 Free Software Foundation, Inc. + + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#ifndef _SYS_HWPROBE_H +#define _SYS_HWPROBE_H 1 + +#include +#include +#ifdef __has_include +# if __has_include () +# include +# endif +#endif + +/* Define a (probably stale) version of the interface if the Linux headers + aren't present. */ +#ifndef RISCV_HWPROBE_KEY_MVENDORID +struct riscv_hwprobe { + signed long long int key; + unsigned long long int value; +}; + +#define RISCV_HWPROBE_KEY_MVENDORID 0 +#define RISCV_HWPROBE_KEY_MARCHID 1 +#define RISCV_HWPROBE_KEY_MIMPID 2 +#define RISCV_HWPROBE_KEY_BASE_BEHAVIOR 3 +#define RISCV_HWPROBE_BASE_BEHAVIOR_IMA (1 << 0) +#define RISCV_HWPROBE_KEY_IMA_EXT_0 4 +#define RISCV_HWPROBE_IMA_FD (1 << 0) +#define RISCV_HWPROBE_IMA_C (1 << 1) +#define RISCV_HWPROBE_IMA_V (1 << 2) +#define RISCV_HWPROBE_EXT_ZBA (1 << 3) +#define RISCV_HWPROBE_EXT_ZBB (1 << 4) +#define RISCV_HWPROBE_EXT_ZBS (1 << 5) +#define RISCV_HWPROBE_KEY_CPUPERF_0 5 +#define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) +#define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) +#define RISCV_HWPROBE_MISALIGNED_SLOW (2 << 0) +#define RISCV_HWPROBE_MISALIGNED_FAST (3 << 0) +#define RISCV_HWPROBE_MISALIGNED_UNSUPPORTED (4 << 0) +#define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) + +#endif /* RISCV_HWPROBE_KEY_MVENDORID */ + +__BEGIN_DECLS + +extern int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) + __nonnull ((1)) __wur + __fortified_attr_access (__read_write__, 1, 2) + __fortified_attr_access (__read_only__, 4, 3); + +__END_DECLS + +#endif /* sys/hwprobe.h */ From patchwork Thu Nov 30 18:32:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 81047 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1392B385C335 for ; Thu, 30 Nov 2023 18:33:27 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by sourceware.org (Postfix) with ESMTPS id 23D3A385C41D for ; Thu, 30 Nov 2023 18:32:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 23D3A385C41D Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 23D3A385C41D Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::42b ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369174; cv=none; b=xKynnQ72ZpcNsf2lhz54f9+yeSkIrf1ADM+lpVq4QOvI7/ySB7pTh8j1C1Q1YdrqxEWy3Q1nNaCahClpbEGeE4cfKtcJNTEZSBiSVNdY2uNuUU+3u5DAHvhuBqgNvO9EfNKf6CN84/EGC2ZR58zON9b0qay4StHDrHF/g/b2DuM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369174; c=relaxed/simple; bh=2CQCiFKGehRU5q8X4tM1luxvwAjjrXZaL8ntousGfWM=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=b5Y3PESdI51GnY4e2B5595nT5BAyNGcva+SsmTUOrU/6zFOBHtLzw/bOCoVVJYw4PfZyivLRg3Xlsb/mpW5imaim6gbZ0rybm2oARHhA83fekIG2/Z7TiPph2+h6MnQahf/z+rCrdcL+29hS7A7P3Us3rnt9GkJwKbF2kBDz8gE= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-6cb55001124so1900438b3a.0 for ; Thu, 30 Nov 2023 10:32:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1701369171; x=1701973971; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aAhWPSgvktYgj2fzRCxEswRwk7DllMq47Tg6ba8f3Mc=; b=ozQO+ldTHzYe0vXloMlnMeBIeDOhk27GhgJradnIY9WKah9O9aEWac5Y5MXLEAQzqq EIf0vYG+4XsinmB9uCna00izo0XBFrquWgCk/3WCNHQsfkEjW7pzId1Y8pesd5r+W8PP FO4wIL5iucxvztCUYPiXA4+xO7VCITNxfxbReuRaDwLZwZTd4CP+cMckABOpUCazEy9r dMo06aMg9HvU5rCPVCUr3sNTrNRaIpiBELTkEfGBD/g3adJaIXjBoM1s5koV98+eAMxa 0YoH0dSkgcQ4OFJhUzw67+WRCVdBOro38TKGYxUIymC/seJlHsoDV3BYLsKtMWk1wNZ0 roow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701369171; x=1701973971; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aAhWPSgvktYgj2fzRCxEswRwk7DllMq47Tg6ba8f3Mc=; b=RWr02N9jCQcHJGDSYHX1RF4OPiZZUY4gw9bpwPp60066SOmEpqjtvvRVWUY/CGZ7tN UvGwAUnGOERbe6UIVwqbUa2bY5JUj9z1rLeu7prR96jDkn1MOS6QcVZgS+NYL1VZEfHN 1HgM5zNnx/6h90ZP141vewqOw9aLlIVkiyg+GnrUOVkiFYVw+d7pVwTRqVAJPWMJ2b0Q bNG6kd2KJ6+lO4w7xFUclejGcVU+5/KfQZcOy5KpRuFWL/eHreHSFW5rKXbtD3cOnc5N ZfAxl8os7s3DOcsSBULkhn/eLEX3NG0xvO63uvrWa4BoiL9k8N/XPtNzrfAGE8Uh+Gqc VUmA== X-Gm-Message-State: AOJu0YzsuJZtfsavNC9CT8axi/br3H7P1W7Glg7j8niPXN6G4aaeih9H xcoe4IIft0srT8lv+PZGaki5DDoFqgR464kEfXw= X-Google-Smtp-Source: AGHT+IFxkwEqnPAsuhpZqfxqX/w2icAX9foOUC9ykZhSNUYaMPAVZy5dXUzrwGP9FfkYPoogd6llrg== X-Received: by 2002:a05:6a20:4401:b0:18a:d4ea:1678 with SMTP id ce1-20020a056a20440100b0018ad4ea1678mr33741954pzb.15.1701369171182; Thu, 30 Nov 2023 10:32:51 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id p3-20020a62ab03000000b0068fd026b496sm1512939pff.46.2023.11.30.10.32.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 10:32:50 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: Florian Weimer , palmer@rivosinc.com, vineetg@rivosinc.com, slewis@rivosinc.com, Evan Green Subject: [PATCH v9 2/6] riscv: Add hwprobe vdso call support Date: Thu, 30 Nov 2023 10:32:35 -0800 Message-Id: <20231130183239.598100-3-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231130183239.598100-1-evan@rivosinc.com> References: <20231130183239.598100-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The new riscv_hwprobe syscall also comes with a vDSO for faster answers to your most common questions. Call in today to speak with a kernel representative near you! Signed-off-by: Evan Green Reviewed-by: Palmer Dabbelt --- (no changes since v7) Changes in v7: - Use INTERNAL_VSYSCALL_CALL (Florian) Changes in v3: - Add the "return" to the vsyscall - Fix up vdso arg types to match kernel v4 version - Remove ifdef around INLINE_VSYSCALL (Adhemerval) Changes in v2: - Add vDSO interface sysdeps/unix/sysv/linux/dl-vdso-setup.c | 10 ++++++++++ sysdeps/unix/sysv/linux/dl-vdso-setup.h | 3 +++ sysdeps/unix/sysv/linux/riscv/hwprobe.c | 17 ++++++++++++++--- sysdeps/unix/sysv/linux/riscv/sysdep.h | 1 + 4 files changed, 28 insertions(+), 3 deletions(-) diff --git a/sysdeps/unix/sysv/linux/dl-vdso-setup.c b/sysdeps/unix/sysv/linux/dl-vdso-setup.c index 97eaaeac37..ed8b1ef426 100644 --- a/sysdeps/unix/sysv/linux/dl-vdso-setup.c +++ b/sysdeps/unix/sysv/linux/dl-vdso-setup.c @@ -71,6 +71,16 @@ PROCINFO_CLASS int (*_dl_vdso_clock_getres_time64) (clockid_t, # ifdef HAVE_GET_TBFREQ PROCINFO_CLASS uint64_t (*_dl_vdso_get_tbfreq)(void) RELRO; # endif + +/* RISC-V specific ones. */ +# ifdef HAVE_RISCV_HWPROBE +PROCINFO_CLASS int (*_dl_vdso_riscv_hwprobe)(void *, + size_t, + size_t, + unsigned long *, + unsigned int) RELRO; +# endif + #endif #undef RELRO diff --git a/sysdeps/unix/sysv/linux/dl-vdso-setup.h b/sysdeps/unix/sysv/linux/dl-vdso-setup.h index 867072b897..39eafd5316 100644 --- a/sysdeps/unix/sysv/linux/dl-vdso-setup.h +++ b/sysdeps/unix/sysv/linux/dl-vdso-setup.h @@ -47,6 +47,9 @@ setup_vdso_pointers (void) #ifdef HAVE_GET_TBFREQ GLRO(dl_vdso_get_tbfreq) = dl_vdso_vsym (HAVE_GET_TBFREQ); #endif +#ifdef HAVE_RISCV_HWPROBE + GLRO(dl_vdso_riscv_hwprobe) = dl_vdso_vsym (HAVE_RISCV_HWPROBE); +#endif } #endif diff --git a/sysdeps/unix/sysv/linux/riscv/hwprobe.c b/sysdeps/unix/sysv/linux/riscv/hwprobe.c index e28194e344..6a9a44657f 100644 --- a/sysdeps/unix/sysv/linux/riscv/hwprobe.c +++ b/sysdeps/unix/sysv/linux/riscv/hwprobe.c @@ -27,9 +27,20 @@ int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, unsigned int __flags) { int r; - - r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, 5, __pairs, __pair_count, - __cpu_count, __cpus, __flags); + __riscv_hwprobe_t vdso_hwprobe = + (__riscv_hwprobe_t)GLRO(dl_vdso_riscv_hwprobe); + + if (vdso_hwprobe != NULL) + { + r = INTERNAL_VSYSCALL_CALL (vdso_hwprobe, 5, __pairs, __pair_count, + __cpu_count, __cpus, __flags); + } + else + { + r = INTERNAL_SYSCALL_CALL (riscv_hwprobe, 5, __pairs, __pair_count, + __cpu_count, __cpus, __flags); + + } /* Negate negative errno values to match pthreads API. */ return -r; diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h index 5583b96d23..ee015dfeb6 100644 --- a/sysdeps/unix/sysv/linux/riscv/sysdep.h +++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h @@ -156,6 +156,7 @@ /* List of system calls which are supported as vsyscalls (for RV32 and RV64). */ # define HAVE_GETCPU_VSYSCALL "__vdso_getcpu" +# define HAVE_RISCV_HWPROBE "__vdso_riscv_hwprobe" # undef HAVE_INTERNAL_BRK_ADDR_SYMBOL # define HAVE_INTERNAL_BRK_ADDR_SYMBOL 1 From patchwork Thu Nov 30 18:32:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 81046 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A2CDA385DC29 for ; Thu, 30 Nov 2023 18:33:25 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by sourceware.org (Postfix) with ESMTPS id 73B18385DC13 for ; Thu, 30 Nov 2023 18:32:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 73B18385DC13 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 73B18385DC13 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::42b ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369176; cv=none; b=xbm5KH4tNGCwR8ihdyOA/+LHlZcnLmY4CQSZML0e7ub7YOMRJC3tnB2PiSr0VCCh0FbgcdDwmorwaGimRuMvS7aMKB58WC0/+fatFjtd8pe8U5ggQoII8XOPANb+emnKXfg5irNe2ixykD+aWKm2PG8E1k1/a8FVZTfl9T06QeU= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369176; c=relaxed/simple; bh=k0qoV4sZV+f6eOmuwD87TJs6T5F/SEkcvPU0mNLa1Is=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=Yr1jY4IWH7FpX7rfGKbKMvGYabEqfCNpA/v+ufjIbtPy/UegEZuKwr+nP5UgPPXNuZHRHSnA1eBAVtgfNqAzxw+8W6+m7AJwVmWxo+dDqO7GFBxamQJQnxN+20Nklj9ncrs5Bz88tn7DrznqrYeC7N51Ynf62ozg/9+rQXb1+40= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-6cbd24d9557so1089028b3a.1 for ; Thu, 30 Nov 2023 10:32:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1701369174; x=1701973974; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GUwAVLzxmk4oYzLKuvyk7oaHk08bVDBm+qjcAc1NUeM=; b=UdkzXmYDZyKQ+SkmBXS9opDe03TKdUvUDCV2+3n0ja/glPlzanSTZk+5qRgFeiNuCG ZOPF9A/mLiaT7KPNQy69yz3OpdXulGvoKlmpzE8xG2hWPFkbBLVeUj9Dp2KwWhpHMORq snfM2KTxzuPfJGiBuy1NncarOwORVnF/cZYUaYuMBs9KU65ENysorfergoEYTYut2HXF RUJzEoJizPRiW5jlhTWN3f+Mn9PQjAqqR58j8f2O0f+uweHJHAHYhN4KI2ta4Mp+l+ur 9QwYbGbIWFmCVqdYqpr8wO2X6U04t5j6Ia7KEkZg7YhXrhc+JdY7+c347vomjJAA+E3s 2flw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701369174; x=1701973974; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GUwAVLzxmk4oYzLKuvyk7oaHk08bVDBm+qjcAc1NUeM=; b=qjR92RaVnVsgDG6zNg0QuU9jLgDWu+BvyQUdzPMyWNxOOL9E1xMmVJNXpHsLeYWEMY NKPBBhyVnrI04uC2ZATeovFeDTlb/6bba40jxusalybM6ZEre0vkOQGTpJ+S4KCCb3OZ dr1E5/ZPlsoWe6gKFtZDlX8G+d62Xoo5Dl9MOVVBUMcy4Z59RWlgqjzu7L3KHmZTwzjo G2172NlHybUL0dTgX+s57mkq7RKyN4KPf60bbUu2bbyZNSUGuMTG9Rr5+iNYKz+ELF1U ToIckGZNfz6gkEb5V201Ze9ARt0i/CZPPvoBcFZtRH0jwF88g4fP/il1MyNzRrLpfrsl zBbw== X-Gm-Message-State: AOJu0YyeAEIITVt6h0aka5rpO9pA4YB6W+luGXzQT1JzWCBBQj9ubxLp R9A1WQl32ZKJpuADf2mCXwBnqXTLURfduLuReDM= X-Google-Smtp-Source: AGHT+IEfahLuCdYkaRS2jFGvNOND1X9izn8U3KKhpe46LGgUEF/U9i8O1zBd2s2BSk7aFDAV/amM8g== X-Received: by 2002:a05:6a20:5482:b0:18c:21aa:6a3f with SMTP id i2-20020a056a20548200b0018c21aa6a3fmr28917082pzk.8.1701369174082; Thu, 30 Nov 2023 10:32:54 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id p3-20020a62ab03000000b0068fd026b496sm1512939pff.46.2023.11.30.10.32.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 10:32:53 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: Florian Weimer , palmer@rivosinc.com, vineetg@rivosinc.com, slewis@rivosinc.com, Evan Green Subject: [PATCH v9 3/6] riscv: Add __riscv_hwprobe pointer to ifunc calls Date: Thu, 30 Nov 2023 10:32:36 -0800 Message-Id: <20231130183239.598100-4-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231130183239.598100-1-evan@rivosinc.com> References: <20231130183239.598100-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The new __riscv_hwprobe() function is designed to be used by ifunc selector functions. This presents a challenge for applications and libraries, as ifunc selectors are invoked before all relocations have been performed, so an external call to __riscv_hwprobe() from an ifunc selector won't work. To address this, pass a pointer to the __riscv_hwprobe() vDSO function into ifunc selectors as the second argument (alongside dl_hwcap, which was already being passed). Include a typedef as well for convenience, so that ifunc users don't have to go through contortions to call this routine. Users will need to remember to check the second argument for NULL, both to account for older glibcs that don't pass the function, and older kernels that don't have the vDSO pointer. Signed-off-by: Evan Green --- (no changes since v7) Changes in v7: - Remove __THROW from function pointer type, as it creates warnings together with __fortified_attr_access. sysdeps/riscv/dl-irel.h | 8 ++++---- sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 10 ++++++++++ 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/sysdeps/riscv/dl-irel.h b/sysdeps/riscv/dl-irel.h index eaeec5467c..2147504458 100644 --- a/sysdeps/riscv/dl-irel.h +++ b/sysdeps/riscv/dl-irel.h @@ -31,10 +31,10 @@ static inline ElfW(Addr) __attribute ((always_inline)) elf_ifunc_invoke (ElfW(Addr) addr) { - /* The second argument is a void pointer to preserve the extension - fexibility. */ - return ((ElfW(Addr) (*) (uint64_t, void *)) (addr)) - (GLRO(dl_hwcap), NULL); + /* The third argument is a void pointer to preserve the extension + flexibility. */ + return ((ElfW(Addr) (*) (uint64_t, void *, void *)) (addr)) + (GLRO(dl_hwcap), GLRO(dl_vdso_riscv_hwprobe), NULL); } static inline void diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h index aa189a4818..fd3be5a411 100644 --- a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -67,6 +67,16 @@ extern int __riscv_hwprobe (struct riscv_hwprobe *__pairs, size_t __pair_count, __fortified_attr_access (__read_write__, 1, 2) __fortified_attr_access (__read_only__, 4, 3); +/* A pointer to the __riscv_hwprobe vDSO function is passed as the second + argument to ifunc selector routines. Include a function pointer type for + convenience in calling the function in those settings. */ +typedef int (*__riscv_hwprobe_t) (struct riscv_hwprobe *__pairs, size_t __pair_count, + size_t __cpu_count, unsigned long int *__cpus, + unsigned int __flags) + __nonnull ((1)) __wur + __fortified_attr_access (__read_write__, 1, 2) + __fortified_attr_access (__read_only__, 4, 3); + __END_DECLS #endif /* sys/hwprobe.h */ From patchwork Thu Nov 30 18:32:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 81048 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3FC84385E008 for ; Thu, 30 Nov 2023 18:33:32 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by sourceware.org (Postfix) with ESMTPS id 69923385C423 for ; Thu, 30 Nov 2023 18:32:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 69923385C423 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 69923385C423 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::42c ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369181; cv=none; b=QhveGIg2t3UZsd6GRdkKAt/jo1MOCSdppIHb/hMCp1BXWd9+i9p0qzvJ9H42jPg3gJvR7mqgM2lflbUDTtV2Feo1Y4L9bss9GJI3iCIyECgp57JqAGMhR6nxIhXsBN1Vc7X7jyNV8aoJarMd71iG5KlsMkAkQZFkd3mSr3QPcFw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369181; c=relaxed/simple; bh=95bimGb4Wi1MoWmr4NkrTJX0yPrzk1X0RlkHN7FXf1Y=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=jl5mUYiS1TZ+85F7zxCR3A6rYxl5zLJhV4TviMiH1UxTwKdkK8W2TTJVVf2JLlp1N16OnlN1GGXLuFguthTTMM7Y0zxdo5s71HgB7CO3PySz28XQn5osGnJkPxXB5/lqSO+RIG1MLLble8MwKWSLwHa+iXHfpoO5pjAXTe6Vtqw= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-6cbe716b511so1262225b3a.3 for ; Thu, 30 Nov 2023 10:32:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1701369177; x=1701973977; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gQBKb7U+e0DvkgZoB/Y+fqXWGiLbnYCp30WGnIDkx6A=; b=Qg3+dkwHc0ScK3n19e6jyShHtqqtETjrDqPSjsjktkN8Ut0F5Re3N5kzQJJjvAR6gA 4iUx5b9iQqioPmSEMZCp4+F9YCIAaJfAYiXJ6y0EufHfGo5imocmDjgW2G5ymFwJRh1n 7mKgh53RpDZ95KV2+y2rJE5ojigmW/SC1QT7kY+n6wQWmg3YylU4/CcfuxzmHTLBUe2n +Cx9VERNqzm1zIQQ/ZgjVtb+b2Ub50aSyOvYVupIA1ncDLYeu/41eNPFLDS5JJzbE9Hu a7vZmnBwYX6nC/Gx5pzPuZXKHnhprJGJhhjILPo9ZkIlG/W3GmLaKbR9fK0aMuoBlPeE 3/wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701369177; x=1701973977; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gQBKb7U+e0DvkgZoB/Y+fqXWGiLbnYCp30WGnIDkx6A=; b=MVqLU8zVa8DrS+LaP5evY69NtwHlH5u5Fas2F3gVcdW5rD7khpWKiJB/VQIhO1lusu lfXYKiuw87zF9N5K0Y5znwjZkZaARv8GxOqInFIfETPt3klrxJO1fO0bMsCMQBd1NRoZ vYA3BGfSr6xFO/7BsiuZhOmvp6EyF9I702dQ6ocqjJ8wBXp47lNfK4Nmg7wzzsZ7kfr4 /WMvIcjsTmi/Zd3ZDNQUJdrWtbDsoK9FaP7QAwvbsIvtreUQo+1H9ayho7W69kwy9Ey6 ciuu3yebHPqlN+DYWmEe4rC13phQ9vBsgSudJP9creMESKWwvCkJR9OsVkN51Jp4lcYr kfpg== X-Gm-Message-State: AOJu0YztnwHlFVbltrwqSqImqvCLvGjg5uiowX/6dUrLWFvWrgNzJhCG 7G/lyT1hpPqlwqA2MyrMNycMdwdoPLAA5NeHHUI= X-Google-Smtp-Source: AGHT+IHR1ubDtvD6CKPTwSTG5/lr+os7B4rNE+SajRaVqPhb3gS3Ismpwjmpq3Gc59lc7UzFPnM8tw== X-Received: by 2002:a05:6a00:2918:b0:6bd:3157:2dfe with SMTP id cg24-20020a056a00291800b006bd31572dfemr23666236pfb.7.1701369177169; Thu, 30 Nov 2023 10:32:57 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id p3-20020a62ab03000000b0068fd026b496sm1512939pff.46.2023.11.30.10.32.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 10:32:56 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: Florian Weimer , palmer@rivosinc.com, vineetg@rivosinc.com, slewis@rivosinc.com, Evan Green Subject: [PATCH v9 4/6] riscv: Enable multi-arg ifunc resolvers Date: Thu, 30 Nov 2023 10:32:37 -0800 Message-Id: <20231130183239.598100-5-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231130183239.598100-1-evan@rivosinc.com> References: <20231130183239.598100-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, URIBL_BLACK autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org RISC-V is apparently the first architecture to pass more than one argument to ifunc resolvers. The helper macros in libc-symbols.h, __ifunc_resolver(), __ifunc(), and __ifunc_hidden(), are incompatible with this. These macros have an "arg" (non-final) parameter that represents the parameter signature of the ifunc resolver. The result is an inability to pass the required comma through in a single preprocessor argument. Rearrange the __ifunc_resolver() macro to be variadic, and pass the types as those variable parameters. Move the guts of __ifunc() and __ifunc_hidden() into new macros, __ifunc_args(), and __ifunc_args_hidden(), that pass the variable arguments down through to __ifunc_resolver(). Then redefine __ifunc() and __ifunc_hidden(), which are used in a bunch of places, to simply shuffle the arguments down into __ifunc_args[_hidden]. Finally, define a riscv-ifunc.h header, which provides convenience macros to those looking to write ifunc selectors that use both arguments. Signed-off-by: Evan Green --- Changes in v9: - Fix a couple of typos causing powerpc not to build (build-many-glibcs) Changes in v6: - Introduced riscv-ifunc.h for multi-arg ifunc selectors. --- include/libc-symbols.h | 28 +++++++++++++++++----------- sysdeps/riscv/riscv-ifunc.h | 27 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 11 deletions(-) create mode 100644 sysdeps/riscv/riscv-ifunc.h diff --git a/include/libc-symbols.h b/include/libc-symbols.h index 5794614488..b45fad72b3 100644 --- a/include/libc-symbols.h +++ b/include/libc-symbols.h @@ -665,9 +665,9 @@ for linking") #endif /* Helper / base macros for indirect function symbols. */ -#define __ifunc_resolver(type_name, name, expr, arg, init, classifier) \ +#define __ifunc_resolver(type_name, name, expr, init, classifier, ...) \ classifier inhibit_stack_protector \ - __typeof (type_name) *name##_ifunc (arg) \ + __typeof (type_name) *name##_ifunc (__VA_ARGS__) \ { \ init (); \ __typeof (type_name) *res = expr; \ @@ -675,13 +675,13 @@ for linking") } #ifdef HAVE_GCC_IFUNC -# define __ifunc(type_name, name, expr, arg, init) \ +# define __ifunc_args(type_name, name, expr, init, ...) \ extern __typeof (type_name) name __attribute__ \ ((ifunc (#name "_ifunc"))); \ - __ifunc_resolver (type_name, name, expr, arg, init, static) + __ifunc_resolver (type_name, name, expr, init, static, __VA_ARGS__) -# define __ifunc_hidden(type_name, name, expr, arg, init) \ - __ifunc (type_name, name, expr, arg, init) +# define __ifunc_args_hidden(type_name, name, expr, init, ...) \ + __ifunc_args (type_name, name, expr, init, __VA_ARGS__) #else /* Gcc does not support __attribute__ ((ifunc (...))). Use the old behaviour as fallback. But keep in mind that the debug information for the ifunc @@ -692,18 +692,24 @@ for linking") different signatures. (Gcc support is disabled at least on a ppc64le Ubuntu 14.04 system.) */ -# define __ifunc(type_name, name, expr, arg, init) \ +# define __ifunc_args(type_name, name, expr, init, ...) \ extern __typeof (type_name) name; \ - __typeof (type_name) *name##_ifunc (arg) __asm__ (#name); \ - __ifunc_resolver (type_name, name, expr, arg, init,) \ + __typeof (type_name) *name##_ifunc (__VA_ARGS__) __asm__ (#name); \ + __ifunc_resolver (type_name, name, expr, init, , __VA_ARGS__) \ __asm__ (".type " #name ", %gnu_indirect_function"); -# define __ifunc_hidden(type_name, name, expr, arg, init) \ +# define __ifunc_args_hidden(type_name, name, expr, init, ...) \ extern __typeof (type_name) __libc_##name; \ - __ifunc (type_name, __libc_##name, expr, arg, init) \ + __ifunc (type_name, __libc_##name, expr, __VA_ARGS__, init) \ strong_alias (__libc_##name, name); #endif /* !HAVE_GCC_IFUNC */ +#define __ifunc(type_name, name, expr, arg, init) \ + __ifunc_args (type_name, name, expr, init, arg) + +#define __ifunc_hidden(type_name, name, expr, arg, init) \ + __ifunc_args_hidden (type_name, name, expr, init, arg) + /* The following macros are used for indirect function symbols in libc.so. First of all, you need to have the function prototyped somewhere, say in foo.h: diff --git a/sysdeps/riscv/riscv-ifunc.h b/sysdeps/riscv/riscv-ifunc.h new file mode 100644 index 0000000000..7bff591d1e --- /dev/null +++ b/sysdeps/riscv/riscv-ifunc.h @@ -0,0 +1,27 @@ +/* Common definition for ifunc resolvers. Linux/RISC-V version. + This file is part of the GNU C Library. + Copyright (C) 2023 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include + +#define INIT_ARCH() + +#define riscv_libc_ifunc(name, expr) \ + __ifunc_args (name, name, expr(hwcap, hwprobe), INIT_ARCH, \ + uint64_t hwcap, __riscv_hwprobe_t hwprobe) From patchwork Thu Nov 30 18:32:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 81049 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 95D47385C40D for ; Thu, 30 Nov 2023 18:33:54 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by sourceware.org (Postfix) with ESMTPS id 705953858289 for ; Thu, 30 Nov 2023 18:33:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 705953858289 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 705953858289 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::435 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369182; cv=none; b=fveMmcj4iJ++CQdt0dXpR0QRtsQUcl/kJjVnk13UPobUzmNErQsf3KZQAG2rfU3/Z7wesQO+qFkdXmAEI92f2zYARrik/d7BIKd5fb38d+IJbFCUUY/x5sDJrCIAwooWnjpg9y33GZX336l7fVdhRX2FDEMuGrzH3CB4gmVunkg= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369182; c=relaxed/simple; bh=eDyim4uvTG6XXlmvYWAkgIGhahfrW5Bt6EAaqKCsMJk=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=nvcoycqDZItGS3BxERBAsDaGzSkhAe5LO7B0BOKH3HlVgm9ONo9qci3qpaPVuN7Hc6WfMmS5a0ELX3ofC6DZp8ydFAJnK3MTlZAfgnbV4QFpWrjsrtaloHYN2HdZ3M7gj1JScWK+QdQizztwl7TKvTasb1DFWHEJYBSd3ZzPruk= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-6c3363a2b93so1209394b3a.3 for ; Thu, 30 Nov 2023 10:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1701369179; x=1701973979; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tadLb3L1akR4SNQCvuoG4lZt9bZ8gdAM7B8yOFdnevk=; b=AICFC++fIHAzwhAaiTg12vunJnAAG/s6plR5c9VDcM7xLpErO8OkuUkUb++4YwvIUs rbvuZwUHMzmP7uW7naTtNrm2hnTg+bUGxcFgFqqc7zThH809J1sPbx4RdG6dppXOmEbi 9Wm2urukoEyaSbsCWtg6RIkBHVVOdh2GqQhht9EunWf/pXbLHX8raFpAGPlLGqkodvRb 26Yvn7yQDg9hhq67IsPtK4MybSqMNdCfUvUrKxDoAJyg1jTTViduaWTZ627Dkx3rjeow SRHeQLDw+WexLXIuS1o7RbhoLKtZAsG/PDW4HPzIxFvTOHaXlGCA8mzuBv/nMjPvYfin rspg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701369179; x=1701973979; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tadLb3L1akR4SNQCvuoG4lZt9bZ8gdAM7B8yOFdnevk=; b=a3jAE//wxtB1S5KsV1K3d+0WjTgiFK0TKSl6KTXxEiaIGgHu1q3huBkPiP935790cs ENJrT8Eu1PvpvRXIBGVUz3IkCwI5tustufVyP1OxwXuLYSYhzYI7mWOMzmuj7wUBiD8G uQDOWty2iB5QER61QpcbxZEVqBiNPwyxzVaKKNjsNiR0BaAKzWo2IGOeQjR5gHwmP6uQ vPhJH2hX3Sjee+52a2FS5QTuj3Q6jvEthyr52WE6OPGEzpXv2rfRA9dMF8g/1e/TVS6q yLMwL0Z18FC/ETVWUm+WKeGAWvXaVk9DFkWqzwzS269BU/xeLeYRwqjQWiqa8Zzs2RsZ uNBw== X-Gm-Message-State: AOJu0YwTHZM0GceiiWnnHAzJSfIvo5QE6T3QvRoqnVXegHhATOZciLKw vYn9sQlytW2maB/86fC0jHor4qeTib+RTEBciFM= X-Google-Smtp-Source: AGHT+IF2TCWnsAQZH5vAWKIfuQnX8b+PFQ840w2brDKjF6tHSHNjqZMMlMC12jnWBhm1HeJf1kO3Rw== X-Received: by 2002:a05:6a20:d396:b0:189:a6b3:ef7b with SMTP id iq22-20020a056a20d39600b00189a6b3ef7bmr30549370pzb.20.1701369179565; Thu, 30 Nov 2023 10:32:59 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id p3-20020a62ab03000000b0068fd026b496sm1512939pff.46.2023.11.30.10.32.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 10:32:59 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: Florian Weimer , palmer@rivosinc.com, vineetg@rivosinc.com, slewis@rivosinc.com, Evan Green Subject: [PATCH v9 5/6] riscv: Add ifunc helper method to hwprobe.h Date: Thu, 30 Nov 2023 10:32:38 -0800 Message-Id: <20231130183239.598100-6-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231130183239.598100-1-evan@rivosinc.com> References: <20231130183239.598100-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Add a little helper method so it's easier to fetch a single value from the hwprobe function when used within an ifunc selector. Signed-off-by: Evan Green --- Changes in v9: - Use __inline rather than inline so c89 compiles (build-many-glibcs) Changes in v7: - Introduced static inline helper (Richard) sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h | 25 +++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h index fd3be5a411..ee7eed3960 100644 --- a/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h +++ b/sysdeps/unix/sysv/linux/riscv/sys/hwprobe.h @@ -22,6 +22,7 @@ #include #include +#include #ifdef __has_include # if __has_include () # include @@ -79,4 +80,28 @@ typedef int (*__riscv_hwprobe_t) (struct riscv_hwprobe *__pairs, size_t __pair_c __END_DECLS +/* Helper function usable from ifunc selectors that probes a single key. */ +static __inline int +__riscv_hwprobe_one(__riscv_hwprobe_t hwprobe_func, + signed long long int key, + unsigned long long int *value) +{ + struct riscv_hwprobe pair; + int rc; + + if (!hwprobe_func) + return ENOSYS; + + pair.key = key; + rc = hwprobe_func(&pair, 1, 0, NULL, 0); + if (rc) + return rc; + + if (pair.key < 0) + return ENOENT; + + *value = pair.value; + return 0; +} + #endif /* sys/hwprobe.h */ From patchwork Thu Nov 30 18:32:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Evan Green X-Patchwork-Id: 81050 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 4C694385803E for ; Thu, 30 Nov 2023 18:34:18 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by sourceware.org (Postfix) with ESMTPS id DDC15385734F for ; Thu, 30 Nov 2023 18:33:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org DDC15385734F Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org DDC15385734F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::436 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369187; cv=none; b=PvyuABJrWla5jckE0Wi/7Yr67kJDWbZHYiERwvBg7OVlgEMCWpjVPxqOwfYmkCr13MsPb+hzg8mjkpcOwhVqbTzHQoxB6eiUaM0d0LXrzK+CTpC+EQznCDB/cOvA3+QX4ex12sdSveoK5OlTuLZkDh9dlzVPFoWKpOUMnDqpe7Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701369187; c=relaxed/simple; bh=mktwsXu8dR4PnTbK/gCT6rPHG12px+fZpESDranSVUc=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=sjzA2kFofpVHdufCTj+SpiESUPu1y11aK8fh5QHQiBWQHWugQzRysHQIHLR8KnTZZOgzDfdsXhkBnzCcPsmZ8l5Wiw2Dw4QGJyf02UbzpFtKPm4nyBc0uHW4PprT5723GKQJHLP6SFz1MOji2XfVXNwaTFNvzFujT8q9ji+nRnI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6c3363a2b93so1209429b3a.3 for ; Thu, 30 Nov 2023 10:33:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1701369182; x=1701973982; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QkP9/BWYF54S2JFmsoNLjYe3SFILFKlRDe8fZrAVdTk=; b=x3M8byiINPWm1oo37VAbk+/nbkdoDWE2NgEB/3XgWO54tNGwBwiGTl8vU6EUpuGAgl PtlCGIlU7RtD8wxzcFSQnaIl3JMJ0qi7qWneI9PJ0lB8L5Szv3Mkvr8y/aQN6Z6uNvB1 nzI5KRs5AfEQBqnkNfy7adC1VcQ/O+y9ukJRp97lh9xHPCPQHZJ3TLNsC2lyUy9dm28T noS7tXR0oIcvYfaaBMkep6WkiURt/w9zo0Zk5sKpRbHHOFhlw9rpxPMDQWBeHKJg6R/B Cc1veN+4w1o4c87CIxnUp2S8DT7ub06ieRLTDdEPnxKs1dyop7nX/llnyCofKe8quxXz 36Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701369182; x=1701973982; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QkP9/BWYF54S2JFmsoNLjYe3SFILFKlRDe8fZrAVdTk=; b=b3jcOyehfzqw4QKk4n70eep2kvH90jTZ1CRIzXTIohBwRhXXMmXQwJ4Av+ewFQbn4w ds/2BLZgWD6+lAYjFnuC8a7n4NTmdo1w4c9j09xAK3WzJEG9JjPjcLyxqn8X/TjXYG68 GfTbqVKfCl6BZdMXwpe55rzhMwhmnjqDMFR8Uq3RKkQnBy5p0hX6zaxHnmVaM/NkhjuO fkwaShdiUT8lT0PW1cgoYpuY6Txi7gcWXrhjl9gvJ48FFTlfI1GIqgKSOEa7UgOHOPny E/bZqIxFe3GMacBjTUHxShS1NA2T5pLLN/V3iu6F+m3nK+A0LyUzPY2i2srRgmidfWtI vPAQ== X-Gm-Message-State: AOJu0YynQcZHv7jAdEeKF+U0qtQZUm1YwAqOfU/TC3AmAQGJnH6yQgH5 63wiX4+kK3yXydYFK5wfnebGgsC1od+GqfuaBd0= X-Google-Smtp-Source: AGHT+IGQQdW8CJyGpdpcpDF2PYAMBwEnDmbuw+wKYfczyy0/btEmVx6kkaOZhHcq5ggDMV77gQttjw== X-Received: by 2002:a05:6a00:1914:b0:6b2:6835:2a7f with SMTP id y20-20020a056a00191400b006b268352a7fmr26815703pfi.22.1701369181864; Thu, 30 Nov 2023 10:33:01 -0800 (PST) Received: from evan.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id p3-20020a62ab03000000b0068fd026b496sm1512939pff.46.2023.11.30.10.33.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 10:33:01 -0800 (PST) From: Evan Green To: libc-alpha@sourceware.org Cc: Florian Weimer , palmer@rivosinc.com, vineetg@rivosinc.com, slewis@rivosinc.com, Evan Green Subject: [PATCH v9 6/6] riscv: Add and use alignment-ignorant memcpy Date: Thu, 30 Nov 2023 10:32:39 -0800 Message-Id: <20231130183239.598100-7-evan@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231130183239.598100-1-evan@rivosinc.com> References: <20231130183239.598100-1-evan@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org For CPU implementations that can perform unaligned accesses with little or no performance penalty, create a memcpy implementation that does not bother aligning buffers. It will use a block of integer registers, a single integer register, and fall back to bytewise copy for the remainder. Signed-off-by: Evan Green Reviewed-by: Palmer Dabbelt --- (no changes since v7) Changes in v7: - Use new helper function in memcpy ifunc selector (Richard) Changes in v6: - Fix a couple regressions in the assembly from v5 :/ - Use passed hwprobe pointer in memcpy ifunc selector. Changes in v5: - Do unaligned word access for final trailing bytes (Richard) Changes in v4: - Fixed comment style (Florian) Changes in v3: - Word align dest for large memcpy()s. - Add tags - Remove spurious blank line from sysdeps/riscv/memcpy.c Changes in v2: - Used _MASK instead of _FAST value itself. --- sysdeps/riscv/memcopy.h | 26 ++++ sysdeps/riscv/memcpy.c | 63 ++++++++ sysdeps/riscv/memcpy_noalignment.S | 138 ++++++++++++++++++ sysdeps/unix/sysv/linux/riscv/Makefile | 4 + .../unix/sysv/linux/riscv/memcpy-generic.c | 24 +++ 5 files changed, 255 insertions(+) create mode 100644 sysdeps/riscv/memcopy.h create mode 100644 sysdeps/riscv/memcpy.c create mode 100644 sysdeps/riscv/memcpy_noalignment.S create mode 100644 sysdeps/unix/sysv/linux/riscv/memcpy-generic.c diff --git a/sysdeps/riscv/memcopy.h b/sysdeps/riscv/memcopy.h new file mode 100644 index 0000000000..2b685c8aa0 --- /dev/null +++ b/sysdeps/riscv/memcopy.h @@ -0,0 +1,26 @@ +/* memcopy.h -- definitions for memory copy functions. RISC-V version. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +/* Redefine the generic memcpy implementation to __memcpy_generic, so + the memcpy ifunc can select between generic and special versions. + In rtld, don't bother with all the ifunciness. */ +#if IS_IN (libc) +#define MEMCPY __memcpy_generic +#endif diff --git a/sysdeps/riscv/memcpy.c b/sysdeps/riscv/memcpy.c new file mode 100644 index 0000000000..285ca85f8e --- /dev/null +++ b/sysdeps/riscv/memcpy.c @@ -0,0 +1,63 @@ +/* Multiple versions of memcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017-2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memcpy so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memcpy +# define memcpy __redirect_memcpy +# include +# include +# include +# include +# include + +# define INIT_ARCH() + +extern __typeof (__redirect_memcpy) __libc_memcpy; + +extern __typeof (__redirect_memcpy) __memcpy_generic attribute_hidden; +extern __typeof (__redirect_memcpy) __memcpy_noalignment attribute_hidden; + +static inline __typeof (__redirect_memcpy) * +select_memcpy_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long int value; + + INIT_ARCH (); + + if (__riscv_hwprobe_one(hwprobe_func, RISCV_HWPROBE_KEY_CPUPERF_0, &value)) + return __memcpy_generic; + + if ((value & RISCV_HWPROBE_MISALIGNED_MASK) == RISCV_HWPROBE_MISALIGNED_FAST) + return __memcpy_noalignment; + + return __memcpy_generic; +} + +riscv_libc_ifunc (__libc_memcpy, select_memcpy_ifunc); + +# undef memcpy +strong_alias (__libc_memcpy, memcpy); +# ifdef SHARED +__hidden_ver1 (memcpy, __GI_memcpy, __redirect_memcpy) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memcpy); +# endif + +#endif diff --git a/sysdeps/riscv/memcpy_noalignment.S b/sysdeps/riscv/memcpy_noalignment.S new file mode 100644 index 0000000000..f3bf8e5867 --- /dev/null +++ b/sysdeps/riscv/memcpy_noalignment.S @@ -0,0 +1,138 @@ +/* memcpy for RISC-V, ignoring buffer alignment + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library. If not, see + . */ + +#include +#include + +/* void *memcpy(void *, const void *, size_t) */ +ENTRY (__memcpy_noalignment) + move t6, a0 /* Preserve return value */ + + /* Bail if 0 */ + beqz a2, 7f + + /* Jump to byte copy if size < SZREG */ + li a4, SZREG + bltu a2, a4, 5f + + /* Round down to the nearest "page" size */ + andi a4, a2, ~((16*SZREG)-1) + beqz a4, 2f + add a3, a1, a4 + + /* Copy the first word to get dest word aligned */ + andi a5, t6, SZREG-1 + beqz a5, 1f + REG_L a6, (a1) + REG_S a6, (t6) + + /* Align dst up to a word, move src and size as well. */ + addi t6, t6, SZREG-1 + andi t6, t6, ~(SZREG-1) + sub a5, t6, a0 + add a1, a1, a5 + sub a2, a2, a5 + + /* Recompute page count */ + andi a4, a2, ~((16*SZREG)-1) + beqz a4, 2f + +1: + /* Copy "pages" (chunks of 16 registers) */ + REG_L a4, 0(a1) + REG_L a5, SZREG(a1) + REG_L a6, 2*SZREG(a1) + REG_L a7, 3*SZREG(a1) + REG_L t0, 4*SZREG(a1) + REG_L t1, 5*SZREG(a1) + REG_L t2, 6*SZREG(a1) + REG_L t3, 7*SZREG(a1) + REG_L t4, 8*SZREG(a1) + REG_L t5, 9*SZREG(a1) + REG_S a4, 0(t6) + REG_S a5, SZREG(t6) + REG_S a6, 2*SZREG(t6) + REG_S a7, 3*SZREG(t6) + REG_S t0, 4*SZREG(t6) + REG_S t1, 5*SZREG(t6) + REG_S t2, 6*SZREG(t6) + REG_S t3, 7*SZREG(t6) + REG_S t4, 8*SZREG(t6) + REG_S t5, 9*SZREG(t6) + REG_L a4, 10*SZREG(a1) + REG_L a5, 11*SZREG(a1) + REG_L a6, 12*SZREG(a1) + REG_L a7, 13*SZREG(a1) + REG_L t0, 14*SZREG(a1) + REG_L t1, 15*SZREG(a1) + addi a1, a1, 16*SZREG + REG_S a4, 10*SZREG(t6) + REG_S a5, 11*SZREG(t6) + REG_S a6, 12*SZREG(t6) + REG_S a7, 13*SZREG(t6) + REG_S t0, 14*SZREG(t6) + REG_S t1, 15*SZREG(t6) + addi t6, t6, 16*SZREG + bltu a1, a3, 1b + andi a2, a2, (16*SZREG)-1 /* Update count */ + +2: + /* Remainder is smaller than a page, compute native word count */ + beqz a2, 7f + andi a5, a2, ~(SZREG-1) + andi a2, a2, (SZREG-1) + add a3, a1, a5 + /* Jump directly to last word if no words. */ + beqz a5, 4f + +3: + /* Use single native register copy */ + REG_L a4, 0(a1) + addi a1, a1, SZREG + REG_S a4, 0(t6) + addi t6, t6, SZREG + bltu a1, a3, 3b + + /* Jump directly out if no more bytes */ + beqz a2, 7f + +4: + /* Copy the last word unaligned */ + add a3, a1, a2 + add a4, t6, a2 + REG_L a5, -SZREG(a3) + REG_S a5, -SZREG(a4) + ret + +5: + /* Copy bytes when the total copy is . */ + +#include + +extern __typeof (memcpy) __memcpy_generic; +hidden_proto(__memcpy_generic) + +#include