From patchwork Mon Jan 20 03:26:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: daichengrong X-Patchwork-Id: 105085 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 887523858406 for ; Mon, 20 Jan 2025 03:28:55 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 887523858406 X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from cstnet.cn (smtp84.cstnet.cn [159.226.251.84]) by sourceware.org (Postfix) with ESMTPS id 4382D385841D for ; Mon, 20 Jan 2025 03:26:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4382D385841D Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=iscas.ac.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=iscas.ac.cn ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4382D385841D Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=159.226.251.84 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1737343621; cv=none; b=Wk98RwpQsOVXzQSoy91R0rWB3kdg7pJT8oDmIj5C9TssvNgA2LpmpUJ1lHYsu28Rm+N4fLzmEs2BVBRZmteqAx0n+DQakyeze2Tp9L0p8bF+Rz0gb3+Qp54rxXi5izTq0n4G8/eOj9X54AfSFOHI18BYqhEgIQwABdL5/T+tWYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1737343621; c=relaxed/simple; bh=v0eeHBbXFnng9ssTU64WimEs4kN+DHCAzMX12gjlsis=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=OkkOwm2hBxl5JGTxVF5rFq9569dc/7kzKfsr285Iomrz/x6FEaQPt8Vf2Cbm/MQIg+UG0T+QSyi+84jkA2pg+6+TIQhgayGTompwnZJIyqnVcefY6wjz2p+PSHoeeXFp2D2bexf3LELVfM7y3wWsGwkh3D55RuTX4sLs+EKTy1o= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4382D385841D Received: from chengrong-ubuntu-02.home.arpa (unknown [124.16.138.129]) by APP-05 (Coremail) with SMTP id zQCowADXQMp+wo1nUoYsCA--.62185S2; Mon, 20 Jan 2025 11:26:55 +0800 (CST) From: daichengrong@iscas.ac.cn To: libc-alpha@sourceware.org Cc: aswaterman@gmail.com, palmer@rivosinc.com Subject: [PATCH v3] RISC-V: add multiarch RVV support for memcpy using FMV IFUNC Date: Mon, 20 Jan 2025 11:26:47 +0800 Message-Id: <20250120032647.3836119-1-daichengrong@iscas.ac.cn> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-CM-TRANSID: zQCowADXQMp+wo1nUoYsCA--.62185S2 X-Coremail-Antispam: 1UD129KBjvJXoWxuryDZFWUWr18Gry5AF4xJFb_yoWrJry3pa n5CF15Kws5Jr1xGrWS9w1jq3W5ZryrGF1Yk34Y93yUJ3yUWr43Jas2ywn8WF9rJrWSkFWr uFnYqFyDu3y5A3JanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUv0b7Iv0xC_Kw4lb4IE77IF4wAFF20E14v26r1j6r4UM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rw A2F7IY1VAKz4vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xII jxv20xvEc7CjxVAFwI0_Gr0_Cr1l84ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwV C2z280aVCY1x0267AKxVW8Jr0_Cr1UM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVAC Y4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r106r15McIj6I8E87Iv67AKxVW8JV WxJwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lw4CEc2x0rVAKj4xxMxkI ecxEwVAFwVW8GwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c 02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jrv_ JF1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7 CjxVAFwI0_Jr0_Gr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v2 6r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Jr0_GrUvcSsGvfC2KfnxnUUI43ZEXa7IU5 Rlk3UUUUU== X-Originating-IP: [124.16.138.129] X-CM-SenderInfo: pgdluxxhqj201qj6x2xfdvhtffof0/ X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org From: daichengrong This patch introduces vector support for memcpy with IFUNC. The implementation select the RVV optimized memcpy version via hwprobe on kernel with RVV enabled. Changes in v3: Remove unnecessary whitespace Changes in v2: delete size-0 branch Signed-off-by: daichengrong --- sysdeps/riscv/multiarch/memcpy_vector.S | 35 +++++++++++++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 2 ++ .../unix/sysv/linux/riscv/multiarch/memcpy.c | 5 +++ 3 files changed, 42 insertions(+) create mode 100644 sysdeps/riscv/multiarch/memcpy_vector.S diff --git a/sysdeps/riscv/multiarch/memcpy_vector.S b/sysdeps/riscv/multiarch/memcpy_vector.S new file mode 100644 index 0000000000..8fddab8432 --- /dev/null +++ b/sysdeps/riscv/multiarch/memcpy_vector.S @@ -0,0 +1,35 @@ +/* memcpy for RISC-V Vector. + Copyright (C) 2024-2025 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + + +#include +#include + +ENTRY (__memcpy_vector) + mv a6, a0 +L(loop): + vsetvli a3,a2,e8,m8,ta,mu + vle8.v v8,(a1) + vse8.v v8,(a6) + add a1,a1,a3 + sub a2,a2,a3 + add a6,a6,a3 + bnez a2,L(loop) +L(ret): + ret +END (__memcpy_vector) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index fcef5659d4..a8b6c22af1 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -3,7 +3,9 @@ sysdep_routines += \ memcpy \ memcpy-generic \ memcpy_noalignment \ + memcpy_vector \ # sysdep_routines CFLAGS-memcpy_noalignment.c += -mno-strict-align +ASFLAGS-memcpy_vector.S += -march=rv64gcv endif diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c index 8544f5402a..105c837101 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c @@ -32,11 +32,16 @@ extern __typeof (__redirect_memcpy) __libc_memcpy; extern __typeof (__redirect_memcpy) __memcpy_generic attribute_hidden; extern __typeof (__redirect_memcpy) __memcpy_noalignment attribute_hidden; +extern __typeof (__redirect_memcpy) __memcpy_vector attribute_hidden; static inline __typeof (__redirect_memcpy) * select_memcpy_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) { unsigned long long int v; + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_IMA_V) == RISCV_HWPROBE_IMA_V) + return __memcpy_vector; + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_CPUPERF_0, &v) == 0 && (v & RISCV_HWPROBE_MISALIGNED_MASK) == RISCV_HWPROBE_MISALIGNED_FAST) return __memcpy_noalignment;