From patchwork Mon Apr 22 07:43:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_M=C3=BCllner?= X-Patchwork-Id: 88836 X-Patchwork-Delegate: carlos@redhat.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 823F43858C66 for ; Mon, 22 Apr 2024 07:48:16 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) by sourceware.org (Postfix) with ESMTPS id 79EA9384AB51 for ; Mon, 22 Apr 2024 07:44:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 79EA9384AB51 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=vrull.eu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=vrull.eu ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 79EA9384AB51 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::12a ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713771859; cv=none; b=PmdoKpSkRki8AgKGpCy+NUedUaYZnTlruioVSnU6LujP1OdFcSzYqWA+O5ucJMPeWjkTnfp6BOYMj3KuwxIPZBTfwTOwSR9NwJpym4l4id1gIZJWBJeTk6738LvlkNqF+WsmoQ04Ryf+STavrt2a7rHzrmJg0ZT+J74YvDxcN6s= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713771859; c=relaxed/simple; bh=tmUgkM2rzq/f5IaXrXLMLR/lS18skn27dbtqZtJVEUQ=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=WNXO7tY5UjEADbgLXS7UTkRlhzhRLskhJMSILrEpYVfj4Grwt++Nz5Cx3j9+fKqJ8DMDpoo1Yzyq5D0aD68A1dTrJugAIkLMgQBbiwePmDXZiAVh6yO1cxFItXHAWVEM6S9013bQlMYT+LWdovI9ILMs2DI592zHoldODts85Os= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-516d2600569so5084242e87.0 for ; Mon, 22 Apr 2024 00:44:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrull.eu; s=google; t=1713771855; x=1714376655; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SU8FBAiXzAb2KnaoarGGG4WXVqvRI5juh0tAbV5EEAQ=; b=MxwIsxYGlqOKqQTguBut+ZaJfsV9dln4xm03eXKcq1wvZvtIpelZx4rA13CJ557nS9 74QAebG/cdIwD0tHoQaMyhAe67Mmrud4s4Uo9hsOxR2pTJzCiBMFJKYN7LnBT78/2F7E DhkEFra41SAwNoCTUd+qyU0+Z0xgMsDi/t0jadbMpPUVZ7h1dZCh3KTCrH9z08yQvJ1F L9bHYQGEokKqqyQkd33ppRfSdjKf4Zq2RajI2uv/zKCR3hHLXBX6NzxMVfAEpGBPsOSz qZ/JN3lehwvCkyNH7736M+42bTZFrRSAlDgFcExNRXU0JKM1Fq/blQhM/c1boaTgcnT2 yz2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713771855; x=1714376655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SU8FBAiXzAb2KnaoarGGG4WXVqvRI5juh0tAbV5EEAQ=; b=K1aaz57i7QO1s1R5amZ8niYIJb56tBYIuhhq9/wUNU9jlG5ft5Wo1Un7DE+5vkYhMu xrX6Qq8j+ToYlIrMAzyrD3rtKFlxE8t9bNDlx9ayRnA/R7Yhmynaug0PSUT3lPbXEujp BsZhxOQVec17oXXcTk7Fyq5Vfx/WVjYXtp/RbkDt0o+eogmlo0TDCbJeH7n/z9zQgkmp MbgdWEzzFBQpnWScEzC77AFx8vq8m+vhJNfeQFpZBlupcyLFI7v4VTNXtDOS3xGmwd1w ONu340xJuRdq/Vn8Vsq5Sg/axtfT3+D3a/G3ZtJ26out//w2bRVaOBdpjximUrk/JiG3 uTrg== X-Gm-Message-State: AOJu0YxuMLDGpPtI+ord/DsqMQMacsnW6MVqACTZpD0dBwMHiw2hX8Xo jYwQhBNQuClamb9ygG7/LJEtUg5pi5jBv0FALqKXxgkK1Hfrha2MfF2Q3REAwUiepDj6ZQ62OFM tKeY= X-Google-Smtp-Source: AGHT+IEkvNyhGV4prdebKttXoGjIt5rk5cafthWUW6DLCv5IHiRJMrNrETX09lrAsf30+cBvivDaCA== X-Received: by 2002:ac2:41da:0:b0:51a:c8ba:d908 with SMTP id d26-20020ac241da000000b0051ac8bad908mr4908012lfi.62.1713771855337; Mon, 22 Apr 2024 00:44:15 -0700 (PDT) Received: from beast.fritz.box (static.239.130.217.95.clients.your-server.de. [95.217.130.239]) by smtp.gmail.com with ESMTPSA id v24-20020a197418000000b0051920234d6dsm1656756lfe.273.2024.04.22.00.44.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 00:44:14 -0700 (PDT) From: =?utf-8?q?Christoph_M=C3=BCllner?= To: libc-alpha@sourceware.org, Adhemerval Zanella , Palmer Dabbelt , Darius Rad , Andrew Waterman , Philipp Tomsich , Evan Green , DJ Delorie , Vineet Gupta , Kito Cheng , Jeff Law Cc: =?utf-8?q?Christoph_M=C3=BCllner?= Subject: [PATCH 3/7] RISC-V: Add Zbb optimized memrchr as ifunc Date: Mon, 22 Apr 2024 09:43:59 +0200 Message-ID: <20240422074403.2399529-4-christoph.muellner@vrull.eu> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240422074403.2399529-1-christoph.muellner@vrull.eu> References: <20240422074403.2399529-1-christoph.muellner@vrull.eu> MIME-Version: 1.0 X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_MANYTO, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org When building with Zbb enabled, memrchr benefits from using orc.b in find_zero_all(). This patch changes the build system such, that a non-Zbb version as well as a Zbb version of this routine is built. Further, a ifunc resolver is provided that selects the right routine based on the outcome of extension probing via hwprobe(). Signed-off-by: Christoph Müllner --- sysdeps/riscv/multiarch/memrchr-generic.c | 26 ++++++++ sysdeps/riscv/multiarch/memrchr-zbb.c | 30 +++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + .../linux/riscv/multiarch/ifunc-impl-list.c | 4 ++ .../unix/sysv/linux/riscv/multiarch/memrchr.c | 61 +++++++++++++++++++ 5 files changed, 124 insertions(+) create mode 100644 sysdeps/riscv/multiarch/memrchr-generic.c create mode 100644 sysdeps/riscv/multiarch/memrchr-zbb.c create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/memrchr.c diff --git a/sysdeps/riscv/multiarch/memrchr-generic.c b/sysdeps/riscv/multiarch/memrchr-generic.c new file mode 100644 index 0000000000..7c1286cb60 --- /dev/null +++ b/sysdeps/riscv/multiarch/memrchr-generic.c @@ -0,0 +1,26 @@ +/* Re-include the default memrchr implementation. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define MEMRCHR __memrchr_generic +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(x) +#endif +#include diff --git a/sysdeps/riscv/multiarch/memrchr-zbb.c b/sysdeps/riscv/multiarch/memrchr-zbb.c new file mode 100644 index 0000000000..c074c7607c --- /dev/null +++ b/sysdeps/riscv/multiarch/memrchr-zbb.c @@ -0,0 +1,30 @@ +/* Re-include the default memrchr implementation for Zbb. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define MEMRCHR __memrchr_zbb +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(x) +#endif +/* Convince preprocessor to have Zbb instructions. */ +#ifndef __riscv_zbb +# define __riscv_zbb +#endif +#include diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index 5586d11c89..d36d0bd6dd 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -6,6 +6,9 @@ sysdep_routines += \ memcpy \ memcpy-generic \ memcpy_noalignment \ + memrchr \ + memrchr-generic \ + memrchr-zbb \ # sysdep_routines CFLAGS-memcpy_noalignment.c += -mno-strict-align diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c index 7321144a32..f4ec58b9d8 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -60,5 +60,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, __memcpy_noalignment) IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_generic)) + IFUNC_IMPL (i, name, memrchr, + IFUNC_IMPL_ADD (array, i, memrchr, has_zbb, __memrchr_zbb) + IFUNC_IMPL_ADD (array, i, memrchr, 1, __memrchr_generic)) + return 0; } diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memrchr.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memrchr.c new file mode 100644 index 0000000000..e19ac471c1 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memrchr.c @@ -0,0 +1,61 @@ +/* Multiple versions of memrchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017-2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memrchr so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memrchr +# undef __memrchr +# define memrchr __redirect_memrchr +# define __memrchr __redirect___memrchr +# include +# include +# include +# include +# include + +extern __typeof (__redirect_memrchr) __libc_memrchr; + +extern __typeof (__redirect_memrchr) __memrchr_generic attribute_hidden; +extern __typeof (__redirect_memrchr) __memrchr_zbb attribute_hidden; + +static inline __typeof (__redirect_memrchr) * +select_memrchr_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long int v; + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_EXT_ZBB)) + return __memrchr_zbb; + + return __memrchr_generic; +} + +riscv_libc_ifunc (__libc_memrchr, select_memrchr_ifunc); + +# undef memrchr +# undef __memrchr +strong_alias (__libc_memrchr, __memrchr); +weak_alias (__memrchr, memrchr) +# ifdef SHARED +__hidden_ver1 (memrchr, __GI___memrchr, __redirect_memrchr) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memrchr); +# endif +#else +# include +#endif