From patchwork Tue Oct 17 13:05:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 78022 X-Patchwork-Delegate: siddhesh@gotplt.org Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A6FB03838A3F for ; Tue, 17 Oct 2023 13:07:23 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by sourceware.org (Postfix) with ESMTPS id 1E216385C6FC for ; Tue, 17 Oct 2023 13:06:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 1E216385C6FC Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 1E216385C6FC Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::432 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1697547996; cv=none; b=O2CvSExD3x4yhZJP99LqlPXx9U4Y6+a7wIViNLNUiFT9rrRsEI5akpmLz41hRoldEarAY0QQzyL+KXW4uaoJdwQqxFBqrVsZwpGgiCzAdtP74ayWGTSsSQ8SB00AbisJBznKxY2PpGysQGYLd005ghXolDWZKyShuQzD++Iadio= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1697547996; c=relaxed/simple; bh=jUPF/QRCDWWHjC57KOQfOqa7bRVA4E/IPgWk92EWTOs=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=IbgJCFUyeBl8cNWwG5JfbWhK8AOBwcFYv1DU0C0SOBv2TrlUHLKRDPElMK6KhRWQvooP6ivNWxzxHEs4EgAj4aMJMNREHHaohlbA28B4gnYjXgXPwPp5rhWZ5/J6OczEnUspwFXtktZBvWvQilxecCQELqechoIHHe3pIxR73go= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-692d2e8c003so5094856b3a.1 for ; Tue, 17 Oct 2023 06:06:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1697547992; x=1698152792; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DD/1ClBe5qIutSn+CAc0Uqp6ps+NPcJnmn8BVpcC8nc=; b=OIZvRjwUS/BpkQe0V5Foft+EtlB5ie9VmJKJnw2YvIA+Jdo6xKyvENrcl5ieOM7KgT oNl/3X+FoSVod6Ha31Vnw8r2Rg1u9o5EE2mB/6Ft1D4Xp02s/7+fIsoQWjDmU9Qj25hO LAwVEGUZjEhqa5fTwfOg2flqTR3hE+NHx+PI3uvq9QznYGjG6NXJ/BR56nh3PAM5tJ9U 7zGidDtC7eRDpQUPOmeFogeG5rV4HhIzJxGdWI1XrjWE/w11xoTR0rUyPRJMBgjg712+ ek9LrXRd33d7As0uJ5mD3K+aPNjmqy79Y2WWn/RhbJkUupja8SBKzYZ65v//uq8kC+Ku 7AOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697547992; x=1698152792; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DD/1ClBe5qIutSn+CAc0Uqp6ps+NPcJnmn8BVpcC8nc=; b=l8CK0lxwvTrhBBhxOgljgljsE+R6pZ4NgQprzyOMzZDvUiv8+c85YIMyFny3OeHHVe 2vEic97OOeLRdFIi2l415iyTiOHfF3G5r6EbOtx3gndogg5+aLL9ZxvOTS8sHlDKhUaT nTIWwAcGeF8WnTUXoObhMDOeef6imzjhWq49BKCVyDp8//5PGdHuG8pHM1hJR3KnSr54 cWIOIt6DqjPXOmj6yYzu2Bmkn7QCv1/ujggFQlFFbIpTDGtmM16aB2YEoyhDLJDvIYib cXSj8eqKdmfvLl2g18zSLSPH8eNPeBkIcJs6T/JdtUwfDc1Cd9bCwYp4gLn67ZmMU2O9 pRDA== X-Gm-Message-State: AOJu0Yw5/F++vFj090tBq0sS3X0JooNB6WS4W33earhMXt0vXOxv4QnM amASpCP2mihuSgz9WX5tv5Zx6VvyD6XMDU/UeTTpdw== X-Google-Smtp-Source: AGHT+IGWjLyoJizStllLWk0XPs5u9Q/SN4TBCd8hAw7xeVGsAYSxTgEKlXuBdJtMrRYtHe5bTxMxbA== X-Received: by 2002:a05:6a20:3d14:b0:15e:9923:3e35 with SMTP id y20-20020a056a203d1400b0015e99233e35mr2588739pzi.19.1697547992395; Tue, 17 Oct 2023 06:06:32 -0700 (PDT) Received: from mandiga.. ([2804:1b3:a7c3:7f2e:11d:92b4:4d78:4197]) by smtp.gmail.com with ESMTPSA id l28-20020a635b5c000000b0056b6d1ac949sm1309788pgm.13.2023.10.17.06.06.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 06:06:31 -0700 (PDT) From: Adhemerval Zanella To: libc-alpha@sourceware.org, Siddhesh Poyarekar Cc: Noah Goldstein Subject: [PATCH v2 09/19] x86: Use dl-symbol-redir-ifunc.h on cpu-tunables Date: Tue, 17 Oct 2023 10:05:16 -0300 Message-Id: <20231017130526.2216827-10-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017130526.2216827-1-adhemerval.zanella@linaro.org> References: <20231017130526.2216827-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org The dl-symbol-redir-ifunc.h redirects compiler-generated libcalls to arch-specific memory implementations to avoid ifun calls where it is not yet possible. The memcmp-isa-default-impl.h aims to fix the same issue by calling the specific memset implementation directly. Using the memcmp symbol directly allows the compile to inline the memset calls (especially because _dl_tunable_set_hwcaps uses constants values), generating better code. For i386, _dl_writev with PIE requires to use the old 'int $0x80' syscall mode because the calling the TLS register (gs) is not yet initialized. Checked on x86_64-linux-gnu. Reviewed-by: Noah Goldstein Reviewed-by: Siddhesh Poyarekar --- .../i686/multiarch/dl-symbol-redir-ifunc.h | 5 +++ .../sysv/linux/i386/dl-writev.h} | 18 ++++----- sysdeps/x86/cpu-tunables.c | 39 ++++++------------- .../x86_64/multiarch/dl-symbol-redir-ifunc.h | 15 +++++++ 4 files changed, 39 insertions(+), 38 deletions(-) rename sysdeps/{x86_64/memcmp-isa-default-impl.h => unix/sysv/linux/i386/dl-writev.h} (62%) diff --git a/sysdeps/i386/i686/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/i386/i686/multiarch/dl-symbol-redir-ifunc.h index dee69d19db..220c586bd2 100644 --- a/sysdeps/i386/i686/multiarch/dl-symbol-redir-ifunc.h +++ b/sysdeps/i386/i686/multiarch/dl-symbol-redir-ifunc.h @@ -19,6 +19,11 @@ #ifndef _DL_IFUNC_GENERIC_H #define _DL_IFUNC_GENERIC_H +#ifndef SHARED + asm ("memset = __memset_ia32"); +asm ("memcmp = __memcmp_ia32"); + +#endif /* SHARED */ #endif diff --git a/sysdeps/x86_64/memcmp-isa-default-impl.h b/sysdeps/unix/sysv/linux/i386/dl-writev.h similarity index 62% rename from sysdeps/x86_64/memcmp-isa-default-impl.h rename to sysdeps/unix/sysv/linux/i386/dl-writev.h index 0962e83c3d..624d0e46b0 100644 --- a/sysdeps/x86_64/memcmp-isa-default-impl.h +++ b/sysdeps/unix/sysv/linux/i386/dl-writev.h @@ -1,5 +1,5 @@ -/* Set default memcmp impl based on ISA level. - Copyright (C) 2022-2023 Free Software Foundation, Inc. +/* Message-writing for the dynamic linker. Linux/i386 version. + Copyright (C) 2013-2023 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -16,13 +16,9 @@ License along with the GNU C Library; if not, see . */ -#include -#if MINIMUM_X86_ISA_LEVEL == 1 || MINIMUM_X86_ISA_LEVEL == 2 -# define DEFAULT_MEMCMP __memcmp_sse2 -#elif MINIMUM_X86_ISA_LEVEL == 3 -# define DEFAULT_MEMCMP __memcmp_avx2_movbe -#elif MINIMUM_X86_ISA_LEVEL == 4 -# define DEFAULT_MEMCMP __memcmp_evex_movbe -#else -# error "Unknown default memcmp implementation" +#if BUILD_PIE_DEFAULT +/* Can't use "call *%gs:SYSINFO_OFFSET" during startup in static PIE. */ +# define I386_USE_SYSENTER 0 #endif + +#include diff --git a/sysdeps/x86/cpu-tunables.c b/sysdeps/x86/cpu-tunables.c index 0d4f328585..5697885226 100644 --- a/sysdeps/x86/cpu-tunables.c +++ b/sysdeps/x86/cpu-tunables.c @@ -24,24 +24,11 @@ #include #include #include - -/* We can't use IFUNC memcmp nor strlen in init_cpu_features from libc.a - since IFUNC must be set up by init_cpu_features. */ -#if defined USE_MULTIARCH && !defined SHARED -# ifdef __x86_64__ -/* DEFAULT_MEMCMP by sysdeps/x86_64/memcmp-isa-default-impl.h. */ -# include -# else -# define DEFAULT_MEMCMP __memcmp_ia32 -# endif -extern __typeof (memcmp) DEFAULT_MEMCMP; -#else -# define DEFAULT_MEMCMP memcmp -#endif +#include #define CHECK_GLIBC_IFUNC_CPU_OFF(f, cpu_features, name, len) \ _Static_assert (sizeof (#name) - 1 == len, #name " != " #len); \ - if (!DEFAULT_MEMCMP (f, #name, len)) \ + if (memcmp (f, #name, len) == 0) \ { \ CPU_FEATURE_UNSET (cpu_features, name) \ break; \ @@ -51,7 +38,7 @@ extern __typeof (memcmp) DEFAULT_MEMCMP; which isn't available. */ #define CHECK_GLIBC_IFUNC_PREFERRED_OFF(f, cpu_features, name, len) \ _Static_assert (sizeof (#name) - 1 == len, #name " != " #len); \ - if (!DEFAULT_MEMCMP (f, #name, len)) \ + if (memcmp (f, #name, len) == 0) \ { \ cpu_features->preferred[index_arch_##name] \ &= ~bit_arch_##name; \ @@ -62,7 +49,7 @@ extern __typeof (memcmp) DEFAULT_MEMCMP; #define CHECK_GLIBC_IFUNC_PREFERRED_BOTH(f, cpu_features, name, \ disable, len) \ _Static_assert (sizeof (#name) - 1 == len, #name " != " #len); \ - if (!DEFAULT_MEMCMP (f, #name, len)) \ + if (memcmp (f, #name, len) == 0) \ { \ if (disable) \ cpu_features->preferred[index_arch_##name] &= ~bit_arch_##name; \ @@ -76,7 +63,7 @@ extern __typeof (memcmp) DEFAULT_MEMCMP; #define CHECK_GLIBC_IFUNC_PREFERRED_NEED_BOTH(f, cpu_features, name, \ need, disable, len) \ _Static_assert (sizeof (#name) - 1 == len, #name " != " #len); \ - if (!DEFAULT_MEMCMP (f, #name, len)) \ + if (memcmp (f, #name, len) == 0) \ { \ if (disable) \ cpu_features->preferred[index_arch_##name] &= ~bit_arch_##name; \ @@ -177,7 +164,7 @@ TUNABLE_CALLBACK (set_hwcaps) (tunable_val_t *valp) CHECK_GLIBC_IFUNC_CPU_OFF (n, cpu_features, POPCNT, 6); CHECK_GLIBC_IFUNC_CPU_OFF (n, cpu_features, SSE4_1, 6); CHECK_GLIBC_IFUNC_CPU_OFF (n, cpu_features, SSE4_2, 6); - if (!DEFAULT_MEMCMP (n, "XSAVEC", 6)) + if (memcmp (n, "XSAVEC", 6) == 0) { /* Update xsave_state_size to XSAVE state size. */ cpu_features->xsave_state_size @@ -290,12 +277,11 @@ attribute_hidden void TUNABLE_CALLBACK (set_x86_ibt) (tunable_val_t *valp) { - if (DEFAULT_MEMCMP (valp->strval, "on", sizeof ("on")) == 0) + if (memcmp (valp->strval, "on", sizeof ("on")) == 0) GL(dl_x86_feature_control).ibt = cet_always_on; - else if (DEFAULT_MEMCMP (valp->strval, "off", sizeof ("off")) == 0) + else if (memcmp (valp->strval, "off", sizeof ("off")) == 0) GL(dl_x86_feature_control).ibt = cet_always_off; - else if (DEFAULT_MEMCMP (valp->strval, "permissive", - sizeof ("permissive")) == 0) + else if (memcmp (valp->strval, "permissive", sizeof ("permissive")) == 0) GL(dl_x86_feature_control).ibt = cet_permissive; } @@ -303,12 +289,11 @@ attribute_hidden void TUNABLE_CALLBACK (set_x86_shstk) (tunable_val_t *valp) { - if (DEFAULT_MEMCMP (valp->strval, "on", sizeof ("on")) == 0) + if (memcmp (valp->strval, "on", sizeof ("on")) == 0) GL(dl_x86_feature_control).shstk = cet_always_on; - else if (DEFAULT_MEMCMP (valp->strval, "off", sizeof ("off")) == 0) + else if (memcmp (valp->strval, "off", sizeof ("off")) == 0) GL(dl_x86_feature_control).shstk = cet_always_off; - else if (DEFAULT_MEMCMP (valp->strval, "permissive", - sizeof ("permissive")) == 0) + else if (memcmp (valp->strval, "permissive", sizeof ("permissive")) == 0) GL(dl_x86_feature_control).shstk = cet_permissive; } #endif diff --git a/sysdeps/x86_64/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/x86_64/multiarch/dl-symbol-redir-ifunc.h index 3fe73ca1c3..c7d8961bb6 100644 --- a/sysdeps/x86_64/multiarch/dl-symbol-redir-ifunc.h +++ b/sysdeps/x86_64/multiarch/dl-symbol-redir-ifunc.h @@ -19,6 +19,8 @@ #ifndef _DL_IFUNC_GENERIC_H #define _DL_IFUNC_GENERIC_H +#ifndef SHARED + #include #if MINIMUM_X86_ISA_LEVEL >= 4 @@ -31,4 +33,17 @@ asm ("memset = " HAVE_MEMSET_IFUNC_GENERIC); + +#if MINIMUM_X86_ISA_LEVEL >= 4 +# define HAVE_MEMCMP_IFUNC_GENERIC "__memcmp_evex_movbe" +#elif MINIMUM_X86_ISA_LEVEL == 3 +# define HAVE_MEMCMP_IFUNC_GENERIC "__memcmp_avx2_movbe" +#else +# define HAVE_MEMCMP_IFUNC_GENERIC "__memcmp_sse2" +#endif + +asm ("memcmp = " HAVE_MEMCMP_IFUNC_GENERIC); + +#endif /* SHARED */ + #endif