From patchwork Wed Jun 29 22:13:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 55565 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B0F16383F85A for ; Wed, 29 Jun 2022 22:14:14 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B0F16383F85A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1656540854; bh=h+e4KeXK3FlVlQ3YzEHQpd9gd2FO6tVSSKtilqQy+vg=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=l6hXLyRmT4uyEwvverTGv6oiJ8LoTiNeesRLIgGCGgrZ66w04rRwjDm7SGvsOL2QV H5GD+93zrDJJoT5xdGBj7Vj8NTY7Jr4/IgkuPf9wX1CTP0/v662KQEBJ+4ZcsA0Mag nY/3Dydda6NNo/4irATf25+wkxgrQ7hmT/VPKGY0= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by sourceware.org (Postfix) with ESMTPS id 972FA3842AD9 for ; Wed, 29 Jun 2022 22:13:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 972FA3842AD9 Received: by mail-pg1-x52b.google.com with SMTP id q140so16614965pgq.6 for ; Wed, 29 Jun 2022 15:13:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h+e4KeXK3FlVlQ3YzEHQpd9gd2FO6tVSSKtilqQy+vg=; b=IZrLzE4WxQdP3PQtAK5Y0nML+pzlqfYWB2W5k8SiUnkbEE2dQyhz5xAFPlYb6QcAmB TjXtgZNWKVI0hKATgoYHCmxedbfuCdWDmBCJ4m++YmcS1h/Fma8+VO1BSHoeKKkLmJDE gtQwFycgRN4McllHl/bEu0MsJjT6qp1JpZc/R8n6kSukKGVutc+w5FjNf1K9Bs3iN2Vi LjoqGo7dhFnZj633C3VEkTepXKGBmt9+cX+6kH28mjlWWcHPKfz29m5/nfDr7Z/qOrFF gvr2sVGv5NlYtQXz63JVixbLcau9LYoIkticRlj1JLNngp58bvaMbthFZlSlEI62uBra EBTQ== X-Gm-Message-State: AJIora8wN4yOef/iIBgu3OZgZH7dYL65fRiwcXAT4UN4K+yogaPDmnuC NkR/W4kmsXQ4JDE53vLvp4r3DoSff2U= X-Google-Smtp-Source: AGRyM1vfr6gVi+/R4VsW0GDnaW47uv/D8CX2el7dUVxxiZPa5+wqfP2bR/ziZShGC9gPZFTwILGiRg== X-Received: by 2002:a63:9042:0:b0:40d:bb3:79e0 with SMTP id a63-20020a639042000000b0040d0bb379e0mr4772509pge.70.1656540832418; Wed, 29 Jun 2022 15:13:52 -0700 (PDT) Received: from noah-tgl.. ([192.55.60.43]) by smtp.gmail.com with ESMTPSA id b15-20020a170902650f00b0016a4ca6517bsm11829239plk.171.2022.06.29.15.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 15:13:52 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 1/2] x86: Move mem{p}{mov|cpy}_{chk_}erms to its own file Date: Wed, 29 Jun 2022 15:13:48 -0700 Message-Id: <20220629221349.1242862-1-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220628152757.17922-1-goldstein.w.n@gmail.com> References: <20220628152757.17922-1-goldstein.w.n@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" The primary memmove_{impl}_unaligned_erms implementations don't interact with this function. Putting them in same file both wastes space and unnecessarily bloats a hot code section. --- sysdeps/x86_64/multiarch/Makefile | 1 + sysdeps/x86_64/multiarch/memmove-erms.S | 53 +++++++++++++++++++ .../multiarch/memmove-vec-unaligned-erms.S | 50 ----------------- 3 files changed, 54 insertions(+), 50 deletions(-) create mode 100644 sysdeps/x86_64/multiarch/memmove-erms.S diff --git a/sysdeps/x86_64/multiarch/Makefile b/sysdeps/x86_64/multiarch/Makefile index 666ee4d5d6..62a4d96fb8 100644 --- a/sysdeps/x86_64/multiarch/Makefile +++ b/sysdeps/x86_64/multiarch/Makefile @@ -18,6 +18,7 @@ sysdep_routines += \ memmove-avx-unaligned-erms-rtm \ memmove-avx512-no-vzeroupper \ memmove-avx512-unaligned-erms \ + memmove-erms \ memmove-evex-unaligned-erms \ memmove-sse2-unaligned-erms \ memmove-ssse3 \ diff --git a/sysdeps/x86_64/multiarch/memmove-erms.S b/sysdeps/x86_64/multiarch/memmove-erms.S new file mode 100644 index 0000000000..d98d21644b --- /dev/null +++ b/sysdeps/x86_64/multiarch/memmove-erms.S @@ -0,0 +1,53 @@ +#include + +#if defined USE_MULTIARCH && IS_IN (libc) + .text +ENTRY (__mempcpy_chk_erms) + cmp %RDX_LP, %RCX_LP + jb HIDDEN_JUMPTARGET (__chk_fail) +END (__mempcpy_chk_erms) + +/* Only used to measure performance of REP MOVSB. */ +ENTRY (__mempcpy_erms) + mov %RDI_LP, %RAX_LP + /* Skip zero length. */ + test %RDX_LP, %RDX_LP + jz 2f + add %RDX_LP, %RAX_LP + jmp L(start_movsb) +END (__mempcpy_erms) + +ENTRY (__memmove_chk_erms) + cmp %RDX_LP, %RCX_LP + jb HIDDEN_JUMPTARGET (__chk_fail) +END (__memmove_chk_erms) + +ENTRY (__memmove_erms) + movq %rdi, %rax + /* Skip zero length. */ + test %RDX_LP, %RDX_LP + jz 2f +L(start_movsb): + mov %RDX_LP, %RCX_LP + cmp %RSI_LP, %RDI_LP + jb 1f + /* Source == destination is less common. */ + je 2f + lea (%rsi,%rcx), %RDX_LP + cmp %RDX_LP, %RDI_LP + jb L(movsb_backward) +1: + rep movsb +2: + ret +L(movsb_backward): + leaq -1(%rdi,%rcx), %rdi + leaq -1(%rsi,%rcx), %rsi + std + rep movsb + cld + ret +END (__memmove_erms) +strong_alias (__memmove_erms, __memcpy_erms) +strong_alias (__memmove_chk_erms, __memcpy_chk_erms) +#endif diff --git a/sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S b/sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S index d1518b8bab..04747133b7 100644 --- a/sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S +++ b/sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S @@ -239,56 +239,6 @@ L(start): #endif #if defined USE_MULTIARCH && IS_IN (libc) END (MEMMOVE_SYMBOL (__memmove, unaligned)) -# if VEC_SIZE == 16 -ENTRY (__mempcpy_chk_erms) - cmp %RDX_LP, %RCX_LP - jb HIDDEN_JUMPTARGET (__chk_fail) -END (__mempcpy_chk_erms) - -/* Only used to measure performance of REP MOVSB. */ -ENTRY (__mempcpy_erms) - mov %RDI_LP, %RAX_LP - /* Skip zero length. */ - test %RDX_LP, %RDX_LP - jz 2f - add %RDX_LP, %RAX_LP - jmp L(start_movsb) -END (__mempcpy_erms) - -ENTRY (__memmove_chk_erms) - cmp %RDX_LP, %RCX_LP - jb HIDDEN_JUMPTARGET (__chk_fail) -END (__memmove_chk_erms) - -ENTRY (__memmove_erms) - movq %rdi, %rax - /* Skip zero length. */ - test %RDX_LP, %RDX_LP - jz 2f -L(start_movsb): - mov %RDX_LP, %RCX_LP - cmp %RSI_LP, %RDI_LP - jb 1f - /* Source == destination is less common. */ - je 2f - lea (%rsi,%rcx), %RDX_LP - cmp %RDX_LP, %RDI_LP - jb L(movsb_backward) -1: - rep movsb -2: - ret -L(movsb_backward): - leaq -1(%rdi,%rcx), %rdi - leaq -1(%rsi,%rcx), %rsi - std - rep movsb - cld - ret -END (__memmove_erms) -strong_alias (__memmove_erms, __memcpy_erms) -strong_alias (__memmove_chk_erms, __memcpy_chk_erms) -# endif # ifdef SHARED ENTRY (MEMMOVE_CHK_SYMBOL (__mempcpy_chk, unaligned_erms))