From patchwork Thu Jan 20 11:27:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Earnshaw X-Patchwork-Id: 50270 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 777333857C7E for ; Thu, 20 Jan 2022 11:29:46 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 777333857C7E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1642678186; bh=h7tjzrYqsOyQVpnLsYBc9vU/CH/sCMlXF4vSEQgs3Cw=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ozjk+tk9ty27Q0pD18Ch9SCv2hrsT3IAgCVmgwvpnm1f5ZMYmPtXU+MSA07eiCXqz Ni6t8zjLklA9RlOWXHm09ln+y12SP0WLjbZo4MAnO4QEoom4XcpQhqe6NWOw5Y3KVG NXgAeyV2dNdmY62UCmGCb+X47GFw7tWBsvyXLqJc= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id DEC4F3858D37 for ; Thu, 20 Jan 2022 11:28:20 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org DEC4F3858D37 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8EA71ED1; Thu, 20 Jan 2022 03:28:20 -0800 (PST) Received: from e126323.arm.com (unknown [10.57.36.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0A1503F774; Thu, 20 Jan 2022 03:28:19 -0800 (PST) To: GCC patches Subject: [PATCH 1/7] arm: Disambiguate multiple crypto patterns with the same name. Date: Thu, 20 Jan 2022 11:27:18 +0000 Message-Id: <20220120112724.830872-2-rearnsha@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220120112724.830872-1-rearnsha@arm.com> References: <20220120112724.830872-1-rearnsha@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Earnshaw via Gcc-patches From: Richard Earnshaw Reply-To: Richard Earnshaw Cc: Richard Earnshaw Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" No functional change, but arm/crypto.md has multiple pattenrs all called crypto_, which makes references to them ambiguous, so add the iterator base to the pattern name so that it is distinct in the commit logs. gcc/ChangeLog: * config/arm/crypto.md (crypto_): Add iterator to pattern name to disambiguate. (crypto_): Likewise. (crypto_): Likewise. (crypto_): Likewise. (crypto_): Likewise. (crypto__lb): Likewise. --- gcc/config/arm/crypto.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/gcc/config/arm/crypto.md b/gcc/config/arm/crypto.md index 2425641e33f..6071ea17eac 100644 --- a/gcc/config/arm/crypto.md +++ b/gcc/config/arm/crypto.md @@ -19,7 +19,7 @@ ;; . -(define_insn "crypto_" +(define_insn "crypto_" [(set (match_operand: 0 "register_operand" "=w") (unspec: [(match_operand: 1 "register_operand" "w")] @@ -29,7 +29,7 @@ (define_insn "crypto_" [(set_attr "type" "")] ) -(define_insn "crypto_" +(define_insn "crypto_" [(set (match_operand:V16QI 0 "register_operand" "=w") (unspec:V16QI [(xor:V16QI @@ -83,7 +83,7 @@ (define_insn "*aarch32_crypto_aesd_fused" (set_attr "length" "8")] ) -(define_insn "crypto_" +(define_insn "crypto_" [(set (match_operand: 0 "register_operand" "=w") (unspec: [(match_operand: 1 "register_operand" "0") @@ -94,7 +94,7 @@ (define_insn "crypto_" [(set_attr "type" "")] ) -(define_insn "crypto_" +(define_insn "crypto_" [(set (match_operand: 0 "register_operand" "=w") (unspec: [(match_operand: 1 "register_operand" "0") (match_operand: 2 "register_operand" "w") @@ -145,7 +145,7 @@ (define_insn "crypto_vmullp64" of the V4SI, adjusted for endianness. Required due to neon_vget_lane and neon_set_lane that change the element ordering in memory for big-endian. */ -(define_expand "crypto_" +(define_expand "crypto_" [(set (match_operand:V4SI 0 "register_operand") (unspec: [(match_operand: 1 "register_operand") @@ -160,7 +160,7 @@ (define_expand "crypto_" DONE; }) -(define_insn "crypto__lb" +(define_insn "crypto__lb" [(set (match_operand:V4SI 0 "register_operand" "=w") (unspec: [(match_operand: 1 "register_operand" "0") From patchwork Thu Jan 20 11:27:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Earnshaw X-Patchwork-Id: 50271 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 520CF3857C7F for ; Thu, 20 Jan 2022 11:30:44 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 520CF3857C7F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1642678244; bh=oCeohBrWw86Nz306ASmYvP/FJGHZlAQHKRM/iGU+xt0=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=bZevVFrYSC4Si2L4MhCF/ai3I3iXo9BOHzgyhoXUJp74r7nn+EVpLv4JHapjR09jY 8vfwOSyBuh6riqnckjOCQ6TrzHFEjSLAV4noUI5scbJ1ni/HJ/+KE/05wkJ+fzCY1Y oMqEIv8gGxAwq7ShdARAwcqxW0rIuJXaQmPLydwk= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id AA1AE3858D35 for ; Thu, 20 Jan 2022 11:28:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org AA1AE3858D35 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5BA3C101E; Thu, 20 Jan 2022 03:28:21 -0800 (PST) Received: from e126323.arm.com (unknown [10.57.36.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CABBA3F774; Thu, 20 Jan 2022 03:28:20 -0800 (PST) To: GCC patches Subject: [PATCH 2/7] arm: Consistently use crypto_mode attribute in crypto patterns Date: Thu, 20 Jan 2022 11:27:19 +0000 Message-Id: <20220120112724.830872-3-rearnsha@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220120112724.830872-1-rearnsha@arm.com> References: <20220120112724.830872-1-rearnsha@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Earnshaw via Gcc-patches From: Richard Earnshaw Reply-To: Richard Earnshaw Cc: Richard Earnshaw Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" A couple of patterns in the crypto support code were hard-coding the mode rather than using the iterators. While not incorrect, it was slightly confusing, so adapt those patterns to the style of the rest of the file. Also fix some white space issues. gcc/ChangeLog: * config/arm/crypto.md (crypto_): Use rather than hard-coding the mode. (crypto_): Fix white space. (crypto_): Likewise. (*aarch32_crypto_aese_fused): Likewise. (*aarch32_crypto_aesd_fused): Likewise. (crypto_): Likewise. (crypto_): Likewise. (crypto_sha1h_lb): Likewise. (crypto_vmullp64): Likewise. (crypto_): Likewise. (crypto__lb): Likewise. --- gcc/config/arm/crypto.md | 94 ++++++++++++++++++++-------------------- 1 file changed, 47 insertions(+), 47 deletions(-) diff --git a/gcc/config/arm/crypto.md b/gcc/config/arm/crypto.md index 6071ea17eac..020dfba7dcf 100644 --- a/gcc/config/arm/crypto.md +++ b/gcc/config/arm/crypto.md @@ -22,7 +22,7 @@ (define_insn "crypto_" [(set (match_operand: 0 "register_operand" "=w") (unspec: - [(match_operand: 1 "register_operand" "w")] + [(match_operand: 1 "register_operand" "w")] CRYPTO_AESMC))] "TARGET_CRYPTO" ".\\t%q0, %q1" @@ -30,12 +30,12 @@ (define_insn "crypto_" ) (define_insn "crypto_" - [(set (match_operand:V16QI 0 "register_operand" "=w") - (unspec:V16QI - [(xor:V16QI - (match_operand:V16QI 1 "register_operand" "%0") - (match_operand:V16QI 2 "register_operand" "w"))] - CRYPTO_AES))] + [(set (match_operand: 0 "register_operand" "=w") + (unspec: + [(xor: + (match_operand: 1 "register_operand" "%0") + (match_operand: 2 "register_operand" "w"))] + CRYPTO_AES))] "TARGET_CRYPTO" ".\\t%q0, %q2" [(set_attr "type" "")] @@ -44,17 +44,16 @@ (define_insn "crypto_" ;; When AESE/AESMC fusion is enabled we really want to keep the two together ;; and enforce the register dependency without scheduling or register ;; allocation messing up the order or introducing moves inbetween. -;; Mash the two together during combine. +;; Mash the two together during combine. (define_insn "*aarch32_crypto_aese_fused" [(set (match_operand:V16QI 0 "register_operand" "=w") (unspec:V16QI - [(unspec:V16QI - [(xor:V16QI - (match_operand:V16QI 1 "register_operand" "%0") - (match_operand:V16QI 2 "register_operand" "w"))] - UNSPEC_AESE)] - UNSPEC_AESMC))] + [(unspec:V16QI [(xor:V16QI + (match_operand:V16QI 1 "register_operand" "%0") + (match_operand:V16QI 2 "register_operand" "w"))] + UNSPEC_AESE)] + UNSPEC_AESMC))] "TARGET_CRYPTO && arm_fusion_enabled_p (tune_params::FUSE_AES_AESMC)" "aese.8\\t%q0, %q2\;aesmc.8\\t%q0, %q0" @@ -65,17 +64,16 @@ (define_insn "*aarch32_crypto_aese_fused" ;; When AESD/AESIMC fusion is enabled we really want to keep the two together ;; and enforce the register dependency without scheduling or register ;; allocation messing up the order or introducing moves inbetween. -;; Mash the two together during combine. +;; Mash the two together during combine. (define_insn "*aarch32_crypto_aesd_fused" [(set (match_operand:V16QI 0 "register_operand" "=w") (unspec:V16QI - [(unspec:V16QI - [(xor:V16QI - (match_operand:V16QI 1 "register_operand" "%0") - (match_operand:V16QI 2 "register_operand" "w"))] - UNSPEC_AESD)] - UNSPEC_AESIMC))] + [(unspec:V16QI [(xor:V16QI + (match_operand:V16QI 1 "register_operand" "%0") + (match_operand:V16QI 2 "register_operand" "w"))] + UNSPEC_AESD)] + UNSPEC_AESIMC))] "TARGET_CRYPTO && arm_fusion_enabled_p (tune_params::FUSE_AES_AESMC)" "aesd.8\\t%q0, %q2\;aesimc.8\\t%q0, %q0" @@ -86,9 +84,9 @@ (define_insn "*aarch32_crypto_aesd_fused" (define_insn "crypto_" [(set (match_operand: 0 "register_operand" "=w") (unspec: - [(match_operand: 1 "register_operand" "0") - (match_operand: 2 "register_operand" "w")] - CRYPTO_BINARY))] + [(match_operand: 1 "register_operand" "0") + (match_operand: 2 "register_operand" "w")] + CRYPTO_BINARY))] "TARGET_CRYPTO" ".\\t%q0, %q2" [(set_attr "type" "")] @@ -96,18 +94,20 @@ (define_insn "crypto_" (define_insn "crypto_" [(set (match_operand: 0 "register_operand" "=w") - (unspec: [(match_operand: 1 "register_operand" "0") - (match_operand: 2 "register_operand" "w") - (match_operand: 3 "register_operand" "w")] - CRYPTO_TERNARY))] + (unspec: + [(match_operand: 1 "register_operand" "0") + (match_operand: 2 "register_operand" "w") + (match_operand: 3 "register_operand" "w")] + CRYPTO_TERNARY))] "TARGET_CRYPTO" ".\\t%q0, %q2, %q3" [(set_attr "type" "")] ) -/* The vec_select operation always selects index 0 from the lower V2SI subreg - of the V4SI, adjusted for endianness. Required due to neon_vget_lane and - neon_set_lane that change the element ordering in memory for big-endian. */ +;; The vec_select operation always selects index 0 from the lower V2SI +;; subreg of the V4SI, adjusted for endianness. Required due to +;; neon_vget_lane and neon_set_lane that change the element ordering +;; in memory for big-endian. (define_expand "crypto_sha1h" [(set (match_operand:V4SI 0 "register_operand") @@ -122,10 +122,10 @@ (define_expand "crypto_sha1h" (define_insn "crypto_sha1h_lb" [(set (match_operand:V4SI 0 "register_operand" "=w") (unspec:V4SI - [(vec_select:SI + [(vec_select:SI (match_operand:V4SI 1 "register_operand" "w") (parallel [(match_operand:SI 2 "immediate_operand" "i")]))] - UNSPEC_SHA1H))] + UNSPEC_SHA1H))] "TARGET_CRYPTO && INTVAL (operands[2]) == NEON_ENDIAN_LANE_N (V2SImode, 0)" "sha1h.32\\t%q0, %q1" [(set_attr "type" "crypto_sha1_fast")] @@ -133,9 +133,9 @@ (define_insn "crypto_sha1h_lb" (define_insn "crypto_vmullp64" [(set (match_operand:TI 0 "register_operand" "=w") - (unspec:TI [(match_operand:DI 1 "register_operand" "w") - (match_operand:DI 2 "register_operand" "w")] - UNSPEC_VMULLP64))] + (unspec:TI [(match_operand:DI 1 "register_operand" "w") + (match_operand:DI 2 "register_operand" "w")] + UNSPEC_VMULLP64))] "TARGET_CRYPTO" "vmull.p64\\t%q0, %P1, %P2" [(set_attr "type" "crypto_pmull")] @@ -148,10 +148,10 @@ (define_insn "crypto_vmullp64" (define_expand "crypto_" [(set (match_operand:V4SI 0 "register_operand") (unspec: - [(match_operand: 1 "register_operand") - (match_operand: 2 "register_operand") - (match_operand: 3 "register_operand")] - CRYPTO_SELECTING))] + [(match_operand: 1 "register_operand") + (match_operand: 2 "register_operand") + (match_operand: 3 "register_operand")] + CRYPTO_SELECTING))] "TARGET_CRYPTO" { rtx op4 = GEN_INT (NEON_ENDIAN_LANE_N (V2SImode, 0)); @@ -162,13 +162,13 @@ (define_expand "crypto_" (define_insn "crypto__lb" [(set (match_operand:V4SI 0 "register_operand" "=w") - (unspec: - [(match_operand: 1 "register_operand" "0") - (vec_select:SI - (match_operand: 2 "register_operand" "w") - (parallel [(match_operand:SI 4 "immediate_operand" "i")])) - (match_operand: 3 "register_operand" "w")] - CRYPTO_SELECTING))] + (unspec: + [(match_operand: 1 "register_operand" "0") + (vec_select:SI + (match_operand: 2 "register_operand" "w") + (parallel [(match_operand:SI 4 "immediate_operand" "i")])) + (match_operand: 3 "register_operand" "w")] + CRYPTO_SELECTING))] "TARGET_CRYPTO && INTVAL (operands[4]) == NEON_ENDIAN_LANE_N (V2SImode, 0)" ".\\t%q0, %q2, %q3" [(set_attr "type" "")] From patchwork Thu Jan 20 11:27:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Earnshaw X-Patchwork-Id: 50272 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 87C78385841D for ; Thu, 20 Jan 2022 11:31:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 87C78385841D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1642678301; bh=gLoLjLIClMNevQR41aYdmBb8YpdeT3XEP6BM/nk9yJs=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=KcD2wn4lNVrcKyR2j3RgSM7uOdgNSVKdq4idkw+X6YHE3mLTmAOWLr0VfjY+dHY9A RT44r5ITC+P4JMFGBafBDcgjXIfla8J5LZnwtc5wc3+joSKRUfTzW4Jma5EectgZ9u 1U4c7ulYypXfqAd7lNZpp7elVor01NuqhjEpvHTA= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 6AE803858D37 for ; Thu, 20 Jan 2022 11:28:22 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6AE803858D37 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2623A6D; Thu, 20 Jan 2022 03:28:22 -0800 (PST) Received: from e126323.arm.com (unknown [10.57.36.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 93FF13F774; Thu, 20 Jan 2022 03:28:21 -0800 (PST) To: GCC patches Subject: [PATCH 3/7] arm: Add option for mitigating against Cortex-A CPU erratum for AES Date: Thu, 20 Jan 2022 11:27:20 +0000 Message-Id: <20220120112724.830872-4-rearnsha@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220120112724.830872-1-rearnsha@arm.com> References: <20220120112724.830872-1-rearnsha@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Earnshaw via Gcc-patches From: Richard Earnshaw Reply-To: Richard Earnshaw Cc: Richard Earnshaw Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Add a new option -mfix-cortex-a-aes for enabling the Cortex-A AES erratum work-around and enable it automatically for the affected products (Cortex-A57 and Cortex-A72). gcc/ChangeLog: * config/arm/arm-cpus.in (quirk_aes_1742098): New quirk feature (ALL_QUIRKS): Add it. (cortex-a57, cortex-a72): Enable it. (cortex-a57.cortex-a53, cortex-a72.cortex-a53): Likewise. * config/arm/arm.opt (mfix-cortex-a57-aes-1742098): New command-line option. (mfix-cortex-a72-aes-1655431): New option alias. * config/arm/arm.cc (arm_option_override): Handle default settings for AES erratum switch. * doc/invoke.texi (Arm Options): Document new options. --- gcc/config/arm/arm-cpus.in | 9 ++++++++- gcc/config/arm/arm.cc | 9 +++++++++ gcc/config/arm/arm.opt | 10 ++++++++++ gcc/doc/invoke.texi | 11 +++++++++++ 4 files changed, 38 insertions(+), 1 deletion(-) diff --git a/gcc/config/arm/arm-cpus.in b/gcc/config/arm/arm-cpus.in index 499e82d790d..0d3082b569f 100644 --- a/gcc/config/arm/arm-cpus.in +++ b/gcc/config/arm/arm-cpus.in @@ -192,6 +192,9 @@ define feature quirk_cm3_ldrd # v8-m/v8.1-m VLLDM errata. define feature quirk_vlldm +# AES errata on some Cortex-A parts +define feature quirk_aes_1742098 + # Don't use .cpu assembly directive define feature quirk_no_asmcpu @@ -329,7 +332,7 @@ define implied vfp_base MVE MVE_FP ALL_FP # architectures. # xscale isn't really a 'quirk', but it isn't an architecture either and we # need to ignore it for matching purposes. -define fgroup ALL_QUIRKS quirk_no_volatile_ce quirk_armv6kz quirk_cm3_ldrd quirk_vlldm xscale quirk_no_asmcpu +define fgroup ALL_QUIRKS quirk_no_volatile_ce quirk_armv6kz quirk_cm3_ldrd quirk_vlldm xscale quirk_no_asmcpu quirk_aes_1742098 define fgroup IGNORE_FOR_MULTILIB cdecp0 cdecp1 cdecp2 cdecp3 cdecp4 cdecp5 cdecp6 cdecp7 @@ -1342,6 +1345,7 @@ begin cpu cortex-a57 cname cortexa57 tune flags LDSCHED architecture armv8-a+crc+simd + isa quirk_aes_1742098 option crypto add FP_ARMv8 CRYPTO costs cortex_a57 vendor 41 @@ -1353,6 +1357,7 @@ begin cpu cortex-a72 tune for cortex-a57 tune flags LDSCHED architecture armv8-a+crc+simd + isa quirk_aes_1742098 option crypto add FP_ARMv8 CRYPTO costs cortex_a57 vendor 41 @@ -1391,6 +1396,7 @@ begin cpu cortex-a57.cortex-a53 tune for cortex-a53 tune flags LDSCHED architecture armv8-a+crc+simd + isa quirk_aes_1742098 option crypto add FP_ARMv8 CRYPTO costs cortex_a57 end cpu cortex-a57.cortex-a53 @@ -1400,6 +1406,7 @@ begin cpu cortex-a72.cortex-a53 tune for cortex-a53 tune flags LDSCHED architecture armv8-a+crc+simd + isa quirk_aes_1742098 option crypto add FP_ARMv8 CRYPTO costs cortex_a57 end cpu cortex-a72.cortex-a53 diff --git a/gcc/config/arm/arm.cc b/gcc/config/arm/arm.cc index 7825e364c01..04354b36606 100644 --- a/gcc/config/arm/arm.cc +++ b/gcc/config/arm/arm.cc @@ -3638,6 +3638,15 @@ arm_option_override (void) fix_vlldm = 0; } + /* Enable fix_aes by default if required. */ + if (fix_aes_erratum_1742098 == 2) + { + if (bitmap_bit_p (arm_active_target.isa, isa_bit_quirk_aes_1742098)) + fix_aes_erratum_1742098 = 1; + else + fix_aes_erratum_1742098 = 0; + } + /* Hot/Cold partitioning is not currently supported, since we can't handle literal pool placement in that case. */ if (flag_reorder_blocks_and_partition) diff --git a/gcc/config/arm/arm.opt b/gcc/config/arm/arm.opt index 587fc932f96..2a4f165033a 100644 --- a/gcc/config/arm/arm.opt +++ b/gcc/config/arm/arm.opt @@ -272,6 +272,16 @@ mfix-cmse-cve-2021-35465 Target Var(fix_vlldm) Init(2) Mitigate issues with VLLDM on some M-profile devices (CVE-2021-35465). +mfix-cortex-a57-aes-1742098 +Target Var(fix_aes_erratum_1742098) Init(2) Save +Mitigate issues with AES instructions on Cortex-A57 and Cortex-A72. +Arm erratum #1742098 + +mfix-cortex-a72-aes-1655431 +Target Alias(mfix-cortex-a57-aes-1742098) +Mitigate issues with AES instructions on Cortex-A57 and Cortex-A72. +Arm erratum #1655431 + munaligned-access Target Var(unaligned_access) Init(2) Save Enable unaligned word and halfword accesses to packed data. diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index 58751c48b8e..67693d6c5cf 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -812,6 +812,8 @@ Objective-C and Objective-C++ Dialects}. -mtp=@var{name} -mtls-dialect=@var{dialect} @gol -mword-relocations @gol -mfix-cortex-m3-ldrd @gol +-mfix-cortex-a57-aes-1742098 @gol +-mfix-cortex-a72-aes-1655431 @gol -munaligned-access @gol -mneon-for-64bits @gol -mslow-flash-data @gol @@ -21281,6 +21283,15 @@ with overlapping destination and base registers are used. This option avoids generating these instructions. This option is enabled by default when @option{-mcpu=cortex-m3} is specified. +@item -mfix-cortex-a57-aes-1742098 +@itemx -mno-fix-cortex-a57-aes-1742098 +@itemx -mfix-cortex-a72-aes-1655431 +@itemx -mno-fix-cortex-a72-aes-1655431 +Enable (disable) mitigation for an erratum on Cortex-A57 and +Cortex-A72 that affects the AES cryptographic instructions. This +option is enabled by default when either @option{-mcpu=cortex-a57} or +@option{-mcpu=cortex-a72} is specified. + @item -munaligned-access @itemx -mno-unaligned-access @opindex munaligned-access From patchwork Thu Jan 20 11:27:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Earnshaw X-Patchwork-Id: 50273 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id AAECF3858D37 for ; Thu, 20 Jan 2022 11:32:45 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org AAECF3858D37 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1642678365; bh=Xr9ki3eVCHbDmnsBmHuiJgLJSF8ZntX9qsDst8EiPjs=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=eyqmrDsCtJVAPZ4c2r/8fOeSDQEGuHCqJwxVCFpQqmU9vtypvsMnOUmTUQ+zU3HSX 4fPp/iQQ/DEXG7khrOa0IxngKw+rxqyOP2OSap6mrwuG0yZJuWHJwtgUzVQx+cj7/J zwI4nctRaErGsUKWQRVQcx3jhRAuhE/RfxbIKfh8= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 2D44B3857C7B for ; Thu, 20 Jan 2022 11:28:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2D44B3857C7B Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E2153ED1; Thu, 20 Jan 2022 03:28:22 -0800 (PST) Received: from e126323.arm.com (unknown [10.57.36.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 642623F774; Thu, 20 Jan 2022 03:28:22 -0800 (PST) To: GCC patches Subject: [PATCH 4/7] arm: add basic mitigation for Cortex-A AES errata Date: Thu, 20 Jan 2022 11:27:21 +0000 Message-Id: <20220120112724.830872-5-rearnsha@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220120112724.830872-1-rearnsha@arm.com> References: <20220120112724.830872-1-rearnsha@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Earnshaw via Gcc-patches From: Richard Earnshaw Reply-To: Richard Earnshaw Cc: Richard Earnshaw Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" This patch adds the basic patterns for mitigation of the erratum, but no attempt is made at this point to optimize the results for the cases where the erratum mitigation is not needed. The mitigation is done by guaranteeing that the input operands are fed from a full-width operation by using an identity operation on the input values. gcc/ChangeLog: * config/arm/crypto.md (crypto_): Convert to define_expand. Add mitigation for the Cortex-A AES erratum when enabled. (*crypto__insn): New pattern, based on original crypto_ insn. (aes_op_protect): New pattern. * config/arm/unspecs.md (unspec): Add UNSPEC_AES_PROTECT. --- gcc/config/arm/crypto.md | 36 +++++++++++++++++++++++++++++++++++- gcc/config/arm/unspecs.md | 1 + 2 files changed, 36 insertions(+), 1 deletion(-) diff --git a/gcc/config/arm/crypto.md b/gcc/config/arm/crypto.md index 020dfba7dcf..fbee1829ce8 100644 --- a/gcc/config/arm/crypto.md +++ b/gcc/config/arm/crypto.md @@ -29,7 +29,28 @@ (define_insn "crypto_" [(set_attr "type" "")] ) -(define_insn "crypto_" +(define_expand "crypto_" + [(set (match_operand: 0 "register_operand" "=w") + (unspec: + [(xor: + (match_operand: 1 "register_operand" "%0") + (match_operand: 2 "register_operand" "w"))] + CRYPTO_AES))] + "TARGET_CRYPTO" +{ + if (fix_aes_erratum_1742098) + { + rtx op1_protect = gen_reg_rtx (V16QImode); + emit_insn (gen_aes_op_protect (op1_protect, operands[1])); + operands[1] = op1_protect; + rtx op2_protect = gen_reg_rtx (V16QImode); + emit_insn (gen_aes_op_protect (op2_protect, operands[2])); + operands[2] = op2_protect; + } + /* Fall through to default expansion. */ +}) + +(define_insn "*crypto__insn" [(set (match_operand: 0 "register_operand" "=w") (unspec: [(xor: @@ -41,6 +62,19 @@ (define_insn "crypto_" [(set_attr "type" "")] ) +; Mitigate against AES erratum on Cortex-A57 and Cortex-A72 by performing +; a 128-bit operation on an operand producer. This can be eliminated only +; if we know that the operand was produced by a full-width operation. +; V16QImode matches for the AES instructions. +(define_insn "aes_op_protect" + [(set (match_operand:V16QI 0 "register_operand" "=w") + (unspec:V16QI [(match_operand:V16QI 1 "register_operand" "0")] + UNSPEC_AES_PROTECT))] + "TARGET_CRYPTO && fix_aes_erratum_1742098" + "vmov\\t%q0, %q1" + [(set_attr "type" "neon_move_q")] +) + ;; When AESE/AESMC fusion is enabled we really want to keep the two together ;; and enforce the register dependency without scheduling or register ;; allocation messing up the order or introducing moves inbetween. diff --git a/gcc/config/arm/unspecs.md b/gcc/config/arm/unspecs.md index 2782af08834..7748e784379 100644 --- a/gcc/config/arm/unspecs.md +++ b/gcc/config/arm/unspecs.md @@ -270,6 +270,7 @@ (define_c_enum "unspec" [ UNSPEC_AESE UNSPEC_AESIMC UNSPEC_AESMC + UNSPEC_AES_PROTECT UNSPEC_SHA1C UNSPEC_SHA1M UNSPEC_SHA1P From patchwork Thu Jan 20 11:27:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Earnshaw X-Patchwork-Id: 50274 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 82E3A3857C77 for ; Thu, 20 Jan 2022 11:33:42 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 82E3A3857C77 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1642678422; bh=U+6Sw+KSGRZe8GYK+sUY0AiuBfF/azW8eegKxGSw5zU=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=XQVGQnrycu2QK1DcmlpqfBRbk0xtWJgs+WAt8cjnH6Wq6YkBreaCO5JFWTjAwghJe 3WMvqWoGpjxlCwnePXw9DUMs/C4wbhY4Py3CJlr5yiSD5F40QJ4qWm8eCQHj+Zrnx1 aiXqF7ei1XSUS554G4t3F7KDl8ttuBNDFdl/PgDc= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id E44AF385841D for ; Thu, 20 Jan 2022 11:28:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org E44AF385841D Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9F597101E; Thu, 20 Jan 2022 03:28:23 -0800 (PST) Received: from e126323.arm.com (unknown [10.57.36.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 25BB93F774; Thu, 20 Jan 2022 03:28:23 -0800 (PST) To: GCC patches Subject: [PATCH 5/7] arm: suppress aes erratum when forwarding from aes Date: Thu, 20 Jan 2022 11:27:22 +0000 Message-Id: <20220120112724.830872-6-rearnsha@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220120112724.830872-1-rearnsha@arm.com> References: <20220120112724.830872-1-rearnsha@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Earnshaw via Gcc-patches From: Richard Earnshaw Reply-To: Richard Earnshaw Cc: Richard Earnshaw Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" AES operations are commonly chained and since the result of one AES operation is never a 32-bit value, they do not need an additional mitigation instruction for the forwarded result. We handle this common case by adding additional patterns that allow for this. gcc/ChangeLog: * config/arm/crypto.md (crypto__protected): New pattern. (aarch32_crypto_aese_fused_protected): Likewise. (aarch32_crypto_aesd_fused_protected): Likewise. --- gcc/config/arm/crypto.md | 50 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/gcc/config/arm/crypto.md b/gcc/config/arm/crypto.md index fbee1829ce8..df857352382 100644 --- a/gcc/config/arm/crypto.md +++ b/gcc/config/arm/crypto.md @@ -75,6 +75,20 @@ (define_insn "aes_op_protect" [(set_attr "type" "neon_move_q")] ) +;; An AESMC operation can feed directly into a subsequent AES +;; operation without needing mitigation. +(define_insn "*crypto__protected" + [(set (match_operand: 0 "register_operand" "=w") + (unspec: + [(unspec: + [(match_operand: 1 "register_operand" "w")] + CRYPTO_AESMC)] + UNSPEC_AES_PROTECT))] + "TARGET_CRYPTO && fix_aes_erratum_1742098" + ".\\t%q0, %q1" + [(set_attr "type" "")] +) + ;; When AESE/AESMC fusion is enabled we really want to keep the two together ;; and enforce the register dependency without scheduling or register ;; allocation messing up the order or introducing moves inbetween. @@ -95,6 +109,25 @@ (define_insn "*aarch32_crypto_aese_fused" (set_attr "length" "8")] ) +;; And similarly when mitigation is enabled, but not needed in this +;; case. +(define_insn "*aarch32_crypto_aese_fused_protected" + [(set (match_operand:V16QI 0 "register_operand" "=w") + (unspec:V16QI + [(unspec:V16QI + [(unspec:V16QI [(xor:V16QI + (match_operand:V16QI 1 "register_operand" "%0") + (match_operand:V16QI 2 "register_operand" "w"))] + UNSPEC_AESE)] + UNSPEC_AESMC)] + UNSPEC_AES_PROTECT))] + "TARGET_CRYPTO && fix_aes_erratum_1742098 + && arm_fusion_enabled_p (tune_params::FUSE_AES_AESMC)" + "aese.8\\t%q0, %q2\;aesmc.8\\t%q0, %q0" + [(set_attr "type" "crypto_aese") + (set_attr "length" "8")] +) + ;; When AESD/AESIMC fusion is enabled we really want to keep the two together ;; and enforce the register dependency without scheduling or register ;; allocation messing up the order or introducing moves inbetween. @@ -115,6 +148,23 @@ (define_insn "*aarch32_crypto_aesd_fused" (set_attr "length" "8")] ) +(define_insn "*aarch32_crypto_aesd_fused_protected" + [(set (match_operand:V16QI 0 "register_operand" "=w") + (unspec:V16QI + [(unspec:V16QI + [(unspec:V16QI [(xor:V16QI + (match_operand:V16QI 1 "register_operand" "%0") + (match_operand:V16QI 2 "register_operand" "w"))] + UNSPEC_AESD)] + UNSPEC_AESIMC)] + UNSPEC_AES_PROTECT))] + "TARGET_CRYPTO && fix_aes_erratum_1742098 + && arm_fusion_enabled_p (tune_params::FUSE_AES_AESMC)" + "aesd.8\\t%q0, %q2\;aesimc.8\\t%q0, %q0" + [(set_attr "type" "crypto_aese") + (set_attr "length" "8")] +) + (define_insn "crypto_" [(set (match_operand: 0 "register_operand" "=w") (unspec: From patchwork Thu Jan 20 11:27:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Earnshaw X-Patchwork-Id: 50275 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A3DD2385800A for ; Thu, 20 Jan 2022 11:34:39 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A3DD2385800A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1642678479; bh=ZkcnF7fPwoUjil1ZH3V2isPzbdoCEUlIuIIkHZUpxPU=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=jPLqBUlbAOCgrVckFwhnQ8cwxsZoU9M0f+3ACQ+4nfBYMijl/0RgGQENQQL3T7C7h jBv37/PTqPhouJup+RIuGf/pJ1FUs5pzrmDofRE63oJ/S6EvGlag0zjyW8arFFR7sk 6Qrnow0W7jeTrxn2gs/mQqWdQZGkRcE2FsdUi8fw= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 8DE413857C76 for ; Thu, 20 Jan 2022 11:28:24 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8DE413857C76 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5EBD86D; Thu, 20 Jan 2022 03:28:24 -0800 (PST) Received: from e126323.arm.com (unknown [10.57.36.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D7DD13F774; Thu, 20 Jan 2022 03:28:23 -0800 (PST) To: GCC patches Subject: [PATCH 6/7] arm: elide some cases where the AES erratum workaround is not required. Date: Thu, 20 Jan 2022 11:27:23 +0000 Message-Id: <20220120112724.830872-7-rearnsha@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220120112724.830872-1-rearnsha@arm.com> References: <20220120112724.830872-1-rearnsha@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Earnshaw via Gcc-patches From: Richard Earnshaw Reply-To: Richard Earnshaw Cc: Richard Earnshaw Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Some common cases where the AES erratum workaround are not required are when there are 64- or 128-bit loads from memory, moving a 128-bit value from core registers, and where a 128-bit constant is being loaded from a literal pool. The loads may also be misaligned or generated via a neon intrinsic function. gcc/ChangeLog: * config/arm/crypto.md (aes_op_protect): Allow moves from core registers and from memory. (aes_op_protect_misalign_load): New pattern. (aes_op_protect_neon_vld1v16qi): New pattern. --- gcc/config/arm/crypto.md | 55 ++++++++++++++++++++++++++++++++++------ 1 file changed, 47 insertions(+), 8 deletions(-) diff --git a/gcc/config/arm/crypto.md b/gcc/config/arm/crypto.md index df857352382..4c785073028 100644 --- a/gcc/config/arm/crypto.md +++ b/gcc/config/arm/crypto.md @@ -62,17 +62,56 @@ (define_insn "*crypto__insn" [(set_attr "type" "")] ) -; Mitigate against AES erratum on Cortex-A57 and Cortex-A72 by performing -; a 128-bit operation on an operand producer. This can be eliminated only -; if we know that the operand was produced by a full-width operation. -; V16QImode matches for the AES instructions. +;; Mitigate against AES erratum on Cortex-A57 and Cortex-A72 by +;; performing a 128-bit operation on an operand producer. This can be +;; eliminated only if we know that the operand was produced by a +;; full-width operation. V16QImode matches for the AES +;; instructions. Handle some very common cases where the source is +;; known to be safe (transfers from core registers and memory). (define_insn "aes_op_protect" - [(set (match_operand:V16QI 0 "register_operand" "=w") - (unspec:V16QI [(match_operand:V16QI 1 "register_operand" "0")] + [(set (match_operand:V16QI 0 "register_operand" "=w,w,w") + (unspec:V16QI [(match_operand:V16QI 1 "general_operand" "w,r,Uni")] + UNSPEC_AES_PROTECT))] + "TARGET_CRYPTO && fix_aes_erratum_1742098" + { + switch (which_alternative) + { + case 0: return "vmov\t%q0, %q1"; + case 1: return "vmov\t%e0, %Q1, %R1 @ V16QI\;vmov\t%f0, %J1, %K1"; + case 2: return output_move_neon (operands); + default: gcc_unreachable (); + } + } + [(set_attr "type" "neon_move_q,neon_from_gp_q,neon_load1_4reg") + (set_attr "length" "4,8,8") + (set_attr "arm_pool_range" "*,*,1020") + (set_attr "thumb2_pool_range" "*,*,1018") + (set_attr "neg_pool_range" "*,*,996")] +) + +;; Another safe case is when a movmisalign load is used as the source. +(define_insn "*aes_op_protect_misalign_load" + [(set (match_operand:V16QI 0 "s_register_operand" "=w") + (unspec:V16QI + [(unspec:V16QI + [(match_operand:V16QI 1 "neon_permissive_struct_operand" "Um")] + UNSPEC_MISALIGNED_ACCESS)] UNSPEC_AES_PROTECT))] "TARGET_CRYPTO && fix_aes_erratum_1742098" - "vmov\\t%q0, %q1" - [(set_attr "type" "neon_move_q")] + "vld1.8\t%{q0}, %A1" + [(set_attr "type" "neon_load1_1reg_q")] +) + +;; Similarly for the vld1 intrinsic +(define_insn "aes_op_protect_neon_vld1v16qi" + [(set (match_operand:V16QI 0 "s_register_operand" "=w") + (unspec:V16QI + [(unspec:V16QI [(match_operand:V16QI 1 "neon_struct_operand" "Um")] + UNSPEC_VLD1)] + UNSPEC_AES_PROTECT))] + "TARGET_NEON" + "vld1.8\t%h0, %A1" + [(set_attr "type" "neon_load1_1reg_q")] ) ;; An AESMC operation can feed directly into a subsequent AES From patchwork Thu Jan 20 11:27:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Earnshaw X-Patchwork-Id: 50276 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D02873857809 for ; Thu, 20 Jan 2022 11:35:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D02873857809 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1642678536; bh=KZiZu4C9NsX3Fra576dyc6S+9/HuYTsQ3qxURbVEs3A=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=fChIr9o//hC4Z2IhbhfbAjjuM6gWu/b5Ky2XfTnalasOduBt9Zjlz3hdOOng+PVQa qGm0U54zOQa0QE4Kq1Gl1VLy1pn/4/y1/rGyqiRaNPsn/I5N2lNd38r3e1vFiFUT5/ yZejfWf/FdpCOLWXvZUDNIQz8yZpuDtU8jnWkv2Q= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 4B4033857C73 for ; Thu, 20 Jan 2022 11:28:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4B4033857C73 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1CB51ED1; Thu, 20 Jan 2022 03:28:25 -0800 (PST) Received: from e126323.arm.com (unknown [10.57.36.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 96CF53F774; Thu, 20 Jan 2022 03:28:24 -0800 (PST) To: GCC patches Subject: [PATCH 7/7] arm: Add test for AES erratum mitigation Date: Thu, 20 Jan 2022 11:27:24 +0000 Message-Id: <20220120112724.830872-8-rearnsha@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220120112724.830872-1-rearnsha@arm.com> References: <20220120112724.830872-1-rearnsha@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Earnshaw via Gcc-patches From: Richard Earnshaw Reply-To: Richard Earnshaw Cc: Richard Earnshaw Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Add a testcase for the erratum mitigation. To improve coverage use -dp on the assembler output and match the pattern names (and where needed the alternative number). gcc/testsuite/ChangeLog: * gcc.target/arm/crypto-vaese-erratum1.c: New test. --- .../gcc.target/arm/crypto-vaese-erratum1.c | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 gcc/testsuite/gcc.target/arm/crypto-vaese-erratum1.c diff --git a/gcc/testsuite/gcc.target/arm/crypto-vaese-erratum1.c b/gcc/testsuite/gcc.target/arm/crypto-vaese-erratum1.c new file mode 100644 index 00000000000..3f16688a8aa --- /dev/null +++ b/gcc/testsuite/gcc.target/arm/crypto-vaese-erratum1.c @@ -0,0 +1,28 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target arm_crypto_ok } */ +/* { dg-options "-O2 -mfix-cortex-a57-aes-1742098 -dp" } */ +/* { dg-add-options arm_crypto } */ + +#include "arm_neon.h" + +uint8x16_t +foo (uint8x16_t v) +{ + const uint8x16_t key1 = {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f}; + const uint8x16_t key2 = {0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, + 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f}; + int i; + + for (i = 0; i < 16; ++i) + { + v = vaesmcq_u8 (vaeseq_u8 (v, key1)); + v = vaesmcq_u8 (vaeseq_u8 (v, key2)); + } + return v; +} + +/* { dg-final { scan-assembler "aese.8\tq\[0-9\]+, q\[0-9\]+" } } */ +/* { dg-final { scan-assembler-times "aes_op_protect/2" 2} } */ +/* { dg-final { scan-assembler-times "aes_op_protect/0" 1} } */ +/* { dg-final { scan-assembler-times "(?:aesmc|aese_fused)_protected" 1} } */