From patchwork Thu Jan 20 11:27:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Earnshaw X-Patchwork-Id: 50275 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A3DD2385800A for ; Thu, 20 Jan 2022 11:34:39 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A3DD2385800A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1642678479; bh=ZkcnF7fPwoUjil1ZH3V2isPzbdoCEUlIuIIkHZUpxPU=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=jPLqBUlbAOCgrVckFwhnQ8cwxsZoU9M0f+3ACQ+4nfBYMijl/0RgGQENQQL3T7C7h jBv37/PTqPhouJup+RIuGf/pJ1FUs5pzrmDofRE63oJ/S6EvGlag0zjyW8arFFR7sk 6Qrnow0W7jeTrxn2gs/mQqWdQZGkRcE2FsdUi8fw= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 8DE413857C76 for ; Thu, 20 Jan 2022 11:28:24 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8DE413857C76 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5EBD86D; Thu, 20 Jan 2022 03:28:24 -0800 (PST) Received: from e126323.arm.com (unknown [10.57.36.197]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D7DD13F774; Thu, 20 Jan 2022 03:28:23 -0800 (PST) To: GCC patches Subject: [PATCH 6/7] arm: elide some cases where the AES erratum workaround is not required. Date: Thu, 20 Jan 2022 11:27:23 +0000 Message-Id: <20220120112724.830872-7-rearnsha@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220120112724.830872-1-rearnsha@arm.com> References: <20220120112724.830872-1-rearnsha@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Earnshaw via Gcc-patches From: Richard Earnshaw Reply-To: Richard Earnshaw Cc: Richard Earnshaw Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Some common cases where the AES erratum workaround are not required are when there are 64- or 128-bit loads from memory, moving a 128-bit value from core registers, and where a 128-bit constant is being loaded from a literal pool. The loads may also be misaligned or generated via a neon intrinsic function. gcc/ChangeLog: * config/arm/crypto.md (aes_op_protect): Allow moves from core registers and from memory. (aes_op_protect_misalign_load): New pattern. (aes_op_protect_neon_vld1v16qi): New pattern. --- gcc/config/arm/crypto.md | 55 ++++++++++++++++++++++++++++++++++------ 1 file changed, 47 insertions(+), 8 deletions(-) diff --git a/gcc/config/arm/crypto.md b/gcc/config/arm/crypto.md index df857352382..4c785073028 100644 --- a/gcc/config/arm/crypto.md +++ b/gcc/config/arm/crypto.md @@ -62,17 +62,56 @@ (define_insn "*crypto__insn" [(set_attr "type" "")] ) -; Mitigate against AES erratum on Cortex-A57 and Cortex-A72 by performing -; a 128-bit operation on an operand producer. This can be eliminated only -; if we know that the operand was produced by a full-width operation. -; V16QImode matches for the AES instructions. +;; Mitigate against AES erratum on Cortex-A57 and Cortex-A72 by +;; performing a 128-bit operation on an operand producer. This can be +;; eliminated only if we know that the operand was produced by a +;; full-width operation. V16QImode matches for the AES +;; instructions. Handle some very common cases where the source is +;; known to be safe (transfers from core registers and memory). (define_insn "aes_op_protect" - [(set (match_operand:V16QI 0 "register_operand" "=w") - (unspec:V16QI [(match_operand:V16QI 1 "register_operand" "0")] + [(set (match_operand:V16QI 0 "register_operand" "=w,w,w") + (unspec:V16QI [(match_operand:V16QI 1 "general_operand" "w,r,Uni")] + UNSPEC_AES_PROTECT))] + "TARGET_CRYPTO && fix_aes_erratum_1742098" + { + switch (which_alternative) + { + case 0: return "vmov\t%q0, %q1"; + case 1: return "vmov\t%e0, %Q1, %R1 @ V16QI\;vmov\t%f0, %J1, %K1"; + case 2: return output_move_neon (operands); + default: gcc_unreachable (); + } + } + [(set_attr "type" "neon_move_q,neon_from_gp_q,neon_load1_4reg") + (set_attr "length" "4,8,8") + (set_attr "arm_pool_range" "*,*,1020") + (set_attr "thumb2_pool_range" "*,*,1018") + (set_attr "neg_pool_range" "*,*,996")] +) + +;; Another safe case is when a movmisalign load is used as the source. +(define_insn "*aes_op_protect_misalign_load" + [(set (match_operand:V16QI 0 "s_register_operand" "=w") + (unspec:V16QI + [(unspec:V16QI + [(match_operand:V16QI 1 "neon_permissive_struct_operand" "Um")] + UNSPEC_MISALIGNED_ACCESS)] UNSPEC_AES_PROTECT))] "TARGET_CRYPTO && fix_aes_erratum_1742098" - "vmov\\t%q0, %q1" - [(set_attr "type" "neon_move_q")] + "vld1.8\t%{q0}, %A1" + [(set_attr "type" "neon_load1_1reg_q")] +) + +;; Similarly for the vld1 intrinsic +(define_insn "aes_op_protect_neon_vld1v16qi" + [(set (match_operand:V16QI 0 "s_register_operand" "=w") + (unspec:V16QI + [(unspec:V16QI [(match_operand:V16QI 1 "neon_struct_operand" "Um")] + UNSPEC_VLD1)] + UNSPEC_AES_PROTECT))] + "TARGET_NEON" + "vld1.8\t%h0, %A1" + [(set_attr "type" "neon_load1_1reg_q")] ) ;; An AESMC operation can feed directly into a subsequent AES