From patchwork Wed Aug 5 10:35:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sivan Shani X-Patchwork-Id: 140643 X-Patchwork-Delegate: alice.carlotti@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2EB934BAE7DE for ; Wed, 5 Aug 2026 10:37:14 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2EB934BAE7DE Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=IQi1V7Qy; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=IQi1V7Qy X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from AM0PR83CU005.outbound.protection.outlook.com (mail-westeuropeazon11010030.outbound.protection.outlook.com [52.101.69.30]) by sourceware.org (Postfix) with ESMTPS id 029FE4BAE7DA for ; Wed, 5 Aug 2026 10:36:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 029FE4BAE7DA Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 029FE4BAE7DA Authentication-Results: sourceware.org; arc=pass smtp.remote-ip=52.101.69.30 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1785926182; cv=pass; b=cMXk9VPzFP2PmzSYMq/W87Uveh+JU3B9sbIEVbsJERBhxQnoOooIf4g1t0TBNAE9iHsKC5GarEegMGgwBmZnPXyVERJmhZ0iyoVpfKLfhObf15loAfnYQ59vW0U3h7SyvZ8zbAwSMaIBidh2cTaeyDtU+jAa+Teg3ptE/RsFbBs= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1785926182; c=relaxed/simple; bh=y3WoCogOL4ZGFHf0Pqy63qVsyfeIyzXi6IrFZnFoWC0=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=Uo1JtvEeCfQrQD107joAnBLZDfYHKH9kp1QUkWzTlNPXTFSS524SDgX01yzPHQVSRcdeU5nIPGmgMWZ44OUQM5nxa7vK/M+cXPdCh3DtPQpPxcb6JrULvyMCvcaU+r+VszU5tvVcRcUZk1+1ggVbTt1FDrf8CEYVlDEU1kOvLyQ= ARC-Authentication-Results: i=3; sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=IQi1V7Qy; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=IQi1V7Qy DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 029FE4BAE7DA ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=ZaNC7JV5dqpYxwm9KkhgpksM+LgmvITWD9YzL9z6pjjeRu8KdsFFgGQ/xZrWnRlFyE1pSQSpvR+mfw9vHN47mVQTFtThJdySTqOOOLvqK+U2jn4sooh9jbO5xVda/kifLPXwRMXHPU8L8BLz1S151fFlibZlU1AoTGRl4ASn5WdFiemXTztJ4RKZ5crMR4HdIpxfZsqtVlGHOv7ND3ItP0lRdRhQFJSpQYP60a/5vXent/DXl95byVx2EUsXpkpfTBRyShD8jYQa5C+qsaMY+3qCL7RwhxuL8v4aCMT1tpCbjzKnTZtFFRC7IvZLN2dlf5M9b+y4+FSSG0mb0hqikw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bbt1OBo+XtAJ9WrMqQJWkmn/wGpbPYwKPRitNzNNtEA=; b=i2rPznRpyJza2D3lvZZpuvNVOBGIdtPspgeZAdJX0j3QO1JMbvD/q79h7W6NusJvRIBGaGnmwg9HOCqYqJ38v3yM3p6yA3JEOKa5BA90jMsFOcHu7BphtRvD4DuuMk0H51ev6iFzL3oPa/0RTJoLWok3skqUas4P/O/MbCdQEi0bwxbnmOLblroX3xmXVOtPMbECZ8TjAbeGAzkm2WvgxbMPGIQr6wsco8DttJbY1dQgWDWEyfahlMPfyvblRAf/5hvM6YkuC1Kv5X1vF2BVcqrlydltCWxrTZNl+wEHZ2eLzuZevcPWMS436xxp4r8+3Yr0ulflnzxiBccKqTdcyA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Bbt1OBo+XtAJ9WrMqQJWkmn/wGpbPYwKPRitNzNNtEA=; b=IQi1V7Qyn44TFufp8cTgzvYMNuj+eXIOAHRkGk9nQ/xKXE10CZx5zQZFLnaSEpoexrk9Dmb2dkQxY627AevSVb5xUkc/QgGJ9DQw8U9nBzLeL58kMc1I/sA7FYldIBeBqWlth0zO1VlDxisbGCb8GEk2vKR8uUBg7cvGBCUY26Y= Received: from DU7P251CA0026.EURP251.PROD.OUTLOOK.COM (2603:10a6:10:551::30) by GV1PR08MB10523.eurprd08.prod.outlook.com (2603:10a6:150:16a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.292.19; Wed, 5 Aug 2026 10:36:16 +0000 Received: from DB5PEPF00014B94.eurprd02.prod.outlook.com (2603:10a6:10:551:cafe::9c) by DU7P251CA0026.outlook.office365.com (2603:10a6:10:551::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.292.16 via Frontend Transport; Wed, 5 Aug 2026 10:36:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by DB5PEPF00014B94.mail.protection.outlook.com (10.167.8.232) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.292.8 via Frontend Transport; Wed, 5 Aug 2026 10:36:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WEx+gguMJ99+TJ7MBvNWu8NvypbKViFLdUAIU+Pi3cq+K8rZ44WAwhJzkU45UCMydFbYxAe2HqIygg9WoVN+gW5iErTtqmnZmfTm3FZ1si3ZwmnAaQwsgXvpfPizinu47glrxe9UPNc2VN3/QNN0hR7+VCEOXu0rCk0TcJSidxzJ9H2dd+6FHvw6s2/jmBFl7K+OHlWdI/ssiZbH1kxnXEJagecQ4aY/EHRRN7p9cBJcS1bFLDlMuwBdAYtLr43zv1aBJAcm0bycvphynt7rZ+QsjwS5hnhqo9TWSiR8mc0kJY/vr835VfbbwpY7KURzSrbczOKAgA4fsTG+/cjckA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bbt1OBo+XtAJ9WrMqQJWkmn/wGpbPYwKPRitNzNNtEA=; b=YsUxdaASJVkiSiLOfSn7CDKfben3GC4elJZQm1lbPUehjRrDRFCt1PDVI//YuKJ9Q06/yC3urrZQVXmrB4R8Ap/7KcPiQ3EzbgsI/UaaJHm6+eEkJ1iKlzbGMIzQk0gHd73EIVziXMc6c6jUEzxPeRVeX3agMeH1tNapkKSVKT4pMcV0J+OnhwTaNx7Bdi5QZvmU6cwCxNOtxsPYSmR+KEi9MxHvdXwPO2wl0bLeIPaNzi0xVtm7nlfVY0QDMvgrBahdJVfU8BAkPSd6czwBj+N+Qf4ezS74IBK+31wOlMAaNvA+ye+Ye+wwmLgldi0XZhoXuaN0YzfB257/Y4cDHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Bbt1OBo+XtAJ9WrMqQJWkmn/wGpbPYwKPRitNzNNtEA=; b=IQi1V7Qyn44TFufp8cTgzvYMNuj+eXIOAHRkGk9nQ/xKXE10CZx5zQZFLnaSEpoexrk9Dmb2dkQxY627AevSVb5xUkc/QgGJ9DQw8U9nBzLeL58kMc1I/sA7FYldIBeBqWlth0zO1VlDxisbGCb8GEk2vKR8uUBg7cvGBCUY26Y= Received: from AM9P193CA0021.EURP193.PROD.OUTLOOK.COM (2603:10a6:20b:21e::26) by DU0PR08MB7761.eurprd08.prod.outlook.com (2603:10a6:10:3bb::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.18; Wed, 5 Aug 2026 10:35:39 +0000 Received: from DB1PEPF000509F2.eurprd02.prod.outlook.com (2603:10a6:20b:21e:cafe::77) by AM9P193CA0021.outlook.office365.com (2603:10a6:20b:21e::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.270.20 via Frontend Transport; Wed, 5 Aug 2026 10:35:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF000509F2.mail.protection.outlook.com (10.167.242.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.292.8 via Frontend Transport; Wed, 5 Aug 2026 10:35:39 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX03.Arm.com (10.240.25.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Wed, 5 Aug 2026 10:35:37 +0000 Received: from ip-10-249-56-69.eu-west-1.compute.internal (10.249.56.69) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Wed, 5 Aug 2026 10:35:37 +0000 From: Sivan Shani To: CC: Subject: [PATCH v3] aarch64: Add SVE AES2 ACLE builtins Date: Wed, 5 Aug 2026 10:35:35 +0000 Message-ID: <20260805103535.182698-1-sivan.shani@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF000509F2:EE_|DU0PR08MB7761:EE_|DB5PEPF00014B94:EE_|GV1PR08MB10523:EE_ X-MS-Office365-Filtering-Correlation-Id: c89f9ff1-588c-428e-4c7a-08def2dd60e2 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|1800799024|82310400026|36860700016|376014|23010399003|22082099003|18002099003|11063799006|5023799004|56012099006|3023799007|13003099007|10067099003|6133799003; X-Microsoft-Antispam-Message-Info-Original: ckN/3ojYGR97ZAo70dMl7IHvATEwObj9Nrjp9Mt+gCVUeSH4eL9qy7DfBQef1t6NMACR5ZtXwL4vKSWuXTtR4ePovCdz1pvs3q16NwXbq3jr4CwXm7QDg9ul0ekB62rdvYIF7dI+pWAyaOJZx6GhGrSWhcIfGKD54tOHbonkwnNcGfRWd37eLmsc5bxIRY5lXAjIoahfPJGfPWdDiYuEMyBOUPFOCLf4wzdYNKv5wjoguIe7Sp68Zsf870Juo0UHgfVwVNvKHwpFPbrcnqo38PxC8WU+jumDA0vt5tznZCTZfLmp4jh4LXjE3ZfdvVFD7MMPltPJEycCiWGcXlq7dsN7XZGbMhVgGtkEa0NmSILlrKjnIHXJQ7m9nsfzdEazgglMMxB9qcP1ssYDMq2J9bUvSmY5dk3zNbKinFVvpOcj46fm/q8LNs6UJjIoVju33ZAk6MMO2rgPGZ8223H0ZYq7Cr+pPIgcSoZ1H9/TWiY1XHHibwNP9U+pMuIiywm5tuElyqCbynLaPnEVPnNr5GNNkOEeWlrGaHmKOrqMR7xLsYkh85TcAFsnB0Og9b7raz60xdizsu4TxVBz54Mm33ZIs99ekjQqjfOTHkSe2w/HR+B2Uj6/JmrM/vg/lAAlhbQDGhY4L7ABGmWqCULLaQ== X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(36860700016)(376014)(23010399003)(22082099003)(18002099003)(11063799006)(5023799004)(56012099006)(3023799007)(13003099007)(10067099003)(6133799003); DIR:OUT; SFP:1101; X-Exchange-RoutingPolicyChecked: KNlhEqnIhaZTHG5v1MtEZoBGoPpScS82Dqj+rTt5H+jWCHWWT0NUAFhrGlkILQEyjnMoJtn7l2eCHTnMvgDBNTnvIO4rk3cVyKnI0XTiw0+iHncWbHju7PyCDvX0b9aYf2HCCCbvGvzRGATIAxj01gzMKR18yUwPDlXrwnj3Jcrk18ey5x6IbgEbHBMa5YQ524ocNd2Y7zGGeQvA4Ep9D86Zud1n8Rp/tF4nbRP93Uw0rfPHiBuo3Pd7eEMlahKI3kSqAT5n2QUXngiALgOmS2XK+iIPDIL+PAGz8tKy22W+uhM/AF8yj1KnzXhVLWl9NWzcoXaS9R+Tg1ib7j2mRA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB7761 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5PEPF00014B94.eurprd02.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 70c91b71-4968-4b5a-e927-08def2dd4b03 X-Microsoft-Antispam: BCL:0; ARA:13230040|14060799003|376014|35042699022|23010399003|36860700016|82310400026|1800799024|13003099007|6133799003|10067099003|11063799006|5023799004|56012099006|18002099003|22082099003|3023799007; X-Microsoft-Antispam-Message-Info: b28YpvqjALlVGVC6dFXsHnN1ezzsiqNWY5K/ggSPfkeYQlEj7LSBSvQ7sPXdHxdpwKThBzDVeCtYhkjuAPm0IafZXQk9p71BhNVZTLiWFi26c/ihZobjX8p0e5am9TdG9VupdQ9AcCHNQwS8JKwv2nzftUXB32IUrURyx5DQ6VgNy/r2uAeAYcrmbdiVcS0CK5hhOnyaW+18gQUjFtT7QlxrUyQYuKVd/L7QNwJqXospi5N2Tbclh0JfU7tQzWHT5l59/iQY4rmIbEWZgaia7Ea95vMa7IqfSeDkXcTFfncIrq8GhYR/0OM+9hTCu3OsujyiAfAEGw5psDVZ5wsZY8x6IvsVoRfbowTO0mg9oCyfynWfZbcGvfJdoP0tvMZShcful0eIuqeyfZiJU0SXpaYhQGeCUraTZU9wy2VRv7tStzOXQm114lHUcbPGVWQzeOtM4qIvCzTMlAC+YwKa7+2bxXQE57+eCqfwp9bDTICFSrClLfsWkR+ld5o+v0lj32B7vkZucufVKfWN3BwSq/piXCADoL8r0R2Zx2LBjO+tlZj0oaCtTzJKrmQvOmi+aOaQOtzVDbTjADGnFdUjD4MAOdfUdKwsF+BSC//feLPDyzWMOMQ42zi9GIPIxLo+PvLJlaeyPlnc5VpUD+daEQ== X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(14060799003)(376014)(35042699022)(23010399003)(36860700016)(82310400026)(1800799024)(13003099007)(6133799003)(10067099003)(11063799006)(5023799004)(56012099006)(18002099003)(22082099003)(3023799007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 5JyUnCvKz6hr7TFFjWq7UDM98rk2mwqnWi0OMyBuc6d2fgOGwqTMaw/chIDKOXiheOwCQDY7qNFyOh87RTTBBmwPGeTmVp0cMeQM9OJ0Q6a255jGnbpLAp5ni31PJvVlWYlHRqgJSrs5E3Kn4DZNmboyxxY7Ud4r8SIca+o/VrmgEdIMUbk+tZUKQTDDYWs88+gM3CXhYNojnnF/peCudT/UrNSlWvmGaqHTELeZAiOTDHKcszUznpnEfSR5OcyxubzLaF9QkDWRPEKEPfi0GSVyIDQABkpZvjTECWKJRVhxNX65OKJQfZkX4+hQy4eBW/rtM/emy9zSAl2/HnRmcseek6iShBoFkk6cTIrOfG81O0zFFoapjuzCTSgunBShD0QkExiKIwG7mxjLAX48yeE8wGrdV1ST9G8Qz7SemQRy0d6MR+qzai12EA+isei7 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Aug 2026 10:36:15.9867 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c89f9ff1-588c-428e-4c7a-08def2dd60e2 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: DB5PEPF00014B94.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB10523 X-Spam-Status: No, score=-10.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org Add ACLE support for the SVE AES2 multi-vector AES indexed instructions and the 128-bit PMULL/PMLAL pair forms. This adds builtin shapes and expanders for svaese_lane, svaesd_lane, svaesemc_lane, svaesdimc_lane, svpmull_pair and svpmlal_pair, together with the corresponding RTL patterns and tests. Also add preprocessor feature macros for SVE AES2 and SSVE AES. gcc/ * config/aarch64/aarch64-c.cc (aarch64_update_cpp_builtins): Define __ARM_FEATURE_SVE_AES2 and __ARM_FEATURE_SSVE_AES. * config/aarch64/aarch64.h (TARGET_SVE_AES2): Define. * config/aarch64/aarch64.md (UNSPEC_SSVE_LANE_SELECT): New unspec. * config/aarch64/aarch64-sve-builtins-functions.h (unspec_based_aes_lane_function): New typedef. (unspec_based_aes_lane_mc_function): Likewise. * config/aarch64/aarch64-sve-builtins-shapes.cc (binary_to_pair_opt_n_def): New shape. (ternary_to_pair_opt_n_def): Likewise. (binary_aes_lane_def): Likewise. * config/aarch64/aarch64-sve-builtins-shapes.h: Declare new shapes. * config/aarch64/aarch64-sve-builtins-sve2.cc: Add new function entries. * config/aarch64/aarch64-sve-builtins-sve2.def: Add new ACLE builtin definitions. * config/aarch64/aarch64-sve-builtins-sve2.h: Declare new functions. * config/aarch64/aarch64-sve2.md: Add PMULL/PMLAL pair and AES indexed multi-vector patterns. * config/aarch64/iterators.md: Add iterators and attrs for new patterns. * config/aarch64/predicates.md (const_0_to_3_operand): New predicate. gcc/testsuite/ * gcc.target/aarch64/pragma_cpp_predefs_5.c: Test new feature macros. * gcc.target/aarch64/sve/acle/asm/test_sve_acle.h (TEST_XN_INDEXED): New macro. * gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c: New test. * gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c: New test. * gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c: New test. * gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c: New test. * gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c: New test. * gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c: New test. * lib/target-supports.exp: Add sve-aes2 assembler probe. --- Thanks for the review. The unused change to TEST_XN_SINGLE has been removed. Some small changes were needed for rebasing on master. gcc/config/aarch64/aarch64-acle-builtins.cc | 18 ++- gcc/config/aarch64/aarch64-acle-builtins.h | 2 + gcc/config/aarch64/aarch64-c.cc | 4 + .../aarch64/aarch64-sve-builtins-functions.h | 5 + .../aarch64/aarch64-sve-builtins-shapes.cc | 116 ++++++++++++++++++ .../aarch64/aarch64-sve-builtins-shapes.h | 3 + .../aarch64/aarch64-sve-builtins-sve2.cc | 16 ++- .../aarch64/aarch64-sve-builtins-sve2.def | 10 ++ .../aarch64/aarch64-sve-builtins-sve2.h | 7 ++ gcc/config/aarch64/aarch64-sve2.md | 71 +++++++++++ gcc/config/aarch64/aarch64.h | 2 + gcc/config/aarch64/aarch64.md | 3 + gcc/config/aarch64/iterators.md | 8 ++ gcc/config/aarch64/predicates.md | 4 + .../gcc.target/aarch64/pragma_cpp_predefs_5.c | 22 ++++ .../aarch64/sve/acle/asm/test_sve_acle.h | 18 +++ .../aarch64/sve2/acle/asm/aesd_lane_u8.c | 92 ++++++++++++++ .../aarch64/sve2/acle/asm/aesdimc_lane_u8.c | 91 ++++++++++++++ .../aarch64/sve2/acle/asm/aese_lane_u8.c | 91 ++++++++++++++ .../aarch64/sve2/acle/asm/aesemc_lane_u8.c | 91 ++++++++++++++ .../aarch64/sve2/acle/asm/pmlal_pair_u64.c | 113 +++++++++++++++++ .../aarch64/sve2/acle/asm/pmull_pair_u64.c | 65 ++++++++++ gcc/testsuite/lib/target-supports.exp | 4 +- 23 files changed, 850 insertions(+), 6 deletions(-) create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c diff --git a/gcc/config/aarch64/aarch64-acle-builtins.cc b/gcc/config/aarch64/aarch64-acle-builtins.cc index e1187859ec5..98ba6c8d347 100644 --- a/gcc/config/aarch64/aarch64-acle-builtins.cc +++ b/gcc/config/aarch64/aarch64-acle-builtins.cc @@ -2326,8 +2326,22 @@ finish_opt_n_resolution (unsigned int argno, unsigned int first_argno, unsigned int expected_bits, type_suffix_index inferred_type) { - if (inferred_type == NUM_TYPE_SUFFIXES) - inferred_type = first_type; + return finish_opt_n_resolution (argno, first_argno, first_type, + expected_tclass, expected_bits, + (inferred_type == NUM_TYPE_SUFFIXES + ? sve_type (first_type) + : sve_type (inferred_type))); +} + +tree function_resolver:: +finish_opt_n_resolution (unsigned int argno, unsigned int first_argno, + type_suffix_index first_type, + type_class_index expected_tclass, + unsigned int expected_bits, + sve_type inferred_type) +{ + if (!inferred_type) + inferred_type = sve_type (first_type); tree scalar_form = lookup_form (MODE_n, inferred_type); /* Allow the final argument to be scalar, if an _n form exists. */ diff --git a/gcc/config/aarch64/aarch64-acle-builtins.h b/gcc/config/aarch64/aarch64-acle-builtins.h index 6314ec5d312..42150a2632b 100644 --- a/gcc/config/aarch64/aarch64-acle-builtins.h +++ b/gcc/config/aarch64/aarch64-acle-builtins.h @@ -590,6 +590,8 @@ public: type_class_index = SAME_TYPE_CLASS, unsigned int = SAME_SIZE, type_suffix_index = NUM_TYPE_SUFFIXES); + tree finish_opt_n_resolution (unsigned int, unsigned int, type_suffix_index, + type_class_index, unsigned int, sve_type); tree finish_opt_single_resolution (unsigned int, unsigned int, sve_type, type_class_index = SAME_TYPE_CLASS); diff --git a/gcc/config/aarch64/aarch64-c.cc b/gcc/config/aarch64/aarch64-c.cc index eb47a57ede4..454c318d0d8 100644 --- a/gcc/config/aarch64/aarch64-c.cc +++ b/gcc/config/aarch64/aarch64-c.cc @@ -227,6 +227,10 @@ aarch64_update_cpp_builtins (cpp_reader *pfile) aarch64_def_or_undef (AARCH64_HAVE_ISA (SVE2), "__ARM_FEATURE_SVE2", pfile); aarch64_def_or_undef (AARCH64_HAVE_ISA (SVE2) && AARCH64_HAVE_ISA (SVE_AES), "__ARM_FEATURE_SVE2_AES", pfile); + aarch64_def_or_undef (AARCH64_HAVE_ISA (SVE_AES2), "__ARM_FEATURE_SVE_AES2", + pfile); + aarch64_def_or_undef (AARCH64_HAVE_ISA (SSVE_AES), "__ARM_FEATURE_SSVE_AES", + pfile); aarch64_def_or_undef (AARCH64_HAVE_ISA (SVE_BITPERM) && AARCH64_HAVE_ISA (SVE2), "__ARM_FEATURE_SVE2_BITPERM", pfile); diff --git a/gcc/config/aarch64/aarch64-sve-builtins-functions.h b/gcc/config/aarch64/aarch64-sve-builtins-functions.h index 072c0bf2c89..ebbefaf739e 100644 --- a/gcc/config/aarch64/aarch64-sve-builtins-functions.h +++ b/gcc/config/aarch64/aarch64-sve-builtins-functions.h @@ -397,6 +397,11 @@ typedef unspec_based_function_exact_insn typedef unspec_based_function_exact_insn unspec_based_sub_lane_function; +typedef unspec_based_function_exact_insn + unspec_based_aes_lane_function; +typedef unspec_based_function_exact_insn + unspec_based_aes_lane_mc_function; + /* A function that has conditional and unconditional forms, with both forms being associated with a single unspec each. */ class cond_or_uncond_unspec_function : public function_base diff --git a/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc b/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc index 90bc80d8195..cf128bc9464 100644 --- a/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc +++ b/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc @@ -1522,6 +1522,122 @@ struct binary_int_opt_single_n_def : public overloaded_base<0> }; SHAPE (binary_int_opt_single_n); +/* svx_t svfoo[_t0_g](sv_t, sv_t) + svx_t svfoo[_n_t0_g](sv_t, _t). */ +struct binary_to_pair_opt_n_def : public overloaded_base<0> +{ + bool explicit_group_suffix_p () const override { return false; } + + void + build (function_builder &b, const function_group_info &group) const override + { + b.add_overloaded_functions (group, MODE_none); + build_all (b, "t0,v0,v0", group, MODE_none); + build_all (b, "t0,v0,s0", group, MODE_n); + } + + tree + resolve (function_resolver &r) const override + { + unsigned int i, nargs; + type_suffix_index type; + if (!r.check_gp_argument (2, i, nargs) + || (type = r.infer_vector_type (i)) == NUM_TYPE_SUFFIXES) + return error_mark_node; + + return r.finish_opt_n_resolution (i + 1, i, type, r.SAME_TYPE_CLASS, + r.SAME_SIZE, sve_type (type, 2)); + } +}; +SHAPE (binary_to_pair_opt_n); + +/* svx_t svfoo[_t0_g](svx_t, sv_t, sv_t) + svx_t svfoo[_n_t0_g](svx_t, sv_t, _t). */ +struct ternary_to_pair_opt_n_def : public overloaded_base<0> +{ + bool explicit_group_suffix_p () const override { return false; } + + void + build (function_builder &b, const function_group_info &group) const override + { + b.add_overloaded_functions (group, MODE_none); + build_all (b, "t0,t0,v0,v0", group, MODE_none); + build_all (b, "t0,t0,v0,s0", group, MODE_n); + } + + tree + resolve (function_resolver &r) const override + { + unsigned int i, nargs; + sve_type type; + if (!r.check_gp_argument (3, i, nargs) + || !(type = r.infer_sve_type (i))) + return error_mark_node; + + if (type.num_vectors != 2) + { + r.report_incorrect_num_vectors (i, type, 2); + return error_mark_node; + } + + if (!r.require_derived_vector_type (i + 1, i, type)) + return error_mark_node; + + return r.finish_opt_n_resolution (i + 2, i + 1, type.type, + r.SAME_TYPE_CLASS, r.SAME_SIZE, type); + } +}; +SHAPE (ternary_to_pair_opt_n); + +/* svuint8x2_t svaes<...>_lane[_u8_x2] (svuint8x2_t zdn, svuint8_t zm, uint64_t + index); + and + svuint8x4_t svaes<...>_lane[_u8_x4] (svuint8x4_t zdn, svuint8_t zm, uint64_t + index); + When index is in range[0-3] +*/ +struct binary_aes_lane_def : public overloaded_base<0> +{ + bool explicit_group_suffix_p () const override { return false; } + + void + build (function_builder &b, const function_group_info &group) const override + { + b.add_overloaded_functions (group, MODE_none); + build_all (b, "t0,t0,v0,su64", group, MODE_none); + } + + tree + resolve (function_resolver &r) const override + { + if (!r.check_num_arguments (3)) + return error_mark_node; + + sve_type type = r.infer_sve_type (0); + if (!type) + return error_mark_node; + + if (type.num_vectors != 2 && type.num_vectors != 4) + return error_mark_node; + + if (!r.require_vector_type (1, VECTOR_TYPE_svuint8_t)) + return error_mark_node; + + if (!r.require_integer_immediate (2)) + return error_mark_node; + + return r.resolve_to (MODE_none, type); + } + + bool + check (function_checker &c) const override + { + return c.require_immediate_lane_index (2, 0, 4); + } +}; +SHAPE (binary_aes_lane); + + /* sv_t svfoo_(sv_t, sv_t, uint64_t) where the final argument is an integer constant expression in the diff --git a/gcc/config/aarch64/aarch64-sve-builtins-shapes.h b/gcc/config/aarch64/aarch64-sve-builtins-shapes.h index ce02b5a45ec..0fca00b36d1 100644 --- a/gcc/config/aarch64/aarch64-sve-builtins-shapes.h +++ b/gcc/config/aarch64/aarch64-sve-builtins-shapes.h @@ -81,6 +81,7 @@ namespace aarch64_acle extern const function_shape *const adr_index; extern const function_shape *const adr_offset; extern const function_shape *const binary; + extern const function_shape *const binary_aes_lane; extern const function_shape *const binary_int_opt_n; extern const function_shape *const binary_int_opt_single_n; extern const function_shape *const binary_lane; @@ -95,6 +96,7 @@ namespace aarch64_acle extern const function_shape *const binary_rotate; extern const function_shape *const binary_scalar; extern const function_shape *const binary_single; + extern const function_shape *const binary_to_pair_opt_n; extern const function_shape *const binary_to_uint; extern const function_shape *const binary_uint; extern const function_shape *const binary_uint_n; @@ -233,6 +235,7 @@ namespace aarch64_acle extern const function_shape *const ternary_rotate; extern const function_shape *const ternary_shift_left_imm; extern const function_shape *const ternary_shift_right_imm; + extern const function_shape *const ternary_to_pair_opt_n; extern const function_shape *const ternary_uint; extern const function_shape *const ternary_uintq_intq; extern const function_shape *const ternary_uintq_intq_lane; diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc b/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc index 738ef201324..c22b5b176be 100644 --- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc +++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc @@ -1070,7 +1070,15 @@ FUNCTION (svaddqv, reduction, (UNSPEC_ADDQV, UNSPEC_ADDQV, UNSPEC_FADDQV)) FUNCTION (svaddwb, unspec_based_function, (UNSPEC_SADDWB, UNSPEC_UADDWB)) FUNCTION (svaddwt, unspec_based_function, (UNSPEC_SADDWT, UNSPEC_UADDWT)) FUNCTION (svaesd, fixed_insn_function, (CODE_FOR_aarch64_sve2_aesd)) +FUNCTION (svaesd_lane, unspec_based_aes_lane_function, + (UNSPEC_NONE, UNSPEC_AESD)) +FUNCTION (svaesdimc_lane, unspec_based_aes_lane_mc_function, + (UNSPEC_NONE, UNSPEC_AESD)) FUNCTION (svaese, fixed_insn_function, (CODE_FOR_aarch64_sve2_aese)) +FUNCTION (svaese_lane, unspec_based_aes_lane_function, + (UNSPEC_NONE, UNSPEC_AESE)) +FUNCTION (svaesemc_lane, unspec_based_aes_lane_mc_function, + (UNSPEC_NONE, UNSPEC_AESE)) FUNCTION (svaesimc, fixed_insn_function, (CODE_FOR_aarch64_sve2_aesimc)) FUNCTION (svaesmc, fixed_insn_function, (CODE_FOR_aarch64_sve2_aesmc)) FUNCTION (svamax, faminmaximpl, (UNSPEC_COND_FAMAX, UNSPEC_FAMAX)) @@ -1194,13 +1202,17 @@ FUNCTION (svnbsl, CODE_FOR_MODE0 (aarch64_sve2_nbsl),) FUNCTION (svnmatch, svmatch_svnmatch_impl, (UNSPEC_NMATCH)) FUNCTION (svorqv, reduction, (UNSPEC_ORQV, UNSPEC_ORQV)) FUNCTION (svpext_lane, svpext_lane_impl,) +FUNCTION (svpmlal_pair, fixed_insn_function, (CODE_FOR_aarch64_sve_pmlal_pair)) FUNCTION (svpmov, svpmov_impl,) FUNCTION (svpmov_lane, svpmov_lane_impl,) FUNCTION (svpmul, CODE_FOR_MODE0 (aarch64_sve2_pmul),) +FUNCTION (svpmull_pair, fixed_insn_function, (CODE_FOR_aarch64_sve_pmull_pair)) FUNCTION (svpmullb, unspec_based_function, (UNSPEC_NONE, UNSPEC_PMULLB)) -FUNCTION (svpmullb_pair, unspec_based_function, (UNSPEC_NONE, UNSPEC_PMULLB_PAIR)) +FUNCTION (svpmullb_pair, unspec_based_function, + (UNSPEC_NONE, UNSPEC_PMULLB_PAIR)) FUNCTION (svpmullt, unspec_based_function, (UNSPEC_NONE, UNSPEC_PMULLT)) -FUNCTION (svpmullt_pair, unspec_based_function, (UNSPEC_NONE, UNSPEC_PMULLT_PAIR)) +FUNCTION (svpmullt_pair, unspec_based_function, + (UNSPEC_NONE, UNSPEC_PMULLT_PAIR)) FUNCTION (svpsel_lane, svpsel_lane_impl,) FUNCTION (svqabs, rtx_code_function, (SS_ABS, UNKNOWN)) FUNCTION (svqcadd, svqcadd_impl,) diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.def b/gcc/config/aarch64/aarch64-sve-builtins-sve2.def index a977b2c1c1a..87f7b472dd9 100644 --- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.def +++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.def @@ -206,6 +206,16 @@ DEF_SVE_FUNCTION (svpmullb_pair, binary_opt_n, d_unsigned, none) DEF_SVE_FUNCTION (svpmullt_pair, binary_opt_n, d_unsigned, none) #undef REQUIRED_EXTENSIONS +#define REQUIRED_EXTENSIONS streaming_compatible (AARCH64_FL_SVE_AES2, \ + AARCH64_FL_SSVE_AES) +DEF_SVE_FUNCTION_GS (svaesd_lane, binary_aes_lane, b_unsigned, x24, none) +DEF_SVE_FUNCTION_GS (svaese_lane, binary_aes_lane, b_unsigned, x24, none) +DEF_SVE_FUNCTION_GS (svaesdimc_lane, binary_aes_lane, b_unsigned, x24, none) +DEF_SVE_FUNCTION_GS (svaesemc_lane, binary_aes_lane, b_unsigned, x24, none) +DEF_SVE_FUNCTION_GS (svpmull_pair, binary_to_pair_opt_n, d_unsigned, x2, none) +DEF_SVE_FUNCTION_GS (svpmlal_pair, ternary_to_pair_opt_n, d_unsigned, x2, none) +#undef REQUIRED_EXTENSIONS + #define REQUIRED_EXTENSIONS streaming_compatible (AARCH64_FL_SVE2 \ | AARCH64_FL_SVE_BITPERM, \ AARCH64_FL_SSVE_BITPERM) diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.h b/gcc/config/aarch64/aarch64-sve-builtins-sve2.h index 4bf8f21776a..0cde240403c 100644 --- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.h +++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.h @@ -42,7 +42,11 @@ namespace aarch64_acle extern const function_base *const svaddwb; extern const function_base *const svaddwt; extern const function_base *const svaesd; + extern const function_base *const svaesd_lane; + extern const function_base *const svaesdimc_lane; extern const function_base *const svaese; + extern const function_base *const svaese_lane; + extern const function_base *const svaesemc_lane; extern const function_base *const svaesimc; extern const function_base *const svaesmc; extern const function_base *const svandqv; @@ -138,13 +142,16 @@ namespace aarch64_acle extern const function_base *const svnmatch; extern const function_base *const svorqv; extern const function_base *const svpext_lane; + extern const function_base *const svpmlal_pair; extern const function_base *const svpmov; extern const function_base *const svpmov_lane; extern const function_base *const svpmul; + extern const function_base *const svpmull_pair; extern const function_base *const svpmullb; extern const function_base *const svpmullb_pair; extern const function_base *const svpmullt; extern const function_base *const svpmullt_pair; + extern const function_base *const svpmlal_pair; extern const function_base *const svpsel_lane; extern const function_base *const svqabs; extern const function_base *const svqcadd; diff --git a/gcc/config/aarch64/aarch64-sve2.md b/gcc/config/aarch64/aarch64-sve2.md index a8bee6b6dc7..33b39412e36 100644 --- a/gcc/config/aarch64/aarch64-sve2.md +++ b/gcc/config/aarch64/aarch64-sve2.md @@ -4225,6 +4225,35 @@ (define_insn "@aarch64_sve_" [(set_attr "sve_type" "sve_int_pmul")] ) +;; Polynomial multiply corresponding D elements and return the widened Q +;; results as a pair of consecutive destination vectors. +;; PMULL { .Q-.Q }, .D, .D +(define_insn "aarch64_sve_pmull_pair" + [(set (match_operand:VNx4DI 0 "aligned_register_operand" "=Uw2") + (unspec:VNx4DI + [(match_operand:VNx2DI 1 "register_operand" "w") + (match_operand:VNx2DI 2 "register_operand" "w")] + UNSPEC_PMULL_PAIR))] + "TARGET_SVE_AES2" + "pmull\t{%S0.q - %T0.q}, %1.d, %2.d" + [(set_attr "sve_type" "sve_int_pmul")] +) + +;; Polynomial multiply corresponding D elements and XOR-accumulate the widened +;; Q results into a pair of consecutive destination vectors. +;; PMLAL { .Q-.Q }, .D, .D +(define_insn "aarch64_sve_pmlal_pair" + [(set (match_operand:VNx4DI 0 "aligned_register_operand" "=Uw2") + (unspec:VNx4DI + [(match_operand:VNx4DI 1 "aligned_register_operand" "0") + (match_operand:VNx2DI 2 "register_operand" "w") + (match_operand:VNx2DI 3 "register_operand" "w")] + UNSPEC_PMLAL_PAIR))] + "TARGET_SVE_AES2" + "pmlal\t{%S0.q - %T0.q}, %2.d, %3.d" + [(set_attr "sve_type" "sve_int_pmul")] +) + ;; ========================================================================= ;; == Comparisons and selects ;; ========================================================================= @@ -4830,6 +4859,10 @@ (define_insn "@aarch64_sve_luti" ;; - AESE ;; - AESIMC ;; - AESMC +;; - AESD (indexed, two registers and four registers) +;; - AESE (indexed, two registers and four registers) +;; - AESEMC (indexed, two registers and four registers) +;; - AESDIMC (indexed, two registers and four registers) ;; ------------------------------------------------------------------------- ;; AESD and AESE. @@ -4891,6 +4924,44 @@ (define_insn "*aarch64_sve2_aesd_fused" (set_attr "length" "8")] ) +;; AESE and AESD, indexed, two registers and four registers. +;; AES { .B-.B }, { .B-.B }, .Q[] + +(define_insn "@aarch64_sve2_aes_lane" + [(set (match_operand:SVE_QIx24 0 "aligned_register_operand" "=Uw") + (unspec:SVE_QIx24 + [(xor:SVE_QIx24 + (match_operand:SVE_QIx24 1 "aligned_register_operand" "0") + (unspec:SVE_QIx24 + [(match_operand:VNx16QI 2 "register_operand" "w") + (match_operand:SI 3 "const_0_to_3_operand")] + UNSPEC_SSVE_LANE_SELECT))] + CRYPTO_AES))] + "TARGET_SVE_AES2" + "aes\t%0, %0, %2.q[%3]" + [(set_attr "type" "crypto_aese")] +) + +;; AESEMC and AESDIMC, indexed, two registers and four registers. +;; AESEMC/AESDIMC { .B-.B }, { .B-.B }, .Q[] + +(define_insn "@aarch64_sve2_aes_lane_mc" + [(set (match_operand:SVE_QIx24 0 "aligned_register_operand" "=Uw") + (unspec:SVE_QIx24 + [(unspec:SVE_QIx24 + [(xor:SVE_QIx24 + (match_operand:SVE_QIx24 1 "aligned_register_operand" "0") + (unspec:SVE_QIx24 + [(match_operand:VNx16QI 2 "register_operand" "w") + (match_operand:SI 3 "const_0_to_3_operand")] + UNSPEC_SSVE_LANE_SELECT))] + CRYPTO_AES)] + ))] + "TARGET_SVE_AES2" + "aes\t%0, %0, %2.q[%3]" + [(set_attr "type" "crypto_aesmc")] +) + ;; ------------------------------------------------------------------------- ;; ---- Optional SHA-3 extensions ;; ------------------------------------------------------------------------- diff --git a/gcc/config/aarch64/aarch64.h b/gcc/config/aarch64/aarch64.h index 50577b136ac..11fb7a9481e 100644 --- a/gcc/config/aarch64/aarch64.h +++ b/gcc/config/aarch64/aarch64.h @@ -295,6 +295,8 @@ constexpr auto AARCH64_FL_DEFAULT_ISA_MODE ATTRIBUTE_UNUSED && (AARCH64_HAVE_ISA (SSVE_AES) \ || TARGET_NON_STREAMING)) +#define TARGET_SVE_AES2 (TARGET_SVE_AES && AARCH64_HAVE_ISA (SVE_AES2)) + /* SVE BITPERM instructions, enabled through +sve-bitperm+sve2 for non-streaming and +ssve-bitperm for streaming. */ #define TARGET_SVE_BITPERM (AARCH64_HAVE_ISA (SVE_BITPERM) \ diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md index 168a8a471fe..9cb55602c36 100644 --- a/gcc/config/aarch64/aarch64.md +++ b/gcc/config/aarch64/aarch64.md @@ -392,6 +392,9 @@ (define_c_enum "unspec" [ ;; Represents an SVE-style lane index, in which the indexing applies ;; within the containing 128-bit block. UNSPEC_SVE_LANE_SELECT + ;; Represents an SVE-style lane index, in which the indexing applies + ;; within the containing 512-bit block. + UNSPEC_SSVE_LANE_SELECT UNSPEC_SVE_CNT_PAT UNSPEC_SVE_PREFETCH UNSPEC_SVE_PREFETCH_GATHER diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md index 0d319751430..e7ae93d1896 100644 --- a/gcc/config/aarch64/iterators.md +++ b/gcc/config/aarch64/iterators.md @@ -711,6 +711,9 @@ (define_mode_iterator SVE_SI [VNx2SI VNx4SI]) (define_mode_iterator SVE_DIx24 [VNx4DI VNx8DI]) +;; SVE integer vector modes with 2 and 4 vectors of 8-bit elements. +(define_mode_iterator SVE_QIx24 [VNx32QI VNx64QI]) + ;; SVE modes with 2 or 4 elements. (define_mode_iterator SVE_24 [VNx2QI VNx2HI VNx2HF VNx2BF VNx2SI VNx2SF VNx2DI VNx2DF @@ -892,6 +895,8 @@ (define_c_enum "unspec" UNSPEC_SQDMULH ; Used in aarch64-simd.md. UNSPEC_SQRDMULH ; Used in aarch64-simd.md. UNSPEC_PMUL ; Used in aarch64-simd.md. + UNSPEC_PMULL_PAIR ; Used in aarch64-sve2.md. + UNSPEC_PMLAL_PAIR ; Used in aarch64-sve2.md. UNSPEC_FMULX ; Used in aarch64-simd.md. UNSPEC_USQADD ; Used in aarch64-simd.md. UNSPEC_SUQADD ; Used in aarch64-simd.md. @@ -3698,6 +3703,9 @@ (define_int_iterator CRC [UNSPEC_CRC32B UNSPEC_CRC32H UNSPEC_CRC32W (define_int_iterator CRYPTO_AES [UNSPEC_AESE UNSPEC_AESD]) (define_int_iterator CRYPTO_AESMC [UNSPEC_AESMC UNSPEC_AESIMC]) +(define_int_attr aes_fused_op [(UNSPEC_AESE "emc") (UNSPEC_AESD "dimc")]) +(define_int_attr aes_mc_unspec [(UNSPEC_AESE "UNSPEC_AESMC") + (UNSPEC_AESD "UNSPEC_AESIMC")]) (define_int_iterator CRYPTO_SHA1 [UNSPEC_SHA1C UNSPEC_SHA1M UNSPEC_SHA1P]) diff --git a/gcc/config/aarch64/predicates.md b/gcc/config/aarch64/predicates.md index f02486c2d9a..e2911a0bccd 100644 --- a/gcc/config/aarch64/predicates.md +++ b/gcc/config/aarch64/predicates.md @@ -50,6 +50,10 @@ (define_predicate "const0_to_1_operand" (and (match_code "const_int") (match_test "IN_RANGE (INTVAL (op), 0, 1)"))) +(define_predicate "const_0_to_3_operand" + (and (match_code "const_int") + (match_test "IN_RANGE (INTVAL (op), 0, 3)"))) + (define_predicate "const_0_to_7_operand" (and (match_code "const_int") (match_test "IN_RANGE (INTVAL (op), 0, 7)"))) diff --git a/gcc/testsuite/gcc.target/aarch64/pragma_cpp_predefs_5.c b/gcc/testsuite/gcc.target/aarch64/pragma_cpp_predefs_5.c index bb83ab434b5..0bec72d70a2 100644 --- a/gcc/testsuite/gcc.target/aarch64/pragma_cpp_predefs_5.c +++ b/gcc/testsuite/gcc.target/aarch64/pragma_cpp_predefs_5.c @@ -41,6 +41,14 @@ #error "__ARM_FEATURE_SVE2_SHA3 is defined but should not be!" #endif +#ifdef __ARM_FEATURE_SSVE_AES +#error "__ARM_FEATURE_SSVE_AES is defined but should not be!" +#endif + +#ifdef __ARM_FEATURE_SVE_AES2 +#error "__ARM_FEATURE_SVE_AES2 is defined but should not be!" +#endif + #pragma GCC pop_options #pragma GCC push_options @@ -225,6 +233,20 @@ #endif #pragma GCC pop_options +#pragma GCC push_options +#pragma GCC target "arch=armv8-a+ssve-aes" +#ifndef __ARM_FEATURE_SSVE_AES +#error "__ARM_FEATURE_SSVE_AES is not defined but should be!" +#endif +#pragma GCC pop_options + +#pragma GCC push_options +#pragma GCC target "arch=armv8-a+sve-aes2" +#ifndef __ARM_FEATURE_SVE_AES2 +#error "__ARM_FEATURE_SVE_AES2 is not defined but should be!" +#endif +#pragma GCC pop_options + int foo (int a) { diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h index aeb58ead0dd..7932a554d1e 100644 --- a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h +++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h @@ -687,6 +687,24 @@ __asm volatile ("" :: "w" (RES)); \ } +#define TEST_XN_INDEXED(NAME, TTYPE, VTYPE, CODE1, CODE2) \ + PROTO (NAME, TTYPE, (TTYPE t, VTYPE v)) \ + { \ + register TTYPE z0 __asm ("z0"); \ + register TTYPE z1 __asm ("z1"); \ + register TTYPE z2 __asm ("z2"); \ + register TTYPE z3 __asm ("z3"); \ + register VTYPE z4 __asm ("z4"); \ + register VTYPE z5 __asm ("z5"); \ + register VTYPE z6 __asm ("z6"); \ + register VTYPE z7 __asm ("z7"); \ + register uint64_t x0 __asm ("x0"); \ + __asm volatile ("" : "=r" (x0)); \ + INVOKE (CODE1, CODE2); \ + __asm volatile ("" :: "w" (t)); \ + return t; \ + } + #define TEST_DUAL_XN(NAME, TTYPE1, TTYPE2, RES, CODE1, CODE2) \ PROTO (NAME, void, ()) \ { \ diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c new file mode 100644 index 00000000000..9d3591a5f39 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c @@ -0,0 +1,92 @@ +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */ +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */ +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */ + +#include "test_sve_acle.h" + +#pragma GCC target "+sve-aes2+ssve-aes" + +/* +** test_aesd_lane_u8_x2: +** aesd {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesd_lane_u8_x2, svuint8x2_t, svuint8_t, + t = svaesd_lane_u8_x2 (t, v, 0), + t = svaesd_lane (t, v, 0)) + +/* +** test_aesd_lane_u8_x2_lane1: +** aesd {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesd_lane_u8_x2_lane1, svuint8x2_t, svuint8_t, + t = svaesd_lane_u8_x2 (t, v, 3), + t = svaesd_lane (t, v, 3)) + +/* +** test_aesd_lane_u8_x4: +** aesd {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesd_lane_u8_x4, svuint8x4_t, svuint8_t, + t = svaesd_lane_u8_x4 (t, v, 0), + t = svaesd_lane (t, v, 0)) + +/* +** test_aesd_lane_u8_x4_lane3: +** aesd {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesd_lane_u8_x4_lane3, svuint8x4_t, svuint8_t, + t = svaesd_lane_u8_x4 (t, v, 3), + t = svaesd_lane (t, v, 3)) + + +/* +** test_aesd_lane_u8_x2_regs_mov: +** mov z0.d, z3.d +** mov z1.d, z4.d +** aesd {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesd_lane_u8_x2_regs_mov, svuint8x2_t, svuint8_t, + t = svaesd_lane_u8_x2 (z3, z4, 0), + t = svaesd_lane (z3, z4, 0)) + +/* +** test_aesd_lane_u8_x2_regs_mov_lane1: +** mov z0.d, z3.d +** mov z1.d, z4.d +** aesd {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesd_lane_u8_x2_regs_mov_lane1, svuint8x2_t, svuint8_t, + t = svaesd_lane_u8_x2 (z3, z4, 3), + t = svaesd_lane (z3, z4, 3)) + +/* +** test_aesd_lane_u8_x4_regs_mov: +** mov z0.d, z3.d +** mov z1.d, z4.d +** mov z2.d, z5.d +** mov z3.d, z6.d +** aesd {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesd_lane_u8_x4_regs_mov, svuint8x4_t, svuint8_t, + t = svaesd_lane_u8_x4 (z3, z4, 0), + t = svaesd_lane (z3, z4, 0)) + +/* +** test_aesd_lane_u8_x4_regs_mov_lane3: +** mov z0.d, z3.d +** mov z1.d, z4.d +** mov z2.d, z5.d +** mov z3.d, z6.d +** aesd {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesd_lane_u8_x4_regs_mov_lane3, svuint8x4_t, svuint8_t, + t = svaesd_lane_u8_x4 (z3, z4, 3), + t = svaesd_lane (z3, z4, 3)) diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c new file mode 100644 index 00000000000..582cb3f742d --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c @@ -0,0 +1,91 @@ +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */ +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */ +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */ + +#include "test_sve_acle.h" + +#pragma GCC target "+sve-aes2+ssve-aes" + +/* +** test_aesdimc_lane_u8_x2: +** aesdimc {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesdimc_lane_u8_x2, svuint8x2_t, svuint8_t, + t = svaesdimc_lane_u8_x2 (t, v, 0), + t = svaesdimc_lane (t, v, 0)) + +/* +** test_aesdimc_lane_u8_x2_lane1: +** aesdimc {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesdimc_lane_u8_x2_lane1, svuint8x2_t, svuint8_t, + t = svaesdimc_lane_u8_x2 (t, v, 3), + t = svaesdimc_lane (t, v, 3)) + +/* +** test_aesdimc_lane_u8_x4: +** aesdimc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesdimc_lane_u8_x4, svuint8x4_t, svuint8_t, + t = svaesdimc_lane_u8_x4 (t, v, 0), + t = svaesdimc_lane (t, v, 0)) + +/* +** test_aesdimc_lane_u8_x4_lane3: +** aesdimc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesdimc_lane_u8_x4_lane3, svuint8x4_t, svuint8_t, + t = svaesdimc_lane_u8_x4 (t, v, 3), + t = svaesdimc_lane (t, v, 3)) + +/* +** test_aesdimc_lane_u8_x2_regs_mov: +** mov z0.d, z3.d +** mov z1.d, z4.d +** aesdimc {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesdimc_lane_u8_x2_regs_mov, svuint8x2_t, svuint8_t, + t = svaesdimc_lane_u8_x2 (z3, z4, 0), + t = svaesdimc_lane (z3, z4, 0)) + +/* +** test_aesdimc_lane_u8_x2_regs_mov_lane1: +** mov z0.d, z3.d +** mov z1.d, z4.d +** aesdimc {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesdimc_lane_u8_x2_regs_mov_lane1, svuint8x2_t, svuint8_t, + t = svaesdimc_lane_u8_x2 (z3, z4, 3), + t = svaesdimc_lane (z3, z4, 3)) + +/* +** test_aesdimc_lane_u8_x4_regs_mov: +** mov z0.d, z3.d +** mov z1.d, z4.d +** mov z2.d, z5.d +** mov z3.d, z6.d +** aesdimc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesdimc_lane_u8_x4_regs_mov, svuint8x4_t, svuint8_t, + t = svaesdimc_lane_u8_x4 (z3, z4, 0), + t = svaesdimc_lane (z3, z4, 0)) + +/* +** test_aesdimc_lane_u8_x4_regs_mov_lane3: +** mov z0.d, z3.d +** mov z1.d, z4.d +** mov z2.d, z5.d +** mov z3.d, z6.d +** aesdimc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesdimc_lane_u8_x4_regs_mov_lane3, svuint8x4_t, svuint8_t, + t = svaesdimc_lane_u8_x4 (z3, z4, 3), + t = svaesdimc_lane (z3, z4, 3)) diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c new file mode 100644 index 00000000000..a494e072cdb --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c @@ -0,0 +1,91 @@ +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */ +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */ +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */ + +#include "test_sve_acle.h" + +#pragma GCC target "+sve-aes2+ssve-aes" + +/* +** test_aese_lane_u8_x2: +** aese {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aese_lane_u8_x2, svuint8x2_t, svuint8_t, + t = svaese_lane_u8_x2 (t, v, 0), + t = svaese_lane (t, v, 0)) + +/* +** test_aese_lane_u8_x2_lane1: +** aese {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aese_lane_u8_x2_lane1, svuint8x2_t, svuint8_t, + t = svaese_lane_u8_x2 (t, v, 3), + t = svaese_lane (t, v, 3)) + +/* +** test_aese_lane_u8_x4: +** aese {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aese_lane_u8_x4, svuint8x4_t, svuint8_t, + t = svaese_lane_u8_x4 (t, v, 0), + t = svaese_lane (t, v, 0)) + +/* +** test_aese_lane_u8_x4_lane3: +** aese {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aese_lane_u8_x4_lane3, svuint8x4_t, svuint8_t, + t = svaese_lane_u8_x4 (t, v, 3), + t = svaese_lane (t, v, 3)) + +/* +** test_aese_lane_u8_x2_regs_mov: +** mov z0.d, z3.d +** mov z1.d, z4.d +** aese {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aese_lane_u8_x2_regs_mov, svuint8x2_t, svuint8_t, + t = svaese_lane_u8_x2 (z3, z4, 0), + t = svaese_lane (z3, z4, 0)) + +/* +** test_aese_lane_u8_x2_regs_mov_lane1: +** mov z0.d, z3.d +** mov z1.d, z4.d +** aese {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aese_lane_u8_x2_regs_mov_lane1, svuint8x2_t, svuint8_t, + t = svaese_lane_u8_x2 (z3, z4, 3), + t = svaese_lane (z3, z4, 3)) + +/* +** test_aese_lane_u8_x4_regs_mov: +** mov z0.d, z3.d +** mov z1.d, z4.d +** mov z2.d, z5.d +** mov z3.d, z6.d +** aese {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aese_lane_u8_x4_regs_mov, svuint8x4_t, svuint8_t, + t = svaese_lane_u8_x4 (z3, z4, 0), + t = svaese_lane (z3, z4, 0)) + +/* +** test_aese_lane_u8_x4_regs_mov_lane3: +** mov z0.d, z3.d +** mov z1.d, z4.d +** mov z2.d, z5.d +** mov z3.d, z6.d +** aese {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aese_lane_u8_x4_regs_mov_lane3, svuint8x4_t, svuint8_t, + t = svaese_lane_u8_x4 (z3, z4, 3), + t = svaese_lane (z3, z4, 3)) diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c new file mode 100644 index 00000000000..e3cb5c29755 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c @@ -0,0 +1,91 @@ +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */ +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */ +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */ + +#include "test_sve_acle.h" + +#pragma GCC target "+sve-aes2+ssve-aes" + +/* +** test_aesemc_lane_u8_x2: +** aesemc {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesemc_lane_u8_x2, svuint8x2_t, svuint8_t, + t = svaesemc_lane_u8_x2 (t, v, 0), + t = svaesemc_lane (t, v, 0)) + +/* +** test_aesemc_lane_u8_x2_lane1: +** aesemc {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesemc_lane_u8_x2_lane1, svuint8x2_t, svuint8_t, + t = svaesemc_lane_u8_x2 (t, v, 3), + t = svaesemc_lane (t, v, 3)) + +/* +** test_aesemc_lane_u8_x4: +** aesemc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesemc_lane_u8_x4, svuint8x4_t, svuint8_t, + t = svaesemc_lane_u8_x4 (t, v, 0), + t = svaesemc_lane (t, v, 0)) + +/* +** test_aesemc_lane_u8_x4_lane3: +** aesemc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesemc_lane_u8_x4_lane3, svuint8x4_t, svuint8_t, + t = svaesemc_lane_u8_x4 (t, v, 3), + t = svaesemc_lane (t, v, 3)) + +/* +** test_aesemc_lane_u8_x2_regs_mov: +** mov z0.d, z3.d +** mov z1.d, z4.d +** aesemc {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesemc_lane_u8_x2_regs_mov, svuint8x2_t, svuint8_t, + t = svaesemc_lane_u8_x2 (z3, z4, 0), + t = svaesemc_lane (z3, z4, 0)) + +/* +** test_aesemc_lane_u8_x2_regs_mov_lane1: +** mov z0.d, z3.d +** mov z1.d, z4.d +** aesemc {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesemc_lane_u8_x2_regs_mov_lane1, svuint8x2_t, svuint8_t, + t = svaesemc_lane_u8_x2 (z3, z4, 3), + t = svaesemc_lane (z3, z4, 3)) + +/* +** test_aesemc_lane_u8_x4_regs_mov: +** mov z0.d, z3.d +** mov z1.d, z4.d +** mov z2.d, z5.d +** mov z3.d, z6.d +** aesemc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\] +** ret +*/ +TEST_XN_INDEXED (test_aesemc_lane_u8_x4_regs_mov, svuint8x4_t, svuint8_t, + t = svaesemc_lane_u8_x4 (z3, z4, 0), + t = svaesemc_lane (z3, z4, 0)) + +/* +** test_aesemc_lane_u8_x4_regs_mov_lane3: +** mov z0.d, z3.d +** mov z1.d, z4.d +** mov z2.d, z5.d +** mov z3.d, z6.d +** aesemc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\] +** ret +*/ +TEST_XN_INDEXED (test_aesemc_lane_u8_x4_regs_mov_lane3, svuint8x4_t, svuint8_t, + t = svaesemc_lane_u8_x4 (z3, z4, 3), + t = svaesemc_lane (z3, z4, 3)) diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c new file mode 100644 index 00000000000..222832adff3 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c @@ -0,0 +1,113 @@ +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */ +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */ +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */ + +#include "test_sve_acle.h" + +#pragma GCC target "+sve-aes2+ssve-aes" + +/* +**test_pmlal_pair_u64: +** pmlal {z0.q - z1.q}, z2.d, z2.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_u64, svuint64x2_t, svuint64_t, + t = svpmlal_pair_u64_x2(t, v, v), + t = svpmlal_pair(t, v, v)) + +/* +**test_pmlal_pair_n_u64_regs: +** pmlal {z0.q - z1.q}, z4.d, z5.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs, svuint64x2_t, svuint64_t, + t = svpmlal_pair_u64_x2(z0, z4, z5), + t = svpmlal_pair(z0, z4, z5)) + +/* +**test_pmlal_pair_n_u64_regs_imm: +** movi d[0-9]{1,2}, #0 +** pmlal {z0.q - z1.q}, z4.d, z[0-9]{1,2}\.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_imm, svuint64x2_t, svuint64_t, + t = svpmlal_pair_n_u64_x2(z0, z4, 0x0), + t = svpmlal_pair(z0, z4, 0x0)) + +/* +**test_pmlal_pair_n_u64_regs_x0: +** mov z([0-9]{1,2})\.d, x0 +** pmlal {z0.q - z1.q}, z4.d, z\1.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_x0, svuint64x2_t, svuint64_t, + t = svpmlal_pair_n_u64_x2(z0, z4, x0), + t = svpmlal_pair(z0, z4, x0)) + +/* +**test_pmlal_pair_n_u64_regs_imm_1: +** mov z[0-9]{1,2}\.d, #65535 +** pmlal {z0.q - z1.q}, z4.d, z[0-9]{1,2}\.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_imm_1, svuint64x2_t, svuint64_t, + t = svpmlal_pair_n_u64_x2(z0, z4, 0xFFFF), + t = svpmlal_pair(z0, z4, 0xFFFF)) + +/* +**test_pmlal_pair_n_u64_regs_x0_1: +** mov z([0-9]{1,2})\.d, x0 +** pmlal {z0.q - z1.q}, z4.d, z\1.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_x0_1, svuint64x2_t, svuint64_t, + t = svpmlal_pair_n_u64_x2(z0, z4, x0), + t = svpmlal_pair(z0, z4, x0)) + +/* +**test_pmlal_pair_n_u64_regs_mov: +** movi d30, #0 +** mov z0.d, z3.d +** mov z1.d, z4.d +** pmlal {z0.q - z1.q}, z4.d, z30.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_mov, svuint64x2_t, svuint64_t, + t = svpmlal_pair_n_u64_x2(z3, z4, 0x0), + t = svpmlal_pair(z3, z4, 0x0)) + +/* +**test_pmlal_pair_n_u64_regs_mov_x0: +** mov z30.d, x0 +** mov z0.d, z3.d +** mov z1.d, z4.d +** pmlal {z0.q - z1.q}, z4.d, z30.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_mov_x0, svuint64x2_t, svuint64_t, + t = svpmlal_pair_n_u64_x2(z3, z4, x0), + t = svpmlal_pair(z3, z4, x0)) + +/* +**test_pmlal_pair_n_u64_regs_mov_1: +** mov z30.d, #65535 +** mov z0.d, z3.d +** mov z1.d, z4.d +** pmlal {z0.q - z1.q}, z4.d, z30.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_mov_1, svuint64x2_t, svuint64_t, + t = svpmlal_pair_n_u64_x2(z3, z4, 0xFFFF), + t = svpmlal_pair(z3, z4, 0xFFFF)) + +/* +**test_pmlal_pair_n_u64_regs_mov_x0_1: +** mov z30.d, x0 +** mov z0.d, z3.d +** mov z1.d, z4.d +** pmlal {z0.q - z1.q}, z4.d, z30.d +** ret +*/ +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_mov_x0_1, svuint64x2_t, svuint64_t, + t = svpmlal_pair_n_u64_x2(z3, z4, x0), + t = svpmlal_pair(z3, z4, x0)) diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c new file mode 100644 index 00000000000..93420ac16c1 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c @@ -0,0 +1,65 @@ +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */ +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */ +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */ + +#include "test_sve_acle.h" + +#pragma GCC target "+sve-aes2+ssve-aes" + +/* +**test_pmull_pair_u64: +** pmull {z0.q - z1.q}, z2.d, z2.d +** ret +*/ +TEST_XN_INDEXED(test_pmull_pair_u64, svuint64x2_t, svuint64_t, + t = svpmull_pair_u64_x2(v, v), + t = svpmull_pair(v, v)) + +/* +**test_pmull_pair_n_u64: +** movi d([0-9]{1,2}), #0 +** pmull {z0.q - z1.q}, z2.d, z\1.d +** ret +*/ +TEST_XN_INDEXED(test_pmull_pair_n_u64, svuint64x2_t, svuint64_t, + t = svpmull_pair_n_u64_x2(v, 0x0), + t = svpmull_pair(v, 0x0)) + +/* +**test_pmull_pair_n_u64_x0: +** mov z([0-9]{1,2})\.d, x0 +** pmull {z0.q - z1.q}, z2.d, z\1.d +** ret +*/ +TEST_XN_INDEXED(test_pmull_pair_n_u64_x0, svuint64x2_t, svuint64_t, + t = svpmull_pair_n_u64_x2(v, x0), + t = svpmull_pair(v, x0)) + +/* +**test_pmull_pair_u64_regs: +** pmull {z0.q - z1.q}, z4.d, z5.d +** ret +*/ +TEST_XN_INDEXED(test_pmull_pair_u64_regs, svuint64x2_t, svuint64_t, + t = svpmull_pair_u64_x2(z4, z5), + t = svpmull_pair(z4, z5)) + +/* +**test_pmull_pair_n_u64_regs: +** movi d([0-9]{1,2}), #0 +** pmull {z0.q - z1.q}, z4.d, z\1.d +** ret +*/ +TEST_XN_INDEXED(test_pmull_pair_n_u64_regs, svuint64x2_t, svuint64_t, + t = svpmull_pair_n_u64_x2(z4, 0x0), + t = svpmull_pair(z4, 0x0)) + +/* +**test_pmull_pair_n_u64_regs_x0: +** mov z([0-9]{1,2})\.d, x0 +** pmull {z0.q - z1.q}, z4.d, z\1.d +** ret +*/ +TEST_XN_INDEXED(test_pmull_pair_n_u64_regs_x0, svuint64x2_t, svuint64_t, + t = svpmull_pair_n_u64_x2(z4, x0), + t = svpmull_pair(z4, x0)) diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp index 42cbb5ce3df..de43c767801 100644 --- a/gcc/testsuite/lib/target-supports.exp +++ b/gcc/testsuite/lib/target-supports.exp @@ -13045,8 +13045,8 @@ proc check_effective_target_aarch64_gas_has_dtprel_reloc { } { set exts { "bf16" "cmpbr" "crc" "crypto" "dotprod" "f32mm" "f64mm" "faminmax" "fp" "fp8" "fp8dot2" "fp8dot4" "fp8fma" "i8mm" "ls64" "lse" "lut" - "sb" "simd" "sve-b16b16" "sve" "sve2" "sve-sm4" "sve-aes" "sve-bitperm" - "sve-sha3" "f8f16mm" "f8f32mm" "sve-f16f32mm" + "sb" "simd" "sve-b16b16" "sve" "sve2" "sve-sm4" "sve-aes" "sve-aes2" + "sve-bitperm" "sve-sha3" "f8f16mm" "f8f32mm" "sve-f16f32mm" "sme-f8f16" "sme-f8f32" "sme-b16b16" "sme-f16f16" "sme-i16i64" "sme" "sme2" "sme2p1" "sme2p2" "sme2p3"