From patchwork Thu Jul 16 14:25:19 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 139314 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B7BC14BA2E22 for ; Thu, 16 Jul 2026 14:28:52 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B7BC14BA2E22 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=AW9EoX2C; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=AW9EoX2C X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazlp170130007.outbound.protection.outlook.com [IPv6:2a01:111:f403:c20a::7]) by sourceware.org (Postfix) with ESMTPS id BE4FD4BA2E24; Thu, 16 Jul 2026 14:26:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BE4FD4BA2E24 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BE4FD4BA2E24 Authentication-Results: sourceware.org; arc=pass smtp.remote-ip=2a01:111:f403:c20a::7 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1784211992; cv=pass; b=wXAZojoGEYokhLW0LU//AzDCeWGcLsjKtjsai8NfNHgqeaWbp+94crACNPrVzlzTWAftrjL9YdziQHpaoephKpLN+xuapHLbDNVCJZtNWsLC7oxYS0dFboV1LGGery7DD8GjXQu7lhr6nGEbEj5hXQQq7of2s1kvjZuDBNy9fyc= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1784211992; c=relaxed/simple; bh=E7JP4zNH+CtM97PXB8OwA17qbTnmFB5z7tg+5gG9cGE=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=j2kl9Gc2x6pqeEiJeuOqtiYBhEIHlL1Swu3q4oQGSmMLhGigNfol0hM5dJb1O5wcal1IzrAjJQlef0S09KnoO22JOfIKAUZZC0225bLXcT5avp/QW6JSm+HO5FDgjd6R6LYlOkA+NaR2Hnn8VBSJWzJNMgDEkVPtYyuAJ9/cUNQ= ARC-Authentication-Results: i=3; sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=AW9EoX2C; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=AW9EoX2C DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org BE4FD4BA2E24 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=wd2GFrqb4u0MSPYyvWAGMFm0Hx+lg9wDyRrfMzhtw5DRBH7W/nx930fus0cDITWU8lrKynFa164f4Zt11zDOfAXEIjvzoQtJqXJJouQsgDPXWgKBcdHaxqwR+jFf7CtP+0Er8+ZQ2TSvei5pyn5H3ksOirPW8BdN96rpWaX7KmSzQSOgSJYg3JlJe8++iPpYJXJ8HR0i3ZQYt9D75kAt8sbYzC/DSFoMGSqCzPigTJ0ly6xsG4h+6FzAl0UNBnM6LsoLIGwSyZTvywHyygUVhrcPz5v2KxMSvw0u1ry4omDqJz7NbewSwmyuDbQLy+3E525rQyPYxqvXVmflhow3xA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Q3wcwWwf8uCgJ4cnp++1hpWtiPRfgIBKSSPzuTgh+zY=; b=DQrPux3b2S4/HGptX3Rk4l2MyPOC2WK5sQ+KMQlLul/ZHuYP9JAG6BMb+ChBp2qa3yKZ58na7FCZfG9lAtKEE2uBvJHg40hD6zvOzus1z42uwfxxjxCcB5kl/J09HvTip3++iwEvmzLHI4eRPPOwRh+0c93sWEkygYumNBzKva/Mzpo4ehTV61lR7pyTIU0wQT+Gjj87VtFe2gjTYZvlOc8BHi36AVf5NJCj0KvYitokYLiB9KtH8Nz1zyQ6Qdc0xrSe7ojERK+LTY2olIkN23rRFj2YTZWe6eLT1fSKCUFU9eQ94NX63Ug3psuzfuKm9eeGpuBgshMtjv7hp/2BzA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Q3wcwWwf8uCgJ4cnp++1hpWtiPRfgIBKSSPzuTgh+zY=; b=AW9EoX2CNPmDzwxDGqKMitijB/yIt4aAQcvghFZvRY8JlQjzgmb+Fo2wQ2K/zx9craM0MyaYgrmwhRHrulIVWpSAINSNa/ZvP3ndz3OS6f1FiOHBz3ZKf6AYZMqN6vpoQQRgGTT+hfP09CosDYLZVyUZFvTNU8QVdNd2wRv3bCw= Received: from AM8P189CA0025.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:218::30) by DU0PR08MB7591.eurprd08.prod.outlook.com (2603:10a6:10:31d::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.223.12; Thu, 16 Jul 2026 14:26:25 +0000 Received: from AMS0EPF000001A8.eurprd05.prod.outlook.com (2603:10a6:20b:218:cafe::6a) by AM8P189CA0025.outlook.office365.com (2603:10a6:20b:218::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.223.12 via Frontend Transport; Thu, 16 Jul 2026 14:26:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AMS0EPF000001A8.mail.protection.outlook.com (10.167.16.148) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.245.5 via Frontend Transport; Thu, 16 Jul 2026 14:26:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XLszvJhDdTHXIFpL6NbTfwahyjT5JAlLALZgu9ntagZc0gzw8ZPe8LaYLrOL+C0fwkKNaOIESD11ksssLOtteT4pzpXdudofS5ctlhBl+IqbJ5cqE2X4xMpDNpodXsX3OMdHv7/qrwaKE9xXiEki57aCzJ/IOqBKoGxqBtQM9FED5n/RjKaseUq75UyjI24o/pL+FEkg1aBb5Xh52p7kTLVSV6xJwG/tAOtaNYtMa30vNj9y6teJvDjxNfqSLar7uFztivekWejBG/z2QT42TX6azfuPRsSBJKsojClhQSpOQ2ITULywecLVY2LuZDYjEL6+9feRsXo5TEg/JhScjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Q3wcwWwf8uCgJ4cnp++1hpWtiPRfgIBKSSPzuTgh+zY=; b=IJhVe0VEo/+hzra+096Ta+uvI+ziU1VuwKdS5wqRNI6kNQ9a5iKH9kUbgeo+cOO1BgheJ/s5SnfdIl/fWH1/BMDp52oQDfFzJSkSVJ7Iu16n16arl/vX8azYy3m5z0QDvzw2yN/cpucLjxsS4/ssN7ry9rRzrORiS4y4PpJ2CKjzhnILLj0fCBcSUCYWZA4qT/09I4vPHVkzlWkxtrJg8uUF8klPmAaErWKEMXmh2H7ft6sRphYGTg5ixKh54g+4MBScbvVom8ApbFM1jFLm0MBYy0BsCb/RRN5Ri5RBtNd4bb7Z98555wD8yEFTkia6fu/bqank1ex2MgePXo+SOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Q3wcwWwf8uCgJ4cnp++1hpWtiPRfgIBKSSPzuTgh+zY=; b=AW9EoX2CNPmDzwxDGqKMitijB/yIt4aAQcvghFZvRY8JlQjzgmb+Fo2wQ2K/zx9craM0MyaYgrmwhRHrulIVWpSAINSNa/ZvP3ndz3OS6f1FiOHBz3ZKf6AYZMqN6vpoQQRgGTT+hfP09CosDYLZVyUZFvTNU8QVdNd2wRv3bCw= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI0PR08MB10392.eurprd08.prod.outlook.com (2603:10a6:800:210::10) by DB4PR08MB9264.eurprd08.prod.outlook.com (2603:10a6:10:3f7::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.223.12; Thu, 16 Jul 2026 14:25:22 +0000 Received: from VI0PR08MB10392.eurprd08.prod.outlook.com ([fe80::cd7c:e19a:2072:75cc]) by VI0PR08MB10392.eurprd08.prod.outlook.com ([fe80::cd7c:e19a:2072:75cc%4]) with mapi id 15.21.0202.014; Thu, 16 Jul 2026 14:25:22 +0000 Date: Thu, 16 Jul 2026 15:25:19 +0100 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, Richard.Earnshaw@arm.com, ktkachov@gcc.gnu.org, alex.coplan@arm.com, andrew.pinski@oss.qualcomm.com, wilco.dijkstra@arm.com, alice.carlotti@arm.com Subject: [patch v3 2/3]AArch64: Move SVE ptrue VL folding helpers Message-ID: Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: LO4P123CA0033.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:151::20) To VI0PR08MB10392.eurprd08.prod.outlook.com (2603:10a6:800:210::10) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI0PR08MB10392:EE_|DB4PR08MB9264:EE_|AMS0EPF000001A8:EE_|DU0PR08MB7591:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f939673-0c62-48cd-b227-08dee34636f3 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|376014|1800799024|23010399003|366016|6049299003|18002099003|22082099003|4053099003|6133799003|3023799007|11063799006|10067099003|56012099006; X-Microsoft-Antispam-Message-Info-Original: AViudxMPbITEz6iBQ2foPYbEzuFnwzTCxTWTTcq7ENcGensdcfTrQXLgoDzZjPIY/l13EAPE4AxGr4VNKxIv6TCMoGvEHTvPGitEoQE5i/D+4TfsU26NBUwm313xtQj68VDS1BRiC3SbJuPzW+TPJZ+CkkEOIKsoQZXBFFqSajwQWaGgymTumU4FjD/mlAPQtXtj2JGuvySgu2iEUfwMQvWx3JMFmyxcYPAOYtT1PVItC4wvSF5q2nRQiC6w0o3eW6Aar4fyOkFt9k1LnuYghC7lNLvjjFH46peVdp0ZRT1eF2mM2CKb2rh7Gx1h8g1fimr1tuyP5fQN3I5KYy3WHGLCOW8HhGrg64qCBtwEZZob/iDeS3I+kG7abzX2x+4sbeZP08E3uu5tA9vHUSIEfWLAl9vgc3akiqibgaZXJC1LDK9OezIpj/EvyLUL082CKHLDgXGUKOAkcSPMcKXqxZ5hWMSFRpzSf3B3nz5A5jBgolmJukULHT0r7HM/aB7fBRmWufRZRBziA/fZ+eOj6eqW19c0JrO0lSUJ/09LjQYETTZokLoQcEFo6g8KaDO+odIe1Au7ymvWmaYn9jNkyPngPsJgjvMph3SsLmYBQBoh+kBYlrjEJAAGtwZxjj61EUuZoDnQUHDoSPYXAhvbYYDCdRTYH6yt/vBeMgNwsps= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI0PR08MB10392.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(6049299003)(18002099003)(22082099003)(4053099003)(6133799003)(3023799007)(11063799006)(10067099003)(56012099006); DIR:OUT; SFP:1101; X-Exchange-RoutingPolicyChecked: CGMp55DUPHAT3OBCF5qrTv7qf/gaL9DEo1aHn+o0ki27J2JjHWVcPBAtSacf4KYjwn2x8vNTjJTPPjcnjKSEPATsTc4eS2gop7yjtsRnfMu6ZArURKiiLpG9YwqZSeYOz51vp5mAICGc2qCz35qKjTnhSdMDjJJrz5Xbf7wt3COGxj/VJwXBvXuR7E3Oz4RdAZJO1c6b2RL5hoO0Mt13fook6kVoFZnuc1VbdcL6XPeVLX3S85C8g9wes0g32m2AgN6HObIop8aa/yqBPHMS6vvw86LNKKnxdxfzM/siN3T/M4zEeqA83ORdpDVDfIydRZdXsIg+I3uplbln8jMxiw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB9264 X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS0EPF000001A8.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 9057496d-eca3-4cdf-fe86-08dee34611c2 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700016|35042699022|376014|14060799003|82310400026|6049299003|1800799024|23010399003|3023799007|56012099006|11063799006|6133799003|10067099003|18002099003|22082099003|4053099003; X-Microsoft-Antispam-Message-Info: ocAOY/5GgPsol9KtDfK9okm2651WBOMsJuPvacn/i0Uc+5MvBTF3eFvAhiZW/KqUpLyiGOfRVI3Cv/+/nGyI4jnFsh3puv6cEqOmmGSQZjH3gWBb8vAqNHQ+IAbnojR4B9R/qtdj65wKVSGJSIeNrtTY3kF6vpEeZCQ/vkXdKcm+x4FPIJ6vR2B+i66xSLzjM/1q4i4zjpokS/aU5QE5iVKVf/fsMI12o3Yy1u6qSciOjxVPk1GBU+CPlUA2YzgaF7nHWb7bCLNgz+GYC+RAH4waHQK5fuV2Lhb5t+eoe/yGQ9uGG3MBOhQCP+XnbRG9WffQqcjc+9ioFLYQdo2fBriihPn2Uccn+kvAG3RdL6vOc8+Lqscdn9oRO1IojOz1rOmo/Io17MFudL75wlua3cKBvPTK/DW+KzK5eHucINkwxlWwfEuRK8pTEk+zmBcc1cgYlcaLvkHdjjKpQ+8iti8niOq6EJoqffpSLNLOnLQK+wltYBKpY4PmnuPS+1MfWknESJj+MMho0OW9vyUg7AFKXfWDX6C/pCPTeD6YiEhz23V9pRMtTmf2oHJ/4VjZNUOxlZXAlh7TQBFk835hhwImIZGaa9hCCgw72kiT38+/BNniZ1uDQqFehveHPLzqHiBO77XyZ7Br5Phr8N4QrOclB1RfLsBwquG0ykMBAKDryPlz0eW+15PIsqy2VpfwJiQJMYWz4o9XrDc2Hq5+Xw== X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(36860700016)(35042699022)(376014)(14060799003)(82310400026)(6049299003)(1800799024)(23010399003)(3023799007)(56012099006)(11063799006)(6133799003)(10067099003)(18002099003)(22082099003)(4053099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ZClStvVLDGaIOIBILMyttuGRixaeyxZ8xVrQaLT7M16UqWeKU7OCuTINPDhZS4LV6nTbcAtfeTP3dgYR0bMuWPkKmNlRRp1IrzQDc0vZNZoe8VOzpXuT9UqO0cE9TzNUyO3TLc2VyKFfrbP0Ig7JlLFbPcEZhXnSokURFPmBd5x8ppyLSHWWrTqJzNotpJgAAKuOEIYyzNl8CjgkGZeSb2D0XzGPRk/SaLbbM3QhbERFbQxLhDQbIqElUArfGpxGSqjt02/se51FACuFLbq68+7xNPudKIP9UtJ51sujc8eS1+Jah8aba/v2iZ2DWTFrNEP7CtBG+VDIWeh965yBEEw1C0nDKeyTR7KPpR+BZ98SMBZSNkAudFQZ32QOmeQcpsZmg3n6FLlMjf1IiE5dGjV+6EGaO9ZArE2rswfY9FNCPGcTYO3W+CM0dvwNZyvu X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jul 2026 14:26:24.1361 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2f939673-0c62-48cd-b227-08dee34636f3 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001A8.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB7591 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_NONE, TXREP shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org The 3rd patch in this series will modify these functions, but I have to re-order and extract some shared functionality to avoid duplicate and having to forward declare stuff. I thus separated out the refactoring bit from the parts that use and change these functions. This patch is NFC. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Pushed. Thanks, Tamar gcc/ChangeLog: * config/aarch64/aarch64-acle-builtins.cc (gimple_folder::fold_to_vl_pred): Refactor folding logic into aarch64_fold_sve_ptrue_vl. * config/aarch64/aarch64-protos.h (aarch64_fold_sve_ptrue_vl): New. * config/aarch64/aarch64.cc (aarch64_instruction_selection): Move down. (aarch64_fold_sve_ptrue_vl): New. --- -- diff --git a/gcc/config/aarch64/aarch64-acle-builtins.cc b/gcc/config/aarch64/aarch64-acle-builtins.cc index 426316001ced864081e798b50c7dcb3281a77834..e64c6f00f91717151be0f4d092ca4d5e1123f061 100644 --- a/gcc/config/aarch64/aarch64-acle-builtins.cc +++ b/gcc/config/aarch64/aarch64-acle-builtins.cc @@ -2983,22 +2983,9 @@ gimple * gimple_folder::fold_to_vl_pred (unsigned int vl) { tree vectype = TREE_TYPE (lhs); - tree element_type = TREE_TYPE (vectype); - tree minus_one = build_all_ones_cst (element_type); - tree zero = build_zero_cst (element_type); unsigned int element_bytes = type_suffix (0).element_bytes; - - /* Construct COUNT elements that contain the ptrue followed by - a repeating sequence of COUNT elements. */ - unsigned int count = constant_lower_bound (TYPE_VECTOR_SUBPARTS (vectype)); - gcc_assert (vl * element_bytes <= count); - tree_vector_builder builder (vectype, count, 2); - for (unsigned int i = 0; i < count * 2; ++i) - { - bool bit = (i & (element_bytes - 1)) == 0 && i < vl * element_bytes; - builder.quick_push (bit ? minus_one : zero); - } - return gimple_build_assign (lhs, builder.build ()); + tree pred = aarch64_fold_sve_ptrue_vl (vectype, vl, element_bytes); + return gimple_build_assign (lhs, pred); } /* Try to fold the call to a constant, given that, for integers, the call diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h index 5580f3148d6db754adad6804b506f5a0fe3ecf66..f9943f506c12aec81b3119ceac5d7cafd223dfea 100644 --- a/gcc/config/aarch64/aarch64-protos.h +++ b/gcc/config/aarch64/aarch64-protos.h @@ -1035,6 +1035,7 @@ unsigned aarch64_debugger_regno (unsigned); unsigned aarch64_trampoline_size (void); void aarch64_asm_output_labelref (FILE *, const char *); void aarch64_cpu_cpp_builtins (cpp_reader *); +tree aarch64_fold_sve_ptrue_vl (tree, unsigned int, unsigned int); const char * aarch64_gen_far_branch (rtx *, int, const char *, const char *); const char * aarch64_output_probe_stack_range (rtx, rtx); const char * aarch64_output_probe_sve_stack_clash (rtx, rtx, rtx, rtx); diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc index 72822ca621f52afee144bffa76bc75aac28adae1..bcd143ac06cc1b3674d05dd087874d9b435cae06 100644 --- a/gcc/config/aarch64/aarch64.cc +++ b/gcc/config/aarch64/aarch64.cc @@ -63,6 +63,7 @@ #include "gimple-iterator.h" #include "gimple-fold.h" #include "tree-vectorizer.h" +#include "tree-vector-builder.h" #include "aarch64-cost-tables.h" #include "dumpfile.h" #include "builtins.h" @@ -2311,72 +2312,6 @@ aarch64_try_widen_mult_by_pow2 (const gassign *convert, return true; } -/* Implement TARGET_INSTRUCTION_SELECTION. The target hook is used to - change generic sequences to a form AArch64 has an easier time expanding - instructions for. It's not supposed to be used for generic rewriting that - all targets would benefit from. */ - -static bool -aarch64_instruction_selection (function * /* fun */, gimple_stmt_iterator *gsi) -{ - auto stmt = gsi_stmt (*gsi); - gassign *assign = dyn_cast (stmt); - - if (!assign) - return false; - - if (CONVERT_EXPR_CODE_P (gimple_assign_rhs_code (assign)) - && aarch64_try_widen_mult_by_pow2 (assign, gsi)) - return true; - - /* Convert - p == q ? s1 : s2; - to - p != q ? s2 : s1; - where p and q are svbool_t expr. Due to the absence of predicate - comparison instructions, we use bitwise xor for checking inequality. - Transforming == to != avoids an extra bitwise inversion to the xor. */ - if (gimple_assign_rhs_code (assign) != VEC_COND_EXPR) - return false; - - tree lhs = gimple_assign_lhs (assign); - tree rhs1 = gimple_assign_rhs1 (assign); - tree rhs2 = gimple_assign_rhs2 (assign); - tree rhs3 = gimple_assign_rhs3 (assign); - - if (TREE_CODE (rhs1) != SSA_NAME || !VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (rhs1))) - return false; - - gassign *da = dyn_cast (SSA_NAME_DEF_STMT (rhs1)); - - if (!da) - return false; - - if (gimple_assign_rhs_code (da) != EQ_EXPR) - return false; - - tree eqa = gimple_assign_rhs1 (da); - tree eqb = gimple_assign_rhs2 (da); - - if (!VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (eqa)) - || !VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (eqb))) - return false; - - tree ne_expr_var = create_tmp_var (TREE_TYPE (rhs1)); - gimple *ne_stmt = gimple_build_assign (ne_expr_var, NE_EXPR, eqa, eqb); - gsi_safe_insert_before (gsi, ne_stmt); - - gimple *g = gimple_build_call_internal (IFN_VCOND_MASK, 3, - ne_expr_var, rhs3, rhs2); - if (!g) - return false; - - gimple_set_lhs (g, lhs); - gsi_replace (gsi, g, false); - - return true; -} - /* Implement TARGET_MAX_NOCE_IFCVT_SEQ_COST. If an explicit max was set then honor it, otherwise apply a tuning specific scale to branch costs. */ @@ -4482,6 +4417,97 @@ aarch64_fold_sve_cnt_pat (aarch64_svpattern pattern, unsigned int nelts_per_vq) return -1; } +/* Build a predicate of type VECTYPE in which the first VL elements of size + ELEMENT_BYTES are set and the rest are clear. */ + +tree +aarch64_fold_sve_ptrue_vl (tree vectype, unsigned int vl, + unsigned int element_bytes) +{ + tree element_type = TREE_TYPE (vectype); + tree minus_one = build_all_ones_cst (element_type); + tree zero = build_zero_cst (element_type); + + /* Construct COUNT elements that contain the ptrue followed by + a repeating sequence of COUNT elements. */ + unsigned int count = constant_lower_bound (TYPE_VECTOR_SUBPARTS (vectype)); + count = MAX (count, vl * element_bytes); + count = pow2p_hwi (count) ? count : 1U << ceil_log2 (count); + tree_vector_builder builder (vectype, count, 2); + for (unsigned int i = 0; i < count * 2; ++i) + { + bool bit = (i & (element_bytes - 1)) == 0 && i < vl * element_bytes; + builder.quick_push (bit ? minus_one : zero); + } + return builder.build (); +} + +/* Implement TARGET_INSTRUCTION_SELECTION. The target hook is used to + change generic sequences to a form AArch64 has an easier time expanding + instructions for. It's not supposed to be used for generic rewriting that + all targets would benefit from. */ + +static bool +aarch64_instruction_selection (function * /* fun */, gimple_stmt_iterator *gsi) +{ + auto stmt = gsi_stmt (*gsi); + gassign *assign = dyn_cast (stmt); + + if (!assign) + return false; + + if (CONVERT_EXPR_CODE_P (gimple_assign_rhs_code (assign)) + && aarch64_try_widen_mult_by_pow2 (assign, gsi)) + return true; + + /* Convert + p == q ? s1 : s2; + to + p != q ? s2 : s1; + where p and q are svbool_t expr. Due to the absence of predicate + comparison instructions, we use bitwise xor for checking inequality. + Transforming == to != avoids an extra bitwise inversion to the xor. */ + if (gimple_assign_rhs_code (assign) != VEC_COND_EXPR) + return false; + + tree lhs = gimple_assign_lhs (assign); + tree rhs1 = gimple_assign_rhs1 (assign); + tree rhs2 = gimple_assign_rhs2 (assign); + tree rhs3 = gimple_assign_rhs3 (assign); + + if (TREE_CODE (rhs1) != SSA_NAME || !VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (rhs1))) + return false; + + gassign *da = dyn_cast (SSA_NAME_DEF_STMT (rhs1)); + + if (!da) + return false; + + if (gimple_assign_rhs_code (da) != EQ_EXPR) + return false; + + tree eqa = gimple_assign_rhs1 (da); + tree eqb = gimple_assign_rhs2 (da); + + if (!VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (eqa)) + || !VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (eqb))) + return false; + + tree ne_expr_var = create_tmp_var (TREE_TYPE (rhs1)); + gimple *ne_stmt = gimple_build_assign (ne_expr_var, NE_EXPR, eqa, eqb); + gsi_safe_insert_before (gsi, ne_stmt); + + gimple *g = gimple_build_call_internal (IFN_VCOND_MASK, 3, + ne_expr_var, rhs3, rhs2); + if (!g) + return false; + + gimple_set_lhs (g, lhs); + gsi_replace (gsi, g, false); + + return true; +} + /* Return true if a single CNT[BHWD] instruction can multiply FACTOR by the number of 128-bit quadwords in an SVE vector. */