From patchwork Sat Dec 6 16:55:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126047 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 133E54209E1A for ; Sat, 6 Dec 2025 16:58:23 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 133E54209E1A Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=EKYkEOqr; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=EKYkEOqr X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from DB3PR0202CU003.outbound.protection.outlook.com (mail-northeuropeazon11010019.outbound.protection.outlook.com [52.101.84.19]) by sourceware.org (Postfix) with ESMTPS id EC2884360BB4 for ; Sat, 6 Dec 2025 16:56:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org EC2884360BB4 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org EC2884360BB4 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.84.19 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040196; cv=pass; b=uwtB8txue/WYyUE3adkPrNjvmWzLkDzCaJj5SwxQBkonW1okTc8CPcP4Di9nVDseXUnRH7wUd0uIC0I2rQSpcA2Gn1N/FVmVPGvywa19aLzbLed/6J0+/CTnS1xAObbQVCa+eRxmbVu141Ma5Bnrt8+0aozM4GU1yusRFSAHDTk= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040196; c=relaxed/simple; bh=FGFW8ZvriWvh9hDxa+7X3MsGMwMeGmC8bdoErM/VUKM=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=xfIOdC48dVbAnFjMnzDzLeDUy/RlXOo5GkL2lwkSMYTyKXxQF69TtsPbBkWJFPol0zAm0MH0PhzTDWBomsRCsJiJQQyrboycR1wlD7/MV+2KUyAXnF0IH7gbio0AA79+OMNkTKkAZB+U8+Z9B2y/Pnt9WPZbVByJ8+RAZ2qODxY= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EC2884360BB4 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=KEpKKTeUDeZhC2ggcRWz0LMZM9bIW4V34wsKfDf3EvcLoq7EwmEtgbIS4wGjB/evWZj9OAB9hSNZK+2JUAamR/PO5eZK1aG2OO7HRUuiepiJy7ZZq6i0PaeOLW73JabBzAX1b6lidGhkiJ4k/Qw94i7HRarxnsGsdRQM74jIJJFEPPFctusPOXM2DYbuP99/UXOqAdeemk19VtTLcyJRndZ99UOCBx02QrzNmPgzZuQYBgjOph6kwZ1dA2WTNkx/56Gb++moYKMr3P0OPGUggMikzvrMer1SvF9uXKvZm6wn9lrj//eaw5yXnhvd3Gcggv2Rv7UmTZU3xgxbjHl0mQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kWHMKDQumqKG6FycW0abDD02V021S+mHIXyhEqjbme4=; b=Cu600TCZXsj1zM1p9uXDBP0dAhP/5pdIMFcg/EqEgjJ9yY9LFMPi/lmMBTVL/8EcR9qHJrPBCHv4taBd/UCZAhdl1Pi9Y+WtzUvcwWiJnRpeNG0yYA734BpI8heXMI/EPtnG7THt+5f8HxT5DQqtfuyypV+bdXMylttDDxOAoyPEPI/LivN+Go/f9GyQR3xWpzhPfgp+uWUucYqkmjnqx0H7VcLZJsOUuVijil5Pae+axLqW7LpiX7+7oHRcbX1gnnTKK4nkTeFmejkaDj9rAWj/luQeAwv3W0rYJ+6wxrV6vst/AnKfykgz4KU52Ru8M7hIKSQcWhWftwKhCNyyXA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kWHMKDQumqKG6FycW0abDD02V021S+mHIXyhEqjbme4=; b=EKYkEOqrDS8Gp2L462yZ5J6s+4HLjMALeJnGDDPyCprUaC09hOL2rqgsaiEzcQRH21QmuinNsRWYXuamIGtIJaYvlnqx9WW42YE0EaP3Is6IskpFoI25/UF5kERFjgfOIWCJBYeQ+5qxJ49HhjXqH5RPQ4UjsARAswEg1IOfhck= Received: from AS4P251CA0022.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:5d3::14) by AS4PR08MB8045.eurprd08.prod.outlook.com (2603:10a6:20b:585::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:56:30 +0000 Received: from AMS0EPF000001AE.eurprd05.prod.outlook.com (2603:10a6:20b:5d3:cafe::ce) by AS4P251CA0022.outlook.office365.com (2603:10a6:20b:5d3::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AMS0EPF000001AE.mail.protection.outlook.com (10.167.16.154) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:56:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ml5mmQ4GXIoWO1DnvqsbsY4N63UH92+SKDmX8KylwFasQkOgxfheZGuK+9oeiPRv7T79tR78+spWO0gwFQUs/ArJL0SsRgIRA/wgbErcfbcE3NYp5WRTbxphrmZiHL9bHKa962fajftU5LCGmb6WV99R7dLknqnHgI8eJ0Sg4tsvdLW72HzY01bDveluQktXfoNZfLH6bmmjyHXkIYj7oEnxB/Gk7ZNi4KJqiDmSObuqJLdnYx4WpABq3kAxBOR2Bact1urwiUP+ZN99dB/WGJz+HTh8us72zR0cx8Z+iJzqrJbURas3fRqMzy4c1vGs71SSw3k2/yZmApVctZTgwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kWHMKDQumqKG6FycW0abDD02V021S+mHIXyhEqjbme4=; b=NH0Rq0FbxsZijo8m25bYAlliI7q0gtyKTz/sqABdj6uwh1LC78GzNw13tasYmeLCqTEZIHUtm+dKI//yvknJsMuB8ENuog5IBzyDKkpx1E4JKz+yMcrx/stO1eexSs0o+HIgveEam1Uff2KLQ/y428zJmaKXo7ceukXhA+H7902Baj/k1cUTOexBL42w2jVLCIN7tppHYd74HolNsrrx4F1oclBqhmeU0GOoRDqPvhFKiEuwx/fBFmjdmLq/4M9OAgPIHpAKrVydkC2Ev0eBabfI/uTcI3Mtb2jw4A6Vue3Dz9V+Zc8GYFdeVhO5uUK1mvHeGz3fbXn0U3t5R7lleA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kWHMKDQumqKG6FycW0abDD02V021S+mHIXyhEqjbme4=; b=EKYkEOqrDS8Gp2L462yZ5J6s+4HLjMALeJnGDDPyCprUaC09hOL2rqgsaiEzcQRH21QmuinNsRWYXuamIGtIJaYvlnqx9WW42YE0EaP3Is6IskpFoI25/UF5kERFjgfOIWCJBYeQ+5qxJ49HhjXqH5RPQ4UjsARAswEg1IOfhck= Received: from DU2PR04CA0360.eurprd04.prod.outlook.com (2603:10a6:10:2b4::28) by AS8PR08MB6616.eurprd08.prod.outlook.com (2603:10a6:20b:319::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.9; Sat, 6 Dec 2025 16:55:24 +0000 Received: from DB1PEPF000509F5.eurprd02.prod.outlook.com (2603:10a6:10:2b4:cafe::8) by DU2PR04CA0360.outlook.office365.com (2603:10a6:10:2b4::28) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:55:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF000509F5.mail.protection.outlook.com (10.167.242.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:55:24 +0000 Received: from AZ-NEU-EXJ02.Arm.com (10.240.25.139) by AZ-NEU-EX04.Arm.com (10.240.25.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:24 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EXJ02.Arm.com (10.240.25.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:23 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:55:23 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 01/11] Preparation to support predicated vector tails for BB SLP Date: Sat, 6 Dec 2025 16:55:08 +0000 Message-ID: <20251206165518.5449-2-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF000509F5:EE_|AS8PR08MB6616:EE_|AMS0EPF000001AE:EE_|AS4PR08MB8045:EE_ X-MS-Office365-Filtering-Correlation-Id: 502c8f29-7098-4b11-2f6c-08de34e8673d x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info-Original: Z5kB1iK3bSmPR0V8C4QHpsVnwc1RCaXe0ZKGr/pcdSWRh/sp85S1fsUHIHOI4VWtGGsXGCea+t97Fhg+lwLcZQ9iFB7IpcmeE5gfTgH0ky2wHIDHnDDszHQlH7a9OyOkRTE/6EETDV6sl4pFO4PHwu1glCEJVp/w0soE5vEETnXBi4pJVfxOfCHUbuKaWQ9UbuBqUkJRtgIwbrEexSGnnMXjKSkIS+RfX3UoZg/zT2XmPpqyM6kVQpPcrh/LkrnNJmbiFwSLfTojVOSqjhC+l381i4K7zkrKnvXm0Pk6b2eAaC5DY7XbEECcSqnOHvbZUhCBdWaot+mvyyAey8sTofegzXhYwRLhC61+9fcnJWoCv1l4JuB9BeF4IGggQIOz1/W3jXn0ZsFx3Px+WJe4z0pnuh+RKtjITieV2uxA9mCU/jeB/Bme911Sl8Ks9Z0hUGZjeaS9byAgypSsVsaYmqkyv82dKr/y4y6q1iKl+KLocVnU8dh7KrFA1/tt/jz62ix04KR5F7+OdCX+hobZxcISgZiIpzDXR+tntm/Gl8qJ/ZqzQ3hZO8K+qKuu0A1Lvg7DoEZTUbzXCq9fECkmhvexO4bNpsoNWulax8ZQyF4LoN11NpmHCwsRZ06GH//UKppAwLTbiKHVwHkLhAHqklgu6DechYYAuLUs7n0GnvcLKcyiLX/QifOGskIVrurC+fUs2O+0dtJbRituUQlK7siTjDL7TJkmS5LTs9mq0F03Qq8Vs1VEOAP6dBj+gJ0CebO6bQIS/phKK/SjIDW+n6955MvpVrfewCuSv3uAS5wY882NszWVUktOVigAX1nojvdhdNfIiAT1EheHzBDWXIA1rbCIMfVoRcy2oQLVmKh34fhyfwAikHS2xzwLpflpo3gZx3v9ZtApzGWIOCSnYoIutCXZeJpsAL3KTSr3kCoX9654TWhZp6+Oq8Nu62KrRWYJdSovAbqMV8XS4orDQc39uO5Nu6NCRbsAYsevbxAlKgZzoeeTPk6T/ZN6q4NF9kJ4w1gsv+wZdfpaYxb/LoP+M9heHbKcKCbHxRsluw/qtnNkm74YtrgdEJyTo0cG2vZmMnCBKaJ4/sLvX+kOneaLdccTw+4/Zp10+z/Il0Sw8NCKtg4hvwV1AtxDmQ69JcS1z6eE+pn4jVpd/o8gwnZVuMEtm3e8P9Gwa2vh2+yiFGEqmy7Go2TiQg70giDFs0HcPIYCD2sW8I6gDG9DP2PT1UBmHTNBF4HedLNdhlAGWP7F7Mpr6otKGeRzXFBNBQnSU8p1o3U5Z6ZPEk2M9PtZYZ9smn8Bis/vaR1EAKyowEQbUAcP9amvE2pqVhiZ+E4AM/Ao6Hs19g/hNxCr/ceigjWaTMyJ7SPy7kA0Oqt4nZmQn/EuFKmv4owMw/AAeQSm5Iqm9pDfqoVCXynqZQ4fwn7Mi3dsBZQGwuMduLbkiGkVtcYhhYornl+/tw/u9PbJ+PxF1k7p94J/0mmcrlXZ+6nOVd1OCkDnVhRGHIaSkiSHM5H9Odz5Fb+uO5lavLFythcYSeDO8nESZp6ejxBozVH0UTLcAF+mc8lsQRE= X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6616 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS0EPF000001AE.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 955b884f-2611-403e-eaf4-08de34e84009 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|35042699022|14060799003|1800799024|36860700013|376014; X-Microsoft-Antispam-Message-Info: PGdCF23hEMK832LJIuELhGg7JGUzSV/7EeKOhls/1REhtlMUKoKF93yjnTNvxI94TKpuqSHOBYKcUYEe+PBMKgz9beGvEOP5HT1yLC0EHFFFvBnMKTk4H9oHy5qJzXL1LvdhWovURQq6xqkcFUdHOmbky4T0pmplnjxEv/VnHKnx76/VwtH58Yfwf9I6RlLOr+q+Jgpuo4iOp8z1tHXf6B+YcYzMrGjnzMr4k10Dw6Ila4oO5Gu5ZuTdFICTbKaM7A4rpiBEEM2sM28iLLv/0bje2h+0YZiomLaTY9JAkUUNLGcHbxWMl4gCScX8jPtr2jy5l1JI4RcOPrj/1CBzLWPBeeh16fAADNETCtnkau/yHo3mUAv1aV3VFbyIB37OpWIcfF1uMdwsrYnMh/wMNqLBKNgKxp/PszLsClUXXNOH9+8IRXHingxhXayExAziY1O5QgTRcbZQjveG6Li2yc4lAxDJD5JlxBZOZNJHEfLrPSFWUkm/f/9S7sSdC9eMBO1WfE1Z1bPMVIWNkcttKrYF1a2AEPAcO3S2cmzerL4IvU3+AxtNvXGylnnAPJnvIQ62l1UoEJ3aYbiB21hUEWOtImLcyNEGHdTcvMjlgpoVBsTBL5LRyn8itsEP4MR9+vehdCitdgidSon45XZo9U5WzzMnfA1AcQiWmFQ3eJzjDZ8dPLR9WoXdLb4szSxxC15BvsDKW9tsQxaT4t6mGmq3P9G5A11CFK54UoUECrkiWpjEydlYBvR8ZYuH4F+r985m5rfLD4Iubgk6qGc9dU3A2C/rA8KzrdmP8pjz8BiB9UlOVtG8k1TiQ/z1WgOi4jofudnhiG0oE+4nQfF+KgXvH2HcSxMFxQ+BkY7yEjuZvu5YMAT94c6jcD/8LIOpKFa6KiCyiFaUc/1Pmpcorvs2WMlPw4GjVCP9wI5YMrzdtKbPnqfmYKfPv2aGCuUqn6gmC5xQ7m0YRBVooiuTibO5A2nL+GLoWUXVoEMQvnUgpKLoURSl+/8fFjjoU6OFSDFUFbOnrxN4GhJmfJpnyvz0V5JFcvaGyMaIpsFHuzRMX0yTRlsalnv3tkgEhsu7TGroJHGg3rtEp6T7xrseuTF5/c+3JmjFh17Bl2mjEPh6fPjfGUmn4VQeXejQFbKleECRXMt3PoG2Oz/IVdR1MaiT5yvJKxAWPfuFN6KermEz2cxqhJ3O4H0/jx/zZbGMEeKiJIbS36O25FaMl/6TsmC3Xp4/BTONiCuqEgSNgZJvsaeP6fknz7BAzQw5wmrBlERvEvKSTol5HVFdNlLWBgOYHpa7T+Yz5Px3vRhx2E6ehuT34NfCr2cnoap/iOwflpKPjfVp8YKQMOGSbmmbOGXX1St12X911z3xk+k2qzwKSq3iB9OTPp1rzEfW3Et23Pm1X2YiI+IfbKmHJqXb2bIQifd4IHx3sEfftK69+mbvWGOyY5mFXSUqvjrB43P0K0TbEiMb//BvfqnV1vo1+rVy75RB4ubWD0lk+d6DAIwMcNLZ642SU3gXxeD+NRjZeN2b8cSopNJzUYt5yZs+GSG2zndWJzRfyb7KRujaUmY= X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(82310400026)(35042699022)(14060799003)(1800799024)(36860700013)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:56:30.1808 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 502c8f29-7098-4b11-2f6c-08de34e8673d X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001AE.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR08MB8045 X-Spam-Status: No, score=-10.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org Calls to vect_(get|record)_loop_(mask|len) are replaced with calls to new wrappers that have an extra (SLP node) parameter and which can operate on any vec_info, not just a loop_vec_info. These wrappers pass calls through to the original functions (and ignore the SLP node) when invoked with a loop_vec_info; otherwise, vect_record_(len|mask) do nothing (for now). Direct use of LOOP_VINFO_FULLY_(MASKED|WITH_LENGTH)_P, and LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P is replaced with wrappers named vect_(fully_masked|fully_with_length)_p, vect_can_use_partial_vectors_p (for queries) and vect_cannot_use_partial_vectors (for updates). For BB SLP, vect_cannot_use_partial_vectors does nothing and the getters return false (for now). To minimize code churn, the new wrappers are only used in code shared between loop vectorization and BB SLP. gcc/ChangeLog: * tree-vect-stmts.cc (vect_record_mask): New function that wraps calls to vect_record_loop_mask. (vect_get_mask): New function that wraps calls to vect_get_loop_mask. (vect_record_len): New function that wraps calls to vect_record_loop_len. (vect_get_len): New function that wraps calls to vect_get_loop_len. (check_load_store_for_partial_vectors): Substitute a parameter of type vec_info * for loop_vec_info. Use vect_record_len instead of vect_record_loop_len. Use vect_record_mask instead of vect_record_loop_mask. Use vect_cannot_use_partial_vectors instead of LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. Delete local aliases for LOOP_VINFO_MASKS and LOOP_VINFO_LENS. (prepare_vec_mask): Substitute a parameter of type vec_info * for loop_vec_info. Rename the loop_mask parameter. (vectorizable_call): Delete local aliases for LOOP_VINFO_MASKS and LOOP_VINFO_LENS. Rename masked_loop_p as masked_p and remove requirement for non-null loop_vinfo. Rename len_loop_p as len_p and remove requirement for non-null loop_vinfo. Use vect_cannot_use_partial_vectors instead of LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. Use vect_record_len instead of vect_record_loop_len. Use vect_record_mask instead of vect_record_loop_mask. Use vect_fully_masked_p instead of LOOP_VINFO_FULLY_MASKED_P. Use vect_fully_with_length_p instead of LOOP_VINFO_FULLY_WITH_LENGTH_P. Use vect_get_mask instead of vect_get_loop_mask. Use vect_get_len instead of vect_get_loop_len. Pass vec_info * instead of loop_vec_info to prepare_vec_mask. (vectorizable_simd_clone_call): Use vect_can_use_partial_vectors_p and vect_cannot_use_partial_vectors instead of LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. Use vect_record_mask instead of vect_record_loop_mask. Use vect_fully_masked_p instead of LOOP_VINFO_FULLY_MASKED_P. Use vect_get_mask instead of vect_get_loop_mask. Pass vec_info * instead of loop_vec_info to prepare_vec_mask. Delete local aliases for LOOP_VINFO_MASKS. (vectorizable_conversion): Use vect_can_use_partial_vectors_p and vect_cannot_use_partial_vectors instead of LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. (vectorizable_operation): Delete local alias for LOOP_VINFO_LENS. Rename masked_loop_p as masked_p and remove requirement for non-null loop_vinfo. Rename len_loop_p as len_p and remove requirement for non-null loop_vinfo. Use vect_can_use_partial_vectors_p and vect_cannot_use_partial_vectors instead of LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. Use vect_record_len instead of vect_record_loop_len. Use vect_record_mask instead of vect_record_loop_mask. Use vect_fully_masked_p instead of LOOP_VINFO_FULLY_MASKED_P. Use vect_fully_with_length_p instead of LOOP_VINFO_FULLY_WITH_LENGTH_P. Use vect_get_mask instead of vect_get_loop_mask. Use vect_get_len instead of vect_get_loop_len. (vectorizable_store): Use vect_can_use_partial_vectors_p instead of LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. Pass vec_info * instead of loop_vec_info to check_load_store_for_partial_vectors. Delete local alias for LOOP_VINFO_MASKS. Use vect_fully_masked_p instead of LOOP_VINFO_FULLY_MASKED_P. Use vect_fully_with_length_p instead of LOOP_VINFO_FULLY_WITH_LENGTH_P. Pass vec_info * instead of loop_vec_info to prepare_vec_mask. (vectorizable_load): As above. * tree-vectorizer.h (class vec_info): Moved declaration of the vec_cond_masked_set member from class _loop_vec_info. (class _loop_vec_info): Moved declaration of the vec_cond_masked_set member to class vec_info. (prepare_vec_mask): Substitute a parameter of type vec_info * for loop_vec_info. (vect_can_use_partial_vectors_p): New function that wraps use of LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P as a getter. (vect_cannot_use_partial_vectors): New function that wraps use of LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P to clear the flag. (vect_fully_with_length_p): New function that wraps use of LOOP_VINFO_FULLY_WITH_LENGTH_P. (vect_fully_masked_p): New function that wraps use of LOOP_VINFO_FULLY_MASKED_P. --- gcc/tree-vect-stmts.cc | 399 ++++++++++++++++++++++------------------- gcc/tree-vectorizer.h | 74 +++++++- 2 files changed, 287 insertions(+), 186 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 641b2835693..078cc63b2d9 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -1384,12 +1384,88 @@ vectorizable_internal_function (combined_fn cfn, tree fndecl, return IFN_LAST; } +/* Record that a complete set of masks associated with VINFO would need to + contain a sequence of NVECTORS masks that each control a vector of type + VECTYPE. If SCALAR_MASK is nonnull, the fully-masked loop would AND + these vector masks with the vector version of SCALAR_MASK. */ +static void +vect_record_mask (vec_info *vinfo, slp_tree slp_node, unsigned int nvectors, + tree vectype, tree scalar_mask) +{ + loop_vec_info loop_vinfo = dyn_cast (vinfo); + if (loop_vinfo) + vect_record_loop_mask (loop_vinfo, &LOOP_VINFO_MASKS (loop_vinfo), nvectors, + vectype, scalar_mask); + else + (void) slp_node; /* FORNOW */ +} + +/* Given a complete set of masks associated with VINFO, extract mask number + INDEX for an rgroup that operates on NVECTORS vectors of type VECTYPE, where + 0 <= INDEX < NVECTORS. Alternatively, if doing basic block vectorization, + materialize an equivalent mask for SLP_NODE. Insert any set-up statements + before GSI. */ +static tree +vect_get_mask (vec_info *vinfo, slp_tree slp_node, gimple_stmt_iterator *gsi, + unsigned int nvectors, tree vectype, unsigned int index) +{ + gcc_assert (vect_fully_masked_p (vinfo, slp_node)); + loop_vec_info loop_vinfo = dyn_cast (vinfo); + if (loop_vinfo) + return vect_get_loop_mask (loop_vinfo, gsi, &LOOP_VINFO_MASKS (loop_vinfo), + nvectors, vectype, index); + else + { + (void) slp_node; /* FORNOW */ + return NULL_TREE; + } +} + +/* Record that a complete set of lengths associated with VINFO would need to + contain a sequence of NVECTORS lengths for controlling an operation on + VECTYPE. The operation splits each element of VECTYPE into FACTOR separate + subelements, measuring the length as a number of these subelements. */ +static void +vect_record_len (vec_info *vinfo, slp_tree slp_node, unsigned int nvectors, + tree vectype, unsigned int factor) +{ + loop_vec_info loop_vinfo = dyn_cast (vinfo); + if (loop_vinfo) + vect_record_loop_len (loop_vinfo, &LOOP_VINFO_LENS (loop_vinfo), nvectors, + vectype, factor); + else + (void) slp_node; /* FORNOW */ +} + +/* Given a complete set of lengths associated with VINFO, extract length number + INDEX for an rgroup that operates on NVECTORS vectors of type VECTYPE, where + 0 <= INDEX < NVECTORS. Alternatively, if doing basic block vectorization, + materialize an equivalent length for SLP_NODE. Return a value that contains + FACTOR multiplied by the number of elements that should be processed. Insert + any set-up statements before GSI. */ + +static tree +vect_get_len (vec_info *vinfo, slp_tree slp_node, gimple_stmt_iterator *gsi, + unsigned int nvectors, tree vectype, unsigned int index, + unsigned int factor) +{ + gcc_assert (vect_fully_with_length_p (vinfo, slp_node)); + loop_vec_info loop_vinfo = dyn_cast (vinfo); + if (loop_vinfo) + return vect_get_loop_len (loop_vinfo, gsi, &LOOP_VINFO_LENS (loop_vinfo), + nvectors, vectype, index, factor); + else + { + (void) slp_node; /* FORNOW */ + return NULL_TREE; + } +} static tree permute_vec_elements (vec_info *, tree, tree, tree, stmt_vec_info, gimple_stmt_iterator *); -/* Check whether a load or store statement in the loop described by - LOOP_VINFO is possible in a loop using partial vectors. This is +/* Check whether a load or store statement in the loop or SLP group described by + VINFO is possible using partial vectors. This is testing whether the vectorizer pass has the appropriate support, as well as whether the target does. @@ -1404,13 +1480,14 @@ static tree permute_vec_elements (vec_info *, tree, tree, tree, stmt_vec_info, Clear LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P if a loop using partial vectors is not supported, otherwise record the required rgroup control - types. + types. For basic block SLP, simply record which style of partial vectors + is available (if any). If partial vectors can be used and ELSVALS is nonzero the supported else values will be added to the vector ELSVALS points to. */ static void -check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, +check_load_store_for_partial_vectors (vec_info *vinfo, tree vectype, slp_tree slp_node, vec_load_store_type vls_type, int group_size, @@ -1418,6 +1495,7 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, slp_tree mask_node, vec *elsvals = nullptr) { + loop_vec_info loop_vinfo = dyn_cast (vinfo); vect_memory_access_type memory_access_type = ls->memory_access_type; /* Invariant loads need no special support. */ @@ -1441,9 +1519,7 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, scalar_mask = def; } - unsigned int nvectors = vect_get_num_copies (loop_vinfo, slp_node); - vec_loop_masks *masks = &LOOP_VINFO_MASKS (loop_vinfo); - vec_loop_lens *lens = &LOOP_VINFO_LENS (loop_vinfo); + unsigned int nvectors = vect_get_num_copies (vinfo, slp_node); machine_mode vecmode = TYPE_MODE (vectype); bool is_load = (vls_type == VLS_LOAD); if (memory_access_type == VMAT_LOAD_STORE_LANES) @@ -1454,10 +1530,10 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, elsvals) : vect_store_lanes_supported (vectype, group_size, true)); if (ifn == IFN_MASK_LEN_LOAD_LANES || ifn == IFN_MASK_LEN_STORE_LANES) - vect_record_loop_len (loop_vinfo, lens, nvectors, vectype, 1); + vect_record_len (vinfo, slp_node, nvectors, vectype, 1); else if (ifn == IFN_MASK_LOAD_LANES || ifn == IFN_MASK_STORE_LANES) - vect_record_loop_mask (loop_vinfo, masks, nvectors, vectype, - scalar_mask); + vect_record_mask (vinfo, slp_node, nvectors, vectype, + scalar_mask); else { if (dump_enabled_p ()) @@ -1465,7 +1541,7 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, "can't operate on partial vectors because" " the target doesn't have an appropriate" " load/store-lanes instruction.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); } return; } @@ -1499,14 +1575,13 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, memory_type, off_vectype, scale, elsvals)) - vect_record_loop_len (loop_vinfo, lens, nvectors, vectype, 1); + vect_record_len (vinfo, slp_node, nvectors, vectype, 1); else if (internal_gather_scatter_fn_supported_p (ifn, vectype, memory_type, off_vectype, scale, elsvals) || memory_access_type == VMAT_GATHER_SCATTER_LEGACY) - vect_record_loop_mask (loop_vinfo, masks, nvectors, vectype, - scalar_mask); + vect_record_mask (vinfo, slp_node, nvectors, vectype, scalar_mask); else { if (dump_enabled_p ()) @@ -1514,7 +1589,7 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, "can't operate on partial vectors because" " the target doesn't have an appropriate" " gather load or scatter store instruction.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); } return; } @@ -1527,7 +1602,7 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't operate on partial vectors because an" " access isn't contiguous.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); return; } @@ -1537,7 +1612,7 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't operate on partial vectors when emulating" " vector operations.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); return; } @@ -1562,7 +1637,7 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, { nvectors = group_memory_nvectors (group_size * vf, nunits); unsigned factor = (vecmode == vmode) ? 1 : GET_MODE_UNIT_SIZE (vecmode); - vect_record_loop_len (loop_vinfo, lens, nvectors, vectype, factor); + vect_record_len (vinfo, slp_node, nvectors, vectype, factor); using_partial_vectors_p = true; } else if (targetm.vectorize.get_mask_mode (vecmode).exists (&mask_mode) @@ -1570,7 +1645,7 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, elsvals)) { nvectors = group_memory_nvectors (group_size * vf, nunits); - vect_record_loop_mask (loop_vinfo, masks, nvectors, vectype, scalar_mask); + vect_record_mask (vinfo, slp_node, nvectors, vectype, scalar_mask); using_partial_vectors_p = true; } @@ -1581,35 +1656,35 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, "can't operate on partial vectors because the" " target doesn't have the appropriate partial" " vectorization load or store.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); } } /* Return the mask input to a masked load or store. VEC_MASK is the vectorized - form of the scalar mask condition and LOOP_MASK, if nonnull, is the mask - that needs to be applied to all loads and stores in a vectorized loop. - Return VEC_MASK if LOOP_MASK is null or if VEC_MASK is already masked, - otherwise return VEC_MASK & LOOP_MASK. + form of the scalar mask condition and LOOP_OR_TAIL_MASK, if nonnull, is the + mask that needs to be applied to all loads and stores in a vectorized loop. + Return VEC_MASK if LOOP_OR_TAIL_MASK is null or if VEC_MASK is already + masked, otherwise return VEC_MASK & LOOP_OR_TAIL_MASK. MASK_TYPE is the type of both masks. If new statements are needed, insert them before GSI. */ tree -prepare_vec_mask (loop_vec_info loop_vinfo, tree mask_type, tree loop_mask, +prepare_vec_mask (vec_info *vinfo, tree mask_type, tree loop_or_tail_mask, tree vec_mask, gimple_stmt_iterator *gsi) { gcc_assert (useless_type_conversion_p (mask_type, TREE_TYPE (vec_mask))); - if (!loop_mask) + if (!loop_or_tail_mask) return vec_mask; - gcc_assert (TREE_TYPE (loop_mask) == mask_type); + gcc_assert (TREE_TYPE (loop_or_tail_mask) == mask_type); - if (loop_vinfo->vec_cond_masked_set.contains ({ vec_mask, loop_mask })) + if (vinfo->vec_cond_masked_set.contains ({ vec_mask, loop_or_tail_mask })) return vec_mask; tree and_res = make_temp_ssa_name (mask_type, NULL, "vec_mask_and"); - gimple *and_stmt = gimple_build_assign (and_res, BIT_AND_EXPR, - vec_mask, loop_mask); + gimple *and_stmt + = gimple_build_assign (and_res, BIT_AND_EXPR, vec_mask, loop_or_tail_mask); gsi_insert_before (gsi, and_stmt, GSI_SAME_STMT); return and_res; @@ -3671,8 +3746,6 @@ vectorizable_call (vec_info *vinfo, ? ifn : get_conditional_internal_fn (ifn)); internal_fn cond_len_fn = get_len_internal_fn (ifn); int len_opno = internal_fn_len_index (cond_len_fn); - vec_loop_masks *masks = (loop_vinfo ? &LOOP_VINFO_MASKS (loop_vinfo) : NULL); - vec_loop_lens *lens = (loop_vinfo ? &LOOP_VINFO_LENS (loop_vinfo) : NULL); unsigned int nvectors = vect_get_num_copies (vinfo, slp_node); if (cost_vec) /* transformation not required. */ { @@ -3690,8 +3763,7 @@ vectorizable_call (vec_info *vinfo, DUMP_VECT_SCOPE ("vectorizable_call"); vect_model_simple_cost (vinfo, 1, slp_node, cost_vec); - if (loop_vinfo - && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) + if (vect_can_use_partial_vectors_p (vinfo, slp_node) && (reduc_idx >= 0 || mask_opno >= 0)) { if (reduc_idx >= 0 @@ -3706,7 +3778,7 @@ vectorizable_call (vec_info *vinfo, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't use a fully-masked loop because no" " conditional operation is available.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); } else { @@ -3716,11 +3788,10 @@ vectorizable_call (vec_info *vinfo, if (cond_len_fn != IFN_LAST && direct_internal_fn_supported_p (cond_len_fn, vectype_out, OPTIMIZE_FOR_SPEED)) - vect_record_loop_len (loop_vinfo, lens, nvectors, vectype_out, - 1); + vect_record_len (vinfo, slp_node, nvectors, vectype_out, 1); else - vect_record_loop_mask (loop_vinfo, masks, nvectors, vectype_out, - scalar_mask); + vect_record_mask (vinfo, slp_node, nvectors, vectype_out, + scalar_mask); } } return true; @@ -3735,10 +3806,10 @@ vectorizable_call (vec_info *vinfo, scalar_dest = gimple_call_lhs (stmt); vec_dest = vect_create_destination_var (scalar_dest, vectype_out); - bool masked_loop_p = loop_vinfo && LOOP_VINFO_FULLY_MASKED_P (loop_vinfo); - bool len_loop_p = loop_vinfo && LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo); + bool masked_p = vect_fully_masked_p (vinfo, slp_node); + bool len_p = vect_fully_with_length_p (vinfo, slp_node); unsigned int vect_nargs = nargs; - if (len_loop_p) + if (len_p) { if (len_opno >= 0) { @@ -3749,7 +3820,7 @@ vectorizable_call (vec_info *vinfo, else if (reduc_idx >= 0) gcc_unreachable (); } - else if (masked_loop_p && mask_opno == -1 && reduc_idx >= 0) + else if (masked_p && mask_opno == -1 && reduc_idx >= 0) { ifn = cond_fn; vect_nargs += 2; @@ -3793,12 +3864,12 @@ vectorizable_call (vec_info *vinfo, { int varg = 0; /* Add the mask if necessary. */ - if (masked_loop_p && mask_opno == -1 && reduc_idx >= 0) + if (masked_p && mask_opno == -1 && reduc_idx >= 0) { gcc_assert (internal_fn_mask_index (ifn) == varg); unsigned int vec_num = vec_oprnds0.length (); - vargs[varg++] = vect_get_loop_mask (loop_vinfo, gsi, masks, - vec_num, vectype_out, i); + vargs[varg++] = vect_get_mask (vinfo, slp_node, gsi, vec_num, + vectype_out, i); } size_t k; for (k = 0; k < nargs; k++) @@ -3807,7 +3878,7 @@ vectorizable_call (vec_info *vinfo, vargs[varg++] = vec_oprndsk[i]; } /* Add the else value if necessary. */ - if (masked_loop_p && mask_opno == -1 && reduc_idx >= 0) + if (masked_p && mask_opno == -1 && reduc_idx >= 0) { gcc_assert (internal_fn_else_index (ifn) == varg); vargs[varg++] = vargs[reduc_idx + 1]; @@ -3838,24 +3909,26 @@ vectorizable_call (vec_info *vinfo, } else { - if (len_opno >= 0 && len_loop_p) + if (len_opno >= 0 && len_p) { unsigned int vec_num = vec_oprnds0.length (); - tree len = vect_get_loop_len (loop_vinfo, gsi, lens, - vec_num, vectype_out, i, 1); + tree len = vect_get_len (vinfo, slp_node, gsi, vec_num, + vectype_out, i, 1); signed char biasval - = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo); + = loop_vinfo + ? LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo) + : 0; tree bias = build_int_cst (intQI_type_node, biasval); vargs[len_opno] = len; vargs[len_opno + 1] = bias; } - else if (mask_opno >= 0 && masked_loop_p) + else if (mask_opno >= 0 && masked_p) { unsigned int vec_num = vec_oprnds0.length (); - tree mask = vect_get_loop_mask (loop_vinfo, gsi, masks, - vec_num, vectype_out, i); + tree mask = vect_get_mask (vinfo, slp_node, gsi, vec_num, + vectype_out, i); vargs[mask_opno] - = prepare_vec_mask (loop_vinfo, TREE_TYPE (mask), mask, + = prepare_vec_mask (vinfo, TREE_TYPE (mask), mask, vargs[mask_opno], gsi); } @@ -4412,15 +4485,13 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info, } break; case SIMD_CLONE_ARG_TYPE_MASK: - if (loop_vinfo - && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)) + if (vect_can_use_partial_vectors_p (vinfo, slp_node)) { if (masked_call_offset) /* When there is an explicit mask we require the number of elements to match up. */ - vect_record_loop_mask (loop_vinfo, - &LOOP_VINFO_MASKS (loop_vinfo), - ncopies_in, vectype, NULL_TREE); + vect_record_mask (vinfo, slp_node, ncopies_in, vectype, + NULL_TREE); else { /* When there is no explicit mask on the call we have @@ -4442,23 +4513,22 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info, callee_nelements = TYPE_VECTOR_SUBPARTS (masktype); } auto o = vector_unroll_factor (nunits, callee_nelements); - vect_record_loop_mask (loop_vinfo, - &LOOP_VINFO_MASKS (loop_vinfo), - ncopies * o, masktype, NULL_TREE); + vect_record_mask (vinfo, slp_node, ncopies * o, masktype, + NULL_TREE); } } break; } } - if (!bestn->simdclone->inbranch && loop_vinfo) + if (!bestn->simdclone->inbranch) { if (dump_enabled_p () - && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)) + && vect_can_use_partial_vectors_p (vinfo, slp_node)) dump_printf_loc (MSG_NOTE, vect_location, "can't use a fully-masked loop because a" " non-masked simd clone was selected.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); } SLP_TREE_TYPE (slp_node) = call_simd_clone_vec_info_type; @@ -4629,23 +4699,16 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info, if (m == 0) vec_oprnds_i[i] = 0; vec_oprnd0 = vec_oprnds[i][vec_oprnds_i[i]++]; - if (loop_vinfo - && LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) + if (vect_fully_masked_p (vinfo, slp_node)) { - vec_loop_masks *loop_masks - = &LOOP_VINFO_MASKS (loop_vinfo); - tree loop_mask - = vect_get_loop_mask (loop_vinfo, gsi, - loop_masks, ncopies_in, - vectype, j); + tree mask + = vect_get_mask (vinfo, slp_node, gsi, + ncopies_in, vectype, j); vec_oprnd0 - = prepare_vec_mask (loop_vinfo, - TREE_TYPE (loop_mask), - loop_mask, vec_oprnd0, - gsi); - loop_vinfo->vec_cond_masked_set.add ({ vec_oprnd0, - loop_mask }); - + = prepare_vec_mask (vinfo, TREE_TYPE (mask), + mask, vec_oprnd0, gsi); + vinfo->vec_cond_masked_set.add ( + {vec_oprnd0, mask}); } vec_oprnd0 = build3 (VEC_COND_EXPR, atype, vec_oprnd0, @@ -4844,12 +4907,9 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info, o = vector_unroll_factor (nunits, callee_nelements); for (m = j * o; m < (j + 1) * o; m++) { - if (loop_vinfo && LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) - { - vec_loop_masks *loop_masks = &LOOP_VINFO_MASKS (loop_vinfo); - mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, - ncopies * o, mask_vectype, m); - } + if (vect_fully_masked_p (vinfo, slp_node)) + mask = vect_get_mask (vinfo, slp_node, gsi, ncopies * o, + mask_vectype, m); else mask = vect_build_all_ones_mask (vinfo, stmt_info, mask_argtype); @@ -5256,7 +5316,6 @@ vectorizable_conversion (vec_info *vinfo, vec vec_oprnds1 = vNULL; tree vop0; bb_vec_info bb_vinfo = dyn_cast (vinfo); - loop_vec_info loop_vinfo = dyn_cast (vinfo); int multi_step_cvt = 0; vec interm_types = vNULL; tree intermediate_type, cvt_type = NULL_TREE; @@ -5600,8 +5659,7 @@ vectorizable_conversion (vec_info *vinfo, } if (modifier == WIDEN - && loop_vinfo - && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) + && vect_can_use_partial_vectors_p (vinfo, slp_node) && (code1 == VEC_WIDEN_MULT_EVEN_EXPR || widening_evenodd_fn_p (code1))) { @@ -5610,7 +5668,7 @@ vectorizable_conversion (vec_info *vinfo, "can't use a fully-masked loop because" " widening operation on even/odd elements" " mixes up lanes.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); } if (cost_vec) /* transformation not required. */ @@ -6690,7 +6748,6 @@ vectorizable_operation (vec_info *vinfo, int reduc_idx = SLP_TREE_REDUC_IDX (slp_node); vec_loop_masks *masks = (loop_vinfo ? &LOOP_VINFO_MASKS (loop_vinfo) : NULL); - vec_loop_lens *lens = (loop_vinfo ? &LOOP_VINFO_LENS (loop_vinfo) : NULL); internal_fn cond_fn = get_conditional_internal_fn (code); internal_fn cond_len_fn = get_conditional_len_internal_fn (code); @@ -6707,27 +6764,24 @@ vectorizable_operation (vec_info *vinfo, if (cost_vec) /* transformation not required. */ { - if (loop_vinfo - && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) + if (vect_can_use_partial_vectors_p (vinfo, slp_node) && mask_out_inactive) { if (cond_len_fn != IFN_LAST && direct_internal_fn_supported_p (cond_len_fn, vectype, OPTIMIZE_FOR_SPEED)) - vect_record_loop_len (loop_vinfo, lens, vec_num, vectype, - 1); + vect_record_len (vinfo, slp_node, vec_num, vectype, 1); else if (cond_fn != IFN_LAST && direct_internal_fn_supported_p (cond_fn, vectype, OPTIMIZE_FOR_SPEED)) - vect_record_loop_mask (loop_vinfo, masks, vec_num, - vectype, NULL); + vect_record_mask (vinfo, slp_node, vec_num, vectype, NULL); else { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't use a fully-masked loop because no" " conditional operation is available.\n"); - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + vect_cannot_use_partial_vectors (vinfo, slp_node); } } @@ -6787,8 +6841,8 @@ vectorizable_operation (vec_info *vinfo, dump_printf_loc (MSG_NOTE, vect_location, "transform binary/unary operation.\n"); - bool masked_loop_p = loop_vinfo && LOOP_VINFO_FULLY_MASKED_P (loop_vinfo); - bool len_loop_p = loop_vinfo && LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo); + bool masked_p = vect_fully_masked_p (vinfo, slp_node); + bool len_p = vect_fully_with_length_p (vinfo, slp_node); /* POINTER_DIFF_EXPR has pointer arguments which are vectorized as vectors with unsigned elements, but the result is signed. So, we @@ -6976,12 +7030,11 @@ vectorizable_operation (vec_info *vinfo, gimple_assign_set_lhs (new_stmt, new_temp); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); } - else if ((masked_loop_p || len_loop_p) && mask_out_inactive) + else if ((masked_p || len_p) && mask_out_inactive) { tree mask; - if (masked_loop_p) - mask = vect_get_loop_mask (loop_vinfo, gsi, masks, - vec_num, vectype, i); + if (masked_p) + mask = vect_get_mask (vinfo, slp_node, gsi, vec_num, vectype, i); else /* Dummy mask. */ mask = build_minus_one_cst (truth_type_for (vectype)); @@ -7005,19 +7058,19 @@ vectorizable_operation (vec_info *vinfo, (cond_fn, vectype, vops.length () - 1, &vops[1]); vops.quick_push (else_value); } - if (len_loop_p) + if (len_p) { - tree len = vect_get_loop_len (loop_vinfo, gsi, lens, - vec_num, vectype, i, 1); + tree len + = vect_get_len (vinfo, slp_node, gsi, vec_num, vectype, i, 1); signed char biasval - = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo); + = loop_vinfo ? LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo) + : 0; tree bias = build_int_cst (intQI_type_node, biasval); vops.quick_push (len); vops.quick_push (bias); } gcall *call - = gimple_build_call_internal_vec (masked_loop_p ? cond_fn - : cond_len_fn, + = gimple_build_call_internal_vec (masked_p ? cond_fn : cond_len_fn, vops); new_temp = make_ssa_name (vec_dest, call); gimple_call_set_lhs (call, new_temp); @@ -7031,8 +7084,7 @@ vectorizable_operation (vec_info *vinfo, /* When combining two masks check if either of them is elsewhere combined with a loop mask, if that's the case we can mark that the new combined mask doesn't need to be combined with a loop mask. */ - if (masked_loop_p - && code == BIT_AND_EXPR + if (loop_vinfo && masked_p && code == BIT_AND_EXPR && VECTOR_BOOLEAN_TYPE_P (vectype)) { if (loop_vinfo->scalar_cond_masked_set.contains ({ op0, vec_num })) @@ -7064,7 +7116,7 @@ vectorizable_operation (vec_info *vinfo, /* Enter the combined value into the vector cond hash so we don't AND it with a loop mask again. */ if (mask) - loop_vinfo->vec_cond_masked_set.add ({ new_temp, mask }); + vinfo->vec_cond_masked_set.add ({ new_temp, mask }); } if (vec_cvt_dest) @@ -8206,9 +8258,8 @@ vectorizable_store (vec_info *vinfo, bool costing_p = cost_vec; if (costing_p) /* transformation not required. */ { - if (loop_vinfo - && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)) - check_load_store_for_partial_vectors (loop_vinfo, vectype, slp_node, + if (vect_can_use_partial_vectors_p (vinfo, slp_node)) + check_load_store_for_partial_vectors (vinfo, vectype, slp_node, vls_type, group_size, &ls, mask_node); @@ -8561,10 +8612,6 @@ vectorizable_store (vec_info *vinfo, } gcc_assert (alignment_support_scheme); - vec_loop_masks *loop_masks - = (loop_vinfo && LOOP_VINFO_FULLY_MASKED_P (loop_vinfo) - ? &LOOP_VINFO_MASKS (loop_vinfo) - : NULL); vec_loop_lens *loop_lens = (loop_vinfo && LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo) ? &LOOP_VINFO_LENS (loop_vinfo) @@ -8576,14 +8623,14 @@ vectorizable_store (vec_info *vinfo, Shouldn't go with length-based approach if fully masked. */ if (cost_vec == NULL) /* The cost_vec is NULL during transfrom. */ - gcc_assert ((!loop_lens || !loop_masks)); + gcc_assert ((!vect_fully_with_length_p (vinfo, slp_node) + || !vect_fully_masked_p (vinfo, slp_node))); /* Targets with store-lane instructions must not require explicit realignment. vect_supportable_dr_alignment always returns either dr_aligned or dr_unaligned_supported for masked operations. */ - gcc_assert ((memory_access_type != VMAT_LOAD_STORE_LANES - && !mask_node - && !loop_masks) + gcc_assert ((memory_access_type != VMAT_LOAD_STORE_LANES && !mask_node + && !vect_fully_masked_p (vinfo, slp_node)) || alignment_support_scheme == dr_aligned || alignment_support_scheme == dr_unaligned_supported); @@ -8713,18 +8760,18 @@ vectorizable_store (vec_info *vinfo, tree final_mask = NULL; tree final_len = NULL; tree bias = NULL; - if (loop_masks) - final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, - ncopies, vectype, j); + if (vect_fully_masked_p (vinfo, slp_node)) + final_mask + = vect_get_mask (vinfo, slp_node, gsi, ncopies, vectype, j); if (vec_mask) - final_mask = prepare_vec_mask (loop_vinfo, mask_vectype, final_mask, + final_mask = prepare_vec_mask (vinfo, mask_vectype, final_mask, vec_mask, gsi); if (lanes_ifn == IFN_MASK_LEN_STORE_LANES) { - if (loop_lens) - final_len = vect_get_loop_len (loop_vinfo, gsi, loop_lens, - ncopies, vectype, j, 1); + if (vect_fully_with_length_p (vinfo, slp_node)) + final_len + = vect_get_len (vinfo, slp_node, gsi, ncopies, vectype, j, 1); else final_len = size_int (TYPE_VECTOR_SUBPARTS (vectype)); signed char biasval @@ -8852,12 +8899,11 @@ vectorizable_store (vec_info *vinfo, tree bias = NULL_TREE; if (!costing_p) { - if (loop_masks) - final_mask = vect_get_loop_mask (loop_vinfo, gsi, - loop_masks, num_stmts, - vectype, j); + if (vect_fully_masked_p (vinfo, slp_node)) + final_mask + = vect_get_mask (vinfo, slp_node, gsi, num_stmts, vectype, j); if (vec_mask) - final_mask = prepare_vec_mask (loop_vinfo, mask_vectype, + final_mask = prepare_vec_mask (vinfo, mask_vectype, final_mask, vec_mask, gsi); } @@ -8915,9 +8961,8 @@ vectorizable_store (vec_info *vinfo, if (ls.gs.ifn == IFN_MASK_LEN_SCATTER_STORE) { if (loop_lens) - final_len = vect_get_loop_len (loop_vinfo, gsi, - loop_lens, num_stmts, - vectype, j, 1); + final_len = vect_get_len (vinfo, slp_node, gsi, num_stmts, + vectype, j, 1); else final_len = size_int (TYPE_VECTOR_SUBPARTS (vectype)); @@ -9202,9 +9247,7 @@ vectorizable_store (vec_info *vinfo, || useless_type_conversion_p (vectype, TREE_TYPE (vec_oprnd))); bool simd_lane_access_p = STMT_VINFO_SIMD_LANE_ACCESS_P (stmt_info) != 0; - if (!costing_p - && simd_lane_access_p - && !loop_masks + if (!costing_p && simd_lane_access_p && !vect_fully_masked_p (vinfo, slp_node) && TREE_CODE (DR_BASE_ADDRESS (first_dr_info->dr)) == ADDR_EXPR && VAR_P (TREE_OPERAND (DR_BASE_ADDRESS (first_dr_info->dr), 0)) && integer_zerop (get_dr_vinfo_offset (vinfo, first_dr_info)) @@ -9258,13 +9301,12 @@ vectorizable_store (vec_info *vinfo, tree final_mask = NULL_TREE; tree final_len = NULL_TREE; tree bias = NULL_TREE; - if (loop_masks) - final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, - vec_num, vectype, i); + if (vect_fully_masked_p (vinfo, slp_node)) + final_mask = vect_get_mask (vinfo, slp_node, gsi, vec_num, vectype, i); if (vec_mask) vec_mask = vec_masks[i]; if (vec_mask) - final_mask = prepare_vec_mask (loop_vinfo, mask_vectype, final_mask, + final_mask = prepare_vec_mask (vinfo, mask_vectype, final_mask, vec_mask, gsi); if (i > 0) @@ -9300,8 +9342,8 @@ vectorizable_store (vec_info *vinfo, new_vmode = new_ovmode.require (); unsigned factor = (new_ovmode == vmode) ? 1 : GET_MODE_UNIT_SIZE (vmode); - final_len = vect_get_loop_len (loop_vinfo, gsi, loop_lens, - vec_num, vectype, i, factor); + final_len + = vect_get_len (vinfo, slp_node, gsi, vec_num, vectype, i, factor); } else if (final_mask) { @@ -9824,9 +9866,8 @@ vectorizable_load (vec_info *vinfo, return false; } - if (loop_vinfo - && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)) - check_load_store_for_partial_vectors (loop_vinfo, vectype, slp_node, + if (vect_can_use_partial_vectors_p (vinfo, slp_node)) + check_load_store_for_partial_vectors (vinfo, vectype, slp_node, VLS_LOAD, group_size, &ls, mask_node, &ls.elsvals); @@ -10432,14 +10473,10 @@ vectorizable_load (vec_info *vinfo, ref_type = reference_alias_ptr_type (DR_REF (first_dr_info->dr)); } - vec_loop_masks *loop_masks - = (loop_vinfo && LOOP_VINFO_FULLY_MASKED_P (loop_vinfo) - ? &LOOP_VINFO_MASKS (loop_vinfo) - : NULL); vec_loop_lens *loop_lens = (loop_vinfo && LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo) - ? &LOOP_VINFO_LENS (loop_vinfo) - : NULL); + ? &LOOP_VINFO_LENS (loop_vinfo) + : NULL); /* The vect_transform_stmt and vect_analyze_stmt will go here but there are some difference here. We cannot enable both the lens and masks @@ -10447,15 +10484,15 @@ vectorizable_load (vec_info *vinfo, Shouldn't go with length-based approach if fully masked. */ if (cost_vec == NULL) /* The cost_vec is NULL during transfrom. */ - gcc_assert ((!loop_lens || !loop_masks)); + gcc_assert ((!vect_fully_with_length_p (vinfo, slp_node) + || !vect_fully_masked_p (vinfo, slp_node))); /* Targets with store-lane instructions must not require explicit realignment. vect_supportable_dr_alignment always returns either dr_aligned or dr_unaligned_supported for (non-length) masked operations. */ - gcc_assert ((memory_access_type != VMAT_LOAD_STORE_LANES - && !mask_node - && !loop_masks) + gcc_assert ((memory_access_type != VMAT_LOAD_STORE_LANES && !mask_node + && !vect_fully_masked_p (vinfo, slp_node)) || mat_gather_scatter_p (memory_access_type) || alignment_support_scheme == dr_aligned || alignment_support_scheme == dr_unaligned_supported); @@ -10654,18 +10691,18 @@ vectorizable_load (vec_info *vinfo, tree final_mask = NULL_TREE; tree final_len = NULL_TREE; tree bias = NULL_TREE; - if (loop_masks) - final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, - ncopies, vectype, j); + if (vect_fully_masked_p (vinfo, slp_node)) + final_mask + = vect_get_mask (vinfo, slp_node, gsi, ncopies, vectype, j); if (vec_mask) - final_mask = prepare_vec_mask (loop_vinfo, mask_vectype, final_mask, + final_mask = prepare_vec_mask (vinfo, mask_vectype, final_mask, vec_mask, gsi); if (lanes_ifn == IFN_MASK_LEN_LOAD_LANES) { - if (loop_lens) - final_len = vect_get_loop_len (loop_vinfo, gsi, loop_lens, - ncopies, vectype, j, 1); + if (vect_fully_with_length_p (vinfo, slp_node)) + final_len + = vect_get_len (vinfo, slp_node, gsi, ncopies, vectype, j, 1); else final_len = size_int (TYPE_VECTOR_SUBPARTS (vectype)); signed char biasval @@ -10801,11 +10838,11 @@ vectorizable_load (vec_info *vinfo, { if (mask_node) vec_mask = vec_masks[i]; - if (loop_masks) - final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, - vec_num, vectype, i); + if (vect_fully_masked_p (vinfo, slp_node)) + final_mask + = vect_get_mask (vinfo, slp_node, gsi, vec_num, vectype, i); if (vec_mask) - final_mask = prepare_vec_mask (loop_vinfo, mask_vectype, + final_mask = prepare_vec_mask (vinfo, mask_vectype, final_mask, vec_mask, gsi); if (i > 0 && !STMT_VINFO_GATHER_SCATTER_P (stmt_info)) @@ -10867,8 +10904,8 @@ vectorizable_load (vec_info *vinfo, if (ls.gs.ifn == IFN_MASK_LEN_GATHER_LOAD) { if (loop_lens) - final_len = vect_get_loop_len (loop_vinfo, gsi, loop_lens, - vec_num, vectype, i, 1); + final_len = vect_get_len (vinfo, slp_node, gsi, vec_num, + vectype, i, 1); else final_len = build_int_cst (sizetype, TYPE_VECTOR_SUBPARTS (vectype)); @@ -11269,11 +11306,11 @@ vectorizable_load (vec_info *vinfo, { if (mask_node) vec_mask = vec_masks[i]; - if (loop_masks) - final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, - vec_num, vectype, i); + if (vect_fully_masked_p (vinfo, slp_node)) + final_mask + = vect_get_mask (vinfo, slp_node, gsi, vec_num, vectype, i); if (vec_mask) - final_mask = prepare_vec_mask (loop_vinfo, mask_vectype, + final_mask = prepare_vec_mask (vinfo, mask_vectype, final_mask, vec_mask, gsi); if (i > 0) @@ -11319,8 +11356,8 @@ vectorizable_load (vec_info *vinfo, new_vmode = new_ovmode.require (); unsigned factor = (new_ovmode == vmode) ? 1 : GET_MODE_UNIT_SIZE (vmode); - final_len = vect_get_loop_len (loop_vinfo, gsi, loop_lens, - vec_num, vectype, i, factor); + final_len = vect_get_len (vinfo, slp_node, gsi, vec_num, + vectype, i, factor); } else if (final_mask) { diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 0356b129e36..2d914dca90b 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -676,6 +676,9 @@ public: and are reset when undoing patterns. */ gimple_seq inv_pattern_def_seq; + /* Set of vector conditions that have loop mask applied. */ + vec_cond_masked_set_type vec_cond_masked_set; + private: stmt_vec_info new_stmt_vec_info (gimple *stmt); void set_vinfo_for_stmt (gimple *, stmt_vec_info, bool = true); @@ -1007,9 +1010,6 @@ public: /* Set of scalar conditions that have loop mask applied. */ scalar_cond_masked_set_type scalar_cond_masked_set; - /* Set of vector conditions that have loop mask applied. */ - vec_cond_masked_set_type vec_cond_masked_set; - /* If we are using a loop mask to align memory addresses, this variable contains the number of vector elements that we should skip in the first iteration of the vector loop (i.e. the number of leading @@ -2760,8 +2760,7 @@ extern slp_tree vect_create_new_slp_node (unsigned, tree_code); extern void vect_free_slp_tree (slp_tree); extern bool compatible_calls_p (gcall *, gcall *, bool); extern int vect_slp_child_index_for_operand (const gimple *, int op, bool); - -extern tree prepare_vec_mask (loop_vec_info, tree, tree, tree, +extern tree prepare_vec_mask (vec_info *, tree, tree, tree, gimple_stmt_iterator *); extern tree vect_get_mask_load_else (int, tree); extern bool vect_load_perm_consecutive_p (slp_tree, unsigned = UINT_MAX); @@ -2897,6 +2896,71 @@ vect_is_reduction (slp_tree slp_node) return SLP_TREE_REDUC_IDX (slp_node) != -1; } +/* Return true if VINFO is vectorizer state for loop vectorization and we + still have the option of vectorizing the loop using partially-populated + vectors; or, true if VINFO is for basic-block vectorization and we still have + the option of vectorizing the given SLP_NODE using partial vectors; + otherwise, return false. */ +inline bool +vect_can_use_partial_vectors_p (vec_info *vinfo, slp_tree slp_node) +{ + loop_vec_info loop_vinfo = dyn_cast (vinfo); + if (loop_vinfo) + return LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo); + + (void) slp_node; /* FORNOW */ + return false; +} + +/* If VINFO is vectorizer state for loop vectorization then record that we no + longer have the option of vectorizing the loop using partially-populated + vectors; or, if VINFO is for basic-block vectorization then record that we no + longer have the option of vectorizing the given SLP_NODE using partial + vectors. */ +inline void +vect_cannot_use_partial_vectors (vec_info *vinfo, slp_tree slp_node) +{ + loop_vec_info loop_vinfo = dyn_cast (vinfo); + if (loop_vinfo) + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; + else + (void) slp_node; /* FORNOW */ +} + +/* Return true if VINFO is vectorizer state for loop vectorization, we've + decided to use partially-populated vectors so that the loop can handle fewer + than VF scalars, and at least one length has been recorded; or, if VINFO is + for basic-block vectorization and SLP_NODE will be vectorized by using + lengths to prevent use of inactive scalar lanes. */ +inline bool +vect_fully_with_length_p (vec_info *vinfo, slp_tree slp_node) +{ + if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) + return LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo); + else + { + (void) slp_node; /* FORNOW */ + return false; + } +} + +/* Return true if VINFO is vectorizer state for loop vectorization, we've + decided to use partially-populated vectors so that the loop can handle fewer + than VF scalars, and at least one mask has been recorded; or, if VINFO is + for basic-block vectorization and SLP_NODE will be vectorized by using masks + to prevent use of inactive scalar lanes. */ +inline bool +vect_fully_masked_p (vec_info *vinfo, slp_tree slp_node) +{ + if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) + return LOOP_VINFO_FULLY_MASKED_P (loop_vinfo); + else + { + (void) slp_node; /* FORNOW */ + return false; + } +} + /* If STMT_INFO describes a reduction, return the vect_reduction_type of the reduction it describes, otherwise return -1. */ inline int From patchwork Sat Dec 6 16:55:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126053 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B9DE94152B57 for ; Sat, 6 Dec 2025 17:03:04 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B9DE94152B57 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=NJukIbdB; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=NJukIbdB X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from DU2PR03CU002.outbound.protection.outlook.com (mail-northeuropeazon11011066.outbound.protection.outlook.com [52.101.65.66]) by sourceware.org (Postfix) with ESMTPS id 2097F42FA793 for ; Sat, 6 Dec 2025 16:56:45 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2097F42FA793 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2097F42FA793 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.65.66 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040209; cv=pass; b=Et0U5cmyDHFmMBRFyerPlftmlBQWFhMBstDb/AKlBbneJDfxSQtxd1x/QQmaYVs+rZkOYjPeYsfyMeSmCOM1A5mfXzCRZbpdnS6D7aHMP5Uc5joz33w4/rA2AexZYTfxCLym/YcqhwOzNd9kR4TmGsnXjPTZwCywcDsXmuAbLaQ= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040209; c=relaxed/simple; bh=2EkDPQ8BtqDVY0QOV07Z2BcH2wwJg2A3N4e3uKa6V88=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=nMQmzapJDh23/2aOU/kZ7Dz73nymGaekt38bsR8+5NJDFG+ZtwDQFL4v7cyRuSBlK3lDtCgv/VNHGtLcqkji2KPLaEr6FcDCHlnqdOTfKErKNwDVP0P7JOxpyWYFuqfxJIv8+jTTsB3s+5LXOf8N32ZtNrEpasS16nDBypelodw= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2097F42FA793 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=hqiZDXf9QC0Pc5xLPBJIzzbeVedzV2LxjQsTt6poGOKrzcyIgG6egD98dgSl2dJPCqUgdpM01ZIbPNgXOuGZ5TjIloaD/CA6pZgXeMFE1ilRfmXVwTT3+PKJBsASqb9vb3ZG2+lfVl00vpdAP0zKtBGGz+04rgXJxzEIjBctlSe8TMLIzZz9UIfmDAAWOOjLUVDC3ZMMC3JE2CNXqL1HIFzbTAmxqU3zKscutfLdt6tjC8BbULfV6ijqNN9OkVsDCPFMZxs7QXPbkhhr+wuOfzvKlAz5t4IbFPjq40FlyQRMT0yTGXkf41gPJ4cUWAFNF/h//8LtW57scNwE4mL1yA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kk3grpAWw2v6FKrA4eVZSIp4/WtWbFvTeHBV8MHLsrk=; b=XYJrI/mXsP6l6LXlvUIcIAOWQ99Rdn83BMbZPZAFDeAbIsFgkCsqiitVmAba3zOBJxfpSzy/yd45EgyFeGlCxXfkSAV2lsu4T9Pqt/yvVz9Qmi4Kh0m56qCsTzJpbT0wrWj7V8DWmJE3iGaYq3bwZ3L1EnIjgMOS9coLm8ffCrxAqzd7b4gEaTDfmPL8aqJ+440B+WprgBpIaJU2yQbykeE34oFN0Y+hA7yECz2/E/T1q+ScdlNELqaA/HbPmznIGliRxBJ8jb6XOF5CUuBNz3E/CKnNHc0qHaYeRinOhyRKlKllUEKl8eJ7HstyfUuA93RRagR5E9fUv//C7WhcJA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kk3grpAWw2v6FKrA4eVZSIp4/WtWbFvTeHBV8MHLsrk=; b=NJukIbdBUjnrJ54CVPB7LRmPhwtMyKz0JHuZy0kp6D/gP6+leQg5HxsgioEX4nDpH1kp9WShMrTCDF9Ur3l0bbPP3+xDEta54nC5Cu7BD1itkn0fhRR3n7zO+SQyAw/zTxYINF4IRqT/f6lUEhfxw3nzYVo1yn/YDBWl1WZX1YI= Received: from DB8PR04CA0008.eurprd04.prod.outlook.com (2603:10a6:10:110::18) by AS4PR08MB8143.eurprd08.prod.outlook.com (2603:10a6:20b:58e::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:56:38 +0000 Received: from DU2PEPF00028D01.eurprd03.prod.outlook.com (2603:10a6:10:110:cafe::62) by DB8PR04CA0008.outlook.office365.com (2603:10a6:10:110::18) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by DU2PEPF00028D01.mail.protection.outlook.com (10.167.242.185) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:56:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xPE7QeAgdZjoMlP0IaM4FjWfRiOwYpGSDavc0QBzeCSfIDn3ReL3/B4ZyjG3Iqd9ZIj2omm8ko+Dj2idmGhAZukW7Q3K0nTTldOcGFjZrgnLzcv09qrk3zCf6gPv6UDaNDeQWJ+HB03FUjyEQWBt7zO7PFQnlFYVqGTGQx78dz1IjjXmrfoozvUOo5YC19l2V7MIyploT07u2Li7v1PNXVaimXfLZc6+0DnZlHZW1C03ndKbpU+IbEwrrE4kvPRxhb8B2lJ60OvuUngXqzWACv2yiJ/L6YNKmM0QRP97ij4JMdAPACA+E8vuycouf4qm6BeifkRAZzO3UaXTkkT0pQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kk3grpAWw2v6FKrA4eVZSIp4/WtWbFvTeHBV8MHLsrk=; b=C5QXci2iAHyRHPhsRdMa89ZX4z2VkYx9g6U42Ni3LVVEVbs1q71oHZgkS/YCmxW1yQnPs6Sg+GaaBnevlO5ZoJuYWDfnAgTdiiWvApuie25MkC7P7/47MgIkdX8XMTh3OMm23fdf6HY55GpPCiIelFLgSUkzeZpQ0HhRb3fUdDgP3KpacXPr+EpcRR0q8pyBE1amuWfn8vOyrWuTFwx+vZ3jWD6Dmb7gERtV1hGHnuddZF/qaHohpzLUldlilLdbiK6olgnJy3jyema2OrDp0fI56txNaKH0MjfTue0xb+/pKde0khYRu1QE799PtD8D+ElAiBIovOe1ot5t88yTLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kk3grpAWw2v6FKrA4eVZSIp4/WtWbFvTeHBV8MHLsrk=; b=NJukIbdBUjnrJ54CVPB7LRmPhwtMyKz0JHuZy0kp6D/gP6+leQg5HxsgioEX4nDpH1kp9WShMrTCDF9Ur3l0bbPP3+xDEta54nC5Cu7BD1itkn0fhRR3n7zO+SQyAw/zTxYINF4IRqT/f6lUEhfxw3nzYVo1yn/YDBWl1WZX1YI= Received: from DU7P195CA0030.EURP195.PROD.OUTLOOK.COM (2603:10a6:10:54d::17) by PAVPR08MB9532.eurprd08.prod.outlook.com (2603:10a6:102:312::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9366.13; Sat, 6 Dec 2025 16:55:30 +0000 Received: from DB1PEPF0003922D.eurprd03.prod.outlook.com (2603:10a6:10:54d:cafe::bd) by DU7P195CA0030.outlook.office365.com (2603:10a6:10:54d::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:55:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF0003922D.mail.protection.outlook.com (10.167.8.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:55:29 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX03.Arm.com (10.240.25.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:29 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:55:29 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 02/11] Implement recording/getting of mask/length for BB SLP Date: Sat, 6 Dec 2025 16:55:09 +0000 Message-ID: <20251206165518.5449-3-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF0003922D:EE_|PAVPR08MB9532:EE_|DU2PEPF00028D01:EE_|AS4PR08MB8143:EE_ X-MS-Office365-Filtering-Correlation-Id: 0014cd41-04b5-4bfb-b34a-08de34e86ba1 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|36860700013|82310400026|376014|1800799024; X-Microsoft-Antispam-Message-Info-Original: cgIUqvojr6oVEt068XxVoAWcGzFhj8IxyA91ikU3s7gW+W3rCQdEFIZzOHukPM9WmLLIJnsr+c0420rbxnr3l4oKZmp6cCzA3G8pDYxLeKrwYBKq5Ckx8QFWSECBOPaP0fBOZV1O1q4cG7Stk3GkCN9aQszKWC/PADGSUsfdGBPJcSgxNN4Qu9v19PA/gztg3xI0qR+bkN/DDgMs6dTNVcef4McIHQIoCBeSKBAKXPNmDcCInObAvrC1gpiO02vTJhyFmZbNLkOwXqIJiECDnysFKeNg8gXBkKS1CTqBimV0E/+e850LISMkxx3qCjaI8xLihVB3y0au3WfassoDTsAR0kboJW8Ff9+E3yqsxjsPMjElXFG8lfBbal4Bt2qr+UBjWmNHL+uHwb/8ZKXFCPLqRiXrhr9Q+Gd1P/WHHZNCBvVMtE/vwYZ2PiDPaRxHvU8TZlAD8B29NM4ejoER5lhvEUS5z1LsRwsYWTBRSMqKmP2F5sq/o6KUbJBLAh1DLCHhg9O82IRvjArf5TablgCYRO9B01CGK0boMLxFQVZ+xboLbu2G97cZR6saYrpB7p83YNH3jHTlLTTDaWXtmUF4NnKfNejNctJkx12Xn1kq+G7NnfVdc9TigHq/13RxECZmm3UL611+8Clwcf2Biur1uGxTc8Hj1x/X9J42mW6vMboG2BFW+uHfAZa5WWZ2jAqCjYxtvQwwH8OjiMVyQa/rEHePBzXiy1uzmshSscvvKUEUBYMnaVJmzaaXW5gSa6ii5jAbvWP3EeI00qGfzrVJr0AAbCnxiOhLLxNmFvyERwMQS5M/a1zwV7MWe9hGR2N0vHZ5YaK3RalpfbsiuMuMxk7kIgxy1S9XW+fLzFs2Wiy+u3Zvw3Xo3R7WUysiLck5EMNkcuE1Ures78j6Ki0Vs1rWczN3OjYExwuW6Z2+uEIY5xY3W4MDnbxwYLASCTfUSlB8TbO8bk4g2wleJI50IB5yINWJGLrWdfI+sSRHF+Tn8/qewdMB7kRdV82UDViZPv1G8B9/a+xZoLPGaATE+r8GGGw9gM9HKM5pvHvNjdUoZgucci/upD8xJFBr/jl8hozYvRDIfLXBVgkFobI+WxFX2+ZQ20Q+LzFR8OBV4ko+wzGcoGDjKAYDoDo5/Ky3iZ3gUOsYtyoZEaTycocdQRnUVpxYuTZTcE1xFQkhn/ANB3evHVbl2m7L0Po2ebYsChJP/mrA2KN93gUgd3ydlgxLS1rtcJUfFp8Ea3UP80x5CiSzMW7UGuhc+aRGmHLOoUFMrBLNO8TuNbAwynIEEDzH7QzRIws6fEnqhIuSHFNseKIsjjrnjJC6fS5C6TQTNq2aD9XWeL1JkWlD8lzjBp7zLIvXaICEryP35eN3njgsa61UCbfDv436vu/Vgpr2m/N+zU3G7s1c7qbe1+2s9cMHb+N115aIY27kz5pyqOMsueGyuRb3vp+ka2vipnAqaS8yvUB+mY2ANZmmDTBjXQJaOsRsIQYDsJCndL60qCN6nR4ylo+93TQhyITJxi90osByE65NrHSvluAUSjkxKg0Xt8f8aEUjiwVw8DQ= X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(36860700013)(82310400026)(376014)(1800799024); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR08MB9532 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DU2PEPF00028D01.eurprd03.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: af57656b-7920-49fe-d322-08de34e84301 X-Microsoft-Antispam: BCL:0; ARA:13230040|14060799003|82310400026|35042699022|1800799024|36860700013|376014; X-Microsoft-Antispam-Message-Info: utIzSf0hFFI3QMbRyU4g5oTOWrxZ8YZDMpb8PbdgjsY2WXYygxenIrsoz1eAzJtyyxQATsyej+3m8lK3Nw1boj8ic7fLR2/QztuBinfD2poTLTHQ+IL6Kh5P01vLh4BAgO+7jHc+7fsZ0Yat6XuH8CqhYJR1SsvBHAGeMQ5Y11aiXzZFUZywWQK3a9Bzs0/fg4A3AIgyaNQByTYQXf0qAHXuTzsv13plDq1Xn7ZRzDooRzZdkBrshCWvp0LC/D4SO+3UKRKzIFXBqF8T9vDs5PSqPVCTOfCbWqboBKkkbZt6O8YtB36vct4f6yyMARPt9C2b3s3twl8FEFq8w8y5dwMvhToeEZ/RzKdRk7Vxh0eVrB0qVwxuaBQOhb56wzpXEqj5LfaNR6ZYcQnipv81nGA6KcaZM7as1t37B7fb0Xgg1uA+5R1Itt3DKxbOFeavCPdwAZMAVWT8f67rMR/lgTVAXjNCTg/lujdBvTj/sKGkXyRKJ2uJipeMAELpTsI53VJoBUxe9NQ/DPR+ZT/gg0Dr3KG+LBcCx/JsXOcp8lZo0+OEdbaYCfmBioPkmVlfF3O0rmhkmM1f+lEe9XaopsRzcMXXyeSNxs4JpJo+oRlOPofwqICAfiL4pyv6Ww/6elZQQHFRjI6f/Y/0nvLb3/AVkrzXhEzR0Fw0GNdj4afBnw+b2rzM6xyTks5zJf1dyZMv5wHWt1u5DKGqhY2Y/GVg3ACMLAtDuPmIBIm/ay5A0B1SWUIwL91mVzfIwKdSmlkMbQ5N9TEsnlRJ5xftygNH9Nfgq8TVdx3WRmCp9sn6rtr9WmkxPguifPUoeBOWjQVFM7lsukPjc3cXLFoThCvMAfaqdMM1OHjgFm9zFAtJkt9DSlHwrPgKPcGi2SqpDE91s5jIffvhpyfmV7oovYypXznl7dLTDfut4JbzifFaOHqOeaRAdJXrbnC1BPtNXQXORhgjtEi2BtkkKEfLPdtW0nz8xPD3PJbWXgb58VQsW7GzRyAYRgmRf3IfZLKc4mGSjNYbh2IbTTrtMcwblkfQrnlxjfIcbHQzIpRJYxl3WmxCEXaCSIARdutHmtEztz9yruCRl9u9gLILd3TJVtm1nV8Yi+q+Y5BfuvsogiYXhowDPI06bpRup3+cVdcA/vtgCKCMxWWmYjHfINHaK5NI2Jm74AJjeABxMxKnTKKlQUUPSSUZs5XOrDJHVU4rkg8WeosMn06WwFe7ykZRy73YDYBXXxkWotOiBN7y5vxZjcg4TfK/9gCmDs4+X3D8TdKKyTfRCo52U3d4PVvwp7Cdzy+j7ZjZbp3SvIz0bzsH883KGtCMEVPvKG2S0ZYQp9IOdW8VJiYFVyimile0jNIDS7bDJC79HamW7wqbB67mVC0Vn1Xzf3j+xoZHFcuIeazs17CyNbY0mQZYb28syA4aBjEhaZDy2Z2cBlq7wvWXbF+YwnrZrF+UR6c+6Qm3fNUt/bIUe8CpyhrY/VmMQ5zRmdv170u4cPgz5q87gNbLoXnakkOEl+tLAgY0DdAkbV6CBzcr6h4lZxRQ5UHzbDUlBpz6VJI9zhWRh5aPqK8= X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(14060799003)(82310400026)(35042699022)(1800799024)(36860700013)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:56:37.5446 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0014cd41-04b5-4bfb-b34a-08de34e86ba1 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028D01.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR08MB8143 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org Add two new fields to SLP tree nodes, which are accessed as SLP_TREE_CAN_USE_PARTIAL_VECTORS_P and SLP_TREE_PARTIAL_VECTORS_STYLE. SLP_TREE_CAN_USE_PARTIAL_VECTORS_P is analogous to the existing predicate LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. It is initialized to true. This flag just records whether the target could vectorize a node using a partial vector; it does not say anything about whether the vector actually is partial, or how the target would support use of a partial vector. Some kinds of node require mask/length for partial vectors; others don't. In the latter case (e.g., for add operations), SLP_TREE_CAN_USE_PARTIAL_VECTORS_P will remain true. SLP_TREE_PARTIAL_VECTORS_STYLE is analogous to the existing field LOOP_VINFO_PARTIAL_VECTORS_STYLE. Both are initialized to 'none'. The vect_partial_vectors_avx512 enumerator is not used for BB SLP. Unlike loop vectorization, a different style of partial vectors can be chosen for each node during analysis of that node. Implement the recently-introduced wrapper functions, vect_record_(len|mask), for BB SLP by setting SLP_TREE_PARTIAL_VECTORS_STYLE to indicate that a mask or length should be used for a given SLP node. The passed-in vec_info is ignored. Implement the vect_fully_(masked|with_length)_p wrapper functions for BB SLP by checking the SLP_TREE_PARTIAL_VECTORS_STYLE. This should be sufficient because at most one of vect_record_(len|mask) and vect_cannot_use_partial_vectors are expected to be called for any given SLP node. SLP_TREE_CAN_USE_PARTIAL_VECTORS_P should be true if the style is not 'none', but its value isn't used beyond the analysis phase. The implementations of vect_get_mask and vect_get_len for BB SLP are non-trivial (albeit simpler than for loop vectorization), therefore they are delegated to SLP-specific functions defined in tree-vect-slp.cc. Implement the vect_cannot_use_partial_vectors wrapper function by setting the SLP_TREE_CAN_USE_PARTIAL_VECTORS_P flag to false. To prevent regressions, vect_can_use_partial_vectors_p still returns false for BB SLP regardless (for now). This prevents vect_record_mask or vect_record_len from being called. gcc/ChangeLog: * tree-vect-slp.cc (_slp_tree::_slp_tree): initialize new partial_vector_style and can_use_partial_vectors members. (vect_slp_record_bb_style): Set the partial vector style of an SLP node, checking that the style does not flip-flop between mask and length. (vect_slp_record_bb_mask): Use vect_slp_record_bb_style to set the partial vectors style of the SLP tree node to vect_partial_vectors_while_ult. (vect_slp_get_bb_mask): New function to materialize a mask for basic block SLP vectorization. (vect_slp_record_bb_len): Use vect_slp_record_bb_style to set the partial vectors style of the SLP tree node to vect_partial_vectors_len. (vect_slp_get_bb_len): New function to materialize a length for basic block SLP vectorization. * tree-vect-stmts.cc (vectorizable_internal_function): (vect_record_mask): Handle the basic block SLP use case by delegating to vect_slp_record_bb_mask. (vect_get_mask): Handle the basic block SLP use case by delegating to vect_slp_get_bb_mask. (vect_record_len): Handle the basic block SLP use case by delegating to vect_slp_record_bb_len. (vect_get_len): Handle the basic block SLP use case by delegating to vect_slp_get_bb_len. (vect_gen_while_ssa_name): New function containing code refactored out of vect_gen_while for reuse by vect_slp_get_bb_mask. (vect_gen_while): Use vect_gen_while_ssa_name instead of custom code for some of the implementation. * tree-vectorizer.h (enum vect_partial_vector_style): Move this definition earlier to allow reuse by struct _slp_tree. (struct _slp_tree): Add a partial_vector_style member to record whether to use a length or mask for the SLP tree node, if partial vectors are required and supported. Add a can_use_partial_vectors member to record whether partial vectors are supported for the SLP tree node. (SLP_TREE_PARTIAL_VECTORS_STYLE): New member accessor macro. (SLP_TREE_CAN_USE_PARTIAL_VECTORS_P): New member accessor macro. (vect_gen_while_ssa_name): Declaration of a new function. (vect_slp_get_bb_mask): As above. (vect_slp_get_bb_len): As above. (vect_cannot_use_partial_vectors): Handle the basic block SLP use-case by setting SLP_TREE_CAN_USE_PARTIAL_VECTORS_P to false. (vect_fully_with_length_p): Handle the basic block SLP use case by checking whether the SLP_TREE_PARTIAL_VECTORS_STYLE is vect_partial_vectors_len. (vect_fully_masked_p): Handle the basic block SLP use case by checking whether the SLP_TREE_PARTIAL_VECTORS_STYLE is vect_partial_vectors_while_ult. --- gcc/tree-vect-slp.cc | 121 +++++++++++++++++++++++++++++++++++++++++ gcc/tree-vect-stmts.cc | 51 ++++++++++------- gcc/tree-vectorizer.h | 44 +++++++++------ 3 files changed, 179 insertions(+), 37 deletions(-) diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index 658ad6dc257..57adef2b5d1 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -124,6 +124,8 @@ _slp_tree::_slp_tree () SLP_TREE_GS_BASE (this) = NULL_TREE; this->ldst_lanes = false; this->avoid_stlf_fail = false; + SLP_TREE_PARTIAL_VECTORS_STYLE (this) = vect_partial_vectors_none; + SLP_TREE_CAN_USE_PARTIAL_VECTORS_P (this) = true; SLP_TREE_VECTYPE (this) = NULL_TREE; SLP_TREE_REPRESENTATIVE (this) = NULL; this->cycle_info.id = -1; @@ -12519,3 +12521,122 @@ vect_schedule_slp (vec_info *vinfo, const vec &slp_instances) } } } + +/* Record that a specific partial vector style could be used to vectorize + SLP_NODE if required. */ + +static void +vect_slp_record_bb_style (slp_tree slp_node, vect_partial_vector_style style) +{ + gcc_assert (style != vect_partial_vectors_none); + gcc_assert (style != vect_partial_vectors_avx512); + + if (SLP_TREE_PARTIAL_VECTORS_STYLE (slp_node) == vect_partial_vectors_none) + SLP_TREE_PARTIAL_VECTORS_STYLE (slp_node) = style; + else + gcc_assert (SLP_TREE_PARTIAL_VECTORS_STYLE (slp_node) == style); +} + +/* Record that a mask could be used to vectorize SLP_NODE if required. + */ + +void +vect_slp_record_bb_mask (slp_tree slp_node) +{ + vect_slp_record_bb_style (slp_node, vect_partial_vectors_while_ult); +} + +/* Materialize mask number INDEX for a group of scalar stmts in SLP_NODE that + operate on NVECTORS vectors of type VECTYPE, where 0 <= INDEX < NVECTORS. + Masking is only required for the tail, therefore NULL_TREE is returned for + every value of INDEX except the last. Insert any set-up statements before + GSI. */ + +tree +vect_slp_get_bb_mask (slp_tree slp_node, gimple_stmt_iterator *gsi, + unsigned int nvectors, tree vectype, unsigned int index) +{ + gcc_checking_assert (SLP_TREE_PARTIAL_VECTORS_STYLE (slp_node) + == vect_partial_vectors_while_ult); + + /* Only the last vector can be a partial vector. */ + if (index < nvectors - 1) + return NULL_TREE; + + /* vect_get_num_copies only allows a partial vector if it is the only + vector. */ + if (nvectors > 1) + return NULL_TREE; + + gcc_checking_assert (nvectors == 1); + + poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); + unsigned int group_size = SLP_TREE_LANES (slp_node); + + /* A single vector can be a full vector, in which case no mask is + * needed. */ + if (known_eq (nunits, group_size)) + return NULL_TREE; + + /* Return a mask for a single partial vector. + FORNOW: don't bother maintaining a set of mask constants to allow + sharing between nodes belonging to the same instance of bb_vec_info. */ + gcc_checking_assert (known_le (group_size, nunits)); + gimple_seq stmts = NULL; + tree cmp_type = size_type_node; + tree start_index = build_zero_cst (cmp_type); + tree end_index = build_int_cst (cmp_type, group_size); + tree masktype = truth_type_for (vectype); + tree mask = make_temp_ssa_name (masktype, NULL, "slp_mask"); + vect_gen_while_ssa_name (&stmts, masktype, start_index, end_index, mask); + gsi_insert_seq_before (gsi, stmts, GSI_SAME_STMT); + + return mask; +} + +/* Record that a length limit could be used to vectorize SLP_NODE if required. + */ + +void +vect_slp_record_bb_len (slp_tree slp_node) +{ + vect_slp_record_bb_style (slp_node, vect_partial_vectors_len); +} + +/* Materialize length number INDEX for a group of scalar stmts in SLP_NODE that + operate on NVECTORS vectors of type VECTYPE, where 0 <= INDEX < NVECTORS. A + length limit is only required for the tail, therefore NULL_TREE is returned + for every value of INDEX except the last; otherwise, return a value that + contains FACTOR multiplied by the number of elements that should be + processed. */ + +tree +vect_slp_get_bb_len (slp_tree slp_node, unsigned int nvectors, tree vectype, + unsigned int index, unsigned int factor) +{ + gcc_checking_assert (SLP_TREE_PARTIAL_VECTORS_STYLE (slp_node) + == vect_partial_vectors_len); + + /* Only the last vector can be a partial vector. */ + if (index < nvectors - 1) + return NULL_TREE; + + /* vect_get_num_copies only allows a partial vector if it is the only + vector. */ + if (nvectors > 1) + return NULL_TREE; + + gcc_checking_assert (nvectors == 1); + + poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); + unsigned int group_size = SLP_TREE_LANES (slp_node); + + /* A single vector can be a full vector, in which case no length limit is + * needed. */ + if (known_eq (nunits, group_size)) + return NULL_TREE; + + /* Return the scaled length of a single partial vector. */ + gcc_checking_assert (known_lt (group_size, nunits)); + return size_int (group_size * factor); +} diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 078cc63b2d9..bd63451b467 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -1387,7 +1387,9 @@ vectorizable_internal_function (combined_fn cfn, tree fndecl, /* Record that a complete set of masks associated with VINFO would need to contain a sequence of NVECTORS masks that each control a vector of type VECTYPE. If SCALAR_MASK is nonnull, the fully-masked loop would AND - these vector masks with the vector version of SCALAR_MASK. */ + these vector masks with the vector version of SCALAR_MASK. Alternatively, + if doing basic block vectorization, record that a mask could be used to + vectorize SLP_NODE if required. */ static void vect_record_mask (vec_info *vinfo, slp_tree slp_node, unsigned int nvectors, tree vectype, tree scalar_mask) @@ -1397,7 +1399,7 @@ vect_record_mask (vec_info *vinfo, slp_tree slp_node, unsigned int nvectors, vect_record_loop_mask (loop_vinfo, &LOOP_VINFO_MASKS (loop_vinfo), nvectors, vectype, scalar_mask); else - (void) slp_node; /* FORNOW */ + vect_slp_record_bb_mask (slp_node); } /* Given a complete set of masks associated with VINFO, extract mask number @@ -1415,16 +1417,15 @@ vect_get_mask (vec_info *vinfo, slp_tree slp_node, gimple_stmt_iterator *gsi, return vect_get_loop_mask (loop_vinfo, gsi, &LOOP_VINFO_MASKS (loop_vinfo), nvectors, vectype, index); else - { - (void) slp_node; /* FORNOW */ - return NULL_TREE; - } + return vect_slp_get_bb_mask (slp_node, gsi, nvectors, vectype, index); } /* Record that a complete set of lengths associated with VINFO would need to contain a sequence of NVECTORS lengths for controlling an operation on VECTYPE. The operation splits each element of VECTYPE into FACTOR separate - subelements, measuring the length as a number of these subelements. */ + subelements, measuring the length as a number of these subelements. + Alternatively, if doing basic block vectorization, record that a length limit + could be used to vectorize SLP_NODE if required. */ static void vect_record_len (vec_info *vinfo, slp_tree slp_node, unsigned int nvectors, tree vectype, unsigned int factor) @@ -1434,7 +1435,7 @@ vect_record_len (vec_info *vinfo, slp_tree slp_node, unsigned int nvectors, vect_record_loop_len (loop_vinfo, &LOOP_VINFO_LENS (loop_vinfo), nvectors, vectype, factor); else - (void) slp_node; /* FORNOW */ + vect_slp_record_bb_len(slp_node); } /* Given a complete set of lengths associated with VINFO, extract length number @@ -1455,10 +1456,7 @@ vect_get_len (vec_info *vinfo, slp_tree slp_node, gimple_stmt_iterator *gsi, return vect_get_loop_len (loop_vinfo, gsi, &LOOP_VINFO_LENS (loop_vinfo), nvectors, vectype, index, factor); else - { - (void) slp_node; /* FORNOW */ - return NULL_TREE; - } + return vect_slp_get_bb_len (slp_node, nvectors, vectype, index, factor); } static tree permute_vec_elements (vec_info *, tree, tree, tree, stmt_vec_info, @@ -14499,24 +14497,35 @@ supportable_indirect_convert_operation (code_helper code, mask[I] is true iff J + START_INDEX < END_INDEX for all J <= I. Add the statements to SEQ. */ +void +vect_gen_while_ssa_name (gimple_seq *seq, tree mask_type, tree start_index, + tree end_index, tree ssa_name) +{ + tree cmp_type = TREE_TYPE (start_index); + gcc_checking_assert (direct_internal_fn_supported_p (IFN_WHILE_ULT, cmp_type, + mask_type, + OPTIMIZE_FOR_SPEED)); + gcall *call + = gimple_build_call_internal (IFN_WHILE_ULT, 3, start_index, end_index, + build_zero_cst (mask_type)); + gimple_call_set_lhs (call, ssa_name); + gimple_seq_add_stmt (seq, call); +} + +/* Like vect_gen_while_ssa_name except that it creates a new SSA_NAME node + for type MASK_TYPE defined in the created GIMPLE_CALL statement. If NAME + is not a null pointer then it is used for the SSA_NAME in dumps. */ + tree vect_gen_while (gimple_seq *seq, tree mask_type, tree start_index, tree end_index, const char *name) { - tree cmp_type = TREE_TYPE (start_index); - gcc_checking_assert (direct_internal_fn_supported_p (IFN_WHILE_ULT, - cmp_type, mask_type, - OPTIMIZE_FOR_SPEED)); - gcall *call = gimple_build_call_internal (IFN_WHILE_ULT, 3, - start_index, end_index, - build_zero_cst (mask_type)); tree tmp; if (name) tmp = make_temp_ssa_name (mask_type, NULL, name); else tmp = make_ssa_name (mask_type); - gimple_call_set_lhs (call, tmp); - gimple_seq_add_stmt (seq, call); + vect_gen_while_ssa_name (seq, mask_type, start_index, end_index, tmp); return tmp; } diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 2d914dca90b..1830c29819a 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -307,6 +307,13 @@ struct vect_load_store_data : vect_data { bool subchain_p; // VMAT_STRIDED_SLP and VMAT_GATHER_SCATTER }; +enum vect_partial_vector_style { + vect_partial_vectors_none, + vect_partial_vectors_while_ult, + vect_partial_vectors_avx512, + vect_partial_vectors_len +}; + /* A computation tree of an SLP instance. Each node corresponds to a group of stmts to be packed in a SIMD stmt. */ struct _slp_tree { @@ -368,6 +375,13 @@ struct _slp_tree { /* For BB vect, flag to indicate this load node should be vectorized as to avoid STLF fails because of related stores. */ bool avoid_stlf_fail; + /* The style used for implementing partial vectors if LANES is less than + the minimum number of lanes implied by the VECTYPE. */ + vect_partial_vector_style partial_vector_style; + /* Flag to indicate whether we still have the option of vectorizing this node + using partial vectors (i.e. using lengths or masks to prevent use of + inactive scalar lanes). */ + bool can_use_partial_vectors; int vertex; @@ -466,6 +480,8 @@ public: #define SLP_TREE_GS_BASE(S) (S)->gs_base #define SLP_TREE_REDUC_IDX(S) (S)->cycle_info.reduc_idx #define SLP_TREE_PERMUTE_P(S) ((S)->code == VEC_PERM_EXPR) +#define SLP_TREE_PARTIAL_VECTORS_STYLE(S) (S)->partial_vector_style +#define SLP_TREE_CAN_USE_PARTIAL_VECTORS_P(S) (S)->can_use_partial_vectors inline vect_memory_access_type SLP_TREE_MEMORY_ACCESS_TYPE (slp_tree node) @@ -476,13 +492,6 @@ SLP_TREE_MEMORY_ACCESS_TYPE (slp_tree node) return VMAT_UNINITIALIZED; } -enum vect_partial_vector_style { - vect_partial_vectors_none, - vect_partial_vectors_while_ult, - vect_partial_vectors_avx512, - vect_partial_vectors_len -}; - /* Key for map that records association between scalar conditions and corresponding loop mask, and is populated by vect_record_loop_mask. */ @@ -2576,6 +2585,7 @@ extern tree vect_gen_perm_mask_checked (tree, const vec_perm_indices &); extern void optimize_mask_stores (class loop*); extern tree vect_gen_while (gimple_seq *, tree, tree, tree, const char * = nullptr); +extern void vect_gen_while_ssa_name (gimple_seq *, tree, tree, tree, tree); extern tree vect_gen_while_not (gimple_seq *, tree, tree, tree); extern opt_result vect_get_vector_types_for_stmt (vec_info *, stmt_vec_info, tree *, @@ -2760,6 +2770,12 @@ extern slp_tree vect_create_new_slp_node (unsigned, tree_code); extern void vect_free_slp_tree (slp_tree); extern bool compatible_calls_p (gcall *, gcall *, bool); extern int vect_slp_child_index_for_operand (const gimple *, int op, bool); +extern void vect_slp_record_bb_mask (slp_tree slp_node); +extern tree vect_slp_get_bb_mask (slp_tree, gimple_stmt_iterator *, + unsigned int, tree, unsigned int); +extern void vect_slp_record_bb_len (slp_tree slp_node); +extern tree vect_slp_get_bb_len (slp_tree, unsigned int, tree, unsigned int, + unsigned int); extern tree prepare_vec_mask (vec_info *, tree, tree, tree, gimple_stmt_iterator *); extern tree vect_get_mask_load_else (int, tree); @@ -2924,7 +2940,7 @@ vect_cannot_use_partial_vectors (vec_info *vinfo, slp_tree slp_node) if (loop_vinfo) LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; else - (void) slp_node; /* FORNOW */ + SLP_TREE_CAN_USE_PARTIAL_VECTORS_P (slp_node) = false; } /* Return true if VINFO is vectorizer state for loop vectorization, we've @@ -2938,10 +2954,8 @@ vect_fully_with_length_p (vec_info *vinfo, slp_tree slp_node) if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) return LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo); else - { - (void) slp_node; /* FORNOW */ - return false; - } + return SLP_TREE_PARTIAL_VECTORS_STYLE (slp_node) + == vect_partial_vectors_len; } /* Return true if VINFO is vectorizer state for loop vectorization, we've @@ -2955,10 +2969,8 @@ vect_fully_masked_p (vec_info *vinfo, slp_tree slp_node) if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) return LOOP_VINFO_FULLY_MASKED_P (loop_vinfo); else - { - (void) slp_node; /* FORNOW */ - return false; - } + return SLP_TREE_PARTIAL_VECTORS_STYLE (slp_node) + == vect_partial_vectors_while_ult; } /* If STMT_INFO describes a reduction, return the vect_reduction_type From patchwork Sat Dec 6 16:55:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126051 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 26F3542FA7AF for ; Sat, 6 Dec 2025 17:02:13 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 26F3542FA7AF Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=GIaURJZK; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=GIaURJZK X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazon11013039.outbound.protection.outlook.com [40.107.162.39]) by sourceware.org (Postfix) with ESMTPS id 2BB3A42DB85F for ; Sat, 6 Dec 2025 16:56:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2BB3A42DB85F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2BB3A42DB85F Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.162.39 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040203; cv=pass; b=S7hTCIuGLCL2/ils7BrEJNBrxC6BMe/nfo2VzTT1izRBw12D77dkJ2vNj3ZtLbeCuOXK38srx/QJt4p5rS+IZZZleUZAxPhbfMcEB5V7jnRMoVmCp6lflKLmdI0vNhPlbHB0Hw6ETDNFkfThZKqXxh1GvSlJX+JpyT9q/DTmIL8= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040203; c=relaxed/simple; bh=yWfv2wrspma2qWczJj+yfZnAIkh1sRw35Zs2t/Id9Ws=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=CgDoU7Et+5m6JCsb7kS920Sn5r5qlsVTq2i3+on2aaM0vBXv1I/S6cjDqKGLsmZW6NbE9LGzdbdxX2onAcizO8WRNbc1RgVNQXmE3QEkvPz8DJ/2zEqAaoG2CXIX1SmpGqv1aBg8ySvR9Okpwe84oOygGhmY0fd1GKecdmScROY= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2BB3A42DB85F ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=EbU055m/7Us2Uu0RIE3HdgJsz52zlmrLZEsQfGjs5pgofFQG6IXYJwHblRkxj242SEnw1qa4mWdUo0CBQqsJwWOAjGqXue4Ixr6XEST9uAH+no659jYpJzPFa6t7xvo9wdHAFpibE86uNEjfk/iWsMdC4sZR6AfiBhsiwkOGVNOeYHNcpharevvE6Zw4eWntKh2eqxeAwXnQoupfMnqoTHo6B41eOX1rpwd+qo0SltaeUW0RqwdHwD5prDwpeMFqSesUqfZepka2oLbnGjFMSQLv8puOtjH1dhXqs/u3Ai2nDGIrR46/iXXsDMrE+QTFnvYr0oZaEJobVLl6U8eLQA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sqn39WPK+CdnMeYeEwDWo1ZfCFaQF8rz1S5msuNH03g=; b=Vx8YgdZEXFl7lExAhJkb/ZlrwIJuo/iPU1AVqZgfAszccHi7ipLHHcmqbjdDAar0P3Q9XSrLBNYcvY1Nuu5kXcPC4OMmkFS+rXraYSCYGS/M1E3NJXuQkPLAWW/aClKFv6fP/Eu4LCNUZC40mKa74lwU3852EHsb9f8Hr9v9nHn3INI/8jXFpDO16vSKgXtEwuzOowxdAPfAgS/K6s5I77cWIZorm97Sy+7+zRM10KbuO7P9e5ZgTv0ml4XwEHWwean8Xdo8D0HqcQFkht91jG2krhVaoZ52Qv8RGIzCkzrsc9ecXZIyfiqeYUyvRA0HvpxC+cUHpHLdcupfcnUcwQ== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sqn39WPK+CdnMeYeEwDWo1ZfCFaQF8rz1S5msuNH03g=; b=GIaURJZK0UZIJuJEdcnwSslu1+TB+TBBy5LSVvKNxvrCLSasrZ5WXPpPpwmn/coiBHYBibPXiVNdCepYuV/9SAX/UXZXnwPgz+iw6jBCxK+YzpRvb2+eV7agwtCoM4JmHGLTPEsqq+YkbavOp5yRigWx4JuY8GQ22Hraf9Bk1gE= Received: from DUZPR01CA0162.eurprd01.prod.exchangelabs.com (2603:10a6:10:4bd::10) by DU0PR08MB7835.eurprd08.prod.outlook.com (2603:10a6:10:3b4::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9366.17; Sat, 6 Dec 2025 16:56:39 +0000 Received: from DB1PEPF000509FD.eurprd03.prod.outlook.com (2603:10a6:10:4bd:cafe::68) by DUZPR01CA0162.outlook.office365.com (2603:10a6:10:4bd::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by DB1PEPF000509FD.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:56:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jWuUbRsPIIvWI+DUwbsYeoNr1bKfReGsIq6SXnO/X8+b/WdPa1RDbifD55sq05jaucz4r80llGBa7SMNAoAFbuDTVf7Xqk+l6K01fo0TYVBf0gjHL0hLEyLvj/g7W2jmMr0y6VsP0sVaVSb4W8v3owdQVFg9JiC7v45J982F1ld3dYsxQp7xrJmB89ZVmKB7CS38AtX3rM7EhlDhegNTZzveaghVOoPv4vPOBsNw99ZW4NxODKytuF6j9ctwHtZ1zch+9ppPoMQ10V+sZ5XU1HkgCmAJQNhMqUjrm4oKBsZa2fCUyT0xIndhUr9fBEK2Drm4/AAyGQpE5yNn8Zzvrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sqn39WPK+CdnMeYeEwDWo1ZfCFaQF8rz1S5msuNH03g=; b=iGcCWNi/YvD5niuB7Ta1CGrf0IYKzLe8Bdp2RBXS1HI+9iFsenTZFUnqydfcUSQOXTx/lyOZsbdYNwKyZoitLXlQkURVhNUp1WyVDfkOqT+UdcjPRv5B90tMbBXQXVw2RVE189WcdrP+tXpVlRDuFYCmSBbdl5E+oA6Jqu0/ehMVG3IGIzKo1oK8iL31t2MknMCY3kZbqDvF8UZg/U1kXuIz3FAquPru4zFQu2UcEviypMUddOqlzCcOj8cMoDFfXT4+QidJAgLcQB1Y6k0+h0k1tcSD/XjVMWVLu7DTBAcD4lBTkXoThKi7mcEA4DlrscrywCkfkbXL0LAKCSBWmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sqn39WPK+CdnMeYeEwDWo1ZfCFaQF8rz1S5msuNH03g=; b=GIaURJZK0UZIJuJEdcnwSslu1+TB+TBBy5LSVvKNxvrCLSasrZ5WXPpPpwmn/coiBHYBibPXiVNdCepYuV/9SAX/UXZXnwPgz+iw6jBCxK+YzpRvb2+eV7agwtCoM4JmHGLTPEsqq+YkbavOp5yRigWx4JuY8GQ22Hraf9Bk1gE= Received: from DU7P191CA0006.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:54e::34) by GV1PR08MB10572.eurprd08.prod.outlook.com (2603:10a6:150:169::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:55:35 +0000 Received: from DB1PEPF00039230.eurprd03.prod.outlook.com (2603:10a6:10:54e:cafe::24) by DU7P191CA0006.outlook.office365.com (2603:10a6:10:54e::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:55:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF00039230.mail.protection.outlook.com (10.167.8.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:55:34 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX03.Arm.com (10.240.25.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:34 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:55:34 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 03/11] Make BIT_FIELD_REF CONSTRUCTOR pattern more robust Date: Sat, 6 Dec 2025 16:55:10 +0000 Message-ID: <20251206165518.5449-4-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF00039230:EE_|GV1PR08MB10572:EE_|DB1PEPF000509FD:EE_|DU0PR08MB7835:EE_ X-MS-Office365-Filtering-Correlation-Id: f1f0cc48-dc43-4a81-dd59-08de34e86c3e x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|376014|36860700013|82310400026|1800799024; X-Microsoft-Antispam-Message-Info-Original: IRUPZmLDK2Ex68UJ2yz+nIrlY4tj8BCPbV2Ur6JAwpKRlLM7BZzI2kPBUDqzc2JOrt8XI7khDkiCcF4Y9AV6ozTXENByPrMO3+e3GNAOfnl3J7yxiSah4fDX1Evu3Ovlmc42wXTGHZ2QbT8liAdwgOoCpmK43G9UtZdQZ/siYI+i60hwFJUAGxfX1rC2RjXEoE10k4Y+k2ROwqYBcSjFCmRtdtIQX5JuoLs3Eri3kRGMups7aEksMlVPJjbRGBpNdVMrcXWY2ItwiD351c2n1Fxki2LSjb0o1wenNguBXoG8OSfH9+kOIXCGHjpK8HA4SqehddLAR0wLJkEcTllmxBREdnPRsyb/QfQa4+DAKhl+qfFvfaNZwIbo035z2qLgiQivCIkXea1OKDmK+ZCKEFHMjAlVu575QEd5kcEzfYPmHjLZAowoh7QTC1eKjHGgMn7t1gan5JFJhrF/igOgu7nxu+/Kjz6sHlKq55oPWEcdCgFZHpdKpVGpMHLBE+ZMyZyXuLivF6FmQ+P8bhjtQnZMo89elZqb7R+WO8savsAky3/enA669N/lOOTeqPc8ccK0es3xLrYZLYd0d0VAEZz2JgxSgdDf6xbM7gOXfzNxLQcKjGz09voPGU/nEnKdBQ8/cHWFoG2Pr8xqqm5rxk2ev+A6+6g/+xwwRChgQKz0mtCppv3zp36cR4GwpHygEZrqwyYLSakJjr9mRi13W9O8FW8gu6qutvjKZFIRIm67c3gxyn9vblgSzDCa1qfu8QbSIqG0QBznEPV9AKpB2AaatEeoTBK4QwWED6Gpemz4BZlhWuR4mwpCId5Nm/4S8J9+nC6zNIGl223pPjAc2rHZbIgGofVdOqcIWSdzRKO+Na2c68nMqZhGsjvKliFtNxmarGOaWJdwpaw45UZPP/RbUTG3BsCDA9R0oF0ivPXiQE+uRNy2uIVJSzbC8zCcz8UG2RrpnH+bNMEn7EMP28R7W/4X99whBztM6sKu4KqbrbmxChTlIMynR6A83/dUqe90wE7Pmy3Vb8gOi2ZVBHrvITEIQGCxp5Nry3rt4I5EAH+F/89K2LCTLldbB746X7UQhwV8HbJ+fjymQWGkisMYp7Zqz+oZJSD55lXxFL8+PY2qvjBCAzOb49BravUmjm2Tpgmabx6v+DUCxrY4SdEOO/1qT6UvIhxiZuEt/5EZm7kdnuMNlsWiiDigitIdPilHE2dcKSB6rGGmLcQrLiQ0k3bJJvaGKVWTaSKSnLiZ0bG2qxem4LrYfoJUl+yvZfn5CKG8+TDM9CE6kUQRhbg1tvk+gvndR14bQdK/0WUEh3cUjVOWHR2Vg+oOxVk/yvfDFdi7fRVxEFqSAQp/3muVg5cqf6tGkiDTSLPJnslOausGaUOryfJpZM9zuL1Y/GGulmDy74UhwB/VrLnesN4QZ9mLmNioxeelIDWRs8cvZ55FnfJ5a0gkB8GWNxDYZjc7CXTsD4pRMOOzyPmE1pcaRGR8z7Y3FHLFzH6dnWR+6/HrVSE1DF1VstqEUeEv2DgkS4upshtLYn10hOND75WoUP+mWldXzob2TQeB+qY= X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(36860700013)(82310400026)(1800799024); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB10572 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB1PEPF000509FD.eurprd03.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 2686a28c-db95-4a0f-d144-08de34e8462f X-Microsoft-Antispam: BCL:0; ARA:13230040|14060799003|82310400026|36860700013|376014|35042699022|1800799024; X-Microsoft-Antispam-Message-Info: nwMR/ANp+Jk88swWQ4Nh8sZO0kHVPcd2+tGPiQ7qTwln+gdTjTJB0QD5i6sSKN1bAWP78m87xXJcmSz+mE0dOSZ3V6BT0sCkOiA0AYIDzt97vne4gubrk9dFHg2n1z13QlQLEV14IIItFfVSLdyyq8yXrgy2+eRm7DgkYi6sEvbGbyNinqlIHTcmiLVRHMuZ9AKCwMDor0UBNr3x0+2m9tTjpBfxO01Vuxe2HNlrJF/xxMI18kNhEGMQUf/4lUfxLn54NkvPl50ZkeDwTtwuYMS2minNdE15GxKVy3AMwr76MCsQt90hUbmn6rQU9+SDCq1w74Q3R+O3yV93JcxdZDzutNAFxVfBMpR7QCGra+BE0zba9YieXa2Jh6puKepZ8PfI6Pozmc4sfM/sYGzZ6hA4ews9PQoH7aFrS3xKgDjR4V/GpwcDJFpeng2RWYTAz0f/jULzwv5z8d5peX75kOmiXoV024NykLai4nz2J3d1CfULUiB6iBX4OHJsTjRwy84GJYuFTHC6grGr3CB5XadeB3azNtDwy5HQNJp8YbPlo6oCx/xgbHajwaQl8j9/yhlHzsur+1yxfpgSSL+nMWvHyBpsH5p1QUvdmxWLzsWKzmL4Gp5l8+r/stxI3WSAlMYvOg5JyRs3CHzkdRjI813/eqsKawJrpkeLVClEmVQzukmgDQTam8tmQ8b4Z/CLBq5zdCtlT6YQRGqn+imDcopx4UpecDvrv/jiglS1/7IFWLOL0B+xDcFdLvSs7Addk+D7nNhwJNFEi0ijMRqMTR0VnY4CI+ZAgiiKz0xCpgTObZsuuIbypGI/1PZloj2okcQFfMNeJy0nEELyignCN5/BCHq315Ld5AqQ0SXXdTQzgn/jGt7CT2DU1S79lJGGNkXkGUq8AEmoai8lIqPJZj0gvpTElBEjICRwH00+Qq1S8ualrz4fyk3byPCTkHY0pKj5xJVwUL7UsiOrzCnJQ3DFKzyULsA7XpFyiVEe65+jnoW2QOimQ1EXqE60yo6kdGpALXpd194nzMLSo/MrhW4hidGHAFhQArY5+dt+VmkoC/Yio+Kb+b+Rtio8qh9TgfCmhYHuWWMv4Fc7jf/+OwMVzEW6WxegBeHi9+2KwunZTM1X0aON4QG3NCgfVWUdmpWwb9W9fRNBkBnbyYArNH5LhZYl2qcyL3ib4ZRQNkgZfHgMF24SShlwgxpGDRuYNOv4am38veBxUTWiG6R2FZTe7enVSx6O92afvoSbCmw4noxSKNmomX2M/0UQKPfJVY2B42rAK21ItZlOcPcspxKICS9iMMBuId3mh2nIS7XTtWQeDgu7BO5lr+1RyGy4ie5oDaShOqSy6LvFsFj+GgIBMPnr+A7+o60xaMpyF/LZxlykSDMdjsN5wRIvooKjAGKedpN03wvzR3zqqTJvm1N9lJ6IzdoLBkJ9Mgq5VGglfq6ZZFeCmh8ieCp9dmYsotwO/2rIHSD7mtq1tIveKCgiZvDPUyfQcyfd95bJsMjEw5ozFTtvX552FGZ19caDd6WwCgRgdJ1fdGdTvTuJAwRHEbQHTICA12gPVU26yeM= X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(14060799003)(82310400026)(36860700013)(376014)(35042699022)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:56:38.5837 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f1f0cc48-dc43-4a81-dd59-08de34e86c3e X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509FD.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB7835 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org e7301f5fcbfa6 required the sizes and offsets of a BIT_FIELD_REF to be poly_uint64s rather than uhwis. When support for predicated vector tails with BB SLP is enabled, GIMPLE will sometimes contain statements such as: float _246; vector([4,4]) float _251; _251 = { _234, _236, _238, _84 } ; 246 = BIT_FIELD_REF <_251, 32, POLY_INT_CST [96, 128]> Update an existing pattern named 'Simplify vector extracts' to guard against invalid invocations of tree_to_uhwi in such circumstances (by checking beforehand with tree_fits_uhwi_p). gcc/ChangeLog: * match.pd: Require the offset and size parameters of a BIT_FIELD_REF to fit type unsigned HOST_WIDE_INT before attempting to convert them. --- gcc/match.pd | 2 ++ 1 file changed, 2 insertions(+) diff --git a/gcc/match.pd b/gcc/match.pd index f164ec59100..db7f4374bbf 100644 --- a/gcc/match.pd +++ b/gcc/match.pd @@ -9969,6 +9969,8 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT) (BIT_FIELD_REF CONSTRUCTOR@0 @1 @2) (if (VECTOR_TYPE_P (TREE_TYPE (@0)) && tree_fits_uhwi_p (TYPE_SIZE (type)) + && tree_fits_uhwi_p (@1) + && tree_fits_uhwi_p (@2) && ((tree_to_uhwi (TYPE_SIZE (type)) == tree_to_uhwi (TYPE_SIZE (TREE_TYPE (TREE_TYPE (@0))))) || (VECTOR_TYPE_P (type) From patchwork Sat Dec 6 16:55:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126048 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B7E004209E2D for ; Sat, 6 Dec 2025 16:58:40 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B7E004209E2D Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=akbZExZO; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=akbZExZO X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from AM0PR83CU005.outbound.protection.outlook.com (mail-westeuropeazon11010067.outbound.protection.outlook.com [52.101.69.67]) by sourceware.org (Postfix) with ESMTPS id BAC99430766D for ; Sat, 6 Dec 2025 16:56:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BAC99430766D Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BAC99430766D Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.69.67 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040211; cv=pass; b=qm9IQnXGGvUbOEBCxvnV39wFkwQE7lIBlbfeQOZzyTUiyFuxosfRInODetDEDfamgElRqPfDY6Zn7mqcKnOiaJa3k3f9jnhimad2iXDVqeaiQhQh+wCWyJi0qGmX5P4W83EzcDZp2/ZJfmqrBx5q2b3LwGZx1bCNMPrqMjzgZQw= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040211; c=relaxed/simple; bh=NxgNWy7HgbAhD+SlF5JBBX/0mf8mv5ZlJ3Wr7kHEBOI=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=d09znfwWPd0sE1t20m8ByTVqzTY+XqWJ2I5qywU4f0en+xwe+x33LpuMHUe/EDRdL995rAcGNcH1W8UaVpeAW5XczR9MNCyAdeRR80IIZ8SZ5m5d1p0te3JTb1nf7+0eoGve5CqWGmyycLaPcMsxxjzKIhsJ3qG7OMc+0mJG/3s= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org BAC99430766D ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=IwKu9IWzDnxG6y4rqK7ksW7T97vbuFi0ypIpZb7+zPoruT2jx9pBffs+8dZjqiC4rmBqrw6fiaKNPmunq2uqsnN64KIuEAjOchw3ES5urAeTimmeYOBeuxAwrFhMKVPFzAxKw2dgiHEcyVvj4sI5Ct/1rcYwJHfDKaPo4GvOHsExC4Tizq0A3/pZHOgLxle0q8VRoSqt96EzeWEKFqGubpwY8o8pp+sLzhjSdivrf4+2DINAxCVcKVE/kOb5sgF3L59Q417oWbvB3FNhb6BH4dNzqlN7cEv6jpbxN0knRf7r7hCOiBccjI1wKPeKCazASbdobVyd5o7IW8ijunUX4Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w1CRHdzIWeJ86AiEse0CxdQUvj74Zsan+xWDsjmprGc=; b=SWBgmuNQdfB2ZwY8ZlSyXeSEhMjHrXKQcRcvU3qUK6WkcNDwx5mOLRkE4dBA7tIkwQPZLttBCl4/r/HZsNjPP/78Cyj0kHVljTKmOEJn8ATERPwsBhbESLIIGBMfT+QdTwxy71Y9ws03nP2VMWxQbnerUVa/6kWIGduYeR+tyJt68pj8QrITkEzYXFUEocNYHn39EATKv7+EK6Uu2ko2L7keb8aJOTjZHsEXzjqNve+Li+46h1pipTL1mKaTg5x8SHoN0kRkV7XTyyVjtT5dVaPPGU/Q9b8hCT8i9huye/SswA3sfooTYpB+hRF0t17bZEKL1H3y5401dBjxlZh1bQ== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w1CRHdzIWeJ86AiEse0CxdQUvj74Zsan+xWDsjmprGc=; b=akbZExZODxX5QHwFHzOn1nQxXn5fGHUOnBGQqBfBqlGW3W+YXP7h6uK70dENrnk3DnKJt2e3yZHti+NB5ib1qbSbgLuO9AS0hlITcnqYo2wKPerylfxex2N0po1G/Qt8rina6TiKvUVc8ul5QaM14y/R8adhItPMb8hdqVvcb8s= Received: from CWLP265CA0273.GBRP265.PROD.OUTLOOK.COM (2603:10a6:401:5c::21) by DBAPR08MB5735.eurprd08.prod.outlook.com (2603:10a6:10:1a1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:56:46 +0000 Received: from AM3PEPF00009B9F.eurprd04.prod.outlook.com (2603:10a6:401:5c:cafe::b3) by CWLP265CA0273.outlook.office365.com (2603:10a6:401:5c::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AM3PEPF00009B9F.mail.protection.outlook.com (10.167.16.24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:56:44 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q7YdZM0d29g+LA+uFWzhMM27DeidqeYMJVai/JelXjRcXr3A5WkEXoUerSObNKqPpr/zKwwoJoa140dQEsn4XxfaRl4d9F16Sa5L1ySvIYxosjF584+tcBpmtPSUbbSP8vEIwMOy6+BqbDziXL12hhmokB2XNt+YFd24vLuuBcZWy1vR+WgIBpspW+gAzL8B5wGh4lmtQYRqrTYpWjHRPHJBLU8C1nNC20cziXAgJ6HiE5LOwBsSJW+6TYIWYbPvm15GYzVY86577RqHwvjZJSVxH0jPBQSmBw8Ip0ogeullOLTSnIJmL4OGQ4oNeDwde7TRnEipuzWcG8F9wZn7lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w1CRHdzIWeJ86AiEse0CxdQUvj74Zsan+xWDsjmprGc=; b=muPRsWG7C5l4BHxs/bqNHgwNmKBPIOx6f6ijptaODjF82Ofnq1x51LL3h4fO3j2quv4aHyV30iighm62ohGvq+ty0vE61aL4LXH40YjSUxXeFQq6jb+CiFJEA8riILvt5mjNupq3yVdAy4719/ieZwFkHaqlAg53x77nJuxZ1J10TvDd/4WrNoKkAzIdO3BZX58JNNRmv2iSB2snb895jU9Pg8tLAFJfhVfVsHp6P3EIlNhOrcuNRKq+0Uy1Nq/LtkSE2vWnh6d4vwBNgWc8Rcba4sGfwi/q0v9YG9tJ2D43KibpxivHtFqdgPXW9AlO2xdPj4rvw+745jY9943J2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w1CRHdzIWeJ86AiEse0CxdQUvj74Zsan+xWDsjmprGc=; b=akbZExZODxX5QHwFHzOn1nQxXn5fGHUOnBGQqBfBqlGW3W+YXP7h6uK70dENrnk3DnKJt2e3yZHti+NB5ib1qbSbgLuO9AS0hlITcnqYo2wKPerylfxex2N0po1G/Qt8rina6TiKvUVc8ul5QaM14y/R8adhItPMb8hdqVvcb8s= Received: from DUZP191CA0041.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:4f8::20) by AS4PR08MB7508.eurprd08.prod.outlook.com (2603:10a6:20b:4fa::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:55:40 +0000 Received: from DB1PEPF00039234.eurprd03.prod.outlook.com (2603:10a6:10:4f8:cafe::5) by DUZP191CA0041.outlook.office365.com (2603:10a6:10:4f8::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:55:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF00039234.mail.protection.outlook.com (10.167.8.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:55:39 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX03.Arm.com (10.240.25.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:39 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:55:39 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 04/11] Update constant creation for BB SLP with predicated tails Date: Sat, 6 Dec 2025 16:55:11 +0000 Message-ID: <20251206165518.5449-5-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF00039234:EE_|AS4PR08MB7508:EE_|AM3PEPF00009B9F:EE_|DBAPR08MB5735:EE_ X-MS-Office365-Filtering-Correlation-Id: 1365d4b9-9920-4bbc-e413-08de34e87009 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info-Original: laQMgJ+27zOJqnp8u3NOuNzliq4ktnH+mJ9TIdcARtixh3WtSmFBYkxe9NKH/9iKYVNF+/vw9udkRHoAAd6q3eV9OS3HFYJDy+PoHdAKwNJ2Qxq4otRN76yPXvCtTc7goyjz4Jhjew2L4oOSUadjcfO787sBFK1AC4eqI6p3ZpBZVKKyNZL58gaHYKkFY83GbqNPo8SSyrrZ+x+8qf4JWul0Z87MWRZUqei0hHK30qc5GCsnM6UPMWpiQsLum2n/Du8TsqQUYfW7AdAFlkm8mvmkOxxDJfO++O6wt7wlF9LIgKsPUhw+wJjatza6OoNC4oecMjIa7zzzJiGAw3ENQUSFQbRFKiOWp1d+mLzFkZFTlEDvEwKeS+KLj6SAGThQ5fMxoG0PaCIrReqPUodrbUWI3hB8dt4RNdYWXlhe8X3uoega+3Ka1FWW3S7xX1Ega+eXymtMnEGJcEkiencCXCDHq/Ypq3Qmhjf9mmuu0FQvGaUZ7VgqARo3KdixerdsQTDMNVFmIx9Wa7hAJBZ1bdre8++LGok4gLgdrn3wAECiq5n9dF1gJyh9ERNCJfeikGqMFUR0AbG9/RI3mQTVaT3kKKZR3HopPfOs8ANmVOUdeSVvN3W0o6fxm7YOu3elI0+XKsOS1I1wu995vpVZ7uRkx2NnHF6aTZsTFjOdBcHFU5ZjsmaUlvrEfJu3fO5RzCv/+A/5pKdd1lIODjkmJXCbQ2W2xX7D01EWMXsA5sR8piVZn9OTWpNo5a8aStV86hDG4H7nesuQMiNw2P/x1JKaaERbJeaAI1pfkt3B0B1/d6jz94D0JK/PqOCH46GRAr1wb1A4PKo1iL2ZOU9+2KyYyVCJmsZoF3CQzU53ZdMapks7zkrvh8mD33YuGYRneVYnCy6Lj1g6HPaBy86myJDg0JvcrGo5HdX7xyXelfz+abKgYlY5KWGlGu6LBt5aMOInakzdRgjQu5nBDEMLc707h+Kat2GeZcmZlUOLjXMUTLjsReo7GY8Ph8rFTUZZrAXLl2eq7f89/lOZgoCu1nw7/v5CGSD7FzhkQUDmc7DtDd/bjkhG+ppH9YPkkukO19d7LwlcOJQwjI3nas0H9Ypx38kK1/Q1gKqTg85rwx8o40my5Thqkw7XIHrPcuUWHYNrTIU2zUGpWkDrCZB4Ns1tJRB0XG41LXshyFtiGlrtZzyFkiVWjEEbr7TCTVweasW2VJbEdxyt+FMTNf1rEbYBR86DmrwrEFAOaZ5PsmW2v3SBLN9x2Q1HSOrNNLM0UQZFpHm7WyN+DKwFGuR5vwLRrLr+4+jDJWy559BWJ+QzKJOLwF4bIppOhuZqj60GMKwi5iU97jEIbwJXmPIPQBhMPr+tRysiEvKxxzCWhwOb/MN8Ld6kl6XG/1QssyNWHclbk0g3IY3I0PTEQfVUqFXrQnJmfPXHVpvoxfUyRaRk7zAEH5MuAUcXiHKhcmpLpsYTdYAU6p8gorybdVtIURdPSXfnXlSBJPEDBFZqEW5FDG2OMLAzTP/rIWkZgCiI3SptURymkwNl2YMNeSM5jjP+qrgT3IerBEuoqEB1auU= X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR08MB7508 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM3PEPF00009B9F.eurprd04.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 37d24a83-38e1-4635-2012-08de34e8491f X-Microsoft-Antispam: BCL:0; ARA:13230040|35042699022|14060799003|36860700013|1800799024|82310400026|376014; X-Microsoft-Antispam-Message-Info: tXJupVPbJq8ETD85PbXzXGCwNb694Vl4qEVXi0KFma/RGtVtK9OW48yar8JuRq2SB1VNb+VQTA6ljdSZnHixYnw/0IUGZ2O7Ot2c+zCnTk5S2tjiPOnFXhHrlX0igHAGkgd9HxYsvdA/Tu0c55CS6A9Uy7QzBugGvUQSQM6kZo+TqxUbIPp/rGDLCSzZWP18pbJXw+UqjYwcg5nagNwgc1rt8dpFeC4VwTcNEdaNlNN2CfpDEW2DGMIe6ZXAcptHcnf9sNoXKkyws4yseB/llTEX8jhSLmxivFmHxcVRp0tia+JT1kWLfsAxeJAngcF6kaJSPxmRUmnygOXXTOof7DObesyGrqn2GXDNxNk4XadzgSuSp2aS+qI/BcolJAAH3WGjHgr1sMlFrl0NPGSKxiT9xHRtbFAF3IX2+Yq0xQVQY2AdE4wVS+UJNeIrormZHzKcO/xpZfXtq9a4/SAiOjUenN9QTj8Nd9ogOHDuHDkdr3KKxzXfSDLmhvc1fhHfJ2Z7vOBhNxqzOg+ocZHH125/MJv6IZL66nc5Z8fvPGJknMjRgXA1lpN/dP4nbHEWRJ05MXfMj5h2c5GgB7Othp7oFWKgCXbBgBhh3CnNAtYBArloX/NXKD/GWO3qse/455654ERVscKirXiYFvfeeCTxX6qKuvtYlQjUPtkRqd2g1rMBiI08VKKFo2XBqJMzjiMzbzXIT+EZxzIR1WY2oMLBCL1oCLSXCyrI1akiW5AEYAGIBtKqJ5vWy9QJbtbpqCCUjEY8bdQY0EV6PcYIZMCtDCA/2fRqjyiEuYgPxsOUZBrGHcW6HG//9N1wkr+mX0qjS6mh1HgHoR9S072BXmwrjLbgqGiVnFT5srFq2a6roQJKrLfDL9J2sRGKMtq517KhNPDMRwznzc9N6bbJPoW5JHJT8eQ41P57cjwkO+xSBKZic8/6pdz/9Y69/YU/7Jnp2Mk3d50SO6hX9arnh3EexuGfxno7Gow1I5Q4k6rDej3Dimv+VHsTqXo//arsvrDWQmrRDz06I3uHQLi4+TXIBdxpUsjArb98r6x6/7+eutEph0w9kGikfeaxNcUWy7daotecWZeibnOF2yoEYk0wC+OjBg6pm2hjtboHXc4Yfc35GataqXBBY9HyoTRwAPsKgYlwYTMl8mBZ4cdf9QJ2srUv7qQsJ6S3mC2WVHJWrGd0BAGxcWkQaG5k/k1AK0jByY6Ql7NV6KJLexzm0d8fd9KFeyQeczcjl8obeCLd/rjQO+c0bt6GK/f11N02XOtn8mH5lhOIu5Lg0R44X0+vJQ24RC8w2uea8F4jWV3zNiWDg65lGZn4wdJBnYzI0Vz7sShgf5XnRooKMHB0a1kiRnCE4o6AGYADabJOElV5SvtjqrnTvZSyBA3dZ7epdIcMI4CJgPIFfVn7Pb8nbqIOpEE1mB/jRYo+DbIHkdR9EIRs1lJcexBV2SbRlqzhaWwW4Z2NNHpv1SimkfgMcLAak6AD/AgXnDgbaGep8BlkUHpJo7hRkekXKJ3iA17WzWYPPdeZngygowAaqTMea1wjt87h7oNj2eMrj5toNfU= X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(35042699022)(14060799003)(36860700013)(1800799024)(82310400026)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:56:44.9521 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1365d4b9-9920-4bbc-e413-08de34e87009 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF00009B9F.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5735 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org Created a new function, gimple_build_vector_from_elems, for use when creating vectorized definitions for basic block vectorization in vect_create_constant_vectors. The existing gimple_build_vector function cannot be used for SVE vector types because it relies on the type associated with the tree_vector_builder having a constant number of subparts. Even if that limitation were lifted, the possibility of tree_vector_builder patterns being used is inappropriate. The new function takes a vector type and vec of tree nodes giving the element values to put into the built vector, instead of an instance of tree_vector_builder. If the number of values is zero then a zero constant is built. If all values are constant then a vector constant is built. Otherwise, a new constructor node is created. gcc/ChangeLog: * gimple-fold.cc (gimple_build_vector_from_elems): Define a new function to build a vector from a list of elements that need not be complete. * gimple-fold.h (gimple_build_vector_from_elems): Declare a new function and a simpler overloaded version with fewer parameters. * tree-vect-slp.cc (vect_create_constant_vectors): For basic block SLP vectorization, pad each constant to the minimum vector length (which is the same as the actual vector length, except in the case of variable-length vector types). Use gimple_build_vector_from_elems instead of duplicate_and_interleave to create non-uniform constant vectors for BB SLP vectorization. --- gcc/gimple-fold.cc | 54 ++++++++++++++++++++++++++++++++++++++++++++ gcc/gimple-fold.h | 14 ++++++++++++ gcc/tree-vect-slp.cc | 42 +++++++++++++++++++++++++++------- 3 files changed, 102 insertions(+), 8 deletions(-) diff --git a/gcc/gimple-fold.cc b/gcc/gimple-fold.cc index 3fc76313622..d7a16569f5a 100644 --- a/gcc/gimple-fold.cc +++ b/gcc/gimple-fold.cc @@ -11274,6 +11274,60 @@ gimple_build_vector (gimple_stmt_iterator *gsi, return builder->build (); } +/* Build a vector of type VECTYPE from a partial list of ELTS, handling the case + in which some elements are non-constant, and padding the tail with zeros + if the list of values may be shorter than the number of subparts implied by + VECTYPE (including when the vector type is variable-length). + + Return a gimple value for the result, inserting any new instructions + to GSI honoring BEFORE and UPDATE. */ + +tree +gimple_build_vector_from_elems (gimple_stmt_iterator *gsi, bool before, + gsi_iterator_update update, location_t loc, + tree vectype, const vec &elts) +{ + unsigned int encoded_nelts = elts.length (); + gimple_seq seq = NULL; + gcc_assert (TREE_CODE (vectype) == VECTOR_TYPE); + unsigned int lower_bound + = constant_lower_bound (TYPE_VECTOR_SUBPARTS (vectype)); + gcc_assert (encoded_nelts <= lower_bound); + + if (encoded_nelts == 0) + return build_zero_cst (vectype); + + /* Prepare a vector of constructor elements and find out whether all + of the element values are constant. */ + vec *v; + vec_alloc (v, encoded_nelts); + bool is_constant = true; + + for (unsigned int i = 0; i < encoded_nelts; ++i) + { + if (!CONSTANT_CLASS_P (elts[i])) + is_constant = false; + + CONSTRUCTOR_APPEND_ELT (v, NULL_TREE, elts[i]); + } + + /* If all element values are constant then we can return a new VECTOR_CST + node. Any elements for which no value is supplied will be zero. */ + if (is_constant) + return build_vector_from_ctor (vectype, v); + + tree res; + if (gimple_in_ssa_p (cfun)) + res = make_ssa_name (vectype); + else + res = create_tmp_reg (vectype); + gimple *stmt = gimple_build_assign (res, build_constructor (vectype, v)); + gimple_set_location (stmt, loc); + gimple_seq_add_stmt_without_update (&seq, stmt); + gimple_build_insert_seq (gsi, before, update, seq); + return res; +} + /* Emit gimple statements into &stmts that take a value given in OLD_SIZE and generate a value guaranteed to be rounded upwards to ALIGN. diff --git a/gcc/gimple-fold.h b/gcc/gimple-fold.h index 7244941722d..358c269aacc 100644 --- a/gcc/gimple-fold.h +++ b/gcc/gimple-fold.h @@ -243,6 +243,20 @@ gimple_build_vector (gimple_seq *seq, tree_vector_builder *builder) UNKNOWN_LOCATION, builder); } +extern tree gimple_build_vector_from_elems (gimple_stmt_iterator *, bool, + enum gsi_iterator_update, + location_t, tree vectype, + const vec &); + +inline tree +gimple_build_vector_from_elems (gimple_seq *seq, tree vectype, + const vec &elts) +{ + gimple_stmt_iterator gsi = gsi_last (*seq); + return gimple_build_vector_from_elems (&gsi, false, GSI_CONTINUE_LINKING, + UNKNOWN_LOCATION, vectype, elts); +} + extern tree gimple_build_round_up (gimple_stmt_iterator *, bool, enum gsi_iterator_update, location_t, tree, tree, diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index 57adef2b5d1..3becf34c09b 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -10796,7 +10796,7 @@ vect_create_constant_vectors (vec_info *vinfo, slp_tree op_node) unsigned j, number_of_places_left_in_vector; tree vector_type; tree vop; - int group_size = op_node->ops.length (); + unsigned int group_size = op_node->ops.length (); unsigned int vec_num, i; unsigned number_of_copies = 1; bool constant_p; @@ -10826,12 +10826,27 @@ vect_create_constant_vectors (vec_info *vinfo, slp_tree op_node) (s1, s2, ..., s8). We will create two vectors {s1, s2, s3, s4} and {s5, s6, s7, s8}. */ - /* When using duplicate_and_interleave, we just need one element for - each scalar statement. */ - if (!TYPE_VECTOR_SUBPARTS (vector_type).is_constant (&nunits)) - nunits = group_size; + unsigned int elt_count = group_size; + if (is_a (vinfo)) + { + /* We don't use duplicate_and_interleave for basic block vectorization. + We know that either the group size is exactly divisible by the vector + length or it fits within a single vector, so all we need to do for + VLA is to pad the constant to the minimum vector length. */ + nunits = constant_lower_bound (TYPE_VECTOR_SUBPARTS (vector_type)); + gcc_checking_assert (multiple_p (group_size, nunits) + || known_le (group_size, nunits)); + elt_count = MAX (nunits, group_size); + } + else + { + /* When using duplicate_and_interleave, we just need one element for + each scalar statement. */ + if (!TYPE_VECTOR_SUBPARTS (vector_type).is_constant (&nunits)) + nunits = group_size; + } - number_of_copies = nunits * number_of_vectors / group_size; + number_of_copies = nunits * number_of_vectors / elt_count; number_of_places_left_in_vector = nunits; constant_p = true; @@ -10841,9 +10856,15 @@ vect_create_constant_vectors (vec_info *vinfo, slp_tree op_node) stmt_vec_info insert_after = NULL; for (j = 0; j < number_of_copies; j++) { - tree op; - for (i = group_size - 1; op_node->ops.iterate (i, &op); i--) + for (i = elt_count; i-- > 0;) { + tree op; + if (i < group_size) + op = op_node->ops[i]; + else + /* Pad with zeros. */ + op = build_zero_cst (TREE_TYPE (vector_type)); + /* Create 'vect_ = {op0,op1,...,opn}'. */ tree orig_op = op; if (number_of_places_left_in_vector == nunits) @@ -10929,6 +10950,11 @@ vect_create_constant_vectors (vec_info *vinfo, slp_tree op_node) ? multiple_p (type_nunits, nunits) : known_eq (type_nunits, nunits)) vec_cst = gimple_build_vector (&ctor_seq, &elts); + else if (is_a (vinfo)) + { + vec_cst = gimple_build_vector_from_elems (&ctor_seq, + elts.type (), elts); + } else { if (permute_results.is_empty ()) From patchwork Sat Dec 6 16:55:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126055 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3E8B94152D0F for ; Sat, 6 Dec 2025 17:06:06 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3E8B94152D0F Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=G8Ovbo4x; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=G8Ovbo4x X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from DB3PR0202CU003.outbound.protection.outlook.com (mail-northeuropeazon11010011.outbound.protection.outlook.com [52.101.84.11]) by sourceware.org (Postfix) with ESMTPS id AE35742DB840 for ; Sat, 6 Dec 2025 16:56:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org AE35742DB840 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org AE35742DB840 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.84.11 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040215; cv=pass; b=UhvP3OdHQq5vi0S35de2RQjbYk92qluG8aZcl0M/6rY944HM8IutxXqYjK+DX4IYGyT1L7Dll8mwuuSNoBicje2lBxbyAxUWiiZo/idXQnyvdQnkN57FNzUq7ulhz2sGKIgxmNNsDPE20L9/ewbwRLFrvx2mvURsE3TbdycpyAE= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040215; c=relaxed/simple; bh=MZy8kr/f0qy12f7BxETJgFUNjfRIXYJOYVnzn/NG7Jk=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=XtHfI2hJ117JYhc2p9c89atxH7NGsTwyJOwRLqDkLQMNIw2falV6KKiwwUpdui9mLa+Apvzyavl4bRDx20cyWP6qCSIdoCzbSVuPBXFkgNZp9+WGidJ4lQfSsSUE9xL9F8oHQ9xnNBRwO5fp8kPa0nKz4HJ+8jKjHbVs0iOBzFA= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org AE35742DB840 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=CESJ21PxrqyjhdtfQS4NFBSnCKCJi8i6Rt6RlbOnwNANjJw7wQA1JxPB7XfOCby33bypN5Sxg8CTYFrpfHTgjPfzrXL5JMpXj9pBP1yKosIHtRuB7xfM4tVPiD0MYvWOz39nzoUYpYb9c5FCIkfjXwd3cln71y1L2Pl+l/kUNEtw41eOwvhg7uv26o73uF412V/ZRzus3Nlrix6QTjXLGdbUHUc5wq7NyuTa/T3TU1buKQYMPHLjvUkDxOF5GneCeoDmKd+OMc0IdZzjGtOlDwkm/KU3o/JcD+Ba/HB9vDyRF83EKfUrmUkEra9EXs4oV32bR4mkyC3FxP9qHrnM6A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kW9MkxQUrdqZFtsDO3SOcOyW+urTwXR/iIumXK/oAFQ=; b=g6ti/A5UuygUPBG5pYhXxTPmM/S8msmZ6z1ubyx9+kxbc9lwjl/XTDWUQfC+DujpBQWjfyXs9a0LDM7Wwyp50Clup9lSD0XXqCTEJwTb8b1FXEkOHWFVap1VtnFs93aLr8x2L2jj1u+YTGP7K5zsvcmdH+Y7pXOZ0TaYFx1qnZTJwcp5TOCG393t9Ppdc6qLpW5QdlljEp0A9LNc5z7dwd4APoSy/B7hSFpkb8rngiBY1d5kYwKI7MrULFDg2h2Sn8vC3+salJxcwphMTiOCZwbhQj79A+YCMUTECwGgs07RpaTgpnABiFZ1czXqeDi5EUqH9OaNpLIJzB0apBHGxw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kW9MkxQUrdqZFtsDO3SOcOyW+urTwXR/iIumXK/oAFQ=; b=G8Ovbo4xpiw5lvRTXqTs82Y1ALinvI+SFUNt5GhcGm44furEbK17pZJv22VvjS176dvZ48iuPR66u5SvfIcFBSBRS1oEfUmus2FCB34kYV4qAHfCEpoo1E74WkrHjlwRywOz+9YnCR2zm2z/7I+q8XNusHA03mi7VRq2awL9YXg= Received: from CWLP123CA0255.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:19e::23) by DBBPR08MB5898.eurprd08.prod.outlook.com (2603:10a6:10:20c::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:56:52 +0000 Received: from AMS0EPF00000192.eurprd05.prod.outlook.com (2603:10a6:400:19e:cafe::f1) by CWLP123CA0255.outlook.office365.com (2603:10a6:400:19e::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AMS0EPF00000192.mail.protection.outlook.com (10.167.16.218) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:56:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=spj4fNhbvOOVqQwqONnnDFswjMF5NCsFD6fcX8J1JEKjj5e/TdtWUZwFB7NCRTJgIq9vtl97uudSy9UjjRGZpVgiajfUZbsQv+1DDG0qdBsWfevsYctoEC+WCBiLrZLmu7ALJJX8QY+M5/DTT/BA1tuNOOKX2sfa6d4WTK0lPtYG9QB2qz/NrqJa5BC9S+yQHFLAooM42pjg5XywIuIU9oRRVZ0nLwTTExgEXXDcQI0gl2ilaa8C7Z1bfGE+KOK5vVOUmuZr63f0JfMeo2Hx1qQ3yg2+p9w4ex8HXmo3bpIut8VjXgOYom0Ie456i2irGmXXOEXf7q2I2ydgSnO07A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kW9MkxQUrdqZFtsDO3SOcOyW+urTwXR/iIumXK/oAFQ=; b=Fpu+JyT5P9RfwT7m4ayf3nIYmeuPLAO09qcoPor0dBfRSr2uHQmtvATFYZYGZFuvMbRhqLfrX+MyM5ZYbAFIMD/ibmLTXoeE0y8WeCJpkizY4VXcDSMK2iyMzFAl6PCogFe6QygES3P+g4o9b3ElsrSyiOJFbeD+X9Oe75K3J31lGOM6JR2ah74G5YeXLFBhX1dr90C7nyn3y7gpuC4Ia78u2LfsqBja+c+C8A6NDlyPphAQQvHLKkk3JCgeAg9v/DxQndw5MBEMDU95vvmzq5cQyLT0XicaVbsxGDRd7Cdrod05R8iYTWJJIv21GKNxxjd67m3cU+d3cloOJoCUlw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kW9MkxQUrdqZFtsDO3SOcOyW+urTwXR/iIumXK/oAFQ=; b=G8Ovbo4xpiw5lvRTXqTs82Y1ALinvI+SFUNt5GhcGm44furEbK17pZJv22VvjS176dvZ48iuPR66u5SvfIcFBSBRS1oEfUmus2FCB34kYV4qAHfCEpoo1E74WkrHjlwRywOz+9YnCR2zm2z/7I+q8XNusHA03mi7VRq2awL9YXg= Received: from DU6P191CA0061.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:53e::14) by GV2PR08MB11315.eurprd08.prod.outlook.com (2603:10a6:150:2cb::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:55:45 +0000 Received: from DB1PEPF00039231.eurprd03.prod.outlook.com (2603:10a6:10:53e:cafe::34) by DU6P191CA0061.outlook.office365.com (2603:10a6:10:53e::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:55:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF00039231.mail.protection.outlook.com (10.167.8.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:55:44 +0000 Received: from AZ-NEU-EXJ02.Arm.com (10.240.25.139) by AZ-NEU-EX03.Arm.com (10.240.25.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:44 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EXJ02.Arm.com (10.240.25.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:44 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:55:44 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 05/11] Refactor check_load_store_for_partial_vectors Date: Sat, 6 Dec 2025 16:55:12 +0000 Message-ID: <20251206165518.5449-6-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF00039231:EE_|GV2PR08MB11315:EE_|AMS0EPF00000192:EE_|DBBPR08MB5898:EE_ X-MS-Office365-Filtering-Correlation-Id: 99d9bab8-027a-45dc-f65e-08de34e87400 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info-Original: KQc5wGOyj+PE607uVUkjZDPrIfsHokOBh/J5Hm7av/o/fSzBqF10OHT2TOO2v5dregEFVeFHniirFSiWg7LXmKlBfOC2LeNO3qtf8/37gEj5+evFDrkb7+4jGDVU0dgUwwNzNEPJlpcLjcQPVXFFdElDtIpBYMHY9dMXePcXfOXyNhQBxx38S0neGrNKaA9BnPyOhs6qeEfsCiiSskVEdnMBzemeO+pWooymeNQQwQKRZgaEVYKTesIdY1kVbPsP42QzrwJjIY2kMKhjZV9e9m1Dmkp/4iVxyiW4FH0ICDdwYrDUqlTC/f8xY75IBn5RFE02xWF1FGIQJ07MpIA1oyK2Y6lTbhKR9BXAoW7AjS453d1mDbdBZjtbWIJLoSm7H0KZmPhR6Qoto6Xm8stS1pEUvC7CGu3gTOEigJLa7XGvxqlo5bVCjlMqBETwF7JcUMNt3Y8VmPFMC5m3bNqUYUOa5S+tDZa26FjelC0Jct5Ac8saEOxYMVWzXJv2MiMeAAWxlrdK1iwyIeFJ2dYmKnSvBbVLAQRzdRXTdLuN0hVIDMZhoaxtXYiBpy6EGIuVY4Vupe2FM3YT0XvcBnwikwLShri7Wjxx5IQBCKxW3smYI+d/lX8+sXJS+VXxzzf8etpQx1CCU8uencWFMBAFSYfgrhmC2ukyrb2ZIRCwE9SxtkGO+dzt44xuXbzBB8fuTZRfJrB0d1jYBlGwCjJL1VocBCOW5wi8KLu1qYZhiBF8aOyI6BSs3xZarXMxXSb2bDf+5T0YkbCzc0TNu0PYXqDZVXPxBHtm8W69j/RPECflvTczLlk2gfdjo261fbzRsC4k6sJ3Itog8RXQsDlErO7TOeFHn4RaDJ34CgWvy7Fh4c7djxVBmQykwd4tcb8mF7Xc4jRvjzutKu0hloVCDq/EXb2TD2hJZL2TZfvrJ/F0TBa0xrdhHGIvgEw1cUftSgYsi1gDtjofDfPIgE5zG90Gc5BLw15W3JgGFMMZLhD7DVyCDXoEPQSvfggYy4ZuXu8tnyTEawAdo0xaJDYFbDEw3333/FutJClx9RHQmUQVJKw9Giu2XLeq11fS8g0293d95B62k10daAAXkOmWQLxAS6aJ4iPeTTkIghdZl10+LyXLI0TXxTtJPOgf7sajz6bJs8fyPOlq9psHjjughxH+/cTnBfPDaZdAtrTUKVIRp6qlBq6FCNhVwiZBvU5iFZiSYothWO5gSF8ArY+WoOBTBYncWYZV51gWWGlMh14SnCbFcr48Q6M684tEFYYoFXYsAXHhDgBoSX1LUw/quA477XgAd605ek3DxzqwOvBsex14745N51C/+EGHSukBtddmd8dDWCySOucQNB837Rs8LCApuzAZnXqRCSjAdR+tZgmEvJnMbv/WEmK3nzLX+b4Io91dD6cuGcD8XtAxiuotd43bqX8UHzx3sFLZImvDlsLVpTyARmN3Hpr8sppE6G1B3SZicPYb2wb6WuhYPVYpaM/vMMCZVv8krS3L2yTI9rd0kSP4XuIOVPfJdcgVdyiGpxiPwODcS/DPHGX0DtiOrM0VTx9XUTl/gr7/GD8= X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB11315 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS0EPF00000192.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: e6103a9f-2529-4bdf-12a4-08de34e84c43 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|35042699022|82310400026|376014|36860700013|14060799003; X-Microsoft-Antispam-Message-Info: wLWwU4dGZsxHroR0TtWX7+9xiQALqkICqFobRZ4RW+drbyNvaVDAElGBG8wswWewXAhiPz9FqHbZb4v7pZyT/Ydh6GXMaxQhVqUxFF8lO4firTDkC0wztgMVNlMu7oh/PHC2AGsPxABjUqC8st8EsgFlV2zZBE1qEUPel19HxbwxO+LUpe3g+w8to159Bn9Rti9do0KwT2nY+FYeebRYCyOF3xpvszN832uzCtOd7lv8MD4ufapd4gtlb6MEZ93RNOJlz6qK3cSt2pZbmrhpABRIcuudK5iTB3qC61sYUrF3D1DbGYy+6RK0ASLw46ETveEjtIH6jIKjfAvn7drgGtwoKXovX/ZumkikPgyc0rQKMFxQVMusTSiRuegZE0rn8ScHbZomr859MjVJrPoIlUSi5vKeQ667QONgriQqhsvhGG2pkGfsZuCIo0QQAa89MZMQfODDQWwA5heVNmDZ3QSBVyrj+mekk7mjTBHhmLkpslT2oMzaj2+RbuP7H2pVBGKaUB544Q3YvsgdkV/XYiP1LUUZFv50tVGVADyz26Gl5PLotPzEOt9Lg5Zm5APkwRjw6SFdk8r7nyaEt1MFa4hEzsNSu3UKMl6ZcNT5N0riM4hCupPktoq1NkMpYh4VLnjwLjr4Y4jbyP/nvuOZyJWSpitvdOYmwVL0lWIjGBS1GAkUiI126z+GvSiv9NFzexlehVEmF1LcDRzEvnrG9CdSCrD6N62IxAMpiXrA1oPaEz/sUN5etTycg5B2qiH8uf44YM0tJUXWXNTGJC+4sB09oMGXmVU47SSDqSdbBcfh0Iu/lSNLc9fzuVIYJ7Vir5druDjd7MLykEVfknQx+O9OBsOlnoTuMLZquVXm/6W1JZmkQU6tT7ybezYq7HqwzXeNlPvY7DNkcolNOEh511ux/w7ddCrDdyHb7y+BAEuQe2TunvtJXef9D8Bxx5bxIid0fJQUlVQEMNaNwjU/cI13vBAbByeF2nSa4Q/7GuH2jM27qx4tO0DeEHgoCymP3HkAYzLofbhhmcFo6A77j0Tip5mbGBrpTI5fqSEkIqnkb1CAcMIN0lDT8MyB0Yvgi1bpjtFdCBDxvAEksw3ikMaiN7KsCbEvW+Fi7yF7Qi6+MgZvIC6bxEMqb/OPFsly+TF8vk3kPvSOmD+RyL4SA9JBKjEsr9jtfmjIFoaZE0oo2zV8nBDKW/JUX97ny5WOqSnEbWR4ixD1TqGgKffq6eJrHFMxG2AHsA4wSy/AlLT6cemfOzcfAPqE/q8HyYb5wxJmjuGxaAvK8aFeBhMdRbFIMTT0M36Wirz3srdEbMhJ3qpiQcXGC27Jufij/KBGx1GuFVuFG2kBy6ALK7UbtrlO8HmV+LEEULqNQk5i1GGZRNOF07P6Im4lJDHTXmlMAy42QKmOoM6Gp33MNDQ3tF7AK3snk/V8+EEHHTL23T5nVvlFl1Px0q7YYBDoHujRRMfhHIlw8KzkUrmSAzVbyXerBhu0rPrtEidhrObe0rdPOUpMm28+zxus15uOW5iVuvB77PEokvFzmyRMmajikokx5x89aTYmDOnbDakjD9E= X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(35042699022)(82310400026)(376014)(36860700013)(14060799003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:56:51.5980 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 99d9bab8-027a-45dc-f65e-08de34e87400 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF00000192.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB5898 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org Moved existing code to determine the partial vector style for a load or store into a new function that will be reused for BB SLP with predicated tails. gcc/ChangeLog: * tree-vect-stmts.cc (vect_get_partial_vector_style): Define a new function to get the partial vectors style supported by the target. (check_load_store_for_partial_vectors): Use the new function in place of existing custom code to do the same thing. --- gcc/tree-vect-stmts.cc | 71 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 58 insertions(+), 13 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index bd63451b467..8ffdaacb5d2 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -1462,6 +1462,56 @@ vect_get_len (vec_info *vinfo, slp_tree slp_node, gimple_stmt_iterator *gsi, static tree permute_vec_elements (vec_info *, tree, tree, tree, stmt_vec_info, gimple_stmt_iterator *); +/* If target supports vector load/store (determined by IS_LOAD) with length for + VECTYPE, return VECT_PARTIAL_VECTORS_LEN; otherwise, if it supports vector + masked load/store for mode for VECTYPE, return + VECT_PARTIAL_VECTORS_WHILE_ULT; otherwise, return VECT_PARTIAL_VECTORS_NONE. + If load/store with length or mask is supported and ELSVALS is not null, + store the possible else values in the vector it points to. Unless FACTOR + is null, *FACTOR is set to the number of separate subelements that each + element of VECTYPE should be split into. (This is always 1 except in the + case of VECT_PARTIAL_VECTORS_LEN when the machine mode of VECTYPE can only + be supported by using a length in bytes instead of a length in lanes.) + */ +static enum vect_partial_vector_style +vect_get_partial_vector_style (tree vectype, bool is_load, + unsigned *factor = nullptr, + vec *elsvals = nullptr) +{ + machine_mode vecmode = TYPE_MODE (vectype); + + /* TYPE_MODE tries to map vector modes that are unsupported by the target to + same-sized scalar modes, so reject scalar modes here. */ + if (VECTOR_MODE_P (vecmode)) + { + machine_mode mask_mode, vmode; + if (get_len_load_store_mode (vecmode, is_load, nullptr, elsvals) + .exists (&vmode)) + { + if (factor) + *factor = (vecmode == vmode) ? 1 : GET_MODE_UNIT_SIZE (vecmode); + return vect_partial_vectors_len; + } + + if (factor) + *factor = 1; + + if (targetm.vectorize.get_mask_mode (vecmode).exists (&mask_mode) + && can_vec_mask_load_store_p (vecmode, mask_mode, is_load, NULL, + elsvals)) + { + return vect_partial_vectors_while_ult; + } + } + + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "the target doesn't have the appropriate partial" + " vectorization %s for vector data type %T.\n", + is_load ? "load" : "store", vectype); + return vect_partial_vectors_none; +} + /* Check whether a load or store statement in the loop or SLP group described by VINFO is possible using partial vectors. This is testing whether the vectorizer pass has the appropriate support, @@ -1627,28 +1677,23 @@ check_load_store_for_partial_vectors (vec_info *vinfo, tree vectype, poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); poly_uint64 vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); - machine_mode mask_mode; - machine_mode vmode; - bool using_partial_vectors_p = false; - if (get_len_load_store_mode - (vecmode, is_load, nullptr, elsvals).exists (&vmode)) + unsigned factor; + vect_partial_vector_style partial_vector_style + = vect_get_partial_vector_style (vectype, is_load, &factor, elsvals); + + if (partial_vector_style == vect_partial_vectors_len) { nvectors = group_memory_nvectors (group_size * vf, nunits); - unsigned factor = (vecmode == vmode) ? 1 : GET_MODE_UNIT_SIZE (vecmode); vect_record_len (vinfo, slp_node, nvectors, vectype, factor); - using_partial_vectors_p = true; } - else if (targetm.vectorize.get_mask_mode (vecmode).exists (&mask_mode) - && can_vec_mask_load_store_p (vecmode, mask_mode, is_load, NULL, - elsvals)) + else if (partial_vector_style == vect_partial_vectors_while_ult) { nvectors = group_memory_nvectors (group_size * vf, nunits); vect_record_mask (vinfo, slp_node, nvectors, vectype, scalar_mask); - using_partial_vectors_p = true; } - - if (!using_partial_vectors_p) + else { + gcc_assert (partial_vector_style == vect_partial_vectors_none); if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't operate on partial vectors because the" From patchwork Sat Dec 6 16:55:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126054 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6A0BD42DB866 for ; Sat, 6 Dec 2025 17:05:03 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6A0BD42DB866 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=SnPa2npY; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=SnPa2npY X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazon11013060.outbound.protection.outlook.com [40.107.162.60]) by sourceware.org (Postfix) with ESMTPS id 3E29C42DED97 for ; Sat, 6 Dec 2025 16:57:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3E29C42DED97 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 3E29C42DED97 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.162.60 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040221; cv=pass; b=fVCQ3UaeNlYyxBqtQEKLm+Df8gt2JzYgnQOr763THsloX3jDaq/BpLjpE7ylsqHhLur62wTRUeoXc7dZTqs/K5Zvdo61YPcHzxnDYh59OCdMXBllWoMD4R25yya+rS9Oq8Ruhhk47pxj4B3EeBdQqDPZL7qOJb+7L3ii2RTfnuk= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040221; c=relaxed/simple; bh=3xsysanz2G93u1DoNwiFSAlXIeFMNgCgHga19Jy2mDM=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=bT38OQNyn2+CSHB0eur1+fcVGrF00eqp/pQ86/Ll/OcL6/A1Q8PqDjaqdibi+u+LOFp1kLY4wbUnE/DfVCnAGSeD6Z1Lces2RL5tuC86ifqdSBq+Gfoq9vMiRlFAErnL6GEhnpbcZIvuAvleN/1ZieTQ8oxcAIcCIiAQ/pFd4XM= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3E29C42DED97 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=RcHdm9GyGFiJED0+W4k44oKNYv44MZN3pTcqAF+YJoqG9fizqIYGqP4Gb2+VGR7mmtlfgOkrT0mRxlRRSdxITZNReRh74KK4ZxwupDragkDOtimdyysfB9sRK7LdJVoPkUosJhok/RSY/2VAF372xJdQQPUmKGDmkrREspGMNJt7SYwHsXdgNlCzWPxZIHwUH44UBPJasMCZe+I84nJfbkbepNv/LoqGNEoP26q4BBwUkcaklYMz/VoR3/R1+mRExVni9xFRDF0twrsRhIj4RxKe5aPyOVBma9X4wMFNLAxf8yRDlSRnVXfaTLLHUF5cxI/765SVGkp8N7pT+piybg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f3H+iQqeINWJ8J/thcYJmV73zMjE1kmpZGcW/YSYAsk=; b=WCQLkXayZhT4Jph3TCc4bFALN6N26waZfP9wIDaEnX/l1Ibj3UVzHDq2E7jvaJaEHrTODOdZWeOQY+gq8Rw9t3q0wzxc/OYm0PHoQm4VUnyk5SY0S/TDlFUXAhpT21SGKB58Y/RCBzeHJhkAiU+iHln9xKotxBwhwraDxyEjnoH6hnOvJKBfwxpLLxT/4ZvrQKw9UqYIr4GoTGY8N0s7s0/ubItXRzcR/OB9n5xFSU4pF6nnPAVSuFk//DDk7Ai60kSnbKvzmDVDXj+s9f7PJ3d4Xc/rS8ShaJ6Zt6x+LqY/cSM6/ox8MtJ65C24AqY8hMsmWmLIg67uuw9nwt4djA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f3H+iQqeINWJ8J/thcYJmV73zMjE1kmpZGcW/YSYAsk=; b=SnPa2npYqmqvTHoQtheUcWenNCmJPog7OkVHX4gtVCw8MUtKqO4a36QzrA9G6bhohAlpA8Sh3ZOaKWzNu3CTR2XRJlyG6E8otXDfKXG6sLfl0Genn9qQU8U9ucaagtvrXdeTUjTvGbfmfpA0PXGO5Sm3zHR8kSc5TzSqheb3gTs= Received: from AM8P191CA0028.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::33) by PAXPR08MB7527.eurprd08.prod.outlook.com (2603:10a6:102:24d::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.12; Sat, 6 Dec 2025 16:56:56 +0000 Received: from AM2PEPF0001C710.eurprd05.prod.outlook.com (2603:10a6:20b:21a:cafe::dd) by AM8P191CA0028.outlook.office365.com (2603:10a6:20b:21a::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AM2PEPF0001C710.mail.protection.outlook.com (10.167.16.180) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:56:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=j/2jpxWhCO7bbE3934QIUDjilsiUHmYV4SgIcu3t44iF77wM9vInGVjrYRU7KRrT5QZtzkiaQuB3kRTil1M5ZKvngL6SQJ1sKitFJJRb5uURGIQmUVh5CKst7zRvqW95409Pm2H9DFEZxNqNqZ2EXSkg5JCa0K2bt0/74Mpwv+crzriLb20I2fIYYCt36yw17LVO58aEpyDiJ6xq0VfxaTbfANbRAQLHmmxpI08IF2nnvQU/zNc7PXqJ2WJZtSQ/viymGo5EGbpul0oPbApoe65+qnKq5bWhdH8sgCInpe5inVWiiA4HGa0AH9iP2ehffsu8h6jbZm41W/U37BUhWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f3H+iQqeINWJ8J/thcYJmV73zMjE1kmpZGcW/YSYAsk=; b=Q7wZIObsaIxLd+6eRIJYgs+w225mzG12et4KEXqGjIAjF7XwXMKCV9Y3UyHhdALm+l4lvGMxFWTW27CnA/dgl+XN41CUclYgULpiTz0HnI8+JhNoQmsmE9wHEzkSFKCPsBeXspTLUwk/QsaiatFb2cy9nRcec9MjQL8vuOT57KcZu75hVgbSmy4SVux0IFYcH78CtaNQI4F65QcncCcCVCFyCVmjY3MgEgGSYo8KnsY34dkMhDGkWD/aeDnnm312PReCDv8Qcj9eUmYQCdTIw+tcyI18WAlOsF6dDM7KPBwzOaI81oS5/dg+Yh1MMaVsuIlOdqcXza8vYi0ulsoLNw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f3H+iQqeINWJ8J/thcYJmV73zMjE1kmpZGcW/YSYAsk=; b=SnPa2npYqmqvTHoQtheUcWenNCmJPog7OkVHX4gtVCw8MUtKqO4a36QzrA9G6bhohAlpA8Sh3ZOaKWzNu3CTR2XRJlyG6E8otXDfKXG6sLfl0Genn9qQU8U9ucaagtvrXdeTUjTvGbfmfpA0PXGO5Sm3zHR8kSc5TzSqheb3gTs= Received: from DUZP191CA0041.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:4f8::20) by AS4PR08MB8094.eurprd08.prod.outlook.com (2603:10a6:20b:589::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.12; Sat, 6 Dec 2025 16:55:50 +0000 Received: from DB1PEPF00039234.eurprd03.prod.outlook.com (2603:10a6:10:4f8:cafe::3a) by DUZP191CA0041.outlook.office365.com (2603:10a6:10:4f8::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:55:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF00039234.mail.protection.outlook.com (10.167.8.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:55:49 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX03.Arm.com (10.240.25.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:49 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:55:49 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 06/11] New parameter for vect_maybe_update_slp_op_vectype Date: Sat, 6 Dec 2025 16:55:13 +0000 Message-ID: <20251206165518.5449-7-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF00039234:EE_|AS4PR08MB8094:EE_|AM2PEPF0001C710:EE_|PAXPR08MB7527:EE_ X-MS-Office365-Filtering-Correlation-Id: 5e44c805-468b-4e85-ad82-08de34e87638 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|376014|36860700013|82310400026|1800799024; X-Microsoft-Antispam-Message-Info-Original: WTwCJ52u8zMCwXZHNZ19IqzXtmwGCKzZEVGM180ru2pIpGKVOTZav3Ytf0PYtwDY32ajDQqnA979SPEh8Sl7mRWK2/Xbjhw55aXD7pqh3co82ekcuh1fMLLRWbQGdJLFjkq6p+z0t0FN4orYgiXBCk6I2namxKqxcj+Ccd7JVe6bR0xB4FAEgZeZG/lN1pVFrpOopTb9/lJHHn0dRcyICF68Pbi5vkY2rUZuuYSCvg6GzVWNujumocMzwgJvaFd6m3g1+vWdffW4KxX1o+sxFjVsLo6Csd/XeDhn2SJ+dHFNnrdFHY9fx6i1eZ445hPg439Ms40hABXQ46m0qmhr2bknFxhrIDCQpC+lm1287xnXvPOJB1flBEu9506w7D5lP6nmCz1Gc1YloWcxy+kHA8tnxdzkd20caMBv/NjPd5e4n0rAG4kvmYQ3OCAFsFJ3ujOuz2DXLm56HxayZQ7oLcbaH0cSpsSUatalslI28SzCnV07HLezBKj8wc4Ez29VVet9qHnq+24ya1ylGFBTJSAE7135DM5U4tXkBk4vy+VVt0BBUw7vDD8+p6zQuW+BUB+Ieshh+WUdhMD8NW0veJj1J63FxSGfBGlx8nB3Po8UYw6PLMbAZrcwY+wXZq+4lgdZGYNtjQ4v+gL30E8geSJ7/5WVEjkVuxIFlfDUj7V/bUaq/U5lHyjLSStKtpVs3RwzzJTJnskWo61Lf1pL3eMT6bz0WXVs0MjjSWo4rHpuacLl5LwIM5dJpftOheAnNXjee6okS55Ir9I5VRXN9g/qUPzefKTAbEzVEjYZNK4XH8AMWKv7VtZ9L7T/PNk7u2HZGAgsOPwmTiwn/dZNZY+g0vRPBjNxg0MVS1PifNp5QXvGUdf62xGJzDJFI2EL8snkt5O49S6LSpuxCpPWGM/Qd/aIjLD2dM9OxWEskOFenqzR5leCfDhl11ijuVNuRiRJbbdWQpsw5HSPpTBJQ6f6PzJQLj3OYShG4oPPQm5aK7QnJNDeoAkXe6WICTwEWpLAYfo4oREe3JbOReQ1yUfj2DF9SkRg7AAgRQcP5VTksY5gVPXZbHEXEFhOauW544EDufLuZ9iIk1Hwg2gf7e4uy311jYFOlt6EEIaZyBBEakj3FDdbU/xKPZaxyBEChAVt2gks6wPb11/FnojuZ5cjSRJ8OwlpUOmp9B6+qo2gNd0l8St5QJCqf5LDnK2vrjB7G3vXNrPpkP4tdRWN3ZOZ464ihVereg0yMUlwA7QMxN46zqQEU7LPIiXszmEoAVt2Urf68qoOM1dpJV/BCRrMRxz41F1I6ZEH9bL6G8zWhGnQbekEmj2SEl6ZOY3UiXTtr2yJ5TTm3hXHaXLauHx3MOigyBGgJsziqrnXQV/8mhiUR/2oEk7lb422PktglU/x27DnJY6S2OuvR+fLOP1sItWtX/rHTftDkeN3/8EeNCS+eqASdF/2h0rTbeeOS06YhVQdEsWRLwUVs4MKrCK3d145RQHR37/2++RkcV2LnxX8IE3JnLz+iRN6molP+j5ANnm6+zqff0csoSGtq23zwJ6N4hfh8BVLqjpamOg= X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(36860700013)(82310400026)(1800799024); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR08MB8094 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM2PEPF0001C710.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 010c576b-db79-4ef3-eff8-08de34e84f3c X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|14060799003|82310400026|35042699022|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: QgaCADHkXSSij4BadywH9MjcyzX3kUn+T0iJqdzSaEacDs3OtQ5HCvpBEEXL0WgDrC4Q14tQARq6MPEPlBFVHOdcSQ93g14pc4KkT1f0BGSeiSSxnkQI+L5p4Nnw3BVlHEed8aArPlVLOy6knrWHKdxm7TVgddGqPURuiN7N8J/oAqNupfsPXPomUt5vTYzdEUBQ28SmwO08BkqmcCAsml1FRJO4zvFLfVTuDBqiTZOX7ugirJUWTQBHGZOIKEGfhKciBuwhrMsDb+AYP4Wp/F1yGHR5Os+dJgIKqPMVWAwSqeugMVYGxZwc/hZofAVRa6/ItcHjOr3qu8hDK+9zwV1NzAiqn6QGqGS6Q2509rpykMinvDu/w40u/WVpHm9h+7wgIwRv4hlv++R/Zx4nyVT2203IrG87O8JJMkCWD5nwsbpLS5dkAeeqTLFMc9TxjKBehZij/NXUFDL0tE5HAcvLBV+V92TG46uDeq0BdZaJkFnOybDKnvFNF0YZohMrj3brsbsuoKCG0Xc2efiT5WdBe3MdOso4zAyZOeWw6gaVyx/xsi2EfFtWRMgvhgehN3yGiQF93DCvCgQYow26RyL8NRA85Cp8cZRneZdbTH60hlrxkKn2HFB6A5lcn89KlEtlj32mjstBEzk7EnI7HtEfG47NDrlz42jsn7Z4Q2CJbuEcXeXrxbx0+ayx7Z4EfnAQuRcTQ/zgCV1hfk36QQHxwHuh1iahnycrfTu8vC2GSQ6Jwf+AsHjwZf0qfNkv4/KdiQFkAdKr3jFLMEO2iOvowU+57r5BKBf4WG//g8OZI93wdLw607blkxCgXMD2N2gB1rleYpDEuOAe581JxkK33KN6y645OUCzeuHbazVqaLqiR5vESlRHdBm2u0u83Ycx8bslrhec8oksPw+C1ZMOVKMuEl8qY/2jWxF1SveENiV5Dx+J0UHq1f2ArMin6sPdiWsnASG5QLpJQcPm+zP3vV1ZMC0TdnNnTUG7V7roymq/X8EymmZfP85GkD93bcQfV/QViYplLMC8UCzrQ/NVGYj16CjS221npjW+UUJ1HpB/iv2K24Ek9W4yD+cC6HVxLgIWyip1W+MFaD/OOglQ0rNmfdI6tGk+7O9WdxVEqTkkTUWK7/L5bwL2RReTSah+lxjl/+im21xW38FYUFKY1KE5qnmXC49ONanbyuD/dIQYWqaSh6gnXg1wgEKvfmzc3IqrEV4Aa7YjYGO1LFaqpEUeFf5ojK5LtpLEYfu+diPGK8gmpbfakg8DCeitt38KcUyfh/GIKTeVhH7rFjtBhQrgdJQo1K23Oz1x696PkXSJErOgGMk4ENrAvh2TE/irS2dO05b6nmf7f/h9P1SCi4jYDYoDcyTO+mjdBjfXmG4JiJdw6qI8Ckqb4XlXhbWRuwplXxNZf3uRFDRJe8NC51ski6pe8IvM8HEqOR5iYLGRSYBYtrlzXuCIB2YhypVvyacUs/HlEQN/PMuRAZ7e+Rh+u1rkO3wKShPs7z9tKwQhZ6V4CR02Y3gJ93EiWGP6Ld070R1KHziXBhMKhUSUfoI2Uko48wOuGIr5rQY= X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(14060799003)(82310400026)(35042699022)(36860700013)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:56:55.3245 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5e44c805-468b-4e85-ad82-08de34e87638 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AM2PEPF0001C710.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7527 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org Update all callers to pass a pointer to the vectorizer state into this helper function. Its value is temporarily unused but will be required for BB SLP with predicated tails. gcc/ChangeLog: * tree-vect-loop.cc (vectorizable_lane_reducing): Pass loop_vinfo to vect_maybe_update_slp_op_vectype. (vectorizable_reduction): As above. (vectorizable_lc_phi): As above. (vectorizable_phi): Pass vinfo to vect_maybe_update_slp_op_vectype. (vectorizable_recurr): Pass loop_vinfo to vect_maybe_update_slp_op_vectype. (vectorizable_induction): As above. * tree-vect-slp.cc (vectorizable_slp_permutation_1): Pass vinfo to vect_maybe_update_slp_op_vectype. * tree-vect-stmts.cc (vectorizable_bswap): As above. (vectorizable_call): As above. (vectorizable_simd_clone_call): As above. (vectorizable_conversion): As above. (vectorizable_assignment): As above. (vectorizable_shift): As above. (vectorizable_operation): As above. (vectorizable_store): As above. (vectorizable_load): As above. (vectorizable_condition): As above. (vectorizable_comparison_1): As above. (vect_maybe_update_slp_op_vectype): Update definition to accept a parameter of type vec_info *. * tree-vectorizer.h (vect_maybe_update_slp_op_vectype): Update declaration to accept a parameter of type vec_info *. --- gcc/tree-vect-loop.cc | 20 ++++++++------- gcc/tree-vect-slp.cc | 2 +- gcc/tree-vect-stmts.cc | 57 ++++++++++++++++++++++-------------------- gcc/tree-vectorizer.h | 2 +- 4 files changed, 43 insertions(+), 38 deletions(-) diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 3ac264f0ce3..9164cef92da 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -6800,7 +6800,7 @@ vectorizable_lane_reducing (loop_vec_info loop_vinfo, stmt_vec_info stmt_info, return false; } - if (!vect_maybe_update_slp_op_vectype (slp_op, vectype)) + if (!vect_maybe_update_slp_op_vectype (loop_vinfo, slp_op, vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -6937,7 +6937,7 @@ vectorizable_reduction (loop_vec_info loop_vinfo, unsigned j; slp_tree child; FOR_EACH_VEC_ELT (SLP_TREE_CHILDREN (slp_node), j, child) - if (!vect_maybe_update_slp_op_vectype (child, + if (!vect_maybe_update_slp_op_vectype (loop_vinfo, child, SLP_TREE_VECTYPE (slp_node))) { if (dump_enabled_p ()) @@ -7055,7 +7055,7 @@ vectorizable_reduction (loop_vec_info loop_vinfo, vectype_op = get_vectype_for_scalar_type (loop_vinfo, type_op); if (!vectype_op - || !vect_maybe_update_slp_op_vectype (op_node, + || !vect_maybe_update_slp_op_vectype (loop_vinfo, op_node, vectype_op)) return false; } @@ -7744,7 +7744,8 @@ vectorizable_reduction (loop_vec_info loop_vinfo, if (single_defuse_cycle || reduction_type == FOLD_LEFT_REDUCTION) for (i = 0; i < (int) op.num_ops; i++) - if (!vect_maybe_update_slp_op_vectype (slp_op[i], vectype_op[i])) + if (!vect_maybe_update_slp_op_vectype (loop_vinfo, slp_op[i], + vectype_op[i])) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -8449,7 +8450,8 @@ vectorizable_lc_phi (loop_vec_info loop_vinfo, /* Deal with copies from externs or constants that disguise as loop-closed PHI nodes (PR97886). */ - if (!vect_maybe_update_slp_op_vectype (SLP_TREE_CHILDREN (slp_node)[0], + if (!vect_maybe_update_slp_op_vectype (loop_vinfo, + SLP_TREE_CHILDREN (slp_node)[0], SLP_TREE_VECTYPE (slp_node))) { if (dump_enabled_p ()) @@ -8525,7 +8527,7 @@ vectorizable_phi (bb_vec_info vinfo, "PHI node with unvectorized backedge def\n"); return false; } - else if (!vect_maybe_update_slp_op_vectype (child, vectype)) + else if (!vect_maybe_update_slp_op_vectype (vinfo, child, vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -8714,7 +8716,7 @@ vectorizable_recurr (loop_vec_info loop_vinfo, stmt_vec_info stmt_info, unsigned j; slp_tree child; FOR_EACH_VEC_ELT (SLP_TREE_CHILDREN (slp_node), j, child) - if (!vect_maybe_update_slp_op_vectype (child, vectype)) + if (!vect_maybe_update_slp_op_vectype (loop_vinfo, child, vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -9591,8 +9593,8 @@ vectorizable_induction (loop_vec_info loop_vinfo, unsigned j; slp_tree child; FOR_EACH_VEC_ELT (SLP_TREE_CHILDREN (slp_node), j, child) - if (!vect_maybe_update_slp_op_vectype - (child, SLP_TREE_VECTYPE (slp_node))) + if (!vect_maybe_update_slp_op_vectype (loop_vinfo, child, + SLP_TREE_VECTYPE (slp_node))) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index 3becf34c09b..d76b18dd7d9 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -11514,7 +11514,7 @@ vectorizable_slp_permutation_1 (vec_info *vinfo, gimple_stmt_iterator *gsi, FOR_EACH_VEC_ELT (children, i, child) { if ((SLP_TREE_DEF_TYPE (child) != vect_internal_def - && !vect_maybe_update_slp_op_vectype (child, op_vectype)) + && !vect_maybe_update_slp_op_vectype (vinfo, child, op_vectype)) || !types_compatible_p (SLP_TREE_VECTYPE (child), op_vectype) || !types_compatible_p (TREE_TYPE (vectype), TREE_TYPE (op_vectype))) { diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 8ffdaacb5d2..44759a360e4 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -3458,7 +3458,7 @@ vectorizable_bswap (vec_info *vinfo, if (cost_vec) { - if (!vect_maybe_update_slp_op_vectype (slp_op[0], vectype_in)) + if (!vect_maybe_update_slp_op_vectype (vinfo, slp_op[0], vectype_in)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -3793,7 +3793,7 @@ vectorizable_call (vec_info *vinfo, if (cost_vec) /* transformation not required. */ { for (i = 0; i < nargs; ++i) - if (!vect_maybe_update_slp_op_vectype (slp_op[i], + if (!vect_maybe_update_slp_op_vectype (vinfo, slp_op[i], vectypes[i] ? vectypes[i] : vectype_in)) { @@ -4486,7 +4486,8 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info, if (cost_vec) /* transformation not required. */ { for (unsigned i = 0; i < nargs; ++i) - if (!vect_maybe_update_slp_op_vectype (slp_op[i], arginfo[i].vectype)) + if (!vect_maybe_update_slp_op_vectype (vinfo, slp_op[i], + arginfo[i].vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -5716,8 +5717,8 @@ vectorizable_conversion (vec_info *vinfo, if (cost_vec) /* transformation not required. */ { - if (!vect_maybe_update_slp_op_vectype (slp_op0, vectype_in) - || !vect_maybe_update_slp_op_vectype (slp_op1, vectype_in)) + if (!vect_maybe_update_slp_op_vectype (vinfo, slp_op0, vectype_in) + || !vect_maybe_update_slp_op_vectype (vinfo, slp_op1, vectype_in)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -6070,7 +6071,7 @@ vectorizable_assignment (vec_info *vinfo, if (cost_vec) /* transformation not required. */ { - if (!vect_maybe_update_slp_op_vectype (slp_op, vectype_in)) + if (!vect_maybe_update_slp_op_vectype (vinfo, slp_op, vectype_in)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -6398,13 +6399,13 @@ vectorizable_shift (vec_info *vinfo, if (cost_vec) /* transformation not required. */ { - if (!vect_maybe_update_slp_op_vectype (slp_op0, vectype) + if (!vect_maybe_update_slp_op_vectype (vinfo, slp_op0, vectype) || ((!scalar_shift_arg || dt[1] == vect_internal_def) - && (!incompatible_op1_vectype_p - || dt[1] == vect_constant_def) - && !vect_maybe_update_slp_op_vectype - (slp_op1, - incompatible_op1_vectype_p ? vectype : op1_vectype))) + && (!incompatible_op1_vectype_p || dt[1] == vect_constant_def) + && !vect_maybe_update_slp_op_vectype (vinfo, slp_op1, + incompatible_op1_vectype_p + ? vectype + : op1_vectype))) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -6829,9 +6830,9 @@ vectorizable_operation (vec_info *vinfo, } /* Put types on constant and invariant SLP children. */ - if (!vect_maybe_update_slp_op_vectype (slp_op0, vectype) - || !vect_maybe_update_slp_op_vectype (slp_op1, vectype) - || !vect_maybe_update_slp_op_vectype (slp_op2, vectype)) + if (!vect_maybe_update_slp_op_vectype (vinfo, slp_op0, vectype) + || !vect_maybe_update_slp_op_vectype (vinfo, slp_op1, vectype) + || !vect_maybe_update_slp_op_vectype (vinfo, slp_op2, vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -8306,9 +8307,9 @@ vectorizable_store (vec_info *vinfo, vls_type, group_size, &ls, mask_node); - if (!vect_maybe_update_slp_op_vectype (op_node, vectype) + if (!vect_maybe_update_slp_op_vectype (vinfo, op_node, vectype) || (mask_node - && !vect_maybe_update_slp_op_vectype (mask_node, + && !vect_maybe_update_slp_op_vectype (vinfo, mask_node, mask_vectype))) { if (dump_enabled_p ()) @@ -8735,7 +8736,7 @@ vectorizable_store (vec_info *vinfo, above only handles the mask and the first store operand node. */ for (slp_tree child : SLP_TREE_CHILDREN (slp_node)) if (child != mask_node - && !vect_maybe_update_slp_op_vectype (child, vectype)) + && !vect_maybe_update_slp_op_vectype (vinfo, child, vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -9900,7 +9901,7 @@ vectorizable_load (vec_info *vinfo, if (costing_p) /* transformation not required. */ { if (mask_node - && !vect_maybe_update_slp_op_vectype (mask_node, + && !vect_maybe_update_slp_op_vectype (vinfo, mask_node, mask_vectype)) { if (dump_enabled_p ()) @@ -12236,13 +12237,14 @@ vectorizable_condition (vec_info *vinfo, || !expand_vec_cond_expr_p (vectype, vec_cmp_type)))) return false; - if (!vect_maybe_update_slp_op_vectype (SLP_TREE_CHILDREN (slp_node)[0], + if (!vect_maybe_update_slp_op_vectype (vinfo, + SLP_TREE_CHILDREN (slp_node)[0], comp_vectype) || (op_adjust == 1 - && !vect_maybe_update_slp_op_vectype - (SLP_TREE_CHILDREN (slp_node)[1], comp_vectype)) - || !vect_maybe_update_slp_op_vectype (then_slp_node, vectype) - || !vect_maybe_update_slp_op_vectype (else_slp_node, vectype)) + && !vect_maybe_update_slp_op_vectype ( + vinfo, SLP_TREE_CHILDREN (slp_node)[1], comp_vectype)) + || !vect_maybe_update_slp_op_vectype (vinfo, then_slp_node, vectype) + || !vect_maybe_update_slp_op_vectype (vinfo, else_slp_node, vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -12683,8 +12685,8 @@ vectorizable_comparison_1 (vec_info *vinfo, tree vectype, } /* Put types on constant and invariant SLP children. */ - if (!vect_maybe_update_slp_op_vectype (slp_rhs1, vectype) - || !vect_maybe_update_slp_op_vectype (slp_rhs2, vectype)) + if (!vect_maybe_update_slp_op_vectype (vinfo, slp_rhs1, vectype) + || !vect_maybe_update_slp_op_vectype (vinfo, slp_rhs2, vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -13864,7 +13866,7 @@ vect_is_simple_use (vec_info *vinfo, slp_tree slp_node, for example when conflicting vector types are present. */ bool -vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype) +vect_maybe_update_slp_op_vectype (vec_info *vinfo, slp_tree op, tree vectype) { if (!op || SLP_TREE_DEF_TYPE (op) == vect_internal_def) return true; @@ -13878,6 +13880,7 @@ vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype) && SLP_TREE_DEF_TYPE (op) == vect_external_def && SLP_TREE_LANES (op) > 1) return false; + (void) vinfo; /* FORNOW */ SLP_TREE_VECTYPE (op) = vectype; return true; } diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 1830c29819a..b66aca240f4 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -2499,7 +2499,7 @@ extern bool vect_is_simple_use (vec_info *, slp_tree, unsigned, tree *, slp_tree *, enum vect_def_type *, tree *, stmt_vec_info * = NULL); -extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree); +extern bool vect_maybe_update_slp_op_vectype (vec_info *, slp_tree, tree); extern tree perm_mask_for_reverse (tree); extern bool supportable_widening_operation (vec_info*, code_helper, stmt_vec_info, tree, tree, From patchwork Sat Dec 6 16:55:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126052 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A3E884143B00 for ; Sat, 6 Dec 2025 17:02:32 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A3E884143B00 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=ZYh+UFCA; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=ZYh+UFCA X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from GVXPR05CU001.outbound.protection.outlook.com (mail-swedencentralazon11013001.outbound.protection.outlook.com [52.101.83.1]) by sourceware.org (Postfix) with ESMTPS id 69CD24360BB4 for ; Sat, 6 Dec 2025 16:57:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 69CD24360BB4 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 69CD24360BB4 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.83.1 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040229; cv=pass; b=niizR+yYY1SyRxLVq5Rr3KBg+iWGIiequLhMnpot57Xqr1YFNsXs/9m/V/N6i/dzHuF9cURFdNYLkn/fzdqOif65rIVeiJuxjAVcaXEaW2zPcnBQtxbiuNR3/plbM6DLjREEs5JVQE8S5ozlXBB8LKFCv6GkjEEyF16o57tRq7o= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040229; c=relaxed/simple; bh=jLuEoHePVGJ/4vupE+2VSyZi+EyECvFxVxsbh4uJL9A=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=MxFc00yXA0YmWrv4HAssJj0VngwRZ0p7T7neMn62hEX8YCsWURfTvTHJYWMiywXArr5XGjlq6GESdmg9CUF582rJHnhBgTiSylXrsu/dZK7AYQoctzDRkWOetLbL8MLmHef9yiy3vTXzPBWCdt5ipY/EdFijGlmTY8BukDE5VQI= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 69CD24360BB4 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=fKsgYS+fbfz0cxkTf9XxlQIFlpxNGFP8YhaRf16RcK+ukLlf0eYlaTwKcPJUvVlxHyYut1YHY6DGfToWe8kLNWEaKWMF4j2s2nvPybxCM94qR0hhYy7HLAW1bViSeavbedvelG955NeJEcSth69tpOvufvaWhrK/oh/XbzjMQSURiJEEiUyGX6n3tH0b4SZzAfOx3dikg/PRWS4yb2lQUiCfZSXFkPE7BZlzldWo7ftt3pjaGL7hR+Ur4EH0/ced8XT74wdDfClv9fw1UpC/GxgDpSj36+7G7wb3iEPk4bxt4fjZy2fUXstEv3O2TIWBAXhdVpFSjrk33HLrL1ZTfA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bFhXNHfiX9b9O7NvV2cIVVEwutlZTezUT9N44b7XmIA=; b=AgNoVjvJjhvkDGy7qSWA5IPQ8HjWBxRhtrpfptJlFH583zul2I2jXpSCQcchMnAlx/Q+Y7E/OjvYCjaE+xAVQCSw1UzC2LRH4U4x9wwx/hF4Ce8idzr5D2c/TtH0Oxu39rPm2sVJ/C5IgOrlSsYdPR+8G28JxpD5FfWALcj2q4rs3ePrzAsnE1cjfWq0eAhnRDR+RTVgbdMLOgM5zjiZlfJ0+6znFFvhZK8tSHgXToVcOWhCTbiPylfGChRpCZyIiZqNHnQgTZbRgtGZyzZij/maong2JfajnMm2qkb5M/pak7bFRCgoGpNCUjp8kcmIVoR+Ofs1mmO1lsauOYCjtA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bFhXNHfiX9b9O7NvV2cIVVEwutlZTezUT9N44b7XmIA=; b=ZYh+UFCAjdCN9/+JAzlLd8Stbk8cl0LlodSpzCiDffOCYX7FYCMaLgrzbbqrw/4d+j8R3Futel7LRA/GuINa5SnGbxvxYmD8uTvZ6KFJfEqChOYAFVPKn5W7+/YHC+Fql1i3RmrRfHECdEHalDtRPMSK60reSJzJIYZ+pAChJ+A= Received: from AM0PR02CA0134.eurprd02.prod.outlook.com (2603:10a6:20b:28c::31) by AM8PR08MB6514.eurprd08.prod.outlook.com (2603:10a6:20b:36b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:56:59 +0000 Received: from AMS1EPF0000004B.eurprd04.prod.outlook.com (2603:10a6:20b:28c:cafe::66) by AM0PR02CA0134.outlook.office365.com (2603:10a6:20b:28c::31) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AMS1EPF0000004B.mail.protection.outlook.com (10.167.16.136) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:56:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CWb11qoxPGEbbo56LhxSeIXXunGhXHjQ/vltbPWiLadLNpblwbcyKF/6JXFmbPNdGMCqOavsqr/rxbNef6L/vQ4eMudHULpDOnI4NTaEAuQj3R+3tR9fs8ORiMKjCbeJY2abHd2YCfLSaG7zP8nMLdvZVm13wsNCFsMB3S3ms2Hh4/dBqtbxDAyqLkkUPkp27H+dUG/Wz0ez396AJM7Po83ePSBVW360bKqgAU5RF+UCNGlK5Lt9/1vRx3AygIZyCCkgeqpungTD8p/fa+j8SE6vtEoPk5gI5vwx4EXhjmbSeEognI/UIbCo5ONxeMJtQwKXd8yabwkYIkduPxGq/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bFhXNHfiX9b9O7NvV2cIVVEwutlZTezUT9N44b7XmIA=; b=E+79Xeim6y7OYKM2ZRPoXAXrJK/JpcX2THQXdFkhYuLJKnDufxeRxddi7h6XoiT3ALrKmQCGcyKAKSmyaTGPFj/K7XXJeEHVmcOc6RXpzxQ7r3AvEB/ocFVG7SQxRvlZeT4rPKgbE2p8WObX1SAJJuhwbRQKC0o/1fiIK78Gu/7hXbQwoROVWlp07vsKkWDFO2DIWiKZmjEPM2NdDrCkhTWjaBHUMZqeNFFlBdfkRBsO0YIre/Ij2bRDlLPKnqZ0FANbxrTlJBtgHHqQCPiaiPaq5B/xsjQZLfTiso8J2iMZxd7xpV9duRhCMo0itCZ94t5MV8TMI9fqqYqXdbx1DQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bFhXNHfiX9b9O7NvV2cIVVEwutlZTezUT9N44b7XmIA=; b=ZYh+UFCAjdCN9/+JAzlLd8Stbk8cl0LlodSpzCiDffOCYX7FYCMaLgrzbbqrw/4d+j8R3Futel7LRA/GuINa5SnGbxvxYmD8uTvZ6KFJfEqChOYAFVPKn5W7+/YHC+Fql1i3RmrRfHECdEHalDtRPMSK60reSJzJIYZ+pAChJ+A= Received: from DB9PR01CA0002.eurprd01.prod.exchangelabs.com (2603:10a6:10:1d8::7) by DU4PR08MB11213.eurprd08.prod.outlook.com (2603:10a6:10:577::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:55:55 +0000 Received: from DB1PEPF000509F5.eurprd02.prod.outlook.com (2603:10a6:10:1d8:cafe::3a) by DB9PR01CA0002.outlook.office365.com (2603:10a6:10:1d8::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:55:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF000509F5.mail.protection.outlook.com (10.167.242.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:55:55 +0000 Received: from AZ-NEU-EXJ02.Arm.com (10.240.25.139) by AZ-NEU-EX04.Arm.com (10.240.25.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:55 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EXJ02.Arm.com (10.240.25.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:54 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:55:54 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 07/11] Handle variable-length vector types in store_constructor Date: Sat, 6 Dec 2025 16:55:14 +0000 Message-ID: <20251206165518.5449-8-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF000509F5:EE_|DU4PR08MB11213:EE_|AMS1EPF0000004B:EE_|AM8PR08MB6514:EE_ X-MS-Office365-Filtering-Correlation-Id: 3111297c-2e31-49ab-719b-08de34e87804 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|82310400026|1800799024|36860700013|376014; X-Microsoft-Antispam-Message-Info-Original: wGSNAFIIrrpOBfJKL8/wl51NppArwZk0rLxnt8AHGZmHUG5WIGzIsklC/aPue6lbPtnxr+NIsabmqJY7aFUeGRArJ6Y4xp2kS5fkbt2LisSuTxxiYWIH8PjSMVcqBXeKgSCZlF97ylLSi2Jmm/6/stZbEcIE+Kl3LTtnDucdiyxdzBdEZGNHHtUTWq/iRt3EZCEA0U2kDt8C+YnWtwDfWCDH2q8K9QlvdMH2ganZW2GIaG5rUDpcA8GtgkEuTcMKPLlJyVG833dx1KkqugrhxF8Hne0RfPQWnOwHhFtISVCUhc/SKcoTRGh8IgH0yrn5xaDYL1kcbjTJwA5qJgwBnkJCZg8fnWLjhyw70CW5b2t2HoC0Yiw1w4jQbYR6zZDt0Zz++Otg7lpPsPnNoZSNiYVCR9wxpL4daH/bdwTbOyG0yx9fKs0gwhwqc2LTfn4VzynyomH6Z9aHV7w8pcX7BEK4kUCqE3/jfDQW8SwDo9tzNN+RPSBY7fGVGBPqifEdLP3bsmeVghfi61oS8aIdi79+hJfK9VeV5OIBdGQidpXSyOYXCOzBNmngI6nuBwhniP1D8ntRJ/yhFLgWMOFMeY3Hxg/cAdFyswDEi1HEi7D7dZDAtAJYsJ6gOquzeSYpezYscsmotAYQk5oROf+8AGv3ZRsskT+ZthK3lyAQBwZ8JCBV3yJeLjlTsPDx00IONAusFk0HGFepsi1DaKK+wsCuB5AQOi9HDa6p6AGG5ClrdN3HYTMB5XcaOUpSQlXVI0GoDTB4h9M+DR+QY9vriXBjsgeGO8BNgnR4knukDn7yQUXDcEYcx81L34Ow9pgweO6Y9inPoQkH0Peo25/X8DyGGe4byk0XTqfxPLz8pq5ZY1LFgGBqPFVZaw++UuJgSqULaLBrNCy4nK733Ba9n7bKQw9M+rQJ+EuifH4aPX4ZOW6TeFBMltw3KhisfR7F7SIMKGPLbw8nnJjjsfdUx/b4mE8uoGUDMFwKBkYwEkG49Bke5MXRuHnmFUSrFv8dwDZYN2T08GiSgB5G+0aM54ztfYVBR93mI+DJgaNswxDxCe4+ovkn+bdtkdFJWJy0SjN4CbAoBUiNxI2bBnhIJ0sQyZrqTTs5Q67ZKraZQsosoY/3x71RI+Ad2J8k7LIJfmFIIIPX+muwi7M3jvbeo+prGJtSnNMRndTxTgWfT1gNYkKjFqXiRuHOdrTmP9yeYky3bTb4KdX+2cmWm41T6l+FVyx5hF/rMzOZmu+diIE/ZZxWN6IDfgGcdE4q21Dvr/wcM7Ttw80lOm981cLCAYgSVj+xwNfGHRI9L2c9MzCQE1RtUVEShW7Kjz62V+S1AzXON88dC4dA3p1KwsEio2/UDRqGPJlYQA3aQ2Lj9bxzlMvqDTH9QCMmGmZDuSYrG1NGL+MPSPENMqc0OgXV1HyBCVzs93mLN8RRzg+NoCjkFvnZ0VRJ/s3IavHufkYz0sGFsKyRZhn6lm5H2r1NB/wdC8Vb4Zi2C8shK2NkVgZNiVcHrx2B415nZENKHLZn1tMaA9QHZCoPp77BNspPH5cgNAhAm1C+XLrcrGNpGvM= X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(82310400026)(1800799024)(36860700013)(376014); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU4PR08MB11213 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS1EPF0000004B.eurprd04.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 1161c670-f74d-49a5-3a8e-08de34e85260 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|35042699022|376014|36860700013|14060799003|82310400026; X-Microsoft-Antispam-Message-Info: 6F4eHCa8lR5F7RX8Wa35r/XjLOVrHQd/QwTVQ9ngXNfN0ntcIpSo7UGS9f7qPiY8BdrsqklnxjydtjNQmwQPqeG7HM9ulm5ws4kVpR9cWMepkiQHw4X7qJ53CesyOg8UXrA3DN1JpLuwyEDxw+bo2P72Tw8e1JPNlNRqPRGJdr/YHKIkd8D95rZdaN1GzBvd71oPrT0nYeiK2exp+BgXCMt2qvQB/kLu5LwFSjPbiPzupL/O+h3b4XiTumynFhPJhsOg+FpikG7cLPDPsFPPeTTAGXl32o1W1fj47kdKogagkd39Cx7DZsoj16PuwkmK9vHqVFPxqyfISsQZqsgqce1Ha8fBjwCbZ+/hIfd0vCSG+8yaoL7KjR84D8WyduHSkMVvvf6WYo6lRM0mdewrfK+kNI/tqF5x+VIZN1GBijUAOuHRc/Xx4HQtuno5PPEuJ2qPeNci+m0XDrW4TNXd9QttBPsepGS+bDSIsnmwXytcrz8mhRzUbYrpFr9d1YRRf2HDgNfQpSNs5+MG4FTMjdSFs1szRriNbdsoiTsboRLRbm+tu3IHzllndUfi0Wq360TkaEkUaSXcbL99aax09C476o5uZnErnipL0adu0AWQbeDY3MXAWGQN0d6a+ATvh2390yxnL7YUu8OiQCg9t3JUG3uN/7ztaRdJf+OsVe1oi2cXWxmrPxJ+J8MF6LPi7j0gChJL4xzvi3PkyrUnCTRghU1fvPRrFaTYSU3uu9aj2Bf7iSBQkbxgvnYGKA4LCmvQV0+P0OjG4CgzYvOVTqkVgUNX8q4PBBxtfOvLxuzmYaQDlMkGfH8gRDhFIgVoNoUNGEVSijlR5q5E9BrBsYu9xiMyb6dtbAopmp+7vfN8CPBnnHdIgbyFK0PmrAlxoMQG8txXBoCG1moUb2V7zp3x1/hhC3OeH6dFAra9Nylz55XdY/91FJamNRRYIUYG8UQ8FiH/aS0kz6jxy1gnc3GMkwjfk+H6pUOhWqAHCNWMtmNgV+xc6FJRk7ZB/G1Yhkqw0a17Pm/zNGOjvG4zGNhdumNH57Lf5EZ8LiweroFWbg8FKlKv6b1t+7EYX9gkftoaHct4Mi4miTxSVYkoPnByX45IOqxGsI0qCFDn1johcpIuF4D8AxmjqmCfFsPTEiklNkdinm4nOe2UWkie/4+6gZ/AiIobbaA7JegXg/s9/02gk/sEmfsSrU+XfyqBsJFoflTEz0HeD2Tdv6UJWHVMMLu1bkVrM+iI8BVYbnReIZdXGw+BCReEXx+9dPxD9aHLE9+MwdizKcqxjvax5YqMsn12pDqYpaArhOPzn2GHg83kU/T48WyH1CQZzauuTJl6fHIjGzkOm2h3xYkyOAL1Z4m3jriYvX7IO6h9FN2j8kcmlgqGjdHxqIHpVgozDE+gEp9NETdWcmdrQmaJWbEpXbkEANuV3punHbH2dEG11ZK5wsv61DUTsWqfzEg0w8ZdlTQ9JmYrL3oGfIUNxnNYtpEsCna6J24PCvMuLLIJZYyf9wxQmFBO9bIXz6x7+AkKggyIjxXpGdOYdhYySovFNfGYTid1m2AB4fAv1bg= X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(35042699022)(376014)(36860700013)(14060799003)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:56:58.3485 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3111297c-2e31-49ab-719b-08de34e87804 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF0000004B.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6514 X-Spam-Status: No, score=-11.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_NONE, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org When given a constructor of variable-length vector type, the store_constructor function now builds a vector with the number of elements specified by the constructor and uses it to emit the body of the kind of insn chosen by the convert_optab_handler function. Previously, this function used a fallback path of calling store_constructor_field upon discovering that the number of subparts in the vector type was not a constant multiple of the number of subparts in the element type. For example, this allows GCC to generate the following AArch64 assembly language output for the tail of a reduction in the slp_6 test: uaddv d31, p6, z31.b uaddv d27, p6, z27.b uaddv d26, p6, z26.b movi d30, #0 insr z30.b, b26 insr z30.b, b27 insr z30.b, b31 add z25.b, z25.b, z30.b instead of the following output (with predicated tails for basic block SLP vectorization but without this change): addvl x0, sp, #2 movi d0, #0 st1b z0.b, p6, [sp, #2, mul vl] uaddv d27, p6, z27.b uaddv d26, p6, z26.b uaddv d25, p6, z25.b str b27, [x0] addvl x0, sp, #1 add x0, x0, 1 ptrue p7.b, vl3 ld1b z0.b, p6/z, [sp, #2, mul vl] st1b z0.b, p6, [sp, #1, mul vl] str b26, [x0] ld1b z0.b, p6/z, [sp, #1, mul vl] st1b z0.b, p6, [sp] str b25, [sp, 2] ld1b z0.b, p6/z, [sp] add z28.b, z28.b, z0.b st1b z28.b, p7, [x1] addvl sp, sp, #3 or the original assembly language output (with neither predicated tails for basic block SLP vectorization nor this change): uaddv d31, p6, z31.b fmov x0, d31 uaddv d31, p6, z26.b add w6, w6, w0 fmov x0, d31 uaddv d31, p6, z27.b add w5, w5, w0 fmov x0, d31 add w4, w4, w0 gcc/ChangeLog: * expr.cc (store_constructor): Add an else block to handle cases of TREE_CODE (TREE_TYPE (exp)) == VECTOR_TYPE in which exact_div (n_elts, GET_MODE_NUNITS (eltmode)).is_constant (&const_n_elts) is false similar to the existing "element type is not a vector type" case except that const_n_elts is taken from the constructor instead of the subparts of the vector type. --- gcc/expr.cc | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/gcc/expr.cc b/gcc/expr.cc index 7d84ad9e6fc..622c5be1d59 100644 --- a/gcc/expr.cc +++ b/gcc/expr.cc @@ -8058,7 +8058,12 @@ store_constructor (tree exp, rtx target, int cleared, poly_int64 size, similarly non-const type vectors. */ icode = convert_optab_handler (vec_init_optab, mode, eltmode); } - + else + { + /* Handle variable-length vector types. */ + icode = convert_optab_handler (vec_init_optab, mode, eltmode); + const_n_elts = CONSTRUCTOR_NELTS (exp); + } if (const_n_elts && icode != CODE_FOR_nothing) { vector = rtvec_alloc (const_n_elts); From patchwork Sat Dec 6 16:55:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126057 X-Patchwork-Delegate: tamar.christina@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 600844143B25 for ; Sat, 6 Dec 2025 17:08:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 600844143B25 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=f8EBZma0; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=f8EBZma0 X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from AS8PR04CU009.outbound.protection.outlook.com (mail-westeuropeazon11011056.outbound.protection.outlook.com [52.101.70.56]) by sourceware.org (Postfix) with ESMTPS id 3007942DB228 for ; Sat, 6 Dec 2025 16:57:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3007942DB228 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 3007942DB228 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.70.56 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040232; cv=pass; b=uRBeRWP5nRKkdpYoUAPSKxlwhfHEYfvZd4fvsbPFdGvQL/cS4aMaiTU4w+14Br+PNrRB/nucPKQLFf2KpFnTLF83BzzB+nQ8W3fqINzGRI+qeSqsaphG4myai0z91cHGsZPMslf8dndSStuSmtogmgi5FS/nKp21c811cjiZ238= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040232; c=relaxed/simple; bh=VVsLhcvlRM6gzrTipWPVG71BhDLcFn2oz7dyN5WVuMQ=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=tix+PvRhls16urQP+GGqHUcHZiE0Kml0v5pPR5Dmq3nv+qK9oda5Ugj05fpajVekk4KXsuez3r9/RQGyVjfkJutgmskJbQZKuRa+wwYC2Tc8ZXmTIpdCtBKbnzt1XUu8JsYIxeIqSMnFOMDT8Jn//rhJkQG6C+n/c2HQ12m7bAQ= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3007942DB228 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=QgOQ0Afqvde246FUx0scTFotyP0VxxmEzAhEOPWSZUYbOJo/brdzCxonDMnVgHTmrep2W1UPBtjA39epzqAbgs57Y9CLb7XyKFMFeoZSnmQrYBU9E/G9P2VuYvvM9V8HqtU7qngTcmIDFmFmxushCsuUJ1UdCr/JuReCHYvWsdWyVK6TH7ohyBiKZEd+A09JngYx9vFg4h0fLKVyFjnWpmOtfjgYzx7IJdsIuEBGRaO1IHKiTmgxOAuIX5iQY6GFURwBaSugJ7YVacBMs2s3mXVHY1Xonpo1vAIiyZHWkTNPJp9aCEQzWdpSvxlpuJl7SpHH77c3JMSaYJOOKU16IQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BVfmNMJWBkV9NHgT8wY6reDw3Dsr1kqS6zt9y4dTjHk=; b=gFqs7hzT/xlaYWOwwHtWS6kx7lHxphzTZPyz1Z4aSPJR2W87ueusy/3Z3avIaXLHX8Ubz9SXQyjVwQC+sai4LyGDVN+N6ypnAFWH3zZ2a/uKZUIQYqucm74Zr9o19RR+bpXSfZ8CNn+z247KOVTaSfYo9ETxIbgsjw8tVri7gfU+Kp4GUplC6vmJfw0GJz9sMwVmoeD1dh2vmpLk0/5uALsSqZSESA/lUQMVa3zQtWc9ZDtpTdNQgXNmOhj4ePSltOkCJkeXNezEUOwDyZV8nZn4acfJgA3WwsQZ4b53R2gTC+rfuYwUxfwK/8RzV9r1pZCW3hsJXG8hAEsUAUPY/A== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BVfmNMJWBkV9NHgT8wY6reDw3Dsr1kqS6zt9y4dTjHk=; b=f8EBZma0fKB33H+njFFkhbS0jDAIr09IZqT2BUSQsj47e3rp33GE3ec5dsGb0l/gMHMjs3Ruuq30sF/osnjU+gCLNLhkQXpRYP82YEUtTZyYE2yAhzy1ObVIfLHIN09G4tzmbgzld/zoC9TUb/2VnwazioQi5vGHPPSj5unLz9o= Received: from AS4P251CA0004.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:5d2::6) by GV1PR08MB10690.eurprd08.prod.outlook.com (2603:10a6:150:161::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:57:04 +0000 Received: from AMS1EPF0000004A.eurprd04.prod.outlook.com (2603:10a6:20b:5d2:cafe::d0) by AS4P251CA0004.outlook.office365.com (2603:10a6:20b:5d2::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AMS1EPF0000004A.mail.protection.outlook.com (10.167.16.134) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:57:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tAtGtgOs6LjTVZjTCMPFx6cRpsnZXthRVW3LT7F6HGuEiLpbQ8UbleVN3YqDgnbJCkKXWcLdhhbeKYuffgrnpJRNS/xG032wdhyEltDqzU226hF712hFnaSdVEbacM6xRBJ9Isj/mnVBLVvV+42RcKWMdFm7W3TJwLjBogSgcyWAEOQ1RsHPzcU03Vrlx7ba1DXrzxX3RfR4L2Qd8dSZ2DSag6toY1KsERR7eKqi/zFOW1VMMPGJLNB7Wc9P8G1ITig77geVi4mWWSsf+7b8J/s3x42SeBjPmj1Zg52JWaQVSX9MDXS5tc5F23wNyquff5oBsoCr3CAHU+DCIH2dUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BVfmNMJWBkV9NHgT8wY6reDw3Dsr1kqS6zt9y4dTjHk=; b=B2htnBIL3aEWKlpNXG/NgPivEdW9jNpEGhJu9ccLP+UFl8PZpxk9AvOM/IfbFcEmENOCjq/EFqYFvsI2TJB/tWnOa85X7RpGFYFZb85jfoxbeJ/XgxmPw5PH/D+wvfrYYURBMjhTFzwT5905mt/ukjDruCG2T8nWir9pn5424rnPiYjxbhi2u0q/QGaP6DyXNPYNmBu1FPtJn4pP0hpmjecUWVBji1Lf1Ckg9TDjhPNyjGxc26sX+8F4bxLdHgVSUhU2iFPJY8+6cwpbi3JTGaDw/uigs/sgEe8EoDyjKyVulyLCjpPpZYB0kkW8jlguYvmB3rzGhSIoagXC2Of1rw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BVfmNMJWBkV9NHgT8wY6reDw3Dsr1kqS6zt9y4dTjHk=; b=f8EBZma0fKB33H+njFFkhbS0jDAIr09IZqT2BUSQsj47e3rp33GE3ec5dsGb0l/gMHMjs3Ruuq30sF/osnjU+gCLNLhkQXpRYP82YEUtTZyYE2yAhzy1ObVIfLHIN09G4tzmbgzld/zoC9TUb/2VnwazioQi5vGHPPSj5unLz9o= Received: from DUZP191CA0039.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:4f8::13) by DB4PR08MB8150.eurprd08.prod.outlook.com (2603:10a6:10:382::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:56:00 +0000 Received: from DB1PEPF00039234.eurprd03.prod.outlook.com (2603:10a6:10:4f8:cafe::52) by DUZP191CA0039.outlook.office365.com (2603:10a6:10:4f8::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF00039234.mail.protection.outlook.com (10.167.8.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:56:00 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX03.Arm.com (10.240.25.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:55:59 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:55:59 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 08/11] AArch64/SVE: Relax the expectations of the popcnt-sve test Date: Sat, 6 Dec 2025 16:55:15 +0000 Message-ID: <20251206165518.5449-9-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF00039234:EE_|DB4PR08MB8150:EE_|AMS1EPF0000004A:EE_|GV1PR08MB10690:EE_ X-MS-Office365-Filtering-Correlation-Id: 0ac66823-be4d-4cab-2111-08de34e87b5d x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|1800799024|82310400026|376014|36860700013|13003099007; X-Microsoft-Antispam-Message-Info-Original: 5j3nInPJeFxyHMNSLs7XdN1P7X3BHTWYO7wtTfBc34rl48VVLobJOMtxb0kwnRaLHXMcRDO97IDapwNlEahlKy75Ab+ziHsinrkX3gPp7jvamzZ72ToHojzxqgpZmambsQs7g3zn4YMlP9/vxP2n67MvhVBh2r0uHjQhVWXvihX1jd8GprBd5DmpgsFKo7mCMAIACSS6SAnGyjP0UlvuqiXk3djSxz2EYHlfRit0qKxjpGwMIYEI6oeHre5V1WUGECZD3thfnfskDGEPNSqLj4bS7vxp1m+Xj7N+NbhLEtwxaVXpQHlWD9CLmqRzujhLa+A3wRC6eE3dNiQmiLVoBDSilRVUNX7cCnvD9MPq/DwO0+HDtkPFWjDvL+FYa8ybuQPg59L6hPPoNLyFBpuNnWA5mtOdhg+/QiPLNn889GxrizS/LITIfiNkoFDEh18F2bfebVxl7bOjF4nbNnGhb3TIJEnf2nxGRNT+18x9W5dDGhmNEuSvLh6y0xXZVrMUL/y7qL684Jw44d3FtQxQYVSqbZJBv6BLvkwcxpV6mqcI6IOu/GdGRQgGw/YoD4VrwAu0f4/Wi8yRl9KXaei6POaKSWyiMSSJvI9OxbfyZ3b7ru5GRlpOlXPguALT+Ro8n8lCQzBFRzqXOfLMcw2VuupERrzAr7LY3XLvDNDRdzNxcaHddN/MskfRBmoO/peTQVwNCaV+ussGwLl3uqNaOcMrBJnJDKBrwCUXaWQx5PqTZu0SDsFElA1Qn0RbrXjqaCKDfha6nSANq57RwG8IcyQfLLaZ3L+Jkh+MozQyrg7n4qLFHp3n0ALcD9nOWE9ltZBWUFWhbkl5O5TEd0vPVEaAsVK/4+O6fT1WPSNILuYjKm6vWtwezoeasRHY4TAIDSR1qVO95Bcs5SJLDN1nQNl3FFq/oWYu74izMSb59QnFM3sxSblNGAe4uydQK4AA+X48di33OPOhYou+yrWbf4HOZJqZi2XEoTQNRCfskzkBQUEkHWh2Evm21KP8tDDa8dm0xjgxdPdQbb63vQfdtpZpXBoayEMngLwBNCjb6QzvJpEueDZHiE/8gXdr3eZaTI6tX/Bv+J6/YPoNLaVoop/AzoJgXUdpQDY2NpqrnsqeP72hb0Pvn2V2TAhrGPohQ5gzjby49paaD09KrWuYyb2Mn8R8tp/oZKJmLV0eb3tObW//vxpRYsIpz9UR6jd7ZlAm+ZkYVumD8Dybk2mCH/Pb9hINQHH/vrvaBf93kE0hMfrpRTEzvXmflQbjNfr4uULrIdEROdajz9NFMc0lyDo0J1fYw7Vj5IIab0puKF/7v/fy6NtkPUcittH4m/aaKYYRQN0bQldYYZCj9a+yLZ2dnRBpJydgNHA975+MQs6JU/KLQqFjJtNSM084WkIrBfWlgV2gPhRs9Mylo0Gu8PH5PJ1KPjQaWtyRk98+WAOUyb/8Ul5/XO5Y/ANvyESn6gUXdvAnMscJgeCk5qndeRAj45r4EK97P4aqdFAigOZZs6yk7MY4jj9QDk4ewBAfPIOSNcyxtW/knE82Q1ytBJO1AnvKLlT+V2ACzWEMnB4HaUpKmhmOhFT2OGYWHoBL X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(376014)(36860700013)(13003099007); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB8150 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS1EPF0000004A.eurprd04.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: e8120d5e-e11f-4d2a-a5c4-08de34e8555c X-Microsoft-Antispam: BCL:0; ARA:13230040|35042699022|376014|14060799003|82310400026|1800799024|36860700013|13003099007; X-Microsoft-Antispam-Message-Info: /oKkYPtUaSLyZGIfpAuHNdsIfcsAUeSmdXJYs47JWLdQ+h7IrbJHMffIoRs4KFsFLrWC4Dt8RX7B8LMqSZQROFqBA5T4KxbrEmVVZYCtz2WHOzpqgvldR4KSC9ERoW/85sbc+U/gxAXsOm0o5mjXB/q1Fp9og5KBvEcUH4WH/jvsg8gGVC8/Ha3TpBrbMcoQny7+Xkt1GIxIbh1c79Dj/YmFY/+umkNPGBTd0//eZdlGmmgzUwk5RGD59oHev/+AP7i1gpNln6B2itJ5a6GlYMQNO9bq2TU+R6+Wp4cTSLUFg34J0Zi0ziRDlM+KKd6adtWCoMqEREXqAujkA1BnMIynQCKhyR/dXDxfZi579J/7StwJY32gFW4GIeCHf/i4MgBHEAfzjGkNE0ttuqw5DUlyP/CQ/X2tzsrpudHt6S8GkwBJkKFD0Q1h55fgeR2GiOxrhD4YqWcxACc7h8uvSfRP6HF0uA6hZgy+cw38lpmo26MwOrrO1iVOAlA9G9wyJEADGEsQaK1l2C8dwNLlpQGd8WiL4Lf4MWUB72XWkz+zTVATwWzOTLKNt7u9PJYhc16lBFToY7WsXsvKgjmIexAf0dl5b9Na6VhUComu0cZGVWElQvejSpfKtrdSjDcqQhoTShPAAV26k2+cbuZuVZj2UCemlbtxfNSZk7aSgXNSDeJ3sxVPPYtStKTwfMy8ba7wcmIEdZdsgaEgBGcgvTLMpuWe1Uif5nTd3BbnWgWoHbUvXZf1GaA1uVq5m5nHoAF/tWYGm5RN3MBfpLKRv1x+9YT/zABs9HCDPQmvPiFiqPYcPFAk9ts6ksfZxY4JtqwEZT8rzyzRRl8NMuLILAlhiFfbHijszVO/QDmt683ckDGQhwm9Pr1NT2IgT0jgKKONJKrroVq9SDgmFxi65fIZxf5UVtf7dfVIDiDXIxvrXDG7rnjDBoBhEXUAbsrTJGJIfA3qs/H4w3HBN3Yonn436shuulapvgpWl09H+rTik1CXBXzAXDSiGD7IiVqijAPcAG1OPLToqv+wn0DHdvsOdyJBv9xSmvVvYXUzcNOInKV6xTazFDSMlbEneWyPADeU8QZD+pXXjTZ9sYzW8wqjmJVry6DGTkhiuDSP/48BAohuljqxKRU1SMHooOLP7JdBeLRpXJtx0GZVClMmIkCADDuJ9oW2KMjtn9CSKigq7G9zaLzi46vK4SoK+eJiySrm9DE3abV88+rAjh1EveBzcCvLuPwWB6GmzCe44BCwVvd4A0SfAlpcqyuoIkEq4sSKjwZMqR1XeXuuioDh1WkJi9ZW/FXQkuqFJEiCidGHKZqfxVnk02ARorW/NQ4S0CqxjcNBSJd+hvBo0FxMqeB+R1BDWUGIdqx4SuYnufZdvToe04CE7Bas/GybWJBUt/+Jj2dd019vUph/zMalKrxM/Gi7MnDV0F2lWTLGnEd3ZyKXDRYVXM65KimIWvCV40QQdOiXslzxvjFWgAnOXUR3l0LypGkUvVWgb/J5OrUZvW8o/TEw8QH+s+IH8cVghxgNkGrOa7qtpIQRFkvCFI7w0acaPAl6P9qR+nGAPY+6GeMWFBHWzKwVxHCIHlpN X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(35042699022)(376014)(14060799003)(82310400026)(1800799024)(36860700013)(13003099007); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:57:03.9023 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0ac66823-be4d-4cab-2111-08de34e87b5d X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF0000004A.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB10690 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org When predicated tails are enabled for basic block SLP vectorization, the assembly language generated by GCC when compiling popcnt-sve.c will change. Relax the regular expressions used by this test in preparation. Currently, analysis of f_v8hi succeeds with vector mode V16QI and the following GIMPLE is produced: vector(8) short unsigned intD.19 vect__1.18D.4648; ... vect__1.18_69 = MEM [(short unsigned intD.19 *)vectp.17_68 clique 1 base 1]; vect_patt_60.19_70 = .POPCOUNT (vect__1.18_69); With predicated tails, analysis instead succeeds with a variable-length vector mode and the following GIMPLE is produced: vector([8,8]) short unsigned intD.19 vect__1.18D.4649; ... slp_mask_45 = .WHILE_ULT (0, 8, { 0, ... }); # VUSE <.MEM_25(D)> vect__1.18_46 = .MASK_LOAD (vectp.17_44, 16B, slp_mask_45, { 0, ... }); vect_patt_36.19_47 = .POPCOUNT (vect__1.18_46); When lowered to RTL, the WHILE_ULT is replaced by reinterpretation of a V16QI as VNx8HI: (insn 7 4 8 2 ( set (reg:V16QI 107) (mem:V16QI (reg/v/f:DI 103 [ b ]) [1 S16 A16]) ) "gcc.target/aarch64/popcnt-sve.c":33:8 discrim 1 -1 (nil)) (insn 8 7 9 2 ( set (reg:VNx8HI 106) (subreg:VNx8HI (reg:V16QI 107) 0)) "gcc.target/aarch64/popcnt-sve.c":33:8 discrim 1 -1 (nil)) A mask is still required to lower POPCOUNT, so an all-ones mask is synthesized: (insn 9 8 10 2 (set (reg:VNx16BI 108) (const_vector:VNx16BI repeat [(const_int 1 [0x1]) ])) "gcc.target/aarch64/popcnt-sve.c":69:8 discrim 1 -1 (nil)) (insn 10 9 11 2 (set (reg:VNx4SI 105) (unspec:VNx4SI [ (subreg:VNx4BI (reg:VNx16BI 108) 0) (popcount:VNx4SI (reg:VNx4SI 106)) ] UNSPEC_PRED_X)) "gcc.target/aarch64/popcnt-sve.c":69:8 discrim 1 -1 (nil)) However, this mask is not the same as the specific-width mask currently expected by the tests. gcc/testsuite/ChangeLog: * gcc.target/aarch64/popcnt-sve.c: Update test expectations to allow both current and alternative valid mask specifications. --- gcc/testsuite/gcc.target/aarch64/popcnt-sve.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/gcc/testsuite/gcc.target/aarch64/popcnt-sve.c b/gcc/testsuite/gcc.target/aarch64/popcnt-sve.c index c3b4c69b4b4..117a5ca8f1b 100644 --- a/gcc/testsuite/gcc.target/aarch64/popcnt-sve.c +++ b/gcc/testsuite/gcc.target/aarch64/popcnt-sve.c @@ -4,7 +4,7 @@ /* ** f_v4hi: -** ptrue (p[0-7]).b, vl8 +** ptrue (p[0-7]).b, (?:vl8|all) ** ldr d([0-9]+), \[x0\] ** cnt z\2.h, \1/m, z\2.h ** str d\2, \[x1\] @@ -21,7 +21,7 @@ f_v4hi (unsigned short *__restrict b, unsigned short *__restrict d) /* ** f_v8hi: -** ptrue (p[0-7]).b, vl16 +** ptrue (p[0-7]).b, (?:vl16|all) ** ldr q([0-9]+), \[x0\] ** cnt z\2.h, \1/m, z\2.h ** str q\2, \[x1\] @@ -42,7 +42,7 @@ f_v8hi (unsigned short *__restrict b, unsigned short *__restrict d) /* ** f_v2si: -** ptrue (p[0-7]).b, vl8 +** ptrue (p[0-7]).b, (?:vl8|all) ** ldr d([0-9]+), \[x0\] ** cnt z\2.s, \1/m, z\2.s ** str d\2, \[x1\] @@ -57,7 +57,7 @@ f_v2si (unsigned int *__restrict b, unsigned int *__restrict d) /* ** f_v4si: -** ptrue (p[0-7]).b, vl16 +** ptrue (p[0-7]).b, (?:vl16|all) ** ldr q([0-9]+), \[x0\] ** cnt z\2.s, \1/m, z\2.s ** str q\2, \[x1\] @@ -74,7 +74,7 @@ f_v4si (unsigned int *__restrict b, unsigned int *__restrict d) /* ** f_v2di: -** ptrue (p[0-7]).b, vl16 +** ptrue (p[0-7]).b, (?:vl16|all) ** ldr q([0-9]+), \[x0\] ** cnt z\2.d, \1/m, z\2.d ** str q\2, \[x1\] From patchwork Sat Dec 6 16:55:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126050 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 501234152B40 for ; Sat, 6 Dec 2025 16:59:33 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 501234152B40 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=VCXh7t+v; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=VCXh7t+v X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from DUZPR83CU001.outbound.protection.outlook.com (mail-northeuropeazon11012056.outbound.protection.outlook.com [52.101.66.56]) by sourceware.org (Postfix) with ESMTPS id 2C0784209E17 for ; Sat, 6 Dec 2025 16:57:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2C0784209E17 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2C0784209E17 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.66.56 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040241; cv=pass; b=pbJ876+F0nmMBOO1Ho9hm8jyYRHMzba1ns2X/JF+qZw9Xhe4T7N+xalnLMVuTbqBdzFsLVdoJUtHvoyYCKyS3yWkQlLRKe4DZjYFDda4Wexg3DTA/W1cKuBVl3KKVkNcvbGuQMbtZ2qsKYmmkgB47Ua1sMW9AWTpuooiJ3ZBBCA= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040241; c=relaxed/simple; bh=7uwYvjx71kAd/w5LXUQv/8p3Odlv4u0RWO4Io2vppVU=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=J1+2U5agh5aqxRLWhJmMwYSHLBttt/9cCrzqcXJeQh94uaUnrTZIY5/DpfNSgCLMnIsYoG0SP4F4upQkh5ouaIT7LQIMyAq6ALmygRacDtn9IRW3980bIW5G9DLCfDMctb3iea6T5JqRszl0gBHffVQ5VVCiKTL/Pi5S4iiowi4= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2C0784209E17 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=Owe2g7OwqpT79xNJL1XmNw12HtGiYEiBVS3eNKorPaTtYeD6/oE/JCtqvi0fzbM+ArNzgM8ocT0TwbMVeSmM46Swh71qVPc96cvccNLTn2Or6RX2VRxNk+pCV/e/2Zt3PcL+XFor2GxqojQnVkk0vje2GV4UzLtMl7gpXJSJho5h4plMNK8Ffnq+P8h8BU33sNewlkWfykD5ipGIoISgrdkQnnW+fo7F4tvdc9dp12H1/fMHuK7AJpUemt7LMu+f0lMCmShA0QPd/Bbiseiy51emW3SQqKZFaj2F/8MKyS7sdziJeKClJV4sOgQqDNLWpZmMc2U8IfDuMeNYTWivXw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gLqq2mSsVmPhzI4RxqIDtHj8dxOD7W7wEuyR57iWEWw=; b=yRhfnJfsujvhlPtmOxzw5h8kL1JlC0bW3tKkL1Rnmke1AHoipmqS0L+5NqaZA7fax4hMK24gessiYEF+pZKxPB9r61xGHPQT8mAU3SOjZM8U6ayJ3vs0SU+IDeA2RcWthIuOklkgw2QZ/YIaP7T0+f4yIaE3Njh+Bt4bl3sbEefnRaVWBbUFP83vps+MyFY8ioSdHuCrZURywy1Gkr4irk6PGv+44u9SImY731jgGvMI8ShX6v60vOpPgOKVC+cMcIpWVI5h9kePS1X3vWDZcSS1rbTvYZfNYnr1drRKrRBtyZoJho5jQqc3XwGB1m+DTlCctwyIHaatDsVNyPOoPA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gLqq2mSsVmPhzI4RxqIDtHj8dxOD7W7wEuyR57iWEWw=; b=VCXh7t+vbQVYwyNCQEKnRfJWDulHkP3fDO+KaK5srFQbr9wK2AbSornFRRwnibED8QWyN2osepQEtJrPHPR88SZMCPSG0W75ja4KsFwqsDSk3QNTpqPeY58Wz1+HwD+riNZsdJ4snagQFlSpR4MkNOcVR3OufIRqp+LHTaHzN48= Received: from AS4P250CA0028.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:5e3::16) by GV1PR08MB10689.eurprd08.prod.outlook.com (2603:10a6:150:164::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:57:13 +0000 Received: from AM1PEPF000252DB.eurprd07.prod.outlook.com (2603:10a6:20b:5e3:cafe::7b) by AS4P250CA0028.outlook.office365.com (2603:10a6:20b:5e3::16) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:57:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AM1PEPF000252DB.mail.protection.outlook.com (10.167.16.53) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9412.4 via Frontend Transport; Sat, 6 Dec 2025 16:57:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HyyAhhVxNPT0wmyMQFjHXKM0QbiuykG+RjxMs9gofATG9TcJA8Vr/KKG3hOV5zQxzBEUqWiEoM7Uf36z5lL/KSKF5Jb6oTzaAs1NxGTNLpqcCnM82IB4VL+IfXRp1FDTDUTZgywrMMYISW6KdwEvdGZRU5gsJgfj+I87UEyQsfQ5B0pJHO7+EAijhbRPc3mbwx7GUb4xLpiSzImQ84tlSkXRF7wrrm92s5zUXpeLXo504w94tAqUQ3eP/Re0sSsWjKfilbfp3bEE82MTvQbV79RdErxm+A0bEXx5YDwCYE0x/YSX6vSM5khacG8TwKNkkUa/seTt8k/CyJQRuDaLJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gLqq2mSsVmPhzI4RxqIDtHj8dxOD7W7wEuyR57iWEWw=; b=DmKLEKAJnQm3pWQPwHw1WT0i38ypO0xA1bjxGjLqLSf8TQ7qnAk4S4AdJLre5Ifo/qs9mlsYtfhZn0jVHR7TOwou/wLiqagCtPazzwH+3EsB3lLTGCdkh534ukmDJxcD00oLFcMJ+IH+va92DtkM0aVt1wIxbnbMLEj41+PWdTPtIou9tCcbDpegOg7lrK2BE7QotSDdH5J1ySCbTPPkX/oD93I8r2GZR5JtNbzWCF8mG1wa72mdepLiIrNlhX4eKqG/e62d0VcqW+0xBkFbUvl4mH8eKSSEOHy1zeDQoqxvd78zInOyugDxsY8zmuy6UYtiYnvSJeTxJZXoYs/6wQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gLqq2mSsVmPhzI4RxqIDtHj8dxOD7W7wEuyR57iWEWw=; b=VCXh7t+vbQVYwyNCQEKnRfJWDulHkP3fDO+KaK5srFQbr9wK2AbSornFRRwnibED8QWyN2osepQEtJrPHPR88SZMCPSG0W75ja4KsFwqsDSk3QNTpqPeY58Wz1+HwD+riNZsdJ4snagQFlSpR4MkNOcVR3OufIRqp+LHTaHzN48= Received: from DB9PR01CA0019.eurprd01.prod.exchangelabs.com (2603:10a6:10:1d8::24) by VI0PR08MB11265.eurprd08.prod.outlook.com (2603:10a6:800:2f8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.11; Sat, 6 Dec 2025 16:56:05 +0000 Received: from DB1PEPF000509F5.eurprd02.prod.outlook.com (2603:10a6:10:1d8:cafe::f9) by DB9PR01CA0019.outlook.office365.com (2603:10a6:10:1d8::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF000509F5.mail.protection.outlook.com (10.167.242.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:56:05 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX04.Arm.com (10.240.25.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:56:05 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:56:05 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 09/11] Extend BB SLP vectorization to use predicated tails Date: Sat, 6 Dec 2025 16:55:16 +0000 Message-ID: <20251206165518.5449-10-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF000509F5:EE_|VI0PR08MB11265:EE_|AM1PEPF000252DB:EE_|GV1PR08MB10689:EE_ X-MS-Office365-Filtering-Correlation-Id: 52131554-3b91-4a16-8c32-08de34e8804a x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|82310400026|376014|36860700013|1800799024|13003099007; X-Microsoft-Antispam-Message-Info-Original: Fa8UJ3XDMU2VYeZvKkfZNU7Z0l77QpAE6cez2X7veRdsF0/c/ojUmCBrTHIp5RRLyVXCnyZbYaxA3G2nDP5za9rF59SeIwLvaYFWVmpc2Vnq7Y5RLm0X2DIZ/yFOcCRw+eMSm1XqCAfiXk0UlEYl4OWe9G4A5+KwV3aDSXbWENHV55CJt7Fg/hdc8/UvkcDsI5RUHi87oRzMiefM8mQCyYSf1syKwWb763PlbbDL4sr7OalwxaPm0f23XtITxEW5RR8DTJrHtAXjSCy7S5mtJbarscHyHAJHRBnktnqpU7SNVNRK/WNo7iXF49/E6MjiZJxDbf/0m15LWGx80zNc9maqwcFPLnxFyvYG3apYOxsUb9YFI6JAgoC4AXfmn6ZUsr2u3YSHQfIDPFfI6QMC3ETWK4K4eBV8fgbINqGR2LfnfaR3vM4mHpyFpzvevAUUnm3rsIwYg941YJ8WxEajBvvOyMFDjfcq7b2RADCwzBp/Rt0Tup44yLfkI0y0cYMBCFW58bC847P5OwiGyErvGJyADuAttTe06IrfJu+6uUa+DI3L2ZuqIukqcKo78k8IOQLRG+x3aWzKrypKfyJxX/AyQ2gPJY4iUIQ9Bjk0ZcfUwoOM8SA3igUYZVchlEPKmzoFHNMd1GZ6sM4j4HVHYUk7d1oorgKdt7sAgdvT1j5/7geepDTxVz6o3skiJoPRNy0GcWy+f1FoHYbpwNbb8oUIIiGzhHDUYwLxKFYDpHEvKxOoN73vkwNTCnoalumO+s+p3l1oyRd2sdYGWY4bNPVfzjDKf5oGfgcge3QLls9EZrIadr4MBkkkF/zTf9MIAKsQguqzBF2Sh6p9GvtYOQ8I5P2peydR21TUeJ0FhABTLdpOJdjn+ZagpqQMwof+t+70oux3RrVa5jXlrjTI1UNsDLdj21DIOvlw3tDHLxNUYx/yb6P7XtX3Lg1tcgKA7j10tGOMyxHqWQBl8aOYuUpYsZX7zabdtZU/eB2i5sKKKhW15ZVAZsX8xFAL4ckXV2OyL4tJUtspa3jBdqR3cclnWwZkYRWmIhzXsKwZMMnUvenLixqCWVWDeg+NkshEe8UHWL8uPquG1vEOqzA8k9Hc2APY1aoKizXa+YQ74nNlYkUCwF/+zCHVGOiUQB2gTmGYch8Q8p4BCzYYRYUzQRyuoo59Snaj8GmPrrVh6FJsFOQ+cKhxPgc/TQwY2KXHcaNyosz4M6j+M31nL7oTVDC8LpSLrMzB5SBaZ5KW/hqT4uvBBsXVfmVck1Vue2P+hsinFZi+Z+4bQHkVwnOfVDszuEthC58cDMzcVqEBsWKsHIDItbbKhhWKy0cDGx9CDcU//cNEDWQRGfTLWb3/Rl50yDlce+E2eFGeTwU87LWzY6BhpevyYPyUDx0MRVYxaMCaWsmGLbBM8lWziz+gPK1Hut0fGN2mpbrpDiPsUzTQvIeViXTxPd9ijaUExQIlnYoZjhu8Ut/f6nAGXQtNg3iSQojswad/aaxwlfyNddzqe5tFknDQLedJbllw4RRG X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024)(13003099007); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI0PR08MB11265 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM1PEPF000252DB.eurprd07.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: d2558a76-7d50-41a5-7a30-08de34e85874 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|14060799003|82310400026|35042699022|36860700013|376014|13003099007; X-Microsoft-Antispam-Message-Info: j+b3Dgyg8fPZzLgLgeYjySIQfp9oF9I9KXa8pJFPj2d+OTpgm2FKLKs5pYN3M2tCiOdulwFv3TAYZaA9YbSiCJrFK+q/wvm9OosnNaJSeRSMxM7ZF9WvWvG4HVWT9VGeAEDnVCgTSeYaXexu41TEOE4eBNjZ5mXDwQ53AXYW703YdP+WvJNzd4LXFjlnIbLg6dG4BiESEE61aFtnTmncHPUKk1oPcjw4cmtjfJCtM8W1YRevvSdjv3W9TIRLfabZ5co5+S2VXgtAlXpWOgIt7kiH2BLV7pkijoFC/7AhgFArCKnifML0/4CDxumZHqbGWke+kMXTn0/m3JTUBIKET3irfOXVm6yLVRUmfYpYzSG9CJBeJG5ntDXJ9nhJbu2bF4ekkGhG2QBjnviZpwidt4DUn6H4PHsrDmp2+iyVwZinE+TkyhzMT4taJpCKkHNvCh3Qa7exJnjhiY7hCIMC+UQB84ULtd4yMQ2lYYnKvrrTiu1m6e5ibT41KU++pUN3KtVHS8k7dAkWhErubDKPcjPUoFMr1AyzagemsboJuaXkDQJ/CXrZNiAKQyzKVEtHqQYujPOrTQtQxj/bJU8BDDijCqRgaDlF6geoPT+oaO6ZyUgETODPR3cXVZ0Se1RLq1gvDipUjFwxWxbta0xvhGmEIjV0mHiGAHm2bt/+rhCQ6eerxPvVkfnbwYVuAqDGUsD/i9QCG90SYFVdt3DAkyl1vB0Y7ZZ/3kXv2nOtzYkl3oV2SJG3+wLO753NlOM94qnc4UkeH54re8bQgR58jtv7AJtsbMmXIMJZv/fX56zGUZfRZk7W46Pa6iDvWNQxx9jOi3gRMSQMuHLxKR2cpvADm0T0mF2P7v6orIz8thHBPuUiwtbJO7efj8X9YTxsXPQ73hIVcwUqrX/2OzAvVXV5jps0EqDO8703fc/U8/P+fkht24lnPm9y4QAcw4xcV3ME6EyRNFgOjN5mI8mlBoVxYqX2ciP0hRBhoH4Cp4/zsS2FMZLlj1F2iZgh0MjV8FzBXBsBeLMtEWgphu3YnpFWyKSJmsF7jhuGrrfqfdKcdi8Pws7JqC9a7B4pfwEG4146rgdlv0oSKvx0EFJIzo2t71SWaaMv5vG/ruDZR76cHIIaydQhTOkHVZPInqdn1XHKkySPswezehUW5mU12iQnYm0mPRFPBBHkV5SiOgAbe/bkI0b+PSUI8E+1/neZ6m+3kTgmDYBxuUylKOQZPjVM248BB9lxci+1hnUdadbOKAu9CHlBLIkRtYIGr3D2myEFfrdpdXXGxR8pWpyzcOG40Me00Ip8T3GFAbVr1+GiSgs5oYqPjI4984094D9ZeIIVQ1eZMjy33ZOFEp/DNy5LN+Z8lW+A+Wb+scatvozCEpbnA8N/BSOZb+tlyd9rrOHz9VUMIXpbR4mRJrPnawNKGDdL/YJ6VfLRoYIR9EQPbbNItZQ3szBld8kZRWypH2NM/Z3X5LA7+MMEe/PaXsATZRbPRvrRUFLqc+KRqMde7EkNNnzPESRy2/WTaYsc X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(14060799003)(82310400026)(35042699022)(36860700013)(376014)(13003099007); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:57:12.2134 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 52131554-3b91-4a16-8c32-08de34e8804a X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AM1PEPF000252DB.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB10689 X-Spam-Status: No, score=-10.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org This enables use of a predicate mask or length limit for vectorization of basic blocks in cases where previously only the equivalent rolled (i.e. loop) form of some source code would have been vectorized. Predication is used for groups whose size is not neatly divisible into vectors of lengths that can be supported directly by the target. The vect_record_nunits function used during building of an SLP tree is updated to prevent it returning failure for BB SLP if the group size is not an integral multiple of the number of lanes in the vector type; it now allows such cases if the vector type might be more than long enough. Instead of giving up if vect_get_vector_types_for_stmt fails for the specified group size, vect_build_slp_tree_1 now calls vect_get_vector_types_for_stmt again without a group size (which defaults to 0) as a fallback. If this succeeds then the initial failure is treated as a 'soft' failure that results in the group being split. Consequently, assertions that "For BB vectorization, we should always have a group size once we've constructed the SLP tree" were deleted in get_vectype_for_scalar_type and vect_get_vector_types_for_stmt. For BB SLP, vect_analyze_slp_instance previously gave up after building an SLP tree if it could not prove that the group size was at least the maximum lane count across all of the vector types in the SLP tree (which is unprovable for scalable vector types), or attempted to split the group if it could prove that the group size was greater than this maximum but not exactly divisible by it (which is also unprovable for scalable vector types). This function will now provisionally create a new SLP instance if the group size definitely does not exceed the minimum number of lanes, even if the group size otherwise satisfies conditions that would require a loop to be unrolled (e.g., a group of size 3 that uses a mixture of V4SI and V8HI types). If the group size lies between the minimum and maximum number of lanes then vectorization is still abandoned (e.g., a group of size 3 that uses a mixture of V2DI and V4SI types). With BB SLP, there is no need for agreement between different SLP nodes about whether to use masks or lengths to support partial vectors. Instead, that decision is made early and per individual SLP node, by vect_analyze_stmt. If a partial vector is required (i.e. if the number of subparts in the vector type may be greater than the number of active lanes for the node) then vect_analyze_stmt now requires SLP_TREE_CAN_USE_PARTIAL_VECTORS_P to be true; otherwise it clears any SLP_TREE_PARTIAL_VECTORS_STYLE that could have been set. The vect_get_num_copies function used during statement analysis is updated to return early with 1 if a vector type is long enough for the specified SLP tree node. This avoids an ICE in vect_get_num_vectors, which cannot cope with SVE vector types. vect_create_vectorized_promotion_stmts no longer pushes more stmts than implied by vect_get_num_copies because it could previously overrun the number of slots allocated for an SLP node (based on its number of lanes and type). e.g., four defs were pushed for a promotion of V8HI to V2DI (8/2=4) even if only two lanes of the V8HI were active. Allowing it later caused ICE in vectorizable_operation for a parent node, because binary ops require both operands to be the same length. Since promotion no longer produces redundant definitions, vectorizable_conversion also had to be modified so that demotion no longer relies on an even number of defs being produced. If necessary, it now pushes a single constant zero def. The whole change is enabled by wiring the wrapper function vect_can_use_partial_vectors_p to SLP_TREE_CAN_USE_PARTIAL_VECTORS_P when invoked for BB SLP vectorization. Update test expectations for gcc.dg/vect/vect-over-widen-*.c and gcc.target/aarch64/sve/slp_6.c. gcc/ChangeLog: * tree-vect-slp.cc (vect_record_nunits): Allow group sizes that are indivisible by the vector length. (vect_build_slp_tree_1): In case of failure of vect_get_vector_types_for_stmt, try to get fallback vector types and continue analysis to allow splitting of groups. (vect_build_slp_tree_2): Don't call can_duplicate_and_interleave_p when doing basic block SLP vectorization. (vect_update_slp_min_nunits_for_node): New recursive function. Update min_nunits to reflect the minimum number of subparts for all of the vector types used by an SLP subgraph. (vect_slp_tree_min_nunits): New function. Initialize min_nunits then call vect_update_slp_min_nunits_for_node. (vect_analyze_slp_instance): For BB SLP vectorization, create a new SLP instance if the group size definitely does not exceed the minimum number of subparts for all of the vector types used in the SLP tree, even if the group size otherwise satisfies conditions that would require a loop to be unrolled. (vectorizable_slp_permutation_1): Instead of asserting that an SLP tree node's number of lanes is compatible with the chosen vector width, return a failure indication if incompatible. * tree-vect-stmts.cc (check_load_store_for_partial_vectors): When calculating the number of vectors, get the group size from SLP_TREE_LANES instead of a parameter (e.g., DR_GROUP_SIZE) if doing BB SLP vectorization. Don't assume it can be divided by the number of subparts in the vector type to get a compile-time constant. (vect_get_data_ptr_increment): Require a parameter of type loop_vec_info instead of vec_info *. (vect_create_vectorized_promotion_stmts): Require an SLP tree node to be passed by the caller, for use by vect_get_num_copies. Stop pushing more stmts than implied by vect_get_num_copies. (vectorizable_conversion): Pass SLP tree node to vect_create_vectorized_promotion_stmts. Demotion no longer relies on an even number of definitions being produced by promotion. If necessary, push a single constant zero definition. (vectorizable_load): Pass loop_vec_info instead of vec_info * when calling vect_get_data_ptr_increment. (vect_analyze_stmt): For BB SLP vectorization, check whether the group needs partial vectors. If it does then return a failure indication if SLP_TREE_CAN_USE_PARTIAL_VECTORS_P was cleared by a callee of this function; if it doesn't need partial vectors then clear any partial vectors style that might have been chosen by callees of this function. (get_vectype_for_scalar_type): For BB SLP vectorization, allow invocation of this function with a group size of zero even if one or more SLP instances have been created. If the number of subparts in the natural choice of vector type could be greater than the group size then pick a shorter vector type only if the target does not support partial vectors. (vect_maybe_update_slp_op_vectype): Reject external definitions that have a number of lanes not divisible by the number of subparts in a vector type naively inferred from the scalar type. (vect_get_vector_types_for_stmt): Add a new output parameter of Boolean type. Set it to true if the statement can't be vectorized because it uses a data type that the target doesn't support in vector form for a group of the given size, otherwise false. * tree-vectorizer.h (vect_get_num_copies): Return early with 1 if a vector type is long enough for the specified SLP tree node to avoid an ICE in vect_get_num_vectors. (vect_get_vector_types_for_stmt): Update function declaration. (vect_can_use_partial_vectors_p): Handle the BB SLP use-case by returning the result of SLP_TREE_CAN_USE_PARTIAL_VECTORS_P. gcc/testsuite/ChangeLog: * gcc.dg/vect/vect-over-widen-10.c: Update test expectations to avoid spurious matching of scan-tree-dump-not pattern. * gcc.dg/vect/vect-over-widen-13.c: As above. * gcc.dg/vect/vect-over-widen-14.c: As above. * gcc.dg/vect/vect-over-widen-17.c: As above. * gcc.dg/vect/vect-over-widen-18.c: As above. * gcc.dg/vect/vect-over-widen-5.c: As above. * gcc.dg/vect/vect-over-widen-6.c: As above. * gcc.dg/vect/vect-over-widen-7.c: As above. * gcc.dg/vect/vect-over-widen-8.c: As above. * gcc.dg/vect/vect-over-widen-9.c: As above. --- .../gcc.dg/vect/vect-over-widen-10.c | 2 +- .../gcc.dg/vect/vect-over-widen-13.c | 2 +- .../gcc.dg/vect/vect-over-widen-14.c | 2 +- .../gcc.dg/vect/vect-over-widen-17.c | 2 +- .../gcc.dg/vect/vect-over-widen-18.c | 2 +- gcc/testsuite/gcc.dg/vect/vect-over-widen-5.c | 2 +- gcc/testsuite/gcc.dg/vect/vect-over-widen-6.c | 2 +- gcc/testsuite/gcc.dg/vect/vect-over-widen-7.c | 2 +- gcc/testsuite/gcc.dg/vect/vect-over-widen-8.c | 2 +- gcc/testsuite/gcc.dg/vect/vect-over-widen-9.c | 2 +- gcc/testsuite/gcc.target/aarch64/sve/slp_6.c | 3 - gcc/tree-vect-slp.cc | 141 ++++++++++-- gcc/tree-vect-stmts.cc | 201 +++++++++++++----- gcc/tree-vectorizer.h | 13 +- 14 files changed, 281 insertions(+), 97 deletions(-) diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-10.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-10.c index f0140e4ef6d..6efcf739db9 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-10.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-10.c @@ -16,5 +16,5 @@ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(unsigned char\)} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-13.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-13.c index 08a65ea5518..720353716cf 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-13.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-13.c @@ -48,5 +48,5 @@ main (void) /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* / 2} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* = \(signed char\)} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-14.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-14.c index dfa09f5d2ca..f1d5f95c543 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-14.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-14.c @@ -15,5 +15,5 @@ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* = \(unsigned char\)} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-17.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-17.c index 53fcfd0c06c..ac1a0f86727 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-17.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-17.c @@ -46,5 +46,5 @@ main (void) adopts realign_load scheme. It requires rs6000_builtin_mask_for_load to generate mask whose return type is vector char. */ /* { dg-final { scan-tree-dump-not {vector[^\n]*char} "vect" { target vect_hw_misalign } } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-18.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-18.c index aa58cd1c957..3ebfaa78270 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-18.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-18.c @@ -47,5 +47,5 @@ main (void) /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* |} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* <<} "vect" } } */ /* { dg-final { scan-tree-dump {vector[^\n]*char} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-5.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-5.c index c2ab11a9d32..1d89789a86d 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-5.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-5.c @@ -49,5 +49,5 @@ main (void) /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(signed char\)} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-6.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-6.c index bda92c965e0..62d5a52587e 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-6.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-6.c @@ -13,5 +13,5 @@ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(unsigned char\)} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-7.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-7.c index 1d55e13fb1f..6e09631009a 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-7.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-7.c @@ -51,5 +51,5 @@ main (void) /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(signed char\)} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-8.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-8.c index 553c0712a79..b6d650beab4 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-8.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-8.c @@ -16,5 +16,5 @@ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(unsigned char\)} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-over-widen-9.c b/gcc/testsuite/gcc.dg/vect/vect-over-widen-9.c index 36bfc68e053..e82f8a571da 100644 --- a/gcc/testsuite/gcc.dg/vect/vect-over-widen-9.c +++ b/gcc/testsuite/gcc.dg/vect/vect-over-widen-9.c @@ -56,5 +56,5 @@ main (void) /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */ /* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(signed char\)} "vect" } } */ -/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */ +/* { dg-final { scan-tree-dump-not {vector[^ ]* int vect__} "vect" } } */ /* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_6.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_6.c index 44d128477d2..1c9ac15a699 100644 --- a/gcc/testsuite/gcc.target/aarch64/sve/slp_6.c +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_6.c @@ -37,9 +37,6 @@ vec_slp_##TYPE (TYPE *restrict a, TYPE *restrict b, int n) \ TEST_ALL (VEC_PERM) /* These loops can't use SLP. */ -/* { dg-final { scan-assembler-not {\tld1b\t} } } */ -/* { dg-final { scan-assembler-not {\tld1h\t} } } */ -/* { dg-final { scan-assembler-not {\tld1w\t} } } */ /* { dg-final { scan-assembler-not {\tld1d\t} } } */ /* { dg-final { scan-assembler {\tld3b\t} } } */ /* { dg-final { scan-assembler {\tld3h\t} } } */ diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index d76b18dd7d9..4cd687a324e 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -1073,8 +1073,12 @@ vect_record_max_nunits (vec_info *vinfo, stmt_vec_info stmt_info, } /* If populating the vector type requires unrolling then fail - before adjusting *max_nunits for basic-block vectorization. */ + before adjusting *max_nunits for basic-block vectorization. + Allow group sizes that are indivisible by the vector length only if they + are known not to exceed the vector length. We may be able to support such + cases by generating constant masks. */ if (is_a (vinfo) + && maybe_gt (group_size, TYPE_VECTOR_SUBPARTS (vectype)) && !multiple_p (group_size, TYPE_VECTOR_SUBPARTS (vectype))) { if (dump_enabled_p ()) @@ -1126,12 +1130,29 @@ vect_build_slp_tree_1 (vec_info *vinfo, unsigned char *swap, tree soft_fail_nunits_vectype = NULL_TREE; tree vectype, nunits_vectype; + bool unsupported_datatype = false; if (!vect_get_vector_types_for_stmt (vinfo, first_stmt_info, &vectype, - &nunits_vectype, group_size)) + &nunits_vectype, &unsupported_datatype, + group_size)) { - /* Fatal mismatch. */ - matches[0] = false; - return false; + /* Try to get fallback vector types and continue analysis, producing + matches[] as if vectype was not an issue. This allows splitting of + groups to happen. */ + if (unsupported_datatype + && vect_get_vector_types_for_stmt (vinfo, first_stmt_info, &vectype, + &nunits_vectype, + &unsupported_datatype)) + { + gcc_assert (is_a (vinfo)); + maybe_soft_fail = true; + soft_fail_nunits_vectype = nunits_vectype; + } + else + { + /* Fatal mismatch. */ + matches[0] = false; + return false; + } } if (is_a (vinfo) && known_le (TYPE_VECTOR_SUBPARTS (vectype), 1U)) @@ -1659,16 +1680,22 @@ vect_build_slp_tree_1 (vec_info *vinfo, unsigned char *swap, if (maybe_soft_fail) { - unsigned HOST_WIDE_INT const_nunits; - if (!TYPE_VECTOR_SUBPARTS - (soft_fail_nunits_vectype).is_constant (&const_nunits) - || const_nunits > group_size) + /* Use the known minimum number of subparts for VLA because we still need + to choose a splitting point although the choice is more arbitrary. */ + unsigned HOST_WIDE_INT const_nunits = constant_lower_bound ( + TYPE_VECTOR_SUBPARTS (soft_fail_nunits_vectype)); + + if (const_nunits > group_size) matches[0] = false; else { /* With constant vector elements simulate a mismatch at the point we need to split. */ + gcc_assert ((const_nunits & (const_nunits - 1)) == 0); unsigned tail = group_size & (const_nunits - 1); + if (tail == 0) + tail = const_nunits; + gcc_assert (group_size >= tail); memset (&matches[group_size - tail], 0, sizeof (bool) * tail); } return false; @@ -2399,13 +2426,21 @@ vect_build_slp_tree_2 (vec_info *vinfo, slp_tree node, /* Check whether we can build the invariant. If we can't we never will be able to. */ tree type = TREE_TYPE (chains[0][n].op); - if (!GET_MODE_SIZE (vinfo->vector_mode).is_constant () - && (TREE_CODE (type) == BOOLEAN_TYPE - || !can_duplicate_and_interleave_p (vinfo, group_size, - type))) + if (!GET_MODE_SIZE (vinfo->vector_mode).is_constant ()) { - matches[0] = false; - goto out; + if (TREE_CODE (type) == BOOLEAN_TYPE) + { + matches[0] = false; + goto out; + } + + if (!is_a (vinfo) + && !can_duplicate_and_interleave_p (vinfo, group_size, + type)) + { + matches[0] = false; + goto out; + } } } else if (dt != vect_internal_def) @@ -2834,7 +2869,7 @@ out: uniform_val = NULL_TREE; break; } - if (!uniform_val + if (!uniform_val && !is_a (vinfo) && !can_duplicate_and_interleave_p (vinfo, oprnd_info->ops.length (), TREE_TYPE (op0))) @@ -4910,6 +4945,53 @@ vect_analyze_slp_reductions (loop_vec_info loop_vinfo, return true; } +/* Update MIN_NUNITS to reflect the minimum number of subparts for all of the + vector types used by the SLP subgraph rooted at NODE. VISITED is used to + avoid reevaluating any node in the subgraph; it thereby prevents infinite + recursion should a cycle be encountered. The value of MIN_NUNITS will only be + updated if any node in the subgraph has a vector type with a number of + subparts that is smaller than the passed-in value of MIN_NUNITS. Before + calling this function for the first time, initialize MIN_NUNITS to + UINT64_MAX. */ + +static void +vect_update_slp_min_nunits_for_node (slp_tree node, poly_uint64 &min_nunits, + hash_set &visited) +{ + if (!node || SLP_TREE_DEF_TYPE (node) != vect_internal_def) + return; + + if (visited.add (node)) + return; + + for (slp_tree child : SLP_TREE_CHILDREN (node)) + vect_update_slp_min_nunits_for_node (child, min_nunits, visited); + + tree vectype = SLP_TREE_VECTYPE (node); + if (!vectype) + return; + + /* All unit counts have the form vec_info::vector_size * X for some + rational X, therefore we know the values are ordered. */ + poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); + min_nunits = known_eq (min_nunits, UINT64_MAX) + ? nunits + : ordered_min (min_nunits, nunits); +} + +/* For NODE, return the minimum number of subparts for all of the vector + types used in the given SLP graph. */ + +static poly_uint64 +vect_slp_tree_min_nunits (slp_tree node) +{ + poly_uint64 min_nunits = UINT64_MAX; + hash_set visited; + vect_update_slp_min_nunits_for_node (node, min_nunits, visited); + gcc_checking_assert (known_ne (min_nunits, UINT64_MAX)); + return min_nunits; +} + /* Analyze an SLP instance starting from a group of grouped stores. Call vect_build_slp_tree to build a tree of packed stmts if possible. Return FALSE if it's impossible to SLP any stmt in the group. */ @@ -4979,8 +5061,8 @@ vect_analyze_slp_instance (vec_info *vinfo, poly_uint64 unrolling_factor = calculate_unrolling_factor (max_nunits, group_size); - if (maybe_ne (unrolling_factor, 1U) - && is_a (vinfo)) + if (maybe_ne (unrolling_factor, 1U) && is_a (vinfo) + && !known_ge (vect_slp_tree_min_nunits (node), group_size)) { unsigned HOST_WIDE_INT const_max_nunits; if (!max_nunits.is_constant (&const_max_nunits) @@ -5065,9 +5147,10 @@ vect_analyze_slp_instance (vec_info *vinfo, = TREE_TYPE (DR_REF (STMT_VINFO_DATA_REF (stmt_info))); tree vectype = get_vectype_for_scalar_type (vinfo, scalar_type, 1 << floor_log2 (i)); - unsigned HOST_WIDE_INT const_nunits; - if (vectype - && TYPE_VECTOR_SUBPARTS (vectype).is_constant (&const_nunits)) + unsigned HOST_WIDE_INT const_nunits + = vectype ? constant_lower_bound (TYPE_VECTOR_SUBPARTS (vectype)) + : 0; + if (const_nunits > 1 && (i % const_nunits) == 0) { /* Split into two groups at the first vector boundary. */ gcc_assert ((const_nunits & (const_nunits - 1)) == 0); @@ -11652,7 +11735,21 @@ vectorizable_slp_permutation_1 (vec_info *vinfo, gimple_stmt_iterator *gsi, unpack_factor = 1; } unsigned olanes = unpack_factor * ncopies * SLP_TREE_LANES (node); - gcc_assert (repeating_p || multiple_p (olanes, nunits)); + + /* With fully-predicated BB-SLP, an external node's number of lanes can be + incompatible with the chosen vector width (e.g., lane packs of 3 with a + natural 2-lane vector type). */ + if (!repeating_p && !multiple_p (olanes, nunits)) + { + if (dump_p) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "unsupported permutation %p: vector type %T," + " nunits=" HOST_WIDE_INT_PRINT_UNSIGNED + " ncopies=%" PRIu64 ", lanes=%u and unpack=%u\n", + (void *) node, vectype, estimated_poly_value (nunits), + ncopies, SLP_TREE_LANES (node), unpack_factor); + return -1; + } /* Compute the { { SLP operand, vector index}, lane } permutation sequence from the { SLP operand, scalar lane } permutation as recorded in the diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 44759a360e4..5bd32e5f287 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -1672,23 +1672,27 @@ check_load_store_for_partial_vectors (vec_info *vinfo, tree vectype, unsigned int nvectors; if (can_div_away_from_zero_p (size, nunits, &nvectors)) return nvectors; - gcc_unreachable (); + + gcc_assert (known_le (size, nunits)); + return 1u; }; poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); - poly_uint64 vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); + poly_uint64 size = loop_vinfo + ? group_size * LOOP_VINFO_VECT_FACTOR (loop_vinfo) + : SLP_TREE_LANES (slp_node); unsigned factor; vect_partial_vector_style partial_vector_style = vect_get_partial_vector_style (vectype, is_load, &factor, elsvals); if (partial_vector_style == vect_partial_vectors_len) { - nvectors = group_memory_nvectors (group_size * vf, nunits); + nvectors = group_memory_nvectors (size, nunits); vect_record_len (vinfo, slp_node, nvectors, vectype, factor); } else if (partial_vector_style == vect_partial_vectors_while_ult) { - nvectors = group_memory_nvectors (group_size * vf, nunits); + nvectors = group_memory_nvectors (size, nunits); vect_record_mask (vinfo, slp_node, nvectors, vectype, scalar_mask); } else @@ -3351,12 +3355,11 @@ vect_get_strided_load_store_ops (stmt_vec_info stmt_info, slp_tree node, static tree vect_get_loop_variant_data_ptr_increment ( - vec_info *vinfo, tree aggr_type, gimple_stmt_iterator *gsi, + loop_vec_info loop_vinfo, tree aggr_type, gimple_stmt_iterator *gsi, vec_loop_lens *loop_lens, dr_vec_info *dr_info, vect_memory_access_type memory_access_type) { - loop_vec_info loop_vinfo = dyn_cast (vinfo); - tree step = vect_dr_behavior (vinfo, dr_info)->step; + tree step = vect_dr_behavior (loop_vinfo, dr_info)->step; /* gather/scatter never reach here. */ gcc_assert (!mat_gather_scatter_p (memory_access_type)); @@ -3400,7 +3403,7 @@ vect_get_data_ptr_increment (vec_info *vinfo, gimple_stmt_iterator *gsi, loop_vec_info loop_vinfo = dyn_cast (vinfo); if (loop_vinfo && LOOP_VINFO_USING_SELECT_VL_P (loop_vinfo)) - return vect_get_loop_variant_data_ptr_increment (vinfo, aggr_type, gsi, + return vect_get_loop_variant_data_ptr_increment (loop_vinfo, aggr_type, gsi, loop_lens, dr_info, memory_access_type); @@ -5215,7 +5218,7 @@ vect_create_vectorized_demotion_stmts (vec_info *vinfo, vec *vec_oprnds, call the function recursively. */ static void -vect_create_vectorized_promotion_stmts (vec_info *vinfo, +vect_create_vectorized_promotion_stmts (vec_info *vinfo, slp_tree slp_node, vec *vec_oprnds0, vec *vec_oprnds1, stmt_vec_info stmt_info, tree vec_dest, @@ -5228,37 +5231,39 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo, gimple *new_stmt1, *new_stmt2; vec vec_tmp = vNULL; - vec_tmp.create (vec_oprnds0->length () * 2); + const unsigned ncopies = vect_get_num_copies (vinfo, slp_node); + vec_tmp.create (ncopies); + gcc_assert (vec_oprnds0->length () <= ncopies); FOR_EACH_VEC_ELT (*vec_oprnds0, i, vop0) { + if (vec_tmp.length () >= ncopies) + break; + if (op_type == binary_op) vop1 = (*vec_oprnds1)[i]; else vop1 = NULL_TREE; /* Generate the two halves of promotion operation. */ - new_stmt1 = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1, - op_type, vec_dest, gsi, - stmt_info); - new_stmt2 = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1, - op_type, vec_dest, gsi, - stmt_info); - if (is_gimple_call (new_stmt1)) - { - new_tmp1 = gimple_call_lhs (new_stmt1); - new_tmp2 = gimple_call_lhs (new_stmt2); - } - else + new_stmt1 + = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1, op_type, + vec_dest, gsi, stmt_info); + new_tmp1 = is_gimple_call (new_stmt1) ? gimple_call_lhs (new_stmt1) + : gimple_assign_lhs (new_stmt1); + vec_tmp.quick_push (new_tmp1); + + if (vec_tmp.length () < ncopies) { - new_tmp1 = gimple_assign_lhs (new_stmt1); - new_tmp2 = gimple_assign_lhs (new_stmt2); + new_stmt2 + = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1, op_type, + vec_dest, gsi, stmt_info); + new_tmp2 = is_gimple_call (new_stmt2) ? gimple_call_lhs (new_stmt2) + : gimple_assign_lhs (new_stmt2); + vec_tmp.quick_push (new_tmp2); } - - /* Store the results for the next step. */ - vec_tmp.quick_push (new_tmp1); - vec_tmp.quick_push (new_tmp2); } + gcc_assert (vec_tmp.length () <= ncopies); vec_oprnds0->release (); *vec_oprnds0 = vec_tmp; } @@ -5470,6 +5475,7 @@ vectorizable_conversion (vec_info *vinfo, from the scalar type. */ if (!vectype_in) vectype_in = get_vectype_for_scalar_type (vinfo, rhs_type, slp_node); + if (!cost_vec) gcc_assert (vectype_in); if (!vectype_in) @@ -5860,12 +5866,15 @@ vectorizable_conversion (vec_info *vinfo, stmt_info, this_dest, gsi, c1, op_type); else - vect_create_vectorized_promotion_stmts (vinfo, &vec_oprnds0, - &vec_oprnds1, stmt_info, - this_dest, gsi, + vect_create_vectorized_promotion_stmts (vinfo, slp_node, + &vec_oprnds0, &vec_oprnds1, + stmt_info, this_dest, gsi, c1, c2, op_type); } + gcc_assert (vec_oprnds0.length () + == vect_get_num_copies (vinfo, slp_node)); + FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) { gimple *new_stmt; @@ -5889,6 +5898,16 @@ vectorizable_conversion (vec_info *vinfo, generate more than one vector stmt - i.e - we need to "unroll" the vector stmt by a factor VF/nunits. */ vect_get_vec_defs (vinfo, slp_node, op0, &vec_oprnds0); + + /* Promotion no longer produces redundant defs (since support was + added for length/mask-predicated BB SLP of awkward-sized groups), + therefore demotion now has to handle that case too. */ + if (vec_oprnds0.length () % 2 != 0) + { + tree vectype = TREE_TYPE (vec_oprnds0[0]); + vec_oprnds0.safe_push (build_zero_cst (vectype)); + } + /* Arguments are ready. Create the new vector stmts. */ if (cvt_type && modifier == NARROW_DST) FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) @@ -10672,7 +10691,7 @@ vectorizable_load (vec_info *vinfo, aggr_type = build_array_type_nelts (elem_type, group_size * nunits); if (!costing_p) - bump = vect_get_data_ptr_increment (vinfo, gsi, dr_info, aggr_type, + bump = vect_get_data_ptr_increment (loop_vinfo, gsi, dr_info, aggr_type, memory_access_type, loop_lens); unsigned int inside_cost = 0, prologue_cost = 0; @@ -13239,6 +13258,37 @@ vect_analyze_stmt (vec_info *vinfo, " live stmt not supported: %G", stmt_info->stmt); + if (bb_vinfo) + { + unsigned int group_size = SLP_TREE_LANES (node); + tree vectype = SLP_TREE_VECTYPE (node); + poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); + bool needs_partial = maybe_lt (group_size, nunits); + if (needs_partial) + { + /* If partial vectors are required then they must be supported by the + target; however, don't assume that a partial vectors style has + been set because a mask or length may not be required for the + statement. */ + if (!SLP_TREE_CAN_USE_PARTIAL_VECTORS_P (node)) + return opt_result::failure_at (stmt_info->stmt, + "not vectorized: SLP node needs but " + "cannot use partial vectors: %G", + stmt_info->stmt); + } + else + { + /* If we don't need partial vectors then we don't care about whether + they are supported or not; however, we need to clear any partial + vectors style that might have been chosen because it will be used + to control generation of lengths or masks. */ + SLP_TREE_PARTIAL_VECTORS_STYLE (node) = vect_partial_vectors_none; + } + + if (maybe_gt (group_size, nunits)) + gcc_assert (multiple_p (group_size, nunits)); + } + return opt_result::success (); } @@ -13541,13 +13591,7 @@ tree get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type, unsigned int group_size) { - /* For BB vectorization, we should always have a group size once we've - constructed the SLP tree; the only valid uses of zero GROUP_SIZEs - are tentative requests during things like early data reference - analysis and pattern recognition. */ - if (is_a (vinfo)) - gcc_assert (vinfo->slp_instances.is_empty () || group_size != 0); - else + if (!is_a (vinfo)) group_size = 0; tree vectype = get_related_vectype_for_scalar_type (vinfo->vector_mode, @@ -13561,10 +13605,18 @@ get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type, vinfo->used_vector_modes.add (TYPE_MODE (vectype)); /* If the natural choice of vector type doesn't satisfy GROUP_SIZE, - try again with an explicit number of elements. */ - if (vectype - && group_size - && maybe_ge (TYPE_VECTOR_SUBPARTS (vectype), group_size)) + try again with an explicit number of elements. A vector type satisfies + GROUP_SIZE if it is definitely not too long to store the whole group, + or we are able to generate masks to handle the unknown number of excess + lanes that might exist. Otherwise, we must substitute a vector type that + can be used to carve up the group. + */ + if (vectype && group_size + && maybe_gt (TYPE_VECTOR_SUBPARTS (vectype), group_size) + && (vect_get_partial_vector_style (vectype, true) + == vect_partial_vectors_none + || vect_get_partial_vector_style (vectype, false) + == vect_partial_vectors_none)) { /* Start with the biggest number of units that fits within GROUP_SIZE and halve it until we find a valid vector type. @@ -13880,7 +13932,36 @@ vect_maybe_update_slp_op_vectype (vec_info *vinfo, slp_tree op, tree vectype) && SLP_TREE_DEF_TYPE (op) == vect_external_def && SLP_TREE_LANES (op) > 1) return false; - (void) vinfo; /* FORNOW */ + + /* When the vectorizer falls back to building vector operands from scalars, + it can create SLP trees with external defs that have a number of lanes not + divisible by the number of subparts in a vector type naively inferred from + the scalar type. Reject such types to avoid ICE when later computing the + prologue cost for invariant operands. */ + if (SLP_TREE_DEF_TYPE (op) == vect_external_def) + { + poly_uint64 vf = 1; + + if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) + vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); + + vf *= SLP_TREE_LANES (op); + + if (maybe_lt (TYPE_VECTOR_SUBPARTS (vectype), vf) + && !multiple_p (vf, TYPE_VECTOR_SUBPARTS (vectype))) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "lanes=" HOST_WIDE_INT_PRINT_UNSIGNED + " is not divisible by " + "subparts=" HOST_WIDE_INT_PRINT_UNSIGNED ".\n", + estimated_poly_value (vf), + estimated_poly_value ( + TYPE_VECTOR_SUBPARTS (vectype))); + return false; + } + } + SLP_TREE_VECTYPE (op) = vectype; return true; } @@ -14602,27 +14683,32 @@ vect_gen_while_not (gimple_seq *seq, tree mask_type, tree start_index, - Set *NUNITS_VECTYPE_OUT to the vector type that contains the maximum number of units needed to vectorize STMT_INFO, or NULL_TREE if the - statement does not help to determine the overall number of units. */ + statement does not help to determine the overall number of units. + + - Set *UNSUPPORTED_DATATYPE to false. + + On failure: + + - Set *UNSUPPORTED_DATATYPE to true if the statement can't be vectorized + because it uses a data type that the target doesn't support in vector form + for a group of the given GROUP_SIZE. + */ opt_result vect_get_vector_types_for_stmt (vec_info *vinfo, stmt_vec_info stmt_info, tree *stmt_vectype_out, tree *nunits_vectype_out, + bool *unsupported_datatype, unsigned int group_size) { gimple *stmt = stmt_info->stmt; - /* For BB vectorization, we should always have a group size once we've - constructed the SLP tree; the only valid uses of zero GROUP_SIZEs - are tentative requests during things like early data reference - analysis and pattern recognition. */ - if (is_a (vinfo)) - gcc_assert (vinfo->slp_instances.is_empty () || group_size != 0); - else + if (!is_a (vinfo)) group_size = 0; *stmt_vectype_out = NULL_TREE; *nunits_vectype_out = NULL_TREE; + *unsupported_datatype = false; if (gimple_get_lhs (stmt) == NULL_TREE /* Allow vector conditionals through here. */ @@ -14695,10 +14781,13 @@ vect_get_vector_types_for_stmt (vec_info *vinfo, stmt_vec_info stmt_info, } vectype = get_vectype_for_scalar_type (vinfo, scalar_type, group_size); if (!vectype) - return opt_result::failure_at (stmt, - "not vectorized:" - " unsupported data-type %T\n", - scalar_type); + { + *unsupported_datatype = true; + return opt_result::failure_at (stmt, + "not vectorized:" + " unsupported data-type %T\n", + scalar_type); + } if (dump_enabled_p ()) dump_printf_loc (MSG_NOTE, vect_location, "vectype: %T\n", vectype); diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index b66aca240f4..f58ec17b2bd 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -2320,6 +2320,8 @@ vect_get_num_copies (vec_info *vinfo, slp_tree node) vf *= SLP_TREE_LANES (node); tree vectype = SLP_TREE_VECTYPE (node); + if (known_ge (TYPE_VECTOR_SUBPARTS (vectype), vf)) + return 1; return vect_get_num_vectors (vf, vectype); } @@ -2587,9 +2589,9 @@ extern tree vect_gen_while (gimple_seq *, tree, tree, tree, const char * = nullptr); extern void vect_gen_while_ssa_name (gimple_seq *, tree, tree, tree, tree); extern tree vect_gen_while_not (gimple_seq *, tree, tree, tree); -extern opt_result vect_get_vector_types_for_stmt (vec_info *, - stmt_vec_info, tree *, - tree *, unsigned int = 0); +extern opt_result vect_get_vector_types_for_stmt (vec_info *, stmt_vec_info, + tree *, tree *, + bool *, unsigned int = 0); extern opt_tree vect_get_mask_type_for_stmt (stmt_vec_info, unsigned int = 0); /* In tree-if-conv.cc. */ @@ -2923,9 +2925,8 @@ vect_can_use_partial_vectors_p (vec_info *vinfo, slp_tree slp_node) loop_vec_info loop_vinfo = dyn_cast (vinfo); if (loop_vinfo) return LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo); - - (void) slp_node; /* FORNOW */ - return false; + else + return SLP_TREE_CAN_USE_PARTIAL_VECTORS_P (slp_node); } /* If VINFO is vectorizer state for loop vectorization then record that we no From patchwork Sat Dec 6 16:55:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126049 X-Patchwork-Delegate: tamar.christina@arm.com Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6CE0C41A9E86 for ; Sat, 6 Dec 2025 16:59:27 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6CE0C41A9E86 Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=oxkD2e3C; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=oxkD2e3C X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from GVXPR05CU001.outbound.protection.outlook.com (mail-swedencentralazon11013006.outbound.protection.outlook.com [52.101.83.6]) by sourceware.org (Postfix) with ESMTPS id 8DE064335880 for ; Sat, 6 Dec 2025 16:57:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8DE064335880 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 8DE064335880 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.83.6 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040247; cv=pass; b=Q1tQJ3ulkVHOrs4eAW33prhPSTKz1ESS2axhVy51WGopO9hRCEWA+49jZYtYAn4/+RQ60QI0i81/ohECI12SNoiXnuTR7dE9OEBb//k73ACHhIgvTACCXowm0r3msBKF48Yz4yynap6mvsFw5M1dyoxRQxvN6F+CotYfgny17t0= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040247; c=relaxed/simple; bh=OvUM6ARVdgazYNYdx31Ox7SovPNkBMv9LvKgdThu0zs=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=sgs7ur0/DCcCP20eK83hQyvyjbdZCfLuVq5FoKSPEIziUiWBCFsqY8OXtwaAzMAYOW1A68rGdAImlNVEP5XV/eCJlbC7bGNCC7P7VpsTQyP+3UvLDJOwO4lYk0T0aYPNrCbtCiJn2f9Dt3K6yegL9HL61fir7vqTEhToPZCyZ/8= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 8DE064335880 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=HWLilptlW9h2xNp5ns5KgLuSpJsuAL9eyfIay6QG4fOpwYJfeS9Wi6zHvg6UbnXqKW1XBU6VGbNLm/p/KdI751vNdfGq8f8Sl3vQZVydgW02+stLi/yHaqyMtHklavQmvwWaKguMOML3ro1U5zfM0TFxCj12BUTA9VdeT0Wb3ZCp4lfhW1Ivm1Bz2fu0PAjbTzZbgxcVBSy1phK5kMd/nNMv5B+MD8Qluxs2OWru+QqMsryg3xHRcFq5mdHXL/ZTI+oOX1cEPBiQWkhsKSPGW3QlSVxRdR+5qvbofwrPM0aSvok7b3xdD7oB/ts1qqJQBXnSwh9XfMf53bntsE8c1Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IIKuhPWpGhJ2WA0MVBaw2ukzU9oSm/bCXGfFOJl1eHo=; b=v/U3VSI1lT3ESoM2fKtZNq3ofshU9sLaEuMYYB3TDxdxlTha5oyN7iaqNdYssDtYgoDhZlHj63f1/2JfQA8TPuCsumhvgVlUyYdu19b1JCxuLJuFsJ6Yo3IT5m7ls6WoWYwcqxFDFqeg9T8j0T/8aO7mcXTIfzN2Am+2A+eK+DD/NsWtweXf0MoQje1N7Rdfe4gCSCoPMT0YvZ9yOQ8WQWg0DqxH3J9StQ2LtpyghzlsnO9kuk4+0dcE4yC1sdkFqrOp39RTXP/DnxcIafpDapa6mKVRZs9qT/kQ6v8S1WPlfLfRjn6Ba++84hxhmGQpNbSdqrvLZDf1FinkRzreqw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IIKuhPWpGhJ2WA0MVBaw2ukzU9oSm/bCXGfFOJl1eHo=; b=oxkD2e3CLVvX9RdQls07t2u+vLiyE6dqw4JXS9Io1dpucbvNIlr/eYH/Iak3B8fRvaMUj0zchZ3W5SEvhmScpTvvTAKrhxPV505RBDB9OzTu0g+EL3UPhDvdrjtM1AawB4mT8xeKT1Zm0PbyecPcdImqLaymq1y36AZb+ujBjMQ= Received: from AS9PR06CA0749.eurprd06.prod.outlook.com (2603:10a6:20b:487::35) by GV2PR08MB10353.eurprd08.prod.outlook.com (2603:10a6:150:db::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.12; Sat, 6 Dec 2025 16:57:19 +0000 Received: from AMS0EPF000001A7.eurprd05.prod.outlook.com (2603:10a6:20b:487:cafe::f4) by AS9PR06CA0749.outlook.office365.com (2603:10a6:20b:487::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:57:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AMS0EPF000001A7.mail.protection.outlook.com (10.167.16.234) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:57:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q9PEOCIrLXuDgJyB4PcHb0qw9WrJ68povCDOaocxGc6qUgDHwrQcQ0A1PFQGBgKrCQUmneH5aPX9uNBU4RIP0XSnT9cG2rRy19wV7L1vVIWrAiBLZKg6JpLuUxRIxntugjGWLi7PnleMkGxPOxjwKurp/gzBEXtWMdwbdjTziBc1AayhQ7h9vBdTfisQXFCaEXWzh/vLUQ/yoCnZEyhOTNzOHzb4UdwyJCjjRpOxaKyXTZduOxznwfRYnF5yXhTcBe/LcYwnf8begv87YaIHZipQHZty+pbR8mHAbH0+IXv7FngypjyIclrJoDVzFTdt5nrGPkCMSkQwjBzqOg/tZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IIKuhPWpGhJ2WA0MVBaw2ukzU9oSm/bCXGfFOJl1eHo=; b=eqTax2NLr+XOu2bCfGagbKp44taBArWJQXL/g3nwPLjqrV3LPwcFzj/9Pw7TC2Ii0W0wOYd5fakwjwie+IJOZYxGtQ8z0ilj+EN62MSBC01rbE7JTvqUGYLnAiECFqs7L755HahXkzPHw9abhjwOi5qmDn6y7Hghb2jmSXppORuuM759UdJ6wj4EwmVKhLOerzk3AQKzp2aDnYzDv9PyT/f0XCYICgfBGnAh/0vjLYKokqx8CcBaNF4lFv1Vjv/qArh6ySO9km92EGvr0T9Mv/4Xz5mu+NSx9CYnv3WkgXa9J1QOBSATAbbbNXoDjN9/Ct826fRoKFSzEk/pkt/ibw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IIKuhPWpGhJ2WA0MVBaw2ukzU9oSm/bCXGfFOJl1eHo=; b=oxkD2e3CLVvX9RdQls07t2u+vLiyE6dqw4JXS9Io1dpucbvNIlr/eYH/Iak3B8fRvaMUj0zchZ3W5SEvhmScpTvvTAKrhxPV505RBDB9OzTu0g+EL3UPhDvdrjtM1AawB4mT8xeKT1Zm0PbyecPcdImqLaymq1y36AZb+ujBjMQ= Received: from DU6P191CA0024.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:540::14) by VE1PR08MB5693.eurprd08.prod.outlook.com (2603:10a6:800:1a5::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.11; Sat, 6 Dec 2025 16:56:11 +0000 Received: from DB1PEPF000509F9.eurprd02.prod.outlook.com (2603:10a6:10:540:cafe::3c) by DU6P191CA0024.outlook.office365.com (2603:10a6:10:540::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF000509F9.mail.protection.outlook.com (10.167.242.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:56:10 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX04.Arm.com (10.240.25.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:56:10 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:56:10 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 10/11] AArch64/SVE: Tests for use of predicated vector tails for BB SLP Date: Sat, 6 Dec 2025 16:55:17 +0000 Message-ID: <20251206165518.5449-11-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF000509F9:EE_|VE1PR08MB5693:EE_|AMS0EPF000001A7:EE_|GV2PR08MB10353:EE_ X-MS-Office365-Filtering-Correlation-Id: 6edbd5ec-1c75-43b7-8151-08de34e883bc x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|82310400026|376014|36860700013|1800799024|13003099007; X-Microsoft-Antispam-Message-Info-Original: 46aYHZIUwx9hgEMAsB4Qeo07aXCpQv9mRgmx0sVbLkFgEneJsQAlzi0h/HdnJAz2C1Zp/KSGCjZLYlloygVGl6NU9NjLETil/6q4ANHxLR2lKJ5CgTapbQH9/lhUamgQfxsMOzOAUMlq7ttGxUD3vhbaPqs/tMw9c4mALJlQQS19t5Z1pOOVtH3hG0Z9s4dUzZKrGcybvfVL951NLprp4JEvBsnoLvBZ07XH6gTTxNBglEkDbuVBTT4SpWPs1PEHSvYczsPFQz4ApNK8wgJdWpq2ZM/OdOUwa529kFnJY2Y9BUE2Q0akcowaIYObTqdoAAXPXK9s0C3W1lUTB/RBqesDQO5f1xWEuXI5TlqJKRQ+k87OHi37m3V3NfH1fXuH2h4IQoLa7+rZkfywKvg1N876l96FtwAJSau5hn4xuQQHjD4N7JvOQzJ9D03+KS9YFpnXNcPH7gW2GnTAHsNMjcXg3U16KKzglhqJymqbfBoy1Qc9KSCP++Q1Ka/J3D5kX7gH6WjUEOjREV5juhyPrcK8iJgzC0/SnCTbWEgdRfhrKIoq7h1LGRWzGgEVzgdXSiX8IPd65fjWyH94t6XK+SS6Df7TjxEP2VFak4CbyLyacc423tCJ/p2Deavn8BxyvDzISjtnzb5YcramvA9K82gzHPMe7azCEriqzu3l8MVy7/ZLMyZQCh+ZyOOCjTx7uSGSNuSkSR34Tyw7MLbb238puXGKFZF165XLu4hD7PNL6VOIFUu8RYKuxyera1Udo6Lmk5XHdcBTwvz8hFsC7OC36efKgk0CJeiFZ09RxhNvy0xi/8QASs8Av8FSIOPJih+MGU5wL6+KxC8rBxXnM2M7ninYxolszWOXKbdhNU6njojM7MjTJMP9L8tvkacsd1SeAKNNlibg+DnG6xlgvJ9Ah+aQpgM5qP1U2I/AJ0hhmto4Rs+E6eG7UGytIvxnR0gKcdELSXGisVcKpQk+ug4shZZ3fpxAMGn0NF52Bo2G4LekK3t3SALP8hw8Z22qfIWe9PZoaZ7wb/tOmCBGLoZJPUVMfk1ZuJ3y9wWlnS9rbuOgJ3aGZrX/9YcRqoTwpcbGiXGnbn/DOJJxYB4etbuhifTOKfWyimRg190hCbehEWLjbjobbMPc7K84aLf4vsH3b57HGKQ6W9RXYiwTQoW/7EHTF1ZlIqoQAnSIazlXDK9yGoSy6ZL9BhY8v2hqp6xEvGA4Xi/rwHq2Z+zivW7RArKN3kc/7mXXsB6N6gsS36oVZ7a8BEePPijmxMoYwYWoN65jgJILR/slH8dw0PptAjgAGyejymlfh3DDJhyCCUrJNS8UJicIcGxfDL1BjxvtZFoPeXw/uhyikdCVdFQj03yZrSplCwWbucJCGqGZQmu2J7oid5MuFHDVNQ9BjkFeT+Sb47rntXwHUm34hgNzuYk8QILjZAN4wkeNWsE2dXx/6o4P9uplrEtr7bop8tengnT78AR6wz6qURsJ/ZJEFEzEweYm/uBryXOJPb7gcJNZaCNSx6tuLC9n+br9 X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024)(13003099007); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5693 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS0EPF000001A7.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 73077304-5639-4622-4edb-08de34e85b83 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|82310400026|376014|35042699022|1800799024|14060799003|13003099007; X-Microsoft-Antispam-Message-Info: QAHjfsxL4moRITMF2VQiBAyX1UuYUTMNokPrQn6X9za1NaZDREFO3b8HiiXoTSNWo7V22snV2/s9Kk8aIpMgLZrsO8IU9E8JNO+BLlaN1mIhFFLBC/j/9MWIFkR27RyOzBVuJrmG0ygbIK6PBjCsWgsl8S2sEARIKhmgu4xiOxNMYodET9A70PAaKm3YqOIo9JNyfoyFmtlZukoLoWA1EZR+6dVNDcl9r3bONaxTpi1zJG32vVrGVH+ljlpzM3XxCeAHNZy7DwV5oNeCNDMoi6pYjbg2J/FjFitcuvr3n9KpxI7yRib3/8cU5DLMrS7Z6hEiyC7I7BAF5uadCchzr2xwDJDGIFi3cwKew+llaccBNcHUK+ecvVA7PHvmwXWNVLQJbFX+NpdmDFsF7OF0TDsGNN2pqMzoALK57rE9P9EcRk5b+BmniBKplCHN2iv7qhdeJvXFWQhI481FM6ejsSm1fIUC5cL+qN+Vh1EHuKdJ1K49LJTVU2BPlxJvJKpP023uWOGGN6vJuTruT/XTOrmECNAldcRSDvwNpOoj/OCdcnkf8vPZTC+avgP5qKZNhKP0cIpW1m73X/BRQmjg2vWEtLlDhsmn1hWEefIc3zYpGWDcTPlCTy8OsD9zx0i+PUacN4Y+nq12bBCz9RoRHcacVIGy1y8B9B2EzkOGAunGVyj73mdspa6a03n1rj7MhFWgcVFFvrRlMLrBdvbj6wzFHNKwDRsh1gVrrRshESmxKDOyv0oq43X5u/lxzDlj4NSSIDj68Z9W+fKL/teCCxOg+P1Fl64WrSgMq7iwrGfc5CtTIC7hdhrruWrLW/YNBEACflS0UBYyVHQ9ManneXVcZ8Rop0Zqq2YKuKeGnwcIaJzijbDAQPdfBTEEHpJUuf8LUVyI7i1a3+x8yJtuFWIajuwnFzdvSRob0Lbsl+7w9Joz0Mvr+9DKQP6welHGFsCRFr+7k7AMzdGRrIJb88QHwwS/7WH+M5xnMhpaLzyGJtMzF3t2D3Gj2dEaNlQpcGplZ270weLiLeRgICZi4MR04N28kH6AF36G3jTDKbwGCFfvYrNaZte7dwCi6KhnSQH6lnU0Ey93rjmoTp9CJbuuqAPgMmpwlz5zj4oJ91ysQF9hKg71VY89wZgV7/HCutRkBog1BCzeUcr/OSeeCI+T3Kn92B8oQbgyVI/R69kHxfSFq9YeIYlinG7zAehjfksUfDOcdCeLXCofYGOnrPTZxJb0nuQXvAQo/4UM/DhoupAr8CFeICmNY0fbx+7Feb/KWg6h6DG3Aei1ZmEvk66BoyXbg9wA4crhnHmkbtDeW1A9J/ReYI3CjjbVVJTNFKyRGFaguv4jGg8u5i+SqvWwvVdBanCwQ89UDQ5s9Kz3PdR1CU4YNdyLPSkMZMAFFlEd3PkayPJHOO0HqYrpLXIG5xZfzU6be80tNsPdPgW579oRFJmPoDx0q1I4CcMg2qaidVrn/0JajNfRtVXCgYyoygduzEzNvNDKHh4sqyVnEFSsDZ8w45Mr3SO8+zbW X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(36860700013)(82310400026)(376014)(35042699022)(1800799024)(14060799003)(13003099007); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:57:17.9920 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6edbd5ec-1c75-43b7-8151-08de34e883bc X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001A7.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB10353 X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_NONE, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org New tests verify that GCC can generate predicated vector-length specific code for AArch64 if the specified vector length is shorter than, equal to, or longer than the number of elements to be processed (including if the specified length is sufficient but the minimum length would not be); other tests verify that GCC can generate predicated vector-length agnostic code for AArch64 if the minimum length (of 16 bytes) is shorter than, equal to, or longer than the number of elements to be processed. gcc/testsuite/ChangeLog: * gcc.target/aarch64/sve/slp_pred_1.c: New test. * gcc.target/aarch64/sve/slp_pred_1_run.c: New test. * gcc.target/aarch64/sve/slp_pred_2.c: New test. * gcc.target/aarch64/sve/slp_pred_3.c: New test. * gcc.target/aarch64/sve/slp_pred_3_run.c: New test. * gcc.target/aarch64/sve/slp_pred_4.c: New test. * gcc.target/aarch64/sve/slp_pred_5.c: New test. * gcc.target/aarch64/sve/slp_pred_6.c: New test. * gcc.target/aarch64/sve/slp_pred_6_run.c: New test. * gcc.target/aarch64/sve/slp_pred_7.c: New test. * gcc.target/aarch64/sve/slp_pred_harness.h: Test harness shared between tests for vectorization with SVE predication. --- .../gcc.target/aarch64/sve/slp_pred_1.c | 33 ++++++++++++++++ .../gcc.target/aarch64/sve/slp_pred_1_run.c | 6 +++ .../gcc.target/aarch64/sve/slp_pred_2.c | 33 ++++++++++++++++ .../gcc.target/aarch64/sve/slp_pred_3.c | 33 ++++++++++++++++ .../gcc.target/aarch64/sve/slp_pred_3_run.c | 6 +++ .../gcc.target/aarch64/sve/slp_pred_4.c | 33 ++++++++++++++++ .../gcc.target/aarch64/sve/slp_pred_5.c | 36 +++++++++++++++++ .../gcc.target/aarch64/sve/slp_pred_6.c | 39 +++++++++++++++++++ .../gcc.target/aarch64/sve/slp_pred_6_run.c | 6 +++ .../gcc.target/aarch64/sve/slp_pred_7.c | 38 ++++++++++++++++++ .../gcc.target/aarch64/sve/slp_pred_harness.h | 28 +++++++++++++ 11 files changed, 291 insertions(+) create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c new file mode 100644 index 00000000000..4e0a78de02a --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */ + +#include + +/* Test that we can vectorize with SVE predication when generating vector-length + agnostic code if the minimum possible vector length (of 16 bytes) is larger + than the number of elements to be processed. */ + +void +f (uint8_t *x) +{ + x[0] += 1; + x[1] += 2; + x[2] += 1; + x[3] += 2; + x[4] += 1; + x[5] += 2; + x[6] += 1; + x[7] += 2; + x[8] += 1; + x[9] += 2; + x[10] += 1; + x[11] += 2; + x[12] += 1; + x[13] += 2; + x[14] += 1; // one less than the minimum vector length +} + +/* { dg-final { scan-assembler-times {\twhilelo\tp[0-7].b, xzr, x[0-9]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+\.b, p[0-7]/z, \[x[0-9]\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b, z[0-9]+\.b\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+\.b, p[0-7], \[x[0-9]\]\n} 1 } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c new file mode 100644 index 00000000000..7d0a88fec2f --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c @@ -0,0 +1,6 @@ +/* { dg-do run { target aarch64_sve_hw } } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */ +#include "slp_pred_harness.h" +#include "slp_pred_1.c" + +HARNESS (15) diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c new file mode 100644 index 00000000000..da120ad36f9 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */ + +#include + +/* Test that we can vectorize with SVE predication when generating vector-length + specific code if the configured vector length is larger than the number of + elements to be processed. */ + +void +f (uint8_t *x) +{ + x[0] += 1; + x[1] += 2; + x[2] += 1; + x[3] += 2; + x[4] += 1; + x[5] += 2; + x[6] += 1; + x[7] += 2; + x[8] += 1; + x[9] += 2; + x[10] += 1; + x[11] += 2; + x[12] += 1; + x[13] += 2; + x[14] += 1; // one less than the configured vector length +} + +/* { dg-final { scan-assembler-times {\tptrue\tp[0-7].b, mul3\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+\.b, p[0-7]/z, \[x[0-9]\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b, z[0-9]+\.b\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+\.b, p[0-7], \[x[0-9]\]\n} 1 } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c new file mode 100644 index 00000000000..184b9615cd9 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */ + +#include + +/* Test that we can vectorize with SVE predication when generating vector-length + agnostic code if the minimum possible vector length (of 16 bytes) is equal to + the number of elements to be processed. */ + +void +f (uint8_t *x) +{ + x[0] += 1; + x[1] += 2; + x[2] += 1; + x[3] += 2; + x[4] += 1; + x[5] += 2; + x[6] += 1; + x[7] += 2; + x[8] += 1; + x[9] += 2; + x[10] += 1; + x[11] += 2; + x[12] += 1; + x[13] += 2; + x[14] += 1; + x[15] += 2; // exactly fits the minimum vector length +} + +/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b, z[0-9]+\.b\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]\]\n} 1 } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c new file mode 100644 index 00000000000..5c92b1e0b39 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c @@ -0,0 +1,6 @@ +/* { dg-do run { target aarch64_sve_hw } } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */ +#include "slp_pred_harness.h" +#include "slp_pred_3.c" + +HARNESS (16) diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c new file mode 100644 index 00000000000..ecb6ee2304a --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */ + +#include + +/* Test that we can vectorize with SVE predication when generating vector-length + specific code if the configured vector length is equal to the number of + elements to be processed. */ + +void +f (uint8_t *x) +{ + x[0] += 1; + x[1] += 2; + x[2] += 1; + x[3] += 2; + x[4] += 1; + x[5] += 2; + x[6] += 1; + x[7] += 2; + x[8] += 1; + x[9] += 2; + x[10] += 1; + x[11] += 2; + x[12] += 1; + x[13] += 2; + x[14] += 1; + x[15] += 2; // exactly fits the configured vector length +} + +/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b, z[0-9]+\.b\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]\]\n} 1 } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c new file mode 100644 index 00000000000..076756ff948 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c @@ -0,0 +1,36 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=256" } */ + +#include + +/* Test that we can vectorize with SVE predication when generating + vector-length specific code if the number of elements to be + processed is greater than the minimum possible vector length + (of 16 bytes) but less the configured vector length. */ + +void +f (uint8_t *x) +{ + x[0] += 1; + x[1] += 2; + x[2] += 1; + x[3] += 2; + x[4] += 1; + x[5] += 2; + x[6] += 1; + x[7] += 2; + x[8] += 1; + x[9] += 2; + x[10] += 1; + x[11] += 2; + x[12] += 1; + x[13] += 2; + x[14] += 1; + x[15] += 2; + x[16] += 1; // one more than the minimum vector length +} + +/* { dg-final { scan-assembler-times {\twhilelo\tp[0-7].b, xzr, x[0-9]+\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+\.b, p[0-7]/z, \[x[0-9]+\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b, z[0-9]+\.b\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+\.b, p[0-7], \[x[0-9]+\]\n} 1 } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c new file mode 100644 index 00000000000..fffb52e8f4b --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c @@ -0,0 +1,39 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */ + +#include + +/* Test that we do not attempt to vectorize with SVE predication when + generating vector-length agnostic code if the minimum possible + vector length (of 16 bytes) is smaller than the number of elements + to be processed. */ + +void +f (uint8_t *x) +{ + x[0] += 1; + x[1] += 2; + x[2] += 1; + x[3] += 2; + x[4] += 1; + x[5] += 2; + x[6] += 1; + x[7] += 2; + x[8] += 1; + x[9] += 2; + x[10] += 1; + x[11] += 2; + x[12] += 1; + x[13] += 2; + x[14] += 1; + x[15] += 2; + x[16] += 1; // one more than the minimum vector length +} + +/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b, z[0-9]+\.b\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]\]\n} 1 } } */ + +/* { dg-final { scan-assembler-times {\tldrb\tw[0-9]+, \[x[0-9], 16\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tw[0-9]+, w[0-9]+, 1\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tstrb\tw[0-9]+, \[x[0-9], 16\]\n} 1 } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c new file mode 100644 index 00000000000..2147a66abe9 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c @@ -0,0 +1,6 @@ +/* { dg-do run { target aarch64_sve_hw } } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */ +#include "slp_pred_harness.h" +#include "slp_pred_6.c" + +HARNESS (17) diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c new file mode 100644 index 00000000000..82f744c8bbc --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c @@ -0,0 +1,38 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -ftree-vectorize -march=armv9-a+sve --param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */ + +#include + +/* Test that we do not attempt to vectorize with SVE predication when + generating vector-length specific code if the configured vector + length is smaller than the number of elements to be processed. */ + +void +f (uint8_t *x) +{ + x[0] += 1; + x[1] += 2; + x[2] += 1; + x[3] += 2; + x[4] += 1; + x[5] += 2; + x[6] += 1; + x[7] += 2; + x[8] += 1; + x[9] += 2; + x[10] += 1; + x[11] += 2; + x[12] += 1; + x[13] += 2; + x[14] += 1; + x[15] += 2; + x[16] += 1; // one more than the configured vector length +} + +/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b, z[0-9]+\.b\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]\]\n} 1 } } */ + +/* { dg-final { scan-assembler-times {\tldrb\tw[0-9]+, \[x[0-9], 16\]\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tadd\tw[0-9]+, w[0-9]+, 1\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tstrb\tw[0-9]+, \[x[0-9], 16\]\n} 1 } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h new file mode 100644 index 00000000000..ac569fc670c --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h @@ -0,0 +1,28 @@ +/* Test harness shared between tests for vectorization with SVE predication. */ + +#define HARNESS(N) \ + int __attribute__ ((optimize (1))) main (void) \ + { \ + uint8_t a[N], b[N]; \ + for (unsigned int i = 0; i < N; ++i) \ + { \ + a[i] = i * 2 + i % 5; \ + b[i] = a[i]; \ + asm volatile ("" ::: "memory"); \ + } \ + f (a); \ + for (unsigned int i = 0; i < N; i += 2) \ + { \ + b[i]++; \ + if (a[i] != b[i]) \ + __builtin_abort (); \ + if (i + 1 < N) \ + { \ + b[i + 1] += 2; \ + if (a[i + 1] != b[i + 1]) \ + __builtin_abort (); \ + } \ + asm volatile ("" ::: "memory"); \ + } \ + return 0; \ + } From patchwork Sat Dec 6 16:55:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Bazley X-Patchwork-Id: 126056 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5535C414380E for ; Sat, 6 Dec 2025 17:06:31 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5535C414380E Authentication-Results: sourceware.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=B20RwXbM; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=B20RwXbM X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from GVXPR05CU001.outbound.protection.outlook.com (mail-swedencentralazon11013067.outbound.protection.outlook.com [52.101.83.67]) by sourceware.org (Postfix) with ESMTPS id CA0BB4360BB9 for ; Sat, 6 Dec 2025 16:57:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CA0BB4360BB9 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org CA0BB4360BB9 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=52.101.83.67 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040248; cv=pass; b=dQFf0W8aLzowJvdCfV/AEN24O3A+s3Al1E5QVW/J5laXAwk1lJK/EgvUw0xbGFkkPqLniVHNDnou32m4WVbjRs/I8mVONvmhQExM8fE4F0YU4qByU+IWCJT5ZERbo9EgQsWoy9hBLG3REk8EFJYwcrmaP+oNfxfw9qM5+ebcsJk= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1765040248; c=relaxed/simple; bh=IloR0b3ON9k16TNPsicw8b2Z1XUi5LBBnc6HAqgGbPE=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=t8WT/th8FVTify6E+adnorL2HAZXh6zf0XtR8rz92k5xYUS+eOsL/gIK6LtXYJEcuaB4rjRmn6vyPFTP/awDBNO7NUZ3TYYddzzxHFFbof1rZrWajX51KmxxPP49YZP3Lg9NydzAGZ2h2oR3q4UnF7bfzILlNZLXyP+36DAu/XI= ARC-Authentication-Results: i=3; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CA0BB4360BB9 ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=WEkyNmOMAsJuF/5nvlJIvfk9WwtIcg8z/0wjb91te5l0KyN1ks2vih++85eS5fB17KEsLZSDfc/fjyhH0LB1MPlqiV4f7NzFKAuhQAwwCzwoez9CGweR5GKvE/7oS7xBmJ8X3wd/PGdmOyLK+1ztKgBJ2molNXkZvqxPro2uOBg51uCyGjz35y8W76Oys2yI+LZQpW5vFiz2MDRRqtAjW1otMwg/HjOE9tTcXuewuevUfCGp7bLdPqyHW6ABKvnZo4DMCyN42fRWmvHLaDQfiA4MMCSAl6wk1kmxfYill9DYRV3qRsJzpE6IWixViWNtb7IgxuflpU4tleL0OtyPvw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XMpm9f41c8VmWvGan9qpBkEaDkh6qT9eYCd71+lytIc=; b=whL7s/XLYDVKbSv5R3yIk+vRQzhancqz3bh4WYyD7VZt2Iit8gvGu44IFc/Y5NFVOXnvqvZPEdSiAlkuA5DkxNOXn5OAUlKX9OisUT96BpYUaRG9RV6LhrpMFP5ttsGAKMsItCFN4n+4PIOoUD+wPGFmPYhKgVngjk/S/49lrhFy5VHsIQibp1M4YiWJ3blFR9bHdJcyrVklPsnzIZZl5hUG9357546xFbDdwJYJm09+Or87LWRYuPxo/HMAPiLPD/WPeSijUSQfEG0QxhdbKgDTpvfU4LCyhnmZcpqyofb3IbNuenU06VuqpDGA6XvXvT52GE/cfRQjKZYC1c3Bmw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XMpm9f41c8VmWvGan9qpBkEaDkh6qT9eYCd71+lytIc=; b=B20RwXbMqS0Fq4p7sIujY20W4G6Pcnrf3QPchPZuPtgMKPMHxmFg2H8OLMzOnVpXqLgLeP5DVNyG7mmlgOo//sFreQvk6t9jIfwRzeaXSjC6dWD2R98SS03n2Tr6fJjIVrHfSe7/DNC8uKAH/iIwLIUlWfnXq9sIPRiVR0NXka4= Received: from PR1P264CA0069.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:2cc::7) by AS8PR08MB8037.eurprd08.prod.outlook.com (2603:10a6:20b:573::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:57:23 +0000 Received: from AM4PEPF00025F95.EURPRD83.prod.outlook.com (2603:10a6:102:2cc:cafe::cf) by PR1P264CA0069.outlook.office365.com (2603:10a6:102:2cc::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:57:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AM4PEPF00025F95.mail.protection.outlook.com (10.167.16.4) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9434.0 via Frontend Transport; Sat, 6 Dec 2025 16:57:21 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=td6Yq9Wu959o+oD9aU7ihFBJLk1F3iiYyikQdLtTA0zuOXmkkdmXHsMdS6Q6fV+PGOTZx2E5HMBZGi5j3SZ+WwPfhRZCPHdxC3PTbYUpVMl3S9eFSVYX8SW4H2fuOKklyae4cY3s1dpvx5hMBz0tgFF/ljboO9C7YTh2cXbl4/l+/2EKZKupFD2zvDPP8k7rCQKWanfW9vqFu58srp/NbnWISNaxMinZ1TtZJcUybElzw0elMyllyi48CXesTaXpx1iEs/K3xZSaC3NPCDFNopx90BIXq6+oFSaKpIs7zqivtMzCyC0yT0Rpqy1KfJvKRpXl+5KMh3Yw3WgeFwdkAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XMpm9f41c8VmWvGan9qpBkEaDkh6qT9eYCd71+lytIc=; b=zC09DVh2y5BxQPfCFfM6AaHyW5Zzy6ITah5sHSpvTMWoqxG4R3vRlHZB2FAu4BtwsG6JJvM4v8NsQSbhKgD0IwbcR2+EDmgx23VnvE6mUMsWeuVBcuNYrPOrIX9a/1RUvev54MGZ2KEXTLVATiNDvL4SfKkIbKELUIJUFYSAyl+ncduz3i62+PSrNyOTSIbIkPE+/6sg4JFCDBrxypbUQPKz/RZ5vZRlVA4iOhTdw+sOkHUY25ovwGSOIrrqEVpWY13fVrW5IhxTvb1OquWCj5H0r+VFzsleVC6vnWmRp0+85SATiFnidU6XrG1e6BCE2LtVhH3xS2PaIO2EDrYJDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 172.205.89.229) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XMpm9f41c8VmWvGan9qpBkEaDkh6qT9eYCd71+lytIc=; b=B20RwXbMqS0Fq4p7sIujY20W4G6Pcnrf3QPchPZuPtgMKPMHxmFg2H8OLMzOnVpXqLgLeP5DVNyG7mmlgOo//sFreQvk6t9jIfwRzeaXSjC6dWD2R98SS03n2Tr6fJjIVrHfSe7/DNC8uKAH/iIwLIUlWfnXq9sIPRiVR0NXka4= Received: from DUZPR01CA0274.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b9::13) by AS8PR08MB8038.eurprd08.prod.outlook.com (2603:10a6:20b:548::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.13; Sat, 6 Dec 2025 16:56:16 +0000 Received: from DB1PEPF000509F3.eurprd02.prod.outlook.com (2603:10a6:10:4b9:cafe::ff) by DUZPR01CA0274.outlook.office365.com (2603:10a6:10:4b9::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.13 via Frontend Transport; Sat, 6 Dec 2025 16:56:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 172.205.89.229) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 172.205.89.229 as permitted sender) receiver=protection.outlook.com; client-ip=172.205.89.229; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (172.205.89.229) by DB1PEPF000509F3.mail.protection.outlook.com (10.167.242.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Sat, 6 Dec 2025 16:56:15 +0000 Received: from AZ-NEU-EX04.Arm.com (10.240.25.138) by AZ-NEU-EX04.Arm.com (10.240.25.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.29; Sat, 6 Dec 2025 16:56:15 +0000 Received: from ip-10-248-139-165.eu-west-1.compute.internal (10.248.139.165) by mail.arm.com (10.240.25.138) with Microsoft SMTP Server id 15.2.2562.29 via Frontend Transport; Sat, 6 Dec 2025 16:56:15 +0000 From: Christopher Bazley To: CC: Subject: [PATCH v6 11/11] Add extra conditional dump output to the vectorizer Date: Sat, 6 Dec 2025 16:55:18 +0000 Message-ID: <20251206165518.5449-12-chris.bazley@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251206165518.5449-1-chris.bazley@arm.com> References: <20251206165518.5449-1-chris.bazley@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DB1PEPF000509F3:EE_|AS8PR08MB8038:EE_|AM4PEPF00025F95:EE_|AS8PR08MB8037:EE_ X-MS-Office365-Filtering-Correlation-Id: bb4e7b81-8511-4db4-a0c8-08de34e8858b x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|1800799024|82310400026|376014|36860700013; X-Microsoft-Antispam-Message-Info-Original: 3ZmQPy2u3n5YQrD3bcV7NlOnEjt4OJz4SOwyIzK8vAZ54iSAHZoNRp7Uf8kwX7o0YSLq4QIRpKdSDF56fFEbh65ythIM642iwx8MWQVNGhTyB0JgTavcDNMgN0/jny3OPKyEtJrHouKkKGwBHiu642ls4bik5L89J2Iel6H+kvdPpkGEN0oVBU+k69V8qIX9nWsV3y55hUBrgpuCeuQzSvAqfXjht0A3wO6tAfom1vQxtujoC1fqcf1aX/6AvMjGSC7SiMZDpYe3wVEc0149fk0ZQ79OktaKabKHEVbRQyipJO3ddBvcq9tUaYhvcnEG5tkwBiz1ibAoiPKr9AUL26LrAWzT5cj9onqN8EQNAJiblY1d0zUwmHnuyHa0XSmewtMnZtcKKJV8aBjjPIbKWIoW7czSc5YSUaXb95hvQVYLwVJhb9LfpGip3ZErkbJRZxg9IgiW6bN4xKfdBdI5wCJMgXF1eunGvKGbxiEzS8OhK7wxPjawADTYJkIX/15sPFiWs/FlyjGFEJjEmrc56ujCleZtloQQIBSFxZ2akj8V+UZM7udtdcI5GlAl7jJTFPxq2bZEuiYeNyroyR0LBMjFQAJmwU97dCQE3bySQr8T3Wmnu719ZTAGToCIj2mkI0eQyYDwpFuC2vLFmDcT4yRzR1fU1KvEkeqNg8OW8riycSOOEqGfdj4xXwfMdPSceeqf9fkorN2tmf4xaPcDhTzhWjH5VB77ZYH+0lpX8fgoH+oEU8iP2hTwJY0ow9qsHK8OET3p4OEx9a+wCTWGS1gnePn49HrPMYwloeeRwhgzZcfojD3YH4wkMBKHCLkiKHks/ywdzcyjxME8zzwAvKxWshxUhtvJjtijYqqGQtGtQAUxhM+l6IjmJMcRczdbS0cXiPuLSvT8vQ9e8Lu4U5Ye1hIAxBTSt+FG8KCFGpiv4O4J92+gjI3KrzOufrRgbYwWdvFqWjPJVK/+nbWVMRDAPZsQejZawTKUskdACOCy8K+lh4NGCn5z3K4Sz9feOHHvxaAjsbnTSArlWNKL7ysFgchFTgNUOmLK1jmKb090XtwB1TnbNcvToaFFwzNdvWAt04sVYCxqqoUaIS3X5uiievcVZN+ojT9RTI7Pdkkd0jYIAxhSro7/LTjJh1MnUuz+Q9dUybOEGBbRUXj+W1TMibiNlfMYcHUfluAhpG/3/A1qmZBpSQ6wqE7S7A5rW+U5bbW69mIE5BpTyUSRXzVgZBKCJt2oMQUfrhNlctoeYqIZg2hpzmyMGsa/u52b33U/tLmuNwmKY0UHfmP1dwrGvuGKgTV5ak7STqrZh3wzOIFbVDhb7L3TDsgVNgqxam0RRnCdvjh66H+feKITcKasdaj/4LPT/YCyNCIdtbseqc6kgy/b/1HX19VwXmzB+UpSUo5+yrkAVcuhFyfbCalDlPa2BNPz+XLNajYIt7T62cjUCNHS9wEs7+Ab59L5QU2DIMwIFgBAfZZdJyaUfAWXwkjbnjHKXBH/9rnKNs0xUtOor0C/uI/Cq0mmMF/Y5/9zGH2dQy2HGJtZ4I+TfYWY/A4IVbY0tbkD7/AQJrU= X-Forefront-Antispam-Report-Untrusted: CIP:172.205.89.229; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(376014)(36860700013); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8038 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM4PEPF00025F95.EURPRD83.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: c301ff69-a8fc-46de-cf42-08de34e85e96 X-Microsoft-Antispam: BCL:0; ARA:13230040|14060799003|82310400026|1800799024|35042699022|36860700013|376014; X-Microsoft-Antispam-Message-Info: fz/kh7pLTHhTmu2hIov6GSlDra3rI31fW00jryFYeN284Kky2F4kyZa4769CZaCcMc0zXFKWkp/l83dM0EUuBRH3m4KegoWc3Q9s4HoisMgs4g1d2XKXBSydeuGMK+tn65BDwQB8ZSezdhxgdIJqcESgyA63d8ap2tNNyzOQnz8zV7ssxuDiqvwSgn9a+Fw8x46j7Ci2UlEaI8OYyjT/TQNUageRTnrlmJhXhnc9SQNwRUfbkXCqPfvFcEK56IBisbj69oE/bNp7EzCuaU0poO8gNMfwzWr1B3Pn74+HJiD4jq5OhXzcWAk+upsL0fNR5gx3VL/t2DZR7KRklVys+2aDzxMqpR+u2DdWrL2JV/ToHa315b2ScALL2zXmP/une9P78KFK73TKAYOrTR6OEG/WQJZkClbXGfZ5Dx+BfWrQvDXl89UivWSwBlU7carPOzowyEXxniNwE+Fl//2NxAsEThpXamgoPATar2jp2Ki+Dp9VehEp6w9m0gSwbf5WmQa+KmEZ435zfM+SCmhBAjO79nyBDLZHRB/7rSAOZbiNriAgYtdGJvAkmcx81xGRcUIP4NW2Q5i4GawVcMKPS6UsDc/qyukeA05zqZzVVmtPkoNT1UP3EedcvvlveAPwfVUhim/YcFUk9dQVXr5XbS9g3ZNme2+KJt61xOan+r8A5U9uSvwrbbhi27bVdat8nBq8utnH6BIH8UZcFD+m8p7BJxJoHRe+tawNpsv8ftqt5eiHKobUq0MJrZHCP5mh86ezJu0I4BYq7wCk/42cE0pnJrbBQaAk47idnjI3l5hIWPjQZo8BFtodL5wuBGBvHVncsJ2AxdENAdfLqzWCbTUA8KTjkAZMcQf51/yt+28nNBOlrEmmM8Xbm/eNRCQ+tTuH1MWW+BULJgLqomV+7rG1lTFiZQL4pWjXLX+3AGl91gXbmRvcBHD82VWo8Jk67unLcDYhR91GEPltZ/i6xthoeHVjUdajOhWpBMlA4tasVpecB4aPsl3laELz7CXOvsA5hKW4aIwh1Yx3jR/8FPHpfxL5gTBM7ARAcPfFHXOImdxwmxLb+LrhWqVuREfrWSpEhTJIqsF0oZRCPyFAYenJlbU2XNUktQJcH5w26zQ1QsZW3AhweB8P9sf+aUju92ybJ/p7Psqd6fID2TpmAysdeu3/lSt0cI1r2XD+L5LzaAVU6lDpS3Li6P9L0KAux94zboxEjdIOKwK7mljcQnlBx3kmaPKzmrBcHuMf2xOUGSl/UWctiHoi8FF/KLQ9izKYS9TYcgkRCxiBs89M3NEmD3Oe/pthKRqTS+p3uwXRpplcVWKMvmnn9giuBiMMZ69FghZDdlBSSTcRJAcvxoJC9HdFELstJCGggGHhhd0ysqi+TReX7jpasrdZPdoJMUVFbnC/UD7B5EvMbPpERit8Y1CcFBAELA12mD2BcFThJs6enG5Jrj3v3J0wnpz9NybxaYUtFenGcXlqgdEWnlHkZqDP96oPfNfvga6HQPxcMxwti0uQ7q4GohzS+YFDpccMTf8LiH4pZV+3KzrdqdHJ2PeHMXMpT5c5SoxfnHA= X-Forefront-Antispam-Report: CIP:4.158.2.129; CTRY:GB; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:outbound-uk1.az.dlp.m.darktrace.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(14060799003)(82310400026)(1800799024)(35042699022)(36860700013)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2025 16:57:21.0387 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bb4e7b81-8511-4db4-a0c8-08de34e8858b X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[4.158.2.129]; Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00025F95.EURPRD83.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8037 X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_NONE, SPF_NONE, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org Instruments those things that needed to be instrumented in order to develop predicated tails for basic block SLP (superword-level parallelism). gcc/ChangeLog: * tree-vect-loop.cc (vect_get_max_nscalars_per_iter): Dump the result. (vect_verify_full_masking): Dump the number of vectors and number of scalars per iteration. * tree-vect-slp.cc (vect_prologue_cost_for_slp): Dump the SLP tree node's address, vector type and group size. * tree-vect-stmts.cc (vectorizable_conversion): Dump the vector type inferred from the scalar type. (vectorizable_comparison_1): Dump the vector type guessed for an invariant comparison. (get_vectype_for_scalar_type): Dump the natural choice of vector type and instrument the fallback loop which searches for a narrower type. (vect_maybe_update_slp_op_vectype): Dump the new vector type and number of lanes of an operand when its vector type is updated. --- gcc/tree-vect-loop.cc | 10 ++++++++++ gcc/tree-vect-slp.cc | 6 ++++++ gcc/tree-vect-stmts.cc | 38 +++++++++++++++++++++++++++++++++++++- 3 files changed, 53 insertions(+), 1 deletion(-) diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 9164cef92da..4f8c500c131 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -906,6 +906,11 @@ vect_get_max_nscalars_per_iter (loop_vec_info loop_vinfo) rgroup_controls *rgm; FOR_EACH_VEC_ELT (LOOP_VINFO_MASKS (loop_vinfo).rgc_vec, i, rgm) res = MAX (res, rgm->max_nscalars_per_iter); + + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, "max_nscalars_per_iter=%u\n", + res); + return res; } @@ -1018,6 +1023,11 @@ vect_verify_full_masking (loop_vec_info loop_vinfo) unsigned int nscalars_per_iter = exact_div (nvectors * TYPE_VECTOR_SUBPARTS (vectype), LOOP_VINFO_VECT_FACTOR (loop_vinfo)).to_constant (); + if (dump_enabled_p ()) + dump_printf_loc ( + MSG_NOTE, vect_location, + "verify_full_masking: nvectors=%u, nscalars_per_iter=%u\n", nvectors, + nscalars_per_iter); if (rgm->max_nscalars_per_iter < nscalars_per_iter) { diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index 4cd687a324e..bcabcb75b6c 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -8781,6 +8781,12 @@ vect_prologue_cost_for_slp (vec_info *vinfo, slp_tree node, When all elements are the same we can use a splat. */ tree vectype = SLP_TREE_VECTYPE (node); unsigned group_size = SLP_TREE_SCALAR_OPS (node).length (); + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "vect_prologue_cost_for_slp: node %p, vector type %T, " + "group_size %u\n", + (void *) node, vectype, group_size); + unsigned HOST_WIDE_INT const_nunits; unsigned nelt_limit; unsigned nvectors = vect_get_num_copies (vinfo, node); diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 5bd32e5f287..ce24dfa11b2 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -5474,7 +5474,12 @@ vectorizable_conversion (vec_info *vinfo, /* If op0 is an external or constant def, infer the vector type from the scalar type. */ if (!vectype_in) - vectype_in = get_vectype_for_scalar_type (vinfo, rhs_type, slp_node); + { + vectype_in = get_vectype_for_scalar_type (vinfo, rhs_type, slp_node); + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, "inferred vector type %T\n", + vectype_in); + } if (!cost_vec) gcc_assert (vectype_in); @@ -12628,6 +12633,16 @@ vectorizable_comparison_1 (vec_info *vinfo, tree vectype, if (!vectype) { vectype = get_vectype_for_scalar_type (vinfo, TREE_TYPE (rhs1), slp_node); + if (dump_enabled_p ()) + { + if (vectype) + dump_printf_loc (MSG_NOTE, vect_location, + "invariant comparison, guessed type %T\n", + vectype); + else + dump_printf_loc (MSG_NOTE, vect_location, + "invariant comparison, no usable vector type\n"); + } if (!vectype || maybe_ne (TYPE_VECTOR_SUBPARTS (vectype), nunits)) return false; } @@ -13602,7 +13617,19 @@ get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type, /* Register the natural choice of vector type, before the group size has been applied. */ if (vectype) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "get_vectype_for_scalar_type: natural type for %T " + "(ignoring group size %u): %T\n", + scalar_type, group_size, vectype); vinfo->used_vector_modes.add (TYPE_MODE (vectype)); + } + else if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "get_vectype_for_scalar_type: no natural type for %T " + "(ignoring group size %u)\n", + scalar_type, group_size); /* If the natural choice of vector type doesn't satisfy GROUP_SIZE, try again with an explicit number of elements. A vector type satisfies @@ -13635,6 +13662,11 @@ get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type, { vectype = get_related_vectype_for_scalar_type (vinfo->vector_mode, scalar_type, nunits); + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "get_vectype_for_scalar_type: trying %u elements " + "of type %T: %T\n", + nunits, scalar_type, vectype); nunits /= 2; } while (nunits > 1 && !vectype); @@ -13963,6 +13995,10 @@ vect_maybe_update_slp_op_vectype (vec_info *vinfo, slp_tree op, tree vectype) } SLP_TREE_VECTYPE (op) = vectype; + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "updated vectype of operand %p with %u lanes to %T\n", + (void *) op, SLP_TREE_LANES (op), vectype); return true; }