From patchwork Tue Dec 19 10:12:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 82445 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id BBD35385DC1D for ; Tue, 19 Dec 2023 10:13:02 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2055.outbound.protection.outlook.com [40.107.22.55]) by sourceware.org (Postfix) with ESMTPS id 0D4D33857437 for ; Tue, 19 Dec 2023 10:12:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0D4D33857437 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0D4D33857437 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.22.55 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1702980739; cv=pass; b=FFoNKHaDKpQEqkxxxipSgH8+4EIWRayXyW42mm6bkUTIH86aqHRRg+qdDT1IkyN9+VRBQfa/o5PJ68hwLaX1rix8o+z9uq8E+rXlHjbNgesCqHmCMiqPlLJqMY910eGPOaykIvh6QKTurvLyTTQ1ZYkOLZaf5w9katI5KSNGM0U= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1702980739; c=relaxed/simple; bh=abawl81nisyfSnmczMpDrTJDMgxh30+E1p7wqHNkTu4=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=aoeRSuJYr+xakte3yY3k2rmjgP1MY0xytu/aFDvJV9m9sShvA2S4z92wCLzEVEGnppHojOnmbyZU/ubTtbg4y5DzJ3/EKKjtgASfSX0G1aA7Up+CI+bDEhpNK1VsaaLQxHKBvyZh2szG7zfr6HZyJ6cCfyCZXpsdqi0BeJWVtHY= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=oVRCjOGhseztfwq0hvihmMMfSSJpv/HM4jaSZRmN6dzoa9Q0nQBgYuvZ3acEGxIKcFMtwJsIIMyQ6wnQ3I6RX2Ym9T53ToxkcQaokR1I4hRONQ7vURSXcopOwikKbKD+jvUow4hSIyLlzK93nvUBTgRHELj+ZsRSauMNnMJCNHTMjuCADgxa1Ns3YbZfLQNpoSL1r0vQQP0FTvnoT99ftXb6Bkd8t0wXp2DGA4WD50jOUgddexUz4BCVNgZ/1pNjnNnrZPV2litTY4APemHH2etagWJgfCMSTgkbF1jGmef5DqQCCQX/R9idWHrfC8a8JIkQqd4r4FdF6fD1gv2LTA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/rmPp9ZCJwOTDoqV2NCaZMhmuTLHRzA2bZqjJ6ukymA=; b=Qv9bG5WyM/ibiNO1YmBa4zZ02Q6jwP1jHUugbZo22vGI3ABaGw/DEIwM3Efj8QMeizscyH9MM76lyvCu2DoRtUi8qf4UMZ7B86ma8gAN6DQSXHtx+xCPnc2voncXJMH9r9lc9ymIDa5UVzFJnUFuqjkzxlTBh7GXpxAoZcaAc5+ksE5wBlxb92F/zhMx0vyWUsQiWQ3U0dZw3iW/5V1PSHmzrxIk4Vk/UhOIF9AZGKICeWQGJXhPLLUYDYoe2PzADzPnP6B9y9Yb+ag5enna3NdZ5RsO0exjcqfsaH8NTev2Bgx4j5fEucAua8c/r7NEj6McQJ4vqzZyQ1rApvch0w== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/rmPp9ZCJwOTDoqV2NCaZMhmuTLHRzA2bZqjJ6ukymA=; b=fhOTkft0Uw9YtOmFehHPVkUyXb6Qhyhfq8BB6bi8sDMsIxfj/DhOrHCdsutY0auboq1W+BdagRervZWUgaNohJsTJAHud8Avu64wPQt8jck098Wz/7yWYTAsVL7oldfxQZIffN1BmqBo5m9IAcEuSw2D5/xXhFO3HSvXBu0Eetk= Received: from AM0PR01CA0141.eurprd01.prod.exchangelabs.com (2603:10a6:208:168::46) by DU0PR08MB7463.eurprd08.prod.outlook.com (2603:10a6:10:356::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.37; Tue, 19 Dec 2023 10:12:15 +0000 Received: from AMS0EPF000001A0.eurprd05.prod.outlook.com (2603:10a6:208:168:cafe::87) by AM0PR01CA0141.outlook.office365.com (2603:10a6:208:168::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.37 via Frontend Transport; Tue, 19 Dec 2023 10:12:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AMS0EPF000001A0.mail.protection.outlook.com (10.167.16.230) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.14 via Frontend Transport; Tue, 19 Dec 2023 10:12:14 +0000 Received: ("Tessian outbound 7671e7ddc218:v228"); Tue, 19 Dec 2023 10:12:14 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: c2b4b390e6c06bc7 X-CR-MTA-TID: 64aa7808 Received: from f8ad79afbff9.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 00937270-1BDC-4132-A225-1E6B64576718.1; Tue, 19 Dec 2023 10:12:07 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f8ad79afbff9.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 19 Dec 2023 10:12:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B1iztovFFPQPvIOAuUyKUxfhTiFZ+0umq3N45RzMQ0kducMSoPPIJA/TbrZHBrvGwAF5/uTmSSqHFyLen0cJTQzo4Ic1/SETo5puXhh+KnoNVUpwesOhK/6gVBIEtZfevE26Awxh/D58aHPY/TwJXaP+2B6MOmNSFQxPx+WeH3HV8axG6WDRLYQhNRtMRn8L/SJMmuhrX5RFYsE4/3OCMo+EO+AKAuDNl+ZXyIEbl6uycuT0Y4/v/ARROYP4TFSfITM+WoJ5KF30PLFdmI08gizylnDh+tSxja3ktsuXnzOg75OLs8GUiNxICxwMe7BULZNZn25uQFIA2vKlncpMVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/rmPp9ZCJwOTDoqV2NCaZMhmuTLHRzA2bZqjJ6ukymA=; b=cGz8Or1bBwq7Z9fGmbbNpBbp4ZzcxI0TrovW4HUEUcG1/SIlQVZY/JhFnupkSPhCcM58dR/ORpK61idWJFIFJiswB0OEMKpTTcRYSsVkRm/YZvEXThd8ZyN6439hm1w4iNUDIyCCSC835KYcZZfsUWxFt14VSCkpn/dBa51UF2GkShTVWQEagDn+mwaIr9eZOVYeC71tbp/yGpJVa1wfGpNMQDBXwQ3bxEz9Ef6SGCeBJK0ckSWAF/d4rT0r1NMqaqzH9f8yHMwroveh6K7J0AQBsojwnfU7m+UQvC4UM/kF6jNp/o1zTgLE270Vlg6DyVX36nW2MtYrHZ2JC/6g7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/rmPp9ZCJwOTDoqV2NCaZMhmuTLHRzA2bZqjJ6ukymA=; b=fhOTkft0Uw9YtOmFehHPVkUyXb6Qhyhfq8BB6bi8sDMsIxfj/DhOrHCdsutY0auboq1W+BdagRervZWUgaNohJsTJAHud8Avu64wPQt8jck098Wz/7yWYTAsVL7oldfxQZIffN1BmqBo5m9IAcEuSw2D5/xXhFO3HSvXBu0Eetk= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by PA4PR08MB5999.eurprd08.prod.outlook.com (2603:10a6:102:f2::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.38; Tue, 19 Dec 2023 10:12:04 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3%6]) with mapi id 15.20.7091.034; Tue, 19 Dec 2023 10:12:04 +0000 Date: Tue, 19 Dec 2023 10:12:02 +0000 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, jlaw@ventanamicro.com Subject: [PATCH]middle-end: Handle hybrid SLP induction vectorization with early breaks. Message-ID: Content-Disposition: inline X-ClientProxiedBy: LO4P123CA0403.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:189::12) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|PA4PR08MB5999:EE_|AMS0EPF000001A0:EE_|DU0PR08MB7463:EE_ X-MS-Office365-Filtering-Correlation-Id: 44caa0ae-6cfb-4c5c-9c2f-08dc007af959 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 2lopNlp0IYKU8eM3juABO7Y3yrdq75G1eZ+K58Dp0lnzNWypFIvVCV1JabH9rrnkoMvr7rYOZfa8LOiVJGt4lPt7GaywB775/SHmpmFmqXwN/baPBjBtiiYB3rPORBQwyxVu0teE3o0NK8jDoEOAGEcWv+XN9JX1f+cJ/xREPjrGTSqJrsXbB+snMWEpUH5xWLbtvcdX8eHAM0e738wWqCuLt5eFsOUZ7rAPoNz0VPaYobdwJNNX8NAwLae6G9ZPi0wFwrWEsqvbytzGCnPvAe7rt8UYXBHTNY+KOBnqPF1KPATXsVZq1aPu55vVTTGyzFY9qmk0ogTcFAxR/QlWSzBV8rcFbaZknvhC3UjyHj9a+EjILx4dorflhvV8xWHBaPzW3Zq4CGp0DKK713PcHHPOBBjr6jl73IqOAKOwxJ/iz4iF/pJfTdjmxSckT/Wmi1UHFBUUKHO4UpQQoMJ0xmaUdb7yMLYVPqQdKDe+dslDqJ9F+oxW8cRJKBEzF/ItsnqeNOBty/EeJoo0+9DAhO3dyiIgz+Wj9BgLp+MutQ30eKKrGGop4O0UmGMhVj3YDMMF590tR4yEjRFKf6tfS1mOXZ6j40XRqbKbprTkslV3EdGRhAYX089UtcfXwbk0 X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(39860400002)(136003)(366004)(346002)(376002)(396003)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(44832011)(2906002)(8936002)(8676002)(4326008)(6512007)(6506007)(33964004)(44144004)(83380400001)(235185007)(6916009)(5660300002)(316002)(4743002)(66556008)(66476007)(66946007)(38100700002)(2616005)(478600001)(26005)(6486002)(41300700001)(36756003)(84970400001)(86362001)(4216001)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5999 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS0EPF000001A0.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 9c7b43e7-cb28-407b-f0d3-08dc007af300 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1CxGVAbyn0NeViaX2e3O8muHE1zwgmStuPYUbt+4fXlafnqVX+Jc69PHD3VjnAYb4Q6zEgFKHOM4cxcKAQOsvjkKH0BC/nCPMsbngprYbOSizsR4fwZ2T2AeA9M8dqvUrUv8tCFjvEibhZWbE47G/xtNJbomfFsBRMr9wOnAQ1MgzHElbCD3EYYBWyUK24LugiN6nhp3u5st0dHRj0hb3H68Akge3d2w19/Sk+wqePue2STRjfsxEIIApHqpAkhwEKB4u5jmnT7Jt4WrY6aHcuVwRiQEm3EbflgMmYOq8S78iAkDoHPdqw9elBqNRalJbBV7Q/axyUXWargDOvD4GR73SElo3alhsUJEArWT/+lEnJuBrg88twalHFYwWuSbvOPVO2CxFsN4DhjmIEyX5Nsso3w63kTLCG6q1eZUpkhMio1xb9fdCTD+CBMFfhXH1udQbORJLl3maAY1Xy68RlQv9I0fnIKJgNRjRfP9X7vb2SktlTVerVdK5g7URzmTwIcd+msc9PVWT+U6Cgo938U8eGMzwm3rSZyX11IgXpleu87pOG83bhFWRNPwjyH/Fee19kuSWQqv4HJG3WHFYMejUePF82UT3GhqkCqqISxSm/llUU/kbe+vpNARIKWadESy4zfrjST4Bv9w5JGiaqD+qq4PKr302P82/1H9r1eNoLN4O7P+0uPnk7+72MgasiJjsso8rDKavaPTb51I23Jg/3GRX/Bj8FfPEoINd+IOGiT+ABWx3CGJPSP4HLy7Q1EqZPu9NjVtFbRVOzpxJLmsMAWTRofmRxqfNXJ0khQ= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(39860400002)(136003)(376002)(396003)(230922051799003)(186009)(1800799012)(451199024)(64100799003)(82310400011)(46966006)(36840700001)(40470700004)(40480700001)(6512007)(6486002)(107886003)(2616005)(70206006)(4743002)(336012)(83380400001)(84970400001)(6916009)(70586007)(316002)(8676002)(5660300002)(44832011)(8936002)(47076005)(4326008)(36860700001)(33964004)(44144004)(26005)(6506007)(478600001)(2906002)(40460700003)(81166007)(356005)(235185007)(82740400003)(86362001)(41300700001)(36756003)(4216001)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Dec 2023 10:12:14.8396 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 44caa0ae-6cfb-4c5c-9c2f-08dc007af959 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001A0.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB7463 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Hi All, While we don't support SLP for early break vectorization, we can land in the situation where the induction was vectorized through hybrid SLP. This means when vectorizing the early break live operation we need to get the results of the SLP operation. Bootstrapped Regtested on aarch64-none-linux-gnu, x86_64-pc-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-loop.cc (vectorizable_live_operation): Handle SLP. gcc/testsuite/ChangeLog: * gcc.dg/vect/vect-early-break_82.c: New test. --- inline copy of patch -- diff --git a/gcc/testsuite/gcc.dg/vect/vect-early-break_82.c b/gcc/testsuite/gcc.dg/vect/vect-early-break_82.c new file mode 100644 index 0000000000000000000000000000000000000000..f2a6d640f9c0c381cc2af09bd824e272bcfee0b8 --- diff --git a/gcc/testsuite/gcc.dg/vect/vect-early-break_82.c b/gcc/testsuite/gcc.dg/vect/vect-early-break_82.c new file mode 100644 index 0000000000000000000000000000000000000000..f2a6d640f9c0c381cc2af09bd824e272bcfee0b8 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-early-break_82.c @@ -0,0 +1,27 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target vect_early_break } */ +/* { dg-require-effective-target vect_int } */ + +/* { dg-additional-options "-Ofast" } */ + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ + +#include + +#define N 1024 +complex double vect_a[N]; +complex double vect_b[N]; + +complex double test4(complex double x, complex double t) +{ + complex double ret = 0; + for (int i = 0; i < N; i++) + { + vect_a[i] = t + i; + if (vect_a[i] == x) + return i; + vect_a[i] += x * vect_a[i]; + + } + return ret; +} diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 85b81d30c5ab869cb1f7323caabd9fe4648bdc50..0993d184afe068784474ac225768d9f38d76c040 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -10856,8 +10856,8 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, bitsize = vector_element_bits_tree (vectype); /* Get the vectorized lhs of STMT and the lane to use (counted in bits). */ - tree vec_lhs, bitstart; - gimple *vec_stmt; + tree vec_lhs, vec_lhs0, bitstart; + gimple *vec_stmt, *vec_stmt0; if (slp_node) { gcc_assert (!loop_vinfo @@ -10868,6 +10868,10 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, vec_lhs = SLP_TREE_VEC_DEFS (slp_node)[vec_entry]; vec_stmt = SSA_NAME_DEF_STMT (vec_lhs); + /* In case we need to early break vectorize also get the first stmt. */ + vec_lhs0 = SLP_TREE_VEC_DEFS (slp_node)[0]; + vec_stmt0 = SSA_NAME_DEF_STMT (vec_lhs0); + /* Get entry to use. */ bitstart = bitsize_int (vec_index); bitstart = int_const_binop (MULT_EXPR, bitsize, bitstart); @@ -10878,6 +10882,10 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, vec_stmt = STMT_VINFO_VEC_STMTS (stmt_info).last (); vec_lhs = gimple_get_lhs (vec_stmt); + /* In case we need to early break vectorize also get the first stmt. */ + vec_stmt0 = STMT_VINFO_VEC_STMTS (stmt_info)[0]; + vec_lhs0 = gimple_get_lhs (vec_stmt0); + /* Get the last lane in the vector. */ bitstart = int_const_binop (MULT_EXPR, bitsize, bitsize_int (nunits - 1)); } @@ -10917,7 +10925,6 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, so use ->src. For main exit the merge block is the destination. */ basic_block dest = main_exit_edge ? main_e->dest : e->src; - gimple *tmp_vec_stmt = vec_stmt; tree tmp_vec_lhs = vec_lhs; tree tmp_bitstart = bitstart; @@ -10928,8 +10935,7 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, if (restart_loop && STMT_VINFO_DEF_TYPE (stmt_info) == vect_induction_def) { - tmp_vec_stmt = STMT_VINFO_VEC_STMTS (stmt_info)[0]; - tmp_vec_lhs = gimple_get_lhs (tmp_vec_stmt); + tmp_vec_lhs = vec_lhs0; tmp_bitstart = build_zero_cst (TREE_TYPE (bitstart)); } --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-early-break_82.c @@ -0,0 +1,27 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target vect_early_break } */ +/* { dg-require-effective-target vect_int } */ + +/* { dg-additional-options "-Ofast" } */ + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ + +#include + +#define N 1024 +complex double vect_a[N]; +complex double vect_b[N]; + +complex double test4(complex double x, complex double t) +{ + complex double ret = 0; + for (int i = 0; i < N; i++) + { + vect_a[i] = t + i; + if (vect_a[i] == x) + return i; + vect_a[i] += x * vect_a[i]; + + } + return ret; +} diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 85b81d30c5ab869cb1f7323caabd9fe4648bdc50..0993d184afe068784474ac225768d9f38d76c040 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -10856,8 +10856,8 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, bitsize = vector_element_bits_tree (vectype); /* Get the vectorized lhs of STMT and the lane to use (counted in bits). */ - tree vec_lhs, bitstart; - gimple *vec_stmt; + tree vec_lhs, vec_lhs0, bitstart; + gimple *vec_stmt, *vec_stmt0; if (slp_node) { gcc_assert (!loop_vinfo @@ -10868,6 +10868,10 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, vec_lhs = SLP_TREE_VEC_DEFS (slp_node)[vec_entry]; vec_stmt = SSA_NAME_DEF_STMT (vec_lhs); + /* In case we need to early break vectorize also get the first stmt. */ + vec_lhs0 = SLP_TREE_VEC_DEFS (slp_node)[0]; + vec_stmt0 = SSA_NAME_DEF_STMT (vec_lhs0); + /* Get entry to use. */ bitstart = bitsize_int (vec_index); bitstart = int_const_binop (MULT_EXPR, bitsize, bitstart); @@ -10878,6 +10882,10 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, vec_stmt = STMT_VINFO_VEC_STMTS (stmt_info).last (); vec_lhs = gimple_get_lhs (vec_stmt); + /* In case we need to early break vectorize also get the first stmt. */ + vec_stmt0 = STMT_VINFO_VEC_STMTS (stmt_info)[0]; + vec_lhs0 = gimple_get_lhs (vec_stmt0); + /* Get the last lane in the vector. */ bitstart = int_const_binop (MULT_EXPR, bitsize, bitsize_int (nunits - 1)); } @@ -10917,7 +10925,6 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, so use ->src. For main exit the merge block is the destination. */ basic_block dest = main_exit_edge ? main_e->dest : e->src; - gimple *tmp_vec_stmt = vec_stmt; tree tmp_vec_lhs = vec_lhs; tree tmp_bitstart = bitstart; @@ -10928,8 +10935,7 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, if (restart_loop && STMT_VINFO_DEF_TYPE (stmt_info) == vect_induction_def) { - tmp_vec_stmt = STMT_VINFO_VEC_STMTS (stmt_info)[0]; - tmp_vec_lhs = gimple_get_lhs (tmp_vec_stmt); + tmp_vec_lhs = vec_lhs0; tmp_bitstart = build_zero_cst (TREE_TYPE (bitstart)); }