From patchwork Mon Feb 28 11:29:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 51440 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C734A385843A for ; Mon, 28 Feb 2022 11:30:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C734A385843A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1646047807; bh=HXi+Dnivd5co1PtT/wAn09bk4K90sXGpmvVo42jJ2M4=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=UXCpoy6X87krUQsCduhdVXq0kS84fyNrtFCAeHlima8bAtz3TBA8ZS0uRTCRRCW8l CXfY+b7IPfPSFZq+Xi2Xo6TSijlJgKgtOeU4FOI9dLG/G2EnT8n9bulPE3u00fhbJc 8zxZ6pdDp1XtZEyZ7Wzjmsh2B9tzWQX4BXxqrjyY= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60083.outbound.protection.outlook.com [40.107.6.83]) by sourceware.org (Postfix) with ESMTPS id B33443858C60 for ; Mon, 28 Feb 2022 11:29:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org B33443858C60 Received: from AM5PR1001CA0019.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:2::32) by DB6PR0802MB2405.eurprd08.prod.outlook.com (2603:10a6:4:9f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.26; Mon, 28 Feb 2022 11:29:28 +0000 Received: from VE1EUR03FT040.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:2:cafe::33) by AM5PR1001CA0019.outlook.office365.com (2603:10a6:206:2::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.23 via Frontend Transport; Mon, 28 Feb 2022 11:29:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT040.mail.protection.outlook.com (10.152.18.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22 via Frontend Transport; Mon, 28 Feb 2022 11:29:28 +0000 Received: ("Tessian outbound 63bb5eb69ee8:v113"); Mon, 28 Feb 2022 11:29:28 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: a540e7ef0bc5cd18 X-CR-MTA-TID: 64aa7808 Received: from 85f8993be9b7.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 4BE5B2AC-8861-438C-88D6-BF299C27AD19.1; Mon, 28 Feb 2022 11:29:20 +0000 Received: from EUR01-DB5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 85f8993be9b7.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 28 Feb 2022 11:29:20 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GOABe2aqdTEvARy60ZWBjMRMnx7MjSvqB/Et4ryTsPmWprZj4Xe4AB73P7YQg0FjicHSQQnYBtA/W4TqF8y9Olor7mGL1g7fU4cTds597DEpYsBcz+zKwHO7dC4AgZnAO7YqQL739ZwsWT3osJgeR2ZgQOl6DxZws3QxEbVuqLFtHDNVRqJUx6q/XKHaT9f8k01FVyxqmOfoSSOluJCILdd0vs128q3eiDOADMAW4k9T1X37wvzWXAo/5BWF6FDNINtv2sqovwSfku41EPFK9a3xFAFUETAGy85yz7cox4Mc6idlOomviFdxZaBvroqSytim5YoVt9rVgDM4Ej4GcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HXi+Dnivd5co1PtT/wAn09bk4K90sXGpmvVo42jJ2M4=; b=IpzUy80Gk9EKvaMtFHToEI5N/alRd8hCEqmGvTGyfxyCxbjZwNOOyTczmzfjhirwz3cVzPtd0xQIXrf6YBnD67f3SUBlxNcZ9+ktfdUUKFd6RfkYFat9hhYscpOsieN9QbM7WjMRrbZQtpvE5YQdMLag46ivE1SbWalxhGOoDoinJGEzJPVgIMntyqS2tLFSIbBsXRf4BjmA2A3GhyZ6jy9FAgoDbXYJYUKNHA8PeTZB7ztHpY/X0DiZl/l7O9/YFi3tk6eosbaaQsSzgkYpwIldt+qC8wWCZ5eFELbjQ6d1AK4tlwN1cGQ0CwemRM7OtQBA40xlZBCJIb8V+qUG4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by DB7PR08MB3260.eurprd08.prod.outlook.com (2603:10a6:5:21::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.26; Mon, 28 Feb 2022 11:29:18 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::a911:a778:877d:7bb2]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::a911:a778:877d:7bb2%9]) with mapi id 15.20.5017.026; Mon, 28 Feb 2022 11:29:18 +0000 Date: Mon, 28 Feb 2022 11:29:16 +0000 To: gcc-patches@gcc.gnu.org Subject: [PATCH 1/2]middle-end Handle difference between complex negations in SLP tree better (GCC 11 backport) Message-ID: Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: LO4P123CA0208.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1a5::15) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: d76e308a-326a-43ea-ae53-08d9faad94d5 X-MS-TrafficTypeDiagnostic: DB7PR08MB3260:EE_|VE1EUR03FT040:EE_|DB6PR0802MB2405:EE_ X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: ArkBNhaIELPSY4BTdaFhQFBijHqOHRiyZZ9SJmhPAb58Ul3tqGjJl4zyA9ia8v5pwdMxr+GYgFC/gKgpqEHR8ly/I/Eb5L7jd1K1bwH/ASBihDEgdqvFpx+ehg5+pED+5WEsfWH2Eu7tBYW+j9+oib64y6fKaupFWGkujHErULJR+MfnC4/e2+o4IOMCu2gr2fxKifT3h0pHmQi5oapFXH/rjenuXkwtN33GCr36BquaScmqxXd6SCwSR9kcdq63AAeXb8K0KIKrHnj/mmYZznPiZap34N3strRsowSI6NPONRS6xja4Tne49pYcatiddfuthlg9GzCc15mTJ2glbBzHVBofJuvgOiBXPWPdyWUEVPgOzkDZtmL9BZf6IN4LWuxFZYaihCNM9z4Ic2M0kLr+lBHiYnOVsDiOa/YcnOvNku0RhIFOixspubFKrsYpC+cvuMVCABCdEt4AaXwkGXuXKcyp9Aiwv7+J0Xgd86cozneIy3gdssocX+fH3Ncm8S9nPy5eP57wqywOH2UzuQUAS4wIEqEarjVgICFiVpqSiM/xoQv9jepCPfjA8oLtkAIxuTrdoTAwCXvFw+6z1kclFto/RfuaRp+j5R+F1psg6wEAbJGDK9wN7Qn7mL37hLArksRKM5xr1/COAOFHKcCwY5Ir9ZketPsRk/2k6ys/BYOfIb8j3ewjBIhLJiXnD9WfNGRM8hCiF1ZEQI8WOmUwMzuomZOpjE9CYfnC2kpccxECN6okdceDhDC+MoQiB8++c0ttt595cid38CHPUefNDy/71s+bDk0MZofaapc= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(2906002)(36756003)(44832011)(316002)(8936002)(235185007)(508600001)(26005)(186003)(2616005)(6486002)(52116002)(6512007)(44144004)(33964004)(6506007)(6916009)(38100700002)(38350700002)(86362001)(83380400001)(66556008)(66946007)(66476007)(4326008)(5660300002)(8676002)(4216001)(2700100001)(309714004); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3260 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT040.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: dee18ebd-b25b-4a0b-689e-08d9faad8ec4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q2mj/MCnsSYF1D1iiDGiXUmZDVNvbq4Ot8PYhlLnQ+a//veO1F2+XoycEN/cRx+q6Fa6JAVgGiBp0bb3i++hlHOi7k2ybnw1T2Q3iyqrjwnX0d2Aj3WGarOjMf1J+7fW4QiAB9mquTe63ouJVIDrg8blqFzy7CkLOY3TWp8zRFFzE//W0ZPI0MNN2RsnAb+J29xSqb3F2vubUeXrC3K1D7gMaa8b/TRlU7V6nZbGJFifoR3p/Mzrr3fizNaJWlNkD7MDajq4f4XL8ZRFlIfa+DbCw28/Qaj7sMS7G+e3V3B2dOqUO5UGavOvMHweOv/u1hx1HaZMZ7DyoPP+b09KfqoOvg2cD8/CfYq8rgcbw6W/FCRPsmzHQoZXFet7uRNs5PVKo5RIx5C0/dFg7/p/Bu90fVJ0/vetuWzzLs/kUKuApZmL01q8PmEDEV/4FcYWe2I2oL3JyxTBXv91J3CFfdG+n5532hX/+cwTVhR1gjM1dPM4aajGocXRn91ClI5d80WtgFeAeRT/SK3OeelmEM5A3UPw8BzcaL6M0yo3JsX9x057QXoORHUx7tkmoqyI/ngNXzTYy0Xt9IzG9l/e6SxF7631tFAmAUdm8BXU1i6xMS3E03siAMqc2z5gqhm5zgrZgtFKx6sE1DCPKZbfXdUyFpcIjzoKmK72SEPBjb8e3QpXrZcXeHTZwLJL16X42MSJshlzi4+1KILPGhuhcHxQc6ISQA1D8yfFE6EvW4gJHHcfLVppzBez2Ahs2pg2 X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(508600001)(235185007)(6486002)(82310400004)(2616005)(107886003)(6512007)(86362001)(2906002)(40460700003)(33964004)(44144004)(6506007)(44832011)(186003)(26005)(81166007)(336012)(47076005)(36860700001)(356005)(70206006)(83380400001)(8676002)(70586007)(6916009)(5660300002)(36756003)(8936002)(316002)(4326008)(4216001)(2700100001)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2022 11:29:28.1860 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d76e308a-326a-43ea-ae53-08d9faad94d5 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT040.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2405 X-Spam-Status: No, score=-13.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Cc: nd@arm.com, rguenther@suse.de Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Hi All, GCC 11 handled negations rather differently than GCC 12. This difference caused the previous backport to regress some of the conjugate cases that it used to handle before. The testsuite in GCC 11 wasn't as robust as that in master so it didn't catch it. The second patch in this series backports the testcases from master to GCC-11 to prevent this in the future. This patch deals with the conjugate cases correctly by updating the detection code to deal with the different order of operands. For MUL the problem is that the presence of an ADD can cause the order of the operands to flip, unlike in GCC 12. So to handle this if we detect the shape of a MUL but the data-flow check fails, we swap both operands and try again. Since a * b == b * a this is fine and allows us to keep the df-check simple. This doesn't cause a compile time issue either as most of the data will be in the caches from the previous call. Bootstrapped Regtested on aarch64-none-linux-gnu, x86_64-pc-linux-gnu and no regressions on updated testsuite. Ok for GCC 11? Thanks, Tamar gcc/ChangeLog: * tree-vect-slp-patterns.c (vect_validate_multiplication): Correctly detect conjugate cases. (complex_mul_pattern::matches): Likewise. (complex_fma_pattern::matches): Move accumulator last as expected. (complex_fma_pattern::build): Likewise. (complex_fms_pattern::matches): Handle different conjugate form. --- inline copy of patch -- diff --git a/gcc/tree-vect-slp-patterns.c b/gcc/tree-vect-slp-patterns.c index a3bd90ff85b4ca5423a94388d480b66051a83e08..8b08a0f33dd1cd23ad9577243524c1feaa5e8ed9 100644 diff --git a/gcc/tree-vect-slp-patterns.c b/gcc/tree-vect-slp-patterns.c index a3bd90ff85b4ca5423a94388d480b66051a83e08..8b08a0f33dd1cd23ad9577243524c1feaa5e8ed9 100644 --- a/gcc/tree-vect-slp-patterns.c +++ b/gcc/tree-vect-slp-patterns.c @@ -873,10 +873,8 @@ compatible_complex_nodes_p (slp_compat_nodes_map_t *compat_cache, static inline bool vect_validate_multiplication (slp_tree_to_load_perm_map_t *perm_cache, slp_compat_nodes_map_t *compat_cache, - vec &left_op, - vec &right_op, - bool subtract, - enum _conj_status *_status) + vec &left_op, vec &right_op, + bool subtract, enum _conj_status *_status) { auto_vec ops; enum _conj_status stats = CONJ_NONE; @@ -902,29 +900,31 @@ vect_validate_multiplication (slp_tree_to_load_perm_map_t *perm_cache, /* Default to style and perm 0, most operations use this one. */ int style = 0; - int perm = subtract ? 1 : 0; + int perm = 0; - /* Check if we have a negate operation, if so absorb the node and continue - looking. */ + /* Determine which style we're looking at. We only have different ones + whenever a conjugate is involved. If so absorb the node and continue. */ bool neg0 = vect_match_expression_p (right_op[0], NEGATE_EXPR); bool neg1 = vect_match_expression_p (right_op[1], NEGATE_EXPR); - /* Determine which style we're looking at. We only have different ones - whenever a conjugate is involved. */ - if (neg0 && neg1) - ; - else if (neg0) - { - right_op[0] = SLP_TREE_CHILDREN (right_op[0])[0]; - stats = CONJ_FST; - if (subtract) - perm = 0; - } - else if (neg1) + /* Determine which style we're looking at. We only have different ones + whenever a conjugate is involved. */ + if (neg0 != neg1 && (neg0 || neg1)) { - right_op[1] = SLP_TREE_CHILDREN (right_op[1])[0]; - stats = CONJ_SND; - perm = 1; + unsigned idx = !!neg1; + right_op[idx] = SLP_TREE_CHILDREN (right_op[idx])[0]; + if (linear_loads_p (perm_cache, left_op[!!!neg1]) == PERM_EVENEVEN) + { + stats = CONJ_FST; + style = 1; + if (subtract && neg0) + perm = 1; + } + else + { + stats = CONJ_SND; + perm = 1; + } } *_status = stats; @@ -1069,7 +1069,16 @@ complex_mul_pattern::matches (complex_operation_t op, enum _conj_status status; if (!vect_validate_multiplication (perm_cache, compat_cache, left_op, right_op, false, &status)) - return IFN_LAST; + { + /* Try swapping the operands and trying again. */ + std::swap (left_op[0], left_op[1]); + right_op.truncate (0); + right_op.safe_splice (SLP_TREE_CHILDREN (muls[1])); + std::swap (right_op[0], right_op[1]); + if (!vect_validate_multiplication (perm_cache, compat_cache, left_op, + right_op, false, &status)) + return IFN_LAST; + } if (status == CONJ_NONE) ifn = IFN_COMPLEX_MUL; @@ -1089,7 +1098,7 @@ complex_mul_pattern::matches (complex_operation_t op, ops->quick_push (right_op[1]); ops->quick_push (left_op[0]); } - else if (kind == PERM_EVENEVEN && status != CONJ_SND) + else if (kind == PERM_EVENEVEN && status == CONJ_NONE) { ops->quick_push (left_op[0]); ops->quick_push (right_op[0]); @@ -1246,15 +1255,15 @@ complex_fma_pattern::matches (complex_operation_t op, if (ifn == IFN_COMPLEX_FMA) { - ops->quick_push (SLP_TREE_CHILDREN (vnode)[0]); ops->quick_push (SLP_TREE_CHILDREN (node)[1]); ops->quick_push (SLP_TREE_CHILDREN (node)[0]); + ops->quick_push (SLP_TREE_CHILDREN (vnode)[0]); } else { - ops->quick_push (SLP_TREE_CHILDREN (vnode)[0]); ops->quick_push (SLP_TREE_CHILDREN (node)[0]); ops->quick_push (SLP_TREE_CHILDREN (node)[1]); + ops->quick_push (SLP_TREE_CHILDREN (vnode)[0]); } return ifn; @@ -1290,8 +1299,8 @@ complex_fma_pattern::build (vec_info *vinfo) SLP_TREE_CHILDREN (*this->m_node).create (3); SLP_TREE_CHILDREN (*this->m_node).safe_splice (this->m_ops); + SLP_TREE_REF_COUNT (this->m_ops[0])++; SLP_TREE_REF_COUNT (this->m_ops[1])++; - SLP_TREE_REF_COUNT (this->m_ops[2])++; vect_free_slp_tree (node); @@ -1397,24 +1406,32 @@ complex_fms_pattern::matches (complex_operation_t op, if (!vect_pattern_validate_optab (ifn, *ref_node)) return IFN_LAST; + child = SLP_TREE_CHILDREN ((*ops)[1])[0]; ops->truncate (0); ops->create (4); complex_perm_kinds_t kind = linear_loads_p (perm_cache, right_op[0]); - if (kind == PERM_EVENODD) + if (kind == PERM_EVENODD || kind == PERM_TOP) { ops->quick_push (child); ops->quick_push (right_op[0]); ops->quick_push (right_op[1]); - ops->quick_push (left_op[1]); + ops->quick_push (left_op[0]); } - else + else if (status == CONJ_NONE) { ops->quick_push (child); ops->quick_push (right_op[1]); ops->quick_push (right_op[0]); ops->quick_push (left_op[0]); } + else + { + ops->quick_push (child); + ops->quick_push (right_op[1]); + ops->quick_push (right_op[0]); + ops->quick_push (left_op[1]); + } return ifn; }