From patchwork Wed May 31 15:08:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 70392 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id F1C8D385772F for ; Wed, 31 May 2023 15:08:52 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbg153.qq.com (smtpbg153.qq.com [13.245.218.24]) by sourceware.org (Postfix) with ESMTPS id E06FD3858D20 for ; Wed, 31 May 2023 15:08:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E06FD3858D20 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp90t1685545702t5k2dpdq Received: from rios-cad5.localdomain ( [58.60.1.11]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 31 May 2023 23:08:20 +0800 (CST) X-QQ-SSF: 01400000000000F0R000000A0000000 X-QQ-FEAT: rZJGTgY0+YPvILj9saQ5Ga2rMcoZbpWli2lz7+OLtjRd3YJ3BktWWMhqGG9Cc YYxB6lIwBKEIJSIN1mGGxODcrn/UH9vG2RFqpOPnKibmzndxX41A4x5WuF19iuGF1IxvA3a 8W+QCEUDYlcjitjrdiTbuM68788zOCOvbUvmF/2h00v9VdQm4bC79F+rGYG52S1uLiTRBmg U5vKBbWnO+OkNncoAxpCsHlszvIJrRnHxPeK5KzpakQ3cCKyfAPbwpapaNvq0lbpr0TAla9 FSsqWMp0kBXxY+xl0mC7boA3tuunfu3JH/rKpxz8YK3LiWz/gzQynxf2qlq4ii9/JVAjJgG 2RehOGcheOnv09/ONCpGJ/0ewoRtjfYaYaJQJGVzJ1sCq6drBynCH2PizCPJH19yauPaye1 oBEAbRarTpK818J87Ll0Ug== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 11942505282679942358 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: richard.sandiford@arm.com, rguenther@suse.de, linkw@linux.ibm.com, Ju-Zhe Zhong Subject: [PATCH V2] VECT: Change flow of decrement IV Date: Wed, 31 May 2023 23:08:19 +0800 Message-Id: <20230531150819.136379-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" From: Ju-Zhe Zhong Follow Richi's suggestion, I change current decrement IV flow from: do { remain -= MIN (vf, remain); } while (remain != 0); into: do { old_remain = remain; len = MIN (vf, remain); remain -= vf; } while (old_remain >= vf); to enhance SCEV. Include fixes from kewen. This patch will need to wait for Kewen's test feedback. Testing on X86 is on-going Co-Authored by: Kewen Lin gcc/ChangeLog: * tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow. (vect_set_loop_condition_partial_vectors): Ditto. --- gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index acf3642ceb2..3f735945e67 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, gimple_stmt_iterator loop_cond_gsi, rgroup_controls *rgc, tree niters, tree niters_skip, bool might_wrap_p, - tree *iv_step) + tree *iv_step, tree *compare_step) { tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo); tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo); @@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, ... vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0); ... - ivtmp_35 = ivtmp_9 - _36; + ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4]; ... - if (ivtmp_35 != 0) + if (ivtmp_9 > POLY_INT_CST [4, 4]) goto ; [83.33%] else goto ; [16.67%] @@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, tree step = rgc->controls.length () == 1 ? rgc->controls[0] : make_ssa_name (iv_type); /* Create decrement IV. */ - create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi, - insert_after, &index_before_incr, &index_after_incr); + create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop, + &incr_gsi, insert_after, &index_before_incr, + &index_after_incr); gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR, index_before_incr, nitems_step)); *iv_step = step; - return index_after_incr; + *compare_step = nitems_step; + return index_before_incr; } /* Create increment IV. */ @@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop, arbitrarily pick the last. */ tree test_ctrl = NULL_TREE; tree iv_step = NULL_TREE; + tree compare_step = NULL_TREE; rgroup_controls *rgc; rgroup_controls *iv_rgc = nullptr; unsigned int i; @@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop, &preheader_seq, &header_seq, loop_cond_gsi, rgc, niters, niters_skip, might_wrap_p, - &iv_step); + &iv_step, &compare_step); iv_rgc = rgc; } @@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop, /* Get a boolean result that tells us whether to iterate. */ edge exit_edge = single_exit (loop); - tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR; - tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl)); - gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl, - NULL_TREE, NULL_TREE); + gcond *cond_stmt; + if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo)) + { + gcc_assert (compare_step); + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR; + cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE, + NULL_TREE); + } + else + { + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR; + tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl)); + cond_stmt + = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE); + } gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT); /* The loop iterates (NITERS - 1) / VF + 1 times.