From patchwork Tue Oct 26 08:58:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 46651 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 347C33858424 for ; Tue, 26 Oct 2021 08:59:23 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 347C33858424 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1635238763; bh=K2r/lx2ceWQGr+a7PnM/J+hlaoaAu2jKb6wnuwZXYlQ=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=AmTemBx1N5i7R9MdmnG6wXv3EaCGL6+xEFPKLxl8v2v9uP7HYEXD7ntk0YUs4wAb6 DgcezjyoNf9sQ9+1HTANNAt/Jh3/wD/XY06wsEmc1UyvaRb8dTb6ZhoISykWpyXppn E97oY8tZzwpE+OyOJ4UhsPRFwMXDfHyVvp5NSk5E= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id 8C0A63858D3C for ; Tue, 26 Oct 2021 08:58:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8C0A63858D3C Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5DF4F218A8 for ; Tue, 26 Oct 2021 08:58:51 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4A29F13B08 for ; Tue, 26 Oct 2021 08:58:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id UuPUEEvDd2G5ZAAAMHmgww (envelope-from ) for ; Tue, 26 Oct 2021 08:58:51 +0000 Date: Tue, 26 Oct 2021 10:58:50 +0200 (CEST) To: gcc-patches@gcc.gnu.org Subject: [PATCH] Turn vect_create_addr_base_for_vector_ref offset into a byte offset Message-ID: <5350n593-2893-84q-97p-qo6132qrq6q@fhfr.qr> MIME-Version: 1.0 X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Biener via Gcc-patches From: Richard Biener Reply-To: Richard Biener Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" This changes the offset in elements for vect_create_addr_base_for_vector_ref and vect_create_data_ref_ptr to an offset in bytes, easing a following refactoring. Bootstrapped and tested on x86_64-unknown-linux-gnu, pushed. 2021-10-26 Richard Biener * tree-vect-data-refs.c (vect_create_addr_base_for_vector_ref): Take offset in bytes. (vect_create_data_ref_ptr): Likewise. * tree-vect-loop-manip.c (get_misalign_in_elems): Multiply offset by element size. (vect_create_cond_for_align_checks): Likewise. * tree-vect-stmts.c (get_negative_load_store_type): Likewise. (vectorizable_load): Remove duplicate leftover from merge conflict. --- gcc/tree-vect-data-refs.c | 15 ++++++--------- gcc/tree-vect-loop-manip.c | 8 ++++++-- gcc/tree-vect-stmts.c | 11 +++++++---- 3 files changed, 19 insertions(+), 15 deletions(-) diff --git a/gcc/tree-vect-data-refs.c b/gcc/tree-vect-data-refs.c index cbcd4b80246..46360c50bd4 100644 --- a/gcc/tree-vect-data-refs.c +++ b/gcc/tree-vect-data-refs.c @@ -4753,8 +4753,7 @@ vect_duplicate_ssa_name_ptr_info (tree name, dr_vec_info *dr_info) if LOOP=i_loop: &in (relative to i_loop) if LOOP=j_loop: &in+i*2B (relative to j_loop) BYTE_OFFSET: Optional, defaulted to NULL. If supplied, it is added to the - initial address. Unlike OFFSET, which is number of elements to - be added, BYTE_OFFSET is measured in bytes. + initial address. Both OFFSET and BYTE_OFFSET are measured in bytes. Output: 1. Return an SSA_NAME whose value is the address of the memory location of @@ -4777,7 +4776,6 @@ vect_create_addr_base_for_vector_ref (vec_info *vinfo, stmt_vec_info stmt_info, tree dest; gimple_seq seq = NULL; tree vect_ptr_type; - tree step = TYPE_SIZE_UNIT (TREE_TYPE (DR_REF (dr))); loop_vec_info loop_vinfo = dyn_cast (vinfo); innermost_loop_behavior *drb = vect_dr_behavior (vinfo, dr_info); @@ -4801,8 +4799,7 @@ vect_create_addr_base_for_vector_ref (vec_info *vinfo, stmt_vec_info stmt_info, if (offset) { - offset = fold_build2 (MULT_EXPR, sizetype, - fold_convert (sizetype, offset), step); + offset = fold_convert (sizetype, offset); base_offset = fold_build2 (PLUS_EXPR, sizetype, base_offset, offset); } @@ -4860,8 +4857,8 @@ vect_create_addr_base_for_vector_ref (vec_info *vinfo, stmt_vec_info stmt_info, 2. AGGR_TYPE: the type of the reference, which should be either a vector or an array. 3. AT_LOOP: the loop where the vector memref is to be created. - 4. OFFSET (optional): an offset to be added to the initial address accessed - by the data-ref in STMT_INFO. + 4. OFFSET (optional): a byte offset to be added to the initial address + accessed by the data-ref in STMT_INFO. 5. BSI: location where the new stmts are to be placed if there is no loop 6. ONLY_INIT: indicate if ap is to be updated in the loop, or remain pointing to the initial address. @@ -4885,7 +4882,7 @@ vect_create_addr_base_for_vector_ref (vec_info *vinfo, stmt_vec_info stmt_info, if OFFSET is not supplied: initial_address = &a[init]; if OFFSET is supplied: - initial_address = &a[init + OFFSET]; + initial_address = &a[init] + OFFSET; if BYTE_OFFSET is supplied: initial_address = &a[init] + BYTE_OFFSET; @@ -5031,7 +5028,7 @@ vect_create_data_ref_ptr (vec_info *vinfo, stmt_vec_info stmt_info, /* (2) Calculate the initial address of the aggregate-pointer, and set the aggregate-pointer to point to it before the loop. */ - /* Create: (&(base[init_val+offset]+byte_offset) in the loop preheader. */ + /* Create: (&(base[init_val]+offset+byte_offset) in the loop preheader. */ new_temp = vect_create_addr_base_for_vector_ref (vinfo, stmt_info, &new_stmt_list, diff --git a/gcc/tree-vect-loop-manip.c b/gcc/tree-vect-loop-manip.c index 378b1026baa..72d583189c4 100644 --- a/gcc/tree-vect-loop-manip.c +++ b/gcc/tree-vect-loop-manip.c @@ -1625,7 +1625,9 @@ get_misalign_in_elems (gimple **seq, loop_vec_info loop_vinfo) bool negative = tree_int_cst_compare (DR_STEP (dr_info->dr), size_zero_node) < 0; tree offset = (negative - ? size_int (-TYPE_VECTOR_SUBPARTS (vectype) + 1) + ? size_int ((-TYPE_VECTOR_SUBPARTS (vectype) + 1) + * TREE_INT_CST_LOW + (TYPE_SIZE_UNIT (TREE_TYPE (vectype)))) : size_zero_node); tree start_addr = vect_create_addr_base_for_vector_ref (loop_vinfo, stmt_info, seq, @@ -3227,7 +3229,9 @@ vect_create_cond_for_align_checks (loop_vec_info loop_vinfo, bool negative = tree_int_cst_compare (DR_STEP (STMT_VINFO_DATA_REF (stmt_info)), size_zero_node) < 0; tree offset = negative - ? size_int (-TYPE_VECTOR_SUBPARTS (vectype) + 1) : size_zero_node; + ? size_int ((-TYPE_VECTOR_SUBPARTS (vectype) + 1) + * TREE_INT_CST_LOW (TYPE_SIZE_UNIT (TREE_TYPE (vectype)))) + : size_zero_node; /* create: addr_tmp = (int)(address_of_first_vector) */ addr_base = diff --git a/gcc/tree-vect-stmts.c b/gcc/tree-vect-stmts.c index c28c9370655..1f56e10709e 100644 --- a/gcc/tree-vect-stmts.c +++ b/gcc/tree-vect-stmts.c @@ -1996,6 +1996,11 @@ get_negative_load_store_type (vec_info *vinfo, return VMAT_ELEMENTWISE; } + /* For backward running DRs the first access in vectype actually is + N-1 elements before the address of the DR. */ + *poffset = ((-TYPE_VECTOR_SUBPARTS (vectype) + 1) + * TREE_INT_CST_LOW (TYPE_SIZE_UNIT (TREE_TYPE (vectype)))); + int misalignment = dr_misalignment (dr_info, vectype); alignment_support_scheme = vect_supportable_dr_alignment (vinfo, dr_info, vectype, misalignment); @@ -2006,6 +2011,7 @@ get_negative_load_store_type (vec_info *vinfo, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "negative step but alignment required.\n"); return VMAT_ELEMENTWISE; + *poffset = 0; } if (vls_type == VLS_STORE_INVARIANT) @@ -2014,7 +2020,6 @@ get_negative_load_store_type (vec_info *vinfo, dump_printf_loc (MSG_NOTE, vect_location, "negative step with invariant source;" " no permute needed.\n"); - *poffset = -TYPE_VECTOR_SUBPARTS (vectype) + 1; return VMAT_CONTIGUOUS_DOWN; } @@ -2023,10 +2028,10 @@ get_negative_load_store_type (vec_info *vinfo, if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "negative step and reversing not supported.\n"); + *poffset = 0; return VMAT_ELEMENTWISE; } - *poffset = -TYPE_VECTOR_SUBPARTS (vectype) + 1; return VMAT_CONTIGUOUS_REVERSE; } @@ -9315,8 +9320,6 @@ vectorizable_load (vec_info *vinfo, tree offset = NULL_TREE; if (!known_eq (poffset, 0)) offset = size_int (poffset); - if (memory_access_type == VMAT_CONTIGUOUS_REVERSE) - offset = size_int (-TYPE_VECTOR_SUBPARTS (vectype) + 1); tree bump; tree vec_offset = NULL_TREE;