From patchwork Thu Oct 24 14:19:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 99491 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 088E2385841C for ; Thu, 24 Oct 2024 14:19:47 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by sourceware.org (Postfix) with ESMTPS id 62CC33858C56 for ; Thu, 24 Oct 2024 14:19:10 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 62CC33858C56 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 62CC33858C56 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729779552; cv=none; b=nUHNyhCtUCANzl5qIP/NlWTPG3+9xpJ1TKU0wZNdJgVKEKHx4aW32sLgZn5N0eA9tI6JbMlq1qAGz+ftKafNrNQjXmDmco4E/D+oJxoOCwmRKxug/revMEoZw/4OKI1DH7874CZa48tmQI7bpbzvMY6UaDorCTuMSMB90cZYuWM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729779552; c=relaxed/simple; bh=yz41KJRQLjemEEb2SM/0NGi5VZDBtk/c6W983GAEfuQ=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version:Message-Id; b=cir4uwK/TG6w6/L2FDWlwbzjI2yios3LqEipIFR4xeqW3s4KzDn2IekuVLPCYSh1P1gdg+n1+d/s3Oszv+J2na/Z+bq2u22rAr6gGxXkQr64k17mcgsUEZgc9row6dLek02SPnITf2hUWC4r31gFtCwZLelLIH+BnOcSiJDOIkI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 642BA21E43 for ; Thu, 24 Oct 2024 14:19:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1729779549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=r4nJ0ge/ag6uSAadWdQvqQt3tbOAYSA5LjZ+/kF7a7Y=; b=ItS2f/DJy8RDOprx6l23A4ItSIPjzPNLc+O13duevmiXJiXFe5vgUwZoJH7kCxNYA4h9aS ERDpPTCBjyIGhV/7rstvtYxBQRpTGGPzjeWnQQCwKJ9MrcHTemi+JBBA/DE7paa5fQlig8 eQVMfNDTvdxK3eM57YsMboo2Av4Ekto= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1729779549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=r4nJ0ge/ag6uSAadWdQvqQt3tbOAYSA5LjZ+/kF7a7Y=; b=tr2piD+ngJM7eVGJYw3nMNqVUcj+bZBnpSAJAMZD+IFr5pz/tSjbxXvW5T/1IrFCtqmBaA LSuXA861SbXJcQCA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1729779549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=r4nJ0ge/ag6uSAadWdQvqQt3tbOAYSA5LjZ+/kF7a7Y=; b=ItS2f/DJy8RDOprx6l23A4ItSIPjzPNLc+O13duevmiXJiXFe5vgUwZoJH7kCxNYA4h9aS ERDpPTCBjyIGhV/7rstvtYxBQRpTGGPzjeWnQQCwKJ9MrcHTemi+JBBA/DE7paa5fQlig8 eQVMfNDTvdxK3eM57YsMboo2Av4Ekto= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1729779549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=r4nJ0ge/ag6uSAadWdQvqQt3tbOAYSA5LjZ+/kF7a7Y=; b=tr2piD+ngJM7eVGJYw3nMNqVUcj+bZBnpSAJAMZD+IFr5pz/tSjbxXvW5T/1IrFCtqmBaA LSuXA861SbXJcQCA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4AE39136F5 for ; Thu, 24 Oct 2024 14:19:09 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 6KiGEF1XGmdwLQAAD6G6ig (envelope-from ) for ; Thu, 24 Oct 2024 14:19:09 +0000 Date: Thu, 24 Oct 2024 16:19:08 +0200 (CEST) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH] tree-optimization/117277 - remove CLOBBERs before SLP code generation MIME-Version: 1.0 Message-Id: <20241024141909.4AE39136F5@imap1.dmz-prg2.suse.org> X-Spam-Score: -4.30 X-Spamd-Result: default: False [-4.30 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; MISSING_XM_UA(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; PREVIOUSLY_DELIVERED(0.00)[gcc-patches@gcc.gnu.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:mid, imap1.dmz-prg2.suse.org:helo] X-Spam-Level: X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org We have to remove CLOBBERs before SLP is code generated since for store-lanes we are inserting our own CLOBBERs that we want to survive. So the following refactors vect_transform_loop to remove unwanted stmts first. This resolves the gcc.target/aarch64/sve/store_lane_spill_1.c FAIL. Bootstrap and regtest running on x86_64-unknown-linux-gnu. PR tree-optimization/117277 * tree-vect-loop.cc (vect_transform_loop): Remove CLOBBERs and prefetches before doing any code generation. --- gcc/tree-vect-loop.cc | 110 ++++++++++++++++++++++-------------------- 1 file changed, 59 insertions(+), 51 deletions(-) diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 50a1531f4c3..322e50882ba 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -12515,6 +12515,25 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) move_early_exit_stmts (loop_vinfo); + /* Remove existing clobber stmts and prefetches. */ + for (i = 0; i < nbbs; i++) + { + basic_block bb = bbs[i]; + for (gimple_stmt_iterator si = gsi_start_bb (bb); !gsi_end_p (si);) + { + stmt = gsi_stmt (si); + if (gimple_clobber_p (stmt) + || gimple_call_builtin_p (stmt, BUILT_IN_PREFETCH)) + { + unlink_stmt_vdef (stmt); + gsi_remove (&si, true); + release_defs (stmt); + } + else + gsi_next (&si); + } + } + /* Schedule the SLP instances first, then handle loop vectorization below. */ if (!loop_vinfo->slp_instances.is_empty ()) @@ -12609,66 +12628,55 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) !gsi_end_p (si);) { stmt = gsi_stmt (si); - /* During vectorization remove existing clobber stmts and - prefetches. */ - if (gimple_clobber_p (stmt) - || gimple_call_builtin_p (stmt, BUILT_IN_PREFETCH)) - { - unlink_stmt_vdef (stmt); - gsi_remove (&si, true); - release_defs (stmt); - } - else + + /* Ignore vector stmts created in the outer loop. */ + stmt_info = loop_vinfo->lookup_stmt (stmt); + + /* vector stmts created in the outer-loop during vectorization of + stmts in an inner-loop may not have a stmt_info, and do not + need to be vectorized. */ + stmt_vec_info seen_store = NULL; + if (stmt_info) { - /* Ignore vector stmts created in the outer loop. */ - stmt_info = loop_vinfo->lookup_stmt (stmt); - - /* vector stmts created in the outer-loop during vectorization of - stmts in an inner-loop may not have a stmt_info, and do not - need to be vectorized. */ - stmt_vec_info seen_store = NULL; - if (stmt_info) + if (STMT_VINFO_IN_PATTERN_P (stmt_info)) { - if (STMT_VINFO_IN_PATTERN_P (stmt_info)) + gimple *def_seq = STMT_VINFO_PATTERN_DEF_SEQ (stmt_info); + for (gimple_stmt_iterator subsi = gsi_start (def_seq); + !gsi_end_p (subsi); gsi_next (&subsi)) { - gimple *def_seq = STMT_VINFO_PATTERN_DEF_SEQ (stmt_info); - for (gimple_stmt_iterator subsi = gsi_start (def_seq); - !gsi_end_p (subsi); gsi_next (&subsi)) - { - stmt_vec_info pat_stmt_info - = loop_vinfo->lookup_stmt (gsi_stmt (subsi)); - vect_transform_loop_stmt (loop_vinfo, pat_stmt_info, - &si, &seen_store); - } stmt_vec_info pat_stmt_info - = STMT_VINFO_RELATED_STMT (stmt_info); - if (vect_transform_loop_stmt (loop_vinfo, pat_stmt_info, - &si, &seen_store)) - maybe_set_vectorized_backedge_value (loop_vinfo, - pat_stmt_info); - } - else - { - if (vect_transform_loop_stmt (loop_vinfo, stmt_info, &si, - &seen_store)) - maybe_set_vectorized_backedge_value (loop_vinfo, - stmt_info); + = loop_vinfo->lookup_stmt (gsi_stmt (subsi)); + vect_transform_loop_stmt (loop_vinfo, pat_stmt_info, + &si, &seen_store); } + stmt_vec_info pat_stmt_info + = STMT_VINFO_RELATED_STMT (stmt_info); + if (vect_transform_loop_stmt (loop_vinfo, pat_stmt_info, + &si, &seen_store)) + maybe_set_vectorized_backedge_value (loop_vinfo, + pat_stmt_info); } - gsi_next (&si); - if (seen_store) + else { - if (STMT_VINFO_GROUPED_ACCESS (seen_store)) - /* Interleaving. If IS_STORE is TRUE, the - vectorization of the interleaving chain was - completed - free all the stores in the chain. */ - vect_remove_stores (loop_vinfo, - DR_GROUP_FIRST_ELEMENT (seen_store)); - else - /* Free the attached stmt_vec_info and remove the stmt. */ - loop_vinfo->remove_stmt (stmt_info); + if (vect_transform_loop_stmt (loop_vinfo, stmt_info, &si, + &seen_store)) + maybe_set_vectorized_backedge_value (loop_vinfo, + stmt_info); } } + gsi_next (&si); + if (seen_store) + { + if (STMT_VINFO_GROUPED_ACCESS (seen_store)) + /* Interleaving. If IS_STORE is TRUE, the + vectorization of the interleaving chain was + completed - free all the stores in the chain. */ + vect_remove_stores (loop_vinfo, + DR_GROUP_FIRST_ELEMENT (seen_store)); + else + /* Free the attached stmt_vec_info and remove the stmt. */ + loop_vinfo->remove_stmt (stmt_info); + } } /* Stub out scalar statements that must not survive vectorization.