From patchwork Fri Mar 8 00:04:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Li, Pan2" X-Patchwork-Id: 86947 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 609DD3858D35 for ; Fri, 8 Mar 2024 00:04:40 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by sourceware.org (Postfix) with ESMTPS id 73A6F3858D35 for ; Fri, 8 Mar 2024 00:04:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 73A6F3858D35 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=intel.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 73A6F3858D35 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=198.175.65.16 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709856252; cv=none; b=cABKGm7FOLygeMW/a/qCO1ToexJeHvfVvjbfNOeTGrsubuX8LMYBlxQ7kd2dkmcrSU1taQNEtKLwLMK3bZEUJyQic9hbg2jP3Jup9WygHE06QLO4+gfjHam7dDvfRtD664ncHzgm9SCOncNJN4cEgbcGcdPTXeyJ2Jy4OxVWI7E= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709856252; c=relaxed/simple; bh=sdIk8eQMvfWs4Oz0gjt8yxmKiZNflyj/FvS8cfKA6Ow=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=ph15rmKWl6Gi+HexPf2C23eN+CPHYtaeTuHlrtYAn2BWHakjaiQfiNE/M2xFi9mEsHV/mpwve4qBFlZttq51KY68WL7watLL2Z5oUstcclq3gwoTxn9ueiOeDal/o284Aor5RWGAlWS8lxXpNrDi2tNMcSqG8OsaxTCk19HYtAc= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1709856249; x=1741392249; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=sdIk8eQMvfWs4Oz0gjt8yxmKiZNflyj/FvS8cfKA6Ow=; b=MsCsMnGNo4S7OIcXuQ4JR4aT33dxKfXB4JcnuJvNvx8F6YpgfQ4jXNCz XqVSHX8vZn0ca8VpyFEY+vHT6bBql/jwgBLZx3+gfaOV0Mu/0ABjuNfs5 gRP0MWdzl5RjsDWwu47EwtLkv1EcO50bJDQHrZ5GpvDDnzddJS808sBm3 UkBZGPWJjAMO2qoxAoNGBzkn+9wUnz+WEtQYMuCHFkgOtiNKsiMp/5JHB St30BtagnPk9LYd5woTqMkoK5TLZgxB5kQ7l/HSsXlWqXaPEfb+tsRe3r OYgp9aKn9CfPAwUjgN5DFtaN8RAJ+w+1w5xKp8H7PhR0D13YAPTG4k1hF Q==; X-IronPort-AV: E=McAfee;i="6600,9927,11006"; a="4685007" X-IronPort-AV: E=Sophos;i="6.07,107,1708416000"; d="scan'208";a="4685007" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Mar 2024 16:04:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,107,1708416000"; d="scan'208";a="10838503" Received: from shvmail03.sh.intel.com ([10.239.245.20]) by orviesa008.jf.intel.com with ESMTP; 07 Mar 2024 16:04:04 -0800 Received: from pli-ubuntu.sh.intel.com (pli-ubuntu.sh.intel.com [10.239.159.47]) by shvmail03.sh.intel.com (Postfix) with ESMTP id A9143100568F; Fri, 8 Mar 2024 08:04:03 +0800 (CST) From: pan2.li@intel.com To: gcc-patches@gcc.gnu.org Cc: juzhe.zhong@rivai.ai, kito.cheng@gmail.com, yanzhang.wang@intel.com, rdapp.gcc@gmail.com, richard.guenther@gmail.com, jeffreyalaw@gmail.com, Pan Li Subject: [PATCH v1] VECT: Bugfix ICE for vectorizable_store when both len and mask Date: Fri, 8 Mar 2024 08:04:01 +0800 Message-Id: <20240308000401.2766685-1-pan2.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, WEIRD_PORT autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org From: Pan Li This patch would like to fix one ICE in vectorizable_store for both the loop_masks and loop_lens. The ICE looks like below with "-march=rv64gcv -O3". during GIMPLE pass: vect test.c: In function ‘d’: test.c:6:6: internal compiler error: in vectorizable_store, at tree-vect-stmts.cc:8691 6 | void d() { | ^ 0x37a6f2f vectorizable_store .../__RISC-V_BUILD__/../gcc/tree-vect-stmts.cc:8691 0x37b861c vect_analyze_stmt(vec_info*, _stmt_vec_info*, bool*, _slp_tree*, _slp_instance*, vec*) .../__RISC-V_BUILD__/../gcc/tree-vect-stmts.cc:13242 0x1db5dca vect_analyze_loop_operations .../__RISC-V_BUILD__/../gcc/tree-vect-loop.cc:2208 0x1db885b vect_analyze_loop_2 .../__RISC-V_BUILD__/../gcc/tree-vect-loop.cc:3041 0x1dba029 vect_analyze_loop_1 .../__RISC-V_BUILD__/../gcc/tree-vect-loop.cc:3481 0x1dbabad vect_analyze_loop(loop*, vec_info_shared*) .../__RISC-V_BUILD__/../gcc/tree-vect-loop.cc:3639 0x1e389d1 try_vectorize_loop_1 .../__RISC-V_BUILD__/../gcc/tree-vectorizer.cc:1066 0x1e38f3d try_vectorize_loop .../__RISC-V_BUILD__/../gcc/tree-vectorizer.cc:1182 0x1e39230 execute .../__RISC-V_BUILD__/../gcc/tree-vectorizer.cc:1298 Given the masks and the lens cannot be enabled simultanously when loop is using partial vectors. Thus, we need to ensure the one is disabled when we would like to record the other in check_load_store_for_partial_vectors. For example, when we try to record loop len, we need to check if the loop mask is disabled or not. Below testsuites are passed for this patch: * The x86 bootstrap tests. * The x86 fully regression tests. * The aarch64 fully regression tests. * The riscv fully regressison tests. PR target/114195 gcc/ChangeLog: * tree-vect-stmts.cc (check_load_store_for_partial_vectors): Add loop mask/len check before recording as they are mutual exclusion. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/base/pr114195-1.c: New test. Signed-off-by: Pan Li --- .../gcc.target/riscv/rvv/base/pr114195-1.c | 15 +++++++++++ gcc/tree-vect-stmts.cc | 26 ++++++++++++++----- 2 files changed, 35 insertions(+), 6 deletions(-) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/base/pr114195-1.c diff --git a/gcc/testsuite/gcc.target/riscv/rvv/base/pr114195-1.c b/gcc/testsuite/gcc.target/riscv/rvv/base/pr114195-1.c new file mode 100644 index 00000000000..b0c9d5b81b8 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/base/pr114195-1.c @@ -0,0 +1,15 @@ +/* Test that we do not have ice when compile */ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize" } */ + +long a, b; +extern short c[]; + +void d() { + for (int e = 0; e < 35; e += 2) { + a = ({ a < 0 ? a : 0; }); + b = ({ b < 0 ? b : 0; }); + + c[e] = 0; + } +} diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 14a3ffb5f02..624947ed271 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -1502,6 +1502,8 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, gather_scatter_info *gs_info, tree scalar_mask) { + gcc_assert (LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)); + /* Invariant loads need no special support. */ if (memory_access_type == VMAT_INVARIANT) return; @@ -1521,9 +1523,17 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, internal_fn ifn = (is_load ? vect_load_lanes_supported (vectype, group_size, true) : vect_store_lanes_supported (vectype, group_size, true)); - if (ifn == IFN_MASK_LEN_LOAD_LANES || ifn == IFN_MASK_LEN_STORE_LANES) + + /* When the loop_vinfo using partial vector, we cannot enable both + the fully mask and length simultaneously. Thus, make sure the + other one is disabled when record one of them. + The same as other place for both the vect_record_loop_len and + vect_record_loop_mask. */ + if ((ifn == IFN_MASK_LEN_LOAD_LANES || ifn == IFN_MASK_LEN_STORE_LANES) + && !LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) vect_record_loop_len (loop_vinfo, lens, nvectors, vectype, 1); - else if (ifn == IFN_MASK_LOAD_LANES || ifn == IFN_MASK_STORE_LANES) + else if ((ifn == IFN_MASK_LOAD_LANES || ifn == IFN_MASK_STORE_LANES) + && !LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) vect_record_loop_mask (loop_vinfo, masks, nvectors, vectype, scalar_mask); else @@ -1549,12 +1559,14 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, if (internal_gather_scatter_fn_supported_p (len_ifn, vectype, gs_info->memory_type, gs_info->offset_vectype, - gs_info->scale)) + gs_info->scale) + && !LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) vect_record_loop_len (loop_vinfo, lens, nvectors, vectype, 1); else if (internal_gather_scatter_fn_supported_p (ifn, vectype, gs_info->memory_type, gs_info->offset_vectype, - gs_info->scale)) + gs_info->scale) + && !LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) vect_record_loop_mask (loop_vinfo, masks, nvectors, vectype, scalar_mask); else @@ -1608,7 +1620,8 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, machine_mode mask_mode; machine_mode vmode; bool using_partial_vectors_p = false; - if (get_len_load_store_mode (vecmode, is_load).exists (&vmode)) + if (get_len_load_store_mode (vecmode, is_load).exists (&vmode) + && !LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) { nvectors = group_memory_nvectors (group_size * vf, nunits); unsigned factor = (vecmode == vmode) ? 1 : GET_MODE_UNIT_SIZE (vecmode); @@ -1616,7 +1629,8 @@ check_load_store_for_partial_vectors (loop_vec_info loop_vinfo, tree vectype, using_partial_vectors_p = true; } else if (targetm.vectorize.get_mask_mode (vecmode).exists (&mask_mode) - && can_vec_mask_load_store_p (vecmode, mask_mode, is_load)) + && can_vec_mask_load_store_p (vecmode, mask_mode, is_load) + && !LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) { nvectors = group_memory_nvectors (group_size * vf, nunits); vect_record_loop_mask (loop_vinfo, masks, nvectors, vectype, scalar_mask);