diff mbox series

[2/4] aarch64: only skip PHI costs for loop vectorization

Message ID 20261001162056.68977-2-ktkachov@nvidia.com
State New
Delegated to: Tamar Christina
Headers
Series [1/4] cfgcleanup: keep tail merging out of an unfactored computed goto [PR127246] |

Commit Message

Kyrylo Tkachov Oct. 1, 2026, 4:20 p.m. UTC 
  From: Kyrylo Tkachov <ktkachov@nvidia.com>

aarch64_detect_vector_stmt_subtype returns zero for every cost recorded
against a PHI.  e4180ab2fea added this for loop inductions, whose scalar
code "generally needs the same induction but doesn't cost it".

Basic block SLP is different.  vect_bb_slp_scalar_cost costs the scalar
PHIs, while the zero removes the cost of the vector PHI and of the lane
extracts recorded against it.  Those extracts are real FMOV and UMOV
instructions, and the asymmetry makes a vector PHI look cheaper than the
scalar PHIs it replaces.

Keep the exemption for loop vectorization and cost PHIs normally for basic
block SLP.

Bootstrapped and regression tested on aarch64-unknown-linux-gnu and
x86_64-pc-linux-gnu.

Ok for trunk?
Thanks,
Kyrill

Cc: Tamar Christina <tamar.christina@arm.com>

gcc/ChangeLog:

	* config/aarch64/aarch64.cc (aarch64_detect_vector_stmt_subtype):
	Only skip costing PHIs for loop vectorization.

gcc/testsuite/ChangeLog:

	* gcc.target/aarch64/slp-phi-cost-1.c: New test.

Signed-off-by: Kyrylo Tkachov <ktkachov@nvidia.com>
---
 gcc/config/aarch64/aarch64.cc                 |  9 +++---
 .../gcc.target/aarch64/slp-phi-cost-1.c       | 31 +++++++++++++++++++
 2 files changed, 36 insertions(+), 4 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/aarch64/slp-phi-cost-1.c
diff mbox series

Patch

diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
index 107271b0404..8e0c45d7bd9 100644
--- a/gcc/config/aarch64/aarch64.cc
+++ b/gcc/config/aarch64/aarch64.cc
@@ -18288,10 +18288,11 @@  aarch64_detect_vector_stmt_subtype (vec_info *vinfo, vect_cost_for_stmt kind,
   if (aarch64_sve_mode_p (TYPE_MODE (vectype)))
     sve_costs = aarch64_tune_params.vec_costs->sve;
 
-  /* It's generally better to avoid costing inductions, since the induction
-     will usually be hidden by other operations.  This is particularly true
-     for things like COND_REDUCTIONS.  */
-  if (is_a<gphi *> (stmt_info->stmt))
+  /* It's generally better to avoid costing loop inductions, since the
+     induction will usually be hidden by other operations.  This is
+     particularly true for things like COND_REDUCTIONS.  For BB SLP, retain
+     the costs recorded for PHIs and their associated lane extracts.  */
+  if (is_a<gphi *> (stmt_info->stmt) && is_a<loop_vec_info> (vinfo))
     return 0;
 
   /* Detect cases in which vec_to_scalar is describing the extraction of a
diff --git a/gcc/testsuite/gcc.target/aarch64/slp-phi-cost-1.c b/gcc/testsuite/gcc.target/aarch64/slp-phi-cost-1.c
new file mode 100644
index 00000000000..18e3f3e89d9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/slp-phi-cost-1.c
@@ -0,0 +1,31 @@ 
+/* { dg-do compile } */
+/* { dg-require-effective-target lp64 } */
+/* { dg-options "-O2 -mtune=generic -fdump-tree-slp-details" } */
+
+/* Basic block SLP costs the scalar PHIs, so it must also cost the vector PHI
+   and the extracts needed for the scalar call arguments.  */
+
+extern unsigned long consume (unsigned long, unsigned long);
+
+unsigned long
+foo (unsigned long *out, const unsigned long *x, const unsigned long *y,
+     int cond)
+{
+  unsigned long a, b;
+  if (cond)
+    {
+      a = x[0] + 1;
+      b = x[1] + 1;
+    }
+  else
+    {
+      a = y[0] ^ 2;
+      b = y[1] ^ 2;
+    }
+  out[0] = a;
+  out[1] = b;
+  return consume (a, b);
+}
+
+/* { dg-final { scan-tree-dump {PHI <[^\n]*> 1 times vector_stmt costs 1 in body} "slp2" } } */
+/* { dg-final { scan-tree-dump-times {PHI <[^\n]*> 1 times vec_to_scalar costs 2 in body} 2 "slp2" } } */