diff mbox series

match.pd: Fold SAD_EXPR with identical inputs

Message ID 20260707075715.5624-1-Pengfei.Li2@arm.com
State Committed
Commit 79c77e1642030361a17b3c892a00092d8285a3b2
Headers
Series match.pd: Fold SAD_EXPR with identical inputs |

Checks

Context Check Description
linaro-tcwg-bot/tcwg_gcc_build--master-arm success Build passed
linaro-tcwg-bot/tcwg_gcc_build--master-aarch64 success Build passed

Commit Message

Pengfei Li July 7, 2026, 7:57 a.m. UTC 
  Normally, SAD_EXPRs with identical first and second operands are not
constructed, since the absolute difference value is zero. However, this
can still appear when vectorizing small iteration-count reduction loops
with an absolute difference operation.

This patch adds a match.pd rule to fold the redundant SAD_EXPR to its
accumulator operand.

Bootstrapped and tested on aarch64-linux-gnu and x86_64-linux-gnu.

gcc/ChangeLog:

	* match.pd: Simplify SAD(x, x, acc) into acc.

gcc/testsuite/ChangeLog:

	* gcc.target/aarch64/sve/sad_2.c: New test.
---
 gcc/match.pd                                 |  7 +++++++
 gcc/testsuite/gcc.target/aarch64/sve/sad_2.c | 16 ++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/sad_2.c

Comments

Richard Biener July 7, 2026, 8:38 a.m. UTC | #1 
On Tue, Jul 7, 2026 at 10:00 AM Pengfei Li <Pengfei.Li2@arm.com> wrote:
>
> Normally, SAD_EXPRs with identical first and second operands are not
> constructed, since the absolute difference value is zero. However, this
> can still appear when vectorizing small iteration-count reduction loops
> with an absolute difference operation.
>
> This patch adds a match.pd rule to fold the redundant SAD_EXPR to its
> accumulator operand.
>
> Bootstrapped and tested on aarch64-linux-gnu and x86_64-linux-gnu.

The pattern looks good to me, but I don't see how the testcase exercises it?

> gcc/ChangeLog:
>
>         * match.pd: Simplify SAD(x, x, acc) into acc.
>
> gcc/testsuite/ChangeLog:
>
>         * gcc.target/aarch64/sve/sad_2.c: New test.
> ---
>  gcc/match.pd                                 |  7 +++++++
>  gcc/testsuite/gcc.target/aarch64/sve/sad_2.c | 16 ++++++++++++++++
>  2 files changed, 23 insertions(+)
>  create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
>
> diff --git a/gcc/match.pd b/gcc/match.pd
> index a7cec25dbad..4520a23a026 100644
> --- a/gcc/match.pd
> +++ b/gcc/match.pd
> @@ -1229,6 +1229,13 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
>   (mult (absu@1 @0) @1)
>   (mult (convert@2 @0) @2))
>
> +#if GIMPLE
> +/* Simplify SAD(x, x, acc) -> acc since the absolute difference is zero.  */
> +(simplify
> + (sad @0 @0 @1)
> + @1)
> +#endif
> +
>  /* cos(copysign(x, y)) -> cos(x).  Similarly for cosh.  */
>  (for coss (COS COSH)
>   (for copysigns (COPYSIGN)
> diff --git a/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c b/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
> new file mode 100644
> index 00000000000..d46517f10d8
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
> @@ -0,0 +1,16 @@
> +/* { dg-do compile } */
> +/* { dg-options "-O3 -msve-vector-bits=128 -mtune=neoverse-v2" } */
> +
> +#include <stdint.h>
> +
> +int foo (uint8_t *pix1, uint8_t *pix2)
> +{
> +    int sum = 0;
> +    for (int x = 0; x < 48; x++)
> +    {
> +        sum += __builtin_abs (pix1[x] - pix2[x]);
> +    }
> +    return sum;
> +}
> +
> +/* { dg-final { scan-assembler-times {\tudot\t} 3 } } */
> --
> 2.43.0
>
Pengfei Li July 7, 2026, 9:48 a.m. UTC | #2 
Hi Richard,

On 07/07/2026 09:38, Richard Biener wrote:
> On Tue, Jul 7, 2026 at 10:00 AM Pengfei Li <Pengfei.Li2@arm.com> wrote:
>> Normally, SAD_EXPRs with identical first and second operands are not
>> constructed, since the absolute difference value is zero. However, this
>> can still appear when vectorizing small iteration-count reduction loops
>> with an absolute difference operation.
>>
>> This patch adds a match.pd rule to fold the redundant SAD_EXPR to its
>> accumulator operand.
>>
>> Bootstrapped and tested on aarch64-linux-gnu and x86_64-linux-gnu.
> The pattern looks good to me, but I don't see how the testcase exercises it?

In this testcase, SAD_EXPR is expanded to AArch64 uabd + udot 
instructions. Without the pattern, the uabd is eliminated but SAD_EXPR 
still generates an extra udot with a zero vector, e.g.

     movi    d31, #0
     ...
     udot    z29.s, z31.b, z28.b

The 48-iteration loop needs 3 udot, instead of 4. That's what the 
testcase checks. Ok for trunk?

Thanks,
Pengfei

>
>> gcc/ChangeLog:
>>
>>          * match.pd: Simplify SAD(x, x, acc) into acc.
>>
>> gcc/testsuite/ChangeLog:
>>
>>          * gcc.target/aarch64/sve/sad_2.c: New test.
>> ---
>>   gcc/match.pd                                 |  7 +++++++
>>   gcc/testsuite/gcc.target/aarch64/sve/sad_2.c | 16 ++++++++++++++++
>>   2 files changed, 23 insertions(+)
>>   create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
>>
>> diff --git a/gcc/match.pd b/gcc/match.pd
>> index a7cec25dbad..4520a23a026 100644
>> --- a/gcc/match.pd
>> +++ b/gcc/match.pd
>> @@ -1229,6 +1229,13 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
>>    (mult (absu@1 @0) @1)
>>    (mult (convert@2 @0) @2))
>>
>> +#if GIMPLE
>> +/* Simplify SAD(x, x, acc) -> acc since the absolute difference is zero.  */
>> +(simplify
>> + (sad @0 @0 @1)
>> + @1)
>> +#endif
>> +
>>   /* cos(copysign(x, y)) -> cos(x).  Similarly for cosh.  */
>>   (for coss (COS COSH)
>>    (for copysigns (COPYSIGN)
>> diff --git a/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c b/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
>> new file mode 100644
>> index 00000000000..d46517f10d8
>> --- /dev/null
>> +++ b/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
>> @@ -0,0 +1,16 @@
>> +/* { dg-do compile } */
>> +/* { dg-options "-O3 -msve-vector-bits=128 -mtune=neoverse-v2" } */
>> +
>> +#include <stdint.h>
>> +
>> +int foo (uint8_t *pix1, uint8_t *pix2)
>> +{
>> +    int sum = 0;
>> +    for (int x = 0; x < 48; x++)
>> +    {
>> +        sum += __builtin_abs (pix1[x] - pix2[x]);
>> +    }
>> +    return sum;
>> +}
>> +
>> +/* { dg-final { scan-assembler-times {\tudot\t} 3 } } */
>> --
>> 2.43.0
>>
Richard Biener July 7, 2026, 11:13 a.m. UTC | #3 
On Tue, Jul 7, 2026 at 11:49 AM Pengfei Li <pengfei.li2@arm.com> wrote:
>
> Hi Richard,
>
> On 07/07/2026 09:38, Richard Biener wrote:
> > On Tue, Jul 7, 2026 at 10:00 AM Pengfei Li <Pengfei.Li2@arm.com> wrote:
> >> Normally, SAD_EXPRs with identical first and second operands are not
> >> constructed, since the absolute difference value is zero. However, this
> >> can still appear when vectorizing small iteration-count reduction loops
> >> with an absolute difference operation.
> >>
> >> This patch adds a match.pd rule to fold the redundant SAD_EXPR to its
> >> accumulator operand.
> >>
> >> Bootstrapped and tested on aarch64-linux-gnu and x86_64-linux-gnu.
> > The pattern looks good to me, but I don't see how the testcase exercises it?
>
> In this testcase, SAD_EXPR is expanded to AArch64 uabd + udot
> instructions. Without the pattern, the uabd is eliminated but SAD_EXPR
> still generates an extra udot with a zero vector, e.g.
>
>      movi    d31, #0
>      ...
>      udot    z29.s, z31.b, z28.b
>
> The 48-iteration loop needs 3 udot, instead of 4. That's what the
> testcase checks. Ok for trunk?

OK.

>
> Thanks,
> Pengfei
>
> >
> >> gcc/ChangeLog:
> >>
> >>          * match.pd: Simplify SAD(x, x, acc) into acc.
> >>
> >> gcc/testsuite/ChangeLog:
> >>
> >>          * gcc.target/aarch64/sve/sad_2.c: New test.
> >> ---
> >>   gcc/match.pd                                 |  7 +++++++
> >>   gcc/testsuite/gcc.target/aarch64/sve/sad_2.c | 16 ++++++++++++++++
> >>   2 files changed, 23 insertions(+)
> >>   create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
> >>
> >> diff --git a/gcc/match.pd b/gcc/match.pd
> >> index a7cec25dbad..4520a23a026 100644
> >> --- a/gcc/match.pd
> >> +++ b/gcc/match.pd
> >> @@ -1229,6 +1229,13 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
> >>    (mult (absu@1 @0) @1)
> >>    (mult (convert@2 @0) @2))
> >>
> >> +#if GIMPLE
> >> +/* Simplify SAD(x, x, acc) -> acc since the absolute difference is zero.  */
> >> +(simplify
> >> + (sad @0 @0 @1)
> >> + @1)
> >> +#endif
> >> +
> >>   /* cos(copysign(x, y)) -> cos(x).  Similarly for cosh.  */
> >>   (for coss (COS COSH)
> >>    (for copysigns (COPYSIGN)
> >> diff --git a/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c b/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
> >> new file mode 100644
> >> index 00000000000..d46517f10d8
> >> --- /dev/null
> >> +++ b/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
> >> @@ -0,0 +1,16 @@
> >> +/* { dg-do compile } */
> >> +/* { dg-options "-O3 -msve-vector-bits=128 -mtune=neoverse-v2" } */
> >> +
> >> +#include <stdint.h>
> >> +
> >> +int foo (uint8_t *pix1, uint8_t *pix2)
> >> +{
> >> +    int sum = 0;
> >> +    for (int x = 0; x < 48; x++)
> >> +    {
> >> +        sum += __builtin_abs (pix1[x] - pix2[x]);
> >> +    }
> >> +    return sum;
> >> +}
> >> +
> >> +/* { dg-final { scan-assembler-times {\tudot\t} 3 } } */
> >> --
> >> 2.43.0
> >>
diff mbox series

Patch

diff --git a/gcc/match.pd b/gcc/match.pd
index a7cec25dbad..4520a23a026 100644
--- a/gcc/match.pd
+++ b/gcc/match.pd
@@ -1229,6 +1229,13 @@  DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
  (mult (absu@1 @0) @1)
  (mult (convert@2 @0) @2))
 
+#if GIMPLE
+/* Simplify SAD(x, x, acc) -> acc since the absolute difference is zero.  */
+(simplify
+ (sad @0 @0 @1)
+ @1)
+#endif
+
 /* cos(copysign(x, y)) -> cos(x).  Similarly for cosh.  */
 (for coss (COS COSH)
  (for copysigns (COPYSIGN)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c b/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
new file mode 100644
index 00000000000..d46517f10d8
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/sad_2.c
@@ -0,0 +1,16 @@ 
+/* { dg-do compile } */
+/* { dg-options "-O3 -msve-vector-bits=128 -mtune=neoverse-v2" } */
+
+#include <stdint.h>
+
+int foo (uint8_t *pix1, uint8_t *pix2)
+{
+    int sum = 0;
+    for (int x = 0; x < 48; x++)
+    {
+        sum += __builtin_abs (pix1[x] - pix2[x]);
+    }
+    return sum;
+}
+
+/* { dg-final { scan-assembler-times {\tudot\t} 3 } } */