RISC-V: Switch RVV cost model to generic vector cost model

  This patch is preparing patch for the following cost model tweak.

Since we don't have vector cost model in default tune info (rocket),
we set the cost model default as generic cost model by default.

The reason we want to switch to generic vector cost model is the default
cost model generates inferior codegen for various benchmarks.

For example, PR113247, we have performance bug that we end up having over 70%
performance drop of SHA256.  Currently, no matter how we adapt cost model,
we are not able to fix the performance bug since we always use default cost model by default.

Also, tweak the generic cost model back to default cost model since we have some FAILs in
current tests.

After this patch, we (me an Robin) can work on cost model tunning together to improve performane
in various benchmarks.

Tested on both RV32 and RV64, ok for trunk ?

gcc/ChangeLog:

	* config/riscv/riscv.cc (get_common_costs): Switch RVV cost model.
	(get_vector_costs): Ditto.
	(riscv_builtin_vectorization_cost): Ditto.

---
 gcc/config/riscv/riscv.cc | 117 ++++++++++++++++++++------------------
 1 file changed, 61 insertions(+), 56 deletions(-)
  

Hi Juzhe,

> The reason we want to switch to generic vector cost model is the default
> cost model generates inferior codegen for various benchmarks.
> 
> For example, PR113247, we have performance bug that we end up having over 70%
> performance drop of SHA256.  Currently, no matter how we adapt cost model,
> we are not able to fix the performance bug since we always use default cost model by default.
> 
> Also, tweak the generic cost model back to default cost model since we have some FAILs in
> current tests.

So to recap:

 - Our current default tune model is rocket which does not have a vector
   cost model.  No other tune model except generic-ooo has one.

 - We want tune models with no vector cost model to fall back to the
   default vector cost model for now, later possibly the generic RVV
   cost model.

 - You're seeing inferior codegen for dynamic-lmul2-7.c with our generic
   RVV (not default) vector cost model (built with -mtune=generic-ooo?).

Therefore the suggestions is to start over freshly with the default
vector cost model?

>  /* Generic costs for VLA vector operations.  */
> @@ -374,13 +374,13 @@ static const scalable_vector_cost generic_vla_vector_cost = {
>      1, /* fp_stmt_cost  */
>      1, /* gather_load_cost  */
>      1, /* scatter_store_cost  */
> -    2, /* vec_to_scalar_cost  */
> +    1, /* vec_to_scalar_cost  */
>      1, /* scalar_to_vec_cost  */
> -    2, /* permute_cost  */
> +    1, /* permute_cost  */
>      1, /* align_load_cost  */
>      1, /* align_store_cost  */
> -    1, /* unalign_load_cost  */
> -    1, /* unalign_store_cost  */
> +    2, /* unalign_load_cost  */
> +    2, /* unalign_store_cost  */
>    },
>  };

So is the idea here to just revert the values to the defaults for now
and change them again soon?  And not to keep this as another default
and add others?

I'm a bit confused here :)  How does this help?  Can't we continue to
fall back to the default vector cost model when a tune model does not
specify a vector cost model?  If generic-ooo using the generic vector
cost model is the problem, then let's just change it to NULL for now?

I suppose at some point we will not want to fall back to the default
vector cost model anymore but always use the generic RVV cost model.
Once we reach the costing part we need to fall back to something
if nothing was defined and generic RVV is supposed to always be better 
than default.

Regards
 Robin
  

>> So is the idea here to just revert the values to the defaults for now
>> and change them again soon?  And not to keep this as another default
>> and add others?

My idea is to revert default for now. Then we can refine the cost gradually.

>> I'm a bit confused here :)  How does this help?  Can't we continue to
>> fall back to the default vector cost model when a tune model does not
>> specify a vector cost model?  If generic-ooo using the generic vector
>> cost model is the problem, then let's just change it to NULL for now?

If you still want to fall back to default vector cost model.
Could you tell me how to fix the XFAILs of slp-*.c tests ?



juzhe.zhong@rivai.ai
 
From: Robin Dapp
Date: 2024-01-10 22:11
To: Juzhe-Zhong; gcc-patches
CC: rdapp.gcc; kito.cheng; kito.cheng; jeffreyalaw
Subject: Re: [PATCH] RISC-V: Switch RVV cost model to generic vector cost model
Hi Juzhe,
 
> The reason we want to switch to generic vector cost model is the default
> cost model generates inferior codegen for various benchmarks.
> 
> For example, PR113247, we have performance bug that we end up having over 70%
> performance drop of SHA256.  Currently, no matter how we adapt cost model,
> we are not able to fix the performance bug since we always use default cost model by default.
> 
> Also, tweak the generic cost model back to default cost model since we have some FAILs in
> current tests.
 
So to recap:
 
- Our current default tune model is rocket which does not have a vector
   cost model.  No other tune model except generic-ooo has one.
 
- We want tune models with no vector cost model to fall back to the
   default vector cost model for now, later possibly the generic RVV
   cost model.
 
- You're seeing inferior codegen for dynamic-lmul2-7.c with our generic
   RVV (not default) vector cost model (built with -mtune=generic-ooo?).
 
Therefore the suggestions is to start over freshly with the default
vector cost model?
 
>  /* Generic costs for VLA vector operations.  */
> @@ -374,13 +374,13 @@ static const scalable_vector_cost generic_vla_vector_cost = {
>      1, /* fp_stmt_cost  */
>      1, /* gather_load_cost  */
>      1, /* scatter_store_cost  */
> -    2, /* vec_to_scalar_cost  */
> +    1, /* vec_to_scalar_cost  */
>      1, /* scalar_to_vec_cost  */
> -    2, /* permute_cost  */
> +    1, /* permute_cost  */
>      1, /* align_load_cost  */
>      1, /* align_store_cost  */
> -    1, /* unalign_load_cost  */
> -    1, /* unalign_store_cost  */
> +    2, /* unalign_load_cost  */
> +    2, /* unalign_store_cost  */
>    },
>  };
 
So is the idea here to just revert the values to the defaults for now
and change them again soon?  And not to keep this as another default
and add others?
 
I'm a bit confused here :)  How does this help?  Can't we continue to
fall back to the default vector cost model when a tune model does not
specify a vector cost model?  If generic-ooo using the generic vector
cost model is the problem, then let's just change it to NULL for now?
 
I suppose at some point we will not want to fall back to the default
vector cost model anymore but always use the generic RVV cost model.
Once we reach the costing part we need to fall back to something
if nothing was defined and generic RVV is supposed to always be better 
than default.
 
Regards
Robin
  

I need to add these costs for segment load/stores:

/* Generic costs for VLA vector operations.  */
static const scalable_vector_cost generic_vla_vector_cost = {
  {
    1, /* int_stmt_cost  */
    1, /* fp_stmt_cost  */
    1, /* gather_load_cost  */
    1, /* scatter_store_cost  */
    1, /* vec_to_scalar_cost  */
    1, /* scalar_to_vec_cost  */
    1, /* permute_cost  */
    1, /* align_load_cost  */
    1, /* align_store_cost  */
    2, /* unalign_load_cost  */
    2, /* unalign_store_cost  */
  },
  2, /* vlseg2_vsseg2_permute_cost  */
  2, /* vlseg3_vsseg3_permute_cost  */
  3, /* vlseg4_vsseg4_permute_cost  */
  3, /* vlseg5_vsseg5_permute_cost  */
  4, /* vlseg6_vsseg6_permute_cost  */
  4, /* vlseg7_vsseg7_permute_cost  */
  4, /* vlseg8_vsseg8_permute_cost  */
};

to fix the SLP issues in the following patches.

If you don't allow me to switch to generic vector cost model and tune it.
How can I fix the FAILs of slp-*.c cases ?

Currently, l let all slp-*.c tests all XFAIL which definitely incorrect.


juzhe.zhong@rivai.ai
 
From: Robin Dapp
Date: 2024-01-10 22:11
To: Juzhe-Zhong; gcc-patches
CC: rdapp.gcc; kito.cheng; kito.cheng; jeffreyalaw
Subject: Re: [PATCH] RISC-V: Switch RVV cost model to generic vector cost model
Hi Juzhe,
 
> The reason we want to switch to generic vector cost model is the default
> cost model generates inferior codegen for various benchmarks.
> 
> For example, PR113247, we have performance bug that we end up having over 70%
> performance drop of SHA256.  Currently, no matter how we adapt cost model,
> we are not able to fix the performance bug since we always use default cost model by default.
> 
> Also, tweak the generic cost model back to default cost model since we have some FAILs in
> current tests.
 
So to recap:
 
- Our current default tune model is rocket which does not have a vector
   cost model.  No other tune model except generic-ooo has one.
 
- We want tune models with no vector cost model to fall back to the
   default vector cost model for now, later possibly the generic RVV
   cost model.
 
- You're seeing inferior codegen for dynamic-lmul2-7.c with our generic
   RVV (not default) vector cost model (built with -mtune=generic-ooo?).
 
Therefore the suggestions is to start over freshly with the default
vector cost model?
 
>  /* Generic costs for VLA vector operations.  */
> @@ -374,13 +374,13 @@ static const scalable_vector_cost generic_vla_vector_cost = {
>      1, /* fp_stmt_cost  */
>      1, /* gather_load_cost  */
>      1, /* scatter_store_cost  */
> -    2, /* vec_to_scalar_cost  */
> +    1, /* vec_to_scalar_cost  */
>      1, /* scalar_to_vec_cost  */
> -    2, /* permute_cost  */
> +    1, /* permute_cost  */
>      1, /* align_load_cost  */
>      1, /* align_store_cost  */
> -    1, /* unalign_load_cost  */
> -    1, /* unalign_store_cost  */
> +    2, /* unalign_load_cost  */
> +    2, /* unalign_store_cost  */
>    },
>  };
 
So is the idea here to just revert the values to the defaults for now
and change them again soon?  And not to keep this as another default
and add others?
 
I'm a bit confused here :)  How does this help?  Can't we continue to
fall back to the default vector cost model when a tune model does not
specify a vector cost model?  If generic-ooo using the generic vector
cost model is the problem, then let's just change it to NULL for now?
 
I suppose at some point we will not want to fall back to the default
vector cost model anymore but always use the generic RVV cost model.
Once we reach the costing part we need to fall back to something
if nothing was defined and generic RVV is supposed to always be better 
than default.
 
Regards
Robin
  

On 1/10/24 15:40, 钟居哲 wrote:
> I need to add these costs for segment load/stores:
> 
> /* Generic costs for VLA vector operations.  */
> static const scalable_vector_cost generic_vla_vector_cost = {
>   {
>     1,/* int_stmt_cost  */
>     1,/* fp_stmt_cost  */
>     1,/* gather_load_cost  */
>     1,/* scatter_store_cost  */
>     1,/* vec_to_scalar_cost  */
>     1,/* scalar_to_vec_cost  */
>     1,/* permute_cost  */
>     1,/* align_load_cost  */
>     1,/* align_store_cost  */
>     2,/* unalign_load_cost  */
>     2,/* unalign_store_cost  */
>   },
>   2,/* vlseg2_vsseg2_permute_cost  */
>   2,/* vlseg3_vsseg3_permute_cost  */
>   3,/* vlseg4_vsseg4_permute_cost  */
>   3,/* vlseg5_vsseg5_permute_cost  */
>   4,/* vlseg6_vsseg6_permute_cost  */
>   4,/* vlseg7_vsseg7_permute_cost  */
>   4,/* vlseg8_vsseg8_permute_cost  */
> };
> 
> to fix the SLP issues in the following patches.
> 
> If you don't allow me to switch to generic vector cost model and tune it.
> How can I fix the FAILs of slp-*.c cases ?
> 
> Currently, l let all slp-*.c tests all XFAIL which definitely incorrect.

Of course we don't want those XFAILs.  It's not a matter of "allowing"
or not but rather that I'd like to understand the reasoning.  The patch
itself seems reasonable to me apart from not really getting the
intention.

Your main point seems to be

> +  const cpu_vector_cost *costs = tune_param->vec_costs;
> +  if (!costs)
> +    return &generic_vector_cost
and that is fine.  What's not clear is whether changing the actual
costs is a temporary thing or whether it is supposed to be another
fallback.  If they are going to be changed anyway, why do we need
to revert to the default model now?  As discussed yesterday
increased permute costs and vec_to_scalar costs make sense, to first
order.  Is that because of dynamic-lmul2-7.c?

Generally we need to make the costs dependent on the
type or mode of course, just as we started to do with the latencies.
Permute is particularly sensitive as you already gathered.

Regards
 Robin
  

Current generic cost model makes dynamic-lmul2-7.c generate inferior codegen.

I found if I tweak the cost a little bit then dynamic-lmul2-7.c codegen can be recovered.
However, it makes other tests failed....
It's complicated story....

So, I'd rather set it as default cost and switch to it.
Then, we can tune the cost gradually, not only fix the issues we faced (e.g. SHA256), but also no matter how we 
tweak the costs later, it won't hurt the codegen of current tests.

It's true that: we can keep current cost model default_builtin_vectorization_cost
And tweak generic cost model, for exampl, add testcase for SHA256 and add -mtune=generic-ooo to test it.
But the question, how do you know whether there is a regression on current testsuite with -mtune=generic-ooo ?

Note that we can tweak generic vector cost model to fix SHA256 issue easily, but we should also make sure 
we don't have regressions on current testsuite with the new cost model.  So I switch the cost model.




juzhe.zhong@rivai.ai
 
From: Robin Dapp
Date: 2024-01-10 23:04
To: 钟居哲; gcc-patches
CC: rdapp.gcc; kito.cheng; kito.cheng; Jeff Law
Subject: Re: [PATCH] RISC-V: Switch RVV cost model to generic vector cost model
On 1/10/24 15:40, 钟居哲 wrote:
> I need to add these costs for segment load/stores:
> 
> /* Generic costs for VLA vector operations.  */
> static const scalable_vector_cost generic_vla_vector_cost = {
>   {
>     1,/* int_stmt_cost  */
>     1,/* fp_stmt_cost  */
>     1,/* gather_load_cost  */
>     1,/* scatter_store_cost  */
>     1,/* vec_to_scalar_cost  */
>     1,/* scalar_to_vec_cost  */
>     1,/* permute_cost  */
>     1,/* align_load_cost  */
>     1,/* align_store_cost  */
>     2,/* unalign_load_cost  */
>     2,/* unalign_store_cost  */
>   },
>   2,/* vlseg2_vsseg2_permute_cost  */
>   2,/* vlseg3_vsseg3_permute_cost  */
>   3,/* vlseg4_vsseg4_permute_cost  */
>   3,/* vlseg5_vsseg5_permute_cost  */
>   4,/* vlseg6_vsseg6_permute_cost  */
>   4,/* vlseg7_vsseg7_permute_cost  */
>   4,/* vlseg8_vsseg8_permute_cost  */
> };
> 
> to fix the SLP issues in the following patches.
> 
> If you don't allow me to switch to generic vector cost model and tune it.
> How can I fix the FAILs of slp-*.c cases ?
> 
> Currently, l let all slp-*.c tests all XFAIL which definitely incorrect.
 
Of course we don't want those XFAILs.  It's not a matter of "allowing"
or not but rather that I'd like to understand the reasoning.  The patch
itself seems reasonable to me apart from not really getting the
intention.
 
Your main point seems to be
 
> +  const cpu_vector_cost *costs = tune_param->vec_costs;
> +  if (!costs)
> +    return &generic_vector_cost
and that is fine.  What's not clear is whether changing the actual
costs is a temporary thing or whether it is supposed to be another
fallback.  If they are going to be changed anyway, why do we need
to revert to the default model now?  As discussed yesterday
increased permute costs and vec_to_scalar costs make sense, to first
order.  Is that because of dynamic-lmul2-7.c?
 
Generally we need to make the costs dependent on the
type or mode of course, just as we started to do with the latencies.
Permute is particularly sensitive as you already gathered.
 
Regards
Robin
  

> Current generic cost model makes dynamic-lmul2-7.c generate inferior codegen.
> 
> I found if I tweak the cost a little bit then dynamic-lmul2-7.c codegen can be recovered.
> However, it makes other tests failed....
> It's complicated story....

Ok, makes sense.  So the plan seems to be:

 (1) Fall back to the generic cost model if the tune model didn't
     specify one, i.e. make sure we always use the generic cost
     model rather than the default one.
 (2) Change this generic (fallback) cost model so we don't have
     regressions on the current trunk, as it's now always used.
 (3) Adjust it piece by piece.

Sure this makes sense and is also what I had in mind.

> It's true that: we can keep current cost model default_builtin_vectorization_cost
> And tweak generic cost model, for exampl, add testcase for SHA256 and add -mtune=generic-ooo to test it.

> But the question, how do you know whether there is a regression on current testsuite with -mtune=generic-ooo ?

That's a valid question and not easily solved.  Ideally the
generic model is generic enough to be a good base for most
uarchs.  Then the uarchs would only do minor adjustments and
have their own tests for that while the bulk of the generic
tests would still pass.

Generally, normal tests should be pretty independent of the
cost model with the exception of checking instruction sequences.
Those that are not should either specify their own -mtune and/or
disable scheduling.  Of course that's easier said than done...

Back to the patch:

I would suggest either renaming generic_vl[sa]_vector_cost to
rvv_vl[sa]_vector_cost (I find generic a bit too close to default)
and/or add comments that those are supposed to be the vector cost models
used by default if no other cost model was specified.

After understanding (2) of the plan the patch is OK to me with
that changed.

Regards
 Robin
  

>> (1) Fall back to the generic cost model if the tune model didn't
>> (specify one, i.e. make sure we always use the generic cost
>> ( model rather than the default one.
>> ((2) Change this generic (fallback) cost model so we don't have
>> (regressions on the current trunk, as it's now always used.
>> ((3) Adjust it piece by piece.

>> (Sure this makes sense and is also what I had in mind.

Yes, that's my plan.

Send in V2:
[PATCH V2] RISC-V: Switch RVV cost model. (gnu.org)



juzhe.zhong@rivai.ai
 
From: Robin Dapp
Date: 2024-01-10 23:36
To: 钟居哲; gcc-patches
CC: rdapp.gcc; kito.cheng; kito.cheng; Jeff Law
Subject: Re: [PATCH] RISC-V: Switch RVV cost model to generic vector cost model
> Current generic cost model makes dynamic-lmul2-7.c generate inferior codegen.
> 
> I found if I tweak the cost a little bit then dynamic-lmul2-7.c codegen can be recovered.
> However, it makes other tests failed....
> It's complicated story....
 
Ok, makes sense.  So the plan seems to be:
 
(1) Fall back to the generic cost model if the tune model didn't
     specify one, i.e. make sure we always use the generic cost
     model rather than the default one.
(2) Change this generic (fallback) cost model so we don't have
     regressions on the current trunk, as it's now always used.
(3) Adjust it piece by piece.
 
Sure this makes sense and is also what I had in mind.
 
> It's true that: we can keep current cost model default_builtin_vectorization_cost
> And tweak generic cost model, for exampl, add testcase for SHA256 and add -mtune=generic-ooo to test it.
 
> But the question, how do you know whether there is a regression on current testsuite with -mtune=generic-ooo ?
 
That's a valid question and not easily solved.  Ideally the
generic model is generic enough to be a good base for most
uarchs.  Then the uarchs would only do minor adjustments and
have their own tests for that while the bulk of the generic
tests would still pass.
 
Generally, normal tests should be pretty independent of the
cost model with the exception of checking instruction sequences.
Those that are not should either specify their own -mtune and/or
disable scheduling.  Of course that's easier said than done...
 
Back to the patch:
 
I would suggest either renaming generic_vl[sa]_vector_cost to
rvv_vl[sa]_vector_cost (I find generic a bit too close to default)
and/or add comments that those are supposed to be the vector cost models
used by default if no other cost model was specified.
 
After understanding (2) of the plan the patch is OK to me with
that changed.
 
Regards
Robin