ira: Use TARGET_CALLEE_SAVE_COST in improve_allocation
Commit Message
Since r15-7895 (PR117477), assign_hard_reg prices a callee-saved
register that is not yet used with targetm.callee_save_cost, given the
callee-saved registers already allocated, and weights it by the entry
and exit block frequencies. improve_allocation still prices it with the
older formula, ((memory store cost + memory load cost)
* saved_nregs / nregs - 1) * entry frequency, which does not see the
registers already allocated.
So an allocno that was coloured before the first callee-saved register
was taken, and could not pay for a new register on its own, is not
moved into the next register at the price the target gives it. On
aarch64 a register that turns the STR/LDR of a saved register into an
STP/LDP costs 1, but improve_allocation charges the full price. For
example (the new test):
int foo (struct s *p, int val)
{
set_off (p, val);
p->flag = val >= 0;
return 0;
}
p takes x19. val stays in w1 and is saved and restored around the
call, because x20 is still charged as a new register:
str x19, [sp, 16] stp x19, x20, [sp, 16]
mov x19, x0 mov x19, x0
str w1, [sp, 44] -> mov w20, w1
bl set_off bl set_off
ldr w1, [sp, 44] mvn w1, w20
... ...
ldr x19, [sp, 16] ldp x19, x20, [sp, 16]
This patch splits the cost calculation out of assign_hard_reg and uses
it in both places. No functional change to assign_hard_reg. In
improve_allocation, targets that use the default hook now weight the
restore by the exit frequency rather than the entry frequency; targets
with their own hook (aarch64, x86) now get the price that hook gives.
Measured with the aarch64 cross compiler on a 789-object Linux kernel
sample: .text -0.032%, sp/x29-relative loads and stores -0.51%. The
sqlite 3.45.1 shell executes +0.0065% instructions under qemu and 194
fewer stack loads and stores, which is neutral. On x86-64, c-torture at
-O2, -Os, -m32 -O2 and -m32 -Os has the same .text size and stack
accesses except one -m32 -Os object, which is 3 bytes larger.
Assisted-by: Claude Opus 5 (Anthropic)
gcc/ChangeLog:
* ira-color.cc (calculate_saved_regs_cost): New function, split
out from...
(assign_hard_reg): ...here.
(improve_allocation): Use it instead of the memory move costs.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/callee_save_4.c: New test.
Signed-off-by: Dominic P <gcc@gcc.dp11.uk>
---
Bootstrapped and regression tested on x86_64-pc-linux-gnu. Cross-tested
aarch64-linux-gnu under qemu (aarch64.exp, aarch64-sve.exp,
aarch64-sve-pcs.exp, aarch64-sme.exp, dg.exp) and arm-none-eabi (arm.exp
and dg.exp, default and -march=armv7-a -mthumb -mfloat-abi=hard boards):
no regressions; the only change is the new test, which fails before and
passes after. sqlite at -O2 on x86-64 and c-torture at -Os for arm
cortex-m0 compile to identical code.
Unlike assign_hard_reg, improve_allocation still charges the premium
when HONOR_REG_ALLOC_ORDER is true; the patch keeps that. For SVE
registers the aarch64 hook can add the frame-allocation cost while
nothing has been spilled or caller-saved; improve_allocation does not
add that cost to the memory alternative, so it is slightly conservative
there.
OK for trunk?
gcc/ira-color.cc | 70 ++++++++++---------
.../gcc.target/aarch64/callee_save_4.c | 16 +++++
2 files changed, 53 insertions(+), 33 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/aarch64/callee_save_4.c
@@ -1820,6 +1820,38 @@ calculate_saved_nregs (int hard_regno, machine_mode mode)
return nregs;
}
+/* Return the cost of saving and restoring SAVED_NREGS callee-saved
+ registers in the prologue and epilogue if we allocate HARD_REGNO to
+ hold a value of MODE, given the callee-saved registers allocated so
+ far. The cost is TARGET_CALLEE_SAVE_COST weighted by the entry and
+ exit block frequencies. */
+static int
+calculate_saved_regs_cost (int hard_regno, machine_mode mode, int saved_nregs)
+{
+ int nregs = hard_regno_nregs (hard_regno, mode);
+ enum reg_class rclass = REGNO_REG_CLASS (hard_regno);
+ bool existing_spills_p = allocated_memory_p || caller_save_needed;
+
+ auto entry_cost = targetm.callee_save_cost
+ (spill_cost_type::SAVE, hard_regno, mode, saved_nregs,
+ ira_memory_move_cost[mode][rclass][0] * saved_nregs / nregs,
+ allocated_callee_save_regs, existing_spills_p);
+ /* In the event of a tie between caller-save and callee-save,
+ prefer callee-save. We apply this to the entry cost rather
+ than the exit cost since the entry frequency must be at
+ least as high as the exit frequency. */
+ if (entry_cost > 1)
+ entry_cost -= 1;
+
+ auto exit_cost = targetm.callee_save_cost
+ (spill_cost_type::RESTORE, hard_regno, mode, saved_nregs,
+ ira_memory_move_cost[mode][rclass][1] * saved_nregs / nregs,
+ allocated_callee_save_regs, existing_spills_p);
+
+ return (entry_cost * REG_FREQ_FROM_BB (ENTRY_BLOCK_PTR_FOR_FN (cfun))
+ + exit_cost * REG_FREQ_FROM_BB (EXIT_BLOCK_PTR_FOR_FN (cfun)));
+}
+
/* Allocnos A1 and A2 are known to conflict. Check whether, in some loop L
that is either the current loop or a nested subloop, the conflict is of
the following form:
@@ -2016,7 +2048,6 @@ assign_hard_reg (ira_allocno_t a, bool retry_p)
bool dep_filter_p;
static int costs[FIRST_PSEUDO_REGISTER], full_costs[FIRST_PSEUDO_REGISTER];
int saved_nregs;
- enum reg_class rclass;
int add_cost;
#ifdef STACK_REGS
bool no_stack_reg_p;
@@ -2261,28 +2292,8 @@ assign_hard_reg (ira_allocno_t a, bool retry_p)
/* We need to save/restore the hard register in
epilogue/prologue. Therefore we increase the cost. */
{
- int nregs = hard_regno_nregs (hard_regno, mode);
- add_cost = 0;
- rclass = REGNO_REG_CLASS (hard_regno);
-
- auto entry_cost = targetm.callee_save_cost
- (spill_cost_type::SAVE, hard_regno, mode, saved_nregs,
- ira_memory_move_cost[mode][rclass][0] * saved_nregs / nregs,
- allocated_callee_save_regs, existing_spills_p);
- /* In the event of a tie between caller-save and callee-save,
- prefer callee-save. We apply this to the entry cost rather
- than the exit cost since the entry frequency must be at
- least as high as the exit frequency. */
- if (entry_cost > 1)
- entry_cost -= 1;
- add_cost += entry_cost * entry_freq;
-
- auto exit_cost = targetm.callee_save_cost
- (spill_cost_type::RESTORE, hard_regno, mode, saved_nregs,
- ira_memory_move_cost[mode][rclass][1] * saved_nregs / nregs,
- allocated_callee_save_regs, existing_spills_p);
- add_cost += exit_cost * exit_freq;
-
+ add_cost = calculate_saved_regs_cost (hard_regno, mode,
+ saved_nregs);
cost += add_cost;
full_cost += add_cost;
}
@@ -3259,7 +3270,7 @@ improve_allocation (void)
int j, k, n, hregno, conflict_hregno, base_cost, class_size, word, nwords;
int check, spill_cost, min_cost, nregs, conflict_nregs, r, best;
bool try_p;
- enum reg_class aclass, rclass;
+ enum reg_class aclass;
machine_mode mode;
int *allocno_costs;
int costs[FIRST_PSEUDO_REGISTER];
@@ -3333,15 +3344,8 @@ improve_allocation (void)
if ((saved_nregs = calculate_saved_nregs (hregno, mode)) != 0)
{
/* We need to save/restore the hard register in
- epilogue/prologue. Therefore we increase the cost.
- Since the prolog is placed in the entry BB, the frequency
- of the entry BB is considered while computing the cost. */
- rclass = REGNO_REG_CLASS (hregno);
- add_cost = ((ira_memory_move_cost[mode][rclass][0]
- + ira_memory_move_cost[mode][rclass][1])
- * saved_nregs / hard_regno_nregs (hregno,
- mode) - 1)
- * REG_FREQ_FROM_BB (ENTRY_BLOCK_PTR_FOR_FN (cfun));
+ epilogue/prologue. Therefore we increase the cost. */
+ add_cost = calculate_saved_regs_cost (hregno, mode, saved_nregs);
costs[hregno] += add_cost;
}
new file mode 100644
@@ -0,0 +1,16 @@
+/* { dg-options "-O2 -fno-omit-frame-pointer -mtune=generic" } */
+
+struct s { int x; unsigned char flag; };
+void set_off (struct s *, int);
+
+int
+foo (struct s *p, int val)
+{
+ set_off (p, val);
+ p->flag = val >= 0;
+ return 0;
+}
+
+/* { dg-final { scan-assembler {\tstp\tx19, x20, \[sp,} } } */
+/* { dg-final { scan-assembler {\tldp\tx19, x20, \[sp,} } } */
+/* { dg-final { scan-assembler-not {\tstr\tw[0-9]+, \[sp,} } } */