[v3] middle-end RISC-V: Recognize conditional negate in ISEL [PR126245]
Checks
| Context |
Check |
Description |
| linaro-tcwg-bot/tcwg_gcc_build--master-arm |
success
|
Build passed
|
| linaro-tcwg-bot/tcwg_simplebootstrap_build--master-arm-bootstrap |
success
|
Build passed
|
| linaro-tcwg-bot/tcwg_gcc_check--master-arm |
success
|
Test passed
|
Commit Message
The patch was using the existing negcc/notcc path through
emit_conditional_neg_or_complement. The two expansions it tried were:
first the target conditional negate/complement sequence and then the
existing conditional-move sequence with a cost comparison when both
succeeded.
Reworked the patch to do the recognition in gimple-isel instead.
It now matches scalar COND_EXPRs of the form
cond ? -x : x
cond ? ~x : x
including the reversed form and rewrites them to the corresponding
direct internal function only when direct_internal_fn_supported_p says
the optab is available.
The existing RISC-V neg<mode>cc expander covers the negcc optab used by RTL
if-conversion but the direct internal function path uses cond_neg. Hence,
this patch adds the corresponding RISC-V cond_neg<mode> expander.
Regression tested on riscv target with no extra failures.
gcc/ChangeLog:
PR target/126245
* gimple-isel.cc (strip_ssa_nop_conversions): New.
(get_unary_op_operand): New.
(get_conditional_unary_operand): New.
(gimple_expand_cond_unary_expr): Expand scalar conditional
negation and complement into conditional internal functions.
(pass_gimple_isel::execute): Use gimple_expand_cond_unary_expr.
* config/riscv/riscv.md (cond_neg<mode>): New RISC-V expansion for
branchless conditional negation.
gcc/testsuite/ChangeLog:
PR target/126245
* gcc.target/riscv/pr126245-cond-neg.c: New test.
Signed-off-by: Naveen <naveen.siddegowda@oss.qualcomm.com>
---
gcc/config/riscv/riscv.md | 32 +++++
gcc/gimple-isel.cc | 131 ++++++++++++++++++
.../gcc.target/riscv/pr126245-cond-neg.c | 18 +++
3 files changed, 181 insertions(+)
create mode 100644 gcc/testsuite/gcc.target/riscv/pr126245-cond-neg.c
Comments
On Thu, Oct 1, 2026, 12:40 PM Naveen <naveen.siddegowda@oss.qualcomm.com>
wrote:
> The patch was using the existing negcc/notcc path through
> emit_conditional_neg_or_complement. The two expansions it tried were:
> first the target conditional negate/complement sequence and then the
> existing conditional-move sequence with a cost comparison when both
> succeeded.
>
> Reworked the patch to do the recognition in gimple-isel instead.
> It now matches scalar COND_EXPRs of the form
>
> cond ? -x : x
> cond ? ~x : x
> including the reversed form and rewrites them to the corresponding
> direct internal function only when direct_internal_fn_supported_p says
> the optab is available.
>
> The existing RISC-V neg<mode>cc expander covers the negcc optab used by RTL
> if-conversion but the direct internal function path uses cond_neg. Hence,
> this patch adds the corresponding RISC-V cond_neg<mode> expander.
>
Yes having both negcc and cond_neg is a problem. I had recorded this in
pr 120240.
I am not sure which one here we should be using. Because for aarch64, a fp
compare can be used with negcc and will work.
Thanks,
Andrea
> Regression tested on riscv target with no extra failures.
>
> gcc/ChangeLog:
>
> PR target/126245
> * gimple-isel.cc (strip_ssa_nop_conversions): New.
> (get_unary_op_operand): New.
> (get_conditional_unary_operand): New.
> (gimple_expand_cond_unary_expr): Expand scalar conditional
> negation and complement into conditional internal functions.
> (pass_gimple_isel::execute): Use gimple_expand_cond_unary_expr.
> * config/riscv/riscv.md (cond_neg<mode>): New RISC-V expansion for
> branchless conditional negation.
>
> gcc/testsuite/ChangeLog:
>
> PR target/126245
> * gcc.target/riscv/pr126245-cond-neg.c: New test.
>
> Signed-off-by: Naveen <naveen.siddegowda@oss.qualcomm.com>
> ---
> gcc/config/riscv/riscv.md | 32 +++++
> gcc/gimple-isel.cc | 131 ++++++++++++++++++
> .../gcc.target/riscv/pr126245-cond-neg.c | 18 +++
> 3 files changed, 181 insertions(+)
> create mode 100644 gcc/testsuite/gcc.target/riscv/pr126245-cond-neg.c
>
> diff --git a/gcc/config/riscv/riscv.md b/gcc/config/riscv/riscv.md
> index cfac4c960bf..a2003248b21 100644
> --- a/gcc/config/riscv/riscv.md
> +++ b/gcc/config/riscv/riscv.md
> @@ -3543,6 +3543,38 @@
> DONE;
> })
>
> +(define_expand "cond_neg<mode>"
> + [(set (match_operand:GPR 0 "register_operand")
> + (if_then_else:GPR (match_operand 1 "register_operand")
> + (neg:GPR (match_operand:GPR 2
> "register_operand"))
> + (match_operand:GPR 3 "register_operand")))]
> + ""
> +{
> + rtx cmp_result = gen_reg_rtx (<MODE>mode);
> + convert_move (cmp_result, operands[1], true);
> +
> + rtx mask = gen_reg_rtx (<MODE>mode);
> + riscv_emit_unary (NEG, mask, cmp_result);
> +
> + if (rtx_equal_p (operands[2], operands[3]))
> + {
> + rtx xor_result = gen_reg_rtx (<MODE>mode);
> + riscv_emit_binary (XOR, xor_result, operands[2], mask);
> + riscv_emit_binary (PLUS, operands[0], xor_result, cmp_result);
> + DONE;
> + }
> +
> + rtx neg = gen_reg_rtx (<MODE>mode);
> + riscv_emit_unary (NEG, neg, operands[2]);
> + rtx true_part = gen_reg_rtx (<MODE>mode);
> + riscv_emit_binary (AND, true_part, neg, mask);
> + riscv_emit_unary (NOT, mask, mask);
> + rtx false_part = gen_reg_rtx (<MODE>mode);
> + riscv_emit_binary (AND, false_part, operands[3], mask);
> + riscv_emit_binary (IOR, operands[0], true_part, false_part);
> + DONE;
> +})
> +
> (define_expand "add<mode>cc"
> [(match_operand:GPR 0 "register_operand")
> (match_operand 1 "comparison_operator")
> diff --git a/gcc/gimple-isel.cc b/gcc/gimple-isel.cc
> index 1d346ee124f..196ee7c12f9 100644
> --- a/gcc/gimple-isel.cc
> +++ b/gcc/gimple-isel.cc
> @@ -315,6 +315,128 @@ gimple_expand_vec_cond_expr (gimple_stmt_iterator
> *gsi)
> return gimple_build_call_internal (IFN_VCOND_MASK, 3, op0, op1, op2);
> }
>
> +/* Strip no-op conversions represented by SSA assignments from EXP. */
> +
> +static tree
> +strip_ssa_nop_conversions (tree exp)
> +{
> + while (TREE_CODE (exp) == SSA_NAME)
> + {
> + gassign *stmt = dyn_cast<gassign *> (SSA_NAME_DEF_STMT (exp));
> + if (!stmt
> + || !CONVERT_EXPR_CODE_P (gimple_assign_rhs_code (stmt))
> + || !tree_nop_conversion_p (TREE_TYPE (exp),
> + TREE_TYPE (gimple_assign_rhs1
> (stmt))))
> + break;
> + exp = gimple_assign_rhs1 (stmt);
> + }
> +
> + return tree_strip_nop_conversions (exp);
> +}
> +
> +/* Return the operand of a unary operation CODE in EXP looking through
> + no-op conversions represented by SSA assignments. */
> +
> +static tree
> +get_unary_op_operand (tree exp, enum tree_code code)
> +{
> + exp = strip_ssa_nop_conversions (exp);
> +
> + if (TREE_CODE (exp) == code)
> + return TREE_OPERAND (exp, 0);
> +
> + if (TREE_CODE (exp) == SSA_NAME)
> + {
> + gassign *stmt = dyn_cast<gassign *> (SSA_NAME_DEF_STMT (exp));
> + if (stmt && gimple_assign_rhs_code (stmt) == code)
> + return gimple_assign_rhs1 (stmt);
> + }
> +
> + return NULL_TREE;
> +}
> +
> +/* Return the operand of a conditional negate or complement in EXP and set
> + IFN to the corresponding conditional internal function. */
> +
> +static tree
> +get_conditional_unary_operand (tree exp, internal_fn *ifn)
> +{
> + tree operand = get_unary_op_operand (exp, NEGATE_EXPR);
> + if (operand)
> + {
> + *ifn = IFN_COND_NEG;
> + return operand;
> + }
> +
> + operand = get_unary_op_operand (exp, BIT_NOT_EXPR);
> + if (operand)
> + *ifn = IFN_COND_NOT;
> + return operand;
> +}
> +
> +/* Expand a scalar COND_EXPR of the form A ? -B : B or A ? ~B : B into a
> + conditional internal function when the target supports the optab. */
> +
> +static gimple *
> +gimple_expand_cond_unary_expr (gimple_stmt_iterator *gsi)
> +{
> + gassign *stmt = dyn_cast<gassign *> (gsi_stmt (*gsi));
> + if (!stmt || gimple_assign_rhs_code (stmt) != COND_EXPR)
> + return NULL;
> +
> + tree lhs = gimple_assign_lhs (stmt);
> + tree type = TREE_TYPE (lhs);
> + if (VECTOR_TYPE_P (type))
> + return NULL;
> +
> + tree op1 = gimple_assign_rhs2 (stmt);
> + tree op2 = gimple_assign_rhs3 (stmt);
> + tree op1_nops = strip_ssa_nop_conversions (op1);
> + tree op2_nops = strip_ssa_nop_conversions (op2);
> + internal_fn ifn = IFN_LAST;
> + bool reverse = false;
> + tree value = NULL_TREE;
> +
> + tree operand = get_conditional_unary_operand (op1, &ifn);
> + if (operand
> + && operand_equal_p (strip_ssa_nop_conversions (operand), op2_nops,
> 0))
> + value = op2;
> + else
> + {
> + operand = get_conditional_unary_operand (op2, &ifn);
> + if (operand
> + && operand_equal_p (strip_ssa_nop_conversions (operand),
> op1_nops, 0))
> + {
> + value = op1;
> + reverse = true;
> + }
> + }
> +
> + if (!value
> + || !types_compatible_p (TREE_TYPE (value), type)
> + || !direct_internal_fn_supported_p (ifn, type, OPTIMIZE_FOR_BOTH))
> + return NULL;
> +
> + location_t loc = gimple_location (stmt);
> + gimple_seq stmts = NULL;
> + tree cond = force_gimple_operand (gimple_assign_rhs1 (stmt), &stmts,
> true,
> + NULL_TREE);
> + if (TREE_CODE (TREE_TYPE (cond)) != BOOLEAN_TYPE)
> + cond = gimple_build (&stmts, loc, NE_EXPR, boolean_type_node, cond,
> + build_zero_cst (TREE_TYPE (cond)));
> + if (reverse)
> + cond = gimple_build (&stmts, loc, EQ_EXPR, boolean_type_node, cond,
> + build_zero_cst (TREE_TYPE (cond)));
> + if (stmts)
> + gsi_insert_seq_before (gsi, stmts, GSI_SAME_STMT);
> +
> + gcall *call = gimple_build_call_internal (ifn, 3, cond, value, value);
> + gimple_call_set_lhs (call, lhs);
> + gimple_set_location (call, loc);
> + gimple_call_set_nothrow (call, true);
> + return call;
> +}
> +
> /* Duplicate COND_EXPR condition defs of STMT located in BB when they are
> comparisons so RTL expansion with the help of TER
> can perform better if conversion. */
> @@ -1379,6 +1501,15 @@ pass_gimple_isel::execute (struct function *fun)
> gsi_replace (&gsi, g, false);
> }
>
> + /* Pre-expand scalar conditional negation and complement to
> + conditional internal functions that map to optabs. */
> + g = gimple_expand_cond_unary_expr (&gsi);
> + if (g != NULL)
> + {
> + gsi_replace (&gsi, g, false);
> + continue;
> + }
> +
> /* Recognize .VEC_SET and .VEC_EXTRACT patterns. */
> cfg_changed |= gimple_expand_vec_set_extract_expr (fun, &gsi);
> if (gsi_end_p (gsi))
> diff --git a/gcc/testsuite/gcc.target/riscv/pr126245-cond-neg.c
> b/gcc/testsuite/gcc.target/riscv/pr126245-cond-neg.c
> new file mode 100644
> index 00000000000..6ce517be7f1
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/pr126245-cond-neg.c
> @@ -0,0 +1,18 @@
> +/* PR target/126245 */
> +/* { dg-do compile { target { ! riscv_abi_e } } } */
> +/* { dg-options "-O2 -march=rv64g -mabi=lp64d -fdump-tree-isel" { target
> { rv64 } } } */
> +/* { dg-options "-O2 -march=rv32g -mabi=ilp32d -fdump-tree-isel" { target
> { rv32 } } } */
> +
> +unsigned long
> +f_cond_neg (unsigned long x, unsigned long y)
> +{
> + unsigned long cmp = x < y;
> + return (x ^ -cmp) + cmp;
> +}
> +
> +/* { dg-final { scan-tree-dump {\.COND_NEG} "isel" } } */
> +/* { dg-final { scan-assembler-times {\msltu\M} 1 } } */
> +/* { dg-final { scan-assembler-times "neg\t" 1 } } */
> +/* { dg-final { scan-assembler-times {\mxor\M} 1 } } */
> +/* { dg-final { scan-assembler-times {\madd\M} 1 } } */
> +/* { dg-final { scan-assembler-not
> {\m(?:beq|bne|blt|bge|ble|bgt)(?:u|z)?\M} } } */
> --
> 2.34.1
>
>
@@ -3543,6 +3543,38 @@
DONE;
})
+(define_expand "cond_neg<mode>"
+ [(set (match_operand:GPR 0 "register_operand")
+ (if_then_else:GPR (match_operand 1 "register_operand")
+ (neg:GPR (match_operand:GPR 2 "register_operand"))
+ (match_operand:GPR 3 "register_operand")))]
+ ""
+{
+ rtx cmp_result = gen_reg_rtx (<MODE>mode);
+ convert_move (cmp_result, operands[1], true);
+
+ rtx mask = gen_reg_rtx (<MODE>mode);
+ riscv_emit_unary (NEG, mask, cmp_result);
+
+ if (rtx_equal_p (operands[2], operands[3]))
+ {
+ rtx xor_result = gen_reg_rtx (<MODE>mode);
+ riscv_emit_binary (XOR, xor_result, operands[2], mask);
+ riscv_emit_binary (PLUS, operands[0], xor_result, cmp_result);
+ DONE;
+ }
+
+ rtx neg = gen_reg_rtx (<MODE>mode);
+ riscv_emit_unary (NEG, neg, operands[2]);
+ rtx true_part = gen_reg_rtx (<MODE>mode);
+ riscv_emit_binary (AND, true_part, neg, mask);
+ riscv_emit_unary (NOT, mask, mask);
+ rtx false_part = gen_reg_rtx (<MODE>mode);
+ riscv_emit_binary (AND, false_part, operands[3], mask);
+ riscv_emit_binary (IOR, operands[0], true_part, false_part);
+ DONE;
+})
+
(define_expand "add<mode>cc"
[(match_operand:GPR 0 "register_operand")
(match_operand 1 "comparison_operator")
@@ -315,6 +315,128 @@ gimple_expand_vec_cond_expr (gimple_stmt_iterator *gsi)
return gimple_build_call_internal (IFN_VCOND_MASK, 3, op0, op1, op2);
}
+/* Strip no-op conversions represented by SSA assignments from EXP. */
+
+static tree
+strip_ssa_nop_conversions (tree exp)
+{
+ while (TREE_CODE (exp) == SSA_NAME)
+ {
+ gassign *stmt = dyn_cast<gassign *> (SSA_NAME_DEF_STMT (exp));
+ if (!stmt
+ || !CONVERT_EXPR_CODE_P (gimple_assign_rhs_code (stmt))
+ || !tree_nop_conversion_p (TREE_TYPE (exp),
+ TREE_TYPE (gimple_assign_rhs1 (stmt))))
+ break;
+ exp = gimple_assign_rhs1 (stmt);
+ }
+
+ return tree_strip_nop_conversions (exp);
+}
+
+/* Return the operand of a unary operation CODE in EXP looking through
+ no-op conversions represented by SSA assignments. */
+
+static tree
+get_unary_op_operand (tree exp, enum tree_code code)
+{
+ exp = strip_ssa_nop_conversions (exp);
+
+ if (TREE_CODE (exp) == code)
+ return TREE_OPERAND (exp, 0);
+
+ if (TREE_CODE (exp) == SSA_NAME)
+ {
+ gassign *stmt = dyn_cast<gassign *> (SSA_NAME_DEF_STMT (exp));
+ if (stmt && gimple_assign_rhs_code (stmt) == code)
+ return gimple_assign_rhs1 (stmt);
+ }
+
+ return NULL_TREE;
+}
+
+/* Return the operand of a conditional negate or complement in EXP and set
+ IFN to the corresponding conditional internal function. */
+
+static tree
+get_conditional_unary_operand (tree exp, internal_fn *ifn)
+{
+ tree operand = get_unary_op_operand (exp, NEGATE_EXPR);
+ if (operand)
+ {
+ *ifn = IFN_COND_NEG;
+ return operand;
+ }
+
+ operand = get_unary_op_operand (exp, BIT_NOT_EXPR);
+ if (operand)
+ *ifn = IFN_COND_NOT;
+ return operand;
+}
+
+/* Expand a scalar COND_EXPR of the form A ? -B : B or A ? ~B : B into a
+ conditional internal function when the target supports the optab. */
+
+static gimple *
+gimple_expand_cond_unary_expr (gimple_stmt_iterator *gsi)
+{
+ gassign *stmt = dyn_cast<gassign *> (gsi_stmt (*gsi));
+ if (!stmt || gimple_assign_rhs_code (stmt) != COND_EXPR)
+ return NULL;
+
+ tree lhs = gimple_assign_lhs (stmt);
+ tree type = TREE_TYPE (lhs);
+ if (VECTOR_TYPE_P (type))
+ return NULL;
+
+ tree op1 = gimple_assign_rhs2 (stmt);
+ tree op2 = gimple_assign_rhs3 (stmt);
+ tree op1_nops = strip_ssa_nop_conversions (op1);
+ tree op2_nops = strip_ssa_nop_conversions (op2);
+ internal_fn ifn = IFN_LAST;
+ bool reverse = false;
+ tree value = NULL_TREE;
+
+ tree operand = get_conditional_unary_operand (op1, &ifn);
+ if (operand
+ && operand_equal_p (strip_ssa_nop_conversions (operand), op2_nops, 0))
+ value = op2;
+ else
+ {
+ operand = get_conditional_unary_operand (op2, &ifn);
+ if (operand
+ && operand_equal_p (strip_ssa_nop_conversions (operand), op1_nops, 0))
+ {
+ value = op1;
+ reverse = true;
+ }
+ }
+
+ if (!value
+ || !types_compatible_p (TREE_TYPE (value), type)
+ || !direct_internal_fn_supported_p (ifn, type, OPTIMIZE_FOR_BOTH))
+ return NULL;
+
+ location_t loc = gimple_location (stmt);
+ gimple_seq stmts = NULL;
+ tree cond = force_gimple_operand (gimple_assign_rhs1 (stmt), &stmts, true,
+ NULL_TREE);
+ if (TREE_CODE (TREE_TYPE (cond)) != BOOLEAN_TYPE)
+ cond = gimple_build (&stmts, loc, NE_EXPR, boolean_type_node, cond,
+ build_zero_cst (TREE_TYPE (cond)));
+ if (reverse)
+ cond = gimple_build (&stmts, loc, EQ_EXPR, boolean_type_node, cond,
+ build_zero_cst (TREE_TYPE (cond)));
+ if (stmts)
+ gsi_insert_seq_before (gsi, stmts, GSI_SAME_STMT);
+
+ gcall *call = gimple_build_call_internal (ifn, 3, cond, value, value);
+ gimple_call_set_lhs (call, lhs);
+ gimple_set_location (call, loc);
+ gimple_call_set_nothrow (call, true);
+ return call;
+}
+
/* Duplicate COND_EXPR condition defs of STMT located in BB when they are
comparisons so RTL expansion with the help of TER
can perform better if conversion. */
@@ -1379,6 +1501,15 @@ pass_gimple_isel::execute (struct function *fun)
gsi_replace (&gsi, g, false);
}
+ /* Pre-expand scalar conditional negation and complement to
+ conditional internal functions that map to optabs. */
+ g = gimple_expand_cond_unary_expr (&gsi);
+ if (g != NULL)
+ {
+ gsi_replace (&gsi, g, false);
+ continue;
+ }
+
/* Recognize .VEC_SET and .VEC_EXTRACT patterns. */
cfg_changed |= gimple_expand_vec_set_extract_expr (fun, &gsi);
if (gsi_end_p (gsi))
new file mode 100644
@@ -0,0 +1,18 @@
+/* PR target/126245 */
+/* { dg-do compile { target { ! riscv_abi_e } } } */
+/* { dg-options "-O2 -march=rv64g -mabi=lp64d -fdump-tree-isel" { target { rv64 } } } */
+/* { dg-options "-O2 -march=rv32g -mabi=ilp32d -fdump-tree-isel" { target { rv32 } } } */
+
+unsigned long
+f_cond_neg (unsigned long x, unsigned long y)
+{
+ unsigned long cmp = x < y;
+ return (x ^ -cmp) + cmp;
+}
+
+/* { dg-final { scan-tree-dump {\.COND_NEG} "isel" } } */
+/* { dg-final { scan-assembler-times {\msltu\M} 1 } } */
+/* { dg-final { scan-assembler-times "neg\t" 1 } } */
+/* { dg-final { scan-assembler-times {\mxor\M} 1 } } */
+/* { dg-final { scan-assembler-times {\madd\M} 1 } } */
+/* { dg-final { scan-assembler-not {\m(?:beq|bne|blt|bge|ble|bgt)(?:u|z)?\M} } } */