[21/22] arm: [MVE intrinsics] factorize several binary operations
Commit Message
Factorize vabdq, vhaddq, vhsubq, vmulhq, vqaddq_u, vqdmulhq,
vqrdmulhq, vqrshlq, vqshlq, vqsubq_u, vrhaddq, vrmulhq, vrshlq
so that they use the same pattern.
2022-09-08 Christophe Lyon <christophe.lyon@arm.com>
gcc/
* config/arm/iterators.md (MVE_INT_SU_BINARY): New.
(mve_insn): Add vabdq, vhaddq, vhsubq, vmulhq, vqaddq, vqdmulhq,
vqrdmulhq, vqrshlq, vqshlq, vqsubq, vrhaddq, vrmulhq, vrshlq.
(supf): Add VQDMULHQ_S, VQRDMULHQ_S.
* config/arm/mve.md (mve_vabdq_<supf><mode>)
(@mve_vhaddq_<supf><mode>, mve_vhsubq_<supf><mode>)
(mve_vmulhq_<supf><mode>, mve_vqaddq_<supf><mode>)
(mve_vqdmulhq_s<mode>, mve_vqrdmulhq_s<mode>)
(mve_vqrshlq_<supf><mode>, mve_vqshlq_<supf><mode>)
(mve_vqsubq_<supf><mode>, @mve_vrhaddq_<supf><mode>)
(mve_vrmulhq_<supf><mode>, mve_vrshlq_<supf><mode>): Merge into
...
(@mve_<mve_insn>q_<supf><mode>): ... this.
* config/arm/vec-common.md (avg<mode>3_floor, uavg<mode>3_floor)
(avg<mode>3_ceil, uavg<mode>3_ceil): Use gen_mve_q instead of
gen_mve_vhaddq / gen_mve_vrhaddq.
---
gcc/config/arm/iterators.md | 31 ++++++
gcc/config/arm/mve.md | 198 +++--------------------------------
gcc/config/arm/vec-common.md | 8 +-
3 files changed, 50 insertions(+), 187 deletions(-)
Comments
> -----Original Message-----
> From: Christophe Lyon <christophe.lyon@arm.com>
> Sent: Tuesday, April 18, 2023 2:46 PM
> To: gcc-patches@gcc.gnu.org; Kyrylo Tkachov <Kyrylo.Tkachov@arm.com>;
> Richard Earnshaw <Richard.Earnshaw@arm.com>; Richard Sandiford
> <Richard.Sandiford@arm.com>
> Cc: Christophe Lyon <Christophe.Lyon@arm.com>
> Subject: [PATCH 21/22] arm: [MVE intrinsics] factorize several binary
> operations
>
> Factorize vabdq, vhaddq, vhsubq, vmulhq, vqaddq_u, vqdmulhq,
> vqrdmulhq, vqrshlq, vqshlq, vqsubq_u, vrhaddq, vrmulhq, vrshlq
> so that they use the same pattern.
>
Ok, as before without the trailing ')'.
Thanks,
Kyrill
> 2022-09-08 Christophe Lyon <christophe.lyon@arm.com>
>
> gcc/
> * config/arm/iterators.md (MVE_INT_SU_BINARY): New.
> (mve_insn): Add vabdq, vhaddq, vhsubq, vmulhq, vqaddq, vqdmulhq,
> vqrdmulhq, vqrshlq, vqshlq, vqsubq, vrhaddq, vrmulhq, vrshlq.
> (supf): Add VQDMULHQ_S, VQRDMULHQ_S.
> * config/arm/mve.md (mve_vabdq_<supf><mode>)
> (@mve_vhaddq_<supf><mode>, mve_vhsubq_<supf><mode>)
> (mve_vmulhq_<supf><mode>, mve_vqaddq_<supf><mode>)
> (mve_vqdmulhq_s<mode>, mve_vqrdmulhq_s<mode>)
> (mve_vqrshlq_<supf><mode>, mve_vqshlq_<supf><mode>)
> (mve_vqsubq_<supf><mode>, @mve_vrhaddq_<supf><mode>)
> (mve_vrmulhq_<supf><mode>, mve_vrshlq_<supf><mode>): Merge
> into
> ...
> (@mve_<mve_insn>q_<supf><mode>): ... this.
> * config/arm/vec-common.md (avg<mode>3_floor,
> uavg<mode>3_floor)
> (avg<mode>3_ceil, uavg<mode>3_ceil): Use gen_mve_q instead of
> gen_mve_vhaddq / gen_mve_vrhaddq.
> ---
> gcc/config/arm/iterators.md | 31 ++++++
> gcc/config/arm/mve.md | 198 +++--------------------------------
> gcc/config/arm/vec-common.md | 8 +-
> 3 files changed, 50 insertions(+), 187 deletions(-)
>
> diff --git a/gcc/config/arm/iterators.md b/gcc/config/arm/iterators.md
> index 60452cdefe3..068ae25e578 100644
> --- a/gcc/config/arm/iterators.md
> +++ b/gcc/config/arm/iterators.md
> @@ -414,6 +414,22 @@ (define_int_iterator MVE_INT_SU_N_BINARY [
> VQSUBQ_N_S VQSUBQ_N_U
> ])
>
> +(define_int_iterator MVE_INT_SU_BINARY [
> + VABDQ_S VABDQ_U
> + VHADDQ_S VHADDQ_U
> + VHSUBQ_S VHSUBQ_U
> + VMULHQ_S VMULHQ_U
> + VQADDQ_S VQADDQ_U
> + VQDMULHQ_S
> + VQRDMULHQ_S
> + VQRSHLQ_S VQRSHLQ_U
> + VQSHLQ_S VQSHLQ_U
> + VQSUBQ_S VQSUBQ_U
> + VRHADDQ_S VRHADDQ_U
> + VRMULHQ_S VRMULHQ_U
> + VRSHLQ_S VRSHLQ_U
> + ])
> +
> (define_int_iterator MVE_INT_N_BINARY_LOGIC [
> VBICQ_N_S VBICQ_N_U
> VORRQ_N_S VORRQ_N_U
> @@ -456,6 +472,7 @@ (define_code_attr mve_addsubmul [
>
> (define_int_attr mve_insn [
> (VABDQ_M_S "vabd") (VABDQ_M_U "vabd")
> + (VABDQ_S "vabd") (VABDQ_U "vabd")
> (VADDQ_M_N_S "vadd") (VADDQ_M_N_U "vadd")
> (VADDQ_M_N_F "vadd")
> (VADDQ_M_S "vadd") (VADDQ_M_U "vadd") (VADDQ_M_F
> "vadd")
> (VADDQ_N_S "vadd") (VADDQ_N_U "vadd") (VADDQ_N_F
> "vadd")
> @@ -468,14 +485,17 @@ (define_int_attr mve_insn [
> (VHADDQ_M_N_S "vhadd") (VHADDQ_M_N_U "vhadd")
> (VHADDQ_M_S "vhadd") (VHADDQ_M_U "vhadd")
> (VHADDQ_N_S "vhadd") (VHADDQ_N_U "vhadd")
> + (VHADDQ_S "vhadd") (VHADDQ_U "vhadd")
> (VHSUBQ_M_N_S "vhsub") (VHSUBQ_M_N_U "vhsub")
> (VHSUBQ_M_S "vhsub") (VHSUBQ_M_U "vhsub")
> (VHSUBQ_N_S "vhsub") (VHSUBQ_N_U "vhsub")
> + (VHSUBQ_S "vhsub") (VHSUBQ_U "vhsub")
> (VMAXQ_M_S "vmax") (VMAXQ_M_U "vmax")
> (VMINQ_M_S "vmin") (VMINQ_M_U "vmin")
> (VMLAQ_M_N_S "vmla") (VMLAQ_M_N_U "vmla")
> (VMLASQ_M_N_S "vmlas") (VMLASQ_M_N_U "vmlas")
> (VMULHQ_M_S "vmulh") (VMULHQ_M_U "vmulh")
> + (VMULHQ_S "vmulh") (VMULHQ_U "vmulh")
> (VMULQ_M_N_S "vmul") (VMULQ_M_N_U "vmul")
> (VMULQ_M_N_F "vmul")
> (VMULQ_M_S "vmul") (VMULQ_M_U "vmul") (VMULQ_M_F
> "vmul")
> (VMULQ_N_S "vmul") (VMULQ_N_U "vmul") (VMULQ_N_F
> "vmul")
> @@ -485,6 +505,7 @@ (define_int_attr mve_insn [
> (VQADDQ_M_N_S "vqadd") (VQADDQ_M_N_U "vqadd")
> (VQADDQ_M_S "vqadd") (VQADDQ_M_U "vqadd")
> (VQADDQ_N_S "vqadd") (VQADDQ_N_U "vqadd")
> + (VQADDQ_S "vqadd") (VQADDQ_U "vqadd")
> (VQDMLADHQ_M_S "vqdmladh")
> (VQDMLADHXQ_M_S "vqdmladhx")
> (VQDMLAHQ_M_N_S "vqdmlah")
> @@ -494,6 +515,7 @@ (define_int_attr mve_insn [
> (VQDMULHQ_M_N_S "vqdmulh")
> (VQDMULHQ_M_S "vqdmulh")
> (VQDMULHQ_N_S "vqdmulh")
> + (VQDMULHQ_S "vqdmulh")
> (VQRDMLADHQ_M_S "vqrdmladh")
> (VQRDMLADHXQ_M_S "vqrdmladhx")
> (VQRDMLAHQ_M_N_S "vqrdmlah")
> @@ -503,14 +525,21 @@ (define_int_attr mve_insn [
> (VQRDMULHQ_M_N_S "vqrdmulh")
> (VQRDMULHQ_M_S "vqrdmulh")
> (VQRDMULHQ_N_S "vqrdmulh")
> + (VQRDMULHQ_S "vqrdmulh")
> (VQRSHLQ_M_S "vqrshl") (VQRSHLQ_M_U "vqrshl")
> + (VQRSHLQ_S "vqrshl") (VQRSHLQ_U "vqrshl")
> (VQSHLQ_M_S "vqshl") (VQSHLQ_M_U "vqshl")
> + (VQSHLQ_S "vqshl") (VQSHLQ_U "vqshl")
> (VQSUBQ_M_N_S "vqsub") (VQSUBQ_M_N_U "vqsub")
> (VQSUBQ_M_S "vqsub") (VQSUBQ_M_U "vqsub")
> (VQSUBQ_N_S "vqsub") (VQSUBQ_N_U "vqsub")
> + (VQSUBQ_S "vqsub") (VQSUBQ_U "vqsub")
> (VRHADDQ_M_S "vrhadd") (VRHADDQ_M_U "vrhadd")
> + (VRHADDQ_S "vrhadd") (VRHADDQ_U "vrhadd")
> (VRMULHQ_M_S "vrmulh") (VRMULHQ_M_U "vrmulh")
> + (VRMULHQ_S "vrmulh") (VRMULHQ_U "vrmulh")
> (VRSHLQ_M_S "vrshl") (VRSHLQ_M_U "vrshl")
> + (VRSHLQ_S "vrshl") (VRSHLQ_U "vrshl")
> (VSHLQ_M_S "vshl") (VSHLQ_M_U "vshl")
> (VSUBQ_M_N_S "vsub") (VSUBQ_M_N_U "vsub")
> (VSUBQ_M_N_F "vsub")
> (VSUBQ_M_S "vsub") (VSUBQ_M_U "vsub") (VSUBQ_M_F
> "vsub")
> @@ -1669,6 +1698,8 @@ (define_int_attr supf [(VCVTQ_TO_F_S "s")
> (VCVTQ_TO_F_U "u") (VREV16Q_S "s")
> (VQRDMLASHQ_M_N_S "s")
> (VQDMULHQ_M_N_S "s")
> (VQRDMULHQ_M_N_S "s")
> + (VQDMULHQ_S "s")
> + (VQRDMULHQ_S "s")
> ])
>
> ;; Both kinds of return insn.
> diff --git a/gcc/config/arm/mve.md b/gcc/config/arm/mve.md
> index d14a04d5f82..b9126af2aa9 100644
> --- a/gcc/config/arm/mve.md
> +++ b/gcc/config/arm/mve.md
> @@ -841,16 +841,28 @@ (define_insn
> "mve_vcmp<mve_cmp_op>q_n_<mode>"
>
> ;;
> ;; [vabdq_s, vabdq_u])
> +;; [vhaddq_s, vhaddq_u])
> +;; [vhsubq_s, vhsubq_u])
> +;; [vmulhq_s, vmulhq_u])
> +;; [vqaddq_u, vqaddq_s])
> +;; [vqdmulhq_s])
> +;; [vqrdmulhq_s])
> +;; [vqrshlq_s, vqrshlq_u])
> +;; [vqshlq_s, vqshlq_u])
> +;; [vqsubq_u, vqsubq_s])
> +;; [vrhaddq_s, vrhaddq_u])
> +;; [vrmulhq_s, vrmulhq_u])
> +;; [vrshlq_s, vrshlq_u])
> ;;
> -(define_insn "mve_vabdq_<supf><mode>"
> +(define_insn "@mve_<mve_insn>q_<supf><mode>"
> [
> (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VABDQ))
> + MVE_INT_SU_BINARY))
> ]
> "TARGET_HAVE_MVE"
> - "vabd.<supf>%#<V_sz_elem> %q0, %q1, %q2"
> + "<mve_insn>.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> [(set_attr "type" "mve_move")
> ])
>
> @@ -1033,21 +1045,6 @@ (define_insn
> "@mve_<mve_insn>q_n_<supf><mode>"
> [(set_attr "type" "mve_move")
> ])
>
> -;;
> -;; [vhaddq_s, vhaddq_u])
> -;;
> -(define_insn "@mve_vhaddq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VHADDQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vhadd.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> ;;
> ;; [vhcaddq_rot270_s])
> ;;
> @@ -1078,21 +1075,6 @@ (define_insn "mve_vhcaddq_rot90_s<mode>"
> [(set_attr "type" "mve_move")
> ])
>
> -;;
> -;; [vhsubq_s, vhsubq_u])
> -;;
> -(define_insn "mve_vhsubq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VHSUBQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vhsub.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> ;;
> ;; [vmaxaq_s])
> ;;
> @@ -1293,21 +1275,6 @@ (define_insn "mve_vmlsdavxq_s<mode>"
> [(set_attr "type" "mve_move")
> ])
>
> -;;
> -;; [vmulhq_s, vmulhq_u])
> -;;
> -(define_insn "mve_vmulhq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VMULHQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vmulh.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> ;;
> ;; [vmullbq_int_u, vmullbq_int_s])
> ;;
> @@ -1405,51 +1372,6 @@ (define_expand "mve_vorrq_u<mode>"
> "TARGET_HAVE_MVE"
> )
>
> -;;
> -;; [vqaddq_u, vqaddq_s])
> -;;
> -(define_insn "mve_vqaddq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VQADDQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vqadd.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> -;;
> -;; [vqdmulhq_s])
> -;;
> -(define_insn "mve_vqdmulhq_s<mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VQDMULHQ_S))
> - ]
> - "TARGET_HAVE_MVE"
> - "vqdmulh.s%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> -;;
> -;; [vqrdmulhq_s])
> -;;
> -(define_insn "mve_vqrdmulhq_s<mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VQRDMULHQ_S))
> - ]
> - "TARGET_HAVE_MVE"
> - "vqrdmulh.s%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> ;;
> ;; [vqrshlq_n_s, vqrshlq_n_u])
> ;;
> @@ -1465,21 +1387,6 @@ (define_insn "mve_vqrshlq_n_<supf><mode>"
> [(set_attr "type" "mve_move")
> ])
>
> -;;
> -;; [vqrshlq_s, vqrshlq_u])
> -;;
> -(define_insn "mve_vqrshlq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VQRSHLQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vqrshl.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> ;;
> ;; [vqshlq_n_s, vqshlq_n_u])
> ;;
> @@ -1510,21 +1417,6 @@ (define_insn "mve_vqshlq_r_<supf><mode>"
> [(set_attr "type" "mve_move")
> ])
>
> -;;
> -;; [vqshlq_s, vqshlq_u])
> -;;
> -(define_insn "mve_vqshlq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VQSHLQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vqshl.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> ;;
> ;; [vqshluq_n_s])
> ;;
> @@ -1540,51 +1432,6 @@ (define_insn "mve_vqshluq_n_s<mode>"
> [(set_attr "type" "mve_move")
> ])
>
> -;;
> -;; [vqsubq_u, vqsubq_s])
> -;;
> -(define_insn "mve_vqsubq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VQSUBQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vqsub.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> -;;
> -;; [vrhaddq_s, vrhaddq_u])
> -;;
> -(define_insn "@mve_vrhaddq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VRHADDQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vrhadd.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> -;;
> -;; [vrmulhq_s, vrmulhq_u])
> -;;
> -(define_insn "mve_vrmulhq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VRMULHQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vrmulh.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> ;;
> ;; [vrshlq_n_u, vrshlq_n_s])
> ;;
> @@ -1600,21 +1447,6 @@ (define_insn "mve_vrshlq_n_<supf><mode>"
> [(set_attr "type" "mve_move")
> ])
>
> -;;
> -;; [vrshlq_s, vrshlq_u])
> -;;
> -(define_insn "mve_vrshlq_<supf><mode>"
> - [
> - (set (match_operand:MVE_2 0 "s_register_operand" "=w")
> - (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand"
> "w")
> - (match_operand:MVE_2 2 "s_register_operand" "w")]
> - VRSHLQ))
> - ]
> - "TARGET_HAVE_MVE"
> - "vrshl.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
> - [(set_attr "type" "mve_move")
> -])
> -
> ;;
> ;; [vrshrq_n_s, vrshrq_n_u])
> ;;
> diff --git a/gcc/config/arm/vec-common.md b/gcc/config/arm/vec-
> common.md
> index f06df4db636..918338ca5c0 100644
> --- a/gcc/config/arm/vec-common.md
> +++ b/gcc/config/arm/vec-common.md
> @@ -573,7 +573,7 @@ (define_expand "avg<mode>3_floor"
> "ARM_HAVE_<MODE>_ARITH"
> {
> if (TARGET_HAVE_MVE)
> - emit_insn (gen_mve_vhaddq (VHADDQ_S, <MODE>mode,
> + emit_insn (gen_mve_q (VHADDQ_S, VHADDQ_S, <MODE>mode,
> operands[0], operands[1], operands[2]));
> else
> emit_insn (gen_neon_vhadd (UNSPEC_VHADD_S, UNSPEC_VHADD_S,
> <MODE>mode,
> @@ -588,7 +588,7 @@ (define_expand "uavg<mode>3_floor"
> "ARM_HAVE_<MODE>_ARITH"
> {
> if (TARGET_HAVE_MVE)
> - emit_insn (gen_mve_vhaddq (VHADDQ_U, <MODE>mode,
> + emit_insn (gen_mve_q (VHADDQ_U, VHADDQ_U, <MODE>mode,
> operands[0], operands[1], operands[2]));
> else
> emit_insn (gen_neon_vhadd (UNSPEC_VHADD_U, UNSPEC_VHADD_U,
> <MODE>mode,
> @@ -603,7 +603,7 @@ (define_expand "avg<mode>3_ceil"
> "ARM_HAVE_<MODE>_ARITH"
> {
> if (TARGET_HAVE_MVE)
> - emit_insn (gen_mve_vrhaddq (VRHADDQ_S, <MODE>mode,
> + emit_insn (gen_mve_q (VRHADDQ_S, VRHADDQ_S, <MODE>mode,
> operands[0], operands[1], operands[2]));
> else
> emit_insn (gen_neon_vhadd (UNSPEC_VRHADD_S, UNSPEC_VRHADD_S,
> <MODE>mode,
> @@ -618,7 +618,7 @@ (define_expand "uavg<mode>3_ceil"
> "ARM_HAVE_<MODE>_ARITH"
> {
> if (TARGET_HAVE_MVE)
> - emit_insn (gen_mve_vrhaddq (VRHADDQ_U, <MODE>mode,
> + emit_insn (gen_mve_q (VRHADDQ_U, VRHADDQ_U, <MODE>mode,
> operands[0], operands[1], operands[2]));
> else
> emit_insn (gen_neon_vhadd (UNSPEC_VRHADD_U, UNSPEC_VRHADD_U,
> <MODE>mode,
> --
> 2.34.1
@@ -414,6 +414,22 @@ (define_int_iterator MVE_INT_SU_N_BINARY [
VQSUBQ_N_S VQSUBQ_N_U
])
+(define_int_iterator MVE_INT_SU_BINARY [
+ VABDQ_S VABDQ_U
+ VHADDQ_S VHADDQ_U
+ VHSUBQ_S VHSUBQ_U
+ VMULHQ_S VMULHQ_U
+ VQADDQ_S VQADDQ_U
+ VQDMULHQ_S
+ VQRDMULHQ_S
+ VQRSHLQ_S VQRSHLQ_U
+ VQSHLQ_S VQSHLQ_U
+ VQSUBQ_S VQSUBQ_U
+ VRHADDQ_S VRHADDQ_U
+ VRMULHQ_S VRMULHQ_U
+ VRSHLQ_S VRSHLQ_U
+ ])
+
(define_int_iterator MVE_INT_N_BINARY_LOGIC [
VBICQ_N_S VBICQ_N_U
VORRQ_N_S VORRQ_N_U
@@ -456,6 +472,7 @@ (define_code_attr mve_addsubmul [
(define_int_attr mve_insn [
(VABDQ_M_S "vabd") (VABDQ_M_U "vabd")
+ (VABDQ_S "vabd") (VABDQ_U "vabd")
(VADDQ_M_N_S "vadd") (VADDQ_M_N_U "vadd") (VADDQ_M_N_F "vadd")
(VADDQ_M_S "vadd") (VADDQ_M_U "vadd") (VADDQ_M_F "vadd")
(VADDQ_N_S "vadd") (VADDQ_N_U "vadd") (VADDQ_N_F "vadd")
@@ -468,14 +485,17 @@ (define_int_attr mve_insn [
(VHADDQ_M_N_S "vhadd") (VHADDQ_M_N_U "vhadd")
(VHADDQ_M_S "vhadd") (VHADDQ_M_U "vhadd")
(VHADDQ_N_S "vhadd") (VHADDQ_N_U "vhadd")
+ (VHADDQ_S "vhadd") (VHADDQ_U "vhadd")
(VHSUBQ_M_N_S "vhsub") (VHSUBQ_M_N_U "vhsub")
(VHSUBQ_M_S "vhsub") (VHSUBQ_M_U "vhsub")
(VHSUBQ_N_S "vhsub") (VHSUBQ_N_U "vhsub")
+ (VHSUBQ_S "vhsub") (VHSUBQ_U "vhsub")
(VMAXQ_M_S "vmax") (VMAXQ_M_U "vmax")
(VMINQ_M_S "vmin") (VMINQ_M_U "vmin")
(VMLAQ_M_N_S "vmla") (VMLAQ_M_N_U "vmla")
(VMLASQ_M_N_S "vmlas") (VMLASQ_M_N_U "vmlas")
(VMULHQ_M_S "vmulh") (VMULHQ_M_U "vmulh")
+ (VMULHQ_S "vmulh") (VMULHQ_U "vmulh")
(VMULQ_M_N_S "vmul") (VMULQ_M_N_U "vmul") (VMULQ_M_N_F "vmul")
(VMULQ_M_S "vmul") (VMULQ_M_U "vmul") (VMULQ_M_F "vmul")
(VMULQ_N_S "vmul") (VMULQ_N_U "vmul") (VMULQ_N_F "vmul")
@@ -485,6 +505,7 @@ (define_int_attr mve_insn [
(VQADDQ_M_N_S "vqadd") (VQADDQ_M_N_U "vqadd")
(VQADDQ_M_S "vqadd") (VQADDQ_M_U "vqadd")
(VQADDQ_N_S "vqadd") (VQADDQ_N_U "vqadd")
+ (VQADDQ_S "vqadd") (VQADDQ_U "vqadd")
(VQDMLADHQ_M_S "vqdmladh")
(VQDMLADHXQ_M_S "vqdmladhx")
(VQDMLAHQ_M_N_S "vqdmlah")
@@ -494,6 +515,7 @@ (define_int_attr mve_insn [
(VQDMULHQ_M_N_S "vqdmulh")
(VQDMULHQ_M_S "vqdmulh")
(VQDMULHQ_N_S "vqdmulh")
+ (VQDMULHQ_S "vqdmulh")
(VQRDMLADHQ_M_S "vqrdmladh")
(VQRDMLADHXQ_M_S "vqrdmladhx")
(VQRDMLAHQ_M_N_S "vqrdmlah")
@@ -503,14 +525,21 @@ (define_int_attr mve_insn [
(VQRDMULHQ_M_N_S "vqrdmulh")
(VQRDMULHQ_M_S "vqrdmulh")
(VQRDMULHQ_N_S "vqrdmulh")
+ (VQRDMULHQ_S "vqrdmulh")
(VQRSHLQ_M_S "vqrshl") (VQRSHLQ_M_U "vqrshl")
+ (VQRSHLQ_S "vqrshl") (VQRSHLQ_U "vqrshl")
(VQSHLQ_M_S "vqshl") (VQSHLQ_M_U "vqshl")
+ (VQSHLQ_S "vqshl") (VQSHLQ_U "vqshl")
(VQSUBQ_M_N_S "vqsub") (VQSUBQ_M_N_U "vqsub")
(VQSUBQ_M_S "vqsub") (VQSUBQ_M_U "vqsub")
(VQSUBQ_N_S "vqsub") (VQSUBQ_N_U "vqsub")
+ (VQSUBQ_S "vqsub") (VQSUBQ_U "vqsub")
(VRHADDQ_M_S "vrhadd") (VRHADDQ_M_U "vrhadd")
+ (VRHADDQ_S "vrhadd") (VRHADDQ_U "vrhadd")
(VRMULHQ_M_S "vrmulh") (VRMULHQ_M_U "vrmulh")
+ (VRMULHQ_S "vrmulh") (VRMULHQ_U "vrmulh")
(VRSHLQ_M_S "vrshl") (VRSHLQ_M_U "vrshl")
+ (VRSHLQ_S "vrshl") (VRSHLQ_U "vrshl")
(VSHLQ_M_S "vshl") (VSHLQ_M_U "vshl")
(VSUBQ_M_N_S "vsub") (VSUBQ_M_N_U "vsub") (VSUBQ_M_N_F "vsub")
(VSUBQ_M_S "vsub") (VSUBQ_M_U "vsub") (VSUBQ_M_F "vsub")
@@ -1669,6 +1698,8 @@ (define_int_attr supf [(VCVTQ_TO_F_S "s") (VCVTQ_TO_F_U "u") (VREV16Q_S "s")
(VQRDMLASHQ_M_N_S "s")
(VQDMULHQ_M_N_S "s")
(VQRDMULHQ_M_N_S "s")
+ (VQDMULHQ_S "s")
+ (VQRDMULHQ_S "s")
])
;; Both kinds of return insn.
@@ -841,16 +841,28 @@ (define_insn "mve_vcmp<mve_cmp_op>q_n_<mode>"
;;
;; [vabdq_s, vabdq_u])
+;; [vhaddq_s, vhaddq_u])
+;; [vhsubq_s, vhsubq_u])
+;; [vmulhq_s, vmulhq_u])
+;; [vqaddq_u, vqaddq_s])
+;; [vqdmulhq_s])
+;; [vqrdmulhq_s])
+;; [vqrshlq_s, vqrshlq_u])
+;; [vqshlq_s, vqshlq_u])
+;; [vqsubq_u, vqsubq_s])
+;; [vrhaddq_s, vrhaddq_u])
+;; [vrmulhq_s, vrmulhq_u])
+;; [vrshlq_s, vrshlq_u])
;;
-(define_insn "mve_vabdq_<supf><mode>"
+(define_insn "@mve_<mve_insn>q_<supf><mode>"
[
(set (match_operand:MVE_2 0 "s_register_operand" "=w")
(unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
(match_operand:MVE_2 2 "s_register_operand" "w")]
- VABDQ))
+ MVE_INT_SU_BINARY))
]
"TARGET_HAVE_MVE"
- "vabd.<supf>%#<V_sz_elem> %q0, %q1, %q2"
+ "<mve_insn>.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
[(set_attr "type" "mve_move")
])
@@ -1033,21 +1045,6 @@ (define_insn "@mve_<mve_insn>q_n_<supf><mode>"
[(set_attr "type" "mve_move")
])
-;;
-;; [vhaddq_s, vhaddq_u])
-;;
-(define_insn "@mve_vhaddq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VHADDQ))
- ]
- "TARGET_HAVE_MVE"
- "vhadd.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
;;
;; [vhcaddq_rot270_s])
;;
@@ -1078,21 +1075,6 @@ (define_insn "mve_vhcaddq_rot90_s<mode>"
[(set_attr "type" "mve_move")
])
-;;
-;; [vhsubq_s, vhsubq_u])
-;;
-(define_insn "mve_vhsubq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VHSUBQ))
- ]
- "TARGET_HAVE_MVE"
- "vhsub.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
;;
;; [vmaxaq_s])
;;
@@ -1293,21 +1275,6 @@ (define_insn "mve_vmlsdavxq_s<mode>"
[(set_attr "type" "mve_move")
])
-;;
-;; [vmulhq_s, vmulhq_u])
-;;
-(define_insn "mve_vmulhq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VMULHQ))
- ]
- "TARGET_HAVE_MVE"
- "vmulh.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
;;
;; [vmullbq_int_u, vmullbq_int_s])
;;
@@ -1405,51 +1372,6 @@ (define_expand "mve_vorrq_u<mode>"
"TARGET_HAVE_MVE"
)
-;;
-;; [vqaddq_u, vqaddq_s])
-;;
-(define_insn "mve_vqaddq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VQADDQ))
- ]
- "TARGET_HAVE_MVE"
- "vqadd.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
-;;
-;; [vqdmulhq_s])
-;;
-(define_insn "mve_vqdmulhq_s<mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VQDMULHQ_S))
- ]
- "TARGET_HAVE_MVE"
- "vqdmulh.s%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
-;;
-;; [vqrdmulhq_s])
-;;
-(define_insn "mve_vqrdmulhq_s<mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VQRDMULHQ_S))
- ]
- "TARGET_HAVE_MVE"
- "vqrdmulh.s%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
;;
;; [vqrshlq_n_s, vqrshlq_n_u])
;;
@@ -1465,21 +1387,6 @@ (define_insn "mve_vqrshlq_n_<supf><mode>"
[(set_attr "type" "mve_move")
])
-;;
-;; [vqrshlq_s, vqrshlq_u])
-;;
-(define_insn "mve_vqrshlq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VQRSHLQ))
- ]
- "TARGET_HAVE_MVE"
- "vqrshl.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
;;
;; [vqshlq_n_s, vqshlq_n_u])
;;
@@ -1510,21 +1417,6 @@ (define_insn "mve_vqshlq_r_<supf><mode>"
[(set_attr "type" "mve_move")
])
-;;
-;; [vqshlq_s, vqshlq_u])
-;;
-(define_insn "mve_vqshlq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VQSHLQ))
- ]
- "TARGET_HAVE_MVE"
- "vqshl.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
;;
;; [vqshluq_n_s])
;;
@@ -1540,51 +1432,6 @@ (define_insn "mve_vqshluq_n_s<mode>"
[(set_attr "type" "mve_move")
])
-;;
-;; [vqsubq_u, vqsubq_s])
-;;
-(define_insn "mve_vqsubq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VQSUBQ))
- ]
- "TARGET_HAVE_MVE"
- "vqsub.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
-;;
-;; [vrhaddq_s, vrhaddq_u])
-;;
-(define_insn "@mve_vrhaddq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VRHADDQ))
- ]
- "TARGET_HAVE_MVE"
- "vrhadd.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
-;;
-;; [vrmulhq_s, vrmulhq_u])
-;;
-(define_insn "mve_vrmulhq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VRMULHQ))
- ]
- "TARGET_HAVE_MVE"
- "vrmulh.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
;;
;; [vrshlq_n_u, vrshlq_n_s])
;;
@@ -1600,21 +1447,6 @@ (define_insn "mve_vrshlq_n_<supf><mode>"
[(set_attr "type" "mve_move")
])
-;;
-;; [vrshlq_s, vrshlq_u])
-;;
-(define_insn "mve_vrshlq_<supf><mode>"
- [
- (set (match_operand:MVE_2 0 "s_register_operand" "=w")
- (unspec:MVE_2 [(match_operand:MVE_2 1 "s_register_operand" "w")
- (match_operand:MVE_2 2 "s_register_operand" "w")]
- VRSHLQ))
- ]
- "TARGET_HAVE_MVE"
- "vrshl.<supf>%#<V_sz_elem>\t%q0, %q1, %q2"
- [(set_attr "type" "mve_move")
-])
-
;;
;; [vrshrq_n_s, vrshrq_n_u])
;;
@@ -573,7 +573,7 @@ (define_expand "avg<mode>3_floor"
"ARM_HAVE_<MODE>_ARITH"
{
if (TARGET_HAVE_MVE)
- emit_insn (gen_mve_vhaddq (VHADDQ_S, <MODE>mode,
+ emit_insn (gen_mve_q (VHADDQ_S, VHADDQ_S, <MODE>mode,
operands[0], operands[1], operands[2]));
else
emit_insn (gen_neon_vhadd (UNSPEC_VHADD_S, UNSPEC_VHADD_S, <MODE>mode,
@@ -588,7 +588,7 @@ (define_expand "uavg<mode>3_floor"
"ARM_HAVE_<MODE>_ARITH"
{
if (TARGET_HAVE_MVE)
- emit_insn (gen_mve_vhaddq (VHADDQ_U, <MODE>mode,
+ emit_insn (gen_mve_q (VHADDQ_U, VHADDQ_U, <MODE>mode,
operands[0], operands[1], operands[2]));
else
emit_insn (gen_neon_vhadd (UNSPEC_VHADD_U, UNSPEC_VHADD_U, <MODE>mode,
@@ -603,7 +603,7 @@ (define_expand "avg<mode>3_ceil"
"ARM_HAVE_<MODE>_ARITH"
{
if (TARGET_HAVE_MVE)
- emit_insn (gen_mve_vrhaddq (VRHADDQ_S, <MODE>mode,
+ emit_insn (gen_mve_q (VRHADDQ_S, VRHADDQ_S, <MODE>mode,
operands[0], operands[1], operands[2]));
else
emit_insn (gen_neon_vhadd (UNSPEC_VRHADD_S, UNSPEC_VRHADD_S, <MODE>mode,
@@ -618,7 +618,7 @@ (define_expand "uavg<mode>3_ceil"
"ARM_HAVE_<MODE>_ARITH"
{
if (TARGET_HAVE_MVE)
- emit_insn (gen_mve_vrhaddq (VRHADDQ_U, <MODE>mode,
+ emit_insn (gen_mve_q (VRHADDQ_U, VRHADDQ_U, <MODE>mode,
operands[0], operands[1], operands[2]));
else
emit_insn (gen_neon_vhadd (UNSPEC_VRHADD_U, UNSPEC_VRHADD_U, <MODE>mode,