From patchwork Wed Jul 8 14:46:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Stubbs X-Patchwork-Id: 138757 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id F21834BA2E18 for ; Wed, 8 Jul 2026 14:49:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org F21834BA2E18 Authentication-Results: sourceware.org; dkim=pass (2048-bit key, secure) header.d=baylibre.com header.i=@baylibre.com header.a=rsa-sha256 header.s=google header.b=Y5aoOKb/ X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by sourceware.org (Postfix) with ESMTPS id E67974BA2E05 for ; Wed, 8 Jul 2026 14:46:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E67974BA2E05 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=baylibre.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=baylibre.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E67974BA2E05 Authentication-Results: sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::32e ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1783522009; cv=none; b=wCFZDlDW/c+vZomM6eiLpS1ygxeOjtTbJF2NfT1gKs+HJRph5KjGYaNjM19879MCNy2LOov2wj87q9+XWx/9r610S8zFd/zEJrRZvbDskZnZCLcA/3JTtki2RfU8xYJzPpy+m3tbabOkdU7TZ0i9vv+FwB85TmpMIjG5u87lPv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1783522009; c=relaxed/simple; bh=NTFVxpgA68LrJKyuSv+SgPC+eCfYOipuhOK0zfNHFPQ=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=IMfvFGbV5if6eZKpk8PImvf7p4i2H1wG++H/WzAPezt8wSW/zVI+Fzw7HgTilH8/Ms+ZOULZQlYljcuS7F3zFl5XV4FkdHC9RhXVy4WVJl0Rhy9Sx4h9ITt5Sdxrb5rJgTJKFdA/E7c669ViaGB4WbKdxTuQqSh9YNG3I4cpL1M= ARC-Authentication-Results: i=1; sourceware.org; dkim=pass (2048-bit key, secure) header.d=baylibre.com header.i=@baylibre.com header.a=rsa-sha256 header.s=google header.b=Y5aoOKb/ DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E67974BA2E05 Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-490cf3000f0so5476545e9.1 for ; Wed, 08 Jul 2026 07:46:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre.com; s=google; t=1783522008; x=1784126808; darn=gcc.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=moS302CW4i84Z0NnuPkuA1twVdjtVOLIGaoih1M1BrE=; b=Y5aoOKb/kpo6xLFLmU1Zxvu37NLsEoLZHI680h91YMRDUc2m6VGqCFZokpl+3zumFL Fg/Y5ymwSZ3ibxXiOldK/cxmT/gPrGHmIseLpSSqIzqdGzv6DdzJ0IRRb4/69mfQLUvB Tg1p/RK1Sk/BsmQF8AUSBhul6xJYCuuGw4pMIO4AOizkgw5naYCGWWsMLC+hIOrcpx3J E60aLblTMGzkWES0HOp35fl4B+kGvSex5ETY7PNLj/6zplnjfenAN1Jc9FrfLw4n+d0N M7erx6rR6RgTYWZCwdt0G6tu9svhNISMVxuI7F4Ft5jXtPkGsjBm+VmZXN/r1D+UHvU5 MHxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783522008; x=1784126808; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=moS302CW4i84Z0NnuPkuA1twVdjtVOLIGaoih1M1BrE=; b=VlXLZ7MBGjyVlHkyzBP/W3TwvS81y8phakdESUpjaa3epNqAFQYHsogLSYSRCb5+O8 coSHyNV7Br8M/KsM9lcJcFQ3xEccqHpUuX7bkBdSWxlHmYvyB7xAb69l3ZK/+jsdsJl9 S45RpkKSe/9eoe9KGG4eZfO2vswqCvgJKWbEcnTHOJxy+qowCF15BbB3F1yRMKg64O+t vwooVaqZoCPAAjE3azg1o8a5tYHmJMxx0J2dlRT8RC7SOog4wmo2w6A7W4NDNM2htgQk AalCDQVNqNef7papJbeB5CQx/ZwqYh1U2V6n3SgaAoUbFA7WiZrzUXwy2t3pBm8L9I7d 0CWg== X-Gm-Message-State: AOJu0YzMKsdJEmptNSRwJLsHni4l6mT2vEK0zlBsj80a2e7VwoVuWNjr BGNQgv3prAhA+62p8iDwu64h6bfiAfViAMDjR4yXQBnvpWFWy1C33ifJCnDUBL73WhtWzC/33Rz eDJer X-Gm-Gg: AfdE7cm5Vt8+xWQWN9KDdl3u6CKOKGYvKqFIexBC1MC3Wpt6LTg0Lo5rb9BCeaUceDz Tcy7BGZPhHeXKuWHSVPbCLHueKX98aNK1bdknfipHckgNCuBmpcwv8+8SY2BpCBHOkfk0CaEa+K Nw9klY3V84fwgK27kjMzfv3gzQF4e0sInVWLwlUkITWiqnXCSY/a33TOebeRMI1Xe6uD30F8rab 0jXvBdeuRLoikLQra7DE5wu3gjqdc1zZiVdK22aBL6htMgRN2wgg4ecVxpkmkJgEOTcekrS86Cw hrxeVXR4gPKSiZ7jtLHllACpf5aHNriiYEH5R1SFy+/KwmakQT2k9qK3bDpvHCeDN+M8pda4GJK j4LCl+acMl74KXuaH/6qwXqxzTXecFGw4/4+jaQr97mPKPtOaCKBaQl3RV9rgR8Lmmgekz3A3ab FrsBSvMhc1CJPZm7xZVOoMST6gIw== X-Received: by 2002:a05:600d:84ca:20b0:493:c984:db9c with SMTP id 5b1f17b1804b1-493e6898b54mr22011365e9.2.1783522007526; Wed, 08 Jul 2026 07:46:47 -0700 (PDT) Received: from vbuild-02.baylibre ([217.13.61.132]) by smtp.googlemail.com with ESMTPSA id ffacd0b85a97d-47a9e4d83bdsm43362109f8f.13.2026.07.08.07.46.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2026 07:46:47 -0700 (PDT) From: Andrew Stubbs To: gcc-patches@gcc.gnu.org Subject: [PATCH 3/3] amdgcn: Add vector atomics Date: Wed, 8 Jul 2026 14:46:33 +0000 Message-ID: <20260708144633.1530935-4-ams@baylibre.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260708144633.1530935-1-ams@baylibre.com> References: <20260708144633.1530935-1-ams@baylibre.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org This patch utilizes the new "(mem (reg:))" support to add vector atomics. These work exactly like the scalar atomic operation, but 64 times in parallel. gcc/ChangeLog: * config/gcn/gcn.md (UNSPEC_PAIR): New. (X): Add vector modes. (ATOMICMODE): New mode iterator. (atomic_fetch_): Use ATOMICMODE iterator. (atomic_): Likewise. (x2): Add vector modes. (size): Delete. (bitsize): Add vector modes. (createpair): New insn. (sync_compare_and_swap): Use ATOMICMODE iterator. (sync_compare_and_swap_insn): Likewise. (sync_compare_and_swap_lds_insn): Likewise. (atomic_load): Likewise. (atomic_store): Likewise. (atomic_exchange): Likewise. --- gcc/config/gcn/gcn.md | 114 +++++++++++++++++++++++++----------------- 1 file changed, 69 insertions(+), 45 deletions(-) diff --git a/gcc/config/gcn/gcn.md b/gcc/config/gcn/gcn.md index 4c56b1b5e96..75949f64dea 100644 --- a/gcc/config/gcn/gcn.md +++ b/gcc/config/gcn/gcn.md @@ -86,7 +86,8 @@ (define_c_enum "unspec" [ UNSPEC_FLBIT_INT UNSPEC_FLOOR UNSPEC_CEIL UNSPEC_SIN UNSPEC_COS UNSPEC_EXP2 UNSPEC_LOG2 UNSPEC_LDEXP UNSPEC_FREXP_EXP UNSPEC_FREXP_MANT - UNSPEC_DIV_SCALE UNSPEC_DIV_FMAS UNSPEC_DIV_FIXUP]) + UNSPEC_DIV_SCALE UNSPEC_DIV_FMAS UNSPEC_DIV_FIXUP + UNSPEC_PAIR]) ;; }}} ;; {{{ Attributes @@ -1921,7 +1922,11 @@ (define_expand "ti3" ; the programmer to get this right. (define_code_iterator atomicops [plus minus and ior xor]) -(define_mode_attr X [(SI "") (DI "_X2")]) +(define_mode_attr X [(SI "") (V64SI "") + (DI "_X2") (V64DI "_X2")]) + +(define_mode_iterator ATOMICMODE + [SI DI V64SI V64DI]) ;; TODO compare_and_swap test_and_set inc dec ;; Hardware also supports min and max, but GCC does not. @@ -1963,13 +1968,13 @@ (define_insn "*memory_barrier" ; reliably - they can cause hangs or incorrect results. ; TODO: flush caches according to memory model (define_insn "atomic_fetch_" - [(set (match_operand:SIDI 0 "register_operand" "=Sm, v, v") - (match_operand:SIDI 1 "memory_operand" "+RS,RF,RM")) + [(set (match_operand:ATOMICMODE 0 "register_operand" "=Sm, v, v") + (match_operand:ATOMICMODE 1 "memory_operand" "+RS,RfRF,RmRM")) (set (match_dup 1) - (unspec_volatile:SIDI - [(atomicops:SIDI + (unspec_volatile:ATOMICMODE + [(atomicops:ATOMICMODE (match_dup 1) - (match_operand:SIDI 2 "register_operand" " Sm, v, v"))] + (match_operand:ATOMICMODE 2 "register_operand" " Sm, v, v"))] UNSPECV_ATOMIC)) (use (match_operand 3 "const_int_operand"))] "0 /* Disabled. */" @@ -1987,11 +1992,11 @@ (define_insn "atomic_fetch_" ; you might expect from a concurrent non-atomic read-modify-write. ; TODO: flush caches according to memory model (define_insn "atomic_" - [(set (match_operand:SIDI 0 "memory_operand" "+RS,RF,RM") - (unspec_volatile:SIDI - [(atomicops:SIDI + [(set (match_operand:ATOMICMODE 0 "memory_operand" "+RS,RfRF,RmRM") + (unspec_volatile:ATOMICMODE + [(atomicops:ATOMICMODE (match_dup 0) - (match_operand:SIDI 1 "register_operand" " Sm, v, v"))] + (match_operand:ATOMICMODE 1 "register_operand" " Sm, v, v"))] UNSPECV_ATOMIC)) (use (match_operand 2 "const_int_operand"))] "0 /* Disabled. */" @@ -2003,15 +2008,35 @@ (define_insn "atomic_" (set_attr "flatmemaccess" "*,atomicwait,atomicwait") (set_attr "length" "12")]) -(define_mode_attr x2 [(SI "DI") (DI "TI")]) -(define_mode_attr size [(SI "4") (DI "8")]) -(define_mode_attr bitsize [(SI "32") (DI "64")]) +(define_mode_attr x2 [(SI "DI") (DI "TI") (V64SI "V64DI") (V64DI "V64TI")]) +(define_mode_attr bitsize [(SI "32") (DI "64") (V64SI "32") (V64DI "64")]) + +(define_insn_and_split "createpair" + [(set (match_operand: 0 "register_operand" "=&v,&Sm") + (unspec: [(match_operand:ATOMICMODE 1 "register_operand" "v, Sm") + (match_operand:ATOMICMODE 2 "register_operand" "v, Sm")] + UNSPEC_PAIR))] + "" + "#" + "reload_completed" + [(const_int 0)] + { + int parts = / 32; + int outpart = 0; + for (int inpart = 0; inpart < parts; inpart++, outpart++) + emit_move_insn (gcn_operand_part (mode, operands[0], outpart), + gcn_operand_part (mode, operands[1], inpart)); + for (int inpart = 0; inpart < parts; inpart++, outpart++) + emit_move_insn (gcn_operand_part (mode, operands[0], outpart), + gcn_operand_part (mode, operands[2], inpart)); + DONE; + }) (define_expand "sync_compare_and_swap" - [(match_operand:SIDI 0 "register_operand") - (match_operand:SIDI 1 "memory_operand") - (match_operand:SIDI 2 "register_operand") - (match_operand:SIDI 3 "register_operand")] + [(match_operand:ATOMICMODE 0 "register_operand") + (match_operand:ATOMICMODE 1 "memory_operand") + (match_operand:ATOMICMODE 2 "register_operand") + (match_operand:ATOMICMODE 3 "register_operand")] "" { if (MEM_ADDR_SPACE (operands[1]) == ADDR_SPACE_LDS) @@ -2024,22 +2049,21 @@ (define_expand "sync_compare_and_swap" } /* Operands 2 and 3 must be placed in consecutive registers, and passed - as a combined value. */ + as a combined value. Subregs would work for the scalar case, but + not for the vector case. */ rtx src_cmp = gen_reg_rtx (mode); - emit_move_insn (gen_rtx_SUBREG (mode, src_cmp, 0), operands[3]); - emit_move_insn (gen_rtx_SUBREG (mode, src_cmp, ), operands[2]); - emit_insn (gen_sync_compare_and_swap_insn (operands[0], - operands[1], + emit_insn (gen_createpair (src_cmp, operands[3], operands[2])); + emit_insn (gen_sync_compare_and_swap_insn (operands[0], operands[1], src_cmp)); DONE; }) (define_insn "sync_compare_and_swap_insn" - [(set (match_operand:SIDI 0 "register_operand" "=Sm, v, v") - (match_operand:SIDI 1 "memory_operand" "+RS,RF,RM")) + [(set (match_operand:ATOMICMODE 0 "register_operand" "=Sm, v, v") + (match_operand:ATOMICMODE 1 "memory_operand" "+RS,RfRF,RmRM")) (set (match_dup 1) - (unspec_volatile:SIDI - [(match_operand: 2 "register_operand" " Sm, v, v")] + (unspec_volatile:ATOMICMODE + [(match_operand: 2 "register_operand" " Sm, v, v")] UNSPECV_ATOMIC))] "" "@ @@ -2051,14 +2075,14 @@ (define_insn "sync_compare_and_swap_insn" (set_attr "flatmemaccess" "*,cmpswapx2,cmpswapx2")]) (define_insn "sync_compare_and_swap_lds_insn" - [(set (match_operand:SIDI 0 "register_operand" "= v") - (unspec_volatile:SIDI - [(match_operand:SIDI 1 "memory_operand" "+RL")] + [(set (match_operand:ATOMICMODE 0 "register_operand" "= v") + (unspec_volatile:ATOMICMODE + [(match_operand:ATOMICMODE 1 "memory_operand" "+RLRl")] UNSPECV_ATOMIC)) (set (match_dup 1) - (unspec_volatile:SIDI - [(match_operand:SIDI 2 "register_operand" " v") - (match_operand:SIDI 3 "register_operand" " v")] + (unspec_volatile:ATOMICMODE + [(match_operand:ATOMICMODE 2 "register_operand" " v") + (match_operand:ATOMICMODE 3 "register_operand" " v")] UNSPECV_ATOMIC))] "" { @@ -2071,11 +2095,11 @@ (define_insn "sync_compare_and_swap_lds_insn" (set_attr "length" "12")]) (define_insn "atomic_load" - [(set (match_operand:SIDI 0 "register_operand" "=Sm, v, v") - (unspec_volatile:SIDI - [(match_operand:SIDI 1 "memory_operand" " RS,RF,RM")] + [(set (match_operand:ATOMICMODE 0 "register_operand" "=Sm, v, v") + (unspec_volatile:ATOMICMODE + [(match_operand:ATOMICMODE 1 "memory_operand" " RS,RfRF,RmRM")] UNSPECV_ATOMIC)) - (use (match_operand:SIDI 2 "immediate_operand" " i, i, i"))] + (use (match_operand:SI 2 "immediate_operand" " i, i, i"))] "" { /* FIXME: RDNA cache instructions may be too conservative? */ @@ -2173,11 +2197,11 @@ (define_insn "atomic_load" (set_attr "rdna" "no,*,*")]) (define_insn "atomic_store" - [(set (match_operand:SIDI 0 "memory_operand" "=RS,RF,RM") - (unspec_volatile:SIDI - [(match_operand:SIDI 1 "register_operand" " Sm, v, v")] + [(set (match_operand:ATOMICMODE 0 "memory_operand" "=RS,RfRF,RmRM") + (unspec_volatile:ATOMICMODE + [(match_operand:ATOMICMODE 1 "register_operand" " Sm, v, v")] UNSPECV_ATOMIC)) - (use (match_operand:SIDI 2 "immediate_operand" " i, i, i"))] + (use (match_operand:SI 2 "immediate_operand" " i, i, i"))] "" { switch (INTVAL (operands[2])) @@ -2260,11 +2284,11 @@ (define_insn "atomic_store" (set_attr "rdna" "no,*,*")]) (define_insn "atomic_exchange" - [(set (match_operand:SIDI 0 "register_operand" "=Sm, v, v") - (match_operand:SIDI 1 "memory_operand" "+RS,RF,RM")) + [(set (match_operand:ATOMICMODE 0 "register_operand" "=Sm, v, v") + (match_operand:ATOMICMODE 1 "memory_operand" "+RS,RfRF,RmRM")) (set (match_dup 1) - (unspec_volatile:SIDI - [(match_operand:SIDI 2 "register_operand" " Sm, v, v")] + (unspec_volatile:ATOMICMODE + [(match_operand:ATOMICMODE 2 "register_operand" " Sm, v, v")] UNSPECV_ATOMIC)) (use (match_operand 3 "immediate_operand"))] ""