From patchwork Sat Feb 19 19:07:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom de Vries X-Patchwork-Id: 51241 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id EE7293851C2A for ; Sat, 19 Feb 2022 19:07:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EE7293851C2A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1645297670; bh=ElmR/QRGAeWvrCxeB/X6fEVmR/xTX0a5M0GbLX3BNgw=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=L4bbwfDLRnrEH8cJ3bVer7B9n3t90Ahg6fo+S9K9VbVaimQ8sREqD4tcLW7ehBW03 4KEakKNt7Uo3TRZF4ZgOk++RzDDu6qsWRwU2aOd6bIQewq4jDOs3uC2D/Msbbg55c3 I9+Ic1t9ASSUBGohKptiVSWp5OBdcqLvEn14RVBw= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id DBAC73858D1E for ; Sat, 19 Feb 2022 19:07:20 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org DBAC73858D1E Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id CEE02219A7 for ; Sat, 19 Feb 2022 19:07:19 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B41E11332F for ; Sat, 19 Feb 2022 19:07:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id P2weKuc/EWLCGwAAMHmgww (envelope-from ) for ; Sat, 19 Feb 2022 19:07:19 +0000 Date: Sat, 19 Feb 2022 20:07:18 +0100 To: gcc-patches@gcc.gnu.org Subject: [committed][nvptx] Use nvptx_warpsync / nvptx_uniform_warp_check for -muniform-simt Message-ID: <20220219190716.GA1470@delia.home> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-12.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_LOW, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tom de Vries via Gcc-patches From: Tom de Vries Reply-To: Tom de Vries Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Hi, With the default ptx isa 6.0, we have for uniform-simt-1.c: ... @%r33 atom.global.cas.b32 %r26, [a], %r28, %r29; shfl.sync.idx.b32 %r26, %r26, %r32, 31, 0xffffffff; ... The atomic insn is predicated by -muniform-simt, and the subsequent insn does a warp sync, at which point the warp is uniform again. But with -mptx=3.1, we have instead: ... @%r33 atom.global.cas.b32 %r26, [a], %r28, %r29; shfl.idx.b32 %r26, %r26, %r32, 31; ... The shfl does not sync the warp, and we want the warp to go back to executing uniformly asap. We cannot enforce this, but at least check this using nvptx_uniform_warp_check, similar to how that is done for openacc. Likewise, detect the case that no shfl insn is emitted, and add a nvptx_uniform_warp_check or nvptx_warpsync. Committed to trunk. Thanks, - Tom [nvptx] Use nvptx_warpsync / nvptx_uniform_warp_check for -muniform-simt gcc/ChangeLog: 2022-02-19 Tom de Vries * config/nvptx/nvptx.cc (nvptx_unisimt_handle_set): Change return type to bool. (nvptx_reorg_uniform_simt): Insert nvptx_uniform_warp_check or nvptx_warpsync, if necessary. gcc/testsuite/ChangeLog: 2022-02-19 Tom de Vries * gcc.target/nvptx/uniform-simt-1.c: Add scan-assembler test. * gcc.target/nvptx/uniform-simt-2.c: New test. --- gcc/config/nvptx/nvptx.cc | 34 ++++++++++++++++++++++--- gcc/testsuite/gcc.target/nvptx/uniform-simt-1.c | 1 + gcc/testsuite/gcc.target/nvptx/uniform-simt-2.c | 20 +++++++++++++++ 3 files changed, 52 insertions(+), 3 deletions(-) diff --git a/gcc/config/nvptx/nvptx.cc b/gcc/config/nvptx/nvptx.cc index afbad5bdde6..4942f1100da 100644 --- a/gcc/config/nvptx/nvptx.cc +++ b/gcc/config/nvptx/nvptx.cc @@ -3248,12 +3248,18 @@ nvptx_call_insn_is_syscall_p (rtx_insn *insn) /* If SET subexpression of INSN sets a register, emit a shuffle instruction to propagate its value from lane MASTER to current lane. */ -static void +static bool nvptx_unisimt_handle_set (rtx set, rtx_insn *insn, rtx master) { rtx reg; if (GET_CODE (set) == SET && REG_P (reg = SET_DEST (set))) - emit_insn_after (nvptx_gen_shuffle (reg, reg, master, SHUFFLE_IDX), insn); + { + emit_insn_after (nvptx_gen_shuffle (reg, reg, master, SHUFFLE_IDX), + insn); + return true; + } + + return false; } /* Adjust code for uniform-simt code generation variant by making atomics and @@ -3275,8 +3281,30 @@ nvptx_reorg_uniform_simt () continue; rtx pat = PATTERN (insn); rtx master = nvptx_get_unisimt_master (); + bool shuffle_p = false; for (int i = 0; i < XVECLEN (pat, 0); i++) - nvptx_unisimt_handle_set (XVECEXP (pat, 0, i), insn, master); + shuffle_p + |= nvptx_unisimt_handle_set (XVECEXP (pat, 0, i), insn, master); + if (shuffle_p && TARGET_PTX_6_0) + { + /* The shuffle is a sync, so uniformity is guaranteed. */ + } + else + { + if (TARGET_PTX_6_0) + { + gcc_assert (!shuffle_p); + /* Emit after the insn, to guarantee uniformity. */ + emit_insn_after (gen_nvptx_warpsync (), insn); + } + else + { + /* Emit after the insn (and before the shuffle, if there are any) + to check uniformity. */ + emit_insn_after (gen_nvptx_uniform_warp_check (), insn); + } + } + rtx pred = nvptx_get_unisimt_predicate (); pred = gen_rtx_NE (BImode, pred, const0_rtx); pat = gen_rtx_COND_EXEC (VOIDmode, pred, pat); diff --git a/gcc/testsuite/gcc.target/nvptx/uniform-simt-1.c b/gcc/testsuite/gcc.target/nvptx/uniform-simt-1.c index 1bc0adae014..77cffc40a66 100644 --- a/gcc/testsuite/gcc.target/nvptx/uniform-simt-1.c +++ b/gcc/testsuite/gcc.target/nvptx/uniform-simt-1.c @@ -16,3 +16,4 @@ f (void) } /* { dg-final { scan-assembler-times "@%r\[0-9\]*\tatom.global.cas" 1 } } */ +/* { dg-final { scan-assembler-times "shfl.sync.idx.b32" 1 } } */ diff --git a/gcc/testsuite/gcc.target/nvptx/uniform-simt-2.c b/gcc/testsuite/gcc.target/nvptx/uniform-simt-2.c new file mode 100644 index 00000000000..0f1e4e780fe --- /dev/null +++ b/gcc/testsuite/gcc.target/nvptx/uniform-simt-2.c @@ -0,0 +1,20 @@ +/* { dg-options "-O2 -muniform-simt -mptx=3.1" } */ + +enum memmodel +{ + MEMMODEL_RELAXED = 0, +}; + +int a = 0; + +int +f (void) +{ + int expected = 1; + return __atomic_compare_exchange_n (&a, &expected, 0, 0, MEMMODEL_RELAXED, + MEMMODEL_RELAXED); +} + +/* { dg-final { scan-assembler-times "@%r\[0-9\]*\tatom.global.cas" 1 } } */ +/* { dg-final { scan-assembler-times "shfl.idx.b32" 1 } } */ +/* { dg-final { scan-assembler-times "vote.ballot.b32" 1 } } */