From patchwork Wed Aug 12 01:19:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Turner X-Patchwork-Id: 141146 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1B3614B9DB5A for ; Wed, 12 Aug 2026 01:21:03 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 1B3614B9DB5A Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20251104 header.b=jlGPUMjy X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-yx1-xb12e.google.com (mail-yx1-xb12e.google.com [IPv6:2607:f8b0:4864:20::b12e]) by sourceware.org (Postfix) with ESMTPS id 5B7624BA7986 for ; Wed, 12 Aug 2026 01:20:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5B7624BA7986 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5B7624BA7986 Authentication-Results: sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::b12e ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1786497603; cv=none; b=KhuUVeF1CX8z/Ba3Pf9ZUCk+nFokuZ/ydWSWin4VhIU6THxdOQXtDvIUd4i0yVdCFSqStUv+xZmxwDRThlXGK/Jal7V0BsjU4MVaGJJx2Fpilc8QckbZ/s6iE2SqolzrsZyObyQkcabhri7wCIE7BU4DIes6TLgwoAIvahIehtg= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1786497603; c=relaxed/simple; bh=SdvdiY1oKgd3fwtaIckKMc7RzhFJPAj+xatBiIxDQPQ=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=THpCyZuyT12tMYq2/wXuEiM4xDYHX66poklq9CJUosr/t/MvkYwhAyyPKM4eMvbKC1LuGdXoJ0oVxxwIWiOKhs/hvy+pJcYnttCsl0auombeQMSVRGUqpF1p2ALcFp68iV246HxZlCyf3kZgx5x+F7rGb0cOfHce4xTJm4Om5gQ= ARC-Authentication-Results: i=1; sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20251104 header.b=jlGPUMjy DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5B7624BA7986 Received: by mail-yx1-xb12e.google.com with SMTP id 956f58d0204a3-66807ba2f0fso817252d50.3 for ; Tue, 11 Aug 2026 18:20:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1786497603; x=1787102403; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=EXhffOexjCpeUoBQBPDq/+mLWgr41okxTQJfkqs3S6Q=; b=jlGPUMjy3kCrubaxyaqgEPuIx9q6sHuFweGmgg7WMdDsgp+Zhen2flmkpvX40bjjTu w1h99rACxHOCH4ngk5+JQv6AIwLWUej8gQPZ4DzNqZAlhPhgS7OXalBqgAdjKONaot2v hA4wqhH0U+mmtipuEYAauct7eZQQ9iGU21w9xsArDR64kRm08urOP0yloDhvQlJd4PwU a658074aL0AfSsxLZQ5GocdRGpHx3aKEcOnGHAafKIqdR2mMI2V0BJlJ1CmETS4rS14R 4qBZx8af6aqBTMkVLZ8UIBiUm+8uf73h6/APMwBuaaHi3mMhWqE7PDt7pfAlwJc9WSJd oa4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786497603; x=1787102403; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=EXhffOexjCpeUoBQBPDq/+mLWgr41okxTQJfkqs3S6Q=; b=WjQ9kpXl8vJhO0/O/RoPeo8a0X5IpHhrBZXGpvqXX8TXINYKNbf9v0afMJ04DS7DM4 I2rgF14KcbpdxCiUTGpbGujpzC309Su/jRZnJAM+SirpEYdZzghTuvEjlOmNU7PQ0mDf D1rTwBUNiAtp2JK0kzSj2OUSn271CUJjU62OgK5++n0zqUvVtDoT/eQkUa6rUWK3NW6j 0aKfQWlVgRAtr4jATJdOMsjT6/PuNUiiZ/+SBFX1FrRgVxid3ZbGr+XcW7uIda4ZjiR8 n3Oc9Y/lzHCwMhHxDHiAQhjUN4tSkhdMoem+OmN1fZWTJsN0EacxQxIJd3pA/PPjpQrX 2h9Q== X-Gm-Message-State: AOJu0YyOrX/0qpAiGSHUnS0xY4FxkoZqeJs4tKjnffH8eRLk4VdDT6DC ofYXbBXpZFJc8qcSD2o0IalnQ/2SH5+msr9M/U+GRCGoNm7xrTZtcp2Gq9jzv6LjJ1Q= X-Gm-Gg: AR+sD11KGg7mio6XaS3MKZHxDYT6AuVejyAIorKVspVhUKDCr3WgxWIkSr9cZw9RLxi ZmCbEgNyUcBswkF5Ug48HLOtaIRBEVZ49+Gv6UKCnjLc3ns+LLYPorHeTgFgZ8t97O7uuLpts1C SKxvvsPCR8jH3O8M5jzZMeYrcyBJv2jOmzxvGBzFovgvcWjETI7OzrME1M70R0sSUuYXW2jU3Pa z5vJGlcR42C0SA5tXWgFDaXcCdP77Z89wjHcIQQg7gdwnoya0bAqspSliNOgPW3pDidQZpx6GsJ ioAgbHNHOzmFMBO644BUqtvXjTJnYNnyHxuoO+tGl0cQ9pA/uogh2qV0BDYvi0MO0biVsdCymHH ZvY4BMNKgd2T7+DzSPtMjFwyd2RWKeHhF0bl0MWB3C2tStiUKZxN8sSYsIQPebChMM4Q9SdbYpU bGl6XkNfLUglWLYRvMVyBPLIXkPGiIwsXabaAhmaRTNfGXhTkETRb4jvOkGEs= X-Received: by 2002:a53:b787:0:b0:669:4879:867d with SMTP id 956f58d0204a3-66b34308752mr537169d50.30.1786497602623; Tue, 11 Aug 2026 18:20:02 -0700 (PDT) Received: from localhost ([2600:1702:7a90:6f9f:482:39fe:7891:c084]) by smtp.gmail.com with ESMTPSA id 956f58d0204a3-66b1f2843edsm2115304d50.18.2026.08.11.18.20.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Aug 2026 18:20:01 -0700 (PDT) From: Matt Turner To: libc-alpha@sourceware.org Cc: Magnus Lindholm , Michael Cree Subject: [PATCH 02/20] alpha: issue the ev6 memset wh64 four lines ahead Date: Tue, 11 Aug 2026 21:19:39 -0400 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-10.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org The 21264 memset loop issued its wh64 write hint one 64-byte line ahead of the stores that consume it. That is far too close, and the cost is not subtle: on an EV68CB (DS15, 2MB off-chip Bcache) a cold 256KB memset ran at 453k cycles with the one-line hint against 406k with no hint issued at all, so the wh64 was not merely ineffective but a net loss. Issue it four lines (256 bytes) ahead instead. The same measurement gives 151k cycles, a 3.0x improvement, and the curve is flat from three lines out to eight, so the depth is not delicately tuned -- it only has to clear the threshold. Four lines is at least as fast as one at every size in both the hot and cold cases, so no size range regresses. This matches the CWG, which asks for at least two cache blocks of lead per stream and up to 8/n blocks for n streams against the 21264's 8-entry MAF; a one-line hint was below the architectural minimum. The mechanism is Mbox replay traps, not fill latency. Counted with perf on the 21264 (raw event r4), the one-line hint takes 69885 replays per cold 256KB call and the four-line hint takes 11467, an 83% reduction, while Bcache misses are unchanged at 4162 against 4125. No extra data is moved; the loop was simply trapping and replaying its own store stream, at close to one replay per instruction retired. wh64 has to claim the line before the stores reach it, and unlike a load prefetch there is no fill latency for a late hint to hide behind -- it just collides. The end guard moves with the distance: the hint consumed at trip N+1 covers [$5+256, $5+320), so that trip needs 40 quads remaining, and it is computed at trip N, which therefore needs 48. When fewer remain the target falls back to the block the next trip stores, which is always in range. This matters because wh64 makes the entire 64-byte block UNPREDICTABLE, so a hint that runs past the end of the region would corrupt memory. 320 does not fit the 8-bit addq literal field, so the address computation becomes an lda; both issue on the E box, so the schedule is unchanged. Verified on an EV68CB against the C memset over every size class from 0 to 64KB at 72 destination alignments, with 512-byte guard regions either side checked for stray writes. --- sysdeps/alpha/alphaev6/memset.S | 31 +++++++++++++++++++++++++------ 1 file changed, 25 insertions(+), 6 deletions(-) diff --git ./sysdeps/alpha/alphaev6/memset.S ./sysdeps/alpha/alphaev6/memset.S index 5c1e307cdf..fb01baa795 100644 --- ./sysdeps/alpha/alphaev6/memset.S +++ ./sysdeps/alpha/alphaev6/memset.S @@ -158,19 +158,38 @@ $bigalign: * Scratch registers available: $7, $2, $4, $1 * We know that we'll be taking a minimum of one trip through. * CWG Section 3.7.6: do not expect a sustained store rate of > 1/cycle - * Assumes the wh64 needs to be for 2 trips through the loop in the future. - * The wh64 is issued on for the starting destination address for trip +2 - * through the loop, and if there are less than two trips left, the target - * address will be for the current trip. + * + * The wh64 is issued five trips ahead of the block it is computed on, + * so that when it is consumed at the top of the next trip it names the + * block four lines (256 bytes) beyond the stores of that trip. A + * shorter lead does not work: wh64 has to claim the line before the + * stores reach it and, unlike a load prefetch, there is no fill latency + * for a late hint to hide behind. Measured on an EV68CB (DS15, 2MB + * off-chip Bcache), a one-line lead is 453k cycles for a cold 256KB + * memset against 406k for issuing no hint at all -- i.e. it is worse + * than useless -- while four lines is 151k, a 3.0x improvement. The + * curve is flat from three lines out to eight, so the exact depth in + * that range does not matter; being under it does. + * + * If fewer than four lines remain the hint would run off the end of the + * region, and wh64 makes the whole 64-byte block UNPREDICTABLE, so the + * target falls back to the block the next trip stores. That is always + * in range and still avoids that block's read-for-ownership. + * + * Guard arithmetic: the hint consumed at trip N+1 covers + * [$5+256, $5+320), so trip N+1 needs 40 quads left; it is computed at + * trip N, which therefore needs 48. */ $do_wh64: wh64 ($4) # L1 : memory subsystem write hint - subq $3, 24, $2 # E : For determining future wh64 addresses + subq $3, 48, $2 # E : >= 48 quads left? (see guard note above) stq $17, 0($5) # L : nop # E : - addq $5, 128, $4 # E : speculative target of next wh64 + lda $4, 320($5) # E : speculative target of next wh64 (4 lines + # ahead of the trip that consumes it; 320 + # exceeds the 8-bit addq literal field) stq $17, 8($5) # L : stq $17, 16($5) # L : addq $5, 64, $7 # E : Fallback address for wh64 (== next trip addr)