On Wed, Aug 12, 2026 at 3:20 AM Matt Turner <mattst88@gmail.com> wrote:
>
> Alpha had no assembly memcmp and fell back to the generic C implementation.
> Add one for the alphaev6 directory (shared by the 21264 and 21364).
>
> When both operands share alignment, byte-align to a quadword and then compare
> 64 bytes per iteration, OR-accumulating the per-quad xors; a non-zero
> accumulator means the difference is somewhere in the block, which a byte scan
> from the block base pinpoints to return the correct sign. Operands of
> differing alignment use a byte compare.
>
> The loop issues a plain read prefetch (LDL to R31) on both streams eight cache
> lines ahead. On the 21364 (EV7), whose on-chip memory controller has a long
> memory latency, this hides the Bcache misses that otherwise stall the compare;
> on the 21264 it is at worst a dropped hint, so no IMPLVER dispatch is needed.
> A read prefetch -- rather than the modify-intent LDS/LDT used by memset/memcpy
> -- is deliberate: memcmp never writes its operands, so requesting the lines for
> ownership would only add coherence traffic on a multiprocessor.
>
> The 1 to 7 bytes left after the loops are compared with a single masked
> quadword rather than a byte at a time, which would cost about eight issue
> slots per byte and dominate any length that is not a multiple of eight. Both
> operands are 0mod8 by then, since the head aligned them and everything since
> has advanced by whole quadwords, and an aligned quadword cannot cross a page
> boundary, so a whole one can be loaded at either pointer even though only some
> of it belongs to the compare; masking the difference discards the rest.
> Nothing is stored, so reading past the end of the operands is not observable.
> A mismatch still falls into the byte loop, which is what determines which byte
> differed and in which direction.
>
> Measured on an AlphaServer ES47 (EV7, 1.3GHz), cold cyc/call vs the generic C:
> size generic asm
> 1024 3617 1502 2.35x
> 4096 14527 6423 2.24x
> 65536 231362 96424 2.40x
> 262144 916913 386234 2.37x
>
>
Hi,
The mismatched-alignment fallback is a large regression against the generic C it
replaces. Hot, one operand at offset 3, whole series against baseline:
size generic(C) asm ratio
128 89.1 401.3 4.50x slower
512 268.7 1569.4 5.84x slower
2048 950.8 6235.3 6.56x slower
16384 7376.3 50163.0 6.80x slower
65536 53170.0 237889.3 4.47x slower
0.45 cycles/byte for the C against 3.06 for the byte loop, widening with length.
The generic C does a shifted word-wise compare for unaligned operands; this
gives that up entirely. Co-aligned is a clear win (1.3-2.5x, ~1.7x at 4KB+), so
the patch is worth having -- but the mismatched path needs to keep a word-wise
compare, or defer to the generic implementation, rather than byte-comparing.
Method: both operands filled identically so the compare runs the full length,
one at offset 0 and one at offset 3 from an 8192-aligned base. rpcc timing,
minimum of 7 trials, call overhead measured as a size-0 call and subtracted,
each build measured twice and the runs used as the tolerance. As a check that
does not involve the baseline at all, within the patched build alone at 16KB:
3714 cycles co-aligned against 50163 mismatched, 13.5x for the same library and
the same buffers, which is the two code paths rather than a measurement
artefact. I ran this on my ES40.
Were the quoted 2.2-2.4x numbers all co-aligned?
Magnus
new file mode 100644
@@ -0,0 +1,226 @@
+/* Copyright (C) 2026 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library. If not, see
+ <https://www.gnu.org/licenses/>. */
+
+/*
+ * EV6/EV7 memcmp. When both operands share alignment, byte-align to a
+ * quadword and compare 64 bytes per iteration, OR-accumulating the per-quad
+ * xors; a non-zero accumulator means a difference lies in the block, which a
+ * byte scan from the block base then pinpoints to return the correct sign.
+ * Operands of differing alignment fall back to a byte compare.
+ *
+ * The loop issues a plain read prefetch (LDL to R31) on both streams several
+ * cache lines ahead. On the 21364 (EV7), whose on-chip memory controller has
+ * a long latency, this hides the Bcache misses that otherwise stall the
+ * compare; on the 21264 it is at worst a dropped hint. A read prefetch (not
+ * the modify-intent LDS/LDT) is deliberate: memcmp never writes its operands,
+ * so requesting lines for ownership would only add coherence traffic on SMP.
+ *
+ * Temp usage:
+ * $0 - result
+ * $1,$2 - scratch
+ * $3,$4 - loaded data
+ * $5,$6 - xor / accumulator
+ */
+
+#include <sysdep.h>
+
+ .arch ev6
+ .set noreorder
+ .set noat
+
+/* Read prefetch distance, in bytes (8 cache lines ahead). The CWG asks for
+ 8/n cache blocks for n streams against the 21264's 8-entry MAF, so four for
+ the two streams here; measured on an EV68CB the curve is flat from four
+ blocks out to eight, and only a distance below two costs anything.
+
+ These must stay plain read prefetches (LDL to R31). Asking for the lines
+ with modify intent instead (LDS to F31) measures 2x slower on an EV68CB --
+ 756446 cycles against 368989 for a cold 256KB compare -- because ReadBlkMod
+ provokes about twice the Mbox replay traps on a stream that is only ever
+ read. Bcache misses are unchanged, so it is not extra memory traffic.
+ ReadBlkModEN (LDT to F31) measures identical to the plain prefetch, since
+ the evict-next variants are 21364-only and degrade here. */
+#define PF (8 * 64)
+
+ENTRY(memcmp)
+ .prologue 0
+
+ mov $31, $0 # E : default result 0 (equal)
+ beq $18, $ret # U : n == 0 -> equal
+ xor $16, $17, $1 # E : do the operands share alignment?
+ and $1, 7, $1 # E : ... mod 8
+
+ bne $1, $bytewise # U : different alignment -> byte compare
+ and $16, 7, $1 # E : already 0mod8?
+ beq $1, $aligned # U : yes
+ nop # E :
+
+ /* Same misalignment: byte-compare up to a 0mod8 boundary. */
+$head:
+ ldbu $3, 0($16) # L : a byte
+ ldbu $4, 0($17) # L : b byte
+ lda $16, 1($16) # E : a++
+ lda $17, 1($17) # E : b++
+
+ subq $3, $4, $0 # E : difference (unsigned bytes)
+ bne $0, $ret # U : differ -> return sign
+ subq $18, 1, $18 # E : n--
+ beq $18, $reteq # U : consumed all -> equal
+
+ and $16, 7, $1 # E : aligned now?
+ bne $1, $head # U : keep going
+
+$aligned:
+ cmple $18, 63, $1 # E : at least one 64B block?
+ bne $1, $quadtail # U : no
+ nop # E :
+ nop # E :
+
+ .align 4
+$loop:
+ ldl $31, PF($16) # L : read prefetch a, PF bytes ahead
+ ldl $31, PF($17) # L : read prefetch b
+ ldq $3, 0($16) # L : a[0..7]
+ ldq $4, 0($17) # L : b[0..7]
+
+ xor $3, $4, $5 # E : seed the accumulator
+ ldq $3, 8($16) # L :
+ ldq $4, 8($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 16($16) # L :
+ ldq $4, 16($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 24($16) # L :
+ ldq $4, 24($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 32($16) # L :
+ ldq $4, 32($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 40($16) # L :
+ ldq $4, 40($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 48($16) # L :
+ ldq $4, 48($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E :
+ ldq $3, 56($16) # L :
+ ldq $4, 56($17) # L :
+ xor $3, $4, $6 # E :
+
+ or $5, $6, $5 # E : full 64B xor summary
+ bne $5, $bytewise # U : a difference is in this block
+ subq $18, 64, $18 # E : n -= 64
+ lda $16, 64($16) # E : a += 64
+
+ lda $17, 64($17) # E : b += 64
+ cmple $18, 63, $1 # E : another full block?
+ beq $1, $loop # U :
+ nop # E :
+
+ /* 0..63 bytes left, still 0mod8: compare remaining quads. */
+$quadtail:
+ subq $18, 8, $2 # E : (n - 8)
+ blt $2, $tailquad # U : < 8 left -> masked quad
+
+$qt:
+ ldq $3, 0($16) # L :
+ ldq $4, 0($17) # L :
+ xor $3, $4, $5 # E :
+ bne $5, $bytewise # U : differ in this quad (n unchanged)
+
+ lda $16, 8($16) # E : a += 8
+ lda $17, 8($17) # E : b += 8
+ mov $2, $18 # E : commit n -= 8
+ subq $18, 8, $2 # E : (n - 8)
+
+ bge $2, $qt # U : another quad
+ nop # E :
+ nop # E :
+ nop # E :
+
+ /*
+ * 1..7 bytes left, both operands 0mod8 -- the head aligned them and
+ * everything since has advanced by whole quadwords. An aligned
+ * quadword cannot cross a page boundary, so loading a whole one at
+ * either pointer is safe even though only $18 bytes belong to the
+ * compare; masking the difference down to those bytes discards
+ * whatever followed. Nothing is stored, so reading past the end of
+ * the operands is not observable.
+ *
+ * This is only the tail. A mismatch still falls into the byte loop
+ * below, which is what works out which byte differed and in which
+ * direction, and which the block and quad loops above also branch to.
+ */
+$tailquad:
+ beq $18, $reteq # U : nothing left -> equal
+ nop # E :
+ nop # E :
+ nop # E :
+
+ ldq $3, 0($16) # L :
+ ldq $4, 0($17) # L :
+ xor $3, $4, $5 # E :
+ mskql $5, $18, $5 # U : only the bytes being compared
+
+ beq $5, $reteq # U : all equal
+ nop # E :
+ nop # E :
+ nop # E :
+
+ /* Tail bytes, and the unaligned / locate path. On entry from a block
+ or quad mismatch, the first differing byte lies within the bytes
+ scanned here, so the loop returns its sign before running off. */
+$bytewise:
+ beq $18, $reteq # U : nothing left -> equal
+ nop # E :
+
+$bl:
+ ldbu $3, 0($16) # L :
+ ldbu $4, 0($17) # L :
+ lda $16, 1($16) # E :
+ lda $17, 1($17) # E :
+
+ subq $3, $4, $0 # E : unsigned byte difference
+ bne $0, $ret # U : differ -> sign
+ subq $18, 1, $18 # E : n--
+ bne $18, $bl # U :
+
+$reteq:
+ mov $31, $0 # E : equal
+$ret:
+ ret $31, ($26), 1 # L0 :
+
+END(memcmp)
+libc_hidden_builtin_def (memcmp)
+#ifdef weak_alias
+# undef bcmp
+weak_alias (memcmp, bcmp)
+#endif
+#undef __memcmpeq
+strong_alias (memcmp, __memcmpeq)
+libc_hidden_def (__memcmpeq)