Skip to content

perf(gemm): read a row-major B in place on the small-product path - #82

Merged
tannevaled merged 1 commit into
mainfrom
perf/gemm-unpacked-b
Oct 31, 2026
Merged

tannevaled merged 1 commit into
mainfrom
perf/gemm-unpacked-b

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

On the small-product path, a row-major B is now read in place, as BLIS's sup kernels do (b += rs_b). The A-in-place kernels take B's row stride ldb; a packed panel passes its own width. Only the partial last panel is still packed, and the parallel path is unchanged.

MatMul vs v0.29.0, abbench 10 rounds, 1 core 64² 256² 1024²
Zen 3 float32 −16% ~ ~
Zen 3 float64 −10% ~ ~
N1 float32 −7.3% ~ ~
N1 float64 ~ ~ ~

Zen 3 float32 64² is now at parity with NumPy (8.02 vs 8.14 µs). The N1 host was contended (rows up to ±39%), so its "~" rows mean only that there is no regression.

Tests and checks:

  • Every GEMM case runs on three paths: A and B both in place, only A in place, and both packed.
  • The fences place a strided B flush against a guard page.
  • A sabotaged kernel that ignores ldb fails 13 tests on arm64 and 9 GEMM tests on amd64 (run on Zen 3).
  • The audit entry is in SECURITY.md.

🤖 Generated with Claude Code

BLIS's sup kernels pack neither operand: the "rv" kernel advances B by
its row stride (b += rs_b, bli_gemmsup_rv_haswell_asm_s6x16m.c line
401). The A-in-place kernels (gemmUA*, amd64 and arm64) now take B's
row stride, ldb, and step B by it instead of by the packed panel's
width. A packed panel passes its own width, so one kernel serves both.

On the serial path (below GemmThreshold), with both operands row-major,
packGemmRows packs only the partial last B panel, and gemmBandUA reads
the full panels in place. The parallel path still packs B once for all
workers.

abbench, 10 rounds, one core, against v0.29.0:
  Zen 3  float32 64² -16%, float64 64² -10%, 256²/1024² unchanged
  N1     float32 64² -7.3%, the rest unchanged (contended host)
Zen 3 float32 64² is now at parity with NumPy (8.02 vs 8.14 µs).

Tests: every GEMM case runs with (unpackA, unpackB) = (on, on),
(on, off) and (off, off). The fences add a strided B flush against a
guard page. A sabotaged kernel that ignores ldb fails 13 tests on arm64
and 9 GEMM tests on amd64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 3120a7d into main Oct 31, 2026
12 checks passed
@tannevaled
tannevaled deleted the perf/gemm-unpacked-b branch October 31, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant