Skip to content

perf(linalg): native float32 GEMM on amd64 and arm64 - #74

Merged
tannevaled merged 2 commits into
mainfrom
perf/gemm-float32
Oct 26, 2026
Merged

tannevaled merged 2 commits into
mainfrom
perf/gemm-float32

Conversation

@tannevaled

@tannevaled tannevaled commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Native float32 GEMM on amd64 (AVX2/FMA 6x16, FMA CPUs only) and arm64 (NEON 8x8). The packed GEMM driver is now generic over float32 and float64: each type has its own micro-kernel, tile and pack-buffer pools. float32 MatMul/Dot accumulate in float32, as NumPy's sgemm does. The other targets still widen to the float64 GEMM.

The row-wise packA for a row-major A also speeds up float64.

MatMul, abbench 10 rounds, 1 core 64² 256² 1024²
Zen 3 float32 −61% −56% −55%
Zen 3 float64 −5.8% −4.2% ~
N1 float32 −64% −60% −57%
N1 float64 ~ ~ ~

Still 1.1–1.9× behind NumPy 2.5.4/OpenBLAS on one core (docs/perf.md lists it as an open defect).

Sabotage check: a wrong A lane (arm64) and a wrong A row (amd64) in the generators made arch amd64, arch arm64, and the macOS, ubuntu and windows test jobs fail, in TestGemm32*, TestMatMulFloat32* and the NumPy fixture tests. The sabotage commit has been removed.

Security: the kernels' reach is bounds-checked in the wrappers and fenced with PROT_NONE in fence_test.go. The audit entry is in SECURITY.md.

🤖 Generated with Claude Code

tannevaled and others added 2 commits October 26, 2026 21:20
The packed GEMM driver is now generic over float32 and float64. Each
element type has its own micro-kernel, tile and pack-buffer pools
(gemmKernel[T]). float64 keeps its kernels and MR/NR. float32 gets:

- arm64: gemmMicro8x8S, NEON 8x8, by-element FMLA .4S (WORD-encoded,
  checked against go tool objdump);
- amd64: gemmMicro6x16FMA, AVX2/FMA 6x16 (the BLIS Haswell sgemm
  shape), only when the CPU has FMA.

Float32 MatMul (and Dot, which goes through it) runs on that kernel and
accumulates in float32, as NumPy's BLAS sgemm does. Without a kernel
(no FMA, or any other architecture) it still widens to the float64
GEMM.

gemmEdge's scratch tile now lives at the end of the worker's A buffer.
The kernel is called through a func value, so a stack tile would escape
to the heap at every edge tile.

Tests: the float32 GEMM bit-exact against a float32 ikj oracle on
integer data (every block configuration, a transposed and offset B
view, serial and parallel), within k*eps32*sum|ab| on random data,
NaN/Inf reaching only their row, a fence test of the kernel's reach,
and the root MatMul on a transposed view and a mixed Int8 operand on
both paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
packA gathered A column by column, paying its loop and bounds checks
every MR elements. At 64² that took 33% of a float32 product, more
than the micro-kernel. A row-major A (CS == 1) is now read one
contiguous row at a time and scattered into its lane of the panel.
packB goes back to element loops: copy and clear on NR-element runs
are runtime calls.

abbench, 10 rounds, one core, against main:
  Zen 3  float32 -61/-56/-55% (64²/256²/1024²), float64 -5.8/-4.2/~
  N1     float32 -64/-60/-57%,                   float64 ~/~/~

perf.md gains the measurements and the NumPy 2.5.4 comparison (still
1.1-1.9x behind OpenBLAS: an open defect). README, plan and SECURITY
are updated too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled marked this pull request as ready for review October 26, 2026 20:31
@tannevaled
tannevaled merged commit 5e51541 into main Oct 26, 2026
12 checks passed
@tannevaled
tannevaled deleted the perf/gemm-float32 branch October 26, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant