Repository navigation
perf(linalg): native float32 GEMM on amd64 and arm64 - #74
Merged
Merged
Conversation
The packed GEMM driver is now generic over float32 and float64. Each element type has its own micro-kernel, tile and pack-buffer pools (gemmKernel[T]). float64 keeps its kernels and MR/NR. float32 gets: - arm64: gemmMicro8x8S, NEON 8x8, by-element FMLA .4S (WORD-encoded, checked against go tool objdump); - amd64: gemmMicro6x16FMA, AVX2/FMA 6x16 (the BLIS Haswell sgemm shape), only when the CPU has FMA. Float32 MatMul (and Dot, which goes through it) runs on that kernel and accumulates in float32, as NumPy's BLAS sgemm does. Without a kernel (no FMA, or any other architecture) it still widens to the float64 GEMM. gemmEdge's scratch tile now lives at the end of the worker's A buffer. The kernel is called through a func value, so a stack tile would escape to the heap at every edge tile. Tests: the float32 GEMM bit-exact against a float32 ikj oracle on integer data (every block configuration, a transposed and offset B view, serial and parallel), within k*eps32*sum|ab| on random data, NaN/Inf reaching only their row, a fence test of the kernel's reach, and the root MatMul on a transposed view and a mixed Int8 operand on both paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
packA gathered A column by column, paying its loop and bounds checks every MR elements. At 64² that took 33% of a float32 product, more than the micro-kernel. A row-major A (CS == 1) is now read one contiguous row at a time and scattered into its lane of the panel. packB goes back to element loops: copy and clear on NR-element runs are runtime calls. abbench, 10 rounds, one core, against main: Zen 3 float32 -61/-56/-55% (64²/256²/1024²), float64 -5.8/-4.2/~ N1 float32 -64/-60/-57%, float64 ~/~/~ perf.md gains the measurements and the NumPy 2.5.4 comparison (still 1.1-1.9x behind OpenBLAS: an open defect). README, plan and SECURITY are updated too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tannevaled
force-pushed
the
perf/gemm-float32
branch
from
October 26, 2026 20:31
11c9ee9 to
59b9df3
Compare
tannevaled
marked this pull request as ready for review
October 26, 2026 20:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Native float32 GEMM on amd64 (AVX2/FMA 6x16, FMA CPUs only) and arm64 (NEON 8x8). The packed GEMM driver is now generic over float32 and float64: each type has its own micro-kernel, tile and pack-buffer pools. float32
MatMul/Dotaccumulate in float32, as NumPy's sgemm does. The other targets still widen to the float64 GEMM.The row-wise
packAfor a row-major A also speeds up float64.MatMul, abbench 10 rounds, 1 coreStill 1.1–1.9× behind NumPy 2.5.4/OpenBLAS on one core (docs/perf.md lists it as an open defect).
Sabotage check: a wrong A lane (arm64) and a wrong A row (amd64) in the generators made arch amd64, arch arm64, and the macOS, ubuntu and windows test jobs fail, in TestGemm32*, TestMatMulFloat32* and the NumPy fixture tests. The sabotage commit has been removed.
Security: the kernels' reach is bounds-checked in the wrappers and fenced with PROT_NONE in
fence_test.go. The audit entry is in SECURITY.md.🤖 Generated with Claude Code