Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
92218cb
dasLLAMA: the MoE router's prefill GEMM takes the row nest
borisbat Oct 25, 2026
f4429de
dasLLAMA tests: the depth knob's last round is the budget's to clip
borisbat Oct 25, 2026
b55adcf
dasLLAMA: gelu_erf runs on the float lattice as a tuned loop kernel
borisbat Oct 25, 2026
7e6af3a
dasLLAMA: a bf16 weight plane takes the fp32 batch tile walk
borisbat Oct 25, 2026
31909fe
dasLLAMA: the qwen3a mel splits its DFT GEMM and mel fold over frames
borisbat Oct 25, 2026
55a8b8a
dasLLAMA: the bf16-weight batch GEMM as a [tune] family with an AMX tile
borisbat Oct 25, 2026
a181b19
dasLLAMA: the generator probe gates and races the bf16-weight family
borisbat Oct 25, 2026
156b4a4
dasLLAMA: the bf16-weight family's section and the probe's envelope bar
borisbat Oct 25, 2026
2fcc7c0
dasLLAMA: a flash-decode unit is one slice over every kv head
borisbat Oct 25, 2026
caa61fa
dasLLAMA: the changed-cell and re-minted-row rules route to the re-mint
borisbat Oct 25, 2026
a936fb3
dasLLAMA: the measurement rules name the re-mint; the CPU ASR races
borisbat Oct 25, 2026
3732433
dasLLAMA: a declined perm is no candidate in the bf16 family's race
borisbat Oct 25, 2026
52ddfef
dasLLAMA: the bf16 arc's ledger entry, rows 240 and 242
borisbat Oct 25, 2026
e2f5d06
dasLLAMA tests: the q8-against-f32 transcribe cells compare word lists
borisbat Oct 25, 2026
8743163
dasLLAMA: the measurement rules define board box, re-mint and held entry
borisbat Oct 25, 2026
0036915
dasLLAMA: the bf16w family on the shared walk, one activation builder
borisbat Oct 25, 2026
c786548
dasLLAMA: the CPU ASR race on the benchmarks checklist, the class seats
borisbat Oct 25, 2026
822d20f
dasLLAMA: the bf16 arc's ledger entry with the four boards' figures
borisbat Oct 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion modules/dasLLAMA/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,8 @@ Each companion's sections are anchored by topic; a citation spells `<doc>.md#<an
host, the score dot classic prefill shares with decode, the module-scope visibility a lifted
worker lambda needs of its kernel, the two interleaves a grp<mr> repack is made of, the bf16
tile that keeps its sum in the matrix unit, the JIT partition a hot leaf instantiates in, and the
fp32 batch tile's lattice (`ARCHITECTURE_CPU_KERNELS.md#f32-batch-tile-lattice`); the q8 chain's
batch tile's lattice (`ARCHITECTURE_CPU_KERNELS.md#f32-batch-tile-lattice`) and the bf16 weight
plane's matrix-unit family (`ARCHITECTURE_CPU_KERNELS.md#bf16-weight-tile-family`); the q8 chain's
activation forms against the reference's
(`ARCHITECTURE_CPU_KERNELS.md#q8-activation-forms`).

Expand Down
71 changes: 53 additions & 18 deletions modules/dasLLAMA/ARCHITECTURE_CPU_KERNELS.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,21 +184,56 @@ out-norm epilogue. One token of a decode step, one position of a prefill and one
B-session batched step all walk those same three pieces, so no caller can round apart from
another; what differs is only the lane split over (token, head) and the projections around them.

### The fp32 batch tile walks an absolute lattice {#f32-batch-tile-lattice}

`matmul_batch_core` computes Y = X*W^T in four-row by four-token tiles (`f32_batch_tile`): each step
along n loads four weights and four inputs and issues sixteen FMAs into sixteen SIMD accumulators,
where one `dot` per output loads two values per FMA. A tile's accumulators and a lone `dot` round
differently, so which kernel computes an output has to be a property of the output alone. The
lattice is the 4x4 grid counted from row 0 and token 0, the same whatever range a chunk covers: the
dispatch unit is a row tile, a token chunk is whole tiles, every token block
(`f32_batch_token_block`: the X rows that fit the smaller of the box's q8 L2 budget and
`g_f32_batch_x_block` bytes, rounded down to whole tiles, or the whole token range when it fits)
starts on the lattice, and only the rows past the last whole row tile of d and the tokens past the
last whole token tile of ntok take `dot` (`f32_dot_rows`). Any row chunking, worker count or token
split then lands the same bits (`test_matmul_batch.das`'s lanes cell holds the threaded walk to the
one-worker walk bit for bit); a batch and the same tokens one at a time do not, and
`test_matmul_batch.das` holds the fp32 path to the per-token dot within `F32_TILE_REL`.
`g_f32_batch_tile_min_n` routes a dot shorter than it to `f32_dot_rows` whole; it ships at 1, so the
tile serves every length, and `harness/f32_batch_race.das` is the lever that races the two constants
and the nest against the tile.
### The batch tile walks an absolute lattice {#f32-batch-tile-lattice}

`matmul_batch_core` computes Y = X*W^T in four-row by four-token tiles over an f32 plane
(`f32_batch_tile`) or a bf16 plane (`bf16_batch_tile`, each weight widened by the exact high-half
shift as it is read, so the outputs are the f32 walk's over the widened plane): each step along n
loads four weights and four inputs and issues sixteen FMAs into sixteen SIMD accumulators, where one
`dot` per output loads two values per FMA. A tile's accumulators and a lone `dot` round differently,
so which kernel computes an output has to be a property of the output alone. The lattice is the 4x4
grid counted from row 0 and token 0, the same whatever range a chunk covers: the dispatch unit is a
row tile, a token chunk is whole tiles, every token block (`f32_batch_token_block`: the X rows that
fit the smaller of the box's q8 L2 budget and `g_f32_batch_x_block` bytes, rounded down to whole
tiles, or the whole token range when it fits) starts on the lattice, and only the rows past the last
whole row tile of d and the tokens past the last whole token tile of ntok take a per-output dot
(`batch_dot_rows`: `dot` on an f32 plane, `dot_bf16` on a bf16 one). Any row chunking, worker count
or token split then lands the same bits (`test_matmul_batch.das`'s lanes cell holds the threaded
walk to the one-worker walk bit for bit); a batch and the same tokens one at a time do not, and
`test_matmul_batch.das` holds the fp32 path and the bf16-plane path to the per-token dot over the
widened weights within `F32_TILE_REL`. `g_f32_batch_tile_min_n` routes a dot shorter than it to
`batch_dot_rows` whole; it ships at 1, so the tile serves every length, and
`harness/f32_batch_race.das` is the lever that races the two constants and the nest against the
tile. `matmul_batch_rows` is the nest as its own entry, under no knob: a batch whose outputs decide
what decode decides on the GEMV's bits takes it - the MoE router's prefill logits (`mm_fblob_rows`),
whose top-k over tile-rounded logits parts from decode's at a near tie and routes the token to
another expert than decode would (`test_parity.das` holds the 26B-A4B's Q4_0 file to its pinned
tokens on the CPU backend; `test_matmul_batch.das` holds the entry to the per-token matmul bit for
bit at the router's shape).

### A bf16 plane multiplies through the matrix unit on bf16 activations {#bf16-weight-tile-family}

A bf16 weight plane - an encoder tower's GEMMs, gemma-E's per-layer projection - multiplies as bf16
times bf16, the f32 activations rounded to bf16 first: `bf16w_batch` runs the `bf16w_tile_gen` family
where the family serves the call; a declined call multiplies the f32 activations unrounded. Once per call, `xbf16_build_f32` rounds the activations to
bf16 (nearest even) and also keeps the rounded values widened back to f32.

For each unit, `bf16w_panel_build` packs the plane's rows into the B-tile layout the quant families'
bf16 tile reads: for each pair of K positions (2k, 2k+1), the pair from each of sixteen rows side by
side. That is one 32-bit move per pair, because the plane already stores each pair adjacent. The
same `amx_bf16` emitter then runs the AMX bf16 tile (`TDPBF16PS`) over the panel and the activation
plane.

A unit is `tileform` groups of sixteen rows (the perm's `nrsplit`, one or two) over every whole
`tokstep` block of tokens (32 under `amx_bf16`). Three tails go through `batch_tile_block`, the
batch tile walk over the bf16 plane, fed the widened rounded copy: a unit short of `tileform`
groups, the tokens past the last whole block, and the rows past the last whole group. So every
output of one call multiplies the same activation bits, whichever kernel computed it.

`bf16w_batch` declines the whole call when the selected perm is not `amx_bf16` or the target did not
emit it (`bf16w_tileform_gen` is 0, or `bf16w_witness_gen` is false), when n is not a multiple of
32, when d is under 16, or when ntok is under `tokstep`. A one-token call never reaches it (the GEMV
path serves). A declined call runs `matmul_batch_core` over the f32 activations, unrounded. The
matrix unit adds in a different order from a sequential dot, so the probe
(`harness/gen_tune_probe.das`) holds the family to `bf16_envelope` - the bf16 rounding step at the
largest output - and not to `batch_tile_block`'s bits.
6 changes: 4 additions & 2 deletions modules/dasLLAMA/ARCHITECTURE_ENGINE_FORMATS.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,8 @@ repack walks that plane's region alone (`shexp_plane_q8`).
wrappers, dispatch shaping, and the array-sizing helpers every tier and the engine share
(`reserve_resize`, `grow_resize`, `ensure_length`, `overwrite_resize`), the `[tuned]` fp32
loop-hint families (`dot`, `axpy`, the elementwise set, `f32_batch_tile`) and the fp32 batch walk
(`matmul_batch_core`, `f32_batch_block`); every other kernel lives in a tier file.
(`matmul_batch_core`, `batch_tile_block`, generic over the f32 and bf16 weight lattices - `bf16_batch_tile`
widens each halfword by the exact shift); every other kernel lives in a tier file.
- **`dasllama_math_default.das`** - the portable backend, always registered, always correct,
out-ranked by any platform tier - the per-32 rail's portable kernels and scalar leaves, which
seat every backend's unset per-32 slot (`#per-32-rail`) - and the ONE body of every Q8*Q8 and
Expand All @@ -268,7 +269,8 @@ repack walks that plane's region alone (`shexp_plane_q8`).
generator, the layout/perm schema shared by generator and runtime, and the `[tune]` family
registration. A hand-written tile that the generator could emit belongs in the generator.
The bf16 tile's operand builders - the activation widen (`xbf16_build`, its `bf16_bits`
narrow) and the row-dequant reference panel (`kq_panel_rows_bf16`) - sit in
narrow, the f32 plane's `xbf16_build_f32`), the row-dequant reference panel
(`kq_panel_rows_bf16`) and the native bf16 plane's k-pair panel (`bf16w_panel_build`) - sit in
`dasllama_math_gen.das` beside the walk that runs them once per batch call, in its JIT
partition (`ARCHITECTURE_CPU_KERNELS.md#jit-partition-inlining`). The JIT cache keys a
generated kernel by its stub's hash and its perm's arguments, never by the generator's text,
Expand Down
4 changes: 3 additions & 1 deletion modules/dasLLAMA/ARCHITECTURE_RUNTIME.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,9 @@ and stop-string tails a tick by design, so an entry contract would name its own
instrument (`harness/served_bench.das`), and its one logging per-token branch - the budget's forced close - is a `[cold_path]` helper.
The batched step's `eval_batch_` (reached through `scheduler_step` and the public `eval_batch`) and the media prefill's
`eval_embd_` are uncovered the same way: each grows its workspace and its row set a call by design, and their perf gates are
the batched-row cells and the served-turn instrument.
the batched-row cells and the served-turn instrument. The qwen3a log-mel (`log_mel_qwen3a`) is uncovered the same
way: it builds its window, twiddles, filterbank and image a clip by design, one call a transcription, and its perf gate
is the ASR cells' encode wall.

### The post-CPU-burn GPU ramp and the residency heartbeat {#the-post-cpu-burn-gpu-ramp-and}

Expand Down
Loading
Loading