From 5bd1e4b038b356959b988286a1c61b25e585d782 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 28 Jul 2026 20:09:04 +0000 Subject: [PATCH 1/2] =?UTF-8?q?TD-T22:=20close=20as=20investigated,=20no?= =?UTF-8?q?=20gap=20=E2=80=94=20the=20AVX2=20int=20polyfills=20are=20not?= =?UTF-8?q?=20scalar?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The ticket asked a question ("Investigation only — needs simd_avx2.rs read first"; the lowering was marked "still optional"). This answers it with an asm artifact instead of a lowering. Docs only: no code, no new `unsafe`. Finding: the `avx2_int_type!` polyfills are scalar in the SOURCE and packed AVX2 in the BINARY. `.cargo/config.toml` pins `-Ctarget-cpu=x86-64-v3` for every x86_64 build, so LLVM auto-vectorizes the macro's loop bodies. Measured on `crate::simd::U32x16` (confirmed macro-generated `[u32; 16]` storage, simd_avx2.rs:1542), release, per-symbol instruction histograms: arx_quarter_round 2 vpaddd, 2 vpxor, 2 vpshufb 0 scalar ALU arx_ten_rounds 8 vpaddd, 8 vpxor, 6 vpshufb, 4 vpsrld/vpslld/vpor, 1 jne 0 scalar ALU reduce_sum 4 vpaddd, 2 vpshufd, 1 vextracti128, 1 vmovd 0 scalar ALU Three things worth naming: - `rotate_left(16)` is strength-reduced to a BYTE SHUFFLE (`vpshufb`) — cheaper than the shl|shr|or triple a hand-written intrinsic emits. - The ARX loop hits the instruction floor: 4 `U32x16` adds = 64 u32 lanes, which on 256-bit AVX2 needs 8 ymm adds minimum, and exactly 8 `vpaddd` are emitted. No headroom exists to recover. - `reduce_sum` compiles to a logarithmic horizontal-reduction tree, not the scalar `wrapping_add` fold its source literally spells out. The float side matches: `F32x16::mul_add` has the same `to_array` → scalar loop → `from_array` shape and `add_mul_f32` built on it emits real `vfmadd213ps` — one rounding, mantissa preserved. `array_chunks` / `array_windows` (+ `_checked`) already exist as the slice-level primitives. Consequences recorded: - The `🟠 scalar polyfill` cells and the `⏳` marker are ACCURATE AS WRITTEN and describe a source-level property. They are not a performance defect. - A lowering can only be justified by `repr(align(64))` cacheline guarantees (which the polyfill HAS and a `repr(transparent)` `__m256i` wrapper LOSES — measured: U32x8 size 64→32, U32x16 align 64→32), non-inlined ABI shape, or opt-level/LLVM-version independence. Never by speed. - `U32x8` must not be introduced as `U32x16`'s building block (operator ruling): it splits the lane vocabulary for no gain. - The guard that was missing is a codegen/bench oracle, not a parity harness — x86_64 is the CI host and `simd.rs`'s `u32x16_arx_ops_match_scalar` already gates the ARX triple cross-backend. Ledger entries annotated append-only; the original text is unchanged. Artifact with probe source, commands, and full histograms: `.claude/knowledge/td-t22-asm-investigation.md`. --- .../knowledge/agnostic-surface-cpu-matrix.md | 22 +++ .claude/knowledge/td-simd-tier-audit.md | 18 +- .claude/knowledge/td-t22-asm-investigation.md | 165 ++++++++++++++++++ 3 files changed, 204 insertions(+), 1 deletion(-) create mode 100644 .claude/knowledge/td-t22-asm-investigation.md diff --git a/.claude/knowledge/agnostic-surface-cpu-matrix.md b/.claude/knowledge/agnostic-surface-cpu-matrix.md index 62eacfd0..598d6bf3 100644 --- a/.claude/knowledge/agnostic-surface-cpu-matrix.md +++ b/.claude/knowledge/agnostic-surface-cpu-matrix.md @@ -98,6 +98,28 @@ two `__m256i` halves; "4×NEON" means four 128-bit NEON registers (e.g. inner ops may currently use scalar storage under `#[target_feature]` rather than real `__m256i` intrinsics. Needs verification (see § J integration plan). +> **⏳ RESOLVED — TD-T22 CLOSED, no gap (2026-07-28).** The audit is done and +> the answer is: the storage IS scalar in the SOURCE, and that costs nothing. +> `.cargo/config.toml` pins `-Ctarget-cpu=x86-64-v3` for every x86_64 build, +> so LLVM auto-vectorizes the `avx2_int_type!` loop bodies into packed AVX2. +> Measured on the ChaCha20 ARX triple over `U32x16`: **zero scalar ALU +> instructions**, `rotate_left(16)` strength-reduced to `vpshufb`, and the +> 10-round double-round loop emits exactly **8 `vpaddd` for 64 u32 lanes** — +> the AVX2 instruction-count floor, with no headroom a hand-written +> `__m256i` version could recover. `reduce_sum` emits a logarithmic +> `vpaddd`/`vpshufd`/`vextracti128` reduction tree, not the scalar fold its +> source spells out. The float side matches: `F32x16::mul_add` is the same +> `to_array` → loop → `from_array` shape and emits real `vfmadd213ps`. +> +> **So these ⏳ cells are ACCURATE AS WRITTEN and must not be read as a +> performance defect.** A lowering can only be justified by `repr(align(64))` +> cacheline guarantees (which the polyfill already has and a +> `repr(transparent)` wrapper would LOSE), non-inlined ABI shape, or +> `opt-level`/LLVM-version independence — never by speed. +> +> Full artifact with probe source, exact commands, and per-symbol instruction +> histograms: `.claude/knowledge/td-t22-asm-investigation.md`. + ### Mask vectors | Type | SKX/CLX/CPL/ICX/SPR/GNR/Z4/Z5 | HSW/ARL | A76/A72/A53 | SCA | diff --git a/.claude/knowledge/td-simd-tier-audit.md b/.claude/knowledge/td-simd-tier-audit.md index c57268dc..8b6fe613 100644 --- a/.claude/knowledge/td-simd-tier-audit.md +++ b/.claude/knowledge/td-simd-tier-audit.md @@ -271,7 +271,23 @@ pub use scalar::{ On aarch64, the only types from `simd_neon::aarch64_simd` (line 349) are `f32x16, f64x8, F32Mask16, F32x16, F64Mask8, F64x8`. Every integer width — `I32x16`, `I8x32`, `U8x64`, `U16x32`, etc. — comes from `scalar::*`. Pi 5 / M2 get scalar integer SIMD even though NEON has `int32x4_t`, `uint8x16_t`, etc. -### TD-T22 · `src/simd.rs:310, 318-321` · 256-bit int types in AVX2 build come from `simd_avx512` +### TD-T22 ✅ CLOSED (2026-07-28) — investigated, NO GAP · `src/simd.rs:310, 318-321` · 256-bit int types in AVX2 build come from `simd_avx512` + +> **Closure note (append-only; the original entry below is unchanged).** +> The deferred question — "are the 256-bit polyfills real `__m256i` +> intrinsics or scalar arrays?" — is answered: **scalar in the source, +> packed AVX2 in the binary.** The mandatory `-Ctarget-cpu=x86-64-v3` +> baseline means LLVM vectorizes the `avx2_int_type!` bodies. Measured: +> zero scalar ALU instructions on the ARX triple, 8 `vpaddd` for 64 u32 +> lanes (the AVX2 floor), `rotate_left(16)` → `vpshufb`, and a logarithmic +> reduction tree for `reduce_sum`. **No `__m256i` lowering is warranted on +> codegen grounds** for `U16x16`/`U32x8`/`U64x4`/`I32x8`/`I64x4` or the +> existing wider types. Artifact: +> `.claude/knowledge/td-t22-asm-investigation.md`. +> +> Also settled: `U32x8` must NOT be introduced as `U32x16`'s building block +> (operator ruling — it splits the lane vocabulary). The lowering PR that +> did so was closed unmerged (ndarray #261). ```rust // 310: AVX2-baseline arm uses simd_avx512 for the 256-bit shapes diff --git a/.claude/knowledge/td-t22-asm-investigation.md b/.claude/knowledge/td-t22-asm-investigation.md new file mode 100644 index 00000000..943279a1 --- /dev/null +++ b/.claude/knowledge/td-t22-asm-investigation.md @@ -0,0 +1,165 @@ +# TD-T22 — the AVX2 int-polyfill investigation (CLOSED, no gap) + +## READ BY: +- Anyone about to "lower the AVX2 scalar polyfills to native `__m256i`" +- `simd-savant`, `truth-architect` — before accepting a SIMD-lowering proposal +- Any session reading a `🟠 scalar polyfill` cell in the parity matrix + +## P0 TRIGGER +About to hand-write `__m256i` wrappers for `U32x8` / `U32x16` / `U16x32` / +`U64x4` / `I32x8` / `I64x4` because the parity matrix says "scalar polyfill"? +**Read this first. The matrix marks a SOURCE-level property, not a codegen +one, and the codegen gap does not exist.** + +--- + +## The question TD-T22 actually asked + +`td-simd-tier-audit.md:339` — `TD-T22 | – | – | Investigation only — needs +simd_avx2.rs read first`. `CHACHA20_MATRYOSHKA_PLAN.md:104` — *"avx2 native +`U32x16` — the TD-SIMD-3 lowering; **still optional**, the scalar polyfill is +correct meanwhile."* + +The ticket asked whether the `avx2_int_type!` scalar-storage polyfills +actually cost anything. It did not ask for a lowering. This note answers it. + +## Answer + +**No gap. The "scalar polyfill" is not scalar in the emitted binary.** +`.cargo/config.toml` pins `-Ctarget-cpu=x86-64-v3` (AVX2+FMA) for **every** +x86_64 build including CI, so LLVM auto-vectorizes the macro's +`for i in 0..N { … }` bodies into packed AVX2. Measured below: **zero scalar +ALU instructions**, and the ARX loop hits the theoretical instruction minimum. + +## Method + +Branch at `1ff8171`. `U32x16` is `avx2_int_type!(U32x16, u32, 16, 0u32)` +(`simd_avx2.rs:1542`) — macro-generated `[u32; 16]` storage, confirmed: no +hand-written `pub struct U32x16` exists in that file. + +```rust +// examples/td_t22_probe.rs — #[inline(never)] so each survives as a symbol +use ndarray::simd::U32x16; + +#[inline(never)] +pub fn arx_quarter_round(a: U32x16, b: U32x16) -> U32x16 { + let s = a + b; + let x = s ^ b; + x.rotate_left(16) +} + +#[inline(never)] +pub fn arx_ten_rounds(mut st: [U32x16; 4]) -> [U32x16; 4] { + for _ in 0..10 { + st[0] = st[0] + st[1]; + st[3] = (st[3] ^ st[0]).rotate_left(16); + st[2] = st[2] + st[3]; + st[1] = (st[1] ^ st[2]).rotate_left(12); + st[0] = st[0] + st[1]; + st[3] = (st[3] ^ st[0]).rotate_left(8); + st[2] = st[2] + st[3]; + st[1] = (st[1] ^ st[2]).rotate_left(7); + } + st +} + +#[inline(never)] +pub fn red_sum(a: U32x16) -> u32 { a.reduce_sum() } +``` + +```sh +cargo rustc --release --example td_t22_probe -p ndarray -- --emit asm -C debuginfo=0 +# then histogram the instructions between each symbol's .type/@function label +# and its retq, in target/release/examples/td_t22_probe-*.s +``` + +## Measured output + +**`arx_quarter_round`** — `(a+b)^b`, then `rotate_left(16)`: + +| instr | count | +|---|---| +| `vpaddd` | 2 | +| `vpxor` | 2 | +| `vpshufb` | 2 | +| `vmovdqa` | 5 | +| **scalar ALU** | **0** | + +16 u32 lanes = 2 ymm registers, so 2 packed ops per lane-op is exactly one +instruction per half. LLVM strength-reduced `rotate_left(16)` into a **byte +shuffle** (`vpshufb`) — cheaper than the `shl|shr|or` triple a hand-written +intrinsic version would emit. + +**`arx_ten_rounds`** — the full ChaCha20 double-round over `[U32x16; 4]`: + +| instr | count | +|---|---| +| `vpaddd` | 8 | +| `vpxor` | 8 | +| `vpshufb` | 6 | +| `vpsrld` / `vpslld` / `vpor` | 4 / 4 / 4 | +| `jne` | 1 | +| **scalar ALU** | **0** | + +The `jne` shows this is one rolled loop body, so the counts are per +iteration. **This is the instruction-count floor:** 4 `U32x16` adds = 64 u32 +lanes; on 256-bit AVX2 that is 8 ymm adds minimum, and exactly 8 `vpaddd` +were emitted. Rotates by 16 and 8 fold to `vpshufb` (byte-granular); rotates +by 12 and 7 use the `vpsrld`/`vpslld`/`vpor` triple. **There is no headroom +for a hand-written version to recover.** + +**`red_sum`** — `reduce_sum`: + +| instr | count | +|---|---| +| `vpaddd` | 4 | +| `vpshufd` | 2 | +| `vextracti128` | 1 | +| `vmovd` | 1 | +| **scalar ALU** | **0** | + +A textbook logarithmic horizontal-reduction tree, not the scalar +`wrapping_add` fold the source literally spells out. + +## The same result holds for the float side + +`F32x16::mul_add` (`simd_avx2.rs`) is likewise written as +`to_array()` → scalar loop → `from_array()`. `add_mul_f32` built on it emits +**`vfmadd213ps`** — real fused multiply-add, one rounding, mantissa +preserved. The doc comment at `simd_ops.rs:132` claiming "AVX2 + FMA: +`_mm256_fmadd_ps`" is correct about the *emitted code* despite the scalar +source. `array_chunks` / `array_windows` (+ `_checked`) already exist in +`simd_ops.rs` as the slice-level primitives. + +## Consequences + +1. **TD-T22 is CLOSED as "no gap".** The `🟠 scalar polyfill` cells in the + parity matrix and the `⏳` in `agnostic-surface-cpu-matrix.md` mark a + **source-level** property. They are accurate as written and must not be + read as a performance defect. +2. **Do not hand-lower these types for codegen reasons.** A lowering can + only be justified by properties LLVM does *not* give you: + - `#[repr(align(64))]` cacheline guarantee (the polyfill HAS it; a + `#[repr(transparent)]` `__m256i` wrapper LOSES it — measured: + `U32x8` size 64→32, `U32x16` align 64→32), + - ABI shape across a non-inlined boundary, + - independence from `opt-level` (auto-vectorization does not hold at + `-O0`/`-O1`) and from LLVM version drift. + Those are real but small, and none of them is a speed argument. +3. **`U32x8` must not become `U32x16`'s building block.** Operator ruling, + 2026-07-28: a half-width type composing the lane the substrate actually + uses is an absolute no-go — it splits the lane vocabulary for no gain. +4. **The guard that was missing is a codegen oracle, not a parity harness.** + x86_64 is the CI host, so `cargo test` already runs the AVX2 tier + natively and `simd.rs`'s `u32x16_arx_ops_match_scalar` already gates every + backend's ARX triple against scalar. What did not exist — and what would + have answered this ticket in twenty minutes — is an **asm-diff / bench** + check. Adding one to CI is the sanctioned follow-up. + +## Provenance + +Investigated 2026-07-28 after a lowering PR (closed unmerged, ndarray #261) +shipped ~700 lines of hand-written intrinsics on the premise that these +polyfills executed scalar code. They do not. The PR was closed on the +operator's `U32x8`-composition ruling; this investigation is the deliverable +the ticket originally asked for. From 5276f2b1c145c529be677a77da1ae1ac7d550c8f Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 28 Jul 2026 20:32:26 +0000 Subject: [PATCH 2/2] TD-T22: tighten the "zero scalar ALU" claim to what was actually measured MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Self-caught overclaim in the previous commit. I wrote "0 scalar ALU" from a narrow grep for `(add|sub|xor|rol|ror|shl|shr|or|and)[lqwb]`. A broad sweep of ALL non-vector instructions across the three probe symbols returns: 3 retq 1 movl 1 jne 1 decl The `decl` IS a scalar ALU instruction — it decrements the `arx_ten_rounds` loop trip count. So "zero scalar ALU instructions" is literally false. The claim that survives, and the one the finding actually rests on: **no scalar operation touches lane data.** Every u32 lane op in all three probes is a packed AVX2 instruction; the only scalar work is loop control. Corrected in both sites and stated explicitly in the artifact, since a knowledge doc that overstates its own measurement is the failure mode this investigation exists to document. --- .claude/knowledge/agnostic-surface-cpu-matrix.md | 6 ++++-- .claude/knowledge/td-t22-asm-investigation.md | 15 ++++++++++++--- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/.claude/knowledge/agnostic-surface-cpu-matrix.md b/.claude/knowledge/agnostic-surface-cpu-matrix.md index 598d6bf3..d3050201 100644 --- a/.claude/knowledge/agnostic-surface-cpu-matrix.md +++ b/.claude/knowledge/agnostic-surface-cpu-matrix.md @@ -102,8 +102,10 @@ than real `__m256i` intrinsics. Needs verification (see § J integration plan). > the answer is: the storage IS scalar in the SOURCE, and that costs nothing. > `.cargo/config.toml` pins `-Ctarget-cpu=x86-64-v3` for every x86_64 build, > so LLVM auto-vectorizes the `avx2_int_type!` loop bodies into packed AVX2. -> Measured on the ChaCha20 ARX triple over `U32x16`: **zero scalar ALU -> instructions**, `rotate_left(16)` strength-reduced to `vpshufb`, and the +> Measured on the ChaCha20 ARX triple over `U32x16`: **no scalar arithmetic +> touches lane data** (the only non-vector ops across all three probes are +> `retq` and the loop's `movl`/`decl`/`jne` trip counter), +> `rotate_left(16)` strength-reduced to `vpshufb`, and the > 10-round double-round loop emits exactly **8 `vpaddd` for 64 u32 lanes** — > the AVX2 instruction-count floor, with no headroom a hand-written > `__m256i` version could recover. `reduce_sum` emits a logarithmic diff --git a/.claude/knowledge/td-t22-asm-investigation.md b/.claude/knowledge/td-t22-asm-investigation.md index 943279a1..1c960dae 100644 --- a/.claude/knowledge/td-t22-asm-investigation.md +++ b/.claude/knowledge/td-t22-asm-investigation.md @@ -83,7 +83,7 @@ cargo rustc --release --example td_t22_probe -p ndarray -- --emit asm -C debugin | `vpxor` | 2 | | `vpshufb` | 2 | | `vmovdqa` | 5 | -| **scalar ALU** | **0** | +| **scalar arithmetic on lane data** | **0** | 16 u32 lanes = 2 ymm registers, so 2 packed ops per lane-op is exactly one instruction per half. LLVM strength-reduced `rotate_left(16)` into a **byte @@ -99,7 +99,7 @@ intrinsic version would emit. | `vpshufb` | 6 | | `vpsrld` / `vpslld` / `vpor` | 4 / 4 / 4 | | `jne` | 1 | -| **scalar ALU** | **0** | +| **scalar arithmetic on lane data** | **0** | The `jne` shows this is one rolled loop body, so the counts are per iteration. **This is the instruction-count floor:** 4 `U32x16` adds = 64 u32 @@ -116,11 +116,20 @@ for a hand-written version to recover.** | `vpshufd` | 2 | | `vextracti128` | 1 | | `vmovd` | 1 | -| **scalar ALU** | **0** | +| **scalar arithmetic on lane data** | **0** | A textbook logarithmic horizontal-reduction tree, not the scalar `wrapping_add` fold the source literally spells out. +**Precision on "0 scalar arithmetic on lane data".** A broad sweep of ALL +non-vector instructions across the three symbols returns exactly: +`3 retq`, `1 movl`, `1 jne`, `1 decl`. The `movl`/`decl`/`jne` are the +`arx_ten_rounds` loop counter and branch — loop control, not lane +arithmetic. So the honest claim is **no scalar op touches lane data**, not +"no scalar instruction exists": one `decl` does, and it decrements the trip +count. Every `u32` lane operation in all three probes is a packed AVX2 +instruction. + ## The same result holds for the float side `F32x16::mul_add` (`simd_avx2.rs`) is likewise written as