D-DYNBAG-1: ℕ-compose for p(X, Z) over DR64² tiles: exact, and measured

.claude/board/STATUS_BOARD.md

@@ -38,7 +38,7 @@ Plan: `.claude/plans/d-dynbag-holographic-bags-v1.md`. Plan only.
 | D-id | Deliverable | Status | Gate |
 |---|---|---|---|
 | D-DYNBAG-0 | PR #1884 layering fix: XLOG intake as lanes, support + bag count via Quack → mask-risc, fold oracles, cost columns | Shipped (#1906): gates 1–7 (plan §17.7); cost columns not done | operator go-ahead (given 2026-11-01) |
-| D-DYNBAG-1 | ℕ-compose probe: `matmul_i8_to_i32` over two DR64² tiles for `p(X, Z)`; B projection = `relcomp::compose` | Queued | D-DYNBAG-0 |
+| D-DYNBAG-1 | ℕ-compose probe: `matmul_i8_to_i32` over two DR64² tiles for `p(X, Z)`; B projection = `relcomp::compose` | In PR: exact vs both oracles (300 cases), projection = compose; MEASURED ~21 µs/tile on the AVX-512 VNNI tier (no AMX) vs 50 ns–2.4 µs for B (plan §7.6) | D-DYNBAG-0 |
 | D-DYNBAG-2 | semiring parameter K (B, ℕ) in relalg `Rel` + Quack surface; typed refusal per (op, K) | Queued | D-DYNBAG-1 |
 | D-DYNBAG-3 | missing ℕ words backend-first (i64 masked select, min, truncated minus, compare-to-mask, masked sum; maybe bit-packed product) | Queued | D-DYNBAG-2 |
 | D-DYNBAG-4 | DR{dynamic}: width from survivor count, multi-tile map, cross-tile fold/compose | Queued | DT64 `dr` owner |

.claude/board/entries/2026-11-01-dynbag-1-n-compose.md

@@ -0,0 +1,23 @@
+# D-DYNBAG-1: ℕ-compose for `p(X, Z)`, exact; `matmul_i8_to_i32` is the wrong carrier on this host as called
+
+**Date 2026-11-01 · ID D-DYNBAG-1 · Status IN PR · Plan `.claude/plans/d-dynbag-holographic-bags-v1.md` §7.6 · Crate `crates/xlog-relcomp-probe` (`src/ncompose.rs`, `tests/litho.rs` part 4, `examples/ncompose_cost.rs`)**
+
+- `p(X, Z) :- r(X, Y), s(Y, Z)` is answered as `matmul_i8_to_i32` over two DR64² 0/1 tiles. It equals both oracles (scalar witnesses per (x, z); XLOG's RIR bag walk) on the fixture and on 300 random cases. Its `> 0` reading equals `relcomp::compose`.
+- Disable runs red: operands swapped (caught by the differential only); B read before counting.
+- MEASURED, AVX-512 VNNI tier (Xeon model 0x55, no AMX; the cost example prints the tier):
+
+  | case | `compose` (B) | `matmul_i8_to_i32` (ℕ) |
+  |---|---|---|
+  | sparse | 50 ns | about 21 µs per tile |
+  | dense | 2.4 µs | about 21 µs per tile |
+
+HYPOTHESIS: the 21 µs is per-call pack/alloc overhead, not arithmetic.
+
+A first version said "no VNNI, scalar tier" from a mis-grepped flag (`avx512_vnni`); this entry is corrected.
+
+**OPEN:**
+- A bit-packed i1 × i1 → i32 word (popcount of R[x] ∧ Sᵀ[z]) in `ndarray::simd`, the D-DYNBAG-3 candidate.
+- The AMX tier: not on this host; never certify it from a shared VM (ndarray AMX_GOTCHAS 14).
+- A non-allocating `matmul_i8_to_i32` entry point.
+- Bounds and chains longer than two atoms (refused).
+- Cross-tile compose (MISSING-7).

.claude/board/entries/README.md

@@ -25,12 +25,13 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
 opposite directions; the stranding this convention prevents shows up in
 exactly one of them, never both.
 
-539 entries, 2026-08-06 .. 2026-11-01.
+540 entries, 2026-08-06 .. 2026-11-01.
 
 | date | entry id | finding | file |
 |---|---|---|---|
 | 2026-11-01 | `D-DT64-7-5` |  | [2026-11-01-maptable-versioned-local-map.md](2026-11-01-maptable-versioned-local-map.md) |
 | 2026-11-01 | `D-XLOG-NATIVE-4` |  | [2026-11-01-lance-graph-xlog-crate.md](2026-11-01-lance-graph-xlog-crate.md) |
+| 2026-11-01 | `D-DYNBAG-1` |  | [2026-11-01-dynbag-1-n-compose.md](2026-11-01-dynbag-1-n-compose.md) |
 | 2026-11-01 | `D-DYNBAG-0` |  | [2026-11-01-dynbag-0-xlog-through-quack-mask-risc.md](2026-11-01-dynbag-0-xlog-through-quack-mask-risc.md) |
 | 2026-10-31 | `x2-dt64-admission-policy` |  | [2026-10-31-x2-dt64-admission-policy.md](2026-10-31-x2-dt64-admission-policy.md) |
 | 2026-10-31 | `r1-r2il-binding-reading` |  | [2026-10-31-r1-r2il-binding-reading.md](2026-10-31-r1-r2il-binding-reading.md) |

.claude/plans/d-dynbag-holographic-bags-v1.md

@@ -612,6 +612,29 @@ Evidence: four read-only harvests at `d3c2a29`/`f3b2cfe`, each claim cited file:
 
 §17 stands. One addition: the binding that selects the executed program is DT64's `MethodBinding` (its `effects` name the polyfill words), and the selection mechanism follows Tract's precedent (resolve once, dispatch through the resolved entry, frame-checked per run) without copying Tract's registry. If X1 needs a type in DT64, D-DYNBAG-0 stops and asks rather than adding it in the probe.
 
+### 7.6 Result (2026-11-01, branch `ccr-85acaff0-oszq40-dynbag1`)
+
+- **Equality: met.** `xlog-relcomp-probe` `ncompose::pair_counts` (two DR64² i8 tiles, `matmul_i8_to_i32`) equals scalar enumeration per (x, z) and a bag walk of XLOG's RIR for `p`, on the multiplicity fixture and on a 300-case differential (ids 0..6 per axis; ℕ ≠ B in over 100 cases).
+- **Projection: met.** `count > 0` equals `relcomp::compose` on the DR64 register, as (x, z) id sets, on every case.
+- **Admission.** `ncompose::admit_pair` takes the join in both orders XLOG's optimizer emits, and a `ChainJoin` only when its own relations match its fallback. Bounds are refused. `q(Y)` is still never admitted as `p(X, Z)`.
+- **Disable runs: red.** Operands swapped: the differential fails; the hand fixture alone passes, so the differential is what catches it. Result read as B before counting: both tests fail.
+- **Cost (MEASURED, `examples/ncompose_cost.rs`; host: Intel Xeon model 0x55 (Skylake-SP / Cascade Lake class), 4 vCPU VM, AVX-512 with VNNI, no AMX. The product ran on its AVX-512 VNNI `VPDPBUSD` tier, as the example now prints):**
+
+  | case | `compose` (B) | `matmul_i8_to_i32` (ℕ, tiles prepared) | `pair_counts` (ℕ, from tuples) |
+  |---|---|---|---|
+  | sparse, 4 × 4 tuples | 50 ns | 20,849 ns | 21,420 ns |
+  | dense 64 × 64 | 2,402 ns | 20,829 ns | 158,637 ns |
+
+- **Answer to open decision 2 (on this host):** through `matmul_i8_to_i32`'s current signature, a 64² tile costs about 21 µs whatever the density. It is exact, but it is the wrong carrier for sparse tiles here: 400× slower than B on sparse data, 9× on dense.
+  - **HYPOTHESIS, not profiled:** the 21 µs is mostly per-call overhead (`pack_contig` of both inputs, the u8 shift copy, a fresh output `Vec`, the bias subtraction, the copy back; §7.4), not arithmetic. 64³ multiply-adds is about 4,096 `VPDPBUSD` instructions.
+  - **Untested:** the AMX tier, a non-allocating entry point, and a bit-packed i1 × i1 → i32 word (popcount of R[x] ∧ Sᵀ[z], which skips empty rows). Per §7.4 either lands in `ndarray::simd`, backend-first; the bit-packed word is the D-DYNBAG-3 candidate.
+  - Building the tiles from tuples (about 17 ns per tuple lookup, dense) also dominates end-to-end time.
+- **Correction (same day):** the first version of this result said "no VNNI, scalar tier". That came from grepping `/proc/cpuinfo` for `avx512vnni`, which the kernel spells `avx512_vnni`. The numbers were unchanged on re-run.
+- **AMX (ndarray `.claude/AMX_GOTCHAS.md`):**
+  - This host has no AMX: model 0x55 is pre-Sapphire Rapids, so `amx_available() == false` is genuine (Gotcha 4 does not apply).
+  - Any later AMX measurement must not be certified from a shared VM (Gotcha 14: tile state is silently corrupted under host contention).
+  - Its parity tests must also run under deliberate CPU contention, and must not count a skipped tile test as a pass (Gotcha 9).
+
 ---
 
 ## 19. Polyfills are composed, not hardcoded (2026-10-31, operator decision before coding)

crates/xlog-relcomp-probe/examples/ncompose_cost.rs

@@ -0,0 +1,125 @@
+//! D-DYNBAG-1 cost probe: ℕ-compose (`matmul_i8_to_i32` over DR64² tiles)
+//! against the Boolean `relcomp::compose` on the DR64 register, same data.
+//! Hot paths only, reported in ns per call (median of 201 runs).
+//!
+//!   cargo run --release --example ncompose_cost --manifest-path crates/xlog-relcomp-probe/Cargo.toml
+use std::hint::black_box;
+use std::time::Instant;
+
+use dr_math_binding_probe::relcomp::{Work, compose};
+use dr_math_binding_probe::{Axis, AxisKind};
+use lance_graph_dt64::dr::{LocalMap, Width};
+use ndarray::Array2;
+use ndarray::simd::{amx_available, matmul_i8_to_i32};
+use xlog_relcomp_probe::ncompose::{PairQuery, pair_counts};
+use xlog_relcomp_probe::{AddressMap, register};
+
+const AX: [Axis; 3] = [
+    Axis {
+        class: 10,
+        kind: AxisKind::Identity,
+    },
+    Axis {
+        class: 11,
+        kind: AxisKind::Property,
+    },
+    Axis {
+        class: 12,
+        kind: AxisKind::Community,
+    },
+];
+
+type Tuples = Vec<(u32, u32)>;
+
+fn median_ns(mut f: impl FnMut()) -> u128 {
+    let mut t: Vec<u128> = (0..201)
+        .map(|_| {
+            let s = Instant::now();
+            f();
+            s.elapsed().as_nanos()
+        })
+        .collect();
+    t.sort_unstable();
+    t[100]
+}
+
+fn local(ids: impl Iterator<Item = u32>) -> LocalMap {
+    let mut v: Vec<u64> = ids.map(u64::from).collect();
+    v.sort_unstable();
+    v.dedup();
+    LocalMap::prepare(Width::W64, &v).unwrap()
+}
+
+fn main() {
+    // The tier `matmul_i8_to_i32` dispatches to, in its own order:
+    // AMX, then AVX-512 VNNI, then AVX-VNNI, then the scalar reference.
+    let tier = if amx_available() {
+        "AMX TDPBUSD"
+    } else if std::is_x86_feature_detected!("avx512vnni") {
+        "AVX-512 VNNI VPDPBUSD"
+    } else if std::is_x86_feature_detected!("avxvnni") {
+        "AVX-VNNI"
+    } else {
+        "scalar reference"
+    };
+    println!("amx_available: {} | matmul tier: {tier}", amx_available());
+    let full: Vec<(u32, u32)> = (0..64).flat_map(|a| (0..64).map(move |b| (a, b))).collect();
+    let cases: [(&str, Tuples, Tuples); 2] = [
+        (
+            "sparse",
+            vec![(1, 2), (5, 2), (1, 3), (9, 7)],
+            vec![(2, 4), (2, 6), (3, 4), (7, 8)],
+        ),
+        ("dense 64x64", full.clone(), full),
+    ];
+    let q = PairQuery {
+        left: xlog_core::RelId(0),
+        right: xlog_core::RelId(1),
+    };
+    for (name, r, s) in cases {
+        // Boolean: the register composition.
+        let xs = AddressMap::new(AX[0], r.iter().map(|t| t.0)).unwrap();
+        let ys =
+            AddressMap::new(AX[1], r.iter().map(|t| t.1).chain(s.iter().map(|t| t.0))).unwrap();
+        let zs = AddressMap::new(AX[2], s.iter().map(|t| t.1)).unwrap();
+        let (rr, ss) = (
+            register(&r, &xs, &ys).unwrap(),
+            register(&s, &ys, &zs).unwrap(),
+        );
+        let mut work = Work::default();
+        let b = median_ns(|| {
+            black_box(compose(black_box(&rr), black_box(&ss), &mut work));
+        });
+        // ℕ: the product alone, on prepared i8 tiles, output reused.
+        let (lx, ly, lz) = (
+            local(r.iter().map(|t| t.0)),
+            local(r.iter().map(|t| t.1).chain(s.iter().map(|t| t.0))),
+            local(s.iter().map(|t| t.1)),
+        );
+        let mut a = Array2::<i8>::zeros((64, 64));
+        let mut bm = Array2::<i8>::zeros((64, 64));
+        for &(x, y) in &r {
+            a[(
+                lx.local(x.into()).unwrap() as usize,
+                ly.local(y.into()).unwrap() as usize,
+            )] = 1;
+        }
+        for &(y, z) in &s {
+            bm[(
+                ly.local(y.into()).unwrap() as usize,
+                lz.local(z.into()).unwrap() as usize,
+            )] = 1;
+        }
+        let mut out = Array2::<i32>::zeros((64, 64));
+        let n = median_ns(|| {
+            matmul_i8_to_i32(black_box(a.view()), black_box(bm.view()), out.view_mut()).unwrap();
+        });
+        // ℕ end to end: tiles built from the tuples, then the product.
+        let e = median_ns(|| {
+            black_box(pair_counts(&q, &r, &s, &lx, &ly, &lz).unwrap());
+        });
+        println!(
+            "{name}: compose (B) {b} ns | matmul (N, tiles prepared) {n} ns | pair_counts (N, from tuples) {e} ns"
+        );
+    }
+}

crates/xlog-relcomp-probe/src/lib.rs

@@ -39,6 +39,7 @@
 use std::collections::BTreeMap;
 
 pub mod native;
+pub mod ncompose;
 pub mod recursive;
 
 use dr_math_binding_probe::relcomp::{ComposeRefusal, Rows, Work, compose};

crates/xlog-relcomp-probe/src/ncompose.rs

@@ -0,0 +1,163 @@
+//! ℕ-compose: `p(X, Z) :- r(X, Y), s(Y, Z)` with both ends kept, as an
+//! integer matrix product over DR tiles (D-DYNBAG-1, plan
+//! `d-dynbag-holographic-bags-v1.md` §7).
+//!
+//! R and S are sets of tuples (§9.2). On local tiles they are 0/1 matrices
+//! A (|X| × |Y|) and B (|Y| × |Z|). Then `(A · B)[x][z]` is the number of `y`
+//! with `r(x, y) ∧ s(y, z)`: the ℕ answer, one per witness. Its projection
+//! `> 0` is the Boolean composition (`relcomp::compose`).
+//!
+//! The product is `ndarray::simd::matmul_i8_to_i32` (AMX `TDPBUSD`, AVX-512
+//! VNNI, or the scalar reference, bit-identical). One hop over 0/1 inputs is
+//! exact: a cell is at most the Y tile width. A chain of hops (counts feeding
+//! the next product) is not admitted here (§7.3).
+//!
+//! Lab only. The axes are DR's `LocalMap`s, one per X, Y, Z, chosen by the
+//! caller; an id a map does not hold is refused. Bounds on X or Z are refused
+//! by [`admit_pair`]; they are a later step.
+
+use lance_graph_dt64::dr::{LocalMap, MapRefusal};
+use ndarray::Array2;
+use ndarray::simd::matmul_i8_to_i32;
+use xlog_core::RelId;
+use xlog_ir::{JoinType, ProjectExpr, RirNode};
+
+use crate::{LowerRefusal, node_name, scan_with_bound};
+
+/// `p(X, Z) :- r(X, Y), s(Y, Z)`, admitted.
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+pub struct PairQuery {
+    /// `r(X, Y)`.
+    pub left: RelId,
+    /// `s(Y, Z)`.
+    pub right: RelId,
+}
+
+/// Why the product did not answer.
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub enum ComposeRefusal {
+    /// An id outside the axis's DR map.
+    Domain(MapRefusal),
+    /// The product itself refused (shape).
+    Matmul(String),
+}
+
+/// Admit XLOG's compiled body for `p(X, Z) :- r(X, Y), s(Y, Z)`: an inner
+/// join on Y keeping X and Z, in either join order. Bounds, other
+/// projections and other joins are refused.
+pub fn admit_pair(body: &RirNode) -> Result<PairQuery, LowerRefusal> {
+    // XLOG's optimizer promotes the chain to `ChainJoin` (in either join
+    // order, chosen from statistics) and keeps the plain plan as `fallback`.
+    // Admit it only when its own relations name the same query as its
+    // fallback; a disagreement is refused, never resolved by picking one.
+    if let RirNode::ChainJoin {
+        left,
+        right,
+        left_key,
+        right_key,
+        output_columns,
+        fallback,
+    } = body
+    {
+        // The chain's columns are left ++ right, as for `Join` below.
+        let (r_side, s_side, xz) = match (*left_key, *right_key) {
+            (1, 0) => (left, right, (0, 3)),
+            (0, 1) => (right, left, (2, 1)),
+            _ => {
+                return Err(LowerRefusal::NeedsInverse {
+                    left_keys: vec![*left_key],
+                    right_keys: vec![*right_key],
+                });
+            }
+        };
+        // The chain must answer the same query as its fallback on its own:
+        // its projection and its inputs are checked, not only its relations.
+        match output_columns.as_slice() {
+            [ProjectExpr::Column(x), ProjectExpr::Column(z)] if (*x, *z) == xz => {}
+            _ => return Err(LowerRefusal::UnsupportedProjection),
+        }
+        let (l, lb) = scan_with_bound(r_side, 0)?;
+        let (r, rb) = scan_with_bound(s_side, 1)?;
+        if lb.is_some() || rb.is_some() {
+            return Err(LowerRefusal::UnsupportedFilter);
+        }
+        let own = PairQuery { left: l, right: r };
+        return match admit_pair(fallback) {
+            Ok(fb) if fb == own => Ok(own),
+            Ok(_) => Err(LowerRefusal::ChainJoinDisagrees),
+            Err(e) => Err(e),
+        };
+    }
+    let RirNode::Project { input, columns } = body else {
+        return Err(LowerRefusal::UnsupportedNode(node_name(body)));
+    };
+    let RirNode::Join {
+        left,
+        right,
+        left_keys,
+        right_keys,
+        join_type,
+    } = input.as_ref()
+    else {
+        return Err(LowerRefusal::UnsupportedNode(node_name(input)));
+    };
+    if *join_type != JoinType::Inner {
+        return Err(LowerRefusal::UnsupportedJoin(*join_type));
+    }
+    // Join output columns are left ++ right:
+    //   r(X,Y) ⋈ s(Y,Z): keys [1]/[0], columns [X,Y,Y,Z]: X at 0, Z at 3;
+    //   s(Y,Z) ⋈ r(X,Y): keys [0]/[1], columns [Y,Z,X,Y]: X at 2, Z at 1.
+    let (r_side, s_side, xz) = match (left_keys.as_slice(), right_keys.as_slice()) {
+        ([1], [0]) => (left, right, (0, 3)),
+        ([0], [1]) => (right, left, (2, 1)),
+        _ => {
+            return Err(LowerRefusal::NeedsInverse {
+                left_keys: left_keys.clone(),
+                right_keys: right_keys.clone(),
+            });
+        }
+    };
+    match columns.as_slice() {
+        [ProjectExpr::Column(x), ProjectExpr::Column(z)] if (*x, *z) == xz => {}
+        _ => return Err(LowerRefusal::UnsupportedProjection),
+    }
+    let (l, lb) = scan_with_bound(r_side, 0)?;
+    let (r, rb) = scan_with_bound(s_side, 1)?;
+    if lb.is_some() || rb.is_some() {
+        return Err(LowerRefusal::UnsupportedFilter);
+    }
+    Ok(PairQuery { left: l, right: r })
+}
+
+/// A relation's 0/1 matrix over two DR axes. A repeated tuple sets the same
+/// cell: base relations are sets.
+fn matrix(
+    tuples: &[(u32, u32)],
+    rows: &LocalMap,
+    cols: &LocalMap,
+) -> Result<Array2<i8>, ComposeRefusal> {
+    let mut m = Array2::<i8>::zeros((rows.width().positions(), cols.width().positions()));
+    for &(a, b) in tuples {
+        let i = rows.local(u64::from(a)).map_err(ComposeRefusal::Domain)?;
+        let j = cols.local(u64::from(b)).map_err(ComposeRefusal::Domain)?;
+        m[(usize::from(i), usize::from(j))] = 1;
+    }
+    Ok(m)
+}
+
+/// Witnesses per (x, z), indexed by local position on the X and Z tiles.
+pub fn pair_counts(
+    _q: &PairQuery,
+    left: &[(u32, u32)],
+    right: &[(u32, u32)],
+    xs: &LocalMap,
+    ys: &LocalMap,
+    zs: &LocalMap,
+) -> Result<Array2<i32>, ComposeRefusal> {
+    let a = matrix(left, xs, ys)?;
+    let b = matrix(right, ys, zs)?;
+    let mut out = Array2::<i32>::zeros((a.nrows(), b.ncols()));
+    matmul_i8_to_i32(a.view(), b.view(), out.view_mut())
+        .map_err(|e| ComposeRefusal::Matmul(format!("{e:?}")))?;
+    Ok(out)
+}