diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index c4a1a52fc..c7b902b32 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,7 @@ +## 2026-10-30 — D-TRACT-HARVEST-0: Tract as a cold ONNX frontend, admitted onto native methods → `.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md` + +**Status:** LAB PROBE IMPLEMENTED, REDESIGNED 2026-10-30 to the ONNX graph as a DR#3 graph resolved through the DT64 ON×WHEN×DO registry (plan §0; the recipe text below is the superseded first version) (`crates/tract-harvest-probe`, own workspace, no native dependents); runs the Tract arm of D-ONNX-INTAKE S1/S3 (#1802, #1810); Burn not run. **Board:** [entry](entries/2026-10-30-d-tract-harvest-0-native-algebra-bridge.md), [STATUS_BOARD](STATUS_BOARD.md) D-TRACT-HARVEST-0. Tiny graph bit-exact with ORT (optimizer off); Jina v5 small layer-0 RMSNorm slice bounded (max 4 ulp, deviation confined to the float mean). Records the Tract provenance gaps (domain ignored, equal constants merged, declutter Div→Mul(recip) and fusion name loss) and the next PR sequence (float reduction tolerance, admitted f32 MatMul first). +**Confidence:** lab evidence on two fixtures; no whole-model claim. ## 2026-10-30 — D-DRCA-3: DR #4 method routing falsifier (lab) → `.claude/plans/d-drca-3-dr4-method-routing-v1.md` **Status:** lab probe in `dr-math-binding-probe` (`src/dr4.rs`, `tests/dr4_method_routing.rs`); first DR #4 falsifier of D-DRDT §5. **Board:** [entry](entries/2026-10-30-d-drca-3-dr4-method-routing.md), [STATUS_BOARD](STATUS_BOARD.md) D-DRCA-3. DT-owned method key, guard witness and effect order are carried opaque; the effect-order oracle is ndarray #395's order law. No DT resolver (D-CMH-1), no MethodKey mint, no contract change. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index c6f17c9d6..5d567c8f7 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -51,11 +51,12 @@ | D-id | scope | status | gate / falsifier | |---|---|---|---| | **D-ONNX-INTAKE-0** | S0 capstone: candidates, ownership crosswalk, staged bake-off, falsifiers | Shipped, docs only (#1802) | no importer, test or model parity claimed | -| **D-ONNX-INTAKE-1** | S1 cold import of one pinned real model plus a synthetic graph, unoptimized; source preservation | Queued (NOT RUN; not authorized) | metadata counted preserved/lost/renamed against raw protobuf/ORT; adversarial fixtures refuse | +| **D-ONNX-INTAKE-1** | S1 cold import of one pinned real model plus a synthetic graph, unoptimized; source preservation | Tract arm RUN as lab (D-TRACT-HARVEST-0); Burn arm NOT RUN | metadata counted preserved/lost/renamed against raw protobuf/ORT; adversarial fixtures refuse | | **D-ONNX-INTAKE-2** | S2 node coverage on the same raw model | Queued (NOT RUN; not authorized) | separate denominators; coverage under declared semantic admission, not op-name similarity | -| **D-ONNX-INTAKE-3** | S3 one complete static subgraph on existing native methods | Queued (NOT RUN; not authorized) | scalar oracle + ORT; no Burn/Tract in the native call path; unsupported nodes refuse | +| **D-ONNX-INTAKE-3** | S3 one complete static subgraph on existing native methods | Tract arm RUN as lab (D-TRACT-HARVEST-0): tiny graph exact, Jina RMSNorm slice bounded | scalar oracle + ORT; no Burn/Tract in the native call path; unsupported nodes refuse | | **D-ONNX-INTAKE-4** | S4 numeric/quantization parity for real model blockers | Queued (conditional; not authorized) | IntegerExact / bounded FP / SourceParity / CertifiedMath kept distinct | | **D-ONNX-INTAKE-5** | S5 whole-model forward pass and cost closure | Goal, not authorized | outputs vs ORT under declared tolerance; every cost stage reported | +| **D-TRACT-HARVEST-0** | ONNX graph as a DR#3 graph (nodes Identity, values Property, dep = do_ ∘ read_by) resolved through a DT64 ON×WHEN×DO registry (op signature → classid → native DO); Tract cold facts; ORT (optimizer off) oracle; `crates/tract-harvest-probe` | Lab probe implemented (this PR, redesigned to the graph); plan [d-tract-harvest-0-native-algebra-bridge-v1.md](../plans/d-tract-harvest-0-native-algebra-bridge-v1.md) §0 | 20 tests, 7 disable runs red; full model 3,751/7,620 nodes WHEN-true, >64 positions untiled, first unregistered groups Unsqueeze/Shape/Gather/Concat/MatMul | ## D-TFLP — TFLite models on the native polyfill: int8 codes as Palette256 (2026-10-25) diff --git a/.claude/board/entries/2026-10-30-d-tract-harvest-0-native-algebra-bridge.md b/.claude/board/entries/2026-10-30-d-tract-harvest-0-native-algebra-bridge.md new file mode 100644 index 000000000..d876f883e --- /dev/null +++ b/.claude/board/entries/2026-10-30-d-tract-harvest-0-native-algebra-bridge.md @@ -0,0 +1,27 @@ +# D-TRACT-HARVEST-0 — Tract as a cold ONNX frontend, admitted onto native methods + +**Date 2026-10-30 · Status LAB PROBE IMPLEMENTED (not production) · Owner D-NAM [EXO-152](https://linear.app/exo-red/issue/EXO-152) · Plan [`d-tract-harvest-0-native-algebra-bridge-v1.md`](../../plans/d-tract-harvest-0-native-algebra-bridge-v1.md) · Crate `crates/tract-harvest-probe` (own workspace)** + +Runs the Tract arm of D-ONNX-INTAKE S1 and S3 (#1802), nominated by #1810. Burn `onnx-ir` was not run. + +**Chain shown:** source-attested ONNX subgraph → Tract `InferenceModel → TypedModel` (cold) → admission (fixed pattern; domain/opset/initializer identity from the raw protobuf) → concrete recipe calling only `lance-graph-onnx` / `ndarray` → ORT with its graph optimizer off. + +| fixture | result vs ORT (`ORT_DISABLE_ALL`) | +|---|---| +| tiny (Greater/Where/ReduceSum + Add/Mul/Abs/Sqrt/Div, opset 18) | bit-exact; 256-input div3 sweep bit-exact | +| Jina v5 small retrieval, layer-0 input RMSNorm (10×1024) | 9,259/10,240 exact, max 4 ulp; all deviation is in the one row whose mean is 1 ulp off | + +**Findings (source-backed):** +- Tract looks operators up by `op_type` only (`onnx/src/model.rs:135`): `com.example:Abs` imports as `Abs`. Admission refuses from the protobuf domain. +- `InferenceModel::add_const` merges equal-valued initializers (`hir/src/infer/model.rs:202`): `T = 0` and `Z = 0` become one node. +- Declutter rewrites `Div(Y,B)` → `Mul(Y,1/B)`: 161/1,536 sweep elements differ from ORT. It also fuses RMSNorm and Silu (use as pattern evidence, not kernels). +- An impossible broadcast is typed symbolically and fails only at run time; the native broadcast check refuses. +- `Flatten` refuses >1 symbolic dim (`hir/src/ops/array/flatten.rs:11`); the full Jina graph types only with pinned inputs. +- ORT's default optimizer changes the RMSNorm bytes (7,136/10,240 equal to its literal run): the oracle setting is part of the contract. +- Tract's `Reduce` is 33 ulp from an f64 reference mean; native and ORT literal are within 2. + +**Cost (reported):** Jina slice native 9 µs (9 allocations, every intermediate materialized) vs Tract fused 22.6 µs; full 7,620-node graph cold import + type + declutter 0.4 s with no weight bytes read. + +**OPEN:** float reduction tolerance contract (D-ONNX-NATIVE-1); admitted f32 MatMul (first whole-model blocker); Gather reclassification; ORT_DISABLE_ALL as the standing oracle setting. + +**Redesign (same day, operator correction "We want the graph"):** the fixed recipes are removed. The ONNX graph is laid onto a DR#3 tile (nodes = Identity, values = Property, `on`/`do_` as 64x64 relations, `dep = do_ ∘ read_by`) and every node resolves through a DT64 ON × WHEN × DO registry: op signature (domain, op_type, dtype, attributes) → one immutable lab classid, WHEN a K3 guard over Tract's facts, DO an existing `lance-graph-onnx` / `ndarray` function. Plans are sealed in a `FrameStore` frame. Tiny fixture bit-exact (incl. div3 sweep; `Where(c, X, 1)` now runs, 17); Jina RMSNorm through the graph 9,259/10,240 exact, max 4 ulp; full model coverage 3,751 WHEN-true / 169 WHEN-not-true (facts Tract drops) / 3,700 unregistered. 20 tests; 7 disable runs each red on the named test. Plan §0. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 5aa3b5cea..67aa558fc 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,10 +25,11 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -503 entries, 2026-08-06 .. 2026-10-30. +504 entries, 2026-08-06 .. 2026-10-30. | date | entry id | finding | file | |---|---|---|---| +| 2026-10-30 | `D-TRACT-HARVEST-0` | | [2026-10-30-d-tract-harvest-0-native-algebra-bridge.md](2026-10-30-d-tract-harvest-0-native-algebra-bridge.md) | | 2026-10-30 | `D-DRCA-3` | | [2026-10-30-d-drca-3-dr4-method-routing.md](2026-10-30-d-drca-3-dr4-method-routing.md) | | 2026-10-29 | `palette512-turbovec9-nsl-worksteal` | | [2026-10-29-palette512-turbovec9-nsl-worksteal.md](2026-10-29-palette512-turbovec9-nsl-worksteal.md) | | 2026-10-28 | `thought-layers-nan-skip` | | [2026-10-28-thought-layers-nan-skip.md](2026-10-28-thought-layers-nan-skip.md) | diff --git a/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md b/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md new file mode 100644 index 000000000..b92923974 --- /dev/null +++ b/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md @@ -0,0 +1,370 @@ +# D-TRACT-HARVEST-0 — the ONNX graph as a DR#3 graph through the DT64 registry (Tract as cold frontend) + +**Date:** 2026-10-30. **Status:** LAB PROBE IMPLEMENTED (crates/tract-harvest-probe, own workspace, excluded from the lance-graph workspace). Not a production path; no native crate depends on it. Implements the S1 (source preservation) and S3 (one admitted native subgraph) stages of D-ONNX-INTAKE (#1802) for the Tract arm only; Burn onnx-ir was not run. **Coordinator:** D-NAM / EXO-152. + +## 0. Redesign (2026-10-30): the ONNX graph as a DR#3 graph, resolved through the DT64 registry + +> **Supersedes** the fixed admission recipes (`admit.rs`, removed) and with them §7 and the recipe-specific rows of §1, §8 and §9. The operator's correction: *"We want the graph … The mapping of the Graph should then resolve to polyfill and predicate methods in DT64 verb 'on when do' schema."* Two decisions taken with it: the graph's home is the lance-graph DR#3 graph, and the binding key is the op signature → one immutable classid. Sections 1-13 below are kept as the record of the recipe probe; their measurements still hold, since the same native functions run. + +**Shape** (`crates/tract-harvest-probe`): + +| layer | file | what it holds | +|---|---|---| +| source | `source.rs`, `value.rs` | raw protobuf: node names, domain, opset, discrete attributes, initializer and `Constant` values decoded once (bool packed into mask words) | +| facts | `dr3::tract_facts` | Tract, cold: dtype and shape per labelled value (no declutter) | +| graph | `dr3::OnnxGraph` | nodes = **Identity**, values = **Property** on one 64x64 tile; `on`, `do_` as `Rows`; `dep = do_ ∘ read_by` by `dr_math_binding_probe::relcomp::compose`; ordered args beside the relations | +| registry | `registry.rs` | `Registration { classid, op signature (domain, op_type, dtype, attribute rules, opsets), WHEN: K3 guard, DO: polyfill, Numeric }`, versioned; 16 lab classids `0x0F01..0x0F51` | +| plan | `plan.rs` | resolve once: signature lookup (unregistered = UNKNOWN, refused), opset, WHEN must be TRUE, schedule from `dep`; sealed in a `FrameStore` frame (source digest, registry version, coordinates); `run` checks the frame, then routes each node to its DO | + +ON × WHEN × DO per node: ON = its input Property positions; WHEN = a three-valued guard over the facts and constant operands (multidirectional broadcast, `Pow` exponent exactly 2.0, single reduction axis, scalar threshold, `Where` operand shapes); DO = an existing function, never a new kernel (`lance_graph_onnx::{add,sub,mul,div,abs_f32,sqrt_f32,greater_i32,where_i32,ReduceSum}`, `ndarray` `mean_axis`). Classids are **lab ordinals**, not OGAR-minted concepts; production would mint them in OGAR and activate through `lance-graph-contract::hotplug`. + +**Results:** + +| check | result | +|---|---| +| tiny `base`, `gen2`, `neg_where_nonzero` through the graph | bit-exact vs ORT literal; `Where(c, X, 1)` now **runs** (17): there is no masked-sum rewrite to refuse, `Where` and `ReduceSum` are separate methods | +| tiny `div3`, 256 seeded inputs | bit-exact on every element (`Div` keeps classid `0x0F23`) | +| `dep` vs the source edges | equal on 3 variants; the schedule is a topological order of it, and a protobuf with the node list reversed still runs bit-exact | +| Jina v5 small layer-0 input RMSNorm (13 nodes, Constants included) | 9,259/10,240 exact, max 4 ulp; mean 9/10 exact, max 1 ulp; `ReduceMean` the only Bounded method; every row with a bit-exact mean is bit-exact on all 1,024 outputs | +| full Jina graph (7,620 nodes, inputs pinned `[1,10]`), registry v1 coverage | **3,751** WHEN-true, **169** registered but WHEN not true, **3,700** unregistered. All 113 RMSNorm `Pow`/`ReduceMean`/`Sqrt` resolve. The 169 are facts Tract does not keep: its typed stage labels only 2,546 of 7,620 source outputs, so those shapes are UNKNOWN and the guard refuses them. The largest unregistered groups: `Unsqueeze` 888, `Shape` 430, `Gather` 373, `Concat` 284, `MatMul` 253, `Reshape` 228, `Mul` (non-f32 / other signatures) 228 | + +**Refusals, each with its own test:** + +| input | refusal | +|---|---| +| `B[4]` vs `F[2,3]` | `When { n_add, 0x0F20, False }` (broadcast guard over the facts) | +| `Erf` | `Unregistered { n_abs, "ai.onnx:Erf(Some(F32)){}" }` | +| `com.example:Abs` (Tract imports it as `Abs`) | `Unregistered`, signature `com.example:Abs…`: the source domain keys the signature | +| `Pow(x, 2.0000002)` | `When { Pow, 0x0F32, False }` | +| `Cast_1` to FLOAT16, `Cast` to DOUBLE | `Unregistered`, signature names `to=10` / `to=11` | +| Tract's shapes removed | `When { n_where, Unknown }`: UNKNOWN never counts as TRUE | +| store advanced to `gen2`, or to a changed Jina weight | `Frame(SourceChanged)` at run time | +| registry version changed | `Frame(MapChanged)` | +| sum past int32 | `Native(OutOfDtypeRange)` at run time | + +`ReduceMean` over axis -2 instead of -1 is a different valid program, not a refusal: nothing assumes the RMSNorm pattern, so the graph runs it with the mean shaped `[1,1,1024]`. + +**Disable runs** (committed first, anchor asserted, `--no-fail-fast`; each went red on exactly the named test, tree restored): + +| guard disabled | red test | +|---|---| +| broadcast WHEN → TRUE | `incompatible_broadcast_is_when_false` | +| `Pow` exponent WHEN → TRUE | `pow_exponent_other_than_two_is_when_false` | +| attribute rules ignored | `cast_to_half_is_unregistered`, `cast_to_double_is_unregistered` | +| domain dropped from the signature | `foreign_domain_is_unregistered_although_tract_accepts_it` | +| frame check skipped in `run` | `stale_generation_is_refused`, `registry_change_is_a_frame_change`, `new_weight_generation_refuses_the_old_plan` | +| UNKNOWN accepted as TRUE | `unknown_shapes_refuse` | +| schedule = source order | `schedule_comes_from_dep_not_source_order` | + +**Cost** (`examples/cost.rs`, release, reported, not gated): tiny cold Tract facts 140 µs, tile + resolve 7.4 µs; hot plan 3.4 µs (54 allocs) vs Tract typed 3.6 µs. Jina slice cold facts 175 µs, tile + resolve 11.9 µs; hot plan 14.9 µs (62 allocs, 263 KB: every value materialized, the plan clones its constants per run) vs Tract typed 113 µs. + +**OPEN (graph design):** +- More than 64 nodes or values needs several DR64 tiles; the full graph is only censused, not tiled. +- Facts for the 5,074 values Tract's typed stage leaves unlabelled: from the inference model (labels kept 7,620/7,620) or from ONNX shape inference. +- Registry entries for the shape-plumbing ops (`Unsqueeze`, `Shape`, `Gather` on shapes, `Concat`, `Reshape`) and the admitted f32 `MatMul`. +- The plan clones constants per run; hot allocation is not yet minimal. +- Classids are lab-local; minting through OGAR is a separate decision. + +## 1. The acceptance chain, demonstrated + +The chain under test: a source-attested ONNX subgraph is imported by Tract, admitted, lowered onto existing native methods, executed without Tract, and verified against ONNX Runtime (ORT). The chain holds for both fixtures: fixture A (tiny, synthetic) is bit-exact, and fixture B (a real model slice) is bounded. The numbers are in §4. Nothing here is certified, production or merged. + +```text +.onnx bytes ──► source::SourceInventory (raw protobuf: domain, opset, initializers) + │ + └──► Tract InferenceModel ─► TypedModel (cold; shapes, dtypes, constants) + │ + admit::admit_*(source, typed) ─► Recipe + Receipt | Refusal + │ + Recipe::run ─► lance-graph-onnx / ndarray only +``` + +Invariants: + +- Recipes are concrete structs whose `run` calls lance-graph-onnx / ndarray directly. +- No graph is read at run time. +- There is no general lowering, interpreter, tensor AST or registry. +- Tract supplies structure, shapes and constants. The raw protobuf supplies domain, opset, initializer identity and digest. +- ORT runs with graph optimization DISABLED (`ORT_DISABLE_ALL`) as the source-semantics oracle. + +## 2. Baseline reconciled (what existed before this probe) + +| Reference | State at the time of this probe | +|---|---| +| #1753 D-NFC | Native-first rule: third-party ONNX is an external compatibility leaf. | +| #1754 / #1758 | `lance-graph-onnx` facade: ALIAS / INLINE / bounded Exp, Tanh. Float ReduceSum is NATIVE-not-exported pending a tolerance contract. Gather and MatMul are REFUSE in the facade. | +| #1752 | One `Greater -> Where -> ReduceSum` native composite. | +| #1779 – #1783 | D-NAM specs. PROPOSED, not implemented. | +| #1802 D-ONNX-INTAKE-0 | Bake-off capstone (docs). S1–S5 NOT RUN, not authorized. | +| #1810 | Board entry nominating Tract for first harvest attention (docs). | +| #1800 / #1801 | TFLite Softmax / Codec8 lab evidence. Merged, lab only, not reused here. | +| #1835 | DR/DT ownership decisions. DR = HHTL16 Cartesian addressing/projection. DT = ClassId -> class schema + method schema. DR #4 routes method ops. Thought-Git sits above. | + +Nothing was re-run from the certified TFLite TANH sweeps. + +## 3. Fixtures + +### 3.1 Fixture A: tiny (committed) + +Location: `crates/tract-harvest-probe/fixtures/tiny`, generated by `tools/gen_tiny.py`. Generated with ORT 1.29.0, onnx 1.23.2, opset 18. + +- `S = ReduceSum(Where(Greater(X, T), X, Z), axes=[0], keepdims=0)` with X int32[N] (symbolic N), T = 0, Z = 0. +- `Q = Div(Sqrt(Abs(Mul(Add(F, B), F))), B)` with F float[2,3] and B float[3] an initializer. + +| Variant | Change | +|---|---| +| base | as above | +| gen2 | new B bytes, same shape | +| div3 | B = [0.5, -2, 3] | +| neg_shape | B has shape [4] | +| neg_where_nonzero | Z = 1 | +| neg_unsupported | Erf in place of Abs | +| neg_domain | `com.example:Abs` | + +Plus a 256-input seeded div3 sweep (seed 20261030) run through ORT. + +### 3.2 Fixture B: real model slice (weights never committed) + +- Model: `jinaai/jina-embeddings-v5-text-small-retrieval` at revision `6856e76bb72982e58de0620458a4e8b3614da340`, license cc-by-nc-4.0. +- Weights are fetched by `tools/fetch_jina_slice.py` with SHA-256-pinned HTTP ranges into gitignored `fixtures/jina/`. The committed manifest `fixtures/jina_manifest.json` holds digests only. +- Architecture: Qwen3Model, 28 layers, hidden 1024, 16 heads / 8 KV heads, head_dim 128, intermediate 3072, rms_norm_eps 1e-6, rope_theta 3500000, vocab 151936, tie_word_embeddings true, pooling last-token (1_Pooling config). +- ONNX graph `onnx/model.onnx`: sha256 `463fb3fec2adf9c47a946626dc2a0884e43091f95466083fb1754950000a7c1a`, 1,266,078 bytes; producer pytorch 2.9.1; ir 8; opset 18; 7,620 nodes; 317 initializers, of which 310 are external FP32 in `model.onnx_data` (2,384,199,936 bytes). + +Op histogram (top): Constant 2759, Unsqueeze 888, Mul 567, Shape 430, Gather 373, Cast 350, Concat 284, Add 254, MatMul 253, Reshape 228, Div 169, Transpose 141, Slice 140, Where 114, Pow / ReduceMean / Sqrt 113 each, Expand 60, ConstantOfShape 59, Equal 58, Neg 56, Softmax 28, IsNaN 28, Sigmoid 28, Range 4, And 2, and one each of LessOrEqual, Flatten, Cos, Sin, ReduceSum, Sub, GatherElements, Squeeze. + +- Tokenizer: `tokenizer.json` sha256 `aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4`. +- Pinned text: "Harvest the algebra. Keep the native substrate." Token ids: `[26490, 7019, 279, 46876, 13, 13655, 279, 9867, 52957, 13]` (offsets recorded in the manifest). +- Slice: `/embed_tokens/Gather`, then `/layers.0/input_layernorm/{Cast, Pow, ReduceMean, Add, Sqrt, Div, Mul, Cast_1, Mul_1}` with its Constants (exponent 2.0, axes [-1], eps 9.999999974752427e-07, numerator 1.0). Weight `layers.0.input_layernorm.weight` [1024] at external offset 622330880, length 4096. +- The whole fetch took 7 s and 44 KB of weight bytes instead of 2.4 GB. + +## 4. Results + +### 4.1 Fixture A + +| Check | Result | +|---|---| +| Native recipe vs ORT on base, gen2, div3 | bit-exact; numeric class Exact | +| div3 sweep, native vs ORT | bit-exact on all 256 inputs (1,536 output elements) | +| div3 sweep, Tract typed vs ORT | bit-exact on all 256 inputs | +| div3 sweep, Tract decluttered vs ORT | differs on 161 of 1,536 elements, all in the divide-by-3 column (powers of two are exact under the rewrite) | + +### 4.2 Fixture B (10 x 1024 outputs), against ORT literal (`ORT_DISABLE_ALL`) + +| Arm | Mean (10 values) | Output (10,240 values) | +|---|---|---| +| Native | 9/10 exact, max 1 ulp | 9,259/10,240 exact, max 4 ulp | +| Tract typed (no declutter) | 0/10 exact, max 31 ulp | 0/10,240 exact, max 26 ulp | +| Tract decluttered (fused RmsNorm) | 0/10 exact, max 31 ulp | 7,401/10,240 exact, max 3 ulp | + +Every row whose mean is bit-equal gives a bit-equal output row (asserted), so the whole native deviation is the one bounded step: float summation order in ReduceMean. + +ORT literal vs ORT optimized (default `ORT_ENABLE_ALL`): 7,136/10,240 exact, max 2 ulp. ORT's own optimizer changes the bytes, so the oracle configuration is part of the contract. + +Against an f64 reference mean (f32 squares summed in f64, divided, rounded once): + +| Arm | Max error | +|---|---| +| Native | 2 ulp | +| ORT literal | 2 ulp | +| Tract typed | 33 ulp | + +Pinned bounds: native output <= 4 ulp vs ORT literal; native mean <= 4 ulp vs ORT literal and <= 4 ulp vs the f64 reference. + +### 4.3 Gather + +Byte ranges computed from the source initializer placement (base offset + row x 4096) equal the pinned ranges and the fetched rows' SHA-256 for all 10 tokens. Index `vocab` (151936) and `-vocab-1` are refused. Index -1 maps to row 151935 (the ONNX negative-index rule). + +## 5. Harvest map: Tract construct, retained semantics, owner, transformation, refusal / missing + +All rows are source-backed. + +| Tract construct | Retained / lost | Owner of the missing check | Falsifier or note | +|---|---|---|---| +| ONNX import (`onnx/src/model.rs:119-158`, `parse_graph`) | Node name kept; empty name becomes the first output name or `{n}-{op}`. Ops are looked up by op_type only (`model.rs:135`, `op_register.0.get(&pbnode.op_type)`), so the DOMAIN IS NOT CONSULTED. Unknown op becomes an `UnimplementedOp` placeholder that fails later at typing. Outlet labels kept for every output. Domain lost. | Admission (source inventory) | `neg_domain`: ORT refuses `com.example:Abs`; Tract imports it as Abs. | +| Opset (`model.rs:264-278`) | Only the default-domain version is kept, as one integer. Outside the tested range 9–18 Tract only logs a warning. | Admission enforces `ADMITTED_OPSETS` 13..=18 from the source. | | +| Model metadata (`model.rs:289-324`) | producer, version, domain, ir_version, doc_string, metadata_props kept as properties. | | | +| Initializers / constants (`hir/src/infer/model.rs:202-218`, `InferenceModel::add_const`) | `add_const` returns an existing Const of equal value, so two source initializers with equal values (T = 0, Z = 0) become ONE node named after the first; the second initializer's identity is lost. | Admission reads initializer identity and digests from the source inventory. | | +| Lazy external data (`model.rs:340-345`, `with_lazy_external_data`) | External weights stay as named references without reading bytes. | | This let the full 7,620-node graph be imported with no weight file. | +| Typing (`into_typed`) | Shapes and dtypes. Rank fix-up Consts / AddAxis nodes are synthesized (`.fix-rank`, `-dispose-dims-0`). ReduceMean is renamed `.mean` and becomes Reduce (doc: "Accumulates in f32"). | Native broadcastable check is the admission gate. | An impossible broadcast (B[4] vs F[2,3]) is typed symbolically as `broadcast((3),(4))` and fails only at run time ("Can not broadcast 3 against 4"); ORT refuses at load. | +| Flatten (`hir/src/ops/array/flatten.rs:11`) | Refuses a shape with more than one symbolic dim ("Can not compute a shape with square of symbols"). | | The Jina export flattens [batch_size, sequence_length], so the full model types only after pinning input facts (pinned [1,10]). | +| Declutter: Div | `Div(Y, B)` becomes `Mul(Y, recip(B))` (`n_div-recip` Const). Not numerically identical (161/1,536 differ) and removes the source Div op. | | `declutter_div_to_reciprocal_is_not_exact` | +| Declutter: RMSNorm | Pattern Pow -> ReduceMean -> Add -> Sqrt -> Div -> Mul becomes one fused RmsNorm op (plus Reduce for a separately requested mean). Source names of Pow/Add/Sqrt/Div/Mul/Cast are lost. | | Worth harvesting as a recognizer, not as a kernel. | +| Declutter: Silu, EinSum | `x * Sigmoid(x)` becomes Silu (28 in Jina). MatMul becomes EinSum (253). | | | + +### 5.1 Classification of Tract transformations + +| Class | Members observed | +|---|---| +| Algebraically equivalent | Cast f32 -> f32 identity; `Pow(x, 2)` -> `x * x` when the exponent is exactly 2.0; rank fix-ups; constant dedup of equal values (numerically equivalent but provenance-losing) | +| Numerically bounded, non-identical | Div -> Mul(recip); ReduceMean summation order; fused RmsNorm; ORT's own optimizer | +| Backend-specific | EinSum lowering, packing (not exercised) | +| Provenance-losing | domain; const dedup; declutter fusion names; shape-folded nodes | +| No native counterpart today | see §6 | + +## 6. Native coverage of the full Jina graph + +Source: the Tract decluttered graph of the pinned [1,10] model, 2,106 nodes. Histogram: Add 140, AddAxis 312, And 1, Cast 1, Concat 57, Const 346, Cos 1, EinSum 253, Gather 2, GatherElements 1, Iff 29, IsNan 28, MoveAxis 56, Mul 309, MultiBroadcastTo 29, Neg 56, Reduce 1, Reshape 196, RmAxis 3, RmsNorm 113, Silu 28, Sin 1, Slice 112, Softmax 28, Source 2, Sub 1. + +| Class | Ops | Notes | +|---|---|---| +| ADMITTED in this probe | RmsNorm (BOUNDED recipe); elementwise Add / Mul / Sub / Div / Neg via the facade (exact) | | +| NATIVE-NOT-EXPORTED | Softmax; Silu | `ndarray::hpc::activations::softmax_axis_f32` exists. `sigmoid_f32` exists but Sigmoid has no measured bound. Exp / Tanh bounded exist. | +| MISSING | MatMul / EinSum | Facade REFUSE until the CompiledLinear identity defect is resolved. `bf16_gemm_f32` / `blas_gemm` exist in ndarray but there is no admitted f32 MatMul contract. | +| MISSING | Gather (f32 embedding) | The probe resolves byte ranges; the facade REFUSEs. | +| MISSING | Cos / Sin (RoPE) | No bounded native kernel exported. | +| MISSING | IsNaN; Iff over f32; GatherElements | | +| MISSING | Layout ops: Reshape, AddAxis, MoveAxis, Slice, Concat, MultiBroadcastTo | ndarray views exist but no admitted contract. | + +Whole-model native inference is NOT reachable today. The first blocker is MatMul. + +### 6.1 Per-stage provenance of the full model + +Pinned [1,10], lazy external data. Timings: import 25.6 ms, into_typed 153 ms, declutter 221 ms. + +| Stage | Nodes | Source names kept | Outputs labelled | +|---|---|---|---| +| Inference | 7,937 | 7,605 (15 source nodes have empty names); 17 synthesized | 7,620/7,620 | +| Typed | 3,170 | 1,985 | 2,546/7,620 | +| Decluttered | 2,106 | 1,046 | 1,164/7,620 | + +At the typed stage, shape computation is folded away: Constant 2,759, Unsqueeze 884, Shape 428, Gather 369, Reshape 228, Concat 227, Transpose 141, Cast 118, Mul 115, ReduceMean 113 (renamed), and others. + +### 6.2 Harvest ledger + +| Item | State | Owner | +|---|---|---| +| Tract `InferenceModel → TypedModel` as cold frontend | IMPLEMENTED (lab, this probe) | D-ONNX-INTAKE / D-NAM (EXO-152) | +| Raw-protobuf source inventory (domain, opset, initializer identity and digest) | IMPLEMENTED (lab) | this probe; would move to the D-NAM intake owner | +| Two admitted methods (`TinyRecipe` exact, `RmsNormRecipe` bounded) | IMPLEMENTED (lab) | D-NAM method contracts | +| `Gather` axis 0 from source placement, ONNX index rule | IMPLEMENTED (lab); facade still REFUSE | `lance-graph-onnx` (reclassification is an owner decision) | +| Elementwise float ops, `Sqrt`, `Abs`, int32 compare / masked sum | REUSED from the facade, unchanged | `lance-graph-onnx` (#1754 / #1758) | +| `ndarray` `mean_axis`, `softmax_axis_f32`, `sigmoid_f32`, `blas_gemm`, `bf16_gemm_f32` | EXIST, not exported for ONNX | ndarray; export decisions D-ONNX-NATIVE-1 | +| Tract fusions (RmsNorm, Silu) | REUSABLE as pattern evidence only | — | +| Tract numerical kernels | REFUSED for harvest (`Reduce` 33 ulp from the reference) | — | +| Tract declutter as an admission source | REFUSED (Div → Mul(recip), name loss) | — | +| f32 MatMul, RoPE Cos/Sin, IsNaN, f32 Where, GatherElements, layout views | MISSING | see §12 | +| Burn `onnx-ir` arm | NOT RUN; DEFER | D-ONNX-INTAKE | +| Earlier TFLite TANH / Softmax evidence (#1800, #1801, #1810) | unchanged, not re-run, not superseded | D-TFLP | + +## 7. Admission contract + +`Refusal` variants: + +| Variant | Meaning | +|---|---| +| Frontend | The frontend could not import or type the graph. | +| ForeignDomain | A node's domain is not the default ONNX domain. | +| Opset | The source opset is outside `ADMITTED_OPSETS` (13..=18). | +| UnsupportedOp | An operator with no admitted method. | +| Pattern | The subgraph does not match the method's pattern. | +| RewriteAssumption | A law the method relies on does not hold (for example the Where else-value is not 0). | +| Dtype | A dtype check failed. | +| Native(lance_graph_onnx::Refusal) | The native facade refused. | +| StaleGeneration | Initializer digests differ from those the recipe was prepared with. | + +`Numeric` is `Exact` or `Bounded { op, reason }`. + +`Receipt` fields: `method`, `model_sha256`, `opset`, `frontend`, `source_nodes` (source names in graph order), `initializers` (name, placement digest), `numeric`. It has a `check_generation` method. + +Two methods exist: + +- **TinyRecipe.** Laws: `ReduceSum(Where(c, X, 0)) == masked_sum(X, c)` requires Z == 0 and the same X; B read twice is one weight; broadcast decided once. +- **RmsNormRecipe.** Both Casts are f32 -> f32 identity; exponent exactly 2.0; numerator exactly 1.0; mean over the last axis with keepdims; weight length equals hidden. Numeric class: Bounded. + +## 8. Falsifiers and disable runs + +17 tests: 10 tiny, 7 Jina. The Jina tests are `#[ignore]` until the fixture is fetched, so a fresh clone reports them as not run. `cargo test` stops at the first failing test binary unless `--no-fail-fast` is given; the disable runs use it. + +| ID | Disabled check | Test that goes RED | +|---|---|---| +| T1 | Where else-value need not be 0 | `invalid_rewrite_assumption_is_refused` | +| T2 | Domain not checked | `foreign_domain_is_refused_although_tract_accepts_it` | +| T3 | Generation not checked | `new_weight_generation_is_refused_by_the_old_recipe`, `stale_weight_generation_is_refused` | +| T4 | Native Div replaced by Mul(recip) | `declutter_div_to_reciprocal_is_not_exact` | +| T5 | Pow exponent not checked | `pow_exponent_other_than_two_is_refused` | +| T6 | Gather negative index not wrapped | `gather_rows_from_source_placement` | +| T7 | Gather upper bound not checked | `gather_rows_from_source_placement` | +| T8 | Mean axis not checked | `reduction_over_another_axis_is_refused` | +| T9 | Cast dtype not checked | `cast_to_double_is_refused_by_the_dtype_check` | +| T10 | Broadcast not checked | `incompatible_shape_is_refused` | +| T11 | Unsupported op admitted | `unsupported_operator_is_refused` | +| T12 | Native mean replaced by a sequential f32 sum | `rmsnorm_slice_against_ort` | + +T9 note: the first attempt used an f16 `Cast_1` and stayed green, because Tract inserts its own cast and the pattern refuses first. The f64 control was added so the dtype check binds. + +## 9. Measured cost (reported, not gated) + +Release build, AVX-512 host, medians shown as ranges. + +| Step | Tiny | Jina slice 10 x 1024 | +|---|---|---| +| Source inventory | 13.7–14.0 us | 18.6–19.8 us | +| Tract import + into_typed | 142–143 us | 176–177 us | +| Admission | 1.3 us | 2.9–3.0 us | +| Native run | 353–363 ns (5 allocations, 120 B) | 8.9–9.4 us (9 allocations, 123,048 B) | +| Tract typed runnable | 3.6–3.8 us (26 allocations, 3,119 B incl. input tensors) | 112–114 us | +| Tract decluttered fused RmsNorm | n/a | 22.5–22.7 us | + +The native Jina run materializes every intermediate, because the facade's broadcasting binary adapters return owned arrays. + +Full model, cold (no weights): 25.6 ms import, 153 ms into_typed, 221 ms declutter. + +## 10. DR/DT DO-arm sketch (design only, nothing built) + +| Selection element | Mapped to | +|---|---| +| Source-qualified operation identity | (model_sha256, opset, domain, source node names) | +| Algebraic method + proof receipt | method id + the laws checked + `Numeric` | +| Prepared numerical recipe | the recipe struct (constants copied out at preparation) | +| Hardware specialization | inside ndarray's own SIMD dispatch, not a lance-graph choice | +| Data / calibration generation | initializer placement digests, checked by `check_generation` before reuse | +| Explicit refusal | the `Refusal` enum | + +DT (ClassId -> method schema, #1835) would hold the method identity binding. DR routes coordinates. The DO arm executes a pre-admitted recipe and never re-reads the ONNX graph. Not built: no store, no cache, no DR cube mask. Open owner decisions are listed in §12. + +## 11. Research trajectory (forward map, not authorization) + +Text -> exact tokenizer ids / offsets (pinned, recorded) -> native transformer -> last-token pooling -> graph-aware retrieval. + +- Token boundaries are byte spans, not semantic relations. Any lexical-graph bridge must keep them separate. +- TensorFlow / TFLite learning: gradients, optimizer state, quantized training, intermediate rounding and weight-generation updates are new semantic contracts. Inference parity does not imply learning parity. +- The generation check here (initializer digests) is the hook a learning loop would have to respect. It is not a learning mechanism. + +## 12. Next PR sequence + +Smallest independent steps: + +1. Owner decision: float reduction tolerance contract (D-ONNX-NATIVE-1), using this probe's measured bounds. Then export a bounded ReduceMean / ReduceSum in `lance-graph-onnx::bounded`. +2. Admitted f32 MatMul contract (resolve the CompiledLinear identity defect; ndarray `blas_gemm` / `bf16_gemm_f32` exist) with one Jina q_proj slice. +3. Gather axis-0 reclassification from REFUSE to INLINE for initializer tables (the index rule is tested here). +4. Softmax and Silu as bounded exports after measured bounds. +5. RoPE Cos / Sin bounded kernels. +6. Layout ops as view contracts. +7. A fused native RMSNorm kernel in ndarray if the 9-pass materializing composition is too costly (missing primitive, owner ndarray). +8. One full decoder layer slice, then the whole model. +9. Burn onnx-ir arm of the bake-off only if a later stage needs source fidelity Tract cannot supply. Tract plus the raw-protobuf inventory was sufficient for both fixtures here, so Burn is DEFER, not run. + +Then the TFLite learning frontier. + +Owner decisions OPEN: + +- Tolerance contract (D-ONNX-NATIVE-1 / EXO-152). +- MatMul identity (CompiledLinear). +- Gather reclassification. +- Whether `ORT_DISABLE_ALL` is the standing oracle setting. + +## 13. Verdict + +Tract is ADOPTED as the cold typed frontend for this lab stage, on the condition that admission always pairs it with the raw-protobuf inventory. Tract loses domain, initializer identity and, after declutter, names and Div exactness. + +- Do not admit from Tract's decluttered graph. Use its fusions (RmsNorm, Silu) as pattern evidence only. +- Tract's numerical kernels are not harvested (Reduce is 33 ulp from the reference mean). +- The primary falsifier (Tract cannot retain enough source information) did NOT fire, because the gaps are recoverable from the protobuf. + +## Reproduce + +```text +cd crates/tract-harvest-probe && python3 tools/gen_tiny.py && python3 tools/fetch_jina_slice.py && cargo test --release --no-fail-fast -- --include-ignored && cargo run --release --example stages -- fixtures/jina/onnx_model.onnx 10 && cargo run --release --example cost +``` + +Versions: lance-graph base 61720fa (main), AdaWorldAPI/tract main bf7ca16, AdaWorldAPI/ndarray master 50381182 (one ndarray package in the probe graph, via `[patch.crates-io]`), Rust 1.99.0, ORT 1.29.0 (python, CPUExecutionProvider), onnx 1.23.2, tokenizers 0.23.3. diff --git a/Cargo.toml b/Cargo.toml index 8707c2413..f4f274c88 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -142,6 +142,9 @@ exclude = [ "crates/lance-graph-ogar", # D-DRF-P1 lab probe (immutable DR64 focus lease). Own [workspace]; ndarray only. "crates/drf-focus-probe", + # D-TRACT-HARVEST-0 lab probe: Tract as a cold ONNX frontend. Own [workspace]; + # no native crate depends on it. `cargo test --manifest-path crates/tract-harvest-probe/Cargo.toml`. + "crates/tract-harvest-probe", # A lance-graph ReportResult as an addressed OGAR ObjectSlot source # (implements ogar-doc-ir's DocObjectSource::grid_of). EXCLUDED for the same # reason as lance-graph-ogar: path-deps the OGAR sibling. Verify via diff --git a/crates/tract-harvest-probe/.gitignore b/crates/tract-harvest-probe/.gitignore new file mode 100644 index 000000000..ce6c2c641 --- /dev/null +++ b/crates/tract-harvest-probe/.gitignore @@ -0,0 +1,3 @@ +target/ +# CC-BY-NC-4.0 model data: fetched by tools/fetch_jina_slice.py, never committed. +fixtures/jina/ diff --git a/crates/tract-harvest-probe/Cargo.toml b/crates/tract-harvest-probe/Cargo.toml new file mode 100644 index 000000000..58c85ad7a --- /dev/null +++ b/crates/tract-harvest-probe/Cargo.toml @@ -0,0 +1,36 @@ +# tract-harvest-probe — D-TRACT-HARVEST-0, lab only. +# Tract is a COLD, optional frontend here: it types an ONNX graph. The graph +# itself is laid onto a DR#3 tile (nodes = Identity, values = Property) and +# every node is resolved through a DT64 ON x WHEN x DO registry onto existing +# native functions (lance-graph-onnx aliases/inlines, ndarray). Execution never +# calls Tract, and no native crate depends on this one. +# Own [workspace] root, excluded from the lance-graph workspace. Run from here: +# cargo test +# cargo run --release --example stages (per-stage provenance inventory) +# cargo run --release --example cost (cold/hot costs, reported not gated) +[package] +name = "tract-harvest-probe" +version = "0.0.0" +edition = "2021" +publish = false +description = "Lab probe D-TRACT-HARVEST-0: ONNX graph as a DR#3 graph resolved through a DT64 ON x WHEN x DO registry onto existing native methods; Tract as cold typing frontend; ORT parity. Results only; no production path." + +[workspace] + +[dependencies] +tract-onnx = { git = "https://github.com/AdaWorldAPI/tract", branch = "main" } +lance-graph-onnx = { path = "../lance-graph-onnx" } +# The DR64 relation register, Boolean composition and the frame store. +dr-math-binding-probe = { path = "../dr-math-binding-probe" } +lance-graph-contract = { path = "../lance-graph-contract" } +ndarray = { git = "https://github.com/AdaWorldAPI/ndarray", branch = "master", default-features = false, features = ["std"] } +sha2 = "0.10" +# The protobuf types are tract-onnx's own (`tract_onnx::pb`); decoding them needs the trait. +prost = "0.14" +serde_json = "1" + +# Tract's own workspace patches crates.io ndarray to the AdaWorldAPI fork; a +# dependent workspace must repeat it, so the graph holds ONE ndarray package +# (the coordinate lance-graph-onnx already uses). +[patch.crates-io] +ndarray = { git = "https://github.com/AdaWorldAPI/ndarray", branch = "master" } diff --git a/crates/tract-harvest-probe/examples/cost.rs b/crates/tract-harvest-probe/examples/cost.rs new file mode 100644 index 000000000..1a34958a6 --- /dev/null +++ b/crates/tract-harvest-probe/examples/cost.rs @@ -0,0 +1,182 @@ +//! Cold and hot costs, reported, not gated. +//! +//! `cargo run --release --example cost` (the Jina rows need +//! `python3 tools/fetch_jina_slice.py` first; they are skipped and said so +//! otherwise). +//! +//! Cold: source inventory (raw protobuf), Tract typing (facts), laying the +//! graph onto the DR#3 tile and resolving every node against the registry. +//! Hot: one plan run (routing each node to its DO), against Tract's own +//! runnable of the same typed graph. Allocations are counted per hot call. + +use std::alloc::{GlobalAlloc, Layout, System}; +use std::hint::black_box; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::time::Instant; + +use dr_math_binding_probe::FrameStore; +use ndarray::{ArrayD, IxDyn}; +use tract_harvest_probe::dr3::{tract_facts, OnnxGraph}; +use tract_harvest_probe::plan::{resolve, revisions}; +use tract_harvest_probe::registry::registry; +use tract_harvest_probe::source::SourceInventory; +use tract_harvest_probe::value::Value; +use tract_onnx::prelude::*; + +struct Counting; +static ALLOCS: AtomicU64 = AtomicU64::new(0); +static BYTES: AtomicU64 = AtomicU64::new(0); +unsafe impl GlobalAlloc for Counting { + unsafe fn alloc(&self, l: Layout) -> *mut u8 { + ALLOCS.fetch_add(1, Ordering::Relaxed); + BYTES.fetch_add(l.size() as u64, Ordering::Relaxed); + // SAFETY: forwarded unchanged to the system allocator. + unsafe { System.alloc(l) } + } + unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + // SAFETY: forwarded unchanged to the system allocator. + unsafe { System.dealloc(p, l) } + } +} +#[global_allocator] +static GLOBAL: Counting = Counting; + +fn counted(f: impl FnOnce() -> T) -> (T, u64, u64) { + let (a, b) = ( + ALLOCS.load(Ordering::Relaxed), + BYTES.load(Ordering::Relaxed), + ); + let r = f(); + ( + r, + ALLOCS.load(Ordering::Relaxed) - a, + BYTES.load(Ordering::Relaxed) - b, + ) +} + +/// Median of `runs` timings, in µs (cold) or ns (hot). +fn median(runs: usize, mut f: impl FnMut()) -> f64 { + f(); + let mut t: Vec = (0..runs) + .map(|_| { + let s = Instant::now(); + f(); + s.elapsed().as_nanos() as f64 + }) + .collect(); + t.sort_by(f64::total_cmp); + t[runs / 2] +} + +fn typed(bytes: &[u8]) -> TypedModel { + let onnx = tract_onnx::onnx(); + onnx.model_for_proto_model(&onnx.proto_model_for_read(&mut &*bytes).unwrap()) + .unwrap() + .into_typed() + .unwrap() +} + +/// Cold costs, then the hot plan run against Tract's typed runnable. +fn measure( + label: &str, + bytes: &[u8], + pins: &[(usize, DatumType, Vec)], + inputs: Vec<(&str, Value)>, + tract_in: impl Fn() -> TVec, + runs: usize, +) { + let reg = registry(); + let src_us = median(51, || { + black_box(SourceInventory::from_bytes(black_box(bytes)).unwrap()); + }) / 1e3; + let facts_us = median(51, || { + black_box(tract_facts(black_box(bytes), pins).unwrap()); + }) / 1e3; + let src = SourceInventory::from_bytes(bytes).unwrap(); + let facts = tract_facts(bytes, pins).unwrap(); + let resolve_us = median(51, || { + let g = OnnxGraph::build(&src, &facts).unwrap(); + let store = FrameStore::new(revisions(&src, ®, &g)); + black_box(resolve(&src, g, ®, &store).unwrap()); + }) / 1e3; + let g = OnnxGraph::build(&src, &facts).unwrap(); + let store = FrameStore::new(revisions(&src, ®, &g)); + let p = resolve(&src, g, ®, &store).unwrap(); + let hot_ns = median(runs, || { + black_box(p.run(&store, black_box(&inputs)).unwrap()); + }); + let (_, na, nb) = counted(|| p.run(&store, &inputs).unwrap()); + let plan = typed(bytes).into_runnable().unwrap(); + let tract_ns = median(runs, || { + black_box(plan.run(tract_in()).unwrap()); + }); + println!("{label:>5} cold: source {src_us:8.1} µs | Tract facts {facts_us:8.1} µs | tile+resolve {resolve_us:7.1} µs ({} steps)", p.steps.len()); + println!("{label:>5} hot: graph plan {hot_ns:9.0} ns ({na} allocs, {nb} B) | Tract typed runnable {tract_ns:9.0} ns (incl. input tensors)"); +} + +fn main() { + let dir = concat!(env!("CARGO_MANIFEST_DIR"), "/fixtures"); + + // ── fixture A ── + let tiny = std::fs::read(format!("{dir}/tiny/div3.onnx")).unwrap(); + let x = vec![3, -1, 4, -1, 5, -9, 2]; + let f = vec![1.5f32, -0.25, 3.0, 0.1, 7.0, -2.5]; + let ins = vec![ + ( + "X", + Value::I32(ArrayD::from_shape_vec(IxDyn(&[7]), x.clone()).unwrap()), + ), + ( + "F", + Value::F32(ArrayD::from_shape_vec(IxDyn(&[2, 3]), f.clone()).unwrap()), + ), + ]; + measure( + "tiny", + &tiny, + &[], + ins, + || { + tvec![ + tract_ndarray::Array1::from(x.clone()) + .into_dyn() + .into_tensor() + .into(), + tract_ndarray::Array2::from_shape_vec((2, 3), f.clone()) + .unwrap() + .into_dyn() + .into_tensor() + .into() + ] + }, + 20001, + ); + + // ── fixture B ── + let slice = match std::fs::read(format!("{dir}/jina/slice.onnx")) { + Ok(b) => b, + Err(_) => { + println!(" jina NOT RUN: fixtures/jina absent (python3 tools/fetch_jina_slice.py)"); + return; + } + }; + let raw = std::fs::read(format!("{dir}/jina/gather_rows.f32")).unwrap(); + let xs: Vec = raw + .as_chunks::<4>() + .0 + .iter() + .map(|c| f32::from_le_bytes(*c)) + .collect(); + let rows = xs.len() / 1024; + let xa = ArrayD::from_shape_vec(IxDyn(&[1, rows, 1024]), xs).unwrap(); + let src = SourceInventory::from_bytes(&slice).unwrap(); + let input = src.inputs[0].clone(); + measure( + "jina", + &slice, + &[(0, DatumType::F32, vec![1, rows, 1024])], + vec![(input.as_str(), Value::F32(xa.clone()))], + || tvec![xa.clone().into_tensor().into()], + 2001, + ); +} diff --git a/crates/tract-harvest-probe/examples/stages.rs b/crates/tract-harvest-probe/examples/stages.rs new file mode 100644 index 000000000..02c7f4858 --- /dev/null +++ b/crates/tract-harvest-probe/examples/stages.rs @@ -0,0 +1,98 @@ +//! Per-stage provenance of a real model against its raw protobuf. +//! +//! `cargo run --release --example stages -- fixtures/jina/onnx_model.onnx` +//! +//! Weights are not needed: Tract's lazy external data keeps every external +//! initializer as a named weight whose bytes are never read. The registry +//! coverage reads facts from the source path (weights stay lazy there too). +use std::time::Instant; +use tract_harvest_probe::{dr3, plan, provenance, registry, source::SourceInventory}; +use tract_onnx::prelude::*; + +fn main() -> TractResult<()> { + let path = std::env::args().nth(1).expect("model path"); + let bytes = std::fs::read(&path)?; + let t = Instant::now(); + let src = SourceInventory::from_bytes(&bytes)?; + println!( + "source: sha256 {} ir {} opsets {:?} nodes {} initializers {} ({:?})", + src.model_sha256, + src.ir_version, + src.opsets, + src.nodes.len(), + src.initializers.len(), + t.elapsed() + ); + println!( + "source nodes with empty name: {}", + src.nodes.iter().filter(|n| n.name.is_empty()).count() + ); + + let onnx = tract_onnx::onnx().with_lazy_external_data(true); + let t = Instant::now(); + let mut inf = onnx.model_for_path(&path)?; + // Tract's Flatten refuses a shape with more than one symbol + // (hir/src/ops/array/flatten.rs:11), and the export flattens + // [batch_size, sequence_length]. Pin the inputs to the slice's shape. + if let Some(seq) = std::env::args() + .nth(2) + .and_then(|s| s.parse::().ok()) + { + for i in 0..inf.inputs.len() { + inf.set_input_fact(i, InferenceFact::dt_shape(i64::datum_type(), [1, seq]))?; + } + println!("inputs pinned to [1, {seq}] i64"); + } + let t_inf = t.elapsed(); + let t = Instant::now(); + let typed = inf.clone().into_typed()?; + let t_typed = t.elapsed(); + let t = Instant::now(); + let decl = typed.clone().into_decluttered()?; + let t_decl = t.elapsed(); + println!("timings: import {t_inf:?}, into_typed {t_typed:?}, declutter {t_decl:?}"); + for p in [ + provenance::of_graph("inference", &src, &inf), + provenance::of_graph("typed", &src, &typed), + provenance::of_graph("decluttered", &src, &decl), + ] { + println!("{}: nodes {}, source names kept {}, lost {:?}, synthesized {}, source outputs labelled {}/{}", + p.stage, p.nodes, p.source_names_kept, p.source_names_lost, p.synthesized_names, + p.source_outputs_labelled, p.source_outputs_total); + } + let mut ops = std::collections::BTreeMap::::new(); + for n in decl.nodes() { + *ops.entry(n.op.name().to_string()).or_default() += 1; + } + println!("decluttered op histogram: {ops:?}"); + + // Registration coverage: per op type, [WHEN true, WHEN not true, unregistered]. + let pins: Vec<_> = std::env::args() + .nth(2) + .and_then(|s| s.parse::().ok()) + .map(|seq| { + (0..inf.inputs.len()) + .map(|i| (i, i64::datum_type(), vec![1, seq])) + .collect() + }) + .unwrap_or_default(); + let facts = dr3::tract_facts_path(std::path::Path::new(&path), &pins); + match facts { + Ok(facts) => { + let reg = registry::registry(); + let cov = plan::coverage(&src, &facts, ®); + let tot = cov + .values() + .fold([0; 3], |a, c| [a[0] + c[0], a[1] + c[1], a[2] + c[2]]); + println!( + "registry v{} coverage [when-true, when-not-true, unregistered] total {tot:?}", + reg.version + ); + for (op, c) in &cov { + println!(" {op:>24} {c:?}"); + } + } + Err(e) => println!("registry coverage NOT RUN: typing failed: {e:?}"), + } + Ok(()) +} diff --git a/crates/tract-harvest-probe/fixtures/jina_manifest.json b/crates/tract-harvest-probe/fixtures/jina_manifest.json new file mode 100644 index 000000000..089c3c121 --- /dev/null +++ b/crates/tract-harvest-probe/fixtures/jina_manifest.json @@ -0,0 +1,192 @@ +{ + "repo": "jinaai/jina-embeddings-v5-text-small-retrieval", + "revision": "6856e76bb72982e58de0620458a4e8b3614da340", + "license": "cc-by-nc-4.0 (weights never committed)", + "pins": { + "onnx/model.onnx": "463fb3fec2adf9c47a946626dc2a0884e43091f95466083fb1754950000a7c1a", + "tokenizer.json": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4" + }, + "text": "Harvest the algebra. Keep the native substrate.", + "token_ids": [ + 26490, + 7019, + 279, + 46876, + 13, + 13655, + 279, + 9867, + 52957, + 13 + ], + "offsets": [ + [ + 0, + 3 + ], + [ + 3, + 7 + ], + [ + 7, + 11 + ], + [ + 11, + 19 + ], + [ + 19, + 20 + ], + [ + 20, + 25 + ], + [ + 25, + 29 + ], + [ + 29, + 36 + ], + [ + 36, + 46 + ], + [ + 46, + 47 + ] + ], + "graph": { + "ir_version": 8, + "opset": [ + [ + "ai.onnx", + 18 + ] + ], + "nodes": 7620, + "initializers": 317 + }, + "slice": { + "nodes": [ + "/layers.0/input_layernorm/Cast", + "/layers.0/input_layernorm/Constant", + "/layers.0/input_layernorm/Pow", + "/layers.0/input_layernorm/Constant_1", + "/layers.0/input_layernorm/ReduceMean", + "/layers.0/input_layernorm/Constant_2", + "/layers.0/input_layernorm/Add", + "/layers.0/input_layernorm/Sqrt", + "/layers.0/input_layernorm/Constant_3", + "/layers.0/input_layernorm/Div", + "/layers.0/input_layernorm/Mul", + "/layers.0/input_layernorm/Cast_1", + "/layers.0/input_layernorm/Mul_1" + ], + "input": "/embed_tokens/Gather_output_0", + "output": "/layers.0/input_layernorm/Mul_1_output_0", + "intermediate": "/layers.0/input_layernorm/ReduceMean_output_0", + "shape": [ + 1, + 10, + 1024 + ], + "sha256": "ba60320a3ae7c814bed9220a53c0ea045085cd9991a37038b5a65a72c91db535" + }, + "embedding": { + "name": "embed_tokens.weight", + "dims": [ + 151936, + 1024 + ], + "location": "model.onnx_data", + "base_offset": 0, + "ranges": [ + { + "token_id": 26490, + "offset": 108503040, + "length": 4096, + "sha256": "aea7a739cdb78cb86f156dcfed697126bda6149abb87cb07e8dba9617190da63" + }, + { + "token_id": 7019, + "offset": 28749824, + "length": 4096, + "sha256": "514eae438ee7d5750d3354f14d33d6e83722394a89de01ff955a7f7acfe37ef1" + }, + { + "token_id": 279, + "offset": 1142784, + "length": 4096, + "sha256": "9e31691e271bbd8718894686deefb844692356afeda9d89d4fafbcad5a9e57cf" + }, + { + "token_id": 46876, + "offset": 192004096, + "length": 4096, + "sha256": "bbdcbba8ff0b0136c659b425be43cf608ee1328ec8bb743b9cde1f9f76eea109" + }, + { + "token_id": 13, + "offset": 53248, + "length": 4096, + "sha256": "6a0841de1494fe9b533dee77da18e8d22bc6084970b68e467147b5caf0346aad" + }, + { + "token_id": 13655, + "offset": 55930880, + "length": 4096, + "sha256": "5f028b53d322247fdf8d23a5a67fea0c3397095c4e703272e8bab68b34ff3abf" + }, + { + "token_id": 279, + "offset": 1142784, + "length": 4096, + "sha256": "9e31691e271bbd8718894686deefb844692356afeda9d89d4fafbcad5a9e57cf" + }, + { + "token_id": 9867, + "offset": 40415232, + "length": 4096, + "sha256": "807f2d5aa261ca0124b930f4062c858079d243beb5c76fb9399f3a6572819014" + }, + { + "token_id": 52957, + "offset": 216911872, + "length": 4096, + "sha256": "e40061a6c1abdfcee309b5fa706dc784aecf88c925062bc5d4edb4dc3451c142" + }, + { + "token_id": 13, + "offset": 53248, + "length": 4096, + "sha256": "6a0841de1494fe9b533dee77da18e8d22bc6084970b68e467147b5caf0346aad" + } + ] + }, + "norm_weight": { + "name": "layers.0.input_layernorm.weight", + "offset": 622330880, + "length": 4096, + "sha256": "f9fd0ee309a218e18408ae63eea065ac46f45e771af6400f6b2e6bed7bb85539" + }, + "ort": { + "version": "1.29.0", + "provider": "CPUExecutionProvider", + "literal": { + "graph_optimization_level": "ORT_DISABLE_ALL", + "out_sha256": "422f89d92d88260a0572dca3f759d88ec192cfdff8e3d412d5fa4420aa1dbc1d", + "mid_sha256": "a4e7c2041e161233c262021a75e54f0303ca19ad2c36fedc1a5dd099d337a3e5" + }, + "optimized": { + "graph_optimization_level": "ORT_ENABLE_ALL", + "out_sha256": "530a0eb261dfa5ead4a69caa514e49d071d8db32559ed56c1bf2623bdae89c8c", + "mid_sha256": "a4e7c2041e161233c262021a75e54f0303ca19ad2c36fedc1a5dd099d337a3e5" + } + } +} \ No newline at end of file diff --git a/crates/tract-harvest-probe/fixtures/tiny/base.onnx b/crates/tract-harvest-probe/fixtures/tiny/base.onnx new file mode 100644 index 000000000..998129e08 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/base.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/div3.onnx b/crates/tract-harvest-probe/fixtures/tiny/div3.onnx new file mode 100644 index 000000000..18d5406eb Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/div3.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/gen2.onnx b/crates/tract-harvest-probe/fixtures/tiny/gen2.onnx new file mode 100644 index 000000000..25ae2dd35 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/gen2.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/neg_domain.onnx b/crates/tract-harvest-probe/fixtures/tiny/neg_domain.onnx new file mode 100644 index 000000000..b5e822b96 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/neg_domain.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/neg_shape.onnx b/crates/tract-harvest-probe/fixtures/tiny/neg_shape.onnx new file mode 100644 index 000000000..ad09fea54 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/neg_shape.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/neg_unsupported.onnx b/crates/tract-harvest-probe/fixtures/tiny/neg_unsupported.onnx new file mode 100644 index 000000000..3d9c9007b Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/neg_unsupported.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/neg_where_nonzero.onnx b/crates/tract-harvest-probe/fixtures/tiny/neg_where_nonzero.onnx new file mode 100644 index 000000000..3683235af Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/neg_where_nonzero.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/ort.json b/crates/tract-harvest-probe/fixtures/tiny/ort.json new file mode 100644 index 000000000..b716d9119 --- /dev/null +++ b/crates/tract-harvest-probe/fixtures/tiny/ort.json @@ -0,0 +1,4285 @@ +{ + "ort_version": "1.29.0", + "ort_graph_optimization_level": "ORT_DISABLE_ALL", + "onnx_version": "1.23.2", + "variants": { + "base": { + "sha256": "927dee7503f9a65dcb09890a113183c194e6719eb71ab110a1350041da69be9d", + "ort": { + "S": { + "shape": [], + "dtype": "int32", + "bits": [ + 14 + ] + }, + "Q": { + "shape": [ + 2, + 3 + ], + "dtype": "float32", + "bits": [ + 1079882711, + 3200253952, + 1066574966, + 1056625640, + 3225243783, + 1056431862 + ] + } + } + }, + "gen2": { + "sha256": "956fd639fd665324d3bd10dd7441aa43b4b90bfec8219dd97d4e871db0e5c5d3", + "ort": { + "S": { + "shape": [], + "dtype": "int32", + "bits": [ + 14 + ] + }, + "Q": { + "shape": [ + 2, + 3 + ], + "dtype": "float32", + "bits": [ + 1079882711, + 3200253952, + 1060623221, + 1056625640, + 3225243783, + 1055740287 + ] + } + } + }, + "div3": { + "sha256": "e1af3eabf36c787e39f232f63415183747884b19404f58059b9b72488e43b18d", + "ort": { + "S": { + "shape": [], + "dtype": "int32", + "bits": [ + 14 + ] + }, + "Q": { + "shape": [ + 2, + 3 + ], + "dtype": "float32", + "bits": [ + 1079882711, + 3200253952, + 1068827891, + 1056625640, + 3225243783, + 1052692391 + ] + } + } + }, + "neg_shape": { + "sha256": "616a64682719e02abb21cb9671cd88d19d8a076f79354a98c2587dc835fa2d95", + "ort_error": "Fail: [ONNXRuntimeError] : 1 : FAIL : Node (n_add) Op (Add) [ShapeInferenceError] Incompatible dimensions" + }, + "neg_where_nonzero": { + "sha256": "dbe60cafa40063c9b92f9ad84dbaece117373e75835c2df031143401d2d7a11a", + "ort": { + "S": { + "shape": [], + "dtype": "int32", + "bits": [ + 17 + ] + }, + "Q": { + "shape": [ + 2, + 3 + ], + "dtype": "float32", + "bits": [ + 1079882711, + 3200253952, + 1066574966, + 1056625640, + 3225243783, + 1056431862 + ] + } + } + }, + "neg_unsupported": { + "sha256": "cf116e70da8451620056b44947d69f3b6d24652039fef91deddaec879b01528c", + "ort": { + "S": { + "shape": [], + "dtype": "int32", + "bits": [ + 14 + ] + }, + "Q": { + "shape": [ + 2, + 3 + ], + "dtype": "float32", + "bits": [ + 1073741638, + 3200378321, + 1048576000, + 1057301546, + 3204448256, + 4290772992 + ] + } + } + }, + "neg_domain": { + "sha256": "2e90e4e9e24c58403cce3e0333ffb035bbc13dfbc4108976470887b7fa66738b", + "ort_error": "Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: com.example:Abs(-1) is not a registered function/op" + } + }, + "div3_sweep": { + "seed": 20261030, + "F_bits": [ + [ + 1082145540, + 1098403062, + 3236773855, + 1082947524, + 1091240558, + 3195272670 + ], + [ + 1080201846, + 1077581980, + 1075371045, + 1087989037, + 3238256091, + 3245719214 + ], + [ + 3229208364, + 3222326869, + 1095236995, + 1071083622, + 3213270037, + 1097812842 + ], + [ + 3232302537, + 1093846710, + 1070190081, + 1094936417, + 3215124028, + 3241628918 + ], + [ + 1080632337, + 1083223872, + 1083003334, + 3228842184, + 1046253854, + 3225871952 + ], + [ + 3203602644, + 1092000168, + 3183982224, + 3231506837, + 1074615167, + 3232031313 + ], + [ + 3210152053, + 3229800998, + 1077752811, + 3241191492, + 3224954582, + 3240897462 + ], + [ + 3241715199, + 3240786950, + 1067880505, + 1085536402, + 1092812339, + 3215031032 + ], + [ + 3214244755, + 3230304513, + 3237901357, + 1078296784, + 3207830138, + 3236456678 + ], + [ + 1091638997, + 3225574759, + 1089980077, + 1082534342, + 3234890736, + 3238666340 + ], + [ + 1084839833, + 3243590782, + 3213593874, + 1083447397, + 3226271089, + 3248561047 + ], + [ + 3239953664, + 3205416393, + 3216139257, + 1083063358, + 1084591626, + 1086687866 + ], + [ + 1038621193, + 1066435460, + 3210768317, + 3226937221, + 3183037605, + 1060157077 + ], + [ + 3241494042, + 3244208611, + 3241757183, + 1087520422, + 1078128303, + 3221451006 + ], + [ + 1094294444, + 3208797460, + 1067246810, + 1072304720, + 3218929643, + 3248874826 + ], + [ + 3238868998, + 3239394589, + 1049778639, + 3234565576, + 1095065799, + 3210801959 + ], + [ + 1080380566, + 1063684688, + 1090039524, + 1092297035, + 1087365587, + 1084394057 + ], + [ + 1067096621, + 3230748192, + 1082678342, + 3241213152, + 1091031129, + 1098958926 + ], + [ + 1074349695, + 1091842838, + 3213605801, + 1091290008, + 3237903505, + 3229982112 + ], + [ + 1028499863, + 3201366507, + 1075676410, + 3238577164, + 3212612164, + 1053441993 + ], + [ + 3246813618, + 3207936985, + 1074217361, + 1085673667, + 3219372029, + 3241551051 + ], + [ + 3213369428, + 1077857670, + 1086182873, + 3239895572, + 1094852851, + 1085703417 + ], + [ + 1090217637, + 3252147862, + 3229414209, + 1086323839, + 3236298702, + 1093268512 + ], + [ + 3218111630, + 3231419228, + 1075150957, + 3217023378, + 3225053030, + 3226723120 + ], + [ + 3220152929, + 3230242832, + 3212812463, + 3209110076, + 1087056972, + 1093212459 + ], + [ + 1084915087, + 3232057366, + 1093234587, + 1055527851, + 1089883549, + 1069226739 + ], + [ + 3242395904, + 1076766353, + 1052837851, + 1082257747, + 1083049070, + 1082818609 + ], + [ + 1069285192, + 3234931946, + 3227185525, + 1069743374, + 3230316291, + 3232262942 + ], + [ + 1099829901, + 3233171956, + 1084729142, + 3230952346, + 3234900111, + 3218899854 + ], + [ + 1084612571, + 1099560887, + 3242049841, + 3224703864, + 1084424204, + 3227658063 + ], + [ + 3207428305, + 3219570886, + 3232026802, + 1071264108, + 1070518086, + 3233676344 + ], + [ + 3233912964, + 3237852485, + 3225806340, + 1070210600, + 3239414568, + 3239226068 + ], + [ + 1085925708, + 1089906549, + 3232062548, + 1077358029, + 3229781570, + 3216848208 + ], + [ + 1089769587, + 1088154777, + 1071336670, + 1058723169, + 1069832038, + 1077136652 + ], + [ + 1089561109, + 1075365150, + 1084197895, + 3229905438, + 3216717410, + 1091715324 + ], + [ + 3233936570, + 3240065453, + 3239132511, + 3236366788, + 1088835152, + 3240519234 + ], + [ + 3241142576, + 3226979065, + 1076110027, + 3241254674, + 3227065756, + 1086103211 + ], + [ + 1083301178, + 1071066271, + 1059075199, + 1091534435, + 3212881171, + 3250714421 + ], + [ + 3230715952, + 1071919518, + 1090261900, + 3238106215, + 1066435702, + 1091588724 + ], + [ + 1089303686, + 1094403232, + 3242685773, + 1085380540, + 3232076885, + 1089275872 + ], + [ + 1091155270, + 1086375076, + 1094574263, + 1085745821, + 3231644438, + 1094757243 + ], + [ + 3250457295, + 3239820116, + 3225252399, + 3205802284, + 3207669809, + 1092340496 + ], + [ + 3237908895, + 1059985327, + 3223615684, + 1087246437, + 1075027008, + 1059442533 + ], + [ + 1088664205, + 3193965844, + 1090149440, + 3234721906, + 3245436712, + 1086696192 + ], + [ + 3181610765, + 3179692978, + 1096573562, + 1077020093, + 1092565768, + 1084674557 + ], + [ + 1064452977, + 1042621693, + 3221969346, + 1079835227, + 1086451947, + 3248158130 + ], + [ + 3232720413, + 3213574025, + 3237806536, + 3237998295, + 1084153162, + 3241298302 + ], + [ + 1094861134, + 1062912643, + 1087016375, + 1089684022, + 1051929275, + 3243524545 + ], + [ + 3242272532, + 1066352381, + 3238152740, + 3240560949, + 3226923739, + 3237899977 + ], + [ + 1088478070, + 3248869758, + 3215036592, + 1052767873, + 3231066904, + 3227154935 + ], + [ + 3231227768, + 1085091282, + 1085343542, + 3248765956, + 3231174852, + 3233588047 + ], + [ + 1093437547, + 3239346702, + 1071475950, + 1086337620, + 3218662235, + 3239459679 + ], + [ + 1088378512, + 1094755947, + 1093011760, + 1062800244, + 1078478801, + 1092633468 + ], + [ + 1084335307, + 1099185097, + 3234184467, + 3229683976, + 3226499211, + 3237233148 + ], + [ + 3235214980, + 1083238091, + 1091612036, + 3228632373, + 3205181666, + 3218147161 + ], + [ + 3241173572, + 1077364904, + 3229717215, + 1093711353, + 3240046429, + 3241296091 + ], + [ + 1081268605, + 1100745659, + 1093971353, + 1090577136, + 3243253133, + 3238529184 + ], + [ + 1071972187, + 3240987589, + 3234726671, + 3231453105, + 1089938256, + 1083725026 + ], + [ + 1075602172, + 1055125283, + 1083013855, + 3236557751, + 3244871022, + 3228654003 + ], + [ + 3228918386, + 1099545161, + 3225692063, + 1091155446, + 1096617626, + 1071294953 + ], + [ + 1089674222, + 1090415365, + 1084720014, + 3240938865, + 1078538098, + 3209979800 + ], + [ + 1092209915, + 1080510698, + 3215350769, + 1091428002, + 3222476428, + 3226774499 + ], + [ + 1083824023, + 1071291968, + 3232583875, + 1102943731, + 1085141951, + 1085555257 + ], + [ + 1086426624, + 3240418176, + 1075535417, + 1083943145, + 1089597918, + 1098718186 + ], + [ + 1092291465, + 1089676668, + 3240406455, + 3244566720, + 3240205053, + 1101173072 + ], + [ + 3233085741, + 1086823683, + 1083546343, + 3238983101, + 1097378112, + 1093664539 + ], + [ + 3237808824, + 1071358560, + 1094380441, + 3219296035, + 3222521563, + 3234411305 + ], + [ + 1066801767, + 1084525981, + 3237965676, + 3247765034, + 3224141770, + 1053364319 + ], + [ + 1088364679, + 1079754089, + 3238981240, + 1057944318, + 3230635689, + 3213784716 + ], + [ + 3243700287, + 1081586463, + 3254058134, + 3232588120, + 1089289430, + 3237486259 + ], + [ + 3228620345, + 3226758005, + 3217680633, + 1078818476, + 3243248452, + 3224631632 + ], + [ + 1095053496, + 3243883878, + 3238594836, + 1087780510, + 3239991951, + 3221229092 + ], + [ + 3240928485, + 3223367610, + 1083563596, + 3214998924, + 3233535413, + 1098532352 + ], + [ + 3225782629, + 1078150007, + 3240777557, + 1096135515, + 1076278399, + 3239589704 + ], + [ + 3241151369, + 3232205647, + 1067449162, + 1093326978, + 3211323617, + 1087207129 + ], + [ + 1091009335, + 3229659247, + 3179275014, + 3243043665, + 3229597165, + 3244833238 + ], + [ + 1093889073, + 3218713753, + 3210548871, + 1091638170, + 1054406913, + 1097094712 + ], + [ + 3246847421, + 1082527150, + 3230954852, + 1083083530, + 3243966118, + 1072252349 + ], + [ + 3230170206, + 1076694420, + 3227833335, + 1057779804, + 1085593379, + 1087309435 + ], + [ + 1091076446, + 3240619095, + 3234077465, + 3218075203, + 1095952034, + 3243737203 + ], + [ + 3217612901, + 1082715213, + 3249998120, + 1063883489, + 3247481784, + 1087674404 + ], + [ + 1098007316, + 3234596515, + 1077790999, + 1072176333, + 1083304802, + 3238640811 + ], + [ + 3237629051, + 1085464855, + 3236007557, + 3248446735, + 3209567921, + 1070464833 + ], + [ + 3238700538, + 1088816327, + 1084444110, + 3243592798, + 3239391527, + 3236175401 + ], + [ + 1073417224, + 3236958650, + 1072450137, + 1089785540, + 3243007873, + 3219838298 + ], + [ + 3224701098, + 1077994443, + 1096003933, + 3217563915, + 1094738160, + 3190644992 + ], + [ + 1094576925, + 1088395309, + 3220704925, + 3240853682, + 3216454378, + 3238566510 + ], + [ + 1069823791, + 3241852704, + 3233515664, + 1070595045, + 3222653289, + 3230398148 + ], + [ + 1084449360, + 3247378616, + 1087885636, + 3221586501, + 1095058097, + 1094265503 + ], + [ + 1048685016, + 3230267529, + 3240934900, + 1054645860, + 1093187070, + 1065078244 + ], + [ + 1092296691, + 3230010543, + 3238188520, + 3207053731, + 3208311523, + 1079629545 + ], + [ + 3242150526, + 1091743126, + 1091863803, + 1092180604, + 1093684083, + 3239383181 + ], + [ + 1085553272, + 3242558497, + 3231993989, + 3223278286, + 1080989905, + 3231543044 + ], + [ + 1077636687, + 3226505310, + 3238261785, + 1083477066, + 3231473702, + 1088762976 + ], + [ + 3213919698, + 3235389383, + 3229655335, + 3244570445, + 3236124959, + 1090738702 + ], + [ + 3232148943, + 1084105706, + 1068275236, + 1093948338, + 3217294205, + 1098502020 + ], + [ + 1076634057, + 3221346085, + 3237054213, + 3238783896, + 1063835058, + 3216116167 + ], + [ + 1083135707, + 3221836926, + 3222628858, + 1053722452, + 1086716669, + 1098167048 + ], + [ + 1091354357, + 3239964789, + 3211657662, + 3245073041, + 3187866917, + 1086273885 + ], + [ + 3238467533, + 1040972459, + 1051508167, + 1080252173, + 3211888311, + 3243280439 + ], + [ + 1085287183, + 3227518117, + 3240169364, + 1088969120, + 1068725320, + 1078410480 + ], + [ + 3242632219, + 1093945948, + 1061336311, + 3236528687, + 1093622673, + 3234616902 + ], + [ + 1077563261, + 3223701568, + 3230070165, + 1086411920, + 3209247207, + 3232468766 + ], + [ + 3225742816, + 3221509481, + 3235490416, + 3232761440, + 1088648416, + 3184533958 + ], + [ + 3235020965, + 3237645813, + 3226097337, + 3224942502, + 1079466477, + 3214841955 + ], + [ + 3241821829, + 1080224506, + 1081831279, + 1087929332, + 3231731171, + 1093136457 + ], + [ + 1064378410, + 1083625392, + 3238909519, + 3223590424, + 3239258237, + 3244595044 + ], + [ + 1073458954, + 3233385476, + 1088668524, + 1049874246, + 1086649147, + 1099676672 + ], + [ + 3226197619, + 3238515906, + 1085598600, + 1092954848, + 1074270408, + 1078464923 + ], + [ + 1069140898, + 3245090736, + 1091010731, + 3233257801, + 1090784739, + 3222043574 + ], + [ + 3224075825, + 1073223194, + 1086354438, + 1083262485, + 1091597677, + 1065743769 + ], + [ + 1087011208, + 1041053113, + 3236724035, + 3237394447, + 1082381784, + 1090227527 + ], + [ + 3226236446, + 3235094328, + 3243129447, + 3238688301, + 3202204278, + 1096524757 + ], + [ + 3222366600, + 1075184729, + 3229458207, + 3230210293, + 1096424653, + 1083297542 + ], + [ + 3242786214, + 3240520843, + 1088156458, + 1073818023, + 3239116029, + 3244464863 + ], + [ + 3240367105, + 1075614069, + 1101438259, + 3234557925, + 3218430574, + 3242558194 + ], + [ + 1078924541, + 3238686229, + 1065583761, + 1083578722, + 3226232349, + 3239724946 + ], + [ + 3231291597, + 3240825408, + 1073079895, + 3230294897, + 3244228760, + 1067253607 + ], + [ + 3224957232, + 1101873145, + 3239612409, + 1082034186, + 3242214037, + 1080445669 + ], + [ + 1076901108, + 1071844339, + 3223978139, + 1090974367, + 3225283449, + 1097887582 + ], + [ + 1065656853, + 1099869837, + 3246062395, + 1070907483, + 3191257675, + 1099017687 + ], + [ + 3230168301, + 1097907856, + 3238229594, + 3230069198, + 1075595452, + 3220712448 + ], + [ + 3229025793, + 1067099015, + 1081247660, + 3240894422, + 3217960081, + 3231504748 + ], + [ + 1083980617, + 1086098819, + 3247315427, + 1068586074, + 1088217070, + 3241417888 + ], + [ + 3233683892, + 3229784089, + 3220072440, + 3246445115, + 1095157210, + 1083206479 + ], + [ + 1090426902, + 3226818592, + 3243862166, + 1083455463, + 3240122477, + 3244383415 + ], + [ + 1071894483, + 1076731866, + 3227534489, + 3242075294, + 3230156487, + 1092273679 + ], + [ + 1083441334, + 1065289514, + 3234586937, + 1083497099, + 1085175295, + 3238240079 + ], + [ + 3239773548, + 3222941339, + 1083951029, + 1081174159, + 1082167173, + 3230739523 + ], + [ + 1083009250, + 1101416903, + 3242257745, + 3238243751, + 3237015038, + 3227621053 + ], + [ + 1086020781, + 3214080849, + 1089134120, + 3241348629, + 1089838049, + 1095848255 + ], + [ + 1067274764, + 978963674, + 3232378543, + 1075043150, + 3223502267, + 1079579182 + ], + [ + 1070246772, + 1082795909, + 1090975181, + 3226294024, + 1088989838, + 1084213084 + ], + [ + 1099867472, + 3225227634, + 1092273735, + 3239053565, + 1092058502, + 1094532699 + ], + [ + 3180604140, + 1085020724, + 1084997145, + 1094815785, + 1093255232, + 1087460472 + ], + [ + 3244587600, + 1088660471, + 1077744747, + 1093376628, + 1090705694, + 1086362119 + ], + [ + 3228354006, + 1089589183, + 3238661380, + 1077141230, + 3226580828, + 3230816024 + ], + [ + 3236760379, + 3242233701, + 3225989143, + 1083068375, + 1078751572, + 1090990349 + ], + [ + 1091036037, + 1078496374, + 1090524731, + 1092153075, + 1089712958, + 3213242749 + ], + [ + 3240299525, + 3218060973, + 3205163236, + 1067173361, + 1097535479, + 1095760542 + ], + [ + 1078508471, + 3231567863, + 3238216011, + 3230253291, + 3188024750, + 3220731384 + ], + [ + 1072563516, + 3227990811, + 3167527306, + 3223023341, + 3240245843, + 1098762684 + ], + [ + 1077905115, + 1086032187, + 3224738193, + 3201897479, + 3209711910, + 3230108855 + ], + [ + 3243258312, + 1090587741, + 3229523726, + 3210368726, + 3220343972, + 3242852386 + ], + [ + 3238831521, + 1066035242, + 1025325364, + 1092796112, + 1076953472, + 3232777465 + ], + [ + 1083655491, + 3243497557, + 3228783547, + 1092121522, + 3238519583, + 1087269010 + ], + [ + 3211343101, + 3223187409, + 3216530407, + 1075913333, + 1070430729, + 1080258906 + ], + [ + 1055300309, + 3234541717, + 1089976207, + 3248112622, + 3225888586, + 3225330603 + ], + [ + 1085245552, + 1082643089, + 1067606450, + 1078427096, + 3240107133, + 1088852043 + ], + [ + 3232339242, + 1073249035, + 1093192930, + 3225624711, + 3231615868, + 1095960034 + ], + [ + 1083320908, + 3238212915, + 1077698834, + 3229509055, + 1083181641, + 3236446352 + ], + [ + 1092663908, + 1089115904, + 1098172267, + 3224741002, + 3236574165, + 3237074416 + ], + [ + 3224664297, + 3227153161, + 1088793062, + 1092444441, + 1079656363, + 3233924721 + ], + [ + 1068429287, + 1082743649, + 3228516565, + 3242056873, + 1092266471, + 3230474288 + ], + [ + 3218208633, + 3234356227, + 3246449158, + 3231009652, + 3245130816, + 1079527056 + ], + [ + 1072805472, + 1087441654, + 1075854154, + 1064464035, + 1099253828, + 3228957584 + ], + [ + 3238755983, + 1090533387, + 3236375575, + 3218993151, + 1101723621, + 1075563050 + ], + [ + 3229879871, + 1096266978, + 1072489461, + 3234367144, + 1081638851, + 3207288489 + ], + [ + 1092379893, + 3173797727, + 3235062930, + 3219152234, + 3225710433, + 1092253874 + ], + [ + 3239152078, + 1067830050, + 3219489756, + 3220253234, + 3230595095, + 1090622513 + ], + [ + 3232699836, + 1077792174, + 1084528167, + 1077897024, + 3238754493, + 1084939997 + ], + [ + 3214926811, + 1082632928, + 1087661789, + 3243648919, + 3237538899, + 3239986215 + ], + [ + 1083876657, + 3239393670, + 3231006664, + 1082188555, + 1069569254, + 3231409090 + ], + [ + 1091853468, + 1075719729, + 3223840083, + 3224838218, + 1078630132, + 1090674465 + ], + [ + 3235280758, + 1093700615, + 1091091945, + 1074771353, + 1065639110, + 1080159715 + ], + [ + 1093161427, + 3216380788, + 1076733492, + 1089131588, + 3246914254, + 1093851068 + ], + [ + 1091561334, + 3244177898, + 3221830644, + 1091116799, + 3235441119, + 3234084724 + ], + [ + 3240107610, + 3240227829, + 1074650393, + 3215989353, + 1094679860, + 1082898585 + ], + [ + 3235153057, + 1079909632, + 3240992505, + 1075404185, + 1090943813, + 3201649455 + ], + [ + 1097576189, + 1092196083, + 1093794725, + 3211465573, + 1082258079, + 3236560564 + ], + [ + 3203541033, + 3230574619, + 1084715279, + 1076264639, + 3244399280, + 1090331377 + ], + [ + 1090578078, + 1087739535, + 3221130890, + 1095864995, + 3236571021, + 3229850541 + ], + [ + 1092203617, + 3215942365, + 3238158093, + 1078722637, + 3250564664, + 3239414795 + ], + [ + 3230701676, + 3234770261, + 1069816296, + 3242105220, + 1074498106, + 1099221816 + ], + [ + 3240297067, + 3239601348, + 1085961305, + 3238327827, + 1074304017, + 3200460039 + ], + [ + 1096318138, + 1086449118, + 3233561818, + 3238596605, + 1076465362, + 3231696298 + ], + [ + 1095593929, + 1093801874, + 3238218603, + 1099671073, + 3245453384, + 1068351271 + ], + [ + 1101589131, + 3220952187, + 3231669437, + 3224345203, + 3229675838, + 1082660656 + ], + [ + 3248351821, + 1089998574, + 1093561863, + 1089464475, + 3244653148, + 3220879816 + ], + [ + 3232628172, + 1085928027, + 3249442221, + 3239267519, + 3231009710, + 1089031876 + ], + [ + 3240670817, + 1101425217, + 3244767203, + 1061333919, + 3202001415, + 3208777532 + ], + [ + 1066289027, + 3206097724, + 1068605589, + 3197012659, + 1094820356, + 1090641240 + ], + [ + 1066861428, + 3210221048, + 1095888007, + 3209668355, + 3221537556, + 3221523831 + ], + [ + 1075983801, + 1050627424, + 1095035361, + 1088860252, + 3228803631, + 1075592152 + ], + [ + 3231367893, + 3214666814, + 1073561768, + 3185819914, + 1073947600, + 3216689609 + ], + [ + 3233944180, + 1094879542, + 1076123102, + 3225204483, + 1075704717, + 3239351952 + ], + [ + 3226823008, + 3232974131, + 1078830292, + 1077911381, + 3221200653, + 1052258898 + ], + [ + 3231957824, + 1090027952, + 1091427413, + 1090677010, + 1095004619, + 1091906502 + ], + [ + 1090597519, + 3235664636, + 3227490267, + 3245050262, + 1091661517, + 1081866196 + ], + [ + 1099512149, + 1090885051, + 1093816112, + 1092965700, + 3224455262, + 3222461457 + ], + [ + 3218518488, + 3238250422, + 3229935530, + 1074759205, + 1099198602, + 1082418076 + ], + [ + 1075846031, + 1078938512, + 1091501553, + 3225745357, + 3239314179, + 3239259628 + ], + [ + 1089753876, + 3222686808, + 3238929840, + 1060914852, + 1086935549, + 1072108409 + ], + [ + 3239292808, + 3240067929, + 1079194729, + 1056407399, + 3229753564, + 1094913482 + ], + [ + 1075362309, + 3241875683, + 1075615306, + 1067975244, + 1090803466, + 1094941533 + ], + [ + 3246619141, + 3215467845, + 1088006991, + 1100270013, + 3226818160, + 3238171575 + ], + [ + 3241533056, + 3239481464, + 1095664224, + 3229804895, + 3241217037, + 3231062966 + ], + [ + 1100074721, + 1089095409, + 3240853172, + 1089527666, + 3168353904, + 1092697689 + ], + [ + 1093211260, + 1091764226, + 1083265088, + 3233241616, + 1070453657, + 1068893551 + ], + [ + 3242097056, + 1095982124, + 1099768035, + 1093070352, + 1055271411, + 3229595705 + ], + [ + 3241606477, + 1100200953, + 1091888868, + 1083771776, + 1083183257, + 1089033663 + ], + [ + 3218084636, + 3225172507, + 1096286789, + 3228813273, + 3211220140, + 3225037824 + ], + [ + 1073156162, + 1088309316, + 1087319758, + 3231115948, + 3247394603, + 3166983447 + ], + [ + 1083560228, + 1074065368, + 1073948251, + 1084247274, + 3213351948, + 3241632009 + ], + [ + 3213512300, + 3232391117, + 3230749125, + 3247233959, + 1092783442, + 3226993333 + ], + [ + 3240395514, + 3206699627, + 1097546625, + 1074154174, + 3234332850, + 3237445786 + ], + [ + 1067732265, + 3243855710, + 1084830688, + 3231362716, + 3216723504, + 3220988999 + ], + [ + 3235595214, + 3225637680, + 3229480561, + 3231659837, + 1063814100, + 1093202168 + ], + [ + 1101177894, + 1090678597, + 1076256385, + 3230437699, + 1094501961, + 3232318439 + ], + [ + 1085168583, + 3225391369, + 1069276569, + 3222341482, + 1090757377, + 3219556767 + ], + [ + 3230756714, + 3240782568, + 3232630838, + 1084277546, + 1047533170, + 1086295366 + ], + [ + 3227546122, + 3236518442, + 1081933414, + 1084023826, + 3205784549, + 1098480467 + ], + [ + 3231780150, + 3238938517, + 3218208002, + 3241609964, + 1088182039, + 3243130916 + ], + [ + 1080717986, + 1096479500, + 3240327191, + 3231726613, + 3215196432, + 3238097329 + ], + [ + 3242159253, + 3234661629, + 1080424176, + 3211115057, + 3245237972, + 3233596761 + ], + [ + 1092550010, + 1083201889, + 3208112965, + 1090193693, + 3233179484, + 3227346678 + ], + [ + 3241893462, + 1053363614, + 1089954286, + 3250900539, + 3207331188, + 3232332474 + ], + [ + 3242954294, + 3230102208, + 3235962252, + 3239699006, + 3238814687, + 3230086965 + ], + [ + 1089829038, + 3224506818, + 1086785421, + 1070390038, + 3231257170, + 1087236952 + ], + [ + 3239536280, + 1044730087, + 1080884243, + 3207424725, + 1066404382, + 3235421195 + ], + [ + 1089819675, + 3249180356, + 3231640730, + 1094283469, + 3217917344, + 1078758806 + ], + [ + 1063038052, + 1083596578, + 1079561232, + 1082472723, + 1041638272, + 3240472663 + ], + [ + 1095559379, + 1077326605, + 1071263597, + 1094795428, + 1084858738, + 3231691701 + ], + [ + 1074735032, + 3225511395, + 3241527926, + 1076525680, + 1096656085, + 1083828240 + ], + [ + 1090888094, + 1086759159, + 1090711040, + 1101053268, + 3228147344, + 3223964847 + ], + [ + 3241070605, + 1063764815, + 1082290584, + 3227668827, + 1090042758, + 1078344268 + ], + [ + 1074009470, + 3238762376, + 3237694369, + 3234008622, + 3238643623, + 3250015224 + ], + [ + 1071460911, + 3238594735, + 1093969714, + 1093340861, + 3198127212, + 3237606524 + ], + [ + 3202589728, + 3222237941, + 1092523634, + 1083101174, + 1089459947, + 1090813901 + ], + [ + 1059344569, + 3232649949, + 3241074033, + 1082275827, + 3240851765, + 1067445529 + ], + [ + 3196138547, + 1091074200, + 3239242400, + 1094415355, + 1072761213, + 1078970190 + ], + [ + 1084785276, + 1088055917, + 3211109575, + 3235565230, + 1096939262, + 1089526876 + ], + [ + 1094128157, + 1078752779, + 3203933656, + 3194768781, + 1087138963, + 3235855860 + ], + [ + 3224936483, + 3246927899, + 1097262069, + 3206210949, + 1092313906, + 1071019362 + ], + [ + 1089047472, + 3237720583, + 3230331913, + 3220036523, + 1097468186, + 1096301510 + ], + [ + 3229618473, + 3217302500, + 1046159207, + 1086106001, + 1092626008, + 3234046438 + ], + [ + 3221818859, + 3215151289, + 1088704927, + 1087664418, + 3214727104, + 1093377573 + ], + [ + 1092952018, + 3222606212, + 3240551545, + 3240152996, + 3235544796, + 1086647389 + ], + [ + 1088833172, + 3187281105, + 1083522500, + 1085650759, + 3242962328, + 1085900163 + ], + [ + 3243420452, + 1097343336, + 3243098270, + 3215490754, + 1052980401, + 1076509486 + ], + [ + 1090812539, + 3237048756, + 1095014753, + 3234131353, + 1082424892, + 3246470210 + ], + [ + 3243271933, + 3248977622, + 1082872502, + 1075951343, + 3170383358, + 3237551596 + ], + [ + 1097086393, + 1071284796, + 1091413692, + 3239944677, + 1083974741, + 1083838250 + ], + [ + 1073271238, + 1093734842, + 3227244507, + 3232774352, + 1093051422, + 1055941954 + ], + [ + 1057966409, + 3233375009, + 3218149026, + 3240614577, + 3237000616, + 1094474280 + ], + [ + 1068733537, + 3214295480, + 3237420985, + 3243971320, + 1078316080, + 3236898375 + ], + [ + 3248948041, + 3213921090, + 1081701083, + 3249433870, + 3243272312, + 3239741801 + ], + [ + 3230009899, + 3238230713, + 3125788114, + 3229935610, + 3240299093, + 3222192959 + ], + [ + 3233581975, + 3245931143, + 3247114638, + 3233314114, + 1075832259, + 3214302024 + ], + [ + 3202204538, + 1093341357, + 1081921830, + 3247192305, + 1091737275, + 1005626958 + ], + [ + 1085806409, + 1084270604, + 3234572242, + 1087814717, + 3233464939, + 3215475456 + ], + [ + 3240089794, + 3242580170, + 3239040649, + 1092255052, + 1086164274, + 3238523054 + ], + [ + 1086801681, + 3240136603, + 3234149662, + 3194154187, + 3243707340, + 3225149015 + ], + [ + 3206178744, + 3247306001, + 3241429806, + 3228839822, + 1093402828, + 1095981480 + ], + [ + 1086114700, + 3174674649, + 3243416294, + 3230552426, + 1090614816, + 3231233158 + ], + [ + 1094976650, + 1098161082, + 3235058006, + 1106937211, + 1079890148, + 3211256667 + ] + ], + "Q_bits": [ + [ + 1091043029, + 3236413372, + 1072960732, + 1091846284, + 3228822993, + 1049260509 + ], + [ + 1089604410, + 3209765174, + 1066995900, + 1096892626, + 3230859161, + 1083373091 + ], + [ + 1089028828, + 3217473928, + 1083471770, + 1081433203, + 3211087247, + 1085197451 + ], + [ + 1092670177, + 3231841514, + 1063598103, + 1103584540, + 3213008072, + 1079116996 + ], + [ + 1090035815, + 3218609985, + 1072967049, + 1088661766, + 3198072896, + 1044746985 + ], + [ + 1046480449, + 3229984088, + 1043699741, + 1091873413, + 3199047408, + 1066277130 + ], + [ + 1065930110, + 3223301471, + 1068698187, + 1101831271, + 3220211818, + 1078129601 + ], + [ + 1102355112, + 3233401852, + 1061806677, + 1094438136, + 3230801565, + 1056751296 + ], + [ + 1071781071, + 3223814534, + 1074126143, + 1087694622, + 3207608420, + 1072523139 + ], + [ + 1100286232, + 3220852743, + 1078104567, + 1091432475, + 3228459147, + 1075102743 + ], + [ + 1093740897, + 3236214117, + 1056326492, + 1092346849, + 3221398009, + 1086728797 + ], + [ + 1100593070, + 3206078492, + 1056922408, + 1091962285, + 3221334560, + 1075874083 + ], + [ + 1057422500, + 3204308046, + 1055447604, + 1086751416, + 3193875437, + 1057501718 + ], + [ + 1102133900, + 3236833396, + 1079289965, + 1096423706, + 3211031308, + 1055779424 + ], + [ + 1102942431, + 3208201125, + 1061303639, + 1082397171, + 3215086253, + 1087148488 + ], + [ + 1099507992, + 3232003671, + 1051027024, + 1094935519, + 3233065923, + 1055457829 + ], + [ + 1089783516, + 3204365081, + 1078144728, + 1100944492, + 3224171091, + 1074309607 + ], + [ + 1077401221, + 3224265922, + 1072518675, + 1101852937, + 3228400449, + 1085998811 + ], + [ + 1083732000, + 3229825614, + 1056330999, + 1099937113, + 3230554319, + 1060965739 + ], + [ + 1051341681, + 3204304237, + 1067216203, + 1099216030, + 3210459266, + 1053140025 + ], + [ + 1107586516, + 3207674385, + 1066156629, + 1094575525, + 3215323668, + 1079011965 + ], + [ + 1069952171, + 3210407666, + 1075530577, + 1100534958, + 3232852127, + 1075204003 + ], + [ + 1099015047, + 3244274170, + 1059425851, + 1095226244, + 3229746387, + 1082150661 + ], + [ + 1076727725, + 3224947536, + 1066836675, + 1075619170, + 3220313669, + 1051532035 + ], + [ + 1078798162, + 3223751731, + 1055999341, + 1064175839, + 3223857090, + 1082095571 + ], + [ + 1093816228, + 3225594692, + 1082125304, + 1068063030, + 3226723980, + 1062858025 + ], + [ + 1103035975, + 3207810284, + 1052807795, + 1091155427, + 3218244981, + 1072712297 + ], + [ + 1079617615, + 3228500725, + 1053610061, + 1080080568, + 3223826525, + 1066614846 + ], + [ + 1108348671, + 3226723032, + 1074538891, + 1091318071, + 3228468606, + 1056778564 + ], + [ + 1093513399, + 3238115488, + 1079684449, + 1084509311, + 3221098552, + 1055546581 + ], + [ + 1060217161, + 3215430168, + 1066270533, + 1081615201, + 3200891155, + 1068640992 + ], + [ + 1094282331, + 3230528647, + 1043740607, + 1080552362, + 3232023744, + 1075864891 + ], + [ + 1094827784, + 3226747240, + 1066322787, + 1086753029, + 3223281656, + 1056962835 + ], + [ + 1098674178, + 3224972798, + 1064467871, + 1070679040, + 3201856793, + 1068261005 + ], + [ + 1098465593, + 3203796861, + 1074175242, + 1090019441, + 3213887196, + 1080081668 + ], + [ + 1094305959, + 3232677516, + 1075737660, + 1096738054, + 3225662719, + 1077618269 + ], + [ + 1101782342, + 3221762042, + 1067527878, + 1101894470, + 3221806563, + 1075476347 + ], + [ + 1092200435, + 3199955949, + 1057000008, + 1100181632, + 3210640283, + 1089607613 + ], + [ + 1091081282, + 3198111811, + 1078294936, + 1098582062, + 3204307983, + 1079911147 + ], + [ + 1098208036, + 3232400609, + 1080540881, + 1094282131, + 3225614473, + 1077628547 + ], + [ + 1099802322, + 3223162142, + 1083027269, + 1094647742, + 3225176036, + 1083150021 + ], + [ + 1111230486, + 3232431131, + 1038786741, + 1054715600, + 3207509192, + 1080923187 + ], + [ + 1098281059, + 3203566345, + 1051946752, + 1096149533, + 3201774998, + 1057171943 + ], + [ + 1097568201, + 3199360717, + 1078218979, + 1095091979, + 3238033375, + 1075879743 + ], + [ + 1052479147, + 3191630079, + 1084367540, + 1086413968, + 3230553481, + 1074501560 + ], + [ + 1075167289, + 3196807132, + 1055156632, + 1089236973, + 3223240571, + 1086189751 + ], + [ + 1093088550, + 3211434454, + 1074061134, + 1098370506, + 3220538954, + 1078670917 + ], + [ + 1103509241, + 3204269792, + 1076097309, + 1098588569, + 3200419693, + 1081669124 + ], + [ + 1102912576, + 3204328820, + 1074401080, + 1101200549, + 3221733624, + 1074125197 + ], + [ + 1097381957, + 3240993366, + 1056752488, + 1066573639, + 3224589810, + 1053479095 + ], + [ + 1091593928, + 3221848363, + 1074958575, + 1109539036, + 3224699448, + 1068515820 + ], + [ + 1102085329, + 3231955557, + 1064572791, + 1095240036, + 3214942400, + 1076182333 + ], + [ + 1097282340, + 3232754826, + 1081825847, + 1074320528, + 3211829949, + 1081317256 + ], + [ + 1093235834, + 3237475214, + 1069358485, + 1089575574, + 3221515398, + 1073593049 + ], + [ + 1095585441, + 3218639642, + 1079942549, + 1088451430, + 3205924932, + 1056898492 + ], + [ + 1101813346, + 3209253481, + 1060102748, + 1102359203, + 3232658413, + 1078667933 + ], + [ + 1090596195, + 3239302235, + 1082622658, + 1099223942, + 3235875623, + 1074915604 + ], + [ + 1082229665, + 3233603221, + 1070119424, + 1091819602, + 3226779304, + 1073850881 + ], + [ + 1084990589, + 3201629230, + 1072981551, + 1096929139, + 3237497239, + 1058112477 + ], + [ + 1088738155, + 3238099732, + 1041776969, + 1099802498, + 3234623119, + 1064436416 + ], + [ + 1098578764, + 3227261602, + 1074532649, + 1101578575, + 3211964151, + 1055196331 + ], + [ + 1100857344, + 3214577377, + 1056814328, + 1100075159, + 3217630961, + 1051774860 + ], + [ + 1092723953, + 3199520085, + 1067079717, + 1111462727, + 3221900379, + 1075102994 + ], + [ + 1095329111, + 3233031670, + 1067114571, + 1092843219, + 3226435042, + 1085803409 + ], + [ + 1100938920, + 3226514717, + 1077465688, + 1105207197, + 3232817693, + 1088959693 + ], + [ + 1093454282, + 3223619523, + 1073714443, + 1099622143, + 3235385805, + 1082416661 + ], + [ + 1098180936, + 3199385064, + 1082897221, + 1077930197, + 3217678318, + 1069677373 + ], + [ + 1077101821, + 3221266930, + 1074170224, + 1108538030, + 3219369158, + 1053097613 + ], + [ + 1097268499, + 3213753194, + 1075531812, + 1069862380, + 3224151522, + 1056396592 + ], + [ + 1104340617, + 3215732934, + 1092296264, + 1092956104, + 3226122821, + 1073841356 + ], + [ + 1088439372, + 3221648463, + 1056942244, + 1088217732, + 3235870930, + 1048059745 + ], + [ + 1103701642, + 3236507915, + 1075005201, + 1096683966, + 3232603705, + 1056001687 + ], + [ + 1101568192, + 3218563316, + 1073738156, + 1073343116, + 3227090505, + 1085679048 + ], + [ + 1085592714, + 3211081045, + 1077967561, + 1104783859, + 3206588190, + 1076358869 + ], + [ + 1101791137, + 3225744941, + 1061464897, + 1101974731, + 3209708481, + 1076226849 + ], + [ + 1099656327, + 3223156863, + 1041381609, + 1103683871, + 3223102136, + 1082778722 + ], + [ + 1102536966, + 3214970135, + 1055379917, + 1100285405, + 3201382906, + 1084716608 + ], + [ + 1107620323, + 3217150008, + 1063998667, + 1091982485, + 3236590348, + 1065155017 + ], + [ + 1090534540, + 3207633065, + 1056230637, + 1069689274, + 3222363161, + 1076296303 + ], + [ + 1099723466, + 3233233367, + 1069207772, + 1076690682, + 3233955385, + 1081954961 + ], + [ + 1076220201, + 3217545521, + 1088650328, + 1074875922, + 3239603715, + 1076543947 + ], + [ + 1106655939, + 3228162236, + 1068725220, + 1082332505, + 3218778715, + 1075067923 + ], + [ + 1098001065, + 3222231517, + 1071902060, + 1109219790, + 3208666549, + 1063807552 + ], + [ + 1099339460, + 3225643643, + 1074343875, + 1104233109, + 3232000594, + 1072134376 + ], + [ + 1082957244, + 3230078757, + 1065302661, + 1098690139, + 3235629725, + 1056541784 + ], + [ + 1084506532, + 3210723662, + 1083985875, + 1076170306, + 3232736965, + 1047285257 + ], + [ + 1103224973, + 3225216826, + 1056233613, + 1101493368, + 3213743097, + 1074966549 + ], + [ + 1080161792, + 3234471232, + 1068413091, + 1080940358, + 3217816440, + 1062284975 + ], + [ + 1093350013, + 3239500404, + 1076687188, + 1080611896, + 3233058191, + 1082820091 + ], + [ + 1063231241, + 3223776879, + 1078180184, + 1067590286, + 3231178451, + 1059646026 + ], + [ + 1100944148, + 3223515100, + 1074450045, + 1059280169, + 3207904617, + 1070019217 + ], + [ + 1102790543, + 3229725155, + 1080281611, + 1100828023, + 3231678084, + 1076078422 + ], + [ + 1094455022, + 3235179122, + 1066222523, + 1083075957, + 3215094194, + 1065558119 + ], + [ + 1087032571, + 3221518535, + 1074550266, + 1092376557, + 3225002817, + 1077281519 + ], + [ + 1071104213, + 3228962128, + 1059898027, + 1105210923, + 3229658854, + 1078765091 + ], + [ + 1092516390, + 3220440852, + 1062117239, + 1102596245, + 3214201943, + 1085658749 + ], + [ + 1086026571, + 3216439392, + 1073346887, + 1099422854, + 3204379426, + 1056920191 + ], + [ + 1092034736, + 3216958217, + 1054127876, + 1067092274, + 3223510484, + 1085434555 + ], + [ + 1100001486, + 3232576427, + 1055704775, + 1105713596, + 3196424194, + 1075592519 + ], + [ + 1099106349, + 3196138306, + 1052055868, + 1089654846, + 3210038815, + 1081340919 + ], + [ + 1094188687, + 3222038690, + 1077144736, + 1097873288, + 3203033027, + 1069162796 + ], + [ + 1103272341, + 3231941231, + 1058033936, + 1096900057, + 3231616364, + 1069965723 + ], + [ + 1086958916, + 3218911365, + 1061248307, + 1095314395, + 3208473479, + 1066913363 + ], + [ + 1085552745, + 3216612362, + 1071184555, + 1093129624, + 3225473466, + 1043939384 + ], + [ + 1095391277, + 3230424650, + 1047785248, + 1084749054, + 3213436778, + 1056708733 + ], + [ + 1102461769, + 3214266904, + 1071553319, + 1096832883, + 3225264004, + 1081993437 + ], + [ + 1075129161, + 3219445619, + 1075434156, + 1083389995, + 3231866667, + 1082618842 + ], + [ + 1082978243, + 3226938936, + 1077217580, + 1064587046, + 3223441662, + 1086959047 + ], + [ + 1086009270, + 3231120470, + 1075132549, + 1101602500, + 3196355265, + 1069201171 + ], + [ + 1079471754, + 3237717403, + 1079132087, + 1093626524, + 3227903069, + 1055054797 + ], + [ + 1083878117, + 3190897726, + 1075647329, + 1092161689, + 3229543087, + 1060084615 + ], + [ + 1095914137, + 3196172703, + 1072892051, + 1097766331, + 3216843463, + 1078271721 + ], + [ + 1086048239, + 3228664541, + 1081137849, + 1099327217, + 3204671138, + 1084334845 + ], + [ + 1082157913, + 3202736730, + 1059499911, + 1090574705, + 3234429547, + 1073372255 + ], + [ + 1103426369, + 3233134738, + 1076870752, + 1083197243, + 3231723761, + 1082531445 + ], + [ + 1101006645, + 3204825680, + 1089314095, + 1094927862, + 3214817556, + 1080369144 + ], + [ + 1088324073, + 3231291731, + 1059952347, + 1092478345, + 3221378065, + 1076542384 + ], + [ + 1091657854, + 3233440452, + 1065562133, + 1090659470, + 3236853590, + 1061309065 + ], + [ + 1084763850, + 3240431199, + 1076389685, + 1090979688, + 3234833666, + 1070589571 + ], + [ + 1086294573, + 3198298188, + 1050874597, + 1099621345, + 3220551889, + 1085247484 + ], + [ + 1075936837, + 3238425004, + 1083603200, + 1081255557, + 3198131371, + 1086077443 + ], + [ + 1090532631, + 3235916948, + 1074506238, + 1090347627, + 3204774072, + 1056230548 + ], + [ + 1088845823, + 3204080906, + 1071148037, + 1101534119, + 3214563346, + 1065501264 + ], + [ + 1092880736, + 3222880087, + 1085061772, + 1078910592, + 3225036010, + 1078832307 + ], + [ + 1094053042, + 3223284226, + 1056467187, + 1107139683, + 3233157689, + 1073246904 + ], + [ + 1099119729, + 3221679601, + 1082122889, + 1092354926, + 3232734778, + 1082476759 + ], + [ + 1082190516, + 3207725434, + 1055054211, + 1102715293, + 3223663795, + 1080833288 + ], + [ + 1092340778, + 3204448135, + 1069923735, + 1092396616, + 3221934601, + 1074520580 + ], + [ + 1100412892, + 3218118037, + 1074005991, + 1090548881, + 3216389450, + 1063341709 + ], + [ + 1091908100, + 3239974398, + 1079964551, + 1098857271, + 3230107148, + 1055400040 + ], + [ + 1094922938, + 3212009411, + 1077532665, + 1101988450, + 3226677962, + 1083881541 + ], + [ + 1077582004, + 3165056772, + 1066782681, + 1084429016, + 3218703736, + 1069983947 + ], + [ + 1080588875, + 3217714886, + 1079084139, + 1086106024, + 3225819434, + 1074185650 + ], + [ + 1108386246, + 3220494206, + 1080833364, + 1099692636, + 3230042836, + 1082999383 + ], + [ + 1051991608, + 3221775905, + 1074722067, + 1103463883, + 3231246988, + 1076398810 + ], + [ + 1105228080, + 3225485685, + 1068692477, + 1102024394, + 3227743438, + 1075652555 + ], + [ + 1088172344, + 3226426203, + 1075095978, + 1086535515, + 3221557375, + 1063575888 + ], + [ + 1097131892, + 3234853388, + 1046393491, + 1091967309, + 3212445299, + 1079104597 + ], + [ + 1099683040, + 3211869746, + 1078476253, + 1100800485, + 3226551431, + 1056186145 + ], + [ + 1100939044, + 3214617936, + 1053102900, + 1077479108, + 3235543599, + 1083822752 + ], + [ + 1087906897, + 3225098356, + 1074487657, + 1090617785, + 3196509175, + 1056222801 + ], + [ + 1082527501, + 3222280945, + 1035606372, + 1082819364, + 3232858649, + 1085833187 + ], + [ + 1087301817, + 3222812008, + 1046988716, + 1052235375, + 3208752935, + 1061371855 + ], + [ + 1103898560, + 3227505172, + 1059609851, + 1066168763, + 3215842988, + 1080765109 + ], + [ + 1099470499, + 3204392713, + 1038688956, + 1101443719, + 3208267177, + 1067358597 + ], + [ + 1092555211, + 3236120663, + 1058341335, + 1100768922, + 3231124168, + 1076268861 + ], + [ + 1067229940, + 3218375224, + 1056951316, + 1085303063, + 3201025831, + 1070459243 + ], + [ + 1067941321, + 3228106924, + 1078101952, + 1108885649, + 3221176472, + 1034668964 + ], + [ + 1094147016, + 3217393975, + 1061589831, + 1087825299, + 3232719370, + 1077341804 + ], + [ + 1092706927, + 3190613654, + 1082069328, + 1085434171, + 3225147055, + 1083956455 + ], + [ + 1092220192, + 3230815729, + 1068659965, + 1089330212, + 3218521874, + 1072508879 + ], + [ + 1101311477, + 3225947118, + 1085438048, + 1084546624, + 3229885141, + 1073374692 + ], + [ + 1084469556, + 3221851437, + 1077301883, + 1101091944, + 3213645893, + 1068992292 + ], + [ + 1078751918, + 3217605226, + 1057867021, + 1102696868, + 3230252236, + 1062522457 + ], + [ + 1076826349, + 3227919673, + 1083901239, + 1091375470, + 3237757563, + 1069947433 + ], + [ + 1082649326, + 3224248441, + 1067344119, + 1075172943, + 3237612959, + 1058645272 + ], + [ + 1099394929, + 3227395350, + 1072411113, + 1077623027, + 3240281497, + 1067134501 + ], + [ + 1089968201, + 3234271370, + 1065331984, + 1094736919, + 3215788807, + 1054156869 + ], + [ + 1101027376, + 3189117113, + 1070589257, + 1077784387, + 3220992744, + 1080806640 + ], + [ + 1099791189, + 3203700268, + 1056641233, + 1078899671, + 3224110234, + 1078608268 + ], + [ + 1093067949, + 3210255723, + 1074401408, + 1087293702, + 3231360362, + 1074683022 + ], + [ + 1073194194, + 3217372611, + 1076535389, + 1104289240, + 3230370845, + 1076896624 + ], + [ + 1092776651, + 3232002747, + 1064155765, + 1091086118, + 3202175352, + 1065358923 + ], + [ + 1100500779, + 3205115912, + 1051305092, + 1084644293, + 3212171960, + 1078678393 + ], + [ + 1095651269, + 3231694699, + 1079241609, + 1084155880, + 3204438510, + 1070389979 + ], + [ + 1101809136, + 3213702714, + 1067973851, + 1098035846, + 3239035357, + 1082541905 + ], + [ + 1100208541, + 3236802613, + 1055338157, + 1099763835, + 3229014298, + 1069218003 + ], + [ + 1100747067, + 3232840562, + 1066473038, + 1074561367, + 3232678423, + 1072822624 + ], + [ + 1095523471, + 3213923549, + 1078258004, + 1084791728, + 3228224221, + 1051790635 + ], + [ + 1106224753, + 3230181371, + 1082504077, + 1067362120, + 3216581984, + 1072666553 + ], + [ + 1046976464, + 3224089407, + 1074529411, + 1085655768, + 3237024489, + 1078341855 + ], + [ + 1099224884, + 3224551183, + 1056049125, + 1104513292, + 3229883557, + 1060539671 + ], + [ + 1100851044, + 3213461492, + 1074408407, + 1088121640, + 3242689837, + 1076121369 + ], + [ + 1091066982, + 3228337582, + 1063312036, + 1102745226, + 3198190444, + 1086350576 + ], + [ + 1100936585, + 3232211388, + 1075379715, + 1098966577, + 3196643957, + 1051361888 + ], + [ + 1104966512, + 3223237684, + 1068478608, + 1099235480, + 3207062257, + 1065784933 + ], + [ + 1104242177, + 3231796459, + 1074491203, + 1108189827, + 3238041728, + 1062176831 + ], + [ + 1110108044, + 3216166509, + 1065745259, + 1084148895, + 3223173793, + 1072494251 + ], + [ + 1109124868, + 3226840298, + 1082347708, + 1098368909, + 3237278907, + 1056160569 + ], + [ + 1092996202, + 3222705547, + 1087907203, + 1099906676, + 3224531708, + 1077463493 + ], + [ + 1101310449, + 3239982722, + 1082734402, + 1073389762, + 3204600194, + 1054768468 + ], + [ + 1076580604, + 3206518913, + 1062375635, + 1056855987, + 3232819499, + 1078633547 + ], + [ + 1077162421, + 3209056984, + 1083908184, + 1065393060, + 3216642055, + 1055700872 + ], + [ + 1085373818, + 3199832705, + 1083336558, + 1097764360, + 3222696841, + 1067155485 + ], + [ + 1091734265, + 3212668618, + 1065740503, + 1054141646, + 3189921283, + 1056958425 + ], + [ + 1094313576, + 3232878926, + 1067537255, + 1085012193, + 3205074944, + 1076035991 + ], + [ + 1086636829, + 3226522931, + 1069458383, + 1087308102, + 3216298376, + 1052484089 + ], + [ + 1092325023, + 3226870003, + 1079693811, + 1099323861, + 3233004503, + 1080339100 + ], + [ + 1099244334, + 3229239659, + 1054875797, + 1105690814, + 3229642921, + 1071577537 + ], + [ + 1108030887, + 3228105602, + 1082518437, + 1101613355, + 3219694546, + 1054416655 + ], + [ + 1077141201, + 3230853459, + 1060815437, + 1084143671, + 3237502280, + 1072158969 + ], + [ + 1085235483, + 3212850550, + 1079793709, + 1085555296, + 3231922878, + 1075910516 + ], + [ + 1098658458, + 3217851566, + 1075461829, + 1072956889, + 3223733464, + 1065047403 + ], + [ + 1099931975, + 3232680002, + 1069714405, + 1068531967, + 3223253090, + 1083254818 + ], + [ + 1084749663, + 3234494283, + 1067172175, + 1078292152, + 3227940883, + 1083273632 + ], + [ + 1107392016, + 3213199104, + 1076769456, + 1108788824, + 3221679379, + 1074426857 + ], + [ + 1102172924, + 3232090953, + 1083758192, + 1089817933, + 3233833476, + 1064326076 + ], + [ + 1108593514, + 3225926362, + 1078069753, + 1098432133, + 3186451118, + 1081403616 + ], + [ + 1101858982, + 3229746414, + 1073327588, + 1093610322, + 3200990837, + 1062599811 + ], + [ + 1102737060, + 3233985576, + 1087081244, + 1101718036, + 3201677679, + 1059730171 + ], + [ + 1102246363, + 3238756685, + 1080315359, + 1092671642, + 3218525247, + 1077464703 + ], + [ + 1076700275, + 3220437220, + 1084175413, + 1088632784, + 3209647715, + 1043312092 + ], + [ + 1082825858, + 3225129599, + 1076303310, + 1091481935, + 3239516413, + 1035358543 + ], + [ + 1092459827, + 3192713773, + 1065959056, + 1093147702, + 3211180987, + 1079121165 + ], + [ + 1070252127, + 3225932807, + 1063371152, + 1108006903, + 3230772495, + 1052773384 + ], + [ + 1101035063, + 3206902014, + 1085019219, + 1083535384, + 3227896070, + 1073813560 + ], + [ + 1078045941, + 3236479681, + 1074608317, + 1091729080, + 3213890531, + 1056113067 + ], + [ + 1095965956, + 3220917680, + 1059537466, + 1092026631, + 3204377509, + 1082081741 + ], + [ + 1109696778, + 3227688709, + 1067632789, + 1090802537, + 3232499768, + 1066695476 + ], + [ + 1094069973, + 3220663385, + 1062896537, + 1082132601, + 3227847815, + 1056623181 + ], + [ + 1091122114, + 3233397454, + 1067147471, + 1093178008, + 3198465317, + 1075607136 + ], + [ + 1087362203, + 3229857075, + 1071624153, + 1092923996, + 3206317416, + 1085644325 + ], + [ + 1092147110, + 3231545350, + 1056891069, + 1102249851, + 3225000463, + 1081139825 + ], + [ + 1090121641, + 3234484566, + 1077358416, + 1092093500, + 3213048369, + 1074325224 + ], + [ + 1102799272, + 3228227953, + 1070574579, + 1066983162, + 3237864933, + 1068528181 + ], + [ + 1101197545, + 3218564125, + 1054507244, + 1099003069, + 3226730646, + 1054287728 + ], + [ + 1102533419, + 3200999848, + 1078087141, + 1111673754, + 3207298617, + 1066715850 + ], + [ + 1103594482, + 3223608501, + 1071839307, + 1100338323, + 3231420875, + 1061301999 + ], + [ + 1098733658, + 3219748011, + 1075940391, + 1080733479, + 3224783033, + 1076247097 + ], + [ + 1100175538, + 3197571771, + 1070895204, + 1060208334, + 3204316014, + 1071088301 + ], + [ + 1098724291, + 3241304285, + 1065702821, + 1102931453, + 3214540210, + 1069408101 + ], + [ + 1074442598, + 3219385772, + 1069971375, + 1091370757, + 3196416818, + 1077555269 + ], + [ + 1104207621, + 3209162621, + 1064412763, + 1103443522, + 3221609438, + 1065778146 + ], + [ + 1084119375, + 3220787332, + 1078980771, + 1085917796, + 3234661695, + 1073921738 + ], + [ + 1099535036, + 3223553783, + 1078727759, + 1109572143, + 3222361102, + 1050917346 + ], + [ + 1101710351, + 3204372895, + 1071982549, + 1087485268, + 3226884973, + 1069116109 + ], + [ + 1083389840, + 3231368287, + 1073984200, + 1094378077, + 3231248893, + 1088673189 + ], + [ + 1081813615, + 3231199737, + 1082621558, + 1101988617, + 3201910364, + 1073923921 + ], + [ + 1050718641, + 3217380471, + 1081169541, + 1092000154, + 3226295422, + 1078866565 + ], + [ + 1071327594, + 3226194873, + 1078368123, + 1091173537, + 3233466906, + 1061462008 + ], + [ + 1056963866, + 3228487365, + 1075887095, + 1103063368, + 3195025532, + 1069556725 + ], + [ + 1093686285, + 3224872461, + 1055549500, + 1095935950, + 3234945715, + 1077798275 + ], + [ + 1102776106, + 3212448011, + 1052536823, + 1056914874, + 3223940542, + 1071691865 + ], + [ + 1084743007, + 3239049023, + 1084828682, + 1057036172, + 3230299988, + 1064228271 + ], + [ + 1097951684, + 3230462276, + 1062077472, + 1078680336, + 3235476125, + 1084185276 + ], + [ + 1089444278, + 3214206458, + 1049463042, + 1095008229, + 3230614098, + 1069164035 + ], + [ + 1081080851, + 3213023249, + 1077242224, + 1096567797, + 3212736134, + 1082223926 + ], + [ + 1101599669, + 3217767092, + 1077661973, + 1100792468, + 3229118837, + 1075846566 + ], + [ + 1097737264, + 3196210901, + 1073681669, + 1094552596, + 3235584059, + 1075338069 + ], + [ + 1104060731, + 3235350934, + 1081095913, + 1074049259, + 3200851169, + 1067813893 + ], + [ + 1099459449, + 3230124123, + 1083322738, + 1094500920, + 3216934447, + 1083929457 + ], + [ + 1103912184, + 3241101344, + 1072786649, + 1085341228, + 3187538501, + 1073886217 + ], + [ + 1105734887, + 3199534446, + 1079675321, + 1100584080, + 3220169913, + 1073928608 + ], + [ + 1082883776, + 3231729096, + 1053860417, + 1093142550, + 3231042042, + 1054462905 + ], + [ + 1069885607, + 3226928354, + 1056898271, + 1101254193, + 3230099886, + 1082960187 + ], + [ + 1079059790, + 3212251523, + 1073796525, + 1104611698, + 3211460357, + 1073132313 + ], + [ + 1109721135, + 3211828680, + 1071462984, + 1110206998, + 3235894852, + 1076565248 + ], + [ + 1090228791, + 3230833633, + 1015357520, + 1090079910, + 3232912114, + 1054839605 + ], + [ + 1093951026, + 3238281056, + 1084792879, + 1093682895, + 3205420366, + 1056565233 + ], + [ + 1051608263, + 3231333579, + 1071616121, + 1107965245, + 3229719259, + 1028319307 + ], + [ + 1094708383, + 3220781572, + 1069903140, + 1096718194, + 3227019269, + 1056835885 + ], + [ + 1100729245, + 3235200856, + 1075612675, + 1100902495, + 3222946945, + 1074907237 + ], + [ + 1095704455, + 3232748962, + 1069309483, + 1056856051, + 3236330957, + 1041536537 + ], + [ + 1056925838, + 3239427686, + 1078848388, + 1088659399, + 3231395378, + 1083970828 + ], + [ + 1095016935, + 3189499995, + 1081523592, + 1090917471, + 3227559870, + 1064838484 + ], + [ + 1103624781, + 3236170807, + 1070582388, + 1115456372, + 3213902237, + 1055592176 + ] + ] + }, + "inputs": { + "X": { + "shape": [ + 7 + ], + "dtype": "int32", + "bits": [ + 3, + -1, + 4, + -1, + 5, + -9, + 2 + ] + }, + "F": { + "shape": [ + 2, + 3 + ], + "dtype": "float32", + "bits": [ + 1069547520, + 3196059648, + 1077936128, + 1036831949, + 1088421888, + 3223322624 + ] + } + } +} \ No newline at end of file diff --git a/crates/tract-harvest-probe/src/dr3.rs b/crates/tract-harvest-probe/src/dr3.rs new file mode 100644 index 000000000..bab11f55d --- /dev/null +++ b/crates/tract-harvest-probe/src/dr3.rs @@ -0,0 +1,298 @@ +//! An ONNX graph as a DR#3 graph (user decision 2026-10-30): every node is an +//! **Identity** position, every tensor value (graph input, initializer, node +//! output) a **Property** position, and the dataflow is two relations over one +//! 64 × 64 tile: +//! +//! - `on[node]`: the Property positions the node reads (its ON); +//! - `do_[node]`: the Property positions the node writes (its DO). +//! +//! ON and DO share the Property domain, which is what makes chaining a +//! composition: `dep = do_ ∘ read_by` (`read_by` = the transpose of `on`) is +//! Identity → Identity, "node i writes a value node j reads". The schedule is +//! read off `dep`; no second dependency structure exists. +//! +//! The relations carry *which* positions; the **ordered** arguments (ONNX +//! operands are positional, `Sub(a, b) ≠ Sub(b, a)`) are kept beside them, +//! because a set loses order (D-DT64-6: a registration must reference ordered +//! behaviour, not a DO set). +//! +//! Facts (dtype, shape) come from Tract, used only as a cold typing frontend; +//! names, attributes and constant values come from the source protobuf. + +use std::collections::HashMap; + +use dr_math_binding_probe::relcomp::{compose, Rows, Work}; +use tract_onnx::prelude::*; + +use crate::registry::{Dim, Fact, Refusal}; +use crate::source::{Attr, SourceInventory}; +use crate::value::{DType, Value}; + +/// Positions per axis in one DR64 tile. +pub const TILE: usize = 64; + +/// The graph on its two axes. +pub struct OnnxGraph { + /// Identity axis: node index in source order. + pub nodes: usize, + /// Property axis: value names, position = index. + pub props: Vec, + pub on: Rows, + pub do_: Rows, + /// Ordered arguments per node (`None`: an omitted optional input). + pub args: Vec>>, + /// Ordered outputs per node. + pub outs: Vec>, + /// Facts per Property position. + pub facts: Vec, + /// Values known before execution: initializers and `Constant` outputs. + pub consts: Vec>, + /// Graph inputs (positions bound by the caller) and outputs. + pub inputs: Vec, + pub outputs: Vec, +} + +fn transpose(r: &Rows) -> Rows { + let mut t = [0u64; 64]; + for (i, &row) in r.iter().enumerate() { + let mut bits = row; + while bits != 0 { + let j = bits.trailing_zeros() as usize; + t[j] |= 1 << i; + bits &= bits - 1; + } + } + t +} + +impl OnnxGraph { + /// Lay the source graph onto the tile. `facts` is keyed by value name. + pub fn build(src: &SourceInventory, facts: &HashMap) -> Result { + if src.nodes.len() > TILE { + return Err(Refusal::TileOverflow { + axis: "Identity", + count: src.nodes.len(), + }); + } + let mut props: Vec = Vec::new(); + let mut index: HashMap = HashMap::new(); + let mut intern = |name: &str, props: &mut Vec| -> usize { + *index.entry(name.to_string()).or_insert_with(|| { + props.push(name.to_string()); + props.len() - 1 + }) + }; + for n in &src.inputs { + intern(n, &mut props); + } + for i in &src.initializers { + intern(&i.name, &mut props); + } + let mut args = Vec::with_capacity(src.nodes.len()); + let mut outs = Vec::with_capacity(src.nodes.len()); + for node in &src.nodes { + args.push( + node.inputs + .iter() + .map(|s| (!s.is_empty()).then(|| intern(s, &mut props))) + .collect::>(), + ); + outs.push( + node.outputs + .iter() + .map(|s| intern(s, &mut props)) + .collect::>(), + ); + } + if props.len() > TILE { + return Err(Refusal::TileOverflow { + axis: "Property", + count: props.len(), + }); + } + + let (mut on, mut do_) = ([0u64; 64], [0u64; 64]); + for (n, (a, o)) in args.iter().zip(&outs).enumerate() { + for &p in a.iter().flatten() { + on[n] |= 1 << p; + } + for &p in o { + do_[n] |= 1 << p; + } + } + + let mut consts: Vec> = vec![None; props.len()]; + let mut fact_of: Vec = vec![Fact::default(); props.len()]; + for (p, name) in props.iter().enumerate() { + if let Some(f) = facts.get(name) { + fact_of[p] = f.clone(); + } + } + for i in &src.initializers { + let p = index_of(&props, &i.name); + consts[p] = i.value.clone(); + if fact_of[p].dtype.is_none() { + fact_of[p] = Fact { + dtype: Some(DType::from_onnx(i.data_type)), + shape: Some(i.dims.iter().map(|&d| Dim::Known(d as usize)).collect()), + }; + } + } + for (node, o) in src.nodes.iter().zip(&outs) { + if node.op_type == "Constant" && node.default_domain() { + if let (Some(Attr::Tensor(Some(v))), Some(&p)) = (node.attr("value"), o.first()) { + consts[p] = Some(v.clone()); + } + } + } + let inputs = src.inputs.iter().map(|n| index_of(&props, n)).collect(); + let outputs = src + .outputs + .iter() + .filter_map(|n| props.iter().position(|p| p == n)) + .collect(); + Ok(Self { + nodes: src.nodes.len(), + props, + on, + do_, + args, + outs, + facts: fact_of, + consts, + inputs, + outputs, + }) + } + + /// `dep[i]` has bit `j` iff node `i` writes a value node `j` reads. + pub fn dep(&self) -> (Rows, Work) { + let mut w = Work::default(); + let d = compose(&self.do_, &transpose(&self.on), &mut w); + (d, w) + } + + /// The same relation computed directly from the source edges, for the + /// check that the composition is the dataflow and nothing else. + pub fn dep_from_edges(&self) -> Rows { + let mut d = [0u64; 64]; + for (i, o) in self.outs.iter().enumerate() { + for (j, a) in self.args.iter().enumerate() { + if a.iter().flatten().any(|p| o.contains(p)) { + d[i] |= 1 << j; + } + } + } + d + } + + /// A topological order of the Identity axis from `dep` (Kahn), stable in + /// source order. `Err(Cycle)` if `dep` is not acyclic. + pub fn schedule(&self) -> Result, Refusal> { + let (dep, _) = self.dep(); + let mut indeg = vec![0usize; self.nodes]; + for row in dep.iter().take(self.nodes) { + let mut bits = *row; + while bits != 0 { + indeg[bits.trailing_zeros() as usize] += 1; + bits &= bits - 1; + } + } + let mut done = 0u64; + let mut order = Vec::with_capacity(self.nodes); + while order.len() < self.nodes { + let next = (0..self.nodes) + .find(|&n| done & (1 << n) == 0 && indeg[n] == 0) + .ok_or(Refusal::Cycle)?; + done |= 1 << next; + order.push(next); + let mut bits = dep[next]; + while bits != 0 { + indeg[bits.trailing_zeros() as usize] -= 1; + bits &= bits - 1; + } + } + Ok(order) + } +} + +fn index_of(props: &[String], name: &str) -> usize { + props.iter().position(|p| p == name).expect("interned") +} + +fn dtype_of(dt: DatumType) -> DType { + match dt { + DatumType::F32 => DType::F32, + DatumType::F64 => DType::F64, + DatumType::F16 => DType::F16, + DatumType::I32 => DType::I32, + DatumType::I64 => DType::I64, + DatumType::Bool => DType::Bool, + _ => DType::Other(-1), + } +} + +/// Tract as the cold typing frontend: import and type the graph (no +/// declutter, so every source value keeps its label) and read one [`Fact`] +/// per labelled outlet. `pins` fixes input shapes Tract cannot resolve alone. +pub fn tract_facts( + bytes: &[u8], + pins: &[(usize, DatumType, Vec)], +) -> Result, Refusal> { + let onnx = tract_onnx::onnx(); + let proto = onnx.proto_model_for_read(&mut &*bytes).map_err(fe)?; + facts_of(onnx.model_for_proto_model(&proto).map_err(fe)?, pins) +} + +/// [`tract_facts`] for a model with external data next to it: the weights +/// stay lazy (named, never read). +pub fn tract_facts_path( + path: &std::path::Path, + pins: &[(usize, DatumType, Vec)], +) -> Result, Refusal> { + let onnx = tract_onnx::onnx().with_lazy_external_data(true); + facts_of(onnx.model_for_path(path).map_err(fe)?, pins) +} + +fn fe(e: TractError) -> Refusal { + Refusal::Frontend(format!("{e:#}")) +} + +fn facts_of( + mut m: InferenceModel, + pins: &[(usize, DatumType, Vec)], +) -> Result, Refusal> { + for (i, dt, shape) in pins { + m.set_input_fact(*i, InferenceFact::dt_shape(*dt, shape.clone())) + .map_err(fe)?; + } + let m = m.into_typed().map_err(fe)?; + let mut out = HashMap::new(); + for node in m.nodes() { + for slot in 0..node.outputs.len() { + let o = OutletId::new(node.id, slot); + let label = m + .outlet_label(o) + .map(str::to_string) + .unwrap_or_else(|| node.name.clone()); + let f = m.outlet_fact(o).map_err(fe)?; + let shape = f + .shape + .iter() + .map(|d| { + d.as_i64() + .map(|v| Dim::Known(v as usize)) + .unwrap_or_else(|| Dim::Symbol(d.to_string())) + }) + .collect(); + out.insert( + label, + Fact { + dtype: Some(dtype_of(f.datum_type)), + shape: Some(shape), + }, + ); + } + } + Ok(out) +} diff --git a/crates/tract-harvest-probe/src/gather.rs b/crates/tract-harvest-probe/src/gather.rs new file mode 100644 index 000000000..c26106c35 --- /dev/null +++ b/crates/tract-harvest-probe/src/gather.rs @@ -0,0 +1,66 @@ +//! `Gather(table, ids)` on axis 0 of an external-data f32 initializer, +//! resolved to byte ranges from the source placement. +//! +//! ONNX `Gather` with numeric indices is a row copy, exact by contract. Here it +//! never loads the table: each index becomes one `(offset, length)` range into +//! the initializer's external file, computed from the source `dims` and dtype. +//! Indices follow the ONNX rule: `-dim <= i < dim`, negative counting from the +//! end; anything else is refused before a byte is read. + +use crate::registry::Refusal; +use crate::source::{Placement, SourceInventory}; + +/// One gathered row: the source index, the normalized row, and its bytes. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct RowRange { + pub index: i64, + pub row: u64, + pub offset: u64, + pub length: u64, +} + +pub fn row_ranges( + src: &SourceInventory, + table: &str, + ids: &[i64], +) -> Result, Refusal> { + let refuse = |what: String| Refusal::Runtime { + node: table.into(), + what, + }; + let init = src + .initializer(table) + .ok_or_else(|| refuse("not an initializer".into()))?; + if init.data_type != 1 || init.dims.len() != 2 { + return Err(refuse(format!( + "dtype {} rank {}", + init.data_type, + init.dims.len() + ))); + } + let (rows, cols) = (init.dims[0], init.dims[1]); + let Placement::External { offset, length, .. } = &init.placement else { + return Err(refuse("inline, expected external data".into())); + }; + let row_bytes = cols as u64 * 4; + if *length != rows as u64 * row_bytes { + return Err(refuse(format!("length {length} != rows * cols * 4"))); + } + ids.iter() + .map(|&i| { + let row = if (0..rows).contains(&i) { + i + } else if (-rows..0).contains(&i) { + i + rows + } else { + return Err(refuse(format!("Gather index {i} outside [-dim, dim)"))); + }; + Ok(RowRange { + index: i, + row: row as u64, + offset: offset + row as u64 * row_bytes, + length: row_bytes, + }) + }) + .collect() +} diff --git a/crates/tract-harvest-probe/src/lib.rs b/crates/tract-harvest-probe/src/lib.rs new file mode 100644 index 000000000..d1b77033b --- /dev/null +++ b/crates/tract-harvest-probe/src/lib.rs @@ -0,0 +1,27 @@ +//! D-TRACT-HARVEST-0 lab probe: an ONNX graph as a DR#3 graph, resolved +//! through the DT64 ON × WHEN × DO registry, with ONNX Runtime as the +//! independent oracle. Plan: `.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md`. +//! +//! ```text +//! .onnx bytes ──► source::SourceInventory (raw protobuf: names, domain, opset, attributes, constants) +//! │ │ +//! └──► Tract (cold) ──►│ dr3::tract_facts (dtype, shape per value) +//! ▼ +//! dr3::OnnxGraph nodes = Identity, values = Property, +//! on / do_ = 64x64 relations, dep = do_ ∘ read_by +//! ▼ +//! plan::resolve op signature -> registry::Registration (classid), +//! WHEN (K3) over the facts, schedule from dep, Frame sealed +//! ▼ +//! Plan::run each node routed to its DO (lance-graph-onnx / ndarray) +//! ``` +//! +//! Execution never calls Tract. No native crate depends on this crate. + +pub mod dr3; +pub mod gather; +pub mod plan; +pub mod provenance; +pub mod registry; +pub mod source; +pub mod value; diff --git a/crates/tract-harvest-probe/src/plan.rs b/crates/tract-harvest-probe/src/plan.rs new file mode 100644 index 000000000..9bfe2839d --- /dev/null +++ b/crates/tract-harvest-probe/src/plan.rs @@ -0,0 +1,232 @@ +//! Resolve once, then route: every Identity position of the DR#3 graph is +//! bound to the registration of its op signature, its WHEN is evaluated over +//! the cold facts, and the result is a schedule of `(node, classid, DO)` read +//! off the `dep` relation. Execution routes each node to its DO and nothing +//! else; it never consults Tract. +//! +//! The plan is sealed in a [`Frame`] whose revisions are (source digest, +//! registry version, coordinate assignment). A plan whose frame is not the +//! store's current seal refuses to run (stale generation, D-DT64-6 D8). + +use std::collections::HashMap; + +use dr_math_binding_probe::{Frame, FrameRevisions, FrameStore}; +use lance_graph_contract::class_view::ClassId; +use sha2::{Digest, Sha256}; + +use crate::dr3::OnnxGraph; +use crate::registry::{ + render_signature, signature_dtype, NodeView, Numeric, Refusal, Registry, K3, +}; +use crate::source::SourceInventory; +use crate::value::Value; + +/// One resolved Identity position. +#[derive(Debug, Clone)] +pub struct Bound { + pub node: usize, + pub name: String, + pub classid: ClassId, + pub method: &'static str, + pub numeric: Numeric, + reg: usize, +} + +pub struct Plan<'r> { + pub graph: OnnxGraph, + /// In schedule order. + pub steps: Vec, + registry: &'r Registry, + nodes: Vec, + frame: Frame, +} + +fn digest64(parts: &[&[u8]]) -> u64 { + let mut h = Sha256::new(); + for p in parts { + h.update((p.len() as u64).to_le_bytes()); + h.update(p); + } + u64::from_le_bytes(h.finalize()[..8].try_into().expect("8 bytes")) +} + +/// The revisions a plan of this source under this registry is sealed at. +pub fn revisions(src: &SourceInventory, registry: &Registry, graph: &OnnxGraph) -> FrameRevisions { + let coords: Vec<&[u8]> = graph.props.iter().map(|p| p.as_bytes()).collect(); + FrameRevisions { + source: digest64(&[src.model_sha256.as_bytes()]), + relation_map: registry.version, + coordinates: digest64(&coords), + } +} + +/// Bind every node of `graph` against `registry`, in schedule order. +pub fn resolve<'r>( + src: &SourceInventory, + graph: OnnxGraph, + registry: &'r Registry, + store: &FrameStore, +) -> Result, Refusal> { + let opset = src.default_opset().unwrap_or(0); + let order = graph.schedule()?; + let mut steps = Vec::with_capacity(order.len()); + for &n in &order { + let node = &src.nodes[n]; + let args: Vec<_> = graph.args[n] + .iter() + .map(|p| p.map(|p| &graph.facts[p])) + .collect(); + let reg = registry + .lookup(node, &args) + .ok_or_else(|| Refusal::Unregistered { + node: node.name.clone(), + signature: render_signature(node, signature_dtype(node, &args, Some(0))), + })?; + if node.default_domain() && !reg.opsets.contains(&opset) { + return Err(Refusal::Opset { + node: node.name.clone(), + opset, + }); + } + let consts = graph.args[n] + .iter() + .map(|p| p.and_then(|p| graph.consts[p].as_ref())) + .collect(); + let truth = (reg.when)(&NodeView { node, args, consts }); + if truth != K3::True { + return Err(Refusal::When { + node: node.name.clone(), + classid: reg.classid, + truth, + guard: reg.guard, + }); + } + let reg_ix = registry + .regs + .iter() + .position(|r| std::ptr::eq(r, reg)) + .expect("from the table"); + steps.push(Bound { + node: n, + name: node.name.clone(), + classid: reg.classid, + method: reg.method, + numeric: reg.numeric, + reg: reg_ix, + }); + } + if store.revisions() != revisions(src, registry, &graph) { + return Err(Refusal::Frame( + dr_math_binding_probe::Refusal::SourceChanged, + )); + } + Ok(Plan { + graph, + steps, + registry, + nodes: src.nodes.clone(), + frame: store.seal(), + }) +} + +impl Plan<'_> { + /// Run the plan: bind the caller's inputs by name, route each step to its + /// DO, return every value by name. Refuses first if the frame is stale. + pub fn run( + &self, + store: &FrameStore, + inputs: &[(&str, Value)], + ) -> Result, Refusal> { + store.check(&self.frame).map_err(Refusal::Frame)?; + let g = &self.graph; + let mut vals: Vec> = g.consts.clone(); + for (name, v) in inputs { + let p = g + .props + .iter() + .position(|p| p == name) + .ok_or_else(|| Refusal::Runtime { + node: (*name).into(), + what: "no such graph input".into(), + })?; + vals[p] = Some(v.clone()); + } + for s in &self.steps { + let node = &self.nodes[s.node]; + let args: Vec<&Value> = g.args[s.node] + .iter() + .flatten() + .map(|&p| { + vals[p].as_ref().ok_or_else(|| Refusal::Runtime { + node: node.name.clone(), + what: format!("value {} unbound", g.props[p]), + }) + }) + .collect::>()?; + let out = (self.registry.regs[s.reg].do_)(&args, node)?; + for (&p, v) in g.outs[s.node].iter().zip(out) { + vals[p] = Some(v); + } + } + Ok(g.props + .iter() + .cloned() + .zip(vals) + .filter_map(|(n, v)| v.map(|v| (n, v))) + .collect()) + } + + /// `(classid, method)` per scheduled node, for receipts. + pub fn census(&self) -> Vec<(ClassId, &'static str)> { + self.steps.iter().map(|s| (s.classid, s.method)).collect() + } +} + +/// Registration coverage of a graph too large for one tile: per op type, how +/// many nodes have a registered signature with WHEN TRUE, a registered +/// signature with WHEN not TRUE, or no registration. Facts and constants are +/// looked up by value name, as [`OnnxGraph::build`] does. +pub fn coverage( + src: &SourceInventory, + facts: &HashMap, + registry: &Registry, +) -> std::collections::BTreeMap { + let mut consts: HashMap<&str, &Value> = src + .initializers + .iter() + .filter_map(|i| i.value.as_ref().map(|v| (i.name.as_str(), v))) + .collect(); + for n in src + .nodes + .iter() + .filter(|n| n.op_type == "Constant" && n.default_domain()) + { + if let (Some(crate::source::Attr::Tensor(Some(v))), Some(o)) = + (n.attr("value"), n.outputs.first()) + { + consts.insert(o.as_str(), v); + } + } + let mut out = std::collections::BTreeMap::new(); + for node in &src.nodes { + let args: Vec<_> = node.inputs.iter().map(|i| facts.get(i)).collect(); + let k = match registry.lookup(node, &args) { + None => 2, + Some(reg) => { + let consts = node + .inputs + .iter() + .map(|i| consts.get(i.as_str()).copied()) + .collect(); + usize::from((reg.when)(&NodeView { node, args, consts }) != K3::True) + } + }; + let key = if node.default_domain() { + node.op_type.clone() + } else { + format!("{}:{}", node.domain, node.op_type) + }; + out.entry(key).or_insert([0; 3])[k] += 1; + } + out +} diff --git a/crates/tract-harvest-probe/src/provenance.rs b/crates/tract-harvest-probe/src/provenance.rs new file mode 100644 index 000000000..5d6b1e79f --- /dev/null +++ b/crates/tract-harvest-probe/src/provenance.rs @@ -0,0 +1,89 @@ +//! What each Tract stage keeps of the source graph, counted against the raw +//! protobuf ([`SourceInventory`]), never assumed. + +use std::collections::{BTreeMap, HashSet}; + +use crate::source::SourceInventory; + +/// One stage's correspondence with the source graph. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct StageProvenance { + pub stage: &'static str, + pub nodes: usize, + /// Source nodes whose (non-empty) name is a node name in this stage. + pub source_names_kept: usize, + /// Source nodes whose name is absent here, by op type. + pub source_names_lost: BTreeMap, + /// Node names in this stage that no source node or initializer carries. + pub synthesized_names: usize, + /// Source node outputs that are still an outlet label of this stage. + pub source_outputs_labelled: usize, + pub source_outputs_total: usize, +} + +/// Count a stage. `node_names` and `outlet_labels` are the stage's; the +/// source inventory is the reference. +pub fn stage<'a>( + stage: &'static str, + src: &SourceInventory, + node_names: impl IntoIterator, + outlet_labels: impl IntoIterator, +) -> StageProvenance { + let names: Vec<&str> = node_names.into_iter().collect(); + let name_set: HashSet<&str> = names.iter().copied().collect(); + let labels: HashSet<&str> = outlet_labels.into_iter().collect(); + let src_names: HashSet<&str> = src + .nodes + .iter() + .map(|n| n.name.as_str()) + .chain(src.initializers.iter().map(|i| i.name.as_str())) + .collect(); + + let mut kept = 0; + let mut lost = BTreeMap::new(); + for n in src.nodes.iter().filter(|n| !n.name.is_empty()) { + if name_set.contains(n.name.as_str()) { + kept += 1; + } else { + *lost.entry(n.op_type.clone()).or_insert(0) += 1; + } + } + let outputs: Vec<&str> = src + .nodes + .iter() + .flat_map(|n| n.outputs.iter().map(String::as_str)) + .filter(|o| !o.is_empty()) + .collect(); + StageProvenance { + stage, + nodes: names.len(), + source_names_kept: kept, + source_names_lost: lost, + synthesized_names: names.iter().filter(|n| !src_names.contains(**n)).count(), + source_outputs_labelled: outputs.iter().filter(|o| labels.contains(**o)).count(), + source_outputs_total: outputs.len(), + } +} + +/// [`stage`] over any Tract graph (inference or typed). +pub fn of_graph( + name: &'static str, + src: &SourceInventory, + g: &tract_onnx::prelude::Graph, +) -> StageProvenance +where + F: tract_onnx::prelude::Fact + Clone + 'static, + O: std::fmt::Debug + + std::fmt::Display + + AsRef + + AsMut + + Clone + + 'static, +{ + stage( + name, + src, + g.nodes().iter().map(|n| n.name.as_str()), + g.outlet_labels.values().map(String::as_str), + ) +} diff --git a/crates/tract-harvest-probe/src/registry.rs b/crates/tract-harvest-probe/src/registry.rs new file mode 100644 index 000000000..3e34c9ff0 --- /dev/null +++ b/crates/tract-harvest-probe/src/registry.rs @@ -0,0 +1,776 @@ +//! The DT64 ON × WHEN × DO registration of ONNX operators (D-DT64-6 §1: +//! "Behaviour is registered as a DTO: classid → polyfill over mask-risc / +//! ndarray"; user decision 2026-10-30: one classid per op signature). +//! +//! An ONNX node is a registered method once its **op signature** +//! `(domain, op_type, dtype, discrete attributes)` matches a +//! [`Registration`]. The registration carries: +//! +//! - **ON**: the node's input positions (the DR#3 `on` relation, see `dr3`); +//! - **WHEN**: a typed three-valued guard over the node's facts (shapes, +//! dtypes, constant operand values). UNKNOWN never counts as TRUE; +//! - **DO**: the polyfill, an existing `lance-graph-onnx` / `ndarray` +//! function, writing the node's output positions. +//! +//! The classid's behaviour is immutable: a new behaviour is a new classid, never +//! an edit (D-DT64-6 D5). An unregistered signature is UNKNOWN (D-SAG-6 F06), +//! never "no behaviour": resolution refuses it. +//! +//! Classids here are **lab ordinals** (0x0F01…), not OGAR-minted concepts. A +//! production registration would mint the concept in OGAR and activate it +//! through `lance-graph-contract::hotplug`. + +use std::ops::RangeInclusive; + +use lance_graph_contract::class_view::ClassId; +use ndarray::{ArrayD, Axis, IxDyn}; + +use crate::source::{Attr, SourceNode}; +use crate::value::{DType, Value}; + +/// Three-valued truth of a WHEN guard. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum K3 { + True, + False, + Unknown, +} + +/// One dimension as Tract's typing knows it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Dim { + Known(usize), + Symbol(String), +} + +/// What the cold frontend knows about one Property position. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct Fact { + pub dtype: Option, + pub shape: Option>, +} + +/// A node as its guard sees it: the source node, the facts of its arguments +/// (in source order) and the values of its constant arguments. +pub struct NodeView<'a> { + pub node: &'a SourceNode, + pub args: Vec>, + pub consts: Vec>, +} + +/// How exactly a DO reproduces the ONNX semantics. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Numeric { + /// Bit-identical to any IEEE-754 / exact-integer implementation. + Exact, + /// Depends on evaluation order; bounded by measurement, not proof. + Bounded(&'static str), +} + +/// Why a graph does not resolve or a plan does not run. +#[derive(Debug, Clone, PartialEq)] +pub enum Refusal { + /// The cold frontend could not import or type the graph. + Frontend(String), + /// More than 64 positions on one axis: the graph needs several DR64 tiles. + TileOverflow { axis: &'static str, count: usize }, + /// No registration for this op signature: UNKNOWN, never "no behaviour". + Unregistered { node: String, signature: String }, + /// A registered signature outside the opsets its registration covers. + Opset { node: String, opset: i64 }, + /// The WHEN guard is FALSE or UNKNOWN for this node. + When { + node: String, + classid: ClassId, + truth: K3, + guard: &'static str, + }, + /// The dataflow relation has a cycle (ONNX graphs are acyclic). + Cycle, + /// A native shape or range refusal from `lance-graph-onnx`. + Native(lance_graph_onnx::Refusal), + /// The plan's frame is not the store's current seal (stale generation). + Frame(dr_math_binding_probe::Refusal), + /// A DO received values its guard did not admit (a run-time fact). + Runtime { node: String, what: String }, +} + +impl From for Refusal { + fn from(r: lance_graph_onnx::Refusal) -> Self { + Refusal::Native(r) + } +} + +/// A DO polyfill: argument values in source order, the source node for its +/// attributes; returns the output values in source order. +pub type Polyfill = fn(&[&Value], &SourceNode) -> Result, Refusal>; + +/// A WHEN guard: truth and the guard's name. +pub type Guard = fn(&NodeView) -> K3; + +/// A required discrete attribute value, with the ONNX default when absent. +#[derive(Debug, Clone, Copy)] +pub struct AttrRule { + pub name: &'static str, + pub value: i64, + pub default: Option, +} + +/// One registered method. +pub struct Registration { + pub classid: ClassId, + pub method: &'static str, + pub domain: &'static str, + pub op_type: &'static str, + /// Which argument's dtype keys the signature (`None`: the output's, for + /// `Constant`). + pub dtype_arg: Option, + pub dtype: DType, + /// Every discrete attribute the node may carry, with its required value. + /// A node attribute not listed here does not match (an unknown attribute + /// is not silently ignored). + pub attrs: &'static [AttrRule], + pub opsets: RangeInclusive, + pub when: Guard, + pub guard: &'static str, + pub do_: Polyfill, + pub numeric: Numeric, +} + +/// The registration table, versioned. Its version is the frame's +/// `relation_map` revision: a different table is a different frame. +pub struct Registry { + pub version: u64, + pub regs: Vec, +} + +/// The dtype a node's signature is keyed by. +pub fn signature_dtype( + node: &SourceNode, + args: &[Option<&Fact>], + dtype_arg: Option, +) -> Option { + match dtype_arg { + Some(i) => args.get(i).copied().flatten().and_then(|f| f.dtype), + None => match node.attr("value") { + Some(Attr::Tensor(Some(v))) => Some(v.dtype()), + _ => None, + }, + } +} + +/// `domain:op_type(dtype){attrs}`, for reporting a signature. +pub fn render_signature(node: &SourceNode, dtype: Option) -> String { + let domain = if node.domain.is_empty() { + "ai.onnx" + } else { + &node.domain + }; + let attrs: Vec = node + .attrs + .iter() + .map(|(n, a)| match a { + Attr::Int(v) => format!("{n}={v}"), + Attr::Ints(v) => format!("{n}={v:?}"), + Attr::Str(v) => format!("{n}={v:?}"), + Attr::Float(v) => format!("{n}={v:e}"), + Attr::Floats(v) => format!("{n}=floats[{}]", v.len()), + Attr::Tensor(Some(v)) => format!("{n}=tensor<{:?}>", v.dtype()), + Attr::Tensor(None) => format!("{n}=tensor"), + Attr::Other(t) => format!("{n}=type{t}"), + }) + .collect(); + format!( + "{domain}:{}({dtype:?}){{{}}}", + node.op_type, + attrs.join(",") + ) +} + +impl Registry { + /// The registration whose signature this node has, if any. + pub fn lookup(&self, node: &SourceNode, args: &[Option<&Fact>]) -> Option<&Registration> { + let domain = if node.domain.is_empty() { + "ai.onnx" + } else { + node.domain.as_str() + }; + self.regs.iter().find(|r| { + r.domain == domain + && r.op_type == node.op_type + && signature_dtype(node, args, r.dtype_arg) == Some(r.dtype) + && attrs_match(node, r.attrs) + }) + } +} + +fn attrs_match(node: &SourceNode, rules: &[AttrRule]) -> bool { + // Every node attribute is either the tensor `value` of a Constant or a + // listed discrete attribute with the required value. + let listed = node.attrs.iter().all(|(n, a)| { + (n == "value" && matches!(a, Attr::Tensor(_))) + || rules + .iter() + .any(|r| r.name == n && matches!(a, Attr::Int(v) if *v == r.value)) + }); + let defaults = rules + .iter() + .all(|r| node.attr(r.name).is_some() || r.default == Some(r.value)); + listed && defaults +} + +// ── guards ────────────────────────────────────────────────────────────────── + +fn known(shape: &Option>) -> Option> { + shape + .as_ref()? + .iter() + .map(|d| { + if let Dim::Known(v) = d { + Some(*v) + } else { + None + } + }) + .collect() +} + +/// Multidirectional broadcasting over Tract's facts. A symbolic dim agrees +/// with 1 or with the same symbol; anything else symbolic is UNKNOWN. +fn broadcast_k3(a: &[Dim], b: &[Dim]) -> K3 { + let mut unknown = false; + for (x, y) in a.iter().rev().zip(b.iter().rev()) { + match (x, y) { + (Dim::Known(p), Dim::Known(q)) if p == q || *p == 1 || *q == 1 => {} + (Dim::Known(_), Dim::Known(_)) => return K3::False, + (Dim::Symbol(s), Dim::Symbol(t)) if s == t => {} + (Dim::Known(1), _) | (_, Dim::Known(1)) => {} + _ => unknown = true, + } + } + if unknown { + K3::Unknown + } else { + K3::True + } +} + +fn guard_true(_: &NodeView) -> K3 { + K3::True +} + +fn guard_broadcast(v: &NodeView) -> K3 { + match ( + v.args.first().copied().flatten(), + v.args.get(1).copied().flatten(), + ) { + (Some(a), Some(b)) => match (&a.shape, &b.shape) { + (Some(x), Some(y)) => broadcast_k3(x, y), + _ => K3::Unknown, + }, + _ => K3::Unknown, + } +} + +fn guard_cast_identity(v: &NodeView) -> K3 { + // The signature already fixes `to = FLOAT` and the input dtype F32. + match v.args.first().copied().flatten().and_then(|f| f.dtype) { + Some(DType::F32) => K3::True, + Some(_) => K3::False, + None => K3::Unknown, + } +} + +fn guard_pow2(v: &NodeView) -> K3 { + match v.consts.get(1).copied().flatten().and_then(Value::f32s) { + Some(e) if !e.is_empty() && e.iter().all(|x| x.to_bits() == 2.0f32.to_bits()) => K3::True, + Some(_) => K3::False, + None => K3::Unknown, + } +} + +fn guard_single_axis(v: &NodeView) -> K3 { + match v.consts.get(1).copied().flatten().and_then(Value::i64s) { + Some(a) if a.len() == 1 => K3::True, + Some(_) => K3::False, + None if v.node.inputs.len() < 2 => K3::False, // absent axes: all axes, not admitted here + None => K3::Unknown, + } +} + +fn guard_rank1_sum(v: &NodeView) -> K3 { + let rank1 = match v + .args + .first() + .copied() + .flatten() + .and_then(|f| f.shape.as_ref()) + { + Some(s) if s.len() == 1 => K3::True, + Some(_) => K3::False, + None => K3::Unknown, + }; + if rank1 != K3::True { + return rank1; + } + match v.node.inputs.get(1).filter(|s| !s.is_empty()) { + None => K3::True, + Some(_) => match v.consts.get(1).copied().flatten().and_then(Value::i64s) { + Some(a) if a == [0] || a == [-1] => K3::True, + Some(_) => K3::False, + None => K3::Unknown, + }, + } +} + +fn guard_scalar_rhs(v: &NodeView) -> K3 { + match v + .args + .get(1) + .copied() + .flatten() + .and_then(|f| known(&f.shape)) + { + Some(s) if s.iter().product::() == 1 => K3::True, + Some(_) => K3::False, + None => K3::Unknown, + } +} + +fn guard_where(v: &NodeView) -> K3 { + let shape = |i: usize| { + v.args + .get(i) + .copied() + .flatten() + .and_then(|f| f.shape.clone()) + }; + match (shape(0), shape(1), shape(2)) { + (Some(c), Some(x), Some(y)) => { + let same_cx = if c == x { K3::True } else { K3::Unknown }; + let y_ok = match known(&Some(y.clone())) { + Some(s) if s.iter().product::() == 1 => K3::True, + _ if y == x => K3::True, + Some(_) => K3::False, + None => K3::Unknown, + }; + match (same_cx, y_ok) { + (_, K3::False) => K3::False, + (K3::True, K3::True) => K3::True, + _ => K3::Unknown, + } + } + _ => K3::Unknown, + } +} + +// ── DO polyfills: existing native functions only ─────────────────────────── + +fn f32_arg<'a>( + args: &[&'a Value], + i: usize, + node: &SourceNode, +) -> Result<&'a ArrayD, Refusal> { + match args.get(i) { + Some(Value::F32(a)) => Ok(a), + other => Err(Refusal::Runtime { + node: node.name.clone(), + what: format!("arg {i}: expected f32, got {:?}", other.map(|v| v.dtype())), + }), + } +} + +fn i32_arg<'a>( + args: &[&'a Value], + i: usize, + node: &SourceNode, +) -> Result<&'a ArrayD, Refusal> { + match args.get(i) { + Some(Value::I32(a)) => Ok(a), + other => Err(Refusal::Runtime { + node: node.name.clone(), + what: format!("arg {i}: expected i32, got {:?}", other.map(|v| v.dtype())), + }), + } +} + +fn do_constant(_: &[&Value], n: &SourceNode) -> Result, Refusal> { + match n.attr("value") { + Some(Attr::Tensor(Some(v))) => Ok(vec![v.clone()]), + _ => Err(Refusal::Runtime { + node: n.name.clone(), + what: "Constant without a decodable value".into(), + }), + } +} + +fn do_identity(a: &[&Value], _: &SourceNode) -> Result, Refusal> { + Ok(vec![a[0].clone()]) +} + +macro_rules! binary_do { + ($name:ident, $f:path) => { + fn $name(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + Ok(vec![Value::F32($f(f32_arg(a, 0, n)?, f32_arg(a, 1, n)?)?)]) + } + }; +} +binary_do!(do_add, lance_graph_onnx::add); +binary_do!(do_sub, lance_graph_onnx::sub); +binary_do!(do_mul, lance_graph_onnx::mul); +binary_do!(do_div, lance_graph_onnx::div); + +fn do_abs(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = f32_arg(a, 0, n)?; + let mut out = ArrayD::::zeros(x.raw_dim()); + lance_graph_onnx::abs_f32(x.view(), out.view_mut()); + Ok(vec![Value::F32(out)]) +} + +fn do_sqrt(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = f32_arg(a, 0, n)?; + let mut out = ArrayD::::zeros(x.raw_dim()); + lance_graph_onnx::sqrt_f32(x.view(), out.view_mut()); + Ok(vec![Value::F32(out)]) +} + +/// `Pow(x, 2)` admitted only with the exponent exactly 2.0 (its WHEN), so it +/// is one correctly rounded multiply, `Mul(x, x)`. +fn do_pow2(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = f32_arg(a, 0, n)?; + Ok(vec![Value::F32(lance_graph_onnx::mul(x, x)?)]) +} + +fn axis_of(a: &[&Value], n: &SourceNode, rank: usize) -> Result { + let ax = a + .get(1) + .and_then(|v| v.i64s()) + .and_then(|v| v.first().copied()); + let ax = ax.ok_or_else(|| Refusal::Runtime { + node: n.name.clone(), + what: "no axis".into(), + })?; + let r = rank as i64; + if (-r..r).contains(&ax) { + Ok(ax.rem_euclid(r) as usize) + } else { + Err(Refusal::Runtime { + node: n.name.clone(), + what: format!("axis {ax} out of rank {rank}"), + }) + } +} + +/// `ReduceMean` (keepdims 1) over one axis: `ndarray`'s `mean_axis`. The +/// float sum reassociates, so the method is Bounded. +fn do_reduce_mean(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = f32_arg(a, 0, n)?; + let ax = Axis(axis_of(a, n, x.ndim())?); + let m = x.mean_axis(ax).ok_or_else(|| Refusal::Runtime { + node: n.name.clone(), + what: "empty axis".into(), + })?; + Ok(vec![Value::F32(m.insert_axis(ax))]) +} + +fn reduce_sum_i32(a: &[&Value], n: &SourceNode, keepdims: i64) -> Result, Refusal> { + let x = i32_arg(a, 0, n)?; + let axes = a.get(1).and_then(|v| v.i64s()); + let r = lance_graph_onnx::ReduceSum::new(axes.as_deref(), keepdims, 0)?; + let xs: Vec = x.iter().copied().collect(); + let s = r.sum_i32(&xs)?; + let shape: &[usize] = if s.keepdims { &[1] } else { &[] }; + Ok(vec![Value::I32( + ArrayD::from_shape_vec(IxDyn(shape), vec![s.value]).expect("one element"), + )]) +} + +fn do_reduce_sum_i32_k0(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + reduce_sum_i32(a, n, 0) +} + +fn do_reduce_sum_i32_k1(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + reduce_sum_i32(a, n, 1) +} + +/// `Greater(X, t)`, scalar `t`: `ndarray::simd::gt_i32_to_mask` into a packed +/// `tensor(bool)`. +fn do_greater_i32(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = i32_arg(a, 0, n)?; + let t = i32_arg(a, 1, n)? + .iter() + .copied() + .next() + .ok_or_else(|| Refusal::Runtime { + node: n.name.clone(), + what: "empty threshold".into(), + })?; + let xs: Vec = x.iter().copied().collect(); + let mut words = vec![0u64; lance_graph_onnx::mask_words(xs.len())]; + lance_graph_onnx::greater_i32(&xs, t, &mut words); + Ok(vec![Value::Bool { + shape: x.shape().to_vec(), + words, + }]) +} + +/// `Where(C, X, Y)`, int32, `Y` scalar or of `X`'s shape: `blend_i32`. +fn do_where_i32(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let Some(Value::Bool { shape, words }) = a.first() else { + return Err(Refusal::Runtime { + node: n.name.clone(), + what: "condition is not a bool tensor".into(), + }); + }; + let x = i32_arg(a, 1, n)?; + let y = i32_arg(a, 2, n)?; + if shape.as_slice() != x.shape() { + return Err(Refusal::Runtime { + node: n.name.clone(), + what: "condition and X shapes differ".into(), + }); + } + let xs: Vec = x.iter().copied().collect(); + let ys: Vec = if y.len() == 1 { + vec![y.iter().copied().next().unwrap_or(0); xs.len()] + } else { + y.iter().copied().collect() + }; + if ys.len() != xs.len() { + return Err(Refusal::Runtime { + node: n.name.clone(), + what: "Y does not broadcast to X".into(), + }); + } + let mut out = vec![0i32; xs.len()]; + lance_graph_onnx::where_i32(words, &xs, &ys, &mut out); + Ok(vec![Value::I32( + ArrayD::from_shape_vec(IxDyn(x.shape()), out).expect("same length"), + )]) +} + +// ── the table ─────────────────────────────────────────────────────────────── + +const OPSETS: RangeInclusive = 13..=18; +const NO_ATTRS: &[AttrRule] = &[]; +const CAST_TO_F32: &[AttrRule] = &[AttrRule { + name: "to", + value: 1, + default: None, +}]; +const KEEPDIMS_1: &[AttrRule] = &[AttrRule { + name: "keepdims", + value: 1, + default: Some(1), +}]; +const KEEPDIMS_0: &[AttrRule] = &[AttrRule { + name: "keepdims", + value: 0, + default: None, +}]; + +macro_rules! reg { + ($cid:expr, $m:expr, $op:expr, $arg:expr, $dt:expr, $attrs:expr, $when:path, $do:path, $num:expr) => { + Registration { + classid: $cid, + method: $m, + domain: "ai.onnx", + op_type: $op, + dtype_arg: $arg, + dtype: $dt, + attrs: $attrs, + opsets: OPSETS, + when: $when, + guard: stringify!($when), + do_: $do, + numeric: $num, + } + }; +} + +/// The lab registry: every op the two fixtures use, nothing else. +pub fn registry() -> Registry { + use DType::*; + use Numeric::*; + Registry { + version: 1, + regs: vec![ + reg!( + 0x0F01, + "Constant: the attribute tensor", + "Constant", + None, + F32, + NO_ATTRS, + guard_true, + do_constant, + Exact + ), + reg!( + 0x0F02, + "Constant: the attribute tensor", + "Constant", + None, + I64, + NO_ATTRS, + guard_true, + do_constant, + Exact + ), + reg!( + 0x0F03, + "Constant: the attribute tensor", + "Constant", + None, + I32, + NO_ATTRS, + guard_true, + do_constant, + Exact + ), + reg!( + 0x0F10, + "Cast f32 -> f32: identity", + "Cast", + Some(0), + F32, + CAST_TO_F32, + guard_cast_identity, + do_identity, + Exact + ), + reg!( + 0x0F20, + "lance_graph_onnx::add (broadcast checked)", + "Add", + Some(0), + F32, + NO_ATTRS, + guard_broadcast, + do_add, + Exact + ), + reg!( + 0x0F21, + "lance_graph_onnx::sub (broadcast checked)", + "Sub", + Some(0), + F32, + NO_ATTRS, + guard_broadcast, + do_sub, + Exact + ), + reg!( + 0x0F22, + "lance_graph_onnx::mul (broadcast checked)", + "Mul", + Some(0), + F32, + NO_ATTRS, + guard_broadcast, + do_mul, + Exact + ), + reg!( + 0x0F23, + "lance_graph_onnx::div (broadcast checked)", + "Div", + Some(0), + F32, + NO_ATTRS, + guard_broadcast, + do_div, + Exact + ), + reg!( + 0x0F30, + "lance_graph_onnx::abs_f32 = vml::vsabs", + "Abs", + Some(0), + F32, + NO_ATTRS, + guard_true, + do_abs, + Exact + ), + reg!( + 0x0F31, + "lance_graph_onnx::sqrt_f32 = vml::vssqrt", + "Sqrt", + Some(0), + F32, + NO_ATTRS, + guard_true, + do_sqrt, + Exact + ), + reg!( + 0x0F32, + "Pow(x, 2.0) = lance_graph_onnx::mul(x, x)", + "Pow", + Some(0), + F32, + NO_ATTRS, + guard_pow2, + do_pow2, + Exact + ), + reg!( + 0x0F40, + "ndarray mean_axis, keepdims 1", + "ReduceMean", + Some(0), + F32, + KEEPDIMS_1, + guard_single_axis, + do_reduce_mean, + Bounded("float sum over the reduced axis; order is the implementation's") + ), + reg!( + 0x0F41, + "lance_graph_onnx::ReduceSum::sum_i32, keepdims 0", + "ReduceSum", + Some(0), + I32, + KEEPDIMS_0, + guard_rank1_sum, + do_reduce_sum_i32_k0, + Exact + ), + reg!( + 0x0F42, + "lance_graph_onnx::ReduceSum::sum_i32, keepdims 1", + "ReduceSum", + Some(0), + I32, + KEEPDIMS_1, + guard_rank1_sum, + do_reduce_sum_i32_k1, + Exact + ), + reg!( + 0x0F50, + "lance_graph_onnx::greater_i32 = simd::gt_i32_to_mask", + "Greater", + Some(0), + I32, + NO_ATTRS, + guard_scalar_rhs, + do_greater_i32, + Exact + ), + reg!( + 0x0F51, + "lance_graph_onnx::where_i32 = simd::blend_i32", + "Where", + Some(1), + I32, + NO_ATTRS, + guard_where, + do_where_i32, + Exact + ), + ], + } +} diff --git a/crates/tract-harvest-probe/src/source.rs b/crates/tract-harvest-probe/src/source.rs new file mode 100644 index 000000000..ff7ad3a17 --- /dev/null +++ b/crates/tract-harvest-probe/src/source.rs @@ -0,0 +1,254 @@ +//! Source evidence read from the raw ONNX protobuf, never from Tract's graph. +//! +//! Tract keeps node names and output labels, but it looks operators up by +//! `op_type` alone, deduplicates equal constants and synthesizes names. So +//! everything admission must not lose (domain, opset, initializer identity and +//! digest, external-data placement) is read here, from the bytes the model was +//! loaded from. + +use sha2::{Digest, Sha256}; +use tract_onnx::pb; + +use crate::value::{decode, Value}; + +/// SHA-256 of a byte string, lower-case hex. +pub fn sha256_hex(bytes: &[u8]) -> String { + let mut h = Sha256::new(); + h.update(bytes); + h.finalize().iter().map(|b| format!("{b:02x}")).collect() +} + +/// One attribute value, as the protobuf states it. +#[derive(Debug, Clone, PartialEq)] +pub enum Attr { + Int(i64), + Float(f32), + Ints(Vec), + Floats(Vec), + Str(String), + /// A tensor attribute (a `Constant`'s value), decoded; `None` if the + /// tensor's dtype has no [`Value`]. + Tensor(Option), + /// Graph, sparse tensor, type proto or other: kept by name only. + Other(i32), +} + +impl Attr { + fn from_proto(a: &pb::AttributeProto) -> Self { + match a.r#type { + 1 => Attr::Float(a.f), + 2 => Attr::Int(a.i), + 3 => Attr::Str(String::from_utf8_lossy(&a.s).into_owned()), + 4 => Attr::Tensor(a.t.as_ref().and_then(decode)), + 6 => Attr::Floats(a.floats.clone()), + 7 => Attr::Ints(a.ints.clone()), + t => Attr::Other(t), + } + } +} + +/// One source node, as the protobuf states it. +#[derive(Debug, Clone, PartialEq)] +pub struct SourceNode { + /// `NodeProto.name`; may be empty in real exports. + pub name: String, + pub op_type: String, + /// `""` and `"ai.onnx"` are the default domain. + pub domain: String, + pub inputs: Vec, + pub outputs: Vec, + pub attributes: Vec, + /// The attribute values, in source order. + pub attrs: Vec<(String, Attr)>, +} + +impl SourceNode { + pub fn default_domain(&self) -> bool { + self.domain.is_empty() || self.domain == "ai.onnx" + } + + pub fn attr(&self, name: &str) -> Option<&Attr> { + self.attrs.iter().find(|(n, _)| n == name).map(|(_, a)| a) + } +} + +/// Where an initializer's bytes are. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Placement { + /// Inline bytes, with their SHA-256. + Inline { sha256: String }, + /// External data: file, byte offset, byte length. + External { + location: String, + offset: u64, + length: u64, + }, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct SourceInitializer { + pub name: String, + pub dims: Vec, + /// `TensorProto.data_type` (1 = FLOAT, 6 = INT32, 7 = INT64). + pub data_type: i32, + pub placement: Placement, + /// The decoded value of an inline initializer; `None` for external data. + pub value: Option, +} + +/// What the model file says, independent of any importer. +#[derive(Debug, Clone)] +pub struct SourceInventory { + pub model_sha256: String, + pub ir_version: i64, + /// `(domain, version)`, `""` normalized to `"ai.onnx"`. + pub opsets: Vec<(String, i64)>, + pub nodes: Vec, + pub initializers: Vec, + /// Graph inputs that are not initializers, in source order. + pub inputs: Vec, + /// Graph outputs, in source order. + pub outputs: Vec, +} + +impl SourceInventory { + /// Read the inventory from model bytes. The protobuf is decoded with the + /// same `prost` types Tract uses, but nothing of Tract's import runs. + pub fn from_bytes(bytes: &[u8]) -> Result { + use prost::Message; + let m = pb::ModelProto::decode(bytes)?; + Ok(Self::from_proto(&m, sha256_hex(bytes))) + } + + pub fn from_proto(m: &pb::ModelProto, model_sha256: String) -> Self { + let g = m.graph.as_ref(); + let nodes = g + .map(|g| { + g.node + .iter() + .map(|n| SourceNode { + name: n.name.clone(), + op_type: n.op_type.clone(), + domain: n.domain.clone(), + inputs: n.input.clone(), + outputs: n.output.clone(), + attributes: n.attribute.iter().map(|a| a.name.clone()).collect(), + attrs: n + .attribute + .iter() + .map(|a| (a.name.clone(), Attr::from_proto(a))) + .collect(), + }) + .collect() + }) + .unwrap_or_default(); + let initializers: Vec = g + .map(|g| g.initializer.iter().map(initializer).collect()) + .unwrap_or_default(); + let inputs = g + .map(|g| { + g.input + .iter() + .filter(|i| !initializers.iter().any(|t| t.name == i.name)) + .map(|i| i.name.clone()) + .collect() + }) + .unwrap_or_default(); + let outputs = g + .map(|g| g.output.iter().map(|o| o.name.clone()).collect()) + .unwrap_or_default(); + let opsets = m + .opset_import + .iter() + .map(|o| { + let d = if o.domain.is_empty() { + "ai.onnx".to_string() + } else { + o.domain.clone() + }; + (d, o.version) + }) + .collect(); + Self { + model_sha256, + ir_version: m.ir_version, + opsets, + nodes, + initializers, + inputs, + outputs, + } + } + + /// The default-domain opset version, if declared. + pub fn default_opset(&self) -> Option { + self.opsets + .iter() + .find(|(d, _)| d == "ai.onnx") + .map(|(_, v)| *v) + } + + pub fn node_producing(&self, output: &str) -> Option<&SourceNode> { + self.nodes + .iter() + .find(|n| n.outputs.iter().any(|o| o == output)) + } + + pub fn node_named(&self, name: &str) -> Option<&SourceNode> { + self.nodes.iter().find(|n| n.name == name) + } + + pub fn initializer(&self, name: &str) -> Option<&SourceInitializer> { + self.initializers.iter().find(|i| i.name == name) + } +} + +fn initializer(t: &pb::TensorProto) -> SourceInitializer { + let external = t.data_location == Some(pb::tensor_proto::DataLocation::External as i32); + let placement = if external { + let get = |k: &str| { + t.external_data + .iter() + .find(|e| e.key == k) + .map(|e| e.value.clone()) + }; + Placement::External { + location: get("location").unwrap_or_default(), + offset: get("offset").and_then(|v| v.parse().ok()).unwrap_or(0), + length: get("length").and_then(|v| v.parse().ok()).unwrap_or(0), + } + } else { + Placement::Inline { + sha256: sha256_hex(&inline_bytes(t)), + } + }; + SourceInitializer { + name: t.name.clone(), + dims: t.dims.clone(), + data_type: t.data_type, + placement, + value: decode(t), + } +} + +/// The inline payload as stored: `raw_data` if present, else the typed field +/// re-encoded little-endian. Two encodings of the same values hash the same. +fn inline_bytes(t: &pb::TensorProto) -> Vec { + if !t.raw_data.is_empty() { + return t.raw_data.to_vec(); + } + let mut v = Vec::new(); + t.float_data + .iter() + .for_each(|x| v.extend_from_slice(&x.to_le_bytes())); + t.int32_data + .iter() + .for_each(|x| v.extend_from_slice(&x.to_le_bytes())); + t.int64_data + .iter() + .for_each(|x| v.extend_from_slice(&x.to_le_bytes())); + t.double_data + .iter() + .for_each(|x| v.extend_from_slice(&x.to_le_bytes())); + v +} diff --git a/crates/tract-harvest-probe/src/value.rs b/crates/tract-harvest-probe/src/value.rs new file mode 100644 index 000000000..cd4dfcbc4 --- /dev/null +++ b/crates/tract-harvest-probe/src/value.rs @@ -0,0 +1,152 @@ +//! Tensor values on Property positions, and the decoding of source tensors. +//! +//! `tensor(bool)` is the packed mask every `ndarray::simd` mask kernel uses +//! (bit `i % 64` of word `i / 64` is element `i`), the `lance-graph-onnx` +//! convention. A bool tensor in the source is decoded into that form here, +//! once, and the packing is the only conversion (falsifier 5 of #1802). + +use ndarray::{ArrayD, IxDyn}; +use tract_onnx::pb; + +/// Element types a Property position can carry. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] +pub enum DType { + F32, + F64, + F16, + I32, + I64, + Bool, + /// Any other `TensorProto.DataType`, by its number. + Other(i32), +} + +impl DType { + /// From `TensorProto.DataType`. + pub fn from_onnx(t: i32) -> Self { + match t { + 1 => DType::F32, + 6 => DType::I32, + 7 => DType::I64, + 9 => DType::Bool, + 10 => DType::F16, + 11 => DType::F64, + o => DType::Other(o), + } + } +} + +/// A value bound to a Property position. +#[derive(Debug, Clone, PartialEq)] +pub enum Value { + F32(ArrayD), + I32(ArrayD), + I64(ArrayD), + /// Packed `tensor(bool)`: logical shape plus mask words. + Bool { + shape: Vec, + words: Vec, + }, +} + +impl Value { + pub fn dtype(&self) -> DType { + match self { + Value::F32(_) => DType::F32, + Value::I32(_) => DType::I32, + Value::I64(_) => DType::I64, + Value::Bool { .. } => DType::Bool, + } + } + + pub fn shape(&self) -> Vec { + match self { + Value::F32(a) => a.shape().to_vec(), + Value::I32(a) => a.shape().to_vec(), + Value::I64(a) => a.shape().to_vec(), + Value::Bool { shape, .. } => shape.clone(), + } + } + + /// Every element as `f32`, for reading a small constant (an exponent, an + /// epsilon); `None` for a non-float value. + pub fn f32s(&self) -> Option> { + match self { + Value::F32(a) => Some(a.iter().copied().collect()), + _ => None, + } + } + + /// Every element as `i64` for an integer value (axes, indices). + pub fn i64s(&self) -> Option> { + match self { + Value::I64(a) => Some(a.iter().copied().collect()), + Value::I32(a) => Some(a.iter().map(|&v| i64::from(v)).collect()), + _ => None, + } + } +} + +fn le(raw: &[u8]) -> impl Iterator + '_ { + raw.as_chunks::().0.iter().copied() +} + +/// Decode an inline `TensorProto` into a [`Value`]. `None` for external data +/// (its bytes are not in the protobuf) or a dtype no Property carries. +pub fn decode(t: &pb::TensorProto) -> Option { + if t.data_location == Some(pb::tensor_proto::DataLocation::External as i32) { + return None; + } + let shape: Vec = t.dims.iter().map(|&d| d as usize).collect(); + let raw = &t.raw_data; + let arr = |n: usize| shape.iter().product::() == n; + match DType::from_onnx(t.data_type) { + DType::F32 => { + let v: Vec = if raw.is_empty() { + t.float_data.clone() + } else { + le::<4>(raw).map(f32::from_le_bytes).collect() + }; + ArrayD::from_shape_vec(IxDyn(&shape), v) + .ok() + .map(Value::F32) + } + DType::I32 => { + let v: Vec = if raw.is_empty() { + t.int32_data.clone() + } else { + le::<4>(raw).map(i32::from_le_bytes).collect() + }; + ArrayD::from_shape_vec(IxDyn(&shape), v) + .ok() + .map(Value::I32) + } + DType::I64 => { + let v: Vec = if raw.is_empty() { + t.int64_data.clone() + } else { + le::<8>(raw).map(i64::from_le_bytes).collect() + }; + ArrayD::from_shape_vec(IxDyn(&shape), v) + .ok() + .map(Value::I64) + } + DType::Bool => { + // ONNX stores bool as one byte per element (raw) or int32_data. + let bits: Vec = if raw.is_empty() { + t.int32_data.iter().map(|&v| v != 0).collect() + } else { + raw.iter().map(|&b| b != 0).collect() + }; + if !arr(bits.len()) { + return None; + } + let mut words = vec![0u64; bits.len().div_ceil(64)]; + for (i, b) in bits.iter().enumerate() { + words[i / 64] |= u64::from(*b) << (i % 64); + } + Some(Value::Bool { shape, words }) + } + _ => None, + } +} diff --git a/crates/tract-harvest-probe/tests/jina.rs b/crates/tract-harvest-probe/tests/jina.rs new file mode 100644 index 000000000..9cb54069f --- /dev/null +++ b/crates/tract-harvest-probe/tests/jina.rs @@ -0,0 +1,429 @@ +//! Fixture B: layer 0's input RMSNorm of the published Jina v5 small +//! (retrieval) ONNX export, Qwen3 architecture, run as a DR#3 graph through +//! the registry. The data is CC-BY-NC-4.0 and never committed; fetch it first, +//! then run the ignored tests: +//! +//! ```text +//! python3 tools/fetch_jina_slice.py +//! cargo test --release --test jina -- --ignored --nocapture +//! ``` +//! +//! The tests are `#[ignore]` so a fresh clone reports them as not run instead +//! of passing on missing data. + +use dr_math_binding_probe::FrameStore; +use ndarray::{ArrayD, IxDyn}; +use prost::Message; +use serde_json::Value as Json; +use tract_harvest_probe::dr3::{tract_facts, OnnxGraph}; +use tract_harvest_probe::gather; +use tract_harvest_probe::plan::{resolve, revisions, Plan}; +use tract_harvest_probe::registry::{registry, Numeric, Refusal, Registry, K3}; +use tract_harvest_probe::source::{sha256_hex, SourceInventory}; +use tract_harvest_probe::value::Value; +use tract_onnx::pb; +use tract_onnx::prelude::*; + +const FIX: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/fixtures"); +const IGNORE: &str = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"; +const P: &str = "/layers.0/input_layernorm/"; + +fn manifest() -> Json { + serde_json::from_str(&std::fs::read_to_string(format!("{FIX}/jina_manifest.json")).unwrap()) + .unwrap() +} + +fn read_f32(name: &str) -> Vec { + let b = std::fs::read(format!("{FIX}/jina/{name}")) + .unwrap_or_else(|e| panic!("{name}: {e} ({IGNORE})")); + b.as_chunks::<4>() + .0 + .iter() + .map(|c| f32::from_le_bytes(*c)) + .collect() +} + +fn slice_bytes() -> Vec { + let b = std::fs::read(format!("{FIX}/jina/slice.onnx")).unwrap(); + assert_eq!( + sha256_hex(&b), + manifest()["slice"]["sha256"].as_str().unwrap(), + "slice is the pinned one" + ); + b +} + +fn shape() -> Vec { + manifest()["slice"]["shape"] + .as_array() + .unwrap() + .iter() + .map(|v| v.as_u64().unwrap() as usize) + .collect() +} + +fn name(key: &str) -> String { + manifest()["slice"][key].as_str().unwrap().to_string() +} + +/// Distance in units in the last place between two finite f32 of either sign. +fn ulp(a: f32, b: f32) -> u32 { + let key = |v: f32| { + let i = v.to_bits() as i32; + if i < 0 { + i32::MIN.wrapping_sub(i) + } else { + i + } + }; + (key(a) as i64 - key(b) as i64).unsigned_abs() as u32 +} + +struct Cmp { + exact: usize, + total: usize, + max_ulp: u32, +} + +fn cmp(a: &[f32], b: &[f32]) -> Cmp { + assert_eq!(a.len(), b.len()); + Cmp { + exact: a + .iter() + .zip(b) + .filter(|(x, y)| x.to_bits() == y.to_bits()) + .count(), + total: a.len(), + max_ulp: a.iter().zip(b).map(|(x, y)| ulp(*x, *y)).max().unwrap_or(0), + } +} + +fn plan_of<'r>(bytes: &[u8], reg: &'r Registry) -> (FrameStore, Result, Refusal>) { + let src = SourceInventory::from_bytes(bytes).unwrap(); + let facts = match tract_facts(bytes, &[(0, DatumType::F32, shape())]) { + Ok(f) => f, + Err(e) => { + return ( + FrameStore::new(dr_math_binding_probe::FrameRevisions { + source: 0, + relation_map: 0, + coordinates: 0, + }), + Err(e), + ) + } + }; + let g = OnnxGraph::build(&src, &facts).unwrap(); + let store = FrameStore::new(revisions(&src, reg, &g)); + let p = resolve(&src, g, reg, &store); + (store, p) +} + +fn input() -> ArrayD { + ArrayD::from_shape_vec(IxDyn(&shape()), read_f32("gather_rows.f32")).unwrap() +} + +fn f32_out(m: &std::collections::HashMap, k: &str) -> ArrayD { + match &m[k] { + Value::F32(a) => a.clone(), + other => panic!("{k}: {other:?}"), + } +} + +/// The slice laid onto the tile, every node bound by its op signature, run +/// node by node, compared with ORT literal (the output, and separately the one +/// Bounded step, the mean). +#[test] +#[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] +fn rmsnorm_slice_through_the_graph_against_ort() { + let reg = registry(); + let bytes = slice_bytes(); + let (store, p) = plan_of(&bytes, ®); + let p = p.unwrap(); + let steps: Vec<(String, u16)> = p + .steps + .iter() + .map(|s| (s.name.trim_start_matches(P).to_string(), s.classid)) + .collect(); + eprintln!("schedule: {steps:?}"); + assert_eq!(steps.len(), 13, "all 13 source nodes, Constants included"); + let cid = |n: &str| steps.iter().find(|(s, _)| s == n).unwrap().1; + assert_eq!( + [ + cid("Cast"), + cid("Pow"), + cid("ReduceMean"), + cid("Add"), + cid("Sqrt"), + cid("Div"), + cid("Mul"), + cid("Cast_1"), + cid("Mul_1") + ], + [0x0F10, 0x0F32, 0x0F40, 0x0F20, 0x0F31, 0x0F23, 0x0F22, 0x0F10, 0x0F22] + ); + let bounded: Vec<&str> = p + .steps + .iter() + .filter(|s| matches!(s.numeric, Numeric::Bounded(_))) + .map(|s| s.name.trim_start_matches(P)) + .collect(); + assert_eq!(bounded, ["ReduceMean"], "the only Bounded method"); + + let x = input(); + let out = p + .run(&store, &[(name("input").as_str(), Value::F32(x.clone()))]) + .unwrap(); + let y = f32_out(&out, &name("output")); + let mean = f32_out(&out, &name("intermediate")); + assert_eq!(y.shape(), shape().as_slice()); + + let ort_out = read_f32("ort_out.f32"); + let ort_mid = read_f32("ort_mid.f32"); + let ort_opt = read_f32("ort_out_opt.f32"); + let c_mid = cmp(mean.as_slice().unwrap(), &ort_mid); + let c_out = cmp(y.as_slice().unwrap(), &ort_out); + let c_opt = cmp(y.as_slice().unwrap(), &ort_opt); + eprintln!( + "graph mean vs ORT literal: {}/{} exact, max {} ulp", + c_mid.exact, c_mid.total, c_mid.max_ulp + ); + eprintln!( + "graph out vs ORT literal: {}/{} exact, max {} ulp", + c_out.exact, c_out.total, c_out.max_ulp + ); + eprintln!( + "graph out vs ORT optimized (report only): {}/{} exact, max {} ulp", + c_opt.exact, c_opt.total, c_opt.max_ulp + ); + + // The mean against mathematics: f32 squares summed in f64, divided, rounded once. + let hidden = *shape().last().unwrap(); + let reference: Vec = x + .as_slice() + .unwrap() + .chunks_exact(hidden) + .map(|r| (r.iter().map(|v| f64::from(v * v)).sum::() / hidden as f64) as f32) + .collect(); + for (arm, mid) in [ + ("graph", mean.as_slice().unwrap()), + ("ORT literal", &ort_mid[..]), + ] { + let c = cmp(mid, &reference); + eprintln!( + "{arm} mean vs f64 reference: {}/{} exact, max {} ulp", + c.exact, c.total, c.max_ulp + ); + } + + // Measured bound, pinned two-sided-in-spirit: a regression fails here. + assert!( + c_mid.max_ulp <= 4, + "graph mean drifted: {} ulp", + c_mid.max_ulp + ); + assert!( + c_out.max_ulp <= 4, + "graph RMSNorm drifted: {} ulp", + c_out.max_ulp + ); + // Every step after the mean is a single correctly rounded IEEE op, so a + // row whose mean matches ORT bit for bit must match on every element. + let mut exact_rows = 0; + for (row, (m, o)) in mean.iter().zip(&ort_mid).enumerate() { + if m.to_bits() == o.to_bits() { + exact_rows += 1; + let a = &y.as_slice().unwrap()[row * hidden..(row + 1) * hidden]; + let b = &ort_out[row * hidden..(row + 1) * hidden]; + assert_eq!( + cmp(a, b).exact, + hidden, + "row {row}: exact mean but inexact output" + ); + } + } + eprintln!("rows with a bit-exact mean: {exact_rows}/{}", mean.len()); +} + +/// `Gather` resolved from the source placement equals the byte ranges the +/// fetch tool pinned; out-of-range and negative indices follow the ONNX rule. +#[test] +#[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] +fn gather_rows_from_source_placement() { + let m = manifest(); + let model = std::fs::read(format!("{FIX}/jina/onnx_model.onnx")).unwrap(); + assert_eq!( + sha256_hex(&model), + m["pins"]["onnx/model.onnx"].as_str().unwrap() + ); + let src = SourceInventory::from_bytes(&model).unwrap(); + let ids: Vec = m["token_ids"] + .as_array() + .unwrap() + .iter() + .map(|v| v.as_i64().unwrap()) + .collect(); + let ranges = gather::row_ranges(&src, "embed_tokens.weight", &ids).unwrap(); + let rows = std::fs::read(format!("{FIX}/jina/gather_rows.f32")).unwrap(); + for (i, (r, want)) in ranges + .iter() + .zip(m["embedding"]["ranges"].as_array().unwrap()) + .enumerate() + { + assert_eq!(r.offset, want["offset"].as_u64().unwrap()); + assert_eq!(r.length, 4096); + assert_eq!( + sha256_hex(&rows[i * 4096..(i + 1) * 4096]), + want["sha256"].as_str().unwrap() + ); + } + let vocab = 151_936i64; + for bad in [vocab, -vocab - 1] { + assert!(matches!( + gather::row_ranges(&src, "embed_tokens.weight", &[bad]), + Err(Refusal::Runtime { .. }) + )); + } + assert_eq!( + gather::row_ranges(&src, "embed_tokens.weight", &[-1]).unwrap()[0].row, + vocab as u64 - 1 + ); +} + +// ── negative controls: mutate the pinned slice in its protobuf ────────────── + +fn mutate(f: impl FnOnce(&mut pb::ModelProto)) -> Vec { + let mut m = pb::ModelProto::decode(&*slice_bytes()).unwrap(); + f(&mut m); + m.encode_to_vec() +} + +fn node_named<'a>(m: &'a mut pb::ModelProto, n: &str) -> &'a mut pb::NodeProto { + m.graph + .as_mut() + .unwrap() + .node + .iter_mut() + .find(|x| x.name == format!("{P}{n}")) + .unwrap() +} + +/// `Pow(x, 2.0000002)`: the Pow registration's WHEN reads the constant +/// exponent from the source and is FALSE. +#[test] +#[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] +fn pow_exponent_other_than_two_is_when_false() { + let b = mutate(|m| { + let t = node_named(m, "Constant").attribute[0].t.as_mut().unwrap(); + t.raw_data = 2.000_000_2f32.to_le_bytes().to_vec(); + t.float_data.clear(); + }); + match plan_of(&b, ®istry()).1 { + Err(Refusal::When { + node, + classid, + truth, + .. + }) => { + assert_eq!( + (node.trim_start_matches(P), classid, truth), + ("Pow", 0x0F32, K3::False) + ) + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// The mean over axis -2 instead of -1 is a different, valid program. Nothing +/// assumed the RMSNorm pattern, so the graph runs the program as written: the +/// mean has the shape the source axis gives. +#[test] +#[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] +fn another_reduction_axis_is_a_different_program_not_a_refusal() { + let b = mutate(|m| { + let t = node_named(m, "Constant_1").attribute[0].t.as_mut().unwrap(); + t.raw_data = (-2i64).to_le_bytes().to_vec(); + t.int64_data.clear(); + }); + let reg = registry(); + let (store, p) = plan_of(&b, ®); + let out = p + .unwrap() + .run(&store, &[(name("input").as_str(), Value::F32(input()))]) + .unwrap(); + let sh = shape(); + assert_eq!( + f32_out(&out, &name("intermediate")).shape(), + &[sh[0], 1, sh[2]] + ); + assert_eq!(f32_out(&out, &name("output")).shape(), sh.as_slice()); +} + +/// `Cast_1` to FLOAT16: the signature `Cast{to=10}` is unregistered. +#[test] +#[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] +fn cast_to_half_is_unregistered() { + let b = mutate(|m| node_named(m, "Cast_1").attribute[0].i = 10); + match plan_of(&b, ®istry()).1 { + Err(Refusal::Unregistered { node, signature }) => { + assert!(node.ends_with("Cast_1"), "{node}"); + assert!(signature.contains("to=10"), "{signature}"); + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// The first Cast widened to f64 (and the constants with it): `Cast{to=11}` is +/// unregistered, and it is the first node of the schedule to fail. +#[test] +#[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] +fn cast_to_double_is_unregistered() { + let b = mutate(|m| { + node_named(m, "Cast").attribute[0].i = 11; + for c in ["Constant", "Constant_2", "Constant_3"] { + let t = node_named(m, c).attribute[0].t.as_mut().unwrap(); + let v = f32::from_le_bytes(t.raw_data[..4].try_into().unwrap()); + t.data_type = 11; + t.raw_data = f64::from(v).to_le_bytes().to_vec(); + } + }); + match plan_of(&b, ®istry()).1 { + Err(Refusal::Unregistered { node, signature }) => { + assert!(node.ends_with("/Cast"), "{node}"); + assert!(signature.contains("to=11"), "{signature}"); + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// One bit of one weight changed: a new source generation. The plan sealed for +/// the pinned slice refuses to run once the store advances to it. +#[test] +#[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] +fn new_weight_generation_refuses_the_old_plan() { + let reg = registry(); + let (mut store, p) = plan_of(&slice_bytes(), ®); + let p = p.unwrap(); + let b = mutate(|m| { + let w = m + .graph + .as_mut() + .unwrap() + .initializer + .iter_mut() + .find(|t| t.name == "layers.0.input_layernorm.weight") + .unwrap(); + w.raw_data[0] ^= 1; + }); + let src2 = SourceInventory::from_bytes(&b).unwrap(); + let g2 = OnnxGraph::build( + &src2, + &tract_facts(&b, &[(0, DatumType::F32, shape())]).unwrap(), + ) + .unwrap(); + store.advance(revisions(&src2, ®, &g2)); + match p.run(&store, &[(name("input").as_str(), Value::F32(input()))]) { + Err(Refusal::Frame(dr_math_binding_probe::Refusal::SourceChanged)) => {} + other => panic!("{:?}", other.map(|_| ())), + } +} diff --git a/crates/tract-harvest-probe/tests/tiny.rs b/crates/tract-harvest-probe/tests/tiny.rs new file mode 100644 index 000000000..92ec53906 --- /dev/null +++ b/crates/tract-harvest-probe/tests/tiny.rs @@ -0,0 +1,387 @@ +//! Fixture A (`fixtures/tiny`, generated by `tools/gen_tiny.py`) through the +//! graph: the ONNX graph laid onto the DR#3 tile, every node resolved to its +//! op-signature classid, its WHEN evaluated, its DO run. ORT (`ORT_DISABLE_ALL`, +//! the literal semantics) is read from `fixtures/tiny/ort.json`; execution +//! never calls Tract or ORT. + +use dr_math_binding_probe::FrameStore; +use ndarray::{ArrayD, IxDyn}; +use serde_json::Value as Json; +use tract_harvest_probe::dr3::{tract_facts, OnnxGraph}; +use tract_harvest_probe::plan::{resolve, revisions, Plan}; +use tract_harvest_probe::registry::{registry, Refusal, Registry, K3}; +use tract_harvest_probe::source::SourceInventory; +use tract_harvest_probe::value::Value; +use tract_onnx::prelude::Framework; + +const DIR: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/fixtures/tiny"); + +fn bytes(v: &str) -> Vec { + std::fs::read(format!("{DIR}/{v}.onnx")).unwrap() +} + +fn ort() -> Json { + serde_json::from_str(&std::fs::read_to_string(format!("{DIR}/ort.json")).unwrap()).unwrap() +} + +fn u32s(v: &Json) -> Vec { + v.as_array() + .unwrap() + .iter() + .map(|b| b.as_u64().unwrap() as u32) + .collect() +} + +fn f32_value(shape: &[usize], bits: &[u32]) -> Value { + Value::F32( + ArrayD::from_shape_vec( + IxDyn(shape), + bits.iter().map(|&b| f32::from_bits(b)).collect(), + ) + .unwrap(), + ) +} + +fn inputs(f_bits: Option<&[u32]>) -> Vec<(&'static str, Value)> { + let o = ort(); + let x: Vec = o["inputs"]["X"]["bits"] + .as_array() + .unwrap() + .iter() + .map(|v| v.as_i64().unwrap() as i32) + .collect(); + let f = f_bits + .map(<[u32]>::to_vec) + .unwrap_or_else(|| u32s(&o["inputs"]["F"]["bits"])); + vec![ + ( + "X", + Value::I32(ArrayD::from_shape_vec(IxDyn(&[x.len()]), x).unwrap()), + ), + ("F", f32_value(&[2, 3], &f)), + ] +} + +fn ort_out(v: &str) -> (i32, Vec) { + let e = &ort()["variants"][v]["ort"]; + ( + e["S"]["bits"][0].as_i64().unwrap() as i32, + u32s(&e["Q"]["bits"]), + ) +} + +struct Fixture { + store: FrameStore, + src: SourceInventory, +} + +fn graph(v: &str) -> (SourceInventory, OnnxGraph) { + let b = bytes(v); + let src = SourceInventory::from_bytes(&b).unwrap(); + let facts = tract_facts(&b, &[]).unwrap(); + let g = OnnxGraph::build(&src, &facts).unwrap(); + (src, g) +} + +fn plan<'r>(v: &str, reg: &'r Registry) -> (Fixture, Result, Refusal>) { + let (src, g) = graph(v); + let store = FrameStore::new(revisions(&src, reg, &g)); + let p = resolve(&src, g, reg, &store); + (Fixture { store, src }, p) +} + +fn outputs(m: &std::collections::HashMap) -> (i32, Vec) { + let s = match &m["S"] { + Value::I32(a) => { + assert_eq!(a.shape(), &[] as &[usize], "keepdims=0 gives a scalar"); + a.iter().copied().next().unwrap() + } + other => panic!("{other:?}"), + }; + let q = match &m["Q"] { + Value::F32(a) => { + assert_eq!(a.shape(), &[2, 3]); + a.iter().map(|v| v.to_bits()).collect() + } + other => panic!("{other:?}"), + }; + (s, q) +} + +/// The graph runs node by node through registered methods and agrees with ORT +/// bit for bit. Every node got a classid; none is a pattern recipe. +#[test] +fn base_through_the_graph_is_bit_exact_against_ort() { + let reg = registry(); + let (fx, p) = plan("base", ®); + let p = p.unwrap(); + let names: Vec<&str> = p.steps.iter().map(|s| s.name.as_str()).collect(); + assert_eq!( + names, + [ + "n_greater", + "n_where", + "n_reducesum", + "n_add", + "n_mul", + "n_abs", + "n_sqrt", + "n_div" + ] + ); + let cids: Vec = p.steps.iter().map(|s| s.classid).collect(); + assert_eq!( + cids, + [0x0F50, 0x0F51, 0x0F41, 0x0F20, 0x0F22, 0x0F30, 0x0F31, 0x0F23] + ); + assert_eq!( + outputs(&p.run(&fx.store, &inputs(None)).unwrap()), + ort_out("base") + ); +} + +/// `dep = do_ ∘ read_by` is exactly the source dataflow (no more, no less), +/// and the schedule is a topological order of it. +#[test] +fn dep_composition_is_the_source_dataflow() { + for v in ["base", "div3", "neg_where_nonzero"] { + let (_, g) = graph(v); + let (dep, work) = g.dep(); + assert_eq!(dep, g.dep_from_edges(), "{v}"); + assert!( + dep.iter().any(|&r| r != 0), + "{v}: anti-vacuity, there are edges" + ); + assert!(work.row_ors > 0); + let order = g.schedule().unwrap(); + let pos: Vec = (0..g.nodes) + .map(|n| order.iter().position(|&o| o == n).unwrap()) + .collect(); + for (i, row) in dep.iter().enumerate().take(g.nodes) { + for j in 0..g.nodes { + if row >> j & 1 == 1 { + assert!(pos[i] < pos[j], "{v}: {i} before {j}"); + } + } + } + } +} + +/// `Where(c, X, 1)` (ORT: 17). With the graph there is no masked-sum rewrite to +/// refuse: `Where` and `ReduceSum` are separate registered methods, and the +/// graph runs and matches ORT. +#[test] +fn where_with_nonzero_else_runs_and_matches_ort() { + let reg = registry(); + let (fx, p) = plan("neg_where_nonzero", ®); + assert_eq!(ort_out("neg_where_nonzero").0, 17); + assert_eq!( + outputs(&p.unwrap().run(&fx.store, &inputs(None)).unwrap()), + ort_out("neg_where_nonzero") + ); +} + +/// `div3` over 256 seeded inputs: the graph keeps `Div` (classid 0x0F23) and +/// matches ORT bit for bit on every element. +#[test] +fn div3_sweep_is_bit_exact() { + let reg = registry(); + let (fx, p) = plan("div3", ®); + let p = p.unwrap(); + let o = ort(); + let sweep = &o["div3_sweep"]; + let fs = sweep["F_bits"].as_array().unwrap(); + assert_eq!(fs.len(), 256); + for (fb, qb) in fs.iter().zip(sweep["Q_bits"].as_array().unwrap()) { + let out = p.run(&fx.store, &inputs(Some(&u32s(fb)))).unwrap(); + assert_eq!(outputs(&out).1, u32s(qb)); + } +} + +/// A plan sealed for `base` refuses to run once the store advances to `gen2` +/// (same shapes, new `B` bytes); `gen2` resolved on its own matches its ORT. +#[test] +fn stale_generation_is_refused() { + let reg = registry(); + let (mut fx, p) = plan("base", ®); + let p = p.unwrap(); + let (src2, g2) = graph("gen2"); + let r2 = revisions(&src2, ®, &g2); + assert_eq!(r2.relation_map, fx.store.revisions().relation_map); + assert_eq!( + r2.coordinates, + fx.store.revisions().coordinates, + "same positions" + ); + assert_ne!(r2.source, fx.store.revisions().source); + fx.store.advance(r2); + match p.run(&fx.store, &inputs(None)) { + Err(Refusal::Frame(dr_math_binding_probe::Refusal::SourceChanged)) => {} + other => panic!("{:?}", other.map(|_| ())), + } + let _ = &fx.src; + let reg2 = registry(); + let (fx2, p2) = plan("gen2", ®2); + assert_eq!( + outputs(&p2.unwrap().run(&fx2.store, &inputs(None)).unwrap()), + ort_out("gen2") + ); +} + +/// A registry with another version is another frame: a plan sealed under v1 +/// refuses once the store moves to the v2 map. +#[test] +fn registry_change_is_a_frame_change() { + let reg = registry(); + let (mut fx, p) = plan("base", ®); + let p = p.unwrap(); + let mut r = fx.store.revisions(); + r.relation_map += 1; + fx.store.advance(r); + match p.run(&fx.store, &inputs(None)) { + Err(Refusal::Frame(dr_math_binding_probe::Refusal::MapChanged)) => {} + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// `B[4]` against `F[2,3]`: ORT refuses at load; Tract types it. The Add's WHEN +/// (multidirectional broadcast over the facts) is FALSE, so resolution refuses +/// before anything runs. +#[test] +fn incompatible_broadcast_is_when_false() { + assert!(ort()["variants"]["neg_shape"]["ort_error"] + .as_str() + .unwrap() + .contains("Incompatible")); + let reg = registry(); + match plan("neg_shape", ®).1 { + Err(Refusal::When { + node, + classid, + truth, + .. + }) => { + assert_eq!( + (node.as_str(), classid, truth), + ("n_add", 0x0F20, K3::False) + ) + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// `Erf` has no registration: UNKNOWN, refused by signature. +#[test] +fn unregistered_op_is_refused() { + let reg = registry(); + match plan("neg_unsupported", ®).1 { + Err(Refusal::Unregistered { node, signature }) => { + assert_eq!(node, "n_abs"); + assert!( + signature.starts_with("ai.onnx:Erf(Some(F32))"), + "{signature}" + ); + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// `com.example:Abs`: Tract imports it as the default-domain `Abs`; the source +/// domain keys the signature, so it is unregistered. +#[test] +fn foreign_domain_is_unregistered_although_tract_accepts_it() { + assert!(ort()["variants"]["neg_domain"]["ort_error"] + .as_str() + .unwrap() + .contains("com.example:Abs")); + let reg = registry(); + match plan("neg_domain", ®).1 { + Err(Refusal::Unregistered { node, signature }) => { + assert_eq!(node, "n_abs"); + assert!(signature.starts_with("com.example:Abs"), "{signature}"); + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// A run-time fact the facts cannot show: a sum past int32 is refused, not wrapped. +#[test] +fn int32_overflow_is_refused_at_run_time() { + let reg = registry(); + let (fx, p) = plan("base", ®); + let mut ins = inputs(None); + ins[0].1 = Value::I32(ArrayD::from_shape_vec(IxDyn(&[3]), vec![i32::MAX, 1, 1]).unwrap()); + match p.unwrap().run(&fx.store, &ins) { + Err(Refusal::Native(lance_graph_onnx::Refusal::OutOfDtypeRange(v))) => { + assert_eq!(v, i32::MAX as i64 + 2) + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// The source keeps both `T = 0` and `Z = 0` as Property positions, although +/// Tract's inference model merges equal constants into one node. +#[test] +fn equal_initializers_stay_distinct_positions() { + let (_, g) = graph("base"); + assert!(g.props.iter().any(|p| p == "T") && g.props.iter().any(|p| p == "Z")); + let onnx = tract_onnx::onnx(); + let b = bytes("base"); + let names: Vec = onnx + .model_for_proto_model(&onnx.proto_model_for_read(&mut &*b).unwrap()) + .unwrap() + .nodes() + .iter() + .map(|n| n.name.clone()) + .collect(); + assert!( + names.contains(&"T".into()) && !names.contains(&"Z".into()), + "{names:?}" + ); +} + +/// The node list reversed in the protobuf (no longer topological): the +/// schedule comes from `dep`, not from source order, so the graph still runs +/// and matches ORT. +#[test] +fn schedule_comes_from_dep_not_source_order() { + use prost::Message; + let mut m = tract_onnx::pb::ModelProto::decode(&*bytes("base")).unwrap(); + m.graph.as_mut().unwrap().node.reverse(); + let b = m.encode_to_vec(); + let src = SourceInventory::from_bytes(&b).unwrap(); + assert_eq!( + src.nodes[0].name, "n_div", + "anti-vacuity: the source order is reversed" + ); + let facts = tract_facts(&bytes("base"), &[]).unwrap(); + let g = OnnxGraph::build(&src, &facts).unwrap(); + let reg = registry(); + let store = FrameStore::new(revisions(&src, ®, &g)); + let p = resolve(&src, g, ®, &store).unwrap(); + assert_eq!( + outputs(&p.run(&store, &inputs(None)).unwrap()), + ort_out("base") + ); +} + +/// Tract's shapes removed: `Greater` still resolves (its threshold `T` takes +/// its shape from the source initializer), `Where` has no shapes left, its +/// WHEN is UNKNOWN, and UNKNOWN refuses: it never counts as TRUE. +#[test] +fn unknown_shapes_refuse() { + let src = SourceInventory::from_bytes(&bytes("base")).unwrap(); + let mut facts = tract_facts(&bytes("base"), &[]).unwrap(); + for f in facts.values_mut() { + f.shape = None; + } + let g = OnnxGraph::build(&src, &facts).unwrap(); + let reg = registry(); + let store = FrameStore::new(revisions(&src, ®, &g)); + match resolve(&src, g, ®, &store) { + Err(Refusal::When { node, truth, .. }) => { + assert_eq!((node.as_str(), truth), ("n_where", K3::Unknown)) + } + other => panic!("{:?}", other.map(|_| ())), + } +} diff --git a/crates/tract-harvest-probe/tools/fetch_jina_slice.py b/crates/tract-harvest-probe/tools/fetch_jina_slice.py new file mode 100644 index 000000000..eb1e1c3bc --- /dev/null +++ b/crates/tract-harvest-probe/tools/fetch_jina_slice.py @@ -0,0 +1,164 @@ +#!/usr/bin/env python3 +"""D-TRACT-HARVEST-0 fixture B: one contiguous subgraph of the published Jina v5 +small retrieval ONNX export, with ONNX Runtime outputs as the oracle. + +The model weights are CC-BY-NC-4.0, so nothing here is committed except the +manifest of digests (fixtures/jina_manifest.json). Every fetched byte range is +SHA-256 pinned; a mismatch aborts. Fetched data goes to fixtures/jina/ +(gitignored). Run from the crate root: + + python3 tools/fetch_jina_slice.py + +The slice is layer 0's input RMSNorm, exactly as exported: + /embed_tokens/Gather -> /layers.0/input_layernorm/{Cast, Pow, ReduceMean, + Add, Sqrt, Div, Mul, Cast_1, Mul_1} +The Gather is resolved by fetching only the embedding rows of the pinned token +ids (one 4096-byte HTTP range each); the rest of the slice is extracted node for +node, with its Constants, and the one weight it reads inlined. +""" +import hashlib, json, os, sys, urllib.request +import numpy as np +import onnx +from onnx import helper as h, numpy_helper as nh, TensorProto as T +import onnxruntime as ort +from tokenizers import Tokenizer + +REPO = "jinaai/jina-embeddings-v5-text-small-retrieval" +REV = "6856e76bb72982e58de0620458a4e8b3614da340" +PINS = { + "onnx/model.onnx": "463fb3fec2adf9c47a946626dc2a0884e43091f95466083fb1754950000a7c1a", + "tokenizer.json": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4", +} +TEXT = "Harvest the algebra. Keep the native substrate." +HERE = os.path.join(os.path.dirname(__file__), "..", "fixtures") +OUT = os.path.join(HERE, "jina") +SLICE_NODES = ["/layers.0/input_layernorm/" + n for n in + ["Cast", "Constant", "Pow", "Constant_1", "ReduceMean", "Constant_2", "Add", + "Sqrt", "Constant_3", "Div", "Mul", "Cast_1", "Mul_1"]] +SLICE_IN = "/embed_tokens/Gather_output_0" +SLICE_MID = "/layers.0/input_layernorm/ReduceMean_output_0" +SLICE_OUT = "/layers.0/input_layernorm/Mul_1_output_0" + + +def url(path): + return f"https://huggingface.co/{REPO}/resolve/{REV}/{path}" + + +def fetch(path, rng=None): + req = urllib.request.Request(url(path)) + if rng: + req.add_header("Range", f"bytes={rng[0]}-{rng[0] + rng[1] - 1}") + with urllib.request.urlopen(req) as r: + data = r.read() + if rng and len(data) != rng[1]: + sys.exit(f"range {path} {rng}: got {len(data)} bytes") + return data + + +def pinned(path): + local = os.path.join(OUT, path.replace("/", "_")) + if not os.path.exists(local): + with open(local, "wb") as f: + f.write(fetch(path)) + data = open(local, "rb").read() + got = hashlib.sha256(data).hexdigest() + if got != PINS[path]: + sys.exit(f"{path}: sha256 {got} != pinned {PINS[path]}") + return local, data + + +def ext(t): + d = {e.key: e.value for e in t.external_data} + return d["location"], int(d["offset"]), int(d["length"]) + + +def main(): + os.makedirs(OUT, exist_ok=True) + model_path, model_bytes = pinned("onnx/model.onnx") + tok_path, _ = pinned("tokenizer.json") + enc = Tokenizer.from_file(tok_path).encode(TEXT, add_special_tokens=False) + ids = list(enc.ids) + + m = onnx.load(model_path, load_external_data=False) + g = m.graph + inits = {t.name: t for t in g.initializer} + emb, w = inits["embed_tokens.weight"], inits["layers.0.input_layernorm.weight"] + hidden = emb.dims[1] + row_bytes = hidden * 4 + loc, base, length = ext(emb) + assert emb.data_type == T.FLOAT and length == emb.dims[0] * row_bytes + + ranges = [] + rows = [] + for tid in ids: + r = (base + tid * row_bytes, row_bytes) + data = fetch("onnx/" + loc, r) + ranges.append({"token_id": tid, "offset": r[0], "length": r[1], + "sha256": hashlib.sha256(data).hexdigest()}) + rows.append(np.frombuffer(data, "