diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 818c47cda..d376dc57c 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -57,7 +57,7 @@ | **D-ONNX-INTAKE-4** | S4 numeric/quantization parity for real model blockers | Queued (conditional; not authorized) | IntegerExact / bounded FP / SourceParity / CertifiedMath kept distinct | | **D-ONNX-INTAKE-5** | S5 whole-model forward pass and cost closure | Goal, not authorized | outputs vs ORT under declared tolerance; every cost stage reported | | **D-TRACT-HARVEST-0** | ONNX graph as a DR#3 graph (nodes Identity, values Property, dep = do_ ∘ read_by) resolved through a DT64 ON×WHEN×DO registry (op signature → classid → native DO); Tract cold facts; ORT (optimizer off) oracle; `crates/tract-harvest-probe` | Lab probe implemented (this PR, redesigned to the graph); plan [d-tract-harvest-0-native-algebra-bridge-v1.md](../plans/d-tract-harvest-0-native-algebra-bridge-v1.md) §0 | 26 tests, 13 disable runs red; full model 4,146/7,620 nodes WHEN-true with inference-stage facts (§0.1), 0 refused for missing facts, >64 positions untiled, first unregistered groups Unsqueeze/Shape/Gather/Concat/MatMul | -| **D-TRACT-SHAPE-1** | Cold shape-subgraph folding (Const/Shape/Symbolic/Data classes; fold through the node's own WHEN/DO; stale-input refusal), shape-op registrations (Shape, Gather i64, Unsqueeze, Concat, Reshape, Cast i64, Erf via `vserf`, scalar-Y Where via `blend_i32_scalar`), producer-checked Tract fact admission | **In PR** (lab probe); plan [d-tract-harvest-0-native-algebra-bridge-v1.md](../plans/d-tract-harvest-0-native-algebra-bridge-v1.md) §0.2 | 40 tests, 9 disable runs red; Jina [1,10] coverage 4,146/0/3,474 → 6,404/56/1,160, 4,440 nodes fold in 7.3 ms; remaining blocker int64 shape arithmetic (owner decision) | +| **D-TRACT-SHAPE-1** | Cold shape-subgraph folding (Const/Shape/Symbolic/Data classes; fold through the node's own WHEN/DO; stale-input refusal), shape-op registrations (Shape, Gather i64, Unsqueeze, Concat, Reshape, Cast i64, Erf via `vserf`, scalar-Y Where via `blend_i32_scalar`), producer-checked Tract fact admission | **Shipped** #1853 + follow-up In PR (lab probe); plan [d-tract-harvest-0-native-algebra-bridge-v1.md](../plans/d-tract-harvest-0-native-algebra-bridge-v1.md) §0.2–§0.3 | #1853 merged (40 tests, 9 disable runs red; coverage 4,146/0/3,474 → 6,404/56/1,160). Follow-up **In PR**: registry v3 (int64 compares/Where, ConstantOfShape, Expand), optional-output producer test; coverage 6,581/114/925; int64 Mul/Div/Add/Sub/Range refused pending the ndarray ask (plan §0.3) | ## D-TFLP — TFLite models on the native polyfill: int8 codes as Palette256 (2026-10-25) diff --git a/.claude/board/entries/2026-10-30-d-tract-shape-1-cold-shape-folding.md b/.claude/board/entries/2026-10-30-d-tract-shape-1-cold-shape-folding.md index ec069923f..bb0650b21 100644 --- a/.claude/board/entries/2026-10-30-d-tract-shape-1-cold-shape-folding.md +++ b/.claude/board/entries/2026-10-30-d-tract-shape-1-cold-shape-folding.md @@ -11,3 +11,7 @@ A Tract fact is admitted only from the value's source-producer outlet (a matchin Fixtures (ORT 1.29.0 `ORT_DISABLE_ALL`): shape chain bit-exact; Erf ≤ 2 ULP vs ORT and f64 over 4,115 points. 40 tests; 9 disable runs red. **OPEN:** int64 shape-arithmetic semantics (owner, then ndarray ask); axis as DO parameter vs classid per axis; `vserf` ~4x slower than Tract's erf; ConstantOfShape/Expand via ndarray core not yet registered; multi-tile and MatMul gates unchanged. + +**Follow-up (after #400 / #1853):** registry v3 adds int64 Equal/LessOrEqual (#400 compares), int64 Where (`blend_i64[_scalar]`), ConstantOfShape (`from_elem`), Expand (`broadcast`). Optional-output fixture (`LSTM ["", "Y_h"]`) pins that `source_producers` follows Tract's compacted outlet numbering and admits `Y_h`'s fact, not the omitted `Y`'s. Jina coverage 6,404/56/1,160 → 6,581/114/925. Remaining int64 arithmetic (Mul/Div/Add/Sub/Range) is refused; exact checked signatures and the `i64_arith` fixture are the ndarray ask (plan §0.3). + +**Hardening + census:** explicit ONNX-position → Tract-slot mapping (`tract_slot`), source-bound facts (`merged_for`), collision/stale/tail-bit/ConstantOfShape/Expand tests (55 green, 5 more disable runs red). Jina census: 6,581/114/925, 4,499 folded; the 56 blocked Reshapes all wait on int64 `Mul`; not folded = 2,461 runtime + 660 shape-only blocked. diff --git a/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md b/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md index 40bb72385..16af5cc2d 100644 --- a/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md +++ b/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md @@ -115,6 +115,42 @@ Tests (40, all green): `shape_chain_folds_and_the_data_nodes_run_against_ort`, ` - `ConstantOfShape` and `Expand` map onto ndarray `from_elem` / `broadcast` and are not registered yet. - Multi-tile orchestration and full Jina MatMul execution remain separate gates. +### 0.3 After #400 / #1853 (2026-10-30) + +Owner decisions: int64 shape arithmetic GO under checked, fail-closed admission (Div truncates toward zero; a zero divisor or an overflow refuses); an axis stays a guarded parameter of one method unless a different axis changes the behaviour law; ConstantOfShape and Expand registered on existing ndarray capabilities; Erf numerically accepted as Bounded, its performance open. + +**Registry v3.** int64 `Equal` 0x0F70 and `LessOrEqual` 0x0F71 (ndarray #400 `eq/le_i64_pair_to_mask`, scalar right side `eq/le_i64_to_mask`; a same-shape or one-element right side only, other broadcasts not admitted), int64 `Where` 0x0F72 (`blend_i64` / `blend_i64_scalar`), `ConstantOfShape` 0x0F73 (`ArrayD::from_elem` of the value attribute, constant non-negative shape), `Expand` int64/f32 0x0F74/0x0F75 (`ndarray` `broadcast` to the multidirectional broadcast). Fixture `shape_i64`: every int64 node over `Shape(F)` folds, `Expand(F)` runs, every output equals ORT. + +**Optional output slots.** Tract numbers a node's outlets after dropping empty output names (`onnx/src/model.rs:150`). Fixtures `lstm_skip` (`LSTM` outputs `["", "Y_h"]`) and `lstm_full` pin that `source_producers` names slot 0 resp. 1 for `Y_h` and that the admitted fact is `Y_h`'s (ORT `[1,1,2]`), not the omitted `Y`'s (`[5,1,1,2]`). Disable run (count empty slots): red. + +**Coverage, Jina pinned `[1,10]`:** 6,404 / 56 / 1,160 → **6,581 / 114 / 925**; 4,499 nodes fold. The 114 not-true: Reshape 56 and Expand 57 UNKNOWN (target computed by unregistered int64 arithmetic), LessOrEqual 1 not admitted (a `[S,1]` vs `[1,S]` broadcast). Unregistered shape-class nodes left: Mul 115, Div 56, Range 4, Add 1, plus Cast to bool/float 2, And, Cos, Sin, MatMul, Transpose 1 each. + +**ndarray ask (D-TRACT-SHAPE-1, after #400).** #400 already supplies the int64 compares (`eq/lt/le/gt/ge_i64[_pair]_to_mask`) and `blend_i64[_scalar]`; they are registered. What is missing, by exact signature (fail-closed: `Err` and an unchanged `dst` content contract is not required, but no partial result may be returned as success): + +```rust +pub enum IntArithError { Overflow { index: usize }, DivideByZero { index: usize }, ZeroStep } +pub fn add_i64_checked(a: &[i64], b: &[i64], dst: &mut [i64]) -> Result<(), IntArithError>; +pub fn sub_i64_checked(a: &[i64], b: &[i64], dst: &mut [i64]) -> Result<(), IntArithError>; +pub fn mul_i64_checked(a: &[i64], b: &[i64], dst: &mut [i64]) -> Result<(), IntArithError>; +/// Truncates toward zero; `i64::MIN / -1` is Overflow. +pub fn div_i64_trunc_checked(a: &[i64], b: &[i64], dst: &mut [i64]) -> Result<(), IntArithError>; +// one-element right side, as blend_i64_scalar +pub fn add_i64_scalar_checked(a: &[i64], b: i64, dst: &mut [i64]) -> Result<(), IntArithError>; +pub fn sub_i64_scalar_checked(a: &[i64], b: i64, dst: &mut [i64]) -> Result<(), IntArithError>; +pub fn mul_i64_scalar_checked(a: &[i64], b: i64, dst: &mut [i64]) -> Result<(), IntArithError>; +pub fn div_i64_trunc_scalar_checked(a: &[i64], b: i64, dst: &mut [i64]) -> Result<(), IntArithError>; +/// ONNX Range: max(ceil((limit - start) / delta), 0) elements; delta 0 is ZeroStep. +pub fn range_i64_checked(start: i64, limit: i64, delta: i64) -> Result, IntArithError>; +``` + +Accuracy: exact (integers). Panics only on a length mismatch, as the #400 kernels. Fixtures: `crates/tract-harvest-probe/fixtures/tiny/i64_arith.onnx` with ORT `ORT_DISABLE_ALL` outputs in `ort.json` (`Div([-7, 7], [2, 3]) = [-3, 2]`, `Mul([2, 3], [3, -4]) = [6, -12]`, `Add([2, 3], [-5, 10]) = [-3, 13]`, `Sub = [7, -7]`, `Range(0, 3, 1) = [0, 1, 2]`); refusal cases ORT does not cover: `add(i64::MAX, 1)`, `mul(i64::MIN, -1)`, `div(i64::MIN, -1)`, `div(x, 0)`, `range(0, 5, 0)` each `Err`; `div(-7, 2) = -3`, `div(7, -2) = -3`, `range(5, 0, -2) = [5, 3, 1]`, `range(0, 0, 1) = []`. Test `int64_arithmetic_is_refused_until_checked_words_exist` pins the current refusal. + +Disable runs (each red): Equal kernel replaced by `le`; slot counted over empty outputs; general broadcast admitted; broadcast mismatch accepted; ConstantOfShape ignoring its value. 46 tests green. + +**Hardening (same PR).** `source_producers` enumerates each node's ONNX output positions first and maps a position to Tract's slot with `dr3::tract_slot` (named outputs before it). `FactSources` records the source it was read from; `merged_for` refuses facts of another model. Tests: `["", "Y"]` and multi-output mapping; producer collisions (two producers, a value that is also an input or initializer, duplicate node names, unnamed node without a first output); a fact from a renamed producer; facts of another model; int64 compares and Where over 70 elements (tail bits clear, scalar and pair); ConstantOfShape (default float 0, empty shape gives a scalar, zero dims, bool fill, 1-D shape input, K3 refusals) and Expand (multidirectional broadcast). Disable runs (each red): ONNX position used as the slot; no binding check; collisions keep a producer; bool fill tail bits; ConstantOfShape rank. 55 tests green. + +**Census (Jina, `[1,10]`, `examples/shape_census.rs`).** WHEN-true / not-true / unregistered **6,581 / 114 / 925** (registry v1 4,146 / 0 / 3,474; v2 6,404 / 56 / 1,160). Cold-folded nodes 4,499 (v2 4,440, +59 ConstantOfShape; v1 plan had no folding). **The 56 Reshapes blocked in v2 still do not resolve: the first unregistered op their target depends on is int64 `Mul` in all 56.** Not folded: runtime 2,461 (Mul 450, Add 253, MatMul 252, Cast 228, Reshape 170, Slice 140, Transpose 140, Div/Pow/ReduceMean/Sqrt 113 each, …) and shape-only blocked 660 (Unsqueeze 181, Cast 119, Mul 117, Equal 58, Where 58, Concat 57, Div 56, Range 4, …). The integration acceptance (a whole shape-arithmetic chain folding or executing through admitted methods and matching ORT) waits on the checked int64 words; `i64_arith` is its fixture. No full Jina runtime is started. + ## 1. The acceptance chain, demonstrated The chain under test: a source-attested ONNX subgraph is imported by Tract, admitted, lowered onto existing native methods, executed without Tract, and verified against ONNX Runtime (ORT). The chain holds for both fixtures: fixture A (tiny, synthetic) is bit-exact, and fixture B (a real model slice) is bounded. The numbers are in §4. Nothing here is certified, production or merged. diff --git a/crates/tract-harvest-probe/examples/shape_census.rs b/crates/tract-harvest-probe/examples/shape_census.rs index 5428a715a..493d37762 100644 --- a/crates/tract-harvest-probe/examples/shape_census.rs +++ b/crates/tract-harvest-probe/examples/shape_census.rs @@ -98,8 +98,107 @@ fn main() { for op in OPS { println!(" {op:>10} {:?}", cov.get(*op)); } + let not_true: Vec<_> = cov + .iter() + .filter(|(_, c)| c[1] > 0) + .map(|(k, c)| (k.as_str(), c[1])) + .collect(); + println!(" WHEN not true by op: {not_true:?}"); println!("foldable by dependency but not folded:"); for (why, c) in by { println!(" {c:>5} {why}"); } + + // The Reshapes whose WHEN is not TRUE: which unregistered ops their + // target depends on (the first unregistered or unfolded-unregistered + // ancestors along the target input). + let producer: std::collections::HashMap<&str, usize> = src + .nodes + .iter() + .enumerate() + .flat_map(|(i, n)| n.outputs.iter().map(move |o| (o.as_str(), i))) + .collect(); + let mut consts: std::collections::HashMap<&str, &tract_harvest_probe::value::Value> = src + .initializers + .iter() + .filter_map(|i| i.value.as_ref().map(|v| (i.name.as_str(), v))) + .collect(); + for (k, v) in &f.values { + consts.insert(k.as_str(), v); + } + let mut blocked_reshape = 0; + let mut roots = std::collections::BTreeMap::::new(); + for n in src.nodes.iter().filter(|n| n.op_type == "Reshape") { + let args: Vec<_> = n.inputs.iter().map(|i| facts.get(i)).collect(); + let Some(r) = reg.lookup(n, &args) else { + continue; + }; + let c = n + .inputs + .iter() + .map(|i| consts.get(i.as_str()).copied()) + .collect(); + if (r.when)(®istry::NodeView { + node: n, + args, + consts: c, + }) == registry::K3::True + { + continue; + } + blocked_reshape += 1; + let mut stack = vec![n.inputs[1].as_str()]; + let mut seen = std::collections::HashSet::new(); + let mut found = std::collections::BTreeSet::new(); + while let Some(v) = stack.pop() { + if consts.contains_key(v) || !seen.insert(v) { + continue; + } + let Some(&p) = producer.get(v) else { continue }; + let pn = &src.nodes[p]; + let pargs: Vec<_> = pn.inputs.iter().map(|i| facts.get(i)).collect(); + if reg.lookup(pn, &pargs).is_none() { + found.insert(format!( + "{}({:?})", + pn.op_type, + registry::signature_dtype(pn, &pargs, Some(0)).unwrap() + )); + } else { + stack.extend(pn.inputs.iter().map(String::as_str)); + } + } + *roots + .entry(found.into_iter().collect::>().join(" + ")) + .or_default() += 1; + } + println!("Reshape with WHEN not TRUE: {blocked_reshape}; unregistered ops their target depends on: {roots:?}"); + // Not folded: runtime (reads an input value) vs shape-only (blocked). + let mut runtime = std::collections::BTreeMap::<&str, usize>::new(); + let mut shape_only = std::collections::BTreeMap::<&str, usize>::new(); + for (i, n) in src.nodes.iter().enumerate() { + if f.nodes.contains(&i) { + continue; + } + let m = if node_dep[i] >= fold::Dep::Symbolic { + &mut runtime + } else { + &mut shape_only + }; + *m.entry(n.op_type.as_str()).or_default() += 1; + } + fn top<'a>(m: &std::collections::BTreeMap<&'a str, usize>) -> Vec<(&'a str, usize)> { + let mut v: Vec<_> = m.iter().map(|(k, c)| (*k, *c)).collect(); + v.sort_by(|a, b| b.1.cmp(&a.1)); + v + } + println!( + "not folded: runtime {} nodes {:?}", + runtime.values().sum::(), + top(&runtime) + ); + println!( + "not folded: shape-only (blocked) {} nodes {:?}", + shape_only.values().sum::(), + top(&shape_only) + ); } diff --git a/crates/tract-harvest-probe/fixtures/tiny/i64_arith.onnx b/crates/tract-harvest-probe/fixtures/tiny/i64_arith.onnx new file mode 100644 index 000000000..a5eebe30b Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/i64_arith.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/lstm_full.onnx b/crates/tract-harvest-probe/fixtures/tiny/lstm_full.onnx new file mode 100644 index 000000000..555b87004 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/lstm_full.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/lstm_skip.onnx b/crates/tract-harvest-probe/fixtures/tiny/lstm_skip.onnx new file mode 100644 index 000000000..9a601dca6 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/lstm_skip.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/ort.json b/crates/tract-harvest-probe/fixtures/tiny/ort.json index 9d5472beb..d6846b05f 100644 --- a/crates/tract-harvest-probe/fixtures/tiny/ort.json +++ b/crates/tract-harvest-probe/fixtures/tiny/ort.json @@ -138,6 +138,86 @@ "neg_concat": { "sha256": "f92f5872d18536685cf38fa0a627ceb7fdd7fda6bf908113d1127358c647aed2", "ort_error": "Fail: [ONNXRuntimeError] : 1 : FAIL : Node (n_concat) Op (Concat) [ShapeInferenceError] Can't merge shape info. Both inferred and declared dimension have values but they differ. Inferred=2 Declared=3 Dimens" + }, + "shape_i64": { + "sha256": "19d1dd7da1b55a36b977d582f8a4b3f4a26b6ffc5bc571027701c9f7352a230b", + "ort": { + "W": { + "shape": [2], + "dtype": "int64", + "bits": [2,7] + }, + "LE": { + "shape": [2], + "dtype": "bool", + "bits": [1,0] + }, + "CS": { + "shape": [2,3], + "dtype": "int64", + "bits": [5,5,5,5,5,5] + }, + "ER": { + "shape": [2,3], + "dtype": "int64", + "bits": [1,2,3,1,2,3] + }, + "EF": { + "shape": [2,2,3], + "dtype": "float32", + "bits": [1069547520,3196059648,1077936128,1036831949,1088421888,3223322624,1069547520,3196059648,1077936128,1036831949,1088421888,3223322624] + } + } + }, + "lstm_skip": { + "sha256": "e979463f372fd8bfa4cb2ca46a06c4459f5866786f2b593933e9f34d8f5819f4", + "ort": { + "Y_h": { + "shape": [1,1,2], + "dtype": "float32", + "bits": [1049333655,1049333655] + } + } + }, + "lstm_full": { + "sha256": "3d5456afad55b4f5fb01ba3c81ed6f7123d72e028f22e6ed5ad00c6a85286ce1", + "ort": { + "Y_h": { + "shape": [1,1,2], + "dtype": "float32", + "bits": [1049333655,1049333655] + } + } + }, + "i64_arith": { + "sha256": "042a71f755ea7ef7ffe16ef1a3dd57bcdadbc3ee5f4b752d5dbc1de9543399db", + "ort": { + "DV": { + "shape": [2], + "dtype": "int64", + "bits": [-3,2] + }, + "ML": { + "shape": [2], + "dtype": "int64", + "bits": [6,-12] + }, + "AD": { + "shape": [2], + "dtype": "int64", + "bits": [-3,13] + }, + "SB": { + "shape": [2], + "dtype": "int64", + "bits": [7,-7] + }, + "RG": { + "shape": [3], + "dtype": "int64", + "bits": [0,1,2] + } + } } }, "erf_sweep": { diff --git a/crates/tract-harvest-probe/fixtures/tiny/shape_i64.onnx b/crates/tract-harvest-probe/fixtures/tiny/shape_i64.onnx new file mode 100644 index 000000000..b0e5a5ce7 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/shape_i64.onnx differ diff --git a/crates/tract-harvest-probe/src/dr3.rs b/crates/tract-harvest-probe/src/dr3.rs index 74b2955f0..bfc826d8b 100644 --- a/crates/tract-harvest-probe/src/dr3.rs +++ b/crates/tract-harvest-probe/src/dr3.rs @@ -295,6 +295,8 @@ pub struct FactSources { /// dimension, or more than one symbol through `Flatten`). pub typed_error: Option, pub rejected: Vec, + /// The `model_sha256` of the source these facts were read from. + pub source_sha256: String, } impl FactSources { @@ -315,6 +317,18 @@ impl FactSources { out } + /// [`merged`](Self::merged), but only for the source the facts were read + /// from: facts of another model (a stale binding) refuse. + pub fn merged_for(&self, src: &SourceInventory) -> Result, Refusal> { + if self.source_sha256 != src.model_sha256 { + return Err(Refusal::Frontend(format!( + "facts read from model {} applied to model {}", + self.source_sha256, src.model_sha256 + ))); + } + Ok(self.merged()) + } + /// Values both stages describe, where they disagree on dtype or shape. pub fn disagreements(&self) -> Vec<(String, Fact, Fact)> { let mut out: Vec<_> = self @@ -382,7 +396,11 @@ pub fn source_producers(src: &SourceInventory) -> HashMap for n in &src.nodes { let name = node_name(n); let unique = name.as_ref().is_some_and(|m| by_name[m] == 1); - for (slot, o) in n.outputs.iter().filter(|o| !o.is_empty()).enumerate() { + for (position, o) in n.outputs.iter().enumerate() { + if o.is_empty() { + continue; // an omitted optional output names no value + } + let slot = tract_slot(&n.outputs, position); put( o, name.clone() @@ -394,6 +412,15 @@ pub fn source_producers(src: &SourceInventory) -> HashMap out } +/// The Tract outlet slot of ONNX output `position`: Tract numbers a node's +/// outlets after dropping empty output names (`onnx/src/model.rs:150`, +/// `output.iter().filter(|s| !s.is_empty()).enumerate()`), so the slot is the +/// number of named outputs before `position`. The ONNX position itself is +/// not the slot whenever an omitted optional output precedes it. +pub fn tract_slot(outputs: &[String], position: usize) -> usize { + outputs[..position].iter().filter(|o| !o.is_empty()).count() +} + /// Admit one stage's facts: a fact for source value `v` is kept only if /// exactly one outlet of the stage carries `v`'s label and that outlet is /// `v`'s producer (see [`source_producers`]). Labels that name no source @@ -504,7 +531,10 @@ fn fact_sources_of( .map_err(fe)?; } let producers = source_producers(src); - let mut out = FactSources::default(); + let mut out = FactSources { + source_sha256: src.model_sha256.clone(), + ..FactSources::default() + }; // Inference stage: analyse without stopping at the first node it cannot // type, then read whatever each labelled outlet determined. An unlabelled @@ -666,6 +696,120 @@ mod tests { assert_eq!(p["v6"], None); } + fn inv( + nodes: Vec, + inputs: &[&str], + inits: &[&str], + ) -> SourceInventory { + SourceInventory { + nodes, + inputs: inputs.iter().map(|s| s.to_string()).collect(), + model_sha256: String::new(), + ir_version: 8, + opsets: vec![], + initializers: inits + .iter() + .map(|n| crate::source::SourceInitializer { + name: n.to_string(), + data_type: 1, + dims: vec![], + placement: crate::source::Placement::Inline { + sha256: String::new(), + }, + value: None, + }) + .collect(), + outputs: vec![], + } + } + + fn node(name: &str, outs: &[&str]) -> crate::source::SourceNode { + crate::source::SourceNode { + name: name.into(), + op_type: "Split".into(), + domain: String::new(), + inputs: vec!["x".into()], + outputs: outs.iter().map(|s| s.to_string()).collect(), + attributes: vec![], + attrs: vec![], + } + } + + /// The ONNX position of an output is enumerated first and then mapped to + /// Tract's slot, which counts only the named outputs before it. + #[test] + fn tract_slot_counts_named_outputs_before_the_position() { + let o = |v: &[&str]| v.iter().map(|s| s.to_string()).collect::>(); + assert_eq!(tract_slot(&o(&["", "Y"]), 1), 0); + assert_eq!(tract_slot(&o(&["A", "", "B", "C"]), 2), 1); + assert_eq!(tract_slot(&o(&["A", "", "B", "C"]), 3), 2); + assert_eq!(tract_slot(&o(&["A", "B"]), 1), 1); + } + + /// `["", "Y"]`: Y is slot 0. Several outputs with a gap: slots 0, 1, 2. + #[test] + fn producers_with_omitted_and_multiple_outputs() { + let p = source_producers(&inv( + vec![node("a", &["", "Y"]), node("b", &["P", "", "Q", "R"])], + &["x"], + &[], + )); + assert_eq!(p["Y"], Some(outlet("a", 0))); + assert_eq!( + (p["P"].clone(), p["Q"].clone(), p["R"].clone()), + ( + Some(outlet("b", 0)), + Some(outlet("b", 1)), + Some(outlet("b", 2)) + ) + ); + assert!(!p.contains_key(""), "an omitted output names no value"); + } + + /// Two producers for one value, a value that is also a graph input or an + /// initializer, two nodes with one name, and an unnamed node without a + /// first output: no producer, so no fact is admitted for those values. + #[test] + fn producer_collisions_have_no_producer() { + let p = source_producers(&inv( + vec![ + node("a", &["v"]), + node("b", &["v"]), + node("c", &["x"]), + node("d", &["w"]), + node("dup", &["u1"]), + node("dup", &["u2"]), + node("", &["", "z"]), + node("e", &["k"]), + ], + &["x"], + &["k"], + )); + for v in ["v", "x", "u1", "u2", "z", "k"] { + assert_eq!(p[v], None, "{v}"); + } + assert_eq!( + p["w"], + Some(outlet("d", 0)), + "a unique producer still counts" + ); + } + + /// A fact bound to a producer of an older source: the same label on an + /// outlet the current source no longer has is rejected. + #[test] + fn a_fact_from_a_renamed_producer_is_rejected() { + let now = source_producers(&inv(vec![node("n_new", &["v"])], &["x"], &[])); + let raw = vec![("v".to_string(), outlet("n_old", 0), fact(DType::F32, &[2]))]; + let (ok, rej) = admit("t", raw, &now); + assert!(ok.is_empty()); + assert!( + rej[0].reason.contains("is not the source producer"), + "{}", + rej[0].reason + ); + } + /// `disagreements` reports a dtype or shape conflict and stays silent on /// agreement or on a value only one stage describes; `merged` prefers the /// inference fact and fills only what it lacks from the typed one. diff --git a/crates/tract-harvest-probe/src/registry.rs b/crates/tract-harvest-probe/src/registry.rs index 58d2f79f0..570405635 100644 --- a/crates/tract-harvest-probe/src/registry.rs +++ b/crates/tract-harvest-probe/src/registry.rs @@ -591,6 +591,86 @@ fn guard_reshape(v: &NodeView) -> K3 { } } +/// A binary int64 comparison as admitted here: the right side has the left +/// side's shape (pairwise) or one element (scalar). Other broadcasts are not +/// admitted. +fn guard_pair_or_scalar(v: &NodeView) -> K3 { + let (Some(a), Some(b)) = (arg_shape(v, 0), arg_shape(v, 1)) else { + return K3::Unknown; + }; + if a == b && a.iter().all(|d| matches!(d, Dim::Known(_))) { + return K3::True; + } + match known(&Some(b.clone())) { + Some(s) if s.iter().product::() == 1 && b.len() <= a.len() => K3::True, + Some(_) if known(&Some(a.clone())).is_some() => K3::False, + _ => K3::Unknown, + } +} + +/// `ConstantOfShape`: a constant, non-negative shape and a one-element value. +fn guard_constant_of_shape(v: &NodeView) -> K3 { + match v.node.attr("value") { + None => {} + Some(Attr::Tensor(Some(t))) if t.shape().iter().product::() == 1 => {} + Some(_) => return K3::False, + } + match arg_shape(v, 0).map(Vec::len) { + Some(1) => {} + Some(_) => return K3::False, + None => return K3::Unknown, + } + match v.consts.first().copied().flatten().and_then(Value::i64s) { + Some(d) if d.iter().all(|&x| x >= 0) => K3::True, + Some(_) => K3::False, + None => K3::Unknown, + } +} + +/// The multidirectional broadcast of two shapes, or `None`. +fn broadcast_shapes(a: &[usize], b: &[usize]) -> Option> { + let r = a.len().max(b.len()); + let at = |s: &[usize], i: usize| { + if i + s.len() >= r { + s[i + s.len() - r] + } else { + 1 + } + }; + (0..r) + .map(|i| match (at(a, i), at(b, i)) { + (x, y) if x == y || y == 1 => Some(x), + (1, y) => Some(y), + _ => None, + }) + .collect() +} + +/// `Expand`: a constant target over a fully known input shape that +/// broadcasts against it. +fn guard_expand(v: &NodeView) -> K3 { + let Some(target) = v.consts.get(1).copied().flatten().and_then(Value::i64s) else { + return K3::Unknown; + }; + let Some(input) = v + .args + .first() + .copied() + .flatten() + .and_then(|f| known(&f.shape)) + else { + return K3::Unknown; + }; + if target.iter().any(|&t| t < 0) { + return K3::False; + } + let t: Vec = target.iter().map(|&t| t as usize).collect(); + match broadcast_shapes(&input, &t) { + Some(_) => K3::True, + None => K3::False, + } +} + // ── DO polyfills: existing native functions only ─────────────────────────── fn f32_arg<'a>( @@ -946,6 +1026,152 @@ fn do_erf(a: &[&Value], n: &SourceNode) -> Result, Refusal> { Ok(vec![Value::F32(out)]) } +/// An int64 comparison into a packed `tensor(bool)`: the `_pair` kernel when +/// the right side has the left side's length, the scalar kernel for a +/// one-element right side. +fn compare_i64( + a: &[&Value], + n: &SourceNode, + pair: fn(&[i64], &[i64], &mut [u64]), + scalar: fn(&[i64], i64, &mut [u64]), +) -> Result, Refusal> { + let x = i64_arg(a, 0, n)?; + let y = i64_arg(a, 1, n)?; + let xs: Vec = x.iter().copied().collect(); + let mut words = vec![0u64; lance_graph_onnx::mask_words(xs.len())]; + if y.shape() == x.shape() { + let ys: Vec = y.iter().copied().collect(); + pair(&xs, &ys, &mut words); + } else if let (1, Some(&t)) = (y.len(), y.iter().next()) { + scalar(&xs, t, &mut words); + } else { + return Err(runtime( + n, + format!("{:?} against {:?} is not admitted", x.shape(), y.shape()), + )); + } + Ok(vec![Value::Bool { + shape: x.shape().to_vec(), + words, + }]) +} + +fn do_equal_i64(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + compare_i64( + a, + n, + ndarray::simd::eq_i64_pair_to_mask, + ndarray::simd::eq_i64_to_mask, + ) +} + +fn do_less_or_equal_i64(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + compare_i64( + a, + n, + ndarray::simd::le_i64_pair_to_mask, + ndarray::simd::le_i64_to_mask, + ) +} + +/// `Where(C, X, Y)` over int64: `blend_i64`, or `blend_i64_scalar` for a +/// scalar `Y`. +fn do_where_i64(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let Some(Value::Bool { shape, words }) = a.first() else { + return Err(runtime(n, "condition is not a bool tensor")); + }; + let x = i64_arg(a, 1, n)?; + let y = i64_arg(a, 2, n)?; + if shape.as_slice() != x.shape() { + return Err(runtime(n, "condition and X shapes differ")); + } + let xs: Vec = x.iter().copied().collect(); + let mut out = vec![0i64; xs.len()]; + if let (1, Some(&yv)) = (y.len(), y.iter().next()) { + ndarray::simd::blend_i64_scalar(words, &xs, yv, &mut out); + } else if y.shape() == x.shape() { + let ys: Vec = y.iter().copied().collect(); + ndarray::simd::blend_i64(words, &xs, &ys, &mut out); + } else { + return Err(runtime(n, "Y does not broadcast to X")); + } + Ok(vec![Value::I64( + ArrayD::from_shape_vec(IxDyn(x.shape()), out).expect("same length"), + )]) +} + +/// `ConstantOfShape`: `ArrayD::from_elem` of the attribute's one element +/// (ONNX default: float 0); a bool value fills the packed mask. +fn do_constant_of_shape(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let dims = a + .first() + .and_then(|v| v.i64s()) + .ok_or_else(|| runtime(n, "no shape"))?; + let shape: Vec = dims + .iter() + .map(|&d| usize::try_from(d).map_err(|_| runtime(n, format!("negative dimension {d}")))) + .collect::>()?; + let len: usize = shape.iter().product(); + let sh = IxDyn(&shape); + let out = match n.attr("value") { + None => Value::F32(ArrayD::from_elem(sh, 0.0)), + Some(Attr::Tensor(Some(t))) => match t { + Value::F32(v) => Value::F32(ArrayD::from_elem( + sh, + v.iter().copied().next().unwrap_or(0.0), + )), + Value::I32(v) => { + Value::I32(ArrayD::from_elem(sh, v.iter().copied().next().unwrap_or(0))) + } + Value::I64(v) => { + Value::I64(ArrayD::from_elem(sh, v.iter().copied().next().unwrap_or(0))) + } + Value::Bool { words, .. } => { + let mut w = vec![0u64; lance_graph_onnx::mask_words(len)]; + if words.first().is_some_and(|w| w & 1 == 1) { + w.fill(u64::MAX); + if !len.is_multiple_of(64) { + if let Some(last) = w.last_mut() { + *last = (1u64 << (len % 64)) - 1; + } + } + } + Value::Bool { shape, words: w } + } + }, + Some(_) => return Err(runtime(n, "value is not a decodable tensor")), + }; + Ok(vec![out]) +} + +/// `Expand`: `ndarray`'s `broadcast` to the multidirectional broadcast of the +/// input shape and the target. +fn do_expand(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = a.first().ok_or_else(|| runtime(n, "no input"))?; + let target: Vec = a + .get(1) + .and_then(|v| v.i64s()) + .ok_or_else(|| runtime(n, "no target shape"))? + .iter() + .map(|&t| usize::try_from(t).map_err(|_| runtime(n, format!("negative dimension {t}")))) + .collect::>()?; + let shape = broadcast_shapes(&x.shape(), &target).ok_or_else(|| { + runtime( + n, + format!("{:?} does not broadcast to {target:?}", x.shape()), + ) + })?; + fn bc(a: &ArrayD, s: &[usize]) -> Option> { + a.broadcast(IxDyn(s)).map(|v| v.to_owned()) + } + let bad = || runtime(n, "broadcast failed"); + Ok(vec![match x { + Value::F32(v) => Value::F32(bc(v, &shape).ok_or_else(bad)?), + Value::I64(v) => Value::I64(bc(v, &shape).ok_or_else(bad)?), + other => return Err(runtime(n, format!("Expand of {:?}", other.dtype()))), + }]) +} + // ── the table ─────────────────────────────────────────────────────────────── const OPSETS: RangeInclusive = 13..=18; @@ -1023,7 +1249,7 @@ pub fn registry() -> Registry { use DType::*; use Numeric::*; Registry { - version: 2, + version: 3, regs: vec![ reg!( 0x0F01, @@ -1213,6 +1439,72 @@ pub fn registry() -> Registry { do_where_i32, Exact ), + reg!( + 0x0F70, + "simd::eq_i64_pair_to_mask / eq_i64_to_mask", + "Equal", + Some(0), + I64, + NO_ATTRS, + guard_pair_or_scalar, + do_equal_i64, + Exact + ), + reg!( + 0x0F71, + "simd::le_i64_pair_to_mask / le_i64_to_mask", + "LessOrEqual", + Some(0), + I64, + NO_ATTRS, + guard_pair_or_scalar, + do_less_or_equal_i64, + Exact + ), + reg!( + 0x0F72, + "simd::blend_i64 / blend_i64_scalar", + "Where", + Some(1), + I64, + NO_ATTRS, + guard_where, + do_where_i64, + Exact + ), + reg!( + 0x0F73, + "ndarray ArrayD::from_elem of the value attribute", + "ConstantOfShape", + Some(0), + I64, + NO_ATTRS, + guard_constant_of_shape, + do_constant_of_shape, + Exact + ), + reg!( + 0x0F74, + "ndarray broadcast, multidirectional", + "Expand", + Some(0), + I64, + NO_ATTRS, + guard_expand, + do_expand, + Exact + ), + reg!( + 0x0F75, + "ndarray broadcast, multidirectional", + "Expand", + Some(0), + F32, + NO_ATTRS, + guard_expand, + do_expand, + Exact + ), reg!( 0x0F33, "ndarray::hpc::vml::vserf", @@ -1484,6 +1776,206 @@ mod tests { ); } + fn i64v(shape: &[usize], v: Vec) -> Value { + Value::I64(ArrayD::from_shape_vec(IxDyn(shape), v).unwrap()) + } + + fn bits_of(v: &Value) -> (Vec, Vec) { + match v { + Value::Bool { shape, words } => (shape.clone(), words.clone()), + other => panic!("{other:?}"), + } + } + + /// 70 elements cross a mask word: the pair and scalar compares set the + /// right bits, and no bit past element 69 is set (tail bits clear). + #[test] + fn int64_compares_pack_tail_bits() { + let n = node("Equal", 2, vec![]); + let xs: Vec = (0..70).map(|i| i % 3).collect(); + let x = i64v(&[70], xs.clone()); + let pair = i64v( + &[70], + (0..70).map(|i| if i == 69 { 2 } else { 0 }).collect(), + ); + let (shape, w) = bits_of(&do_equal_i64(&[&x, &pair], &n).unwrap()[0]); + assert_eq!(shape, [70]); + assert_eq!(w.len(), 2); + for i in 0..70 { + let want = xs[i] == if i == 69 { 2 } else { 0 }; + assert_eq!((w[i / 64] >> (i % 64)) & 1 == 1, want, "pair {i}"); + } + assert_eq!(w[1] >> 6, 0, "tail bits past element 69"); + let one = i64v(&[], vec![1]); + let (_, w) = bits_of(&do_less_or_equal_i64(&[&x, &one], &n).unwrap()[0]); + for i in 0..70 { + assert_eq!((w[i / 64] >> (i % 64)) & 1 == 1, xs[i] <= 1, "scalar {i}"); + } + assert_eq!(w[1] >> 6, 0); + let bad = i64v(&[2], vec![1, 2]); + assert!( + do_equal_i64(&[&x, &bad], &n).is_err(), + "a non-admitted broadcast refuses at run time too" + ); + } + + /// int64 `Where` over a 70-element mask, scalar and tensor `Y`. + #[test] + fn int64_where_scalar_and_tensor() { + let n = node("Where", 3, vec![]); + let mut words = vec![0u64; 2]; + for i in (0..70).filter(|i| i % 5 == 0) { + words[i / 64] |= 1 << (i % 64); + } + let c = Value::Bool { + shape: vec![70], + words, + }; + let x = i64v(&[70], (0..70).collect()); + let ys = i64v(&[70], (0..70).map(|i| -i).collect()); + let s = i64v(&[], vec![i64::MIN]); + for (y, f) in [ + (&s, (|_: i64| i64::MIN) as fn(i64) -> i64), + (&ys, |i: i64| -i), + ] { + let out = do_where_i64(&[&c, &x, y], &n).unwrap(); + let Value::I64(o) = &out[0] else { panic!() }; + for (i, &v) in o.iter().enumerate() { + assert_eq!(v, if i % 5 == 0 { i as i64 } else { f(i as i64) }, "{i}"); + } + } + } + + /// ONNX `ConstantOfShape`: the value attribute's one element (default + /// float 0), an empty shape gives a scalar, a zero dimension an empty + /// tensor, a negative dimension or a multi-element value is FALSE, a + /// non-constant shape UNKNOWN; a bool value fills the mask without tail + /// bits. + #[test] + fn constant_of_shape_semantics() { + let with = |v: Option| { + node( + "ConstantOfShape", + 1, + v.map(|v| vec![("value".to_string(), Attr::Tensor(Some(v)))]) + .unwrap_or_default(), + ) + }; + let run = |n: &SourceNode, d: Vec| { + do_constant_of_shape(&[&i64v(&[d.len()], d)], n) + .unwrap() + .remove(0) + }; + assert_eq!( + run(&with(None), vec![2]), + Value::F32(ArrayD::zeros(IxDyn(&[2]))) + ); + assert_eq!( + run(&with(Some(i64v(&[1], vec![5]))), vec![]), + i64v(&[], vec![5]), + "empty shape: a scalar" + ); + assert_eq!( + run(&with(Some(i64v(&[1], vec![5]))), vec![2, 0]).shape(), + [2, 0] + ); + let tru = Value::Bool { + shape: vec![1], + words: vec![1], + }; + let (shape, w) = bits_of(&run(&with(Some(tru)), vec![70])); + assert_eq!((shape, w), (vec![70], vec![u64::MAX, (1 << 6) - 1])); + let guard = |n: &SourceNode, d: Option>| { + let c = d.map(|d| i64v(&[d.len()], d)); + let fact = Fact { + dtype: Some(DType::I64), + shape: Some(vec![Dim::Known(1)]), + }; + guard_constant_of_shape(&NodeView { + node: n, + args: vec![Some(&fact)], + consts: vec![c.as_ref()], + }) + }; + assert_eq!(guard(&with(None), Some(vec![2, 3])), K3::True); + assert_eq!(guard(&with(None), Some(vec![-1])), K3::False); + assert_eq!( + guard(&with(Some(i64v(&[2], vec![1, 2]))), Some(vec![2])), + K3::False + ); + assert_eq!(guard(&with(None), None), K3::Unknown); + let rank2 = Fact { + dtype: Some(DType::I64), + shape: Some(vec![Dim::Known(1), Dim::Known(2)]), + }; + let c = i64v(&[1, 2], vec![2, 3]); + let n = with(None); + assert_eq!( + guard_constant_of_shape(&NodeView { + node: &n, + args: vec![Some(&rank2)], + consts: vec![Some(&c)] + }), + K3::False, + "the shape input is 1-D" + ); + } + + /// ONNX `Expand`: multidirectional broadcast of the input shape and the + /// target, so a target dimension 1 keeps the input's and a shorter + /// target is padded on the left. + #[test] + fn expand_semantics() { + let n = node("Expand", 2, vec![]); + let x = i64v(&[3, 1], vec![1, 2, 3]); + let out = do_expand(&[&x, &i64v(&[2], vec![1, 4])], &n) + .unwrap() + .remove(0); + assert_eq!(out.shape(), [3, 4], "target 1 keeps the input dimension 3"); + assert_eq!(out.i64s().unwrap()[..4], [1, 1, 1, 1]); + let out = do_expand(&[&x, &i64v(&[3], vec![2, 1, 1])], &n) + .unwrap() + .remove(0); + assert_eq!(out.shape(), [2, 3, 1]); + assert!( + do_expand(&[&x, &i64v(&[2], vec![2, 1])], &n).is_err(), + "3 against 2" + ); + assert!(do_expand(&[&x, &i64v(&[1], vec![-1])], &n).is_err()); + } + + /// int64 compares admit a same-shape or a one-element right side only. + #[test] + fn pair_or_scalar_admission() { + use Dim::*; + let n = node("Equal", 2, vec![]); + let check = |a: Vec, b: Vec| { + let (fa, fb) = (f(a), f(b)); + guard_pair_or_scalar(&NodeView { + node: &n, + args: vec![Some(&fa), Some(&fb)], + consts: vec![None, None], + }) + }; + assert_eq!(check(vec![Known(3)], vec![Known(3)]), K3::True); + assert_eq!(check(vec![Known(3)], vec![Known(1)]), K3::True); + assert_eq!(check(vec![Known(3)], vec![]), K3::True); + assert_eq!( + check(vec![Known(3), Known(1)], vec![Known(1), Known(3)]), + K3::False, + "general broadcast not admitted" + ); + assert_eq!(check(vec![Symbol("N".into())], vec![Known(3)]), K3::Unknown); + } + + #[test] + fn broadcast_shapes_rules() { + assert_eq!(broadcast_shapes(&[1, 3], &[2, 1]), Some(vec![2, 3])); + assert_eq!(broadcast_shapes(&[2, 3], &[2, 2, 3]), Some(vec![2, 2, 3])); + assert_eq!(broadcast_shapes(&[3], &[1]), Some(vec![3])); + assert_eq!(broadcast_shapes(&[2, 3], &[4, 3]), None); + } + /// `0` copies, one `-1` infers, the element count must match. #[test] fn reshape_target_rules() { diff --git a/crates/tract-harvest-probe/tests/jina.rs b/crates/tract-harvest-probe/tests/jina.rs index 0eedbae00..98ad0aaab 100644 --- a/crates/tract-harvest-probe/tests/jina.rs +++ b/crates/tract-harvest-probe/tests/jina.rs @@ -446,7 +446,7 @@ fn new_weight_generation_refuses_the_old_plan() { /// 7,620 source values with a full fact; the typed stage labels 2,546, of /// which 1,988 are admitted (a typed fact is admitted only from the outlet /// that is the value's source producer, `dr3::admit`); the two agree wherever -/// both speak. Registry v2 coverage with these facts, cold folding included, +/// both speak. Registry v3 coverage with these facts, cold folding included, /// pinned exactly so any change forces a re-measure. #[test] #[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] @@ -492,5 +492,5 @@ fn full_graph_facts_and_coverage() { .fold([0; 3], |a, c| [a[0] + c[0], a[1] + c[1], a[2] + c[2]]) }; assert_eq!(tot(&fs.typed), [3841, 226, 3553]); - assert_eq!(tot(&fs.merged()), [6404, 56, 1160]); + assert_eq!(tot(&fs.merged()), [6581, 114, 925]); } diff --git a/crates/tract-harvest-probe/tests/tiny.rs b/crates/tract-harvest-probe/tests/tiny.rs index 558bcec24..59429d7e3 100644 --- a/crates/tract-harvest-probe/tests/tiny.rs +++ b/crates/tract-harvest-probe/tests/tiny.rs @@ -767,3 +767,158 @@ fn erf_sweep_against_f64_and_ort() { assert!(all <= 2, "{all} ULP overall"); assert!(vs_ort <= 2, "{vs_ort} ULP vs ORT"); } + +// ── int64 compares, Where, ConstantOfShape, Expand (registry v3) ──────────── + +fn ort_ints(v: &str, out: &str) -> (Vec, Vec) { + let e = &ort()["variants"][v]["ort"][out]; + let shape = e["shape"] + .as_array() + .unwrap() + .iter() + .map(|d| d.as_u64().unwrap() as usize) + .collect(); + ( + shape, + e["bits"] + .as_array() + .unwrap() + .iter() + .map(|b| b.as_i64().unwrap()) + .collect(), + ) +} + +fn ints(m: &std::collections::HashMap, k: &str) -> (Vec, Vec) { + match &m[k] { + Value::I64(a) => (a.shape().to_vec(), a.iter().copied().collect()), + Value::Bool { shape, words } => ( + shape.clone(), + (0..shape.iter().product::()) + .map(|i| ((words[i / 64] >> (i % 64)) & 1) as i64) + .collect(), + ), + other => panic!("{k}: {other:?}"), + } +} + +/// Every int64 node over `Shape(F)` folds through ndarray #400's compares and +/// blends, `from_elem` and `broadcast`; `Expand(F)` reads F and runs. Every +/// output agrees with ORT exactly. +#[test] +fn int64_shape_ops_fold_and_match_ort() { + let reg = registry(); + let (fx, p) = plan("shape_i64", ®); + let p = p.unwrap(); + let folded: Vec<(&str, u16)> = p + .folded + .iter() + .map(|b| (b.name.as_str(), b.classid)) + .collect(); + assert_eq!( + folded, + [ + ("n_shape", 0x0F60), + ("n_equal", 0x0F70), + ("n_where", 0x0F72), + ("n_le", 0x0F71), + ("n_cos", 0x0F73), + ("n_expand_row", 0x0F74) + ] + ); + let steps: Vec<(&str, u16)> = p + .steps + .iter() + .map(|b| (b.name.as_str(), b.classid)) + .collect(); + assert_eq!(steps, [("n_expand_f", 0x0F75)]); + let out = p.run(&fx.store, &[("F", f_input())]).unwrap(); + for k in ["W", "LE", "CS", "ER"] { + assert_eq!(ints(&out, k), ort_ints("shape_i64", k), "{k}"); + } + assert_eq!(f32_bits(&out, "EF"), ort_f32("shape_i64", "EF")); +} + +/// An omitted optional output before a present one (`LSTM` outputs +/// `["", "Y_h"]`): Tract numbers outlets after dropping empty names, so +/// `Y_h` is slot 0, not 1. `source_producers` must name that outlet, and the +/// fact admitted for `Y_h` must be `Y_h`'s (ORT's shape `[1, 1, 2]`), not the +/// omitted `Y`'s (`[5, 1, 1, 2]`). With both outputs present `Y_h` is slot 1. +#[test] +fn an_omitted_optional_output_does_not_shift_the_producer() { + use tract_harvest_probe::dr3::{source_producers, tract_fact_sources, Outlet}; + use tract_harvest_probe::registry::{Dim, Fact}; + let dims = |s: &[usize]| Some(s.iter().map(|&d| Dim::Known(d)).collect::>()); + for (v, slot) in [("lstm_skip", 0), ("lstm_full", 1)] { + let b = bytes(v); + let src = SourceInventory::from_bytes(&b).unwrap(); + let p = source_producers(&src); + assert_eq!( + p["Y_h"], + Some(Outlet { + node: "n_lstm".into(), + slot + }), + "{v}" + ); + let (shape, _) = ort_f32(v, "Y_h"); + assert_eq!(shape, [1, 1, 2]); + let fs = tract_fact_sources(&b, &[]).unwrap(); + assert_eq!( + fs.inference.get("Y_h"), + Some(&Fact { + dtype: Some(tract_harvest_probe::value::DType::F32), + shape: dims(&shape) + }), + "{v}" + ); + } + let fs = tract_fact_sources(&bytes("lstm_full"), &[]).unwrap(); + assert_eq!( + fs.inference["Y"].shape, + dims(&[5, 1, 1, 2]), + "the other output's shape differs" + ); +} + +/// int64 shape arithmetic is refused until ndarray supplies checked words +/// (the D-TRACT-SHAPE-1 ndarray ask): `Div` must truncate toward zero and +/// refuse a zero divisor; `Mul`/`Add`/`Sub` must refuse overflow; `Range` +/// must refuse a zero step. ORT's outputs for this fixture are frozen in +/// `ort.json` (`i64_arith`) as the oracle for the registration that follows. +#[test] +fn int64_arithmetic_is_refused_until_checked_words_exist() { + let o = ort(); + let e = &o["variants"]["i64_arith"]["ort"]; + assert_eq!( + e["DV"]["bits"], + serde_json::json!([-3, 2]), + "Div truncates toward zero" + ); + let reg = registry(); + match plan("i64_arith", ®).1 { + Err(Refusal::Unregistered { node, signature }) => { + assert_eq!(node, "n_div"); + assert!( + signature.starts_with("ai.onnx:Div(Some(I64))"), + "{signature}" + ); + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// Facts read from one model refuse to bind to another (a stale binding): +/// `gen2` has `base`'s value names but different bytes. +#[test] +fn facts_of_another_model_refuse_to_bind() { + use tract_harvest_probe::dr3::tract_fact_sources; + let base = SourceInventory::from_bytes(&bytes("base")).unwrap(); + let gen2 = tract_fact_sources(&bytes("gen2"), &[]).unwrap(); + match gen2.merged_for(&base) { + Err(Refusal::Frontend(e)) => assert!(e.contains("applied to model"), "{e}"), + other => panic!("{:?}", other.map(|m| m.len())), + } + let own = tract_fact_sources(&bytes("base"), &[]).unwrap(); + assert!(own.merged_for(&base).is_ok()); +} diff --git a/crates/tract-harvest-probe/tools/gen_tiny.py b/crates/tract-harvest-probe/tools/gen_tiny.py index d074f3c9b..5ca3fbb3b 100644 --- a/crates/tract-harvest-probe/tools/gen_tiny.py +++ b/crates/tract-harvest-probe/tools/gen_tiny.py @@ -135,6 +135,52 @@ def build_shape(variant): inits.append(nh.from_array(np.zeros((2, 2), np.float32), "G")) nodes = [h.make_node("Concat", ["F", "G"], ["C"], name="n_concat", axis=0)] outs = [h.make_tensor_value_info("C", T.FLOAT, [4, 3])] + elif variant == "shape_i64": + # int64 compares, Where, ConstantOfShape, Expand over Shape(F) = [2, 3]. + inp = [h.make_tensor_value_info("F", T.FLOAT, [2, 3])] + inits += [i64("k24", [2, 4]), i64("seven", 7), i64("k33", [3, 3]), i64("row", [[1, 2, 3]]), + i64("k223", [2, 2, 3])] + nodes = [ + h.make_node("Shape", ["F"], ["s"], name="n_shape"), + h.make_node("Equal", ["s", "k24"], ["eq"], name="n_equal"), + h.make_node("Where", ["eq", "s", "seven"], ["W"], name="n_where"), + h.make_node("LessOrEqual", ["W", "k33"], ["LE"], name="n_le"), + h.make_node("ConstantOfShape", ["s"], ["CS"], name="n_cos", + value=nh.from_array(np.array([5], np.int64))), + h.make_node("Expand", ["row", "s"], ["ER"], name="n_expand_row"), + h.make_node("Expand", ["F", "k223"], ["EF"], name="n_expand_f"), + ] + outs = [h.make_tensor_value_info("W", T.INT64, [2]), + h.make_tensor_value_info("LE", T.BOOL, [2]), + h.make_tensor_value_info("CS", T.INT64, [2, 3]), + h.make_tensor_value_info("ER", T.INT64, [2, 3]), + h.make_tensor_value_info("EF", T.FLOAT, [2, 2, 3])] + elif variant == "i64_arith": + # The int64 shape arithmetic no ndarray word covers yet (the ndarray + # ask): truncating Div on negatives, Mul, Add, Sub, Range. + inp = [h.make_tensor_value_info("F", T.FLOAT, [2, 3])] + inits += [i64("negs", [-7, 7]), i64("mul_k", [3, -4]), i64("add_k", [-5, 10]), + i64("r0", 0), i64("r1", 1)] + nodes = [ + h.make_node("Shape", ["F"], ["s"], name="n_shape"), + h.make_node("Div", ["negs", "s"], ["DV"], name="n_div"), + h.make_node("Mul", ["s", "mul_k"], ["ML"], name="n_mul"), + h.make_node("Add", ["s", "add_k"], ["AD"], name="n_add"), + h.make_node("Sub", ["s", "add_k"], ["SB"], name="n_sub"), + h.make_node("Gather", ["s", "r1"], ["n3"], name="n_gather", axis=0), + h.make_node("Range", ["r0", "n3", "r1"], ["RG"], name="n_range"), + ] + outs = [h.make_tensor_value_info(o, T.INT64, None) for o in ["DV", "ML", "AD", "SB", "RG"]] + elif variant in ("lstm_skip", "lstm_full"): + # An omitted optional output before a present one: Tract numbers a + # node's outlets after dropping empty names (onnx/src/model.rs:150). + hs = 2 + inp = [h.make_tensor_value_info("X", T.FLOAT, [5, 1, 3])] + inits = [nh.from_array(np.full((1, 4 * hs, 3), 0.1, np.float32), "W"), + nh.from_array(np.full((1, 4 * hs, hs), 0.2, np.float32), "R")] + outs_ = ["", "Y_h"] if variant == "lstm_skip" else ["Y", "Y_h"] + nodes = [h.make_node("LSTM", ["X", "W", "R"], outs_, name="n_lstm", hidden_size=hs)] + outs = [h.make_tensor_value_info("Y_h", T.FLOAT, [1, 1, hs])] g = h.make_graph(nodes, f"tiny_{variant}", inp, outs, inits) return h.make_model(g, opset_imports=[h.make_opsetid("", OPSET)], producer_name="d-tract-shape-1", ir_version=8) @@ -163,6 +209,8 @@ def erf_sweep_inputs(): def bits(a): a = np.ascontiguousarray(a) + if a.dtype == np.bool_: + return [int(v) for v in a.ravel()] if a.dtype == np.float32: return [int(v) for v in a.view(np.uint32).ravel()] return [int(v) for v in a.ravel()] @@ -201,12 +249,14 @@ def main(): except Exception as e: # ORT refusing a variant is itself evidence entry["ort_error"] = type(e).__name__ + ": " + str(e).splitlines()[0][:200] manifest["variants"][v] = entry - for v in ["shape_fold", "shape_sym", "neg_gather_index", "neg_concat"]: + for v in ["shape_fold", "shape_sym", "neg_gather_index", "neg_concat", "shape_i64", "lstm_skip", "lstm_full", "i64_arith"]: data = build_shape(v).SerializeToString() with open(os.path.join(OUT, f"{v}.onnx"), "wb") as f: f.write(data) entry = {"sha256": hashlib.sha256(data).hexdigest()} - feeds = {"F": INPUTS["F"]} if v != "shape_sym" else {"X": np.arange(6, dtype=np.float32)} + feeds = {"shape_sym": {"X": np.arange(6, dtype=np.float32)}, + "lstm_skip": {"X": np.ones((5, 1, 3), np.float32)}, + "lstm_full": {"X": np.ones((5, 1, 3), np.float32)}}.get(v, {"F": INPUTS["F"]}) try: sess = session(data) outs = sess.run(None, feeds)