diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 7eb51cbed..818c47cda 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -57,6 +57,7 @@ | **D-ONNX-INTAKE-4** | S4 numeric/quantization parity for real model blockers | Queued (conditional; not authorized) | IntegerExact / bounded FP / SourceParity / CertifiedMath kept distinct | | **D-ONNX-INTAKE-5** | S5 whole-model forward pass and cost closure | Goal, not authorized | outputs vs ORT under declared tolerance; every cost stage reported | | **D-TRACT-HARVEST-0** | ONNX graph as a DR#3 graph (nodes Identity, values Property, dep = do_ ∘ read_by) resolved through a DT64 ON×WHEN×DO registry (op signature → classid → native DO); Tract cold facts; ORT (optimizer off) oracle; `crates/tract-harvest-probe` | Lab probe implemented (this PR, redesigned to the graph); plan [d-tract-harvest-0-native-algebra-bridge-v1.md](../plans/d-tract-harvest-0-native-algebra-bridge-v1.md) §0 | 26 tests, 13 disable runs red; full model 4,146/7,620 nodes WHEN-true with inference-stage facts (§0.1), 0 refused for missing facts, >64 positions untiled, first unregistered groups Unsqueeze/Shape/Gather/Concat/MatMul | +| **D-TRACT-SHAPE-1** | Cold shape-subgraph folding (Const/Shape/Symbolic/Data classes; fold through the node's own WHEN/DO; stale-input refusal), shape-op registrations (Shape, Gather i64, Unsqueeze, Concat, Reshape, Cast i64, Erf via `vserf`, scalar-Y Where via `blend_i32_scalar`), producer-checked Tract fact admission | **In PR** (lab probe); plan [d-tract-harvest-0-native-algebra-bridge-v1.md](../plans/d-tract-harvest-0-native-algebra-bridge-v1.md) §0.2 | 40 tests, 9 disable runs red; Jina [1,10] coverage 4,146/0/3,474 → 6,404/56/1,160, 4,440 nodes fold in 7.3 ms; remaining blocker int64 shape arithmetic (owner decision) | ## D-TFLP — TFLite models on the native polyfill: int8 codes as Palette256 (2026-10-25) diff --git a/.claude/board/entries/2026-10-30-d-tract-shape-1-cold-shape-folding.md b/.claude/board/entries/2026-10-30-d-tract-shape-1-cold-shape-folding.md new file mode 100644 index 000000000..ec069923f --- /dev/null +++ b/.claude/board/entries/2026-10-30-d-tract-shape-1-cold-shape-folding.md @@ -0,0 +1,13 @@ +# D-TRACT-SHAPE-1 — cold shape folding and ON × WHEN × DO admission of the shape ops + +**Date 2026-10-30 · Status LAB PROBE · Plan [`d-tract-harvest-0-native-algebra-bridge-v1.md`](../../plans/d-tract-harvest-0-native-algebra-bridge-v1.md) §0.2 · Crate `crates/tract-harvest-probe`** + +Every value is classed Const / Shape / Symbolic / Data. A node folds at resolution only through its own registration (WHEN TRUE, then its DO) and only when invariant under the source and the resolved input facts; a run with other input dims is refused. Registry v2 adds Shape, Gather (int64 data), Unsqueeze, Concat, Reshape, Cast i64→i64 and Erf (`vserf`); scalar-Y Where uses `blend_i32_scalar`. + +A Tract fact is admitted only from the value's source-producer outlet (a matching label is not authority): typed 1,988 of 2,546 admitted, 558 rejected; inference 0 rejected. Tract `TDim` is read as int64. A Tract panic (out-of-range constant Gather index) is a frontend refusal. + +**Jina coverage [WHEN-true, not-true, unregistered], pinned [1,10]:** 4,146 / 0 / 3,474 → **6,404 / 56 / 1,160**; 4,440 nodes fold in 7.3 ms, 1,972 folded values retained (~20 KB). The remaining shape blockers are int64 arithmetic (Mul 115, ConstantOfShape 59, Equal 58, Where 58, Div 56, …), which no ndarray word covers. + +Fixtures (ORT 1.29.0 `ORT_DISABLE_ALL`): shape chain bit-exact; Erf ≤ 2 ULP vs ORT and f64 over 4,115 points. 40 tests; 9 disable runs red. + +**OPEN:** int64 shape-arithmetic semantics (owner, then ndarray ask); axis as DO parameter vs classid per axis; `vserf` ~4x slower than Tract's erf; ConstantOfShape/Expand via ndarray core not yet registered; multi-tile and MatMul gates unchanged. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 67aa558fc..c57bf4965 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,10 +25,11 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -504 entries, 2026-08-06 .. 2026-10-30. +505 entries, 2026-08-06 .. 2026-10-30. | date | entry id | finding | file | |---|---|---|---| +| 2026-10-30 | `D-TRACT-SHAPE-1` | | [2026-10-30-d-tract-shape-1-cold-shape-folding.md](2026-10-30-d-tract-shape-1-cold-shape-folding.md) | | 2026-10-30 | `D-TRACT-HARVEST-0` | | [2026-10-30-d-tract-harvest-0-native-algebra-bridge.md](2026-10-30-d-tract-harvest-0-native-algebra-bridge.md) | | 2026-10-30 | `D-DRCA-3` | | [2026-10-30-d-drca-3-dr4-method-routing.md](2026-10-30-d-drca-3-dr4-method-routing.md) | | 2026-10-29 | `palette512-turbovec9-nsl-worksteal` | | [2026-10-29-palette512-turbovec9-nsl-worksteal.md](2026-10-29-palette512-turbovec9-nsl-worksteal.md) | diff --git a/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md b/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md index c1f10e512..40bb72385 100644 --- a/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md +++ b/.claude/plans/d-tract-harvest-0-native-algebra-bridge-v1.md @@ -61,7 +61,7 @@ ON × WHEN × DO per node: ON = its input Property positions; WHEN = a three-val **OPEN (graph design):** - More than 64 nodes or values needs several DR64 tiles; the full graph is only censused, not tiled. - ~~Facts for the 5,074 values Tract's typed stage leaves unlabelled~~ CLOSED 2026-10-30, see §0.1. -- Registry entries for the shape-plumbing ops (`Unsqueeze`, `Shape`, `Gather` on shapes, `Concat`, `Reshape`) and the admitted f32 `MatMul`. +- ~~Registry entries for the shape-plumbing ops~~ registered 2026-10-30, see §0.2. Still open: the admitted f32 `MatMul`, and int64 shape arithmetic (§0.2). - The plan clones constants per run; hot allocation is not yet minimal. - Classids are lab-local; minting through OGAR is a separate decision. @@ -79,6 +79,42 @@ The 169 refusals came entirely from missing facts. A further 226 nodes became re Tests: `inference_stage_facts_cover_every_value_and_agree_with_typed` (tiny), `full_graph_facts_and_coverage` (Jina, counts pinned exactly), `undetermined_input_dim_gives_no_shape_and_typing_failure_is_recorded`, `dr3::tests::fact_sources_disagree_and_merge`. `unknown_shapes_refuse` re-pinned: `T` now has a Tract fact (the typed stage had merged it with `Z`), so `n_greater` is the first UNKNOWN. Disable runs, each red on the named tests: merge from typed only; undetermined dim as a symbol; typed failure propagated; `disagreements` always empty. +### 0.2 Cold shape folding (D-TRACT-SHAPE-1, 2026-10-30) + +**Classification.** `fold::classify` gives every value a dependency class: `Const` (source constants only), `Shape` (constants and fully known input shapes), `Symbolic` (a shape with a symbolic dimension) or `Data` (an input value, or a shape the facts do not determine). A `Shape` node takes the class of its input's shape, every other node the strongest class of its inputs. Jina, inputs pinned `[1,10]`: + +| op | Const | Shape | Symbolic | Data | +|---|---|---|---|---| +| Unsqueeze (888) | 0 | 830 | 0 | 58 | +| Shape (430) | 0 | 430 | 0 | 0 | +| Gather (373) | 0 | 371 | 0 | 2 | +| Concat (284) | 0 | 228 | 0 | 56 | +| Reshape (228) | 0 | 58 | 0 | 170 | +| all 7,620 nodes | 2,760 | 2,399 | 0 | 2,461 | + +Unpinned, Tract's `analyse` fails at `/Flatten` (two symbols), so the Symbolic class is shown on the `shape_sym` fixture instead. + +**Folding.** `fold::fold` executes a node at resolution only when its class is `Const` or `Shape` and either every input is a known constant or it is `Shape` over a fully known shape. It uses the node's own registration: lookup, WHEN must be TRUE, then the same DO a run would call. `plan::resolve` folds before binding, writes the folded values into the tile's constants (so later guards read them) and keeps folded nodes in `Plan::folded`, not in the steps. A run whose inputs differ from the resolved facts is refused before any step (`check_against_fact`), so a folded shape is never applied to an input it was not computed for. Symbolic and Data nodes stay dynamic. + +**Registry v2.** Lab classids: `Shape` F32/I64/I32/Bool 0x0F60–0x0F63 (`start`/`end` clamped), `Gather` on int64 data 0x0F64 (`ndarray` `select`; constant indices in `[-d, d)`, else FALSE; run-time indices UNKNOWN), `Unsqueeze` I64/F32 0x0F65/0x0F66, `Concat` I64/F32 0x0F67/0x0F68 (`ndarray::concatenate`; equal non-axis dims, two different symbols UNKNOWN), `Reshape` I64/F32 0x0F69/0x0F6A (`allowzero = 0`, constant target, known input shape), `Cast` i64→i64 identity 0x0F11, `Erf` 0x0F33 (`ndarray::hpc::vml::vserf`, Bounded ≤ 2 ULP on [-5, 5]). Scalar-Y `Where` now calls `ndarray::simd::blend_i32_scalar` (one allocation and 28 B less per call on fixture A; no new kernel). `AttrRule.value: None` admits any integer attribute (an axis) read by the DO; a fixed value still keys a separate signature. + +**Fact identity.** A Tract fact is admitted for a source value only if exactly one outlet of the stage carries the label and that outlet is the value's source producer by Tract's own naming rule (`dr3::source_producers`, `onnx/src/model.rs:122,150`); a matching label alone is not authority (`dr3::admit`). Jina: inference 7,937 admitted, 0 rejected; typed 1,988 of 2,546 full facts admitted, 558 rejected (557 from an outlet Tract renamed during typing, 1 label on two outlets). Coverage is unaffected because the inference stage covers every value. Tract's `TDim` (the type its `Shape` emits) is read as int64, the ONNX type. A Tract panic during `analyse` or typing is a frontend refusal (Tract indexes out of bounds on a constant out-of-range `Gather` index, `core/src/ops/array/gather.rs:75`). + +**Coverage, Jina pinned `[1,10]`, [WHEN-true, WHEN-not-true, unregistered]:** registry v1 (§0.1) 4,146 / 0 / 3,474 → registry v2 with folding **6,404 / 56 / 1,160**. 4,440 nodes fold in 7.3 ms (facts 221 ms); 1,972 folded values are still read by non-folded nodes (~20 KB). Per op: Unsqueeze 888/0/0, Shape 430/0/0, Gather 371/0/2 (f32 embedding and bool data), Concat 284/0/0, Reshape 170/56/2 (56 UNKNOWN: target computed by unregistered int64 arithmetic; 2 bool data). Every registered shape-class node that does not fold is blocked by an upstream unregistered node, none by its own WHEN: `Unsqueeze` 181, `Cast` 117, `Concat` 57, `Mul` 2. The unregistered shape-class nodes are int64 arithmetic: `Mul` 115, `ConstantOfShape` 59, `Equal` 58, `Where` 58, `Div` 56, `Range` 4, `Expand` 2, `Add` 1, `LessOrEqual` 1, `And` 1, plus `Cast` to bool/float 2, `Cos`/`Sin`/`MatMul`/`Transpose` 1 each. + +**Fixtures and oracle.** New in `tools/gen_tiny.py`, frozen ORT 1.29.0 `ORT_DISABLE_ALL`: `shape_fold` (the chain folds; `Reshape(F)` and `Unsqueeze(F)` run bit-exact), `shape_sym` (nothing folds; Reshape WHEN UNKNOWN, refused), `neg_gather_index` (ORT fails the run, Tract panics, the frontend refuses), `neg_concat` (ORT and Tract both reject), `erf` (fixture A with Erf; ≤ 2 ULP vs ORT), `erf_sweep` (4,115 points: dense [-6, 6], signed zeros, infinities, NaN, subnormals; max 2 ULP vs `math.erf` rounded to f32 and vs ORT; specials exact). `neg_unsupported` now uses `Acosh`. + +**Cost (reported, `examples/cost.rs`, AVX-512 host, `target-cpu=native`):** `shape_fold` cold tile+fold+resolve 12.4 µs (2 steps, 4 folded), hot 1.7 µs, 37 allocations; `erf` 4,096 elements hot 8.5 µs vs Tract 2.0 µs (`vserf` alone 7.9 µs, 1.9 ns/element); fixture A hot 3.3 µs, 55 allocations (56 before scalar-Y Where); Jina slice tile+fold+resolve 20 µs (9 steps, 4 folded Constants). + +Tests (40, all green): `shape_chain_folds_and_the_data_nodes_run_against_ort`, `folded_shapes_refuse_a_run_with_other_input_dims`, `a_node_reading_input_values_never_folds`, `a_symbolic_shape_is_not_folded_and_unknown_is_not_true`, `an_out_of_range_gather_index_is_refused_not_a_panic`, `a_concat_dimension_mismatch_is_refused_by_the_frontend`, `erf_in_the_graph_against_ort`, `erf_sweep_against_f64_and_ort`, and unit tests `a_label_alone_does_not_admit_a_fact`, `source_producers_follow_tract_naming`, `concat_never_aliases_two_symbols`, `gather_index_range_is_three_valued`, `reshape_target_rules`, `attr_rules_any_and_fixed`, `shape_start_end_clamp`. Re-pinned: `full_graph_facts_and_coverage` (typed 1,988 admitted, 558 rejected, coverage above), the RMSNorm slice (9 steps + 4 folded Constants), `unregistered_op_is_refused` (Acosh). Disable runs, each red on the named test: no folding; label as authority; two symbols aliased; Reshape UNKNOWN as TRUE; symbolic shape classed as known; no stale-input check; Gather range always TRUE; no panic catch; Shape `end` ignored. Not load-bearing (documented in the code): the fold's dependency-class skip, since a Symbolic or Data node never has every input known. + +**OPEN (owner decisions):** +- int64 shape arithmetic (`Mul`, `Div`, `Add`, `Equal`, `Where`, `Range`) has no ndarray word; its overflow and division semantics (wrap or refuse; truncation; division by zero) need a decision before an ndarray ask. +- An axis attribute is a DO parameter under one classid per (op, dtype); the alternative is a classid per axis value. +- `Erf` is Bounded at 2 ULP; `vserf` is about 4x slower than Tract's `erf` on this host. +- `ConstantOfShape` and `Expand` map onto ndarray `from_elem` / `broadcast` and are not registered yet. +- Multi-tile orchestration and full Jina MatMul execution remain separate gates. + ## 1. The acceptance chain, demonstrated The chain under test: a source-attested ONNX subgraph is imported by Tract, admitted, lowered onto existing native methods, executed without Tract, and verified against ONNX Runtime (ORT). The chain holds for both fixtures: fixture A (tiny, synthetic) is bit-exact, and fixture B (a real model slice) is bounded. The numbers are in §4. Nothing here is certified, production or merged. diff --git a/crates/tract-harvest-probe/examples/cost.rs b/crates/tract-harvest-probe/examples/cost.rs index 1a34958a6..1582cc085 100644 --- a/crates/tract-harvest-probe/examples/cost.rs +++ b/crates/tract-harvest-probe/examples/cost.rs @@ -110,7 +110,13 @@ fn measure( let tract_ns = median(runs, || { black_box(plan.run(tract_in()).unwrap()); }); - println!("{label:>5} cold: source {src_us:8.1} µs | Tract facts {facts_us:8.1} µs | tile+resolve {resolve_us:7.1} µs ({} steps)", p.steps.len()); + let retained: usize = p + .folded + .iter() + .flat_map(|b| p.graph.outs[b.node].iter()) + .filter_map(|&o| p.graph.consts[o].as_ref()) + .count(); + println!("{label:>5} cold: source {src_us:8.1} µs | Tract facts {facts_us:8.1} µs | tile+fold+resolve {resolve_us:7.1} µs ({} steps, {} folded, {retained} folded constants)", p.steps.len(), p.folded.len()); println!("{label:>5} hot: graph plan {hot_ns:9.0} ns ({na} allocs, {nb} B) | Tract typed runnable {tract_ns:9.0} ns (incl. input tensors)"); } @@ -152,6 +158,44 @@ fn main() { 20001, ); + // ── shape folding (D-TRACT-SHAPE-1) ── + let shape = std::fs::read(format!("{dir}/tiny/shape_fold.onnx")).unwrap(); + measure( + "shape", + &shape, + &[], + vec![( + "F", + Value::F32(ArrayD::from_shape_vec(IxDyn(&[2, 3]), f.clone()).unwrap()), + )], + || { + tvec![tract_ndarray::Array2::from_shape_vec((2, 3), f.clone()) + .unwrap() + .into_dyn() + .into_tensor() + .into()] + }, + 20001, + ); + let erf = std::fs::read(format!("{dir}/tiny/erf_sweep.onnx")).unwrap(); + let ex: Vec = (0..4096).map(|i| -6.0 + 12.0 * i as f32 / 4095.0).collect(); + measure( + "erf", + &erf, + &[(0, DatumType::F32, vec![ex.len()])], + vec![( + "X", + Value::F32(ArrayD::from_shape_vec(IxDyn(&[ex.len()]), ex.clone()).unwrap()), + )], + || { + tvec![tract_ndarray::Array1::from(ex.clone()) + .into_dyn() + .into_tensor() + .into()] + }, + 20001, + ); + // ── fixture B ── let slice = match std::fs::read(format!("{dir}/jina/slice.onnx")) { Ok(b) => b, diff --git a/crates/tract-harvest-probe/examples/shape_census.rs b/crates/tract-harvest-probe/examples/shape_census.rs new file mode 100644 index 000000000..5428a715a --- /dev/null +++ b/crates/tract-harvest-probe/examples/shape_census.rs @@ -0,0 +1,105 @@ +//! Dependency census of the shape-plumbing ops, and what folds. +//! +//! `cargo run --release --example shape_census -- fixtures/jina/onnx_model.onnx [seq]` +//! +//! Without `seq` the inputs keep their symbolic dims; with it they are pinned +//! to `[1, seq]`. +use std::time::Instant; +use tract_harvest_probe::{dr3, fold, registry, source::SourceInventory}; +use tract_onnx::prelude::*; + +const OPS: &[&str] = &["Unsqueeze", "Shape", "Gather", "Concat", "Reshape"]; + +fn main() { + let path = std::env::args().nth(1).expect("model path"); + let src = SourceInventory::from_bytes(&std::fs::read(&path).unwrap()).unwrap(); + let pins: Vec<_> = std::env::args() + .nth(2) + .and_then(|s| s.parse::().ok()) + .map(|seq| { + (0..src.inputs.len()) + .map(|i| (i, i64::datum_type(), vec![1, seq])) + .collect() + }) + .unwrap_or_default(); + let t = Instant::now(); + let fs = dr3::tract_fact_sources_path(std::path::Path::new(&path), &pins).unwrap(); + let t_facts = t.elapsed(); + let facts = fs.merged(); + let mut rj = std::collections::BTreeMap::::new(); + for r in &fs.rejected { + *rj.entry(format!( + "{}: {}", + r.stage, + r.reason + .split(" (") + .next() + .unwrap_or(&r.reason) + .chars() + .take(40) + .collect::() + )) + .or_default() += 1; + } + println!( + "admitted facts: inference {}, typed {}; rejected {:?}", + fs.inference.len(), + fs.typed.len(), + rj + ); + let (_, node_dep) = fold::classify(&src, &facts); + println!( + "pins {:?}; typed stage error: {:?}", + pins.first().map(|p| &p.2), + fs.typed_error.as_deref().map(|e| &e[..e.len().min(80)]) + ); + println!("census [Const, Shape, Symbolic, Data]:"); + for (op, c) in fold::census(&src, &node_dep, OPS) { + println!(" {op:>10} {c:?}"); + } + let all = fold::census( + &src, + &node_dep, + &src.nodes + .iter() + .map(|n| n.op_type.as_str()) + .collect::>(), + ); + let tot = all.values().fold([0; 4], |a, c| { + [a[0] + c[0], a[1] + c[1], a[2] + c[2], a[3] + c[3]] + }); + println!(" all nodes {tot:?}"); + let reg = registry::registry(); + let t = Instant::now(); + let f = fold::fold(&src, &facts, ®); + let t_fold = t.elapsed(); + let retained = f.retained(&src); + let bytes: usize = retained + .iter() + .map(|(_, v)| v.shape().iter().product::() * 8) + .sum(); + println!( + "folded {} nodes in {t_fold:?} (facts {t_facts:?}); retained constants {} (~{bytes} B)", + f.nodes.len(), + retained.len() + ); + let mut by = std::collections::BTreeMap::::new(); + for (_, why) in &f.blocked { + *by.entry(why.clone()).or_default() += 1; + } + let cov = tract_harvest_probe::plan::coverage(&src, &facts, ®); + let tot = cov + .values() + .fold([0; 3], |a, c| [a[0] + c[0], a[1] + c[1], a[2] + c[2]]); + println!( + "registry v{} coverage [WHEN true, WHEN not true, unregistered] {tot:?}", + reg.version + ); + for op in OPS { + println!(" {op:>10} {:?}", cov.get(*op)); + } + println!("foldable by dependency but not folded:"); + for (why, c) in by { + println!(" {c:>5} {why}"); + } +} diff --git a/crates/tract-harvest-probe/fixtures/tiny/erf.onnx b/crates/tract-harvest-probe/fixtures/tiny/erf.onnx new file mode 100644 index 000000000..ddbe99396 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/erf.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/erf_sweep.onnx b/crates/tract-harvest-probe/fixtures/tiny/erf_sweep.onnx new file mode 100644 index 000000000..0b8e31276 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/erf_sweep.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/neg_concat.onnx b/crates/tract-harvest-probe/fixtures/tiny/neg_concat.onnx new file mode 100644 index 000000000..4335ce908 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/neg_concat.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/neg_gather_index.onnx b/crates/tract-harvest-probe/fixtures/tiny/neg_gather_index.onnx new file mode 100644 index 000000000..f4beee5b8 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/neg_gather_index.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/neg_unsupported.onnx b/crates/tract-harvest-probe/fixtures/tiny/neg_unsupported.onnx index 3d9c9007b..23d99c77e 100644 Binary files a/crates/tract-harvest-probe/fixtures/tiny/neg_unsupported.onnx and b/crates/tract-harvest-probe/fixtures/tiny/neg_unsupported.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/ort.json b/crates/tract-harvest-probe/fixtures/tiny/ort.json index b716d9119..9d5472beb 100644 --- a/crates/tract-harvest-probe/fixtures/tiny/ort.json +++ b/crates/tract-harvest-probe/fixtures/tiny/ort.json @@ -9,24 +9,12 @@ "S": { "shape": [], "dtype": "int32", - "bits": [ - 14 - ] + "bits": [14] }, "Q": { - "shape": [ - 2, - 3 - ], + "shape": [2,3], "dtype": "float32", - "bits": [ - 1079882711, - 3200253952, - 1066574966, - 1056625640, - 3225243783, - 1056431862 - ] + "bits": [1079882711,3200253952,1066574966,1056625640,3225243783,1056431862] } } }, @@ -36,24 +24,12 @@ "S": { "shape": [], "dtype": "int32", - "bits": [ - 14 - ] + "bits": [14] }, "Q": { - "shape": [ - 2, - 3 - ], + "shape": [2,3], "dtype": "float32", - "bits": [ - 1079882711, - 3200253952, - 1060623221, - 1056625640, - 3225243783, - 1055740287 - ] + "bits": [1079882711,3200253952,1060623221,1056625640,3225243783,1055740287] } } }, @@ -63,24 +39,12 @@ "S": { "shape": [], "dtype": "int32", - "bits": [ - 14 - ] + "bits": [14] }, "Q": { - "shape": [ - 2, - 3 - ], + "shape": [2,3], "dtype": "float32", - "bits": [ - 1079882711, - 3200253952, - 1068827891, - 1056625640, - 3225243783, - 1052692391 - ] + "bits": [1079882711,3200253952,1068827891,1056625640,3225243783,1052692391] } } }, @@ -94,4192 +58,623 @@ "S": { "shape": [], "dtype": "int32", - "bits": [ - 17 - ] + "bits": [17] }, "Q": { - "shape": [ - 2, - 3 - ], + "shape": [2,3], "dtype": "float32", - "bits": [ - 1079882711, - 3200253952, - 1066574966, - 1056625640, - 3225243783, - 1056431862 - ] + "bits": [1079882711,3200253952,1066574966,1056625640,3225243783,1056431862] } } }, "neg_unsupported": { - "sha256": "cf116e70da8451620056b44947d69f3b6d24652039fef91deddaec879b01528c", + "sha256": "25f0686d588c4e3e0c944d5aa1c305061ba9a6c93fc924dbc9bce822045b117c", "ort": { "S": { "shape": [], "dtype": "int32", - "bits": [ - 14 - ] + "bits": [14] }, "Q": { - "shape": [ - 2, - 3 - ], + "shape": [2,3], "dtype": "float32", - "bits": [ - 1073741638, - 3200378321, - 1048576000, - 1057301546, - 3204448256, - 4290772992 - ] + "bits": [1076490644,4290772992,1056403902,4290772992,3213093297,4290772992] } } }, "neg_domain": { "sha256": "2e90e4e9e24c58403cce3e0333ffb035bbc13dfbc4108976470887b7fa66738b", "ort_error": "Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: com.example:Abs(-1) is not a registered function/op" + }, + "erf": { + "sha256": "150ecd6c47b490e27dfb96d9bb72382b98d95816615d88a23f7a726473db3bd6", + "ort": { + "S": { + "shape": [], + "dtype": "int32", + "bits": [14] + }, + "Q": { + "shape": [2,3], + "dtype": "float32", + "bits": [1073741638,3200378321,1048576000,1057301546,3204448256,4290772992] + }, + "A": { + "shape": [2,3], + "dtype": "float32", + "bits": [1065352845,1058200660,1065353216,1032486193,1065353216,3212836862] + } + } + }, + "shape_fold": { + "sha256": "887757977da634f903c3f6d8be3ceee737263a6d75b3c07353806b586886ebb4", + "ort": { + "R": { + "shape": [3,2], + "dtype": "float32", + "bits": [1069547520,3196059648,1077936128,1036831949,1088421888,3223322624] + }, + "U": { + "shape": [1,2,3], + "dtype": "float32", + "bits": [1069547520,3196059648,1077936128,1036831949,1088421888,3223322624] + } + } + }, + "shape_sym": { + "sha256": "09f347cf277da59ef1785103cd9208b9611a747a1ffbf774f659fc702d4ee07f", + "ort": { + "R": { + "shape": [6,1], + "dtype": "float32", + "bits": [0,1065353216,1073741824,1077936128,1082130432,1084227584] + } + } + }, + "neg_gather_index": { + "sha256": "b630d91193b2a8195f6591b1818575744ced4d7fb54a5a67874a5bece7354e70", + "ort_error": "InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Gather node. Name:'n_gather' Status Message: indices element out of data bounds, idx=5 must be within the inclus" + }, + "neg_concat": { + "sha256": "f92f5872d18536685cf38fa0a627ceb7fdd7fda6bf908113d1127358c647aed2", + "ort_error": "Fail: [ONNXRuntimeError] : 1 : FAIL : Node (n_concat) Op (Concat) [ShapeInferenceError] Can't merge shape info. Both inferred and declared dimension have values but they differ. Inferred=2 Declared=3 Dimens" } }, + "erf_sweep": { + "sha256": "0a8fedb70fb7f4844eaf03aa33ac49a34c4fea199f9326416e83cf4dfe9f22b5", + "X_bits": [3233808384,3233802240,3233796096,3233789952,3233783808,3233777664,3233771520,3233765376,3233759232,3233753088,3233746944,3233740800,3233734656,3233728512,3233722368,3233716224,3233710080,3233703936,3233697792,3233691648,3233685504,3233679360,3233673216,3233667072,3233660928,3233654784,3233648640,3233642496,3233636352,3233630208,3233624064,3233617920,3233611776,3233605632,3233599488,3233593344,3233587200,3233581056,3233574912,3233568768,3233562624,3233556480,3233550336,3233544192,3233538048,3233531904,3233525760,3233519616,3233513472,3233507328,3233501184,3233495040,3233488896,3233482752,3233476608,3233470464,3233464320,3233458176,3233452032,3233445888,3233439744,3233433600,3233427456,3233421312,3233415168,3233409024,3233402880,3233396736,3233390592,3233384448,3233378304,3233372160,3233366016,3233359872,3233353728,3233347584,3233341440,3233335296,3233329152,3233323008,3233316864,3233310720,3233304576,3233298432,3233292288,3233286144,3233280000,3233273856,3233267712,3233261568,3233255424,3233249280,3233243136,3233236992,3233230848,3233224704,3233218560,3233212416,3233206272,3233200128,3233193984,3233187840,3233181696,3233175552,3233169408,3233163264,3233157120,3233150976,3233144832,3233138688,3233132544,3233126400,3233120256,3233114112,3233107968,3233101824,3233095680,3233089536,3233083392,3233077248,3233071104,3233064960,3233058816,3233052672,3233046528,3233040384,3233034240,3233028096,3233021952,3233015808,3233009664,3233003520,3232997376,3232991232,3232985088,3232978944,3232972800,3232966656,3232960512,3232954368,3232948224,3232942080,3232935936,3232929792,3232923648,3232917504,3232911360,3232905216,3232899072,3232892928,3232886784,3232880640,3232874496,3232868352,3232862208,3232856064,3232849920,3232843776,3232837632,3232831488,3232825344,3232819200,3232813056,3232806912,3232800768,3232794624,3232788480,3232782336,3232776192,3232770048,3232763904,3232757760,3232751616,3232745472,3232739328,3232733184,3232727040,3232720896,3232714752,3232708608,3232702464,3232696320,3232690176,3232684032,3232677888,3232671744,3232665600,3232659456,3232653312,3232647168,3232641024,3232634880,3232628736,3232622592,3232616448,3232610304,3232604160,3232598016,3232591872,3232585728,3232579584,3232573440,3232567296,3232561152,3232555008,3232548864,3232542720,3232536576,3232530432,3232524288,3232518144,3232512000,3232505856,3232499712,3232493568,3232487424,3232481280,3232475136,3232468992,3232462848,3232456704,3232450560,3232444416,3232438272,3232432128,3232425984,3232419840,3232413696,3232407552,3232401408,3232395264,3232389120,3232382976,3232376832,3232370688,3232364544,3232358400,3232352256,3232346112,3232339968,3232333824,3232327680,3232321536,3232315392,3232309248,3232303104,3232296960,3232290816,3232284672,3232278528,3232272384,3232266240,3232260096,3232253952,3232247808,3232241664,3232235520,3232229376,3232223232,3232217088,3232210944,3232204800,3232198656,3232192512,3232186368,3232180224,3232174080,3232167936,3232161792,3232155648,3232149504,3232143360,3232137216,3232131072,3232124928,3232118784,3232112640,3232106496,3232100352,3232094208,3232088064,3232081920,3232075776,3232069632,3232063488,3232057344,3232051200,3232045056,3232038912,3232032768,3232026624,3232020480,3232014336,3232008192,3232002048,3231995904,3231989760,3231983616,3231977472,3231971328,3231965184,3231959040,3231952896,3231946752,3231940608,3231934464,3231928320,3231922176,3231916032,3231909888,3231903744,3231897600,3231891456,3231885312,3231879168,3231873024,3231866880,3231860736,3231854592,3231848448,3231842304,3231836160,3231830016,3231823872,3231817728,3231811584,3231805440,3231799296,3231793152,3231787008,3231780864,3231774720,3231768576,3231762432,3231756288,3231750144,3231744000,3231737856,3231731712,3231725568,3231719424,3231713280,3231707136,3231700992,3231694848,3231688704,3231682560,3231676416,3231670272,3231664128,3231657984,3231651840,3231645696,3231639552,3231633408,3231627264,3231621120,3231614976,3231608832,3231602688,3231596544,3231590400,3231584256,3231578112,3231571968,3231565824,3231559680,3231553536,3231547392,3231541248,3231535104,3231528960,3231522816,3231516672,3231510528,3231504384,3231498240,3231492096,3231485952,3231479808,3231473664,3231467520,3231461376,3231455232,3231449088,3231442944,3231436800,3231430656,3231424512,3231418368,3231412224,3231406080,3231399936,3231393792,3231387648,3231381504,3231375360,3231369216,3231363072,3231356928,3231350784,3231344640,3231338496,3231332352,3231326208,3231320064,3231313920,3231307776,3231301632,3231295488,3231289344,3231283200,3231277056,3231270912,3231264768,3231258624,3231252480,3231246336,3231240192,3231234048,3231227904,3231221760,3231215616,3231209472,3231203328,3231197184,3231191040,3231184896,3231178752,3231172608,3231166464,3231160320,3231154176,3231148032,3231141888,3231135744,3231129600,3231123456,3231117312,3231111168,3231105024,3231098880,3231092736,3231086592,3231080448,3231074304,3231068160,3231062016,3231055872,3231049728,3231043584,3231037440,3231031296,3231025152,3231019008,3231012864,3231006720,3231000576,3230994432,3230988288,3230982144,3230976000,3230969856,3230963712,3230957568,3230951424,3230945280,3230939136,3230932992,3230926848,3230920704,3230914560,3230908416,3230902272,3230896128,3230889984,3230883840,3230877696,3230871552,3230865408,3230859264,3230853120,3230846976,3230840832,3230834688,3230828544,3230822400,3230816256,3230810112,3230803968,3230797824,3230791680,3230785536,3230779392,3230773248,3230767104,3230760960,3230754816,3230748672,3230742528,3230736384,3230730240,3230724096,3230717952,3230711808,3230705664,3230699520,3230693376,3230687232,3230681088,3230674944,3230668800,3230662656,3230656512,3230650368,3230644224,3230638080,3230631936,3230625792,3230619648,3230613504,3230607360,3230601216,3230595072,3230588928,3230582784,3230576640,3230570496,3230564352,3230558208,3230552064,3230545920,3230539776,3230533632,3230527488,3230521344,3230515200,3230509056,3230502912,3230496768,3230490624,3230484480,3230478336,3230472192,3230466048,3230459904,3230453760,3230447616,3230441472,3230435328,3230429184,3230423040,3230416896,3230410752,3230404608,3230398464,3230392320,3230386176,3230380032,3230373888,3230367744,3230361600,3230355456,3230349312,3230343168,3230337024,3230330880,3230324736,3230318592,3230312448,3230306304,3230300160,3230294016,3230287872,3230281728,3230275584,3230269440,3230263296,3230257152,3230251008,3230244864,3230238720,3230232576,3230226432,3230220288,3230214144,3230208000,3230201856,3230195712,3230189568,3230183424,3230177280,3230171136,3230164992,3230158848,3230152704,3230146560,3230140416,3230134272,3230128128,3230121984,3230115840,3230109696,3230103552,3230097408,3230091264,3230085120,3230078976,3230072832,3230066688,3230060544,3230054400,3230048256,3230042112,3230035968,3230029824,3230023680,3230017536,3230011392,3230005248,3229999104,3229992960,3229986816,3229980672,3229974528,3229968384,3229962240,3229956096,3229949952,3229943808,3229937664,3229931520,3229925376,3229919232,3229913088,3229906944,3229900800,3229894656,3229888512,3229882368,3229876224,3229870080,3229863936,3229857792,3229851648,3229845504,3229839360,3229833216,3229827072,3229820928,3229814784,3229808640,3229802496,3229796352,3229790208,3229784064,3229777920,3229771776,3229765632,3229759488,3229753344,3229747200,3229741056,3229734912,3229728768,3229722624,3229716480,3229710336,3229704192,3229698048,3229691904,3229685760,3229679616,3229673472,3229667328,3229661184,3229655040,3229648896,3229642752,3229636608,3229630464,3229624320,3229618176,3229609984,3229597696,3229585408,3229573120,3229560832,3229548544,3229536256,3229523968,3229511680,3229499392,3229487104,3229474816,3229462528,3229450240,3229437952,3229425664,3229413376,3229401088,3229388800,3229376512,3229364224,3229351936,3229339648,3229327360,3229315072,3229302784,3229290496,3229278208,3229265920,3229253632,3229241344,3229229056,3229216768,3229204480,3229192192,3229179904,3229167616,3229155328,3229143040,3229130752,3229118464,3229106176,3229093888,3229081600,3229069312,3229057024,3229044736,3229032448,3229020160,3229007872,3228995584,3228983296,3228971008,3228958720,3228946432,3228934144,3228921856,3228909568,3228897280,3228884992,3228872704,3228860416,3228848128,3228835840,3228823552,3228811264,3228798976,3228786688,3228774400,3228762112,3228749824,3228737536,3228725248,3228712960,3228700672,3228688384,3228676096,3228663808,3228651520,3228639232,3228626944,3228614656,3228602368,3228590080,3228577792,3228565504,3228553216,3228540928,3228528640,3228516352,3228504064,3228491776,3228479488,3228467200,3228454912,3228442624,3228430336,3228418048,3228405760,3228393472,3228381184,3228368896,3228356608,3228344320,3228332032,3228319744,3228307456,3228295168,3228282880,3228270592,3228258304,3228246016,3228233728,3228221440,3228209152,3228196864,3228184576,3228172288,3228160000,3228147712,3228135424,3228123136,3228110848,3228098560,3228086272,3228073984,3228061696,3228049408,3228037120,3228024832,3228012544,3228000256,3227987968,3227975680,3227963392,3227951104,3227938816,3227926528,3227914240,3227901952,3227889664,3227877376,3227865088,3227852800,3227840512,3227828224,3227815936,3227803648,3227791360,3227779072,3227766784,3227754496,3227742208,3227729920,3227717632,3227705344,3227693056,3227680768,3227668480,3227656192,3227643904,3227631616,3227619328,3227607040,3227594752,3227582464,3227570176,3227557888,3227545600,3227533312,3227521024,3227508736,3227496448,3227484160,3227471872,3227459584,3227447296,3227435008,3227422720,3227410432,3227398144,3227385856,3227373568,3227361280,3227348992,3227336704,3227324416,3227312128,3227299840,3227287552,3227275264,3227262976,3227250688,3227238400,3227226112,3227213824,3227201536,3227189248,3227176960,3227164672,3227152384,3227140096,3227127808,3227115520,3227103232,3227090944,3227078656,3227066368,3227054080,3227041792,3227029504,3227017216,3227004928,3226992640,3226980352,3226968064,3226955776,3226943488,3226931200,3226918912,3226906624,3226894336,3226882048,3226869760,3226857472,3226845184,3226832896,3226820608,3226808320,3226796032,3226783744,3226771456,3226759168,3226746880,3226734592,3226722304,3226710016,3226697728,3226685440,3226673152,3226660864,3226648576,3226636288,3226624000,3226611712,3226599424,3226587136,3226574848,3226562560,3226550272,3226537984,3226525696,3226513408,3226501120,3226488832,3226476544,3226464256,3226451968,3226439680,3226427392,3226415104,3226402816,3226390528,3226378240,3226365952,3226353664,3226341376,3226329088,3226316800,3226304512,3226292224,3226279936,3226267648,3226255360,3226243072,3226230784,3226218496,3226206208,3226193920,3226181632,3226169344,3226157056,3226144768,3226132480,3226120192,3226107904,3226095616,3226083328,3226071040,3226058752,3226046464,3226034176,3226021888,3226009600,3225997312,3225985024,3225972736,3225960448,3225948160,3225935872,3225923584,3225911296,3225899008,3225886720,3225874432,3225862144,3225849856,3225837568,3225825280,3225812992,3225800704,3225788416,3225776128,3225763840,3225751552,3225739264,3225726976,3225714688,3225702400,3225690112,3225677824,3225665536,3225653248,3225640960,3225628672,3225616384,3225604096,3225591808,3225579520,3225567232,3225554944,3225542656,3225530368,3225518080,3225505792,3225493504,3225481216,3225468928,3225456640,3225444352,3225432064,3225419776,3225407488,3225395200,3225382912,3225370624,3225358336,3225346048,3225333760,3225321472,3225309184,3225296896,3225284608,3225272320,3225260032,3225247744,3225235456,3225223168,3225210880,3225198592,3225186304,3225174016,3225161728,3225149440,3225137152,3225124864,3225112576,3225100288,3225088000,3225075712,3225063424,3225051136,3225038848,3225026560,3225014272,3225001984,3224989696,3224977408,3224965120,3224952832,3224940544,3224928256,3224915968,3224903680,3224891392,3224879104,3224866816,3224854528,3224842240,3224829952,3224817664,3224805376,3224793088,3224780800,3224768512,3224756224,3224743936,3224731648,3224719360,3224707072,3224694784,3224682496,3224670208,3224657920,3224645632,3224633344,3224621056,3224608768,3224596480,3224584192,3224571904,3224559616,3224547328,3224535040,3224522752,3224510464,3224498176,3224485888,3224473600,3224461312,3224449024,3224436736,3224424448,3224412160,3224399872,3224387584,3224375296,3224363008,3224350720,3224338432,3224326144,3224313856,3224301568,3224289280,3224276992,3224264704,3224252416,3224240128,3224227840,3224215552,3224203264,3224190976,3224178688,3224166400,3224154112,3224141824,3224129536,3224117248,3224104960,3224092672,3224080384,3224068096,3224055808,3224043520,3224031232,3224018944,3224006656,3223994368,3223982080,3223969792,3223957504,3223945216,3223932928,3223920640,3223908352,3223896064,3223883776,3223871488,3223859200,3223846912,3223834624,3223822336,3223810048,3223797760,3223785472,3223773184,3223760896,3223748608,3223736320,3223724032,3223711744,3223699456,3223687168,3223674880,3223662592,3223650304,3223638016,3223625728,3223613440,3223601152,3223588864,3223576576,3223564288,3223552000,3223539712,3223527424,3223515136,3223502848,3223490560,3223478272,3223465984,3223453696,3223441408,3223429120,3223416832,3223404544,3223392256,3223379968,3223367680,3223355392,3223343104,3223330816,3223318528,3223306240,3223293952,3223281664,3223269376,3223257088,3223244800,3223232512,3223220224,3223207936,3223195648,3223183360,3223171072,3223158784,3223146496,3223134208,3223121920,3223109632,3223097344,3223085056,3223072768,3223060480,3223048192,3223035904,3223023616,3223011328,3222999040,3222986752,3222974464,3222962176,3222949888,3222937600,3222925312,3222913024,3222900736,3222888448,3222876160,3222863872,3222851584,3222839296,3222827008,3222814720,3222802432,3222790144,3222777856,3222765568,3222753280,3222740992,3222728704,3222716416,3222704128,3222691840,3222679552,3222667264,3222654976,3222642688,3222630400,3222618112,3222605824,3222593536,3222581248,3222568960,3222556672,3222544384,3222532096,3222519808,3222507520,3222495232,3222482944,3222470656,3222458368,3222446080,3222433792,3222421504,3222409216,3222396928,3222384640,3222372352,3222360064,3222347776,3222335488,3222323200,3222310912,3222298624,3222286336,3222274048,3222261760,3222249472,3222237184,3222224896,3222212608,3222200320,3222188032,3222175744,3222163456,3222151168,3222138880,3222126592,3222114304,3222102016,3222089728,3222077440,3222065152,3222052864,3222040576,3222028288,3222016000,3222003712,3221991424,3221979136,3221966848,3221954560,3221942272,3221929984,3221917696,3221905408,3221893120,3221880832,3221868544,3221856256,3221843968,3221831680,3221819392,3221807104,3221794816,3221782528,3221770240,3221757952,3221745664,3221733376,3221721088,3221708800,3221696512,3221684224,3221671936,3221659648,3221647360,3221635072,3221622784,3221610496,3221598208,3221585920,3221573632,3221561344,3221549056,3221536768,3221524480,3221512192,3221499904,3221487616,3221475328,3221463040,3221450752,3221438464,3221426176,3221413888,3221401600,3221389312,3221377024,3221364736,3221352448,3221340160,3221327872,3221315584,3221303296,3221291008,3221278720,3221266432,3221254144,3221241856,3221229568,3221209088,3221184512,3221159936,3221135360,3221110784,3221086208,3221061632,3221037056,3221012480,3220987904,3220963328,3220938752,3220914176,3220889600,3220865024,3220840448,3220815872,3220791296,3220766720,3220742144,3220717568,3220692992,3220668416,3220643840,3220619264,3220594688,3220570112,3220545536,3220520960,3220496384,3220471808,3220447232,3220422656,3220398080,3220373504,3220348928,3220324352,3220299776,3220275200,3220250624,3220226048,3220201472,3220176896,3220152320,3220127744,3220103168,3220078592,3220054016,3220029440,3220004864,3219980288,3219955712,3219931136,3219906560,3219881984,3219857408,3219832832,3219808256,3219783680,3219759104,3219734528,3219709952,3219685376,3219660800,3219636224,3219611648,3219587072,3219562496,3219537920,3219513344,3219488768,3219464192,3219439616,3219415040,3219390464,3219365888,3219341312,3219316736,3219292160,3219267584,3219243008,3219218432,3219193856,3219169280,3219144704,3219120128,3219095552,3219070976,3219046400,3219021824,3218997248,3218972672,3218948096,3218923520,3218898944,3218874368,3218849792,3218825216,3218800640,3218776064,3218751488,3218726912,3218702336,3218677760,3218653184,3218628608,3218604032,3218579456,3218554880,3218530304,3218505728,3218481152,3218456576,3218432000,3218407424,3218382848,3218358272,3218333696,3218309120,3218284544,3218259968,3218235392,3218210816,3218186240,3218161664,3218137088,3218112512,3218087936,3218063360,3218038784,3218014208,3217989632,3217965056,3217940480,3217915904,3217891328,3217866752,3217842176,3217817600,3217793024,3217768448,3217743872,3217719296,3217694720,3217670144,3217645568,3217620992,3217596416,3217571840,3217547264,3217522688,3217498112,3217473536,3217448960,3217424384,3217399808,3217375232,3217350656,3217326080,3217301504,3217276928,3217252352,3217227776,3217203200,3217178624,3217154048,3217129472,3217104896,3217080320,3217055744,3217031168,3217006592,3216982016,3216957440,3216932864,3216908288,3216883712,3216859136,3216834560,3216809984,3216785408,3216760832,3216736256,3216711680,3216687104,3216662528,3216637952,3216613376,3216588800,3216564224,3216539648,3216515072,3216490496,3216465920,3216441344,3216416768,3216392192,3216367616,3216343040,3216318464,3216293888,3216269312,3216244736,3216220160,3216195584,3216171008,3216146432,3216121856,3216097280,3216072704,3216048128,3216023552,3215998976,3215974400,3215949824,3215925248,3215900672,3215876096,3215851520,3215826944,3215802368,3215777792,3215753216,3215728640,3215704064,3215679488,3215654912,3215630336,3215605760,3215581184,3215556608,3215532032,3215507456,3215482880,3215458304,3215433728,3215409152,3215384576,3215360000,3215335424,3215310848,3215286272,3215261696,3215237120,3215212544,3215187968,3215163392,3215138816,3215114240,3215089664,3215065088,3215040512,3215015936,3214991360,3214966784,3214942208,3214917632,3214893056,3214868480,3214843904,3214819328,3214794752,3214770176,3214745600,3214721024,3214696448,3214671872,3214647296,3214622720,3214598144,3214573568,3214548992,3214524416,3214499840,3214475264,3214450688,3214426112,3214401536,3214376960,3214352384,3214327808,3214303232,3214278656,3214254080,3214229504,3214204928,3214180352,3214155776,3214131200,3214106624,3214082048,3214057472,3214032896,3214008320,3213983744,3213959168,3213934592,3213910016,3213885440,3213860864,3213836288,3213811712,3213787136,3213762560,3213737984,3213713408,3213688832,3213664256,3213639680,3213615104,3213590528,3213565952,3213541376,3213516800,3213492224,3213467648,3213443072,3213418496,3213393920,3213369344,3213344768,3213320192,3213295616,3213271040,3213246464,3213221888,3213197312,3213172736,3213148160,3213123584,3213099008,3213074432,3213049856,3213025280,3213000704,3212976128,3212951552,3212926976,3212902400,3212877824,3212853248,3212820480,3212771328,3212722176,3212673024,3212623872,3212574720,3212525568,3212476416,3212427264,3212378112,3212328960,3212279808,3212230656,3212181504,3212132352,3212083200,3212034048,3211984896,3211935744,3211886592,3211837440,3211788288,3211739136,3211689984,3211640832,3211591680,3211542528,3211493376,3211444224,3211395072,3211345920,3211296768,3211247616,3211198464,3211149312,3211100160,3211051008,3211001856,3210952704,3210903552,3210854400,3210805248,3210756096,3210706944,3210657792,3210608640,3210559488,3210510336,3210461184,3210412032,3210362880,3210313728,3210264576,3210215424,3210166272,3210117120,3210067968,3210018816,3209969664,3209920512,3209871360,3209822208,3209773056,3209723904,3209674752,3209625600,3209576448,3209527296,3209478144,3209428992,3209379840,3209330688,3209281536,3209232384,3209183232,3209134080,3209084928,3209035776,3208986624,3208937472,3208888320,3208839168,3208790016,3208740864,3208691712,3208642560,3208593408,3208544256,3208495104,3208445952,3208396800,3208347648,3208298496,3208249344,3208200192,3208151040,3208101888,3208052736,3208003584,3207954432,3207905280,3207856128,3207806976,3207757824,3207708672,3207659520,3207610368,3207561216,3207512064,3207462912,3207413760,3207364608,3207315456,3207266304,3207217152,3207168000,3207118848,3207069696,3207020544,3206971392,3206922240,3206873088,3206823936,3206774784,3206725632,3206676480,3206627328,3206578176,3206529024,3206479872,3206430720,3206381568,3206332416,3206283264,3206234112,3206184960,3206135808,3206086656,3206037504,3205988352,3205939200,3205890048,3205840896,3205791744,3205742592,3205693440,3205644288,3205595136,3205545984,3205496832,3205447680,3205398528,3205349376,3205300224,3205251072,3205201920,3205152768,3205103616,3205054464,3205005312,3204956160,3204907008,3204857856,3204808704,3204759552,3204710400,3204661248,3204612096,3204562944,3204513792,3204464640,3204382720,3204284416,3204186112,3204087808,3203989504,3203891200,3203792896,3203694592,3203596288,3203497984,3203399680,3203301376,3203203072,3203104768,3203006464,3202908160,3202809856,3202711552,3202613248,3202514944,3202416640,3202318336,3202220032,3202121728,3202023424,3201925120,3201826816,3201728512,3201630208,3201531904,3201433600,3201335296,3201236992,3201138688,3201040384,3200942080,3200843776,3200745472,3200647168,3200548864,3200450560,3200352256,3200253952,3200155648,3200057344,3199959040,3199860736,3199762432,3199664128,3199565824,3199467520,3199369216,3199270912,3199172608,3199074304,3198976000,3198877696,3198779392,3198681088,3198582784,3198484480,3198386176,3198287872,3198189568,3198091264,3197992960,3197894656,3197796352,3197698048,3197599744,3197501440,3197403136,3197304832,3197206528,3197108224,3197009920,3196911616,3196813312,3196715008,3196616704,3196518400,3196420096,3196321792,3196223488,3196125184,3195994112,3195797504,3195600896,3195404288,3195207680,3195011072,3194814464,3194617856,3194421248,3194224640,3194028032,3193831424,3193634816,3193438208,3193241600,3193044992,3192848384,3192651776,3192455168,3192258560,3192061952,3191865344,3191668736,3191472128,3191275520,3191078912,3190882304,3190685696,3190489088,3190292480,3190095872,3189899264,3189702656,3189506048,3189309440,3189112832,3188916224,3188719616,3188523008,3188326400,3188129792,3187933184,3187736576,3187408896,3187015680,3186622464,3186229248,3185836032,3185442816,3185049600,3184656384,3184263168,3183869952,3183476736,3183083520,3182690304,3182297088,3181903872,3181510656,3181117440,3180724224,3180331008,3179937792,3179544576,3179020288,3178233856,3177447424,3176660992,3175874560,3175088128,3174301696,3173515264,3172728832,3171942400,3171155968,3169845248,3168272384,3166699520,3165126656,3163553792,3161456640,3158310912,3155165184,3149922304,3141533696,0,994050048,1002438656,1007681536,1010827264,1013972992,1016070144,1017643008,1019215872,1020788736,1022361600,1023672320,1024458752,1025245184,1026031616,1026818048,1027604480,1028390912,1029177344,1029963776,1030750208,1031536640,1032060928,1032454144,1032847360,1033240576,1033633792,1034027008,1034420224,1034813440,1035206656,1035599872,1035993088,1036386304,1036779520,1037172736,1037565952,1037959168,1038352384,1038745600,1039138816,1039532032,1039925248,1040252928,1040449536,1040646144,1040842752,1041039360,1041235968,1041432576,1041629184,1041825792,1042022400,1042219008,1042415616,1042612224,1042808832,1043005440,1043202048,1043398656,1043595264,1043791872,1043988480,1044185088,1044381696,1044578304,1044774912,1044971520,1045168128,1045364736,1045561344,1045757952,1045954560,1046151168,1046347776,1046544384,1046740992,1046937600,1047134208,1047330816,1047527424,1047724032,1047920640,1048117248,1048313856,1048510464,1048641536,1048739840,1048838144,1048936448,1049034752,1049133056,1049231360,1049329664,1049427968,1049526272,1049624576,1049722880,1049821184,1049919488,1050017792,1050116096,1050214400,1050312704,1050411008,1050509312,1050607616,1050705920,1050804224,1050902528,1051000832,1051099136,1051197440,1051295744,1051394048,1051492352,1051590656,1051688960,1051787264,1051885568,1051983872,1052082176,1052180480,1052278784,1052377088,1052475392,1052573696,1052672000,1052770304,1052868608,1052966912,1053065216,1053163520,1053261824,1053360128,1053458432,1053556736,1053655040,1053753344,1053851648,1053949952,1054048256,1054146560,1054244864,1054343168,1054441472,1054539776,1054638080,1054736384,1054834688,1054932992,1055031296,1055129600,1055227904,1055326208,1055424512,1055522816,1055621120,1055719424,1055817728,1055916032,1056014336,1056112640,1056210944,1056309248,1056407552,1056505856,1056604160,1056702464,1056800768,1056899072,1056980992,1057030144,1057079296,1057128448,1057177600,1057226752,1057275904,1057325056,1057374208,1057423360,1057472512,1057521664,1057570816,1057619968,1057669120,1057718272,1057767424,1057816576,1057865728,1057914880,1057964032,1058013184,1058062336,1058111488,1058160640,1058209792,1058258944,1058308096,1058357248,1058406400,1058455552,1058504704,1058553856,1058603008,1058652160,1058701312,1058750464,1058799616,1058848768,1058897920,1058947072,1058996224,1059045376,1059094528,1059143680,1059192832,1059241984,1059291136,1059340288,1059389440,1059438592,1059487744,1059536896,1059586048,1059635200,1059684352,1059733504,1059782656,1059831808,1059880960,1059930112,1059979264,1060028416,1060077568,1060126720,1060175872,1060225024,1060274176,1060323328,1060372480,1060421632,1060470784,1060519936,1060569088,1060618240,1060667392,1060716544,1060765696,1060814848,1060864000,1060913152,1060962304,1061011456,1061060608,1061109760,1061158912,1061208064,1061257216,1061306368,1061355520,1061404672,1061453824,1061502976,1061552128,1061601280,1061650432,1061699584,1061748736,1061797888,1061847040,1061896192,1061945344,1061994496,1062043648,1062092800,1062141952,1062191104,1062240256,1062289408,1062338560,1062387712,1062436864,1062486016,1062535168,1062584320,1062633472,1062682624,1062731776,1062780928,1062830080,1062879232,1062928384,1062977536,1063026688,1063075840,1063124992,1063174144,1063223296,1063272448,1063321600,1063370752,1063419904,1063469056,1063518208,1063567360,1063616512,1063665664,1063714816,1063763968,1063813120,1063862272,1063911424,1063960576,1064009728,1064058880,1064108032,1064157184,1064206336,1064255488,1064304640,1064353792,1064402944,1064452096,1064501248,1064550400,1064599552,1064648704,1064697856,1064747008,1064796160,1064845312,1064894464,1064943616,1064992768,1065041920,1065091072,1065140224,1065189376,1065238528,1065287680,1065336832,1065369600,1065394176,1065418752,1065443328,1065467904,1065492480,1065517056,1065541632,1065566208,1065590784,1065615360,1065639936,1065664512,1065689088,1065713664,1065738240,1065762816,1065787392,1065811968,1065836544,1065861120,1065885696,1065910272,1065934848,1065959424,1065984000,1066008576,1066033152,1066057728,1066082304,1066106880,1066131456,1066156032,1066180608,1066205184,1066229760,1066254336,1066278912,1066303488,1066328064,1066352640,1066377216,1066401792,1066426368,1066450944,1066475520,1066500096,1066524672,1066549248,1066573824,1066598400,1066622976,1066647552,1066672128,1066696704,1066721280,1066745856,1066770432,1066795008,1066819584,1066844160,1066868736,1066893312,1066917888,1066942464,1066967040,1066991616,1067016192,1067040768,1067065344,1067089920,1067114496,1067139072,1067163648,1067188224,1067212800,1067237376,1067261952,1067286528,1067311104,1067335680,1067360256,1067384832,1067409408,1067433984,1067458560,1067483136,1067507712,1067532288,1067556864,1067581440,1067606016,1067630592,1067655168,1067679744,1067704320,1067728896,1067753472,1067778048,1067802624,1067827200,1067851776,1067876352,1067900928,1067925504,1067950080,1067974656,1067999232,1068023808,1068048384,1068072960,1068097536,1068122112,1068146688,1068171264,1068195840,1068220416,1068244992,1068269568,1068294144,1068318720,1068343296,1068367872,1068392448,1068417024,1068441600,1068466176,1068490752,1068515328,1068539904,1068564480,1068589056,1068613632,1068638208,1068662784,1068687360,1068711936,1068736512,1068761088,1068785664,1068810240,1068834816,1068859392,1068883968,1068908544,1068933120,1068957696,1068982272,1069006848,1069031424,1069056000,1069080576,1069105152,1069129728,1069154304,1069178880,1069203456,1069228032,1069252608,1069277184,1069301760,1069326336,1069350912,1069375488,1069400064,1069424640,1069449216,1069473792,1069498368,1069522944,1069547520,1069572096,1069596672,1069621248,1069645824,1069670400,1069694976,1069719552,1069744128,1069768704,1069793280,1069817856,1069842432,1069867008,1069891584,1069916160,1069940736,1069965312,1069989888,1070014464,1070039040,1070063616,1070088192,1070112768,1070137344,1070161920,1070186496,1070211072,1070235648,1070260224,1070284800,1070309376,1070333952,1070358528,1070383104,1070407680,1070432256,1070456832,1070481408,1070505984,1070530560,1070555136,1070579712,1070604288,1070628864,1070653440,1070678016,1070702592,1070727168,1070751744,1070776320,1070800896,1070825472,1070850048,1070874624,1070899200,1070923776,1070948352,1070972928,1070997504,1071022080,1071046656,1071071232,1071095808,1071120384,1071144960,1071169536,1071194112,1071218688,1071243264,1071267840,1071292416,1071316992,1071341568,1071366144,1071390720,1071415296,1071439872,1071464448,1071489024,1071513600,1071538176,1071562752,1071587328,1071611904,1071636480,1071661056,1071685632,1071710208,1071734784,1071759360,1071783936,1071808512,1071833088,1071857664,1071882240,1071906816,1071931392,1071955968,1071980544,1072005120,1072029696,1072054272,1072078848,1072103424,1072128000,1072152576,1072177152,1072201728,1072226304,1072250880,1072275456,1072300032,1072324608,1072349184,1072373760,1072398336,1072422912,1072447488,1072472064,1072496640,1072521216,1072545792,1072570368,1072594944,1072619520,1072644096,1072668672,1072693248,1072717824,1072742400,1072766976,1072791552,1072816128,1072840704,1072865280,1072889856,1072914432,1072939008,1072963584,1072988160,1073012736,1073037312,1073061888,1073086464,1073111040,1073135616,1073160192,1073184768,1073209344,1073233920,1073258496,1073283072,1073307648,1073332224,1073356800,1073381376,1073405952,1073430528,1073455104,1073479680,1073504256,1073528832,1073553408,1073577984,1073602560,1073627136,1073651712,1073676288,1073700864,1073725440,1073745920,1073758208,1073770496,1073782784,1073795072,1073807360,1073819648,1073831936,1073844224,1073856512,1073868800,1073881088,1073893376,1073905664,1073917952,1073930240,1073942528,1073954816,1073967104,1073979392,1073991680,1074003968,1074016256,1074028544,1074040832,1074053120,1074065408,1074077696,1074089984,1074102272,1074114560,1074126848,1074139136,1074151424,1074163712,1074176000,1074188288,1074200576,1074212864,1074225152,1074237440,1074249728,1074262016,1074274304,1074286592,1074298880,1074311168,1074323456,1074335744,1074348032,1074360320,1074372608,1074384896,1074397184,1074409472,1074421760,1074434048,1074446336,1074458624,1074470912,1074483200,1074495488,1074507776,1074520064,1074532352,1074544640,1074556928,1074569216,1074581504,1074593792,1074606080,1074618368,1074630656,1074642944,1074655232,1074667520,1074679808,1074692096,1074704384,1074716672,1074728960,1074741248,1074753536,1074765824,1074778112,1074790400,1074802688,1074814976,1074827264,1074839552,1074851840,1074864128,1074876416,1074888704,1074900992,1074913280,1074925568,1074937856,1074950144,1074962432,1074974720,1074987008,1074999296,1075011584,1075023872,1075036160,1075048448,1075060736,1075073024,1075085312,1075097600,1075109888,1075122176,1075134464,1075146752,1075159040,1075171328,1075183616,1075195904,1075208192,1075220480,1075232768,1075245056,1075257344,1075269632,1075281920,1075294208,1075306496,1075318784,1075331072,1075343360,1075355648,1075367936,1075380224,1075392512,1075404800,1075417088,1075429376,1075441664,1075453952,1075466240,1075478528,1075490816,1075503104,1075515392,1075527680,1075539968,1075552256,1075564544,1075576832,1075589120,1075601408,1075613696,1075625984,1075638272,1075650560,1075662848,1075675136,1075687424,1075699712,1075712000,1075724288,1075736576,1075748864,1075761152,1075773440,1075785728,1075798016,1075810304,1075822592,1075834880,1075847168,1075859456,1075871744,1075884032,1075896320,1075908608,1075920896,1075933184,1075945472,1075957760,1075970048,1075982336,1075994624,1076006912,1076019200,1076031488,1076043776,1076056064,1076068352,1076080640,1076092928,1076105216,1076117504,1076129792,1076142080,1076154368,1076166656,1076178944,1076191232,1076203520,1076215808,1076228096,1076240384,1076252672,1076264960,1076277248,1076289536,1076301824,1076314112,1076326400,1076338688,1076350976,1076363264,1076375552,1076387840,1076400128,1076412416,1076424704,1076436992,1076449280,1076461568,1076473856,1076486144,1076498432,1076510720,1076523008,1076535296,1076547584,1076559872,1076572160,1076584448,1076596736,1076609024,1076621312,1076633600,1076645888,1076658176,1076670464,1076682752,1076695040,1076707328,1076719616,1076731904,1076744192,1076756480,1076768768,1076781056,1076793344,1076805632,1076817920,1076830208,1076842496,1076854784,1076867072,1076879360,1076891648,1076903936,1076916224,1076928512,1076940800,1076953088,1076965376,1076977664,1076989952,1077002240,1077014528,1077026816,1077039104,1077051392,1077063680,1077075968,1077088256,1077100544,1077112832,1077125120,1077137408,1077149696,1077161984,1077174272,1077186560,1077198848,1077211136,1077223424,1077235712,1077248000,1077260288,1077272576,1077284864,1077297152,1077309440,1077321728,1077334016,1077346304,1077358592,1077370880,1077383168,1077395456,1077407744,1077420032,1077432320,1077444608,1077456896,1077469184,1077481472,1077493760,1077506048,1077518336,1077530624,1077542912,1077555200,1077567488,1077579776,1077592064,1077604352,1077616640,1077628928,1077641216,1077653504,1077665792,1077678080,1077690368,1077702656,1077714944,1077727232,1077739520,1077751808,1077764096,1077776384,1077788672,1077800960,1077813248,1077825536,1077837824,1077850112,1077862400,1077874688,1077886976,1077899264,1077911552,1077923840,1077936128,1077948416,1077960704,1077972992,1077985280,1077997568,1078009856,1078022144,1078034432,1078046720,1078059008,1078071296,1078083584,1078095872,1078108160,1078120448,1078132736,1078145024,1078157312,1078169600,1078181888,1078194176,1078206464,1078218752,1078231040,1078243328,1078255616,1078267904,1078280192,1078292480,1078304768,1078317056,1078329344,1078341632,1078353920,1078366208,1078378496,1078390784,1078403072,1078415360,1078427648,1078439936,1078452224,1078464512,1078476800,1078489088,1078501376,1078513664,1078525952,1078538240,1078550528,1078562816,1078575104,1078587392,1078599680,1078611968,1078624256,1078636544,1078648832,1078661120,1078673408,1078685696,1078697984,1078710272,1078722560,1078734848,1078747136,1078759424,1078771712,1078784000,1078796288,1078808576,1078820864,1078833152,1078845440,1078857728,1078870016,1078882304,1078894592,1078906880,1078919168,1078931456,1078943744,1078956032,1078968320,1078980608,1078992896,1079005184,1079017472,1079029760,1079042048,1079054336,1079066624,1079078912,1079091200,1079103488,1079115776,1079128064,1079140352,1079152640,1079164928,1079177216,1079189504,1079201792,1079214080,1079226368,1079238656,1079250944,1079263232,1079275520,1079287808,1079300096,1079312384,1079324672,1079336960,1079349248,1079361536,1079373824,1079386112,1079398400,1079410688,1079422976,1079435264,1079447552,1079459840,1079472128,1079484416,1079496704,1079508992,1079521280,1079533568,1079545856,1079558144,1079570432,1079582720,1079595008,1079607296,1079619584,1079631872,1079644160,1079656448,1079668736,1079681024,1079693312,1079705600,1079717888,1079730176,1079742464,1079754752,1079767040,1079779328,1079791616,1079803904,1079816192,1079828480,1079840768,1079853056,1079865344,1079877632,1079889920,1079902208,1079914496,1079926784,1079939072,1079951360,1079963648,1079975936,1079988224,1080000512,1080012800,1080025088,1080037376,1080049664,1080061952,1080074240,1080086528,1080098816,1080111104,1080123392,1080135680,1080147968,1080160256,1080172544,1080184832,1080197120,1080209408,1080221696,1080233984,1080246272,1080258560,1080270848,1080283136,1080295424,1080307712,1080320000,1080332288,1080344576,1080356864,1080369152,1080381440,1080393728,1080406016,1080418304,1080430592,1080442880,1080455168,1080467456,1080479744,1080492032,1080504320,1080516608,1080528896,1080541184,1080553472,1080565760,1080578048,1080590336,1080602624,1080614912,1080627200,1080639488,1080651776,1080664064,1080676352,1080688640,1080700928,1080713216,1080725504,1080737792,1080750080,1080762368,1080774656,1080786944,1080799232,1080811520,1080823808,1080836096,1080848384,1080860672,1080872960,1080885248,1080897536,1080909824,1080922112,1080934400,1080946688,1080958976,1080971264,1080983552,1080995840,1081008128,1081020416,1081032704,1081044992,1081057280,1081069568,1081081856,1081094144,1081106432,1081118720,1081131008,1081143296,1081155584,1081167872,1081180160,1081192448,1081204736,1081217024,1081229312,1081241600,1081253888,1081266176,1081278464,1081290752,1081303040,1081315328,1081327616,1081339904,1081352192,1081364480,1081376768,1081389056,1081401344,1081413632,1081425920,1081438208,1081450496,1081462784,1081475072,1081487360,1081499648,1081511936,1081524224,1081536512,1081548800,1081561088,1081573376,1081585664,1081597952,1081610240,1081622528,1081634816,1081647104,1081659392,1081671680,1081683968,1081696256,1081708544,1081720832,1081733120,1081745408,1081757696,1081769984,1081782272,1081794560,1081806848,1081819136,1081831424,1081843712,1081856000,1081868288,1081880576,1081892864,1081905152,1081917440,1081929728,1081942016,1081954304,1081966592,1081978880,1081991168,1082003456,1082015744,1082028032,1082040320,1082052608,1082064896,1082077184,1082089472,1082101760,1082114048,1082126336,1082134528,1082140672,1082146816,1082152960,1082159104,1082165248,1082171392,1082177536,1082183680,1082189824,1082195968,1082202112,1082208256,1082214400,1082220544,1082226688,1082232832,1082238976,1082245120,1082251264,1082257408,1082263552,1082269696,1082275840,1082281984,1082288128,1082294272,1082300416,1082306560,1082312704,1082318848,1082324992,1082331136,1082337280,1082343424,1082349568,1082355712,1082361856,1082368000,1082374144,1082380288,1082386432,1082392576,1082398720,1082404864,1082411008,1082417152,1082423296,1082429440,1082435584,1082441728,1082447872,1082454016,1082460160,1082466304,1082472448,1082478592,1082484736,1082490880,1082497024,1082503168,1082509312,1082515456,1082521600,1082527744,1082533888,1082540032,1082546176,1082552320,1082558464,1082564608,1082570752,1082576896,1082583040,1082589184,1082595328,1082601472,1082607616,1082613760,1082619904,1082626048,1082632192,1082638336,1082644480,1082650624,1082656768,1082662912,1082669056,1082675200,1082681344,1082687488,1082693632,1082699776,1082705920,1082712064,1082718208,1082724352,1082730496,1082736640,1082742784,1082748928,1082755072,1082761216,1082767360,1082773504,1082779648,1082785792,1082791936,1082798080,1082804224,1082810368,1082816512,1082822656,1082828800,1082834944,1082841088,1082847232,1082853376,1082859520,1082865664,1082871808,1082877952,1082884096,1082890240,1082896384,1082902528,1082908672,1082914816,1082920960,1082927104,1082933248,1082939392,1082945536,1082951680,1082957824,1082963968,1082970112,1082976256,1082982400,1082988544,1082994688,1083000832,1083006976,1083013120,1083019264,1083025408,1083031552,1083037696,1083043840,1083049984,1083056128,1083062272,1083068416,1083074560,1083080704,1083086848,1083092992,1083099136,1083105280,1083111424,1083117568,1083123712,1083129856,1083136000,1083142144,1083148288,1083154432,1083160576,1083166720,1083172864,1083179008,1083185152,1083191296,1083197440,1083203584,1083209728,1083215872,1083222016,1083228160,1083234304,1083240448,1083246592,1083252736,1083258880,1083265024,1083271168,1083277312,1083283456,1083289600,1083295744,1083301888,1083308032,1083314176,1083320320,1083326464,1083332608,1083338752,1083344896,1083351040,1083357184,1083363328,1083369472,1083375616,1083381760,1083387904,1083394048,1083400192,1083406336,1083412480,1083418624,1083424768,1083430912,1083437056,1083443200,1083449344,1083455488,1083461632,1083467776,1083473920,1083480064,1083486208,1083492352,1083498496,1083504640,1083510784,1083516928,1083523072,1083529216,1083535360,1083541504,1083547648,1083553792,1083559936,1083566080,1083572224,1083578368,1083584512,1083590656,1083596800,1083602944,1083609088,1083615232,1083621376,1083627520,1083633664,1083639808,1083645952,1083652096,1083658240,1083664384,1083670528,1083676672,1083682816,1083688960,1083695104,1083701248,1083707392,1083713536,1083719680,1083725824,1083731968,1083738112,1083744256,1083750400,1083756544,1083762688,1083768832,1083774976,1083781120,1083787264,1083793408,1083799552,1083805696,1083811840,1083817984,1083824128,1083830272,1083836416,1083842560,1083848704,1083854848,1083860992,1083867136,1083873280,1083879424,1083885568,1083891712,1083897856,1083904000,1083910144,1083916288,1083922432,1083928576,1083934720,1083940864,1083947008,1083953152,1083959296,1083965440,1083971584,1083977728,1083983872,1083990016,1083996160,1084002304,1084008448,1084014592,1084020736,1084026880,1084033024,1084039168,1084045312,1084051456,1084057600,1084063744,1084069888,1084076032,1084082176,1084088320,1084094464,1084100608,1084106752,1084112896,1084119040,1084125184,1084131328,1084137472,1084143616,1084149760,1084155904,1084162048,1084168192,1084174336,1084180480,1084186624,1084192768,1084198912,1084205056,1084211200,1084217344,1084223488,1084229632,1084235776,1084241920,1084248064,1084254208,1084260352,1084266496,1084272640,1084278784,1084284928,1084291072,1084297216,1084303360,1084309504,1084315648,1084321792,1084327936,1084334080,1084340224,1084346368,1084352512,1084358656,1084364800,1084370944,1084377088,1084383232,1084389376,1084395520,1084401664,1084407808,1084413952,1084420096,1084426240,1084432384,1084438528,1084444672,1084450816,1084456960,1084463104,1084469248,1084475392,1084481536,1084487680,1084493824,1084499968,1084506112,1084512256,1084518400,1084524544,1084530688,1084536832,1084542976,1084549120,1084555264,1084561408,1084567552,1084573696,1084579840,1084585984,1084592128,1084598272,1084604416,1084610560,1084616704,1084622848,1084628992,1084635136,1084641280,1084647424,1084653568,1084659712,1084665856,1084672000,1084678144,1084684288,1084690432,1084696576,1084702720,1084708864,1084715008,1084721152,1084727296,1084733440,1084739584,1084745728,1084751872,1084758016,1084764160,1084770304,1084776448,1084782592,1084788736,1084794880,1084801024,1084807168,1084813312,1084819456,1084825600,1084831744,1084837888,1084844032,1084850176,1084856320,1084862464,1084868608,1084874752,1084880896,1084887040,1084893184,1084899328,1084905472,1084911616,1084917760,1084923904,1084930048,1084936192,1084942336,1084948480,1084954624,1084960768,1084966912,1084973056,1084979200,1084985344,1084991488,1084997632,1085003776,1085009920,1085016064,1085022208,1085028352,1085034496,1085040640,1085046784,1085052928,1085059072,1085065216,1085071360,1085077504,1085083648,1085089792,1085095936,1085102080,1085108224,1085114368,1085120512,1085126656,1085132800,1085138944,1085145088,1085151232,1085157376,1085163520,1085169664,1085175808,1085181952,1085188096,1085194240,1085200384,1085206528,1085212672,1085218816,1085224960,1085231104,1085237248,1085243392,1085249536,1085255680,1085261824,1085267968,1085274112,1085280256,1085286400,1085292544,1085298688,1085304832,1085310976,1085317120,1085323264,1085329408,1085335552,1085341696,1085347840,1085353984,1085360128,1085366272,1085372416,1085378560,1085384704,1085390848,1085396992,1085403136,1085409280,1085415424,1085421568,1085427712,1085433856,1085440000,1085446144,1085452288,1085458432,1085464576,1085470720,1085476864,1085483008,1085489152,1085495296,1085501440,1085507584,1085513728,1085519872,1085526016,1085532160,1085538304,1085544448,1085550592,1085556736,1085562880,1085569024,1085575168,1085581312,1085587456,1085593600,1085599744,1085605888,1085612032,1085618176,1085624320,1085630464,1085636608,1085642752,1085648896,1085655040,1085661184,1085667328,1085673472,1085679616,1085685760,1085691904,1085698048,1085704192,1085710336,1085716480,1085722624,1085728768,1085734912,1085741056,1085747200,1085753344,1085759488,1085765632,1085771776,1085777920,1085784064,1085790208,1085796352,1085802496,1085808640,1085814784,1085820928,1085827072,1085833216,1085839360,1085845504,1085851648,1085857792,1085863936,1085870080,1085876224,1085882368,1085888512,1085894656,1085900800,1085906944,1085913088,1085919232,1085925376,1085931520,1085937664,1085943808,1085949952,1085956096,1085962240,1085968384,1085974528,1085980672,1085986816,1085992960,1085999104,1086005248,1086011392,1086017536,1086023680,1086029824,1086035968,1086042112,1086048256,1086054400,1086060544,1086066688,1086072832,1086078976,1086085120,1086091264,1086097408,1086103552,1086109696,1086115840,1086121984,1086128128,1086134272,1086140416,1086146560,1086152704,1086158848,1086164992,1086171136,1086177280,1086183424,1086189568,1086195712,1086201856,1086208000,1086214144,1086220288,1086226432,1086232576,1086238720,1086244864,1086251008,1086257152,1086263296,1086269440,1086275584,1086281728,1086287872,1086294016,1086300160,1086306304,1086312448,1086318592,1086324736,0,2147483648,2139095040,4286578688,2143289344,228737632,2376221280,786163455,2933647103,1,2139081118,4286564766,1056964608,1065353216,1073741824,1081711002,1082130432,1084227584], + "ort_bits": [3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836859,3212836859,3212836859,3212836859,3212836859,3212836859,3212836859,3212836859,3212836859,3212836858,3212836858,3212836858,3212836858,3212836858,3212836858,3212836858,3212836858,3212836857,3212836857,3212836857,3212836857,3212836857,3212836857,3212836856,3212836856,3212836856,3212836856,3212836856,3212836856,3212836855,3212836855,3212836855,3212836855,3212836855,3212836854,3212836854,3212836854,3212836854,3212836854,3212836853,3212836853,3212836853,3212836853,3212836852,3212836852,3212836852,3212836852,3212836851,3212836851,3212836851,3212836851,3212836850,3212836850,3212836850,3212836849,3212836849,3212836849,3212836848,3212836848,3212836848,3212836847,3212836847,3212836847,3212836846,3212836846,3212836846,3212836845,3212836845,3212836844,3212836844,3212836843,3212836843,3212836843,3212836842,3212836842,3212836841,3212836841,3212836840,3212836840,3212836839,3212836839,3212836838,3212836838,3212836837,3212836837,3212836836,3212836835,3212836835,3212836834,3212836834,3212836833,3212836832,3212836832,3212836831,3212836830,3212836830,3212836829,3212836828,3212836827,3212836827,3212836826,3212836825,3212836824,3212836823,3212836823,3212836822,3212836821,3212836820,3212836819,3212836818,3212836817,3212836816,3212836815,3212836814,3212836813,3212836812,3212836811,3212836810,3212836809,3212836808,3212836807,3212836806,3212836804,3212836803,3212836802,3212836801,3212836800,3212836798,3212836797,3212836796,3212836794,3212836793,3212836791,3212836790,3212836788,3212836787,3212836785,3212836784,3212836782,3212836781,3212836779,3212836777,3212836775,3212836774,3212836772,3212836770,3212836768,3212836766,3212836764,3212836762,3212836760,3212836758,3212836756,3212836754,3212836752,3212836750,3212836747,3212836745,3212836743,3212836740,3212836738,3212836736,3212836733,3212836731,3212836728,3212836725,3212836723,3212836720,3212836717,3212836714,3212836711,3212836708,3212836705,3212836702,3212836699,3212836696,3212836693,3212836689,3212836686,3212836682,3212836679,3212836675,3212836672,3212836668,3212836664,3212836660,3212836657,3212836653,3212836649,3212836644,3212836640,3212836636,3212836632,3212836627,3212836623,3212836618,3212836613,3212836609,3212836604,3212836599,3212836594,3212836589,3212836584,3212836578,3212836573,3212836567,3212836562,3212836556,3212836550,3212836544,3212836538,3212836532,3212836526,3212836520,3212836513,3212836507,3212836500,3212836493,3212836486,3212836479,3212836472,3212836465,3212836458,3212836450,3212836442,3212836435,3212836427,3212836419,3212836410,3212836402,3212836393,3212836385,3212836376,3212836367,3212836358,3212836349,3212836339,3212836330,3212836320,3212836310,3212836300,3212836289,3212836279,3212836268,3212836257,3212836246,3212836235,3212836224,3212836212,3212836200,3212836188,3212836176,3212836164,3212836151,3212836138,3212836125,3212836112,3212836098,3212836085,3212836071,3212836056,3212836042,3212836027,3212836012,3212835997,3212835982,3212835966,3212835950,3212835934,3212835917,3212835900,3212835883,3212835866,3212835848,3212835830,3212835812,3212835793,3212835774,3212835755,3212835736,3212835716,3212835695,3212835675,3212835654,3212835633,3212835611,3212835589,3212835567,3212835545,3212835521,3212835498,3212835474,3212835450,3212835426,3212835401,3212835375,3212835349,3212835323,3212835297,3212835269,3212835242,3212835214,3212835185,3212835157,3212835127,3212835097,3212835067,3212835036,3212835005,3212834973,3212834941,3212834908,3212834875,3212834841,3212834806,3212834771,3212834736,3212834699,3212834663,3212834625,3212834588,3212834549,3212834510,3212834470,3212834430,3212834389,3212834347,3212834305,3212834262,3212834218,3212834174,3212834129,3212834083,3212834036,3212833989,3212833941,3212833893,3212833843,3212833793,3212833742,3212833690,3212833637,3212833584,3212833530,3212833474,3212833418,3212833362,3212833304,3212833245,3212833186,3212833125,3212833064,3212833002,3212832938,3212832874,3212832809,3212832742,3212832675,3212832607,3212832538,3212832467,3212832396,3212832323,3212832250,3212832175,3212832099,3212832022,3212831944,3212831864,3212831784,3212831702,3212831619,3212831534,3212831449,3212831362,3212831274,3212831184,3212831094,3212831001,3212830908,3212830813,3212830717,3212830619,3212830520,3212830419,3212830317,3212830213,3212830108,3212830001,3212829892,3212829782,3212829671,3212829558,3212829443,3212829326,3212829208,3212829088,3212828966,3212828843,3212828717,3212828590,3212828461,3212828330,3212828197,3212828063,3212827926,3212827787,3212827647,3212827504,3212827359,3212827212,3212827063,3212826912,3212826759,3212826604,3212826446,3212826286,3212826124,3212825959,3212825793,3212825623,3212825452,3212825278,3212825101,3212824922,3212824741,3212824557,3212824370,3212824181,3212823989,3212823794,3212823597,3212823397,3212823194,3212822988,3212822779,3212822568,3212822353,3212822136,3212821915,3212821692,3212821465,3212821235,3212821002,3212820766,3212820527,3212820284,3212820038,3212819788,3212819536,3212819279,3212819019,3212818756,3212818489,3212818219,3212817944,3212817666,3212817384,3212817099,3212816809,3212816516,3212816219,3212815917,3212815612,3212815303,3212814989,3212814671,3212814349,3212814023,3212813692,3212813357,3212813017,3212812673,3212812324,3212811971,3212811613,3212811250,3212810882,3212810510,3212810132,3212809750,3212809363,3212808970,3212808573,3212808170,3212807762,3212807348,3212806930,3212806505,3212806075,3212805640,3212805199,3212804752,3212804300,3212803842,3212803377,3212802907,3212802431,3212801948,3212801460,3212800965,3212800464,3212799956,3212799442,3212798921,3212798394,3212797860,3212797319,3212796772,3212796217,3212795656,3212795087,3212794511,3212793928,3212793338,3212792740,3212792134,3212791522,3212790901,3212790273,3212789637,3212788992,3212788340,3212787680,3212787012,3212786335,3212785650,3212784957,3212784255,3212783544,3212782825,3212782096,3212781359,3212780613,3212779858,3212779093,3212778319,3212777536,3212776743,3212775941,3212775129,3212774307,3212773475,3212772633,3212771780,3212770918,3212770045,3212769162,3212768268,3212767364,3212766448,3212765522,3212764585,3212763636,3212762677,3212761706,3212760723,3212759729,3212758723,3212757705,3212756675,3212755633,3212754579,3212753513,3212752434,3212751342,3212750238,3212749121,3212747990,3212746847,3212745690,3212744520,3212743337,3212742140,3212740929,3212739704,3212738465,3212737211,3212735944,3212734662,3212733365,3212732053,3212730727,3212729385,3212728028,3212726656,3212725268,3212723865,3212722445,3212721010,3212719558,3212718090,3212716606,3212715105,3212713588,3212712053,3212710501,3212708932,3212707346,3212705741,3212704120,3212702480,3212700822,3212699146,3212697451,3212695738,3212694006,3212692254,3212690484,3212688695,3212686885,3212685057,3212683208,3212681339,3212679450,3212677541,3212675611,3212673660,3212671688,3212669695,3212667680,3212665644,3212663586,3212661506,3212659404,3212657280,3212655132,3212652963,3212650770,3212648554,3212646314,3212644051,3212641764,3212639454,3212637118,3212634759,3212632375,3212629965,3212627531,3212625072,3212622586,3212620075,3212617539,3212614976,3212612386,3212609770,3212607126,3212604456,3212601758,3212599033,3212596280,3212593499,3212590690,3212587851,3212584985,3212582089,3212579164,3212576209,3212573224,3212570210,3212567165,3212564090,3212560984,3212557847,3212554679,3212551479,3212548247,3212544984,3212541688,3212538359,3212534998,3212531603,3212528175,3212524714,3212521218,3212517688,3212514124,3212510525,3212506892,3212503222,3212499518,3212495777,3212492000,3212488187,3212484337,3212480450,3212476526,3212472565,3212468565,3212464527,3212460451,3212456336,3212452183,3212447989,3212443757,3212439484,3212435171,3212430817,3212426423,3212421987,3212417510,3212412991,3212408431,3212403827,3212399181,3212394492,3212389760,3212384984,3212380164,3212375300,3212370391,3212365437,3212360438,3212355393,3212350302,3212345165,3212339982,3212334751,3212329474,3212324148,3212318775,3212313354,3212307884,3212302364,3212296796,3212291178,3212285510,3212279792,3212274023,3212268203,3212262331,3212256408,3212250433,3212244405,3212238325,3212232191,3212226004,3212219763,3212213468,3212207118,3212200713,3212194253,3212187737,3212181166,3212174537,3212167852,3212161110,3212154310,3212147453,3212140537,3212133562,3212126529,3212119436,3212112283,3212105070,3212097796,3212090462,3212083066,3212075608,3212068089,3212060507,3212052862,3212045154,3212037382,3212029546,3212021646,3212013681,3212005651,3211997555,3211989393,3211981164,3211972869,3211964507,3211956077,3211947579,3211939013,3211930378,3211921674,3211912900,3211904056,3211895142,3211886156,3211877100,3211867972,3211858772,3211849499,3211840154,3211830735,3211821243,3211811676,3211802035,3211792319,3211782528,3211772661,3211762718,3211752698,3211742601,3211732427,3211722175,3211711845,3211701436,3211690948,3211680380,3211669732,3211659005,3211648196,3211637306,3211626334,3211615281,3211604145,3211592926,3211581624,3211570238,3211558768,3211547213,3211535574,3211523848,3211512037,3211500140,3211488156,3211476085,3211463926,3211451680,3211439345,3211426921,3211414408,3211401805,3211389112,3211376328,3211363454,3211350489,3211337431,3211324281,3211311038,3211297704,3211284274,3211270752,3211257134,3211243422,3211229615,3211215712,3211201713,3211187618,3211173426,3211159136,3211144749,3211130264,3211115680,3211100997,3211086215,3211071333,3211056350,3211041268,3211026084,3211010798,3210995411,3210979921,3210964329,3210948634,3210932835,3210916932,3210900925,3210884812,3210868596,3210852273,3210835844,3210819309,3210802667,3210785918,3210769061,3210752097,3210735024,3210717842,3210700551,3210683150,3210665640,3210648018,3210630287,3210612444,3210594490,3210576423,3210558245,3210539954,3210521550,3210503032,3210484400,3210465654,3210446794,3210427818,3210408728,3210389522,3210370199,3210350760,3210331204,3210311532,3210291741,3210271833,3210251806,3210231661,3210211397,3210191014,3210170510,3210149887,3210129144,3210108279,3210087294,3210066188,3210044960,3210023609,3210002136,3209980540,3209958822,3209936980,3209915014,3209892924,3209870710,3209848372,3209825908,3209803319,3209780604,3209757764,3209734797,3209711704,3209688484,3209665136,3209641662,3209618060,3209594330,3209570471,3209546484,3209522369,3209498124,3209473750,3209449246,3209424612,3209399848,3209374954,3209349929,3209324773,3209299486,3209274067,3209248517,3209222835,3209197021,3209171074,3209144994,3209118782,3209092437,3209065958,3209039346,3209012600,3208985721,3208958707,3208931558,3208904276,3208876858,3208849306,3208821618,3208793795,3208765837,3208737743,3208709513,3208681147,3208652645,3208624006,3208595231,3208566320,3208537271,3208508086,3208478764,3208449304,3208419707,3208389972,3208360100,3208330090,3208299942,3208269657,3208239233,3208208671,3208177971,3208147132,3208116155,3208085039,3208053785,3208022392,3207990860,3207959189,3207927379,3207895430,3207863343,3207831116,3207798750,3207766245,3207733600,3207700816,3207667893,3207634831,3207601629,3207568288,3207534808,3207501188,3207467429,3207433530,3207399492,3207365315,3207330998,3207296542,3207261947,3207227213,3207192339,3207157326,3207122174,3207086884,3207051453,3207015885,3206980177,3206944330,3206908345,3206872221,3206835958,3206799558,3206763018,3206726341,3206689525,3206652571,3206615480,3206578250,3206540883,3206503378,3206465736,3206427957,3206390041,3206351987,3206313797,3206275471,3206237008,3206198408,3206159673,3206120802,3206081795,3206042652,3206003375,3205963962,3205924414,3205884732,3205844915,3205804964,3205764879,3205724660,3205684308,3205643823,3205603205,3205562454,3205521570,3205480555,3205439407,3205398128,3205356718,3205315177,3205273505,3205231702,3205189770,3205147708,3205105516,3205063195,3205020746,3204978167,3204935461,3204892627,3204849666,3204806578,3204763363,3204720022,3204676554,3204632962,3204589244,3204545401,3204501434,3204457343,3204378001,3204289326,3204200405,3204111240,3204021831,3203932179,3203842286,3203752151,3203661776,3203571162,3203480309,3203389220,3203297893,3203206332,3203114536,3203022506,3202930244,3202837750,3202745026,3202652072,3202558890,3202465480,3202371844,3202277983,3202183898,3202089589,3201995058,3201900307,3201805335,3201710145,3201614737,3201519113,3201423274,3201327220,3201230953,3201134475,3201037786,3200940888,3200843782,3200746469,3200648950,3200551226,3200453300,3200355171,3200256842,3200158313,3200059587,3199960663,3199861544,3199762231,3199662725,3199563028,3199463141,3199363064,3199262800,3199162351,3199061716,3198960898,3198859899,3198758719,3198657359,3198555823,3198454110,3198352222,3198250161,3198147928,3198045525,3197942952,3197840213,3197737307,3197634237,3197531005,3197427610,3197324056,3197220344,3197116475,3197012450,3196908272,3196803942,3196699461,3196594831,3196490054,3196385132,3196280064,3196174855,3196069504,3195868380,3195657125,3195445597,3195233801,3195021739,3194809414,3194596831,3194383992,3194170901,3193957561,3193743976,3193530148,3193316082,3193101780,3192887247,3192672485,3192457498,3192242289,3192026863,3191811222,3191595369,3191379310,3191163046,3190946581,3190729920,3190513065,3190296020,3190078788,3189861374,3189643781,3189426012,3189208071,3188989961,3188771687,3188553252,3188334659,3188115912,3187897015,3187677971,3187246529,3186807877,3186368955,3185929768,3185490325,3185050633,3184610699,3184170531,3183730136,3183289522,3182848697,3182407667,3181966440,3181525023,3181083425,3180641652,3180199713,3179757614,3179315363,3178463504,3177578441,3176693119,3175807554,3174921761,3174035756,3173149552,3172263166,3171376613,3170085990,3168312304,3166538375,3164764233,3162989908,3159925646,3156376447,3151537525,3143149039,0,995665391,1004053877,1008892799,1012441998,1015506260,1017280585,1019054727,1020828656,1022602342,1023892965,1024779518,1025665904,1026552108,1027438113,1028323906,1029209471,1030094793,1030979856,1031831715,1032273966,1032716065,1033158004,1033599777,1034041375,1034482792,1034924019,1035365049,1035805874,1036246488,1036686883,1037127051,1037566985,1038006677,1038446120,1038885307,1039324229,1039762881,1040194323,1040413367,1040632264,1040851011,1041069604,1041288039,1041506313,1041724423,1041942364,1042160133,1042377726,1042595140,1042812372,1043029417,1043246272,1043462933,1043679398,1043895662,1044111721,1044327574,1044543215,1044758641,1044973850,1045188837,1045403599,1045618132,1045832434,1046046500,1046260328,1046473913,1046687253,1046900344,1047113183,1047325766,1047538091,1047750153,1047961949,1048173477,1048384732,1048585856,1048691207,1048796416,1048901484,1049006406,1049111183,1049215813,1049320294,1049424624,1049528802,1049632827,1049736696,1049840408,1049943962,1050047357,1050150589,1050253659,1050356565,1050459304,1050561877,1050664280,1050766513,1050868574,1050970462,1051072175,1051173711,1051275071,1051376251,1051477250,1051578068,1051678703,1051779152,1051879416,1051979493,1052079380,1052179077,1052278583,1052377896,1052477015,1052575939,1052674665,1052773194,1052871523,1052969652,1053067578,1053165302,1053262821,1053360134,1053457240,1053554138,1053650827,1053747305,1053843572,1053939626,1054035465,1054131089,1054226497,1054321687,1054416659,1054511410,1054605941,1054700250,1054794335,1054888196,1054981832,1055075242,1055168424,1055261378,1055354102,1055446596,1055538858,1055630888,1055722684,1055814245,1055905572,1055996661,1056087514,1056178128,1056268503,1056358638,1056448531,1056538183,1056627592,1056716757,1056805678,1056894353,1056973695,1057017786,1057061753,1057105596,1057149314,1057192906,1057236374,1057279715,1057322930,1057366018,1057408979,1057451813,1057494519,1057537098,1057579547,1057621868,1057664060,1057706122,1057748054,1057789857,1057831529,1057873070,1057914480,1057955759,1057996907,1058037922,1058078806,1058119557,1058160175,1058200660,1058241012,1058281231,1058321316,1058361267,1058401084,1058440766,1058480314,1058519727,1058559004,1058598147,1058637154,1058676025,1058714760,1058753360,1058791823,1058830149,1058868339,1058906393,1058944309,1058982088,1059019730,1059057235,1059094602,1059131832,1059168923,1059205877,1059242693,1059279370,1059315910,1059352310,1059388573,1059424697,1059460682,1059496529,1059532237,1059567805,1059603236,1059638526,1059673678,1059708691,1059743565,1059778299,1059812894,1059847350,1059881667,1059915844,1059949882,1059983781,1060017540,1060051160,1060084640,1060117981,1060151183,1060184245,1060217168,1060249952,1060282597,1060315102,1060347468,1060379695,1060411782,1060443731,1060475541,1060507212,1060538744,1060570137,1060601391,1060632507,1060663484,1060694323,1060725023,1060755585,1060786009,1060816294,1060846442,1060876452,1060906324,1060936059,1060965656,1060995116,1061024438,1061053623,1061082672,1061111583,1061140358,1061168997,1061197499,1061225865,1061254095,1061282189,1061310147,1061337970,1061365658,1061393210,1061420628,1061447910,1061475059,1061502073,1061528952,1061555698,1061582310,1061608789,1061635134,1061661346,1061687426,1061713373,1061739187,1061764869,1061790419,1061815838,1061841125,1061866281,1061891306,1061916200,1061940964,1061965598,1061990102,1062014476,1062038721,1062062836,1062086823,1062110682,1062134412,1062158014,1062181488,1062204836,1062228056,1062251149,1062274116,1062296956,1062319671,1062342260,1062364724,1062387062,1062409276,1062431366,1062453332,1062475174,1062496892,1062518488,1062539961,1062561312,1062582540,1062603646,1062624631,1062645496,1062666239,1062686862,1062707366,1062727749,1062748013,1062768158,1062788185,1062808093,1062827884,1062847556,1062867112,1062886551,1062905874,1062925080,1062944170,1062963146,1062982006,1063000752,1063019384,1063037902,1063056306,1063074597,1063092775,1063110842,1063128796,1063146639,1063164370,1063181992,1063199502,1063216903,1063234194,1063251376,1063268449,1063285413,1063302270,1063319019,1063335661,1063352196,1063368625,1063384948,1063401164,1063417277,1063433284,1063449187,1063464986,1063480681,1063496273,1063511763,1063527150,1063542436,1063557620,1063572702,1063587685,1063602567,1063617349,1063632032,1063646616,1063661101,1063675488,1063689778,1063703970,1063718065,1063732064,1063745967,1063759774,1063773486,1063787104,1063800626,1063814056,1063827390,1063840633,1063853783,1063866841,1063879806,1063892680,1063905464,1063918157,1063930760,1063943273,1063955697,1063968032,1063980278,1063992437,1064004508,1064016492,1064028389,1064040200,1064051926,1064063565,1064075120,1064086590,1064097976,1064109278,1064120497,1064131633,1064142686,1064153658,1064164548,1064175357,1064186084,1064196732,1064207300,1064217788,1064228197,1064238527,1064248779,1064258953,1064269050,1064279070,1064289013,1064298880,1064308671,1064318387,1064328028,1064337595,1064347087,1064356506,1064365851,1064375124,1064384324,1064393452,1064402508,1064411494,1064420408,1064429252,1064438026,1064446730,1064455365,1064463931,1064472429,1064480859,1064489221,1064497516,1064505745,1064513907,1064522003,1064530033,1064537998,1064545898,1064553734,1064561506,1064569214,1064576859,1064584441,1064591960,1064599418,1064606814,1064614148,1064621422,1064628635,1064635788,1064642881,1064649914,1064656889,1064663805,1064670662,1064677462,1064684204,1064690889,1064697518,1064704089,1064710605,1064717065,1064723470,1064729820,1064736115,1064742356,1064748543,1064754677,1064760757,1064766785,1064772760,1064778683,1064784555,1064790375,1064796144,1064801862,1064807530,1064813148,1064818716,1064824236,1064829706,1064835127,1064840500,1064845826,1064851103,1064856334,1064861517,1064866654,1064871745,1064876790,1064881789,1064886743,1064891652,1064896516,1064901336,1064906112,1064910844,1064915533,1064920179,1064924783,1064929343,1064933862,1064938339,1064942775,1064947169,1064951523,1064955836,1064960109,1064964341,1064968535,1064972688,1064976803,1064980879,1064984917,1064988917,1064992878,1064996802,1065000689,1065004539,1065008352,1065012129,1065015870,1065019574,1065023244,1065026877,1065030476,1065034040,1065037570,1065041066,1065044527,1065047955,1065051350,1065054711,1065058040,1065061336,1065064599,1065067831,1065071031,1065074199,1065077336,1065080442,1065083517,1065086562,1065089576,1065092561,1065095516,1065098441,1065101337,1065104203,1065107042,1065109851,1065112632,1065115385,1065118110,1065120808,1065123478,1065126122,1065128738,1065131328,1065133891,1065136427,1065138938,1065141424,1065143883,1065146317,1065148727,1065151111,1065153470,1065155806,1065158116,1065160403,1065162666,1065164906,1065167122,1065169315,1065171484,1065173632,1065175756,1065177858,1065179938,1065181996,1065184032,1065186047,1065188040,1065190012,1065191963,1065193893,1065195802,1065197691,1065199560,1065201409,1065203237,1065205047,1065206836,1065208606,1065210358,1065212090,1065213803,1065215498,1065217174,1065218832,1065220472,1065222093,1065223698,1065225284,1065226853,1065228405,1065229940,1065231457,1065232958,1065234442,1065235910,1065237362,1065238797,1065240217,1065241620,1065243008,1065244380,1065245737,1065247079,1065248405,1065249717,1065251014,1065252296,1065253563,1065254817,1065256056,1065257281,1065258492,1065259689,1065260872,1065262042,1065263199,1065264342,1065265473,1065266590,1065267694,1065268786,1065269865,1065270931,1065271985,1065273027,1065274057,1065275075,1065276081,1065277075,1065278058,1065279029,1065279988,1065280937,1065281874,1065282800,1065283716,1065284620,1065285514,1065286397,1065287270,1065288132,1065288985,1065289827,1065290659,1065291481,1065292293,1065293095,1065293888,1065294671,1065295445,1065296210,1065296965,1065297711,1065298448,1065299177,1065299896,1065300607,1065301309,1065302002,1065302687,1065303364,1065304032,1065304692,1065305344,1065305989,1065306625,1065307253,1065307874,1065308486,1065309092,1065309690,1065310280,1065310863,1065311439,1065312008,1065312569,1065313124,1065313671,1065314212,1065314746,1065315273,1065315794,1065316308,1065316816,1065317317,1065317812,1065318300,1065318783,1065319259,1065319729,1065320194,1065320652,1065321104,1065321551,1065321992,1065322427,1065322857,1065323282,1065323700,1065324114,1065324522,1065324925,1065325322,1065325715,1065326102,1065326484,1065326862,1065327234,1065327602,1065327965,1065328323,1065328676,1065329025,1065329369,1065329709,1065330044,1065330375,1065330701,1065331023,1065331341,1065331655,1065331964,1065332269,1065332571,1065332868,1065333161,1065333451,1065333736,1065334018,1065334296,1065334571,1065334841,1065335108,1065335371,1065335631,1065335888,1065336140,1065336390,1065336636,1065336879,1065337118,1065337354,1065337587,1065337817,1065338044,1065338267,1065338488,1065338705,1065338920,1065339131,1065339340,1065339546,1065339749,1065339949,1065340146,1065340341,1065340533,1065340722,1065340909,1065341093,1065341274,1065341453,1065341630,1065341804,1065341975,1065342145,1065342311,1065342476,1065342638,1065342798,1065342956,1065343111,1065343264,1065343415,1065343564,1065343711,1065343856,1065343999,1065344139,1065344278,1065344415,1065344549,1065344682,1065344813,1065344942,1065345069,1065345195,1065345318,1065345440,1065345560,1065345678,1065345795,1065345910,1065346023,1065346134,1065346244,1065346353,1065346460,1065346565,1065346669,1065346771,1065346872,1065346971,1065347069,1065347165,1065347260,1065347353,1065347446,1065347536,1065347626,1065347714,1065347801,1065347886,1065347971,1065348054,1065348136,1065348216,1065348296,1065348374,1065348451,1065348527,1065348602,1065348675,1065348748,1065348819,1065348890,1065348959,1065349027,1065349094,1065349161,1065349226,1065349290,1065349354,1065349416,1065349477,1065349538,1065349597,1065349656,1065349714,1065349770,1065349826,1065349882,1065349936,1065349989,1065350042,1065350094,1065350145,1065350195,1065350245,1065350293,1065350341,1065350388,1065350435,1065350481,1065350526,1065350570,1065350614,1065350657,1065350699,1065350741,1065350782,1065350822,1065350862,1065350901,1065350940,1065350977,1065351015,1065351051,1065351088,1065351123,1065351158,1065351193,1065351227,1065351260,1065351293,1065351325,1065351357,1065351388,1065351419,1065351449,1065351479,1065351509,1065351537,1065351566,1065351594,1065351621,1065351649,1065351675,1065351701,1065351727,1065351753,1065351778,1065351802,1065351826,1065351850,1065351873,1065351897,1065351919,1065351941,1065351963,1065351985,1065352006,1065352027,1065352047,1065352068,1065352088,1065352107,1065352126,1065352145,1065352164,1065352182,1065352200,1065352218,1065352235,1065352252,1065352269,1065352286,1065352302,1065352318,1065352334,1065352349,1065352364,1065352379,1065352394,1065352408,1065352423,1065352437,1065352450,1065352464,1065352477,1065352490,1065352503,1065352516,1065352528,1065352540,1065352552,1065352564,1065352576,1065352587,1065352598,1065352609,1065352620,1065352631,1065352641,1065352652,1065352662,1065352672,1065352682,1065352691,1065352701,1065352710,1065352719,1065352728,1065352737,1065352745,1065352754,1065352762,1065352771,1065352779,1065352787,1065352794,1065352802,1065352810,1065352817,1065352824,1065352831,1065352838,1065352845,1065352852,1065352859,1065352865,1065352872,1065352878,1065352884,1065352890,1065352896,1065352902,1065352908,1065352914,1065352919,1065352925,1065352930,1065352936,1065352941,1065352946,1065352951,1065352956,1065352961,1065352965,1065352970,1065352975,1065352979,1065352984,1065352988,1065352992,1065352996,1065353001,1065353005,1065353009,1065353012,1065353016,1065353020,1065353024,1065353027,1065353031,1065353034,1065353038,1065353041,1065353045,1065353048,1065353051,1065353054,1065353057,1065353060,1065353063,1065353066,1065353069,1065353072,1065353075,1065353077,1065353080,1065353083,1065353085,1065353088,1065353090,1065353092,1065353095,1065353097,1065353099,1065353102,1065353104,1065353106,1065353108,1065353110,1065353112,1065353114,1065353116,1065353118,1065353120,1065353122,1065353124,1065353126,1065353127,1065353129,1065353131,1065353133,1065353134,1065353136,1065353137,1065353139,1065353140,1065353142,1065353143,1065353145,1065353146,1065353148,1065353149,1065353150,1065353152,1065353153,1065353154,1065353155,1065353156,1065353158,1065353159,1065353160,1065353161,1065353162,1065353163,1065353164,1065353165,1065353166,1065353167,1065353168,1065353169,1065353170,1065353171,1065353172,1065353173,1065353174,1065353175,1065353175,1065353176,1065353177,1065353178,1065353179,1065353179,1065353180,1065353181,1065353182,1065353182,1065353183,1065353184,1065353184,1065353185,1065353186,1065353186,1065353187,1065353187,1065353188,1065353189,1065353189,1065353190,1065353190,1065353191,1065353191,1065353192,1065353192,1065353193,1065353193,1065353194,1065353194,1065353195,1065353195,1065353195,1065353196,1065353196,1065353197,1065353197,1065353198,1065353198,1065353198,1065353199,1065353199,1065353199,1065353200,1065353200,1065353200,1065353201,1065353201,1065353201,1065353202,1065353202,1065353202,1065353203,1065353203,1065353203,1065353203,1065353204,1065353204,1065353204,1065353204,1065353205,1065353205,1065353205,1065353205,1065353206,1065353206,1065353206,1065353206,1065353206,1065353207,1065353207,1065353207,1065353207,1065353207,1065353208,1065353208,1065353208,1065353208,1065353208,1065353208,1065353209,1065353209,1065353209,1065353209,1065353209,1065353209,1065353210,1065353210,1065353210,1065353210,1065353210,1065353210,1065353210,1065353210,1065353211,1065353211,1065353211,1065353211,1065353211,1065353211,1065353211,1065353211,1065353211,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,0,2147483648,1065353216,3212836864,2143289344,230102793,2377586441,788013594,2935497242,1,1065353216,3212836864,1057308539,1062714173,1065274737,1065353215,1065353216,1065353216], + "f64_ref_bits": [3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836864,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836863,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836862,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836861,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836860,3212836859,3212836859,3212836859,3212836859,3212836859,3212836859,3212836859,3212836859,3212836859,3212836858,3212836858,3212836858,3212836858,3212836858,3212836858,3212836858,3212836858,3212836857,3212836857,3212836857,3212836857,3212836857,3212836857,3212836856,3212836856,3212836856,3212836856,3212836856,3212836856,3212836855,3212836855,3212836855,3212836855,3212836855,3212836854,3212836854,3212836854,3212836854,3212836854,3212836853,3212836853,3212836853,3212836853,3212836852,3212836852,3212836852,3212836852,3212836851,3212836851,3212836851,3212836851,3212836850,3212836850,3212836850,3212836849,3212836849,3212836849,3212836848,3212836848,3212836848,3212836847,3212836847,3212836847,3212836846,3212836846,3212836846,3212836845,3212836845,3212836844,3212836844,3212836843,3212836843,3212836843,3212836842,3212836842,3212836841,3212836841,3212836840,3212836840,3212836839,3212836839,3212836838,3212836838,3212836837,3212836837,3212836836,3212836835,3212836835,3212836834,3212836834,3212836833,3212836832,3212836832,3212836831,3212836830,3212836830,3212836829,3212836828,3212836827,3212836827,3212836826,3212836825,3212836824,3212836823,3212836823,3212836822,3212836821,3212836820,3212836819,3212836818,3212836817,3212836816,3212836815,3212836814,3212836813,3212836812,3212836811,3212836810,3212836809,3212836808,3212836807,3212836806,3212836804,3212836803,3212836802,3212836801,3212836800,3212836798,3212836797,3212836796,3212836794,3212836793,3212836791,3212836790,3212836788,3212836787,3212836785,3212836784,3212836782,3212836781,3212836779,3212836777,3212836775,3212836774,3212836772,3212836770,3212836768,3212836766,3212836764,3212836762,3212836760,3212836758,3212836756,3212836754,3212836752,3212836750,3212836747,3212836745,3212836743,3212836740,3212836738,3212836736,3212836733,3212836731,3212836728,3212836725,3212836723,3212836720,3212836717,3212836714,3212836711,3212836708,3212836705,3212836702,3212836699,3212836696,3212836693,3212836689,3212836686,3212836682,3212836679,3212836675,3212836672,3212836668,3212836664,3212836660,3212836657,3212836653,3212836649,3212836644,3212836640,3212836636,3212836632,3212836627,3212836623,3212836618,3212836613,3212836609,3212836604,3212836599,3212836594,3212836589,3212836584,3212836578,3212836573,3212836567,3212836562,3212836556,3212836550,3212836544,3212836538,3212836532,3212836526,3212836520,3212836513,3212836507,3212836500,3212836493,3212836486,3212836479,3212836472,3212836465,3212836458,3212836450,3212836442,3212836435,3212836427,3212836419,3212836410,3212836402,3212836393,3212836385,3212836376,3212836367,3212836358,3212836349,3212836339,3212836330,3212836320,3212836310,3212836300,3212836289,3212836279,3212836268,3212836257,3212836246,3212836235,3212836224,3212836212,3212836200,3212836188,3212836176,3212836164,3212836151,3212836138,3212836125,3212836112,3212836098,3212836085,3212836071,3212836056,3212836042,3212836027,3212836012,3212835997,3212835982,3212835966,3212835950,3212835934,3212835917,3212835900,3212835883,3212835866,3212835848,3212835830,3212835812,3212835793,3212835774,3212835755,3212835735,3212835716,3212835695,3212835675,3212835654,3212835633,3212835611,3212835589,3212835567,3212835544,3212835521,3212835498,3212835474,3212835450,3212835425,3212835401,3212835375,3212835349,3212835323,3212835296,3212835269,3212835242,3212835214,3212835185,3212835157,3212835127,3212835097,3212835067,3212835036,3212835005,3212834973,3212834941,3212834908,3212834875,3212834841,3212834806,3212834771,3212834736,3212834699,3212834663,3212834625,3212834587,3212834549,3212834510,3212834470,3212834430,3212834389,3212834347,3212834305,3212834262,3212834218,3212834174,3212834129,3212834083,3212834036,3212833989,3212833941,3212833892,3212833843,3212833793,3212833742,3212833690,3212833637,3212833584,3212833529,3212833474,3212833418,3212833362,3212833304,3212833245,3212833186,3212833125,3212833064,3212833001,3212832938,3212832874,3212832809,3212832742,3212832675,3212832607,3212832538,3212832467,3212832396,3212832323,3212832249,3212832175,3212832099,3212832022,3212831943,3212831864,3212831783,3212831702,3212831619,3212831534,3212831449,3212831362,3212831274,3212831184,3212831093,3212831001,3212830908,3212830813,3212830716,3212830619,3212830519,3212830419,3212830316,3212830213,3212830108,3212830001,3212829892,3212829782,3212829671,3212829558,3212829443,3212829326,3212829208,3212829088,3212828966,3212828842,3212828717,3212828590,3212828461,3212828330,3212828197,3212828062,3212827926,3212827787,3212827646,3212827504,3212827359,3212827212,3212827063,3212826912,3212826759,3212826604,3212826446,3212826286,3212826124,3212825959,3212825792,3212825623,3212825452,3212825278,3212825101,3212824922,3212824741,3212824557,3212824370,3212824181,3212823989,3212823794,3212823597,3212823396,3212823193,3212822988,3212822779,3212822567,3212822353,3212822136,3212821915,3212821691,3212821465,3212821235,3212821002,3212820766,3212820526,3212820284,3212820038,3212819788,3212819535,3212819279,3212819019,3212818756,3212818489,3212818218,3212817944,3212817666,3212817384,3212817099,3212816809,3212816516,3212816219,3212815917,3212815612,3212815302,3212814989,3212814671,3212814349,3212814022,3212813691,3212813356,3212813017,3212812672,3212812324,3212811970,3212811612,3212811249,3212810882,3212810509,3212810132,3212809750,3212809362,3212808970,3212808572,3212808169,3212807761,3212807348,3212806929,3212806505,3212806075,3212805640,3212805199,3212804752,3212804300,3212803841,3212803377,3212802907,3212802431,3212801948,3212801459,3212800965,3212800463,3212799956,3212799442,3212798921,3212798394,3212797860,3212797319,3212796771,3212796217,3212795655,3212795087,3212794511,3212793928,3212793337,3212792739,3212792134,3212791521,3212790901,3212790272,3212789636,3212788992,3212788340,3212787680,3212787012,3212786335,3212785650,3212784956,3212784254,3212783544,3212782824,3212782096,3212781359,3212780613,3212779857,3212779093,3212778319,3212777536,3212776743,3212775940,3212775128,3212774306,3212773474,3212772632,3212771780,3212770918,3212770045,3212769162,3212768268,3212767363,3212766448,3212765522,3212764585,3212763636,3212762676,3212761705,3212760723,3212759729,3212758723,3212757705,3212756675,3212755633,3212754579,3212753512,3212752433,3212751342,3212750237,3212749120,3212747990,3212746847,3212745690,3212744520,3212743337,3212742139,3212740928,3212739703,3212738464,3212737211,3212735943,3212734661,3212733364,3212732053,3212730726,3212729385,3212728028,3212726655,3212725268,3212723864,3212722445,3212721009,3212719558,3212718090,3212716606,3212715105,3212713587,3212712053,3212710501,3212708932,3212707345,3212705741,3212704119,3212702479,3212700822,3212699145,3212697451,3212695737,3212694005,3212692254,3212690484,3212688694,3212686885,3212685056,3212683208,3212681339,3212679450,3212677540,3212675610,3212673659,3212671687,3212669694,3212667680,3212665644,3212663586,3212661506,3212659404,3212657279,3212655132,3212652962,3212650769,3212648553,3212646314,3212644051,3212641764,3212639453,3212637118,3212634759,3212632374,3212629965,3212627531,3212625071,3212622586,3212620075,3212617538,3212614975,3212612386,3212609769,3212607126,3212604456,3212601758,3212599033,3212596280,3212593499,3212590689,3212587851,3212584984,3212582089,3212579163,3212576209,3212573224,3212570210,3212567165,3212564090,3212560984,3212557847,3212554679,3212551479,3212548247,3212544983,3212541687,3212538359,3212534997,3212531603,3212528175,3212524713,3212521218,3212517688,3212514124,3212510525,3212506891,3212503222,3212499517,3212495777,3212492000,3212488187,3212484337,3212480450,3212476526,3212472565,3212468565,3212464527,3212460451,3212456336,3212452183,3212447989,3212443756,3212439484,3212435171,3212430817,3212426423,3212421987,3212417510,3212412991,3212408431,3212403827,3212399181,3212394492,3212389760,3212384984,3212380164,3212375300,3212370391,3212365437,3212360438,3212355393,3212350302,3212345165,3212339982,3212334751,3212329474,3212324148,3212318775,3212313354,3212307884,3212302364,3212296796,3212291178,3212285510,3212279792,3212274023,3212268203,3212262331,3212256408,3212250433,3212244405,3212238325,3212232191,3212226004,3212219763,3212213468,3212207118,3212200713,3212194253,3212187738,3212181166,3212174537,3212167852,3212161110,3212154311,3212147453,3212140537,3212133562,3212126529,3212119436,3212112283,3212105070,3212097796,3212090462,3212083066,3212075609,3212068089,3212060507,3212052862,3212045154,3212037382,3212029546,3212021646,3212013681,3212005651,3211997555,3211989393,3211981165,3211972869,3211964507,3211956077,3211947579,3211939013,3211930378,3211921674,3211912900,3211904056,3211895142,3211886156,3211877100,3211867972,3211858772,3211849499,3211840154,3211830735,3211821243,3211811676,3211802035,3211792320,3211782528,3211772661,3211762718,3211752698,3211742602,3211732427,3211722175,3211711845,3211701436,3211690948,3211680380,3211669733,3211659005,3211648196,3211637306,3211626335,3211615281,3211604145,3211592926,3211581624,3211570238,3211558768,3211547213,3211535574,3211523848,3211512037,3211500140,3211488156,3211476085,3211463926,3211451680,3211439345,3211426921,3211414408,3211401805,3211389112,3211376329,3211363454,3211350489,3211337431,3211324281,3211311039,3211297703,3211284274,3211270751,3211257134,3211243422,3211229615,3211215712,3211201713,3211187618,3211173426,3211159136,3211144749,3211130264,3211115680,3211100997,3211086215,3211071333,3211056351,3211041268,3211026084,3211010798,3210995411,3210979922,3210964329,3210948634,3210932835,3210916932,3210900925,3210884813,3210868596,3210852273,3210835844,3210819309,3210802667,3210785918,3210769061,3210752097,3210735024,3210717842,3210700551,3210683150,3210665640,3210648019,3210630287,3210612444,3210594490,3210576424,3210558245,3210539954,3210521550,3210503032,3210484400,3210465654,3210446794,3210427819,3210408728,3210389522,3210370199,3210350760,3210331204,3210311532,3210291741,3210271833,3210251806,3210231661,3210211397,3210191013,3210170510,3210149887,3210129144,3210108279,3210087294,3210066188,3210044959,3210023609,3210002136,3209980540,3209958822,3209936980,3209915014,3209892924,3209870710,3209848372,3209825908,3209803319,3209780604,3209757764,3209734797,3209711704,3209688484,3209665137,3209641662,3209618060,3209594330,3209570471,3209546484,3209522369,3209498124,3209473750,3209449246,3209424612,3209399848,3209374954,3209349929,3209324773,3209299486,3209274067,3209248517,3209222835,3209197020,3209171074,3209144994,3209118782,3209092437,3209065958,3209039346,3209012600,3208985721,3208958707,3208931559,3208904276,3208876858,3208849306,3208821618,3208793795,3208765837,3208737743,3208709513,3208681147,3208652645,3208624007,3208595232,3208566320,3208537272,3208508086,3208478764,3208449304,3208419707,3208389972,3208360100,3208330090,3208299943,3208269657,3208239233,3208208671,3208177971,3208147132,3208116155,3208085039,3208053785,3208022392,3207990860,3207959189,3207927379,3207895430,3207863343,3207831116,3207798750,3207766244,3207733600,3207700816,3207667893,3207634831,3207601629,3207568288,3207534807,3207501188,3207467428,3207433530,3207399492,3207365315,3207330998,3207296542,3207261947,3207227213,3207192339,3207157326,3207122174,3207086883,3207051453,3207015884,3206980177,3206944330,3206908345,3206872221,3206835958,3206799557,3206763018,3206726340,3206689525,3206652571,3206615479,3206578250,3206540883,3206503378,3206465736,3206427957,3206390041,3206351987,3206313797,3206275471,3206237008,3206198408,3206159673,3206120802,3206081795,3206042652,3206003374,3205963962,3205924414,3205884732,3205844915,3205804964,3205764879,3205724660,3205684308,3205643823,3205603205,3205562454,3205521570,3205480555,3205439407,3205398128,3205356718,3205315177,3205273505,3205231702,3205189770,3205147708,3205105516,3205063195,3205020746,3204978167,3204935461,3204892627,3204849666,3204806578,3204763363,3204720022,3204676554,3204632962,3204589244,3204545401,3204501434,3204457343,3204378001,3204289326,3204200405,3204111240,3204021831,3203932179,3203842286,3203752151,3203661776,3203571162,3203480309,3203389220,3203297893,3203206332,3203114536,3203022506,3202930244,3202837750,3202745026,3202652072,3202558890,3202465480,3202371844,3202277983,3202183898,3202089589,3201995058,3201900307,3201805335,3201710145,3201614737,3201519113,3201423273,3201327220,3201230953,3201134475,3201037786,3200940888,3200843782,3200746468,3200648949,3200551226,3200453299,3200355171,3200256842,3200158313,3200059587,3199960663,3199861544,3199762231,3199662725,3199563028,3199463140,3199363064,3199262800,3199162350,3199061716,3198960898,3198859898,3198758718,3198657359,3198555822,3198454109,3198352222,3198250161,3198147928,3198045524,3197942952,3197840213,3197737307,3197634237,3197531004,3197427610,3197324056,3197220344,3197116475,3197012450,3196908272,3196803942,3196699461,3196594831,3196490054,3196385131,3196280064,3196174855,3196069504,3195868380,3195657124,3195445597,3195233800,3195021738,3194809414,3194596831,3194383992,3194170901,3193957561,3193743976,3193530148,3193316082,3193101780,3192887247,3192672485,3192457498,3192242289,3192026863,3191811222,3191595369,3191379309,3191163046,3190946581,3190729920,3190513064,3190296020,3190078788,3189861374,3189643781,3189426012,3189208071,3188989961,3188771687,3188553252,3188334659,3188115912,3187897015,3187677971,3187246529,3186807877,3186368955,3185929768,3185490325,3185050633,3184610699,3184170531,3183730136,3183289523,3182848697,3182407667,3181966440,3181525023,3181083425,3180641652,3180199713,3179757614,3179315363,3178463504,3177578441,3176693119,3175807555,3174921762,3174035756,3173149552,3172263166,3171376613,3170085990,3168312304,3166538375,3164764233,3162989908,3159925646,3156376448,3151537525,3143149039,0,995665391,1004053877,1008892800,1012441998,1015506260,1017280585,1019054727,1020828656,1022602342,1023892965,1024779518,1025665904,1026552108,1027438114,1028323907,1029209471,1030094793,1030979856,1031831715,1032273966,1032716065,1033158004,1033599777,1034041375,1034482792,1034924019,1035365049,1035805875,1036246488,1036686883,1037127051,1037566985,1038006677,1038446120,1038885307,1039324229,1039762881,1040194323,1040413367,1040632264,1040851011,1041069604,1041288039,1041506313,1041724423,1041942364,1042160133,1042377726,1042595140,1042812372,1043029416,1043246272,1043462933,1043679398,1043895661,1044111721,1044327574,1044543215,1044758641,1044973850,1045188837,1045403599,1045618132,1045832434,1046046500,1046260328,1046473913,1046687253,1046900344,1047113183,1047325766,1047538090,1047750152,1047961949,1048173476,1048384732,1048585856,1048691207,1048796416,1048901483,1049006406,1049111183,1049215813,1049320294,1049424624,1049528802,1049632827,1049736696,1049840408,1049943962,1050047356,1050150589,1050253659,1050356565,1050459304,1050561876,1050664280,1050766513,1050868574,1050970461,1051072174,1051173711,1051275070,1051376250,1051477250,1051578068,1051678702,1051779152,1051879416,1051979492,1052079380,1052179077,1052278583,1052377896,1052477015,1052575939,1052674665,1052773194,1052871523,1052969651,1053067578,1053165301,1053262820,1053360134,1053457240,1053554138,1053650827,1053747305,1053843572,1053939625,1054035465,1054131089,1054226497,1054321687,1054416659,1054511410,1054605941,1054700250,1054794335,1054888196,1054981832,1055075242,1055168424,1055261378,1055354102,1055446596,1055538858,1055630888,1055722684,1055814245,1055905572,1055996661,1056087514,1056178128,1056268503,1056358638,1056448531,1056538183,1056627592,1056716757,1056805678,1056894353,1056973695,1057017786,1057061753,1057105596,1057149314,1057192906,1057236374,1057279715,1057322930,1057366018,1057408979,1057451813,1057494519,1057537098,1057579547,1057621868,1057664060,1057706122,1057748054,1057789857,1057831529,1057873070,1057914480,1057955759,1057996907,1058037922,1058078806,1058119557,1058160175,1058200660,1058241012,1058281231,1058321316,1058361267,1058401084,1058440766,1058480314,1058519726,1058559004,1058598147,1058637154,1058676025,1058714760,1058753360,1058791823,1058830149,1058868339,1058906393,1058944309,1058982088,1059019730,1059057235,1059094602,1059131831,1059168923,1059205877,1059242692,1059279370,1059315909,1059352310,1059388573,1059424697,1059460682,1059496529,1059532236,1059567805,1059603235,1059638526,1059673678,1059708691,1059743565,1059778299,1059812894,1059847350,1059881667,1059915844,1059949882,1059983780,1060017540,1060051159,1060084640,1060117981,1060151183,1060184245,1060217168,1060249952,1060282596,1060315102,1060347468,1060379695,1060411782,1060443731,1060475541,1060507212,1060538744,1060570137,1060601391,1060632507,1060663484,1060694323,1060725023,1060755585,1060786009,1060816295,1060846442,1060876452,1060906324,1060936059,1060965656,1060995116,1061024438,1061053624,1061082672,1061111584,1061140359,1061168997,1061197499,1061225865,1061254095,1061282189,1061310147,1061337970,1061365658,1061393210,1061420628,1061447911,1061475059,1061502073,1061528952,1061555698,1061582310,1061608789,1061635134,1061661346,1061687426,1061713372,1061739187,1061764869,1061790419,1061815838,1061841125,1061866281,1061891306,1061916200,1061940964,1061965598,1061990102,1062014476,1062038721,1062062836,1062086823,1062110682,1062134412,1062158014,1062181489,1062204836,1062228056,1062251149,1062274116,1062296956,1062319671,1062342260,1062364724,1062387062,1062409276,1062431366,1062453332,1062475174,1062496892,1062518488,1062539961,1062561311,1062582540,1062603646,1062624631,1062645496,1062666239,1062686862,1062707365,1062727749,1062748013,1062768158,1062788185,1062808093,1062827884,1062847556,1062867112,1062886551,1062905874,1062925080,1062944171,1062963146,1062982006,1063000752,1063019384,1063037902,1063056306,1063074597,1063092776,1063110842,1063128796,1063146639,1063164371,1063181992,1063199502,1063216903,1063234194,1063251376,1063268449,1063285413,1063302270,1063319019,1063335661,1063352196,1063368625,1063384948,1063401165,1063417277,1063433284,1063449187,1063464986,1063480681,1063496274,1063511763,1063527150,1063542436,1063557620,1063572703,1063587685,1063602567,1063617349,1063632032,1063646616,1063661101,1063675488,1063689778,1063703970,1063718065,1063732064,1063745967,1063759774,1063773486,1063787103,1063800626,1063814055,1063827391,1063840633,1063853783,1063866841,1063879806,1063892681,1063905464,1063918157,1063930760,1063943273,1063955697,1063968032,1063980278,1063992437,1064004508,1064016492,1064028389,1064040200,1064051926,1064063565,1064075120,1064086590,1064097976,1064109278,1064120497,1064131633,1064142687,1064153658,1064164548,1064175357,1064186085,1064196732,1064207300,1064217788,1064228197,1064238527,1064248779,1064258954,1064269050,1064279070,1064289013,1064298880,1064308672,1064318387,1064328028,1064337595,1064347087,1064356506,1064365851,1064375124,1064384324,1064393452,1064402508,1064411494,1064420408,1064429252,1064438026,1064446730,1064455365,1064463931,1064472429,1064480859,1064489221,1064497517,1064505745,1064513907,1064522003,1064530033,1064537998,1064545898,1064553734,1064561506,1064569214,1064576859,1064584441,1064591961,1064599418,1064606814,1064614148,1064621422,1064628635,1064635788,1064642881,1064649914,1064656889,1064663805,1064670663,1064677462,1064684204,1064690889,1064697518,1064704090,1064710605,1064717065,1064723470,1064729820,1064736115,1064742356,1064748543,1064754677,1064760757,1064766785,1064772760,1064778683,1064784555,1064790375,1064796144,1064801862,1064807530,1064813148,1064818716,1064824236,1064829706,1064835127,1064840500,1064845826,1064851103,1064856334,1064861517,1064866654,1064871745,1064876790,1064881789,1064886743,1064891652,1064896516,1064901336,1064906112,1064910844,1064915533,1064920179,1064924783,1064929343,1064933862,1064938339,1064942775,1064947169,1064951523,1064955836,1064960108,1064964341,1064968535,1064972688,1064976803,1064980879,1064984917,1064988917,1064992878,1064996802,1065000689,1065004539,1065008352,1065012129,1065015869,1065019574,1065023243,1065026877,1065030476,1065034040,1065037570,1065041065,1065044527,1065047955,1065051349,1065054711,1065058039,1065061335,1065064599,1065067831,1065071031,1065074199,1065077336,1065080442,1065083517,1065086562,1065089576,1065092561,1065095515,1065098441,1065101336,1065104203,1065107041,1065109851,1065112632,1065115385,1065118110,1065120808,1065123478,1065126121,1065128738,1065131327,1065133890,1065136427,1065138938,1065141423,1065143883,1065146317,1065148726,1065151111,1065153470,1065155805,1065158116,1065160403,1065162666,1065164905,1065167121,1065169314,1065171484,1065173631,1065175756,1065177858,1065179938,1065181996,1065184032,1065186046,1065188039,1065190011,1065191962,1065193892,1065195802,1065197691,1065199560,1065201408,1065203237,1065205046,1065206836,1065208606,1065210357,1065212089,1065213803,1065215497,1065217174,1065218831,1065220471,1065222093,1065223697,1065225284,1065226853,1065228405,1065229939,1065231457,1065232958,1065234442,1065235910,1065237361,1065238797,1065240216,1065241620,1065243007,1065244380,1065245737,1065247078,1065248405,1065249716,1065251013,1065252295,1065253563,1065254816,1065256055,1065257280,1065258491,1065259689,1065260872,1065262042,1065263199,1065264342,1065265472,1065266589,1065267694,1065268785,1065269864,1065270931,1065271985,1065273027,1065274057,1065275075,1065276081,1065277075,1065278057,1065279028,1065279988,1065280937,1065281874,1065282800,1065283715,1065284620,1065285514,1065286397,1065287270,1065288132,1065288984,1065289826,1065290658,1065291480,1065292292,1065293095,1065293888,1065294671,1065295445,1065296209,1065296965,1065297711,1065298448,1065299176,1065299896,1065300606,1065301308,1065302002,1065302687,1065303364,1065304032,1065304692,1065305344,1065305988,1065306624,1065307253,1065307873,1065308486,1065309091,1065309689,1065310280,1065310863,1065311439,1065312007,1065312569,1065313123,1065313671,1065314212,1065314746,1065315273,1065315794,1065316308,1065316815,1065317317,1065317811,1065318300,1065318783,1065319259,1065319729,1065320193,1065320652,1065321104,1065321551,1065321992,1065322427,1065322857,1065323281,1065323700,1065324113,1065324521,1065324924,1065325322,1065325714,1065326102,1065326484,1065326861,1065327234,1065327601,1065327964,1065328322,1065328676,1065329024,1065329369,1065329708,1065330043,1065330374,1065330701,1065331023,1065331341,1065331654,1065331964,1065332269,1065332571,1065332868,1065333161,1065333451,1065333736,1065334018,1065334296,1065334570,1065334841,1065335108,1065335371,1065335631,1065335887,1065336140,1065336390,1065336636,1065336878,1065337118,1065337354,1065337587,1065337817,1065338043,1065338267,1065338488,1065338705,1065338919,1065339131,1065339340,1065339545,1065339748,1065339949,1065340146,1065340341,1065340533,1065340722,1065340909,1065341093,1065341274,1065341453,1065341630,1065341804,1065341975,1065342144,1065342311,1065342476,1065342638,1065342798,1065342956,1065343111,1065343264,1065343415,1065343564,1065343711,1065343856,1065343998,1065344139,1065344278,1065344414,1065344549,1065344682,1065344813,1065344942,1065345069,1065345194,1065345318,1065345440,1065345560,1065345678,1065345795,1065345910,1065346023,1065346134,1065346244,1065346353,1065346460,1065346565,1065346668,1065346771,1065346871,1065346971,1065347068,1065347165,1065347260,1065347353,1065347445,1065347536,1065347626,1065347714,1065347801,1065347886,1065347971,1065348054,1065348135,1065348216,1065348295,1065348374,1065348451,1065348527,1065348601,1065348675,1065348748,1065348819,1065348890,1065348959,1065349027,1065349094,1065349161,1065349226,1065349290,1065349353,1065349416,1065349477,1065349538,1065349597,1065349656,1065349714,1065349770,1065349826,1065349881,1065349936,1065349989,1065350042,1065350094,1065350145,1065350195,1065350244,1065350293,1065350341,1065350388,1065350435,1065350481,1065350526,1065350570,1065350614,1065350657,1065350699,1065350741,1065350782,1065350822,1065350862,1065350901,1065350939,1065350977,1065351015,1065351051,1065351088,1065351123,1065351158,1065351193,1065351227,1065351260,1065351293,1065351325,1065351357,1065351388,1065351419,1065351449,1065351479,1065351509,1065351537,1065351566,1065351594,1065351621,1065351648,1065351675,1065351701,1065351727,1065351753,1065351777,1065351802,1065351826,1065351850,1065351873,1065351896,1065351919,1065351941,1065351963,1065351985,1065352006,1065352027,1065352047,1065352068,1065352087,1065352107,1065352126,1065352145,1065352164,1065352182,1065352200,1065352218,1065352235,1065352252,1065352269,1065352286,1065352302,1065352318,1065352334,1065352349,1065352364,1065352379,1065352394,1065352408,1065352423,1065352437,1065352450,1065352464,1065352477,1065352490,1065352503,1065352516,1065352528,1065352540,1065352552,1065352564,1065352576,1065352587,1065352598,1065352609,1065352620,1065352631,1065352641,1065352652,1065352662,1065352672,1065352682,1065352691,1065352701,1065352710,1065352719,1065352728,1065352737,1065352745,1065352754,1065352762,1065352771,1065352779,1065352787,1065352794,1065352802,1065352810,1065352817,1065352824,1065352831,1065352838,1065352845,1065352852,1065352859,1065352865,1065352872,1065352878,1065352884,1065352890,1065352896,1065352902,1065352908,1065352914,1065352919,1065352925,1065352930,1065352936,1065352941,1065352946,1065352951,1065352956,1065352961,1065352965,1065352970,1065352975,1065352979,1065352984,1065352988,1065352992,1065352996,1065353001,1065353005,1065353009,1065353012,1065353016,1065353020,1065353024,1065353027,1065353031,1065353034,1065353038,1065353041,1065353045,1065353048,1065353051,1065353054,1065353057,1065353060,1065353063,1065353066,1065353069,1065353072,1065353075,1065353077,1065353080,1065353083,1065353085,1065353088,1065353090,1065353092,1065353095,1065353097,1065353099,1065353102,1065353104,1065353106,1065353108,1065353110,1065353112,1065353114,1065353116,1065353118,1065353120,1065353122,1065353124,1065353126,1065353127,1065353129,1065353131,1065353133,1065353134,1065353136,1065353137,1065353139,1065353140,1065353142,1065353143,1065353145,1065353146,1065353148,1065353149,1065353150,1065353152,1065353153,1065353154,1065353155,1065353156,1065353158,1065353159,1065353160,1065353161,1065353162,1065353163,1065353164,1065353165,1065353166,1065353167,1065353168,1065353169,1065353170,1065353171,1065353172,1065353173,1065353174,1065353175,1065353175,1065353176,1065353177,1065353178,1065353179,1065353179,1065353180,1065353181,1065353182,1065353182,1065353183,1065353184,1065353184,1065353185,1065353186,1065353186,1065353187,1065353187,1065353188,1065353189,1065353189,1065353190,1065353190,1065353191,1065353191,1065353192,1065353192,1065353193,1065353193,1065353194,1065353194,1065353195,1065353195,1065353195,1065353196,1065353196,1065353197,1065353197,1065353198,1065353198,1065353198,1065353199,1065353199,1065353199,1065353200,1065353200,1065353200,1065353201,1065353201,1065353201,1065353202,1065353202,1065353202,1065353203,1065353203,1065353203,1065353203,1065353204,1065353204,1065353204,1065353204,1065353205,1065353205,1065353205,1065353205,1065353206,1065353206,1065353206,1065353206,1065353206,1065353207,1065353207,1065353207,1065353207,1065353207,1065353208,1065353208,1065353208,1065353208,1065353208,1065353208,1065353209,1065353209,1065353209,1065353209,1065353209,1065353209,1065353210,1065353210,1065353210,1065353210,1065353210,1065353210,1065353210,1065353210,1065353211,1065353211,1065353211,1065353211,1065353211,1065353211,1065353211,1065353211,1065353211,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353212,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353213,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353214,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353215,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,1065353216,0,2147483648,1065353216,3212836864,2143289344,230102793,2377586441,788013594,2935497242,1,1065353216,3212836864,1057308539,1062714173,1065274737,1065353215,1065353216,1065353216] + }, "div3_sweep": { "seed": 20261030, "F_bits": [ - [ - 1082145540, - 1098403062, - 3236773855, - 1082947524, - 1091240558, - 3195272670 - ], - [ - 1080201846, - 1077581980, - 1075371045, - 1087989037, - 3238256091, - 3245719214 - ], - [ - 3229208364, - 3222326869, - 1095236995, - 1071083622, - 3213270037, - 1097812842 - ], - [ - 3232302537, - 1093846710, - 1070190081, - 1094936417, - 3215124028, - 3241628918 - ], - [ - 1080632337, - 1083223872, - 1083003334, - 3228842184, - 1046253854, - 3225871952 - ], - [ - 3203602644, - 1092000168, - 3183982224, - 3231506837, - 1074615167, - 3232031313 - ], - [ - 3210152053, - 3229800998, - 1077752811, - 3241191492, - 3224954582, - 3240897462 - ], - [ - 3241715199, - 3240786950, - 1067880505, - 1085536402, - 1092812339, - 3215031032 - ], - [ - 3214244755, - 3230304513, - 3237901357, - 1078296784, - 3207830138, - 3236456678 - ], - [ - 1091638997, - 3225574759, - 1089980077, - 1082534342, - 3234890736, - 3238666340 - ], - [ - 1084839833, - 3243590782, - 3213593874, - 1083447397, - 3226271089, - 3248561047 - ], - [ - 3239953664, - 3205416393, - 3216139257, - 1083063358, - 1084591626, - 1086687866 - ], - [ - 1038621193, - 1066435460, - 3210768317, - 3226937221, - 3183037605, - 1060157077 - ], - [ - 3241494042, - 3244208611, - 3241757183, - 1087520422, - 1078128303, - 3221451006 - ], - [ - 1094294444, - 3208797460, - 1067246810, - 1072304720, - 3218929643, - 3248874826 - ], - [ - 3238868998, - 3239394589, - 1049778639, - 3234565576, - 1095065799, - 3210801959 - ], - [ - 1080380566, - 1063684688, - 1090039524, - 1092297035, - 1087365587, - 1084394057 - ], - [ - 1067096621, - 3230748192, - 1082678342, - 3241213152, - 1091031129, - 1098958926 - ], - [ - 1074349695, - 1091842838, - 3213605801, - 1091290008, - 3237903505, - 3229982112 - ], - [ - 1028499863, - 3201366507, - 1075676410, - 3238577164, - 3212612164, - 1053441993 - ], - [ - 3246813618, - 3207936985, - 1074217361, - 1085673667, - 3219372029, - 3241551051 - ], - [ - 3213369428, - 1077857670, - 1086182873, - 3239895572, - 1094852851, - 1085703417 - ], - [ - 1090217637, - 3252147862, - 3229414209, - 1086323839, - 3236298702, - 1093268512 - ], - [ - 3218111630, - 3231419228, - 1075150957, - 3217023378, - 3225053030, - 3226723120 - ], - [ - 3220152929, - 3230242832, - 3212812463, - 3209110076, - 1087056972, - 1093212459 - ], - [ - 1084915087, - 3232057366, - 1093234587, - 1055527851, - 1089883549, - 1069226739 - ], - [ - 3242395904, - 1076766353, - 1052837851, - 1082257747, - 1083049070, - 1082818609 - ], - [ - 1069285192, - 3234931946, - 3227185525, - 1069743374, - 3230316291, - 3232262942 - ], - [ - 1099829901, - 3233171956, - 1084729142, - 3230952346, - 3234900111, - 3218899854 - ], - [ - 1084612571, - 1099560887, - 3242049841, - 3224703864, - 1084424204, - 3227658063 - ], - [ - 3207428305, - 3219570886, - 3232026802, - 1071264108, - 1070518086, - 3233676344 - ], - [ - 3233912964, - 3237852485, - 3225806340, - 1070210600, - 3239414568, - 3239226068 - ], - [ - 1085925708, - 1089906549, - 3232062548, - 1077358029, - 3229781570, - 3216848208 - ], - [ - 1089769587, - 1088154777, - 1071336670, - 1058723169, - 1069832038, - 1077136652 - ], - [ - 1089561109, - 1075365150, - 1084197895, - 3229905438, - 3216717410, - 1091715324 - ], - [ - 3233936570, - 3240065453, - 3239132511, - 3236366788, - 1088835152, - 3240519234 - ], - [ - 3241142576, - 3226979065, - 1076110027, - 3241254674, - 3227065756, - 1086103211 - ], - [ - 1083301178, - 1071066271, - 1059075199, - 1091534435, - 3212881171, - 3250714421 - ], - [ - 3230715952, - 1071919518, - 1090261900, - 3238106215, - 1066435702, - 1091588724 - ], - [ - 1089303686, - 1094403232, - 3242685773, - 1085380540, - 3232076885, - 1089275872 - ], - [ - 1091155270, - 1086375076, - 1094574263, - 1085745821, - 3231644438, - 1094757243 - ], - [ - 3250457295, - 3239820116, - 3225252399, - 3205802284, - 3207669809, - 1092340496 - ], - [ - 3237908895, - 1059985327, - 3223615684, - 1087246437, - 1075027008, - 1059442533 - ], - [ - 1088664205, - 3193965844, - 1090149440, - 3234721906, - 3245436712, - 1086696192 - ], - [ - 3181610765, - 3179692978, - 1096573562, - 1077020093, - 1092565768, - 1084674557 - ], - [ - 1064452977, - 1042621693, - 3221969346, - 1079835227, - 1086451947, - 3248158130 - ], - [ - 3232720413, - 3213574025, - 3237806536, - 3237998295, - 1084153162, - 3241298302 - ], - [ - 1094861134, - 1062912643, - 1087016375, - 1089684022, - 1051929275, - 3243524545 - ], - [ - 3242272532, - 1066352381, - 3238152740, - 3240560949, - 3226923739, - 3237899977 - ], - [ - 1088478070, - 3248869758, - 3215036592, - 1052767873, - 3231066904, - 3227154935 - ], - [ - 3231227768, - 1085091282, - 1085343542, - 3248765956, - 3231174852, - 3233588047 - ], - [ - 1093437547, - 3239346702, - 1071475950, - 1086337620, - 3218662235, - 3239459679 - ], - [ - 1088378512, - 1094755947, - 1093011760, - 1062800244, - 1078478801, - 1092633468 - ], - [ - 1084335307, - 1099185097, - 3234184467, - 3229683976, - 3226499211, - 3237233148 - ], - [ - 3235214980, - 1083238091, - 1091612036, - 3228632373, - 3205181666, - 3218147161 - ], - [ - 3241173572, - 1077364904, - 3229717215, - 1093711353, - 3240046429, - 3241296091 - ], - [ - 1081268605, - 1100745659, - 1093971353, - 1090577136, - 3243253133, - 3238529184 - ], - [ - 1071972187, - 3240987589, - 3234726671, - 3231453105, - 1089938256, - 1083725026 - ], - [ - 1075602172, - 1055125283, - 1083013855, - 3236557751, - 3244871022, - 3228654003 - ], - [ - 3228918386, - 1099545161, - 3225692063, - 1091155446, - 1096617626, - 1071294953 - ], - [ - 1089674222, - 1090415365, - 1084720014, - 3240938865, - 1078538098, - 3209979800 - ], - [ - 1092209915, - 1080510698, - 3215350769, - 1091428002, - 3222476428, - 3226774499 - ], - [ - 1083824023, - 1071291968, - 3232583875, - 1102943731, - 1085141951, - 1085555257 - ], - [ - 1086426624, - 3240418176, - 1075535417, - 1083943145, - 1089597918, - 1098718186 - ], - [ - 1092291465, - 1089676668, - 3240406455, - 3244566720, - 3240205053, - 1101173072 - ], - [ - 3233085741, - 1086823683, - 1083546343, - 3238983101, - 1097378112, - 1093664539 - ], - [ - 3237808824, - 1071358560, - 1094380441, - 3219296035, - 3222521563, - 3234411305 - ], - [ - 1066801767, - 1084525981, - 3237965676, - 3247765034, - 3224141770, - 1053364319 - ], - [ - 1088364679, - 1079754089, - 3238981240, - 1057944318, - 3230635689, - 3213784716 - ], - [ - 3243700287, - 1081586463, - 3254058134, - 3232588120, - 1089289430, - 3237486259 - ], - [ - 3228620345, - 3226758005, - 3217680633, - 1078818476, - 3243248452, - 3224631632 - ], - [ - 1095053496, - 3243883878, - 3238594836, - 1087780510, - 3239991951, - 3221229092 - ], - [ - 3240928485, - 3223367610, - 1083563596, - 3214998924, - 3233535413, - 1098532352 - ], - [ - 3225782629, - 1078150007, - 3240777557, - 1096135515, - 1076278399, - 3239589704 - ], - [ - 3241151369, - 3232205647, - 1067449162, - 1093326978, - 3211323617, - 1087207129 - ], - [ - 1091009335, - 3229659247, - 3179275014, - 3243043665, - 3229597165, - 3244833238 - ], - [ - 1093889073, - 3218713753, - 3210548871, - 1091638170, - 1054406913, - 1097094712 - ], - [ - 3246847421, - 1082527150, - 3230954852, - 1083083530, - 3243966118, - 1072252349 - ], - [ - 3230170206, - 1076694420, - 3227833335, - 1057779804, - 1085593379, - 1087309435 - ], - [ - 1091076446, - 3240619095, - 3234077465, - 3218075203, - 1095952034, - 3243737203 - ], - [ - 3217612901, - 1082715213, - 3249998120, - 1063883489, - 3247481784, - 1087674404 - ], - [ - 1098007316, - 3234596515, - 1077790999, - 1072176333, - 1083304802, - 3238640811 - ], - [ - 3237629051, - 1085464855, - 3236007557, - 3248446735, - 3209567921, - 1070464833 - ], - [ - 3238700538, - 1088816327, - 1084444110, - 3243592798, - 3239391527, - 3236175401 - ], - [ - 1073417224, - 3236958650, - 1072450137, - 1089785540, - 3243007873, - 3219838298 - ], - [ - 3224701098, - 1077994443, - 1096003933, - 3217563915, - 1094738160, - 3190644992 - ], - [ - 1094576925, - 1088395309, - 3220704925, - 3240853682, - 3216454378, - 3238566510 - ], - [ - 1069823791, - 3241852704, - 3233515664, - 1070595045, - 3222653289, - 3230398148 - ], - [ - 1084449360, - 3247378616, - 1087885636, - 3221586501, - 1095058097, - 1094265503 - ], - [ - 1048685016, - 3230267529, - 3240934900, - 1054645860, - 1093187070, - 1065078244 - ], - [ - 1092296691, - 3230010543, - 3238188520, - 3207053731, - 3208311523, - 1079629545 - ], - [ - 3242150526, - 1091743126, - 1091863803, - 1092180604, - 1093684083, - 3239383181 - ], - [ - 1085553272, - 3242558497, - 3231993989, - 3223278286, - 1080989905, - 3231543044 - ], - [ - 1077636687, - 3226505310, - 3238261785, - 1083477066, - 3231473702, - 1088762976 - ], - [ - 3213919698, - 3235389383, - 3229655335, - 3244570445, - 3236124959, - 1090738702 - ], - [ - 3232148943, - 1084105706, - 1068275236, - 1093948338, - 3217294205, - 1098502020 - ], - [ - 1076634057, - 3221346085, - 3237054213, - 3238783896, - 1063835058, - 3216116167 - ], - [ - 1083135707, - 3221836926, - 3222628858, - 1053722452, - 1086716669, - 1098167048 - ], - [ - 1091354357, - 3239964789, - 3211657662, - 3245073041, - 3187866917, - 1086273885 - ], - [ - 3238467533, - 1040972459, - 1051508167, - 1080252173, - 3211888311, - 3243280439 - ], - [ - 1085287183, - 3227518117, - 3240169364, - 1088969120, - 1068725320, - 1078410480 - ], - [ - 3242632219, - 1093945948, - 1061336311, - 3236528687, - 1093622673, - 3234616902 - ], - [ - 1077563261, - 3223701568, - 3230070165, - 1086411920, - 3209247207, - 3232468766 - ], - [ - 3225742816, - 3221509481, - 3235490416, - 3232761440, - 1088648416, - 3184533958 - ], - [ - 3235020965, - 3237645813, - 3226097337, - 3224942502, - 1079466477, - 3214841955 - ], - [ - 3241821829, - 1080224506, - 1081831279, - 1087929332, - 3231731171, - 1093136457 - ], - [ - 1064378410, - 1083625392, - 3238909519, - 3223590424, - 3239258237, - 3244595044 - ], - [ - 1073458954, - 3233385476, - 1088668524, - 1049874246, - 1086649147, - 1099676672 - ], - [ - 3226197619, - 3238515906, - 1085598600, - 1092954848, - 1074270408, - 1078464923 - ], - [ - 1069140898, - 3245090736, - 1091010731, - 3233257801, - 1090784739, - 3222043574 - ], - [ - 3224075825, - 1073223194, - 1086354438, - 1083262485, - 1091597677, - 1065743769 - ], - [ - 1087011208, - 1041053113, - 3236724035, - 3237394447, - 1082381784, - 1090227527 - ], - [ - 3226236446, - 3235094328, - 3243129447, - 3238688301, - 3202204278, - 1096524757 - ], - [ - 3222366600, - 1075184729, - 3229458207, - 3230210293, - 1096424653, - 1083297542 - ], - [ - 3242786214, - 3240520843, - 1088156458, - 1073818023, - 3239116029, - 3244464863 - ], - [ - 3240367105, - 1075614069, - 1101438259, - 3234557925, - 3218430574, - 3242558194 - ], - [ - 1078924541, - 3238686229, - 1065583761, - 1083578722, - 3226232349, - 3239724946 - ], - [ - 3231291597, - 3240825408, - 1073079895, - 3230294897, - 3244228760, - 1067253607 - ], - [ - 3224957232, - 1101873145, - 3239612409, - 1082034186, - 3242214037, - 1080445669 - ], - [ - 1076901108, - 1071844339, - 3223978139, - 1090974367, - 3225283449, - 1097887582 - ], - [ - 1065656853, - 1099869837, - 3246062395, - 1070907483, - 3191257675, - 1099017687 - ], - [ - 3230168301, - 1097907856, - 3238229594, - 3230069198, - 1075595452, - 3220712448 - ], - [ - 3229025793, - 1067099015, - 1081247660, - 3240894422, - 3217960081, - 3231504748 - ], - [ - 1083980617, - 1086098819, - 3247315427, - 1068586074, - 1088217070, - 3241417888 - ], - [ - 3233683892, - 3229784089, - 3220072440, - 3246445115, - 1095157210, - 1083206479 - ], - [ - 1090426902, - 3226818592, - 3243862166, - 1083455463, - 3240122477, - 3244383415 - ], - [ - 1071894483, - 1076731866, - 3227534489, - 3242075294, - 3230156487, - 1092273679 - ], - [ - 1083441334, - 1065289514, - 3234586937, - 1083497099, - 1085175295, - 3238240079 - ], - [ - 3239773548, - 3222941339, - 1083951029, - 1081174159, - 1082167173, - 3230739523 - ], - [ - 1083009250, - 1101416903, - 3242257745, - 3238243751, - 3237015038, - 3227621053 - ], - [ - 1086020781, - 3214080849, - 1089134120, - 3241348629, - 1089838049, - 1095848255 - ], - [ - 1067274764, - 978963674, - 3232378543, - 1075043150, - 3223502267, - 1079579182 - ], - [ - 1070246772, - 1082795909, - 1090975181, - 3226294024, - 1088989838, - 1084213084 - ], - [ - 1099867472, - 3225227634, - 1092273735, - 3239053565, - 1092058502, - 1094532699 - ], - [ - 3180604140, - 1085020724, - 1084997145, - 1094815785, - 1093255232, - 1087460472 - ], - [ - 3244587600, - 1088660471, - 1077744747, - 1093376628, - 1090705694, - 1086362119 - ], - [ - 3228354006, - 1089589183, - 3238661380, - 1077141230, - 3226580828, - 3230816024 - ], - [ - 3236760379, - 3242233701, - 3225989143, - 1083068375, - 1078751572, - 1090990349 - ], - [ - 1091036037, - 1078496374, - 1090524731, - 1092153075, - 1089712958, - 3213242749 - ], - [ - 3240299525, - 3218060973, - 3205163236, - 1067173361, - 1097535479, - 1095760542 - ], - [ - 1078508471, - 3231567863, - 3238216011, - 3230253291, - 3188024750, - 3220731384 - ], - [ - 1072563516, - 3227990811, - 3167527306, - 3223023341, - 3240245843, - 1098762684 - ], - [ - 1077905115, - 1086032187, - 3224738193, - 3201897479, - 3209711910, - 3230108855 - ], - [ - 3243258312, - 1090587741, - 3229523726, - 3210368726, - 3220343972, - 3242852386 - ], - [ - 3238831521, - 1066035242, - 1025325364, - 1092796112, - 1076953472, - 3232777465 - ], - [ - 1083655491, - 3243497557, - 3228783547, - 1092121522, - 3238519583, - 1087269010 - ], - [ - 3211343101, - 3223187409, - 3216530407, - 1075913333, - 1070430729, - 1080258906 - ], - [ - 1055300309, - 3234541717, - 1089976207, - 3248112622, - 3225888586, - 3225330603 - ], - [ - 1085245552, - 1082643089, - 1067606450, - 1078427096, - 3240107133, - 1088852043 - ], - [ - 3232339242, - 1073249035, - 1093192930, - 3225624711, - 3231615868, - 1095960034 - ], - [ - 1083320908, - 3238212915, - 1077698834, - 3229509055, - 1083181641, - 3236446352 - ], - [ - 1092663908, - 1089115904, - 1098172267, - 3224741002, - 3236574165, - 3237074416 - ], - [ - 3224664297, - 3227153161, - 1088793062, - 1092444441, - 1079656363, - 3233924721 - ], - [ - 1068429287, - 1082743649, - 3228516565, - 3242056873, - 1092266471, - 3230474288 - ], - [ - 3218208633, - 3234356227, - 3246449158, - 3231009652, - 3245130816, - 1079527056 - ], - [ - 1072805472, - 1087441654, - 1075854154, - 1064464035, - 1099253828, - 3228957584 - ], - [ - 3238755983, - 1090533387, - 3236375575, - 3218993151, - 1101723621, - 1075563050 - ], - [ - 3229879871, - 1096266978, - 1072489461, - 3234367144, - 1081638851, - 3207288489 - ], - [ - 1092379893, - 3173797727, - 3235062930, - 3219152234, - 3225710433, - 1092253874 - ], - [ - 3239152078, - 1067830050, - 3219489756, - 3220253234, - 3230595095, - 1090622513 - ], - [ - 3232699836, - 1077792174, - 1084528167, - 1077897024, - 3238754493, - 1084939997 - ], - [ - 3214926811, - 1082632928, - 1087661789, - 3243648919, - 3237538899, - 3239986215 - ], - [ - 1083876657, - 3239393670, - 3231006664, - 1082188555, - 1069569254, - 3231409090 - ], - [ - 1091853468, - 1075719729, - 3223840083, - 3224838218, - 1078630132, - 1090674465 - ], - [ - 3235280758, - 1093700615, - 1091091945, - 1074771353, - 1065639110, - 1080159715 - ], - [ - 1093161427, - 3216380788, - 1076733492, - 1089131588, - 3246914254, - 1093851068 - ], - [ - 1091561334, - 3244177898, - 3221830644, - 1091116799, - 3235441119, - 3234084724 - ], - [ - 3240107610, - 3240227829, - 1074650393, - 3215989353, - 1094679860, - 1082898585 - ], - [ - 3235153057, - 1079909632, - 3240992505, - 1075404185, - 1090943813, - 3201649455 - ], - [ - 1097576189, - 1092196083, - 1093794725, - 3211465573, - 1082258079, - 3236560564 - ], - [ - 3203541033, - 3230574619, - 1084715279, - 1076264639, - 3244399280, - 1090331377 - ], - [ - 1090578078, - 1087739535, - 3221130890, - 1095864995, - 3236571021, - 3229850541 - ], - [ - 1092203617, - 3215942365, - 3238158093, - 1078722637, - 3250564664, - 3239414795 - ], - [ - 3230701676, - 3234770261, - 1069816296, - 3242105220, - 1074498106, - 1099221816 - ], - [ - 3240297067, - 3239601348, - 1085961305, - 3238327827, - 1074304017, - 3200460039 - ], - [ - 1096318138, - 1086449118, - 3233561818, - 3238596605, - 1076465362, - 3231696298 - ], - [ - 1095593929, - 1093801874, - 3238218603, - 1099671073, - 3245453384, - 1068351271 - ], - [ - 1101589131, - 3220952187, - 3231669437, - 3224345203, - 3229675838, - 1082660656 - ], - [ - 3248351821, - 1089998574, - 1093561863, - 1089464475, - 3244653148, - 3220879816 - ], - [ - 3232628172, - 1085928027, - 3249442221, - 3239267519, - 3231009710, - 1089031876 - ], - [ - 3240670817, - 1101425217, - 3244767203, - 1061333919, - 3202001415, - 3208777532 - ], - [ - 1066289027, - 3206097724, - 1068605589, - 3197012659, - 1094820356, - 1090641240 - ], - [ - 1066861428, - 3210221048, - 1095888007, - 3209668355, - 3221537556, - 3221523831 - ], - [ - 1075983801, - 1050627424, - 1095035361, - 1088860252, - 3228803631, - 1075592152 - ], - [ - 3231367893, - 3214666814, - 1073561768, - 3185819914, - 1073947600, - 3216689609 - ], - [ - 3233944180, - 1094879542, - 1076123102, - 3225204483, - 1075704717, - 3239351952 - ], - [ - 3226823008, - 3232974131, - 1078830292, - 1077911381, - 3221200653, - 1052258898 - ], - [ - 3231957824, - 1090027952, - 1091427413, - 1090677010, - 1095004619, - 1091906502 - ], - [ - 1090597519, - 3235664636, - 3227490267, - 3245050262, - 1091661517, - 1081866196 - ], - [ - 1099512149, - 1090885051, - 1093816112, - 1092965700, - 3224455262, - 3222461457 - ], - [ - 3218518488, - 3238250422, - 3229935530, - 1074759205, - 1099198602, - 1082418076 - ], - [ - 1075846031, - 1078938512, - 1091501553, - 3225745357, - 3239314179, - 3239259628 - ], - [ - 1089753876, - 3222686808, - 3238929840, - 1060914852, - 1086935549, - 1072108409 - ], - [ - 3239292808, - 3240067929, - 1079194729, - 1056407399, - 3229753564, - 1094913482 - ], - [ - 1075362309, - 3241875683, - 1075615306, - 1067975244, - 1090803466, - 1094941533 - ], - [ - 3246619141, - 3215467845, - 1088006991, - 1100270013, - 3226818160, - 3238171575 - ], - [ - 3241533056, - 3239481464, - 1095664224, - 3229804895, - 3241217037, - 3231062966 - ], - [ - 1100074721, - 1089095409, - 3240853172, - 1089527666, - 3168353904, - 1092697689 - ], - [ - 1093211260, - 1091764226, - 1083265088, - 3233241616, - 1070453657, - 1068893551 - ], - [ - 3242097056, - 1095982124, - 1099768035, - 1093070352, - 1055271411, - 3229595705 - ], - [ - 3241606477, - 1100200953, - 1091888868, - 1083771776, - 1083183257, - 1089033663 - ], - [ - 3218084636, - 3225172507, - 1096286789, - 3228813273, - 3211220140, - 3225037824 - ], - [ - 1073156162, - 1088309316, - 1087319758, - 3231115948, - 3247394603, - 3166983447 - ], - [ - 1083560228, - 1074065368, - 1073948251, - 1084247274, - 3213351948, - 3241632009 - ], - [ - 3213512300, - 3232391117, - 3230749125, - 3247233959, - 1092783442, - 3226993333 - ], - [ - 3240395514, - 3206699627, - 1097546625, - 1074154174, - 3234332850, - 3237445786 - ], - [ - 1067732265, - 3243855710, - 1084830688, - 3231362716, - 3216723504, - 3220988999 - ], - [ - 3235595214, - 3225637680, - 3229480561, - 3231659837, - 1063814100, - 1093202168 - ], - [ - 1101177894, - 1090678597, - 1076256385, - 3230437699, - 1094501961, - 3232318439 - ], - [ - 1085168583, - 3225391369, - 1069276569, - 3222341482, - 1090757377, - 3219556767 - ], - [ - 3230756714, - 3240782568, - 3232630838, - 1084277546, - 1047533170, - 1086295366 - ], - [ - 3227546122, - 3236518442, - 1081933414, - 1084023826, - 3205784549, - 1098480467 - ], - [ - 3231780150, - 3238938517, - 3218208002, - 3241609964, - 1088182039, - 3243130916 - ], - [ - 1080717986, - 1096479500, - 3240327191, - 3231726613, - 3215196432, - 3238097329 - ], - [ - 3242159253, - 3234661629, - 1080424176, - 3211115057, - 3245237972, - 3233596761 - ], - [ - 1092550010, - 1083201889, - 3208112965, - 1090193693, - 3233179484, - 3227346678 - ], - [ - 3241893462, - 1053363614, - 1089954286, - 3250900539, - 3207331188, - 3232332474 - ], - [ - 3242954294, - 3230102208, - 3235962252, - 3239699006, - 3238814687, - 3230086965 - ], - [ - 1089829038, - 3224506818, - 1086785421, - 1070390038, - 3231257170, - 1087236952 - ], - [ - 3239536280, - 1044730087, - 1080884243, - 3207424725, - 1066404382, - 3235421195 - ], - [ - 1089819675, - 3249180356, - 3231640730, - 1094283469, - 3217917344, - 1078758806 - ], - [ - 1063038052, - 1083596578, - 1079561232, - 1082472723, - 1041638272, - 3240472663 - ], - [ - 1095559379, - 1077326605, - 1071263597, - 1094795428, - 1084858738, - 3231691701 - ], - [ - 1074735032, - 3225511395, - 3241527926, - 1076525680, - 1096656085, - 1083828240 - ], - [ - 1090888094, - 1086759159, - 1090711040, - 1101053268, - 3228147344, - 3223964847 - ], - [ - 3241070605, - 1063764815, - 1082290584, - 3227668827, - 1090042758, - 1078344268 - ], - [ - 1074009470, - 3238762376, - 3237694369, - 3234008622, - 3238643623, - 3250015224 - ], - [ - 1071460911, - 3238594735, - 1093969714, - 1093340861, - 3198127212, - 3237606524 - ], - [ - 3202589728, - 3222237941, - 1092523634, - 1083101174, - 1089459947, - 1090813901 - ], - [ - 1059344569, - 3232649949, - 3241074033, - 1082275827, - 3240851765, - 1067445529 - ], - [ - 3196138547, - 1091074200, - 3239242400, - 1094415355, - 1072761213, - 1078970190 - ], - [ - 1084785276, - 1088055917, - 3211109575, - 3235565230, - 1096939262, - 1089526876 - ], - [ - 1094128157, - 1078752779, - 3203933656, - 3194768781, - 1087138963, - 3235855860 - ], - [ - 3224936483, - 3246927899, - 1097262069, - 3206210949, - 1092313906, - 1071019362 - ], - [ - 1089047472, - 3237720583, - 3230331913, - 3220036523, - 1097468186, - 1096301510 - ], - [ - 3229618473, - 3217302500, - 1046159207, - 1086106001, - 1092626008, - 3234046438 - ], - [ - 3221818859, - 3215151289, - 1088704927, - 1087664418, - 3214727104, - 1093377573 - ], - [ - 1092952018, - 3222606212, - 3240551545, - 3240152996, - 3235544796, - 1086647389 - ], - [ - 1088833172, - 3187281105, - 1083522500, - 1085650759, - 3242962328, - 1085900163 - ], - [ - 3243420452, - 1097343336, - 3243098270, - 3215490754, - 1052980401, - 1076509486 - ], - [ - 1090812539, - 3237048756, - 1095014753, - 3234131353, - 1082424892, - 3246470210 - ], - [ - 3243271933, - 3248977622, - 1082872502, - 1075951343, - 3170383358, - 3237551596 - ], - [ - 1097086393, - 1071284796, - 1091413692, - 3239944677, - 1083974741, - 1083838250 - ], - [ - 1073271238, - 1093734842, - 3227244507, - 3232774352, - 1093051422, - 1055941954 - ], - [ - 1057966409, - 3233375009, - 3218149026, - 3240614577, - 3237000616, - 1094474280 - ], - [ - 1068733537, - 3214295480, - 3237420985, - 3243971320, - 1078316080, - 3236898375 - ], - [ - 3248948041, - 3213921090, - 1081701083, - 3249433870, - 3243272312, - 3239741801 - ], - [ - 3230009899, - 3238230713, - 3125788114, - 3229935610, - 3240299093, - 3222192959 - ], - [ - 3233581975, - 3245931143, - 3247114638, - 3233314114, - 1075832259, - 3214302024 - ], - [ - 3202204538, - 1093341357, - 1081921830, - 3247192305, - 1091737275, - 1005626958 - ], - [ - 1085806409, - 1084270604, - 3234572242, - 1087814717, - 3233464939, - 3215475456 - ], - [ - 3240089794, - 3242580170, - 3239040649, - 1092255052, - 1086164274, - 3238523054 - ], - [ - 1086801681, - 3240136603, - 3234149662, - 3194154187, - 3243707340, - 3225149015 - ], - [ - 3206178744, - 3247306001, - 3241429806, - 3228839822, - 1093402828, - 1095981480 - ], - [ - 1086114700, - 3174674649, - 3243416294, - 3230552426, - 1090614816, - 3231233158 - ], - [ - 1094976650, - 1098161082, - 3235058006, - 1106937211, - 1079890148, - 3211256667 - ] + [1082145540,1098403062,3236773855,1082947524,1091240558,3195272670], + [1080201846,1077581980,1075371045,1087989037,3238256091,3245719214], + [3229208364,3222326869,1095236995,1071083622,3213270037,1097812842], + [3232302537,1093846710,1070190081,1094936417,3215124028,3241628918], + [1080632337,1083223872,1083003334,3228842184,1046253854,3225871952], + [3203602644,1092000168,3183982224,3231506837,1074615167,3232031313], + [3210152053,3229800998,1077752811,3241191492,3224954582,3240897462], + [3241715199,3240786950,1067880505,1085536402,1092812339,3215031032], + [3214244755,3230304513,3237901357,1078296784,3207830138,3236456678], + [1091638997,3225574759,1089980077,1082534342,3234890736,3238666340], + [1084839833,3243590782,3213593874,1083447397,3226271089,3248561047], + [3239953664,3205416393,3216139257,1083063358,1084591626,1086687866], + [1038621193,1066435460,3210768317,3226937221,3183037605,1060157077], + [3241494042,3244208611,3241757183,1087520422,1078128303,3221451006], + [1094294444,3208797460,1067246810,1072304720,3218929643,3248874826], + [3238868998,3239394589,1049778639,3234565576,1095065799,3210801959], + [1080380566,1063684688,1090039524,1092297035,1087365587,1084394057], + [1067096621,3230748192,1082678342,3241213152,1091031129,1098958926], + [1074349695,1091842838,3213605801,1091290008,3237903505,3229982112], + [1028499863,3201366507,1075676410,3238577164,3212612164,1053441993], + [3246813618,3207936985,1074217361,1085673667,3219372029,3241551051], + [3213369428,1077857670,1086182873,3239895572,1094852851,1085703417], + [1090217637,3252147862,3229414209,1086323839,3236298702,1093268512], + [3218111630,3231419228,1075150957,3217023378,3225053030,3226723120], + [3220152929,3230242832,3212812463,3209110076,1087056972,1093212459], + [1084915087,3232057366,1093234587,1055527851,1089883549,1069226739], + [3242395904,1076766353,1052837851,1082257747,1083049070,1082818609], + [1069285192,3234931946,3227185525,1069743374,3230316291,3232262942], + [1099829901,3233171956,1084729142,3230952346,3234900111,3218899854], + [1084612571,1099560887,3242049841,3224703864,1084424204,3227658063], + [3207428305,3219570886,3232026802,1071264108,1070518086,3233676344], + [3233912964,3237852485,3225806340,1070210600,3239414568,3239226068], + [1085925708,1089906549,3232062548,1077358029,3229781570,3216848208], + [1089769587,1088154777,1071336670,1058723169,1069832038,1077136652], + [1089561109,1075365150,1084197895,3229905438,3216717410,1091715324], + [3233936570,3240065453,3239132511,3236366788,1088835152,3240519234], + [3241142576,3226979065,1076110027,3241254674,3227065756,1086103211], + [1083301178,1071066271,1059075199,1091534435,3212881171,3250714421], + [3230715952,1071919518,1090261900,3238106215,1066435702,1091588724], + [1089303686,1094403232,3242685773,1085380540,3232076885,1089275872], + [1091155270,1086375076,1094574263,1085745821,3231644438,1094757243], + [3250457295,3239820116,3225252399,3205802284,3207669809,1092340496], + [3237908895,1059985327,3223615684,1087246437,1075027008,1059442533], + [1088664205,3193965844,1090149440,3234721906,3245436712,1086696192], + [3181610765,3179692978,1096573562,1077020093,1092565768,1084674557], + [1064452977,1042621693,3221969346,1079835227,1086451947,3248158130], + [3232720413,3213574025,3237806536,3237998295,1084153162,3241298302], + [1094861134,1062912643,1087016375,1089684022,1051929275,3243524545], + [3242272532,1066352381,3238152740,3240560949,3226923739,3237899977], + [1088478070,3248869758,3215036592,1052767873,3231066904,3227154935], + [3231227768,1085091282,1085343542,3248765956,3231174852,3233588047], + [1093437547,3239346702,1071475950,1086337620,3218662235,3239459679], + [1088378512,1094755947,1093011760,1062800244,1078478801,1092633468], + [1084335307,1099185097,3234184467,3229683976,3226499211,3237233148], + [3235214980,1083238091,1091612036,3228632373,3205181666,3218147161], + [3241173572,1077364904,3229717215,1093711353,3240046429,3241296091], + [1081268605,1100745659,1093971353,1090577136,3243253133,3238529184], + [1071972187,3240987589,3234726671,3231453105,1089938256,1083725026], + [1075602172,1055125283,1083013855,3236557751,3244871022,3228654003], + [3228918386,1099545161,3225692063,1091155446,1096617626,1071294953], + [1089674222,1090415365,1084720014,3240938865,1078538098,3209979800], + [1092209915,1080510698,3215350769,1091428002,3222476428,3226774499], + [1083824023,1071291968,3232583875,1102943731,1085141951,1085555257], + [1086426624,3240418176,1075535417,1083943145,1089597918,1098718186], + [1092291465,1089676668,3240406455,3244566720,3240205053,1101173072], + [3233085741,1086823683,1083546343,3238983101,1097378112,1093664539], + [3237808824,1071358560,1094380441,3219296035,3222521563,3234411305], + [1066801767,1084525981,3237965676,3247765034,3224141770,1053364319], + [1088364679,1079754089,3238981240,1057944318,3230635689,3213784716], + [3243700287,1081586463,3254058134,3232588120,1089289430,3237486259], + [3228620345,3226758005,3217680633,1078818476,3243248452,3224631632], + [1095053496,3243883878,3238594836,1087780510,3239991951,3221229092], + [3240928485,3223367610,1083563596,3214998924,3233535413,1098532352], + [3225782629,1078150007,3240777557,1096135515,1076278399,3239589704], + [3241151369,3232205647,1067449162,1093326978,3211323617,1087207129], + [1091009335,3229659247,3179275014,3243043665,3229597165,3244833238], + [1093889073,3218713753,3210548871,1091638170,1054406913,1097094712], + [3246847421,1082527150,3230954852,1083083530,3243966118,1072252349], + [3230170206,1076694420,3227833335,1057779804,1085593379,1087309435], + [1091076446,3240619095,3234077465,3218075203,1095952034,3243737203], + [3217612901,1082715213,3249998120,1063883489,3247481784,1087674404], + [1098007316,3234596515,1077790999,1072176333,1083304802,3238640811], + [3237629051,1085464855,3236007557,3248446735,3209567921,1070464833], + [3238700538,1088816327,1084444110,3243592798,3239391527,3236175401], + [1073417224,3236958650,1072450137,1089785540,3243007873,3219838298], + [3224701098,1077994443,1096003933,3217563915,1094738160,3190644992], + [1094576925,1088395309,3220704925,3240853682,3216454378,3238566510], + [1069823791,3241852704,3233515664,1070595045,3222653289,3230398148], + [1084449360,3247378616,1087885636,3221586501,1095058097,1094265503], + [1048685016,3230267529,3240934900,1054645860,1093187070,1065078244], + [1092296691,3230010543,3238188520,3207053731,3208311523,1079629545], + [3242150526,1091743126,1091863803,1092180604,1093684083,3239383181], + [1085553272,3242558497,3231993989,3223278286,1080989905,3231543044], + [1077636687,3226505310,3238261785,1083477066,3231473702,1088762976], + [3213919698,3235389383,3229655335,3244570445,3236124959,1090738702], + [3232148943,1084105706,1068275236,1093948338,3217294205,1098502020], + [1076634057,3221346085,3237054213,3238783896,1063835058,3216116167], + [1083135707,3221836926,3222628858,1053722452,1086716669,1098167048], + [1091354357,3239964789,3211657662,3245073041,3187866917,1086273885], + [3238467533,1040972459,1051508167,1080252173,3211888311,3243280439], + [1085287183,3227518117,3240169364,1088969120,1068725320,1078410480], + [3242632219,1093945948,1061336311,3236528687,1093622673,3234616902], + [1077563261,3223701568,3230070165,1086411920,3209247207,3232468766], + [3225742816,3221509481,3235490416,3232761440,1088648416,3184533958], + [3235020965,3237645813,3226097337,3224942502,1079466477,3214841955], + [3241821829,1080224506,1081831279,1087929332,3231731171,1093136457], + [1064378410,1083625392,3238909519,3223590424,3239258237,3244595044], + [1073458954,3233385476,1088668524,1049874246,1086649147,1099676672], + [3226197619,3238515906,1085598600,1092954848,1074270408,1078464923], + [1069140898,3245090736,1091010731,3233257801,1090784739,3222043574], + [3224075825,1073223194,1086354438,1083262485,1091597677,1065743769], + [1087011208,1041053113,3236724035,3237394447,1082381784,1090227527], + [3226236446,3235094328,3243129447,3238688301,3202204278,1096524757], + [3222366600,1075184729,3229458207,3230210293,1096424653,1083297542], + [3242786214,3240520843,1088156458,1073818023,3239116029,3244464863], + [3240367105,1075614069,1101438259,3234557925,3218430574,3242558194], + [1078924541,3238686229,1065583761,1083578722,3226232349,3239724946], + [3231291597,3240825408,1073079895,3230294897,3244228760,1067253607], + [3224957232,1101873145,3239612409,1082034186,3242214037,1080445669], + [1076901108,1071844339,3223978139,1090974367,3225283449,1097887582], + [1065656853,1099869837,3246062395,1070907483,3191257675,1099017687], + [3230168301,1097907856,3238229594,3230069198,1075595452,3220712448], + [3229025793,1067099015,1081247660,3240894422,3217960081,3231504748], + [1083980617,1086098819,3247315427,1068586074,1088217070,3241417888], + [3233683892,3229784089,3220072440,3246445115,1095157210,1083206479], + [1090426902,3226818592,3243862166,1083455463,3240122477,3244383415], + [1071894483,1076731866,3227534489,3242075294,3230156487,1092273679], + [1083441334,1065289514,3234586937,1083497099,1085175295,3238240079], + [3239773548,3222941339,1083951029,1081174159,1082167173,3230739523], + [1083009250,1101416903,3242257745,3238243751,3237015038,3227621053], + [1086020781,3214080849,1089134120,3241348629,1089838049,1095848255], + [1067274764,978963674,3232378543,1075043150,3223502267,1079579182], + [1070246772,1082795909,1090975181,3226294024,1088989838,1084213084], + [1099867472,3225227634,1092273735,3239053565,1092058502,1094532699], + [3180604140,1085020724,1084997145,1094815785,1093255232,1087460472], + [3244587600,1088660471,1077744747,1093376628,1090705694,1086362119], + [3228354006,1089589183,3238661380,1077141230,3226580828,3230816024], + [3236760379,3242233701,3225989143,1083068375,1078751572,1090990349], + [1091036037,1078496374,1090524731,1092153075,1089712958,3213242749], + [3240299525,3218060973,3205163236,1067173361,1097535479,1095760542], + [1078508471,3231567863,3238216011,3230253291,3188024750,3220731384], + [1072563516,3227990811,3167527306,3223023341,3240245843,1098762684], + [1077905115,1086032187,3224738193,3201897479,3209711910,3230108855], + [3243258312,1090587741,3229523726,3210368726,3220343972,3242852386], + [3238831521,1066035242,1025325364,1092796112,1076953472,3232777465], + [1083655491,3243497557,3228783547,1092121522,3238519583,1087269010], + [3211343101,3223187409,3216530407,1075913333,1070430729,1080258906], + [1055300309,3234541717,1089976207,3248112622,3225888586,3225330603], + [1085245552,1082643089,1067606450,1078427096,3240107133,1088852043], + [3232339242,1073249035,1093192930,3225624711,3231615868,1095960034], + [1083320908,3238212915,1077698834,3229509055,1083181641,3236446352], + [1092663908,1089115904,1098172267,3224741002,3236574165,3237074416], + [3224664297,3227153161,1088793062,1092444441,1079656363,3233924721], + [1068429287,1082743649,3228516565,3242056873,1092266471,3230474288], + [3218208633,3234356227,3246449158,3231009652,3245130816,1079527056], + [1072805472,1087441654,1075854154,1064464035,1099253828,3228957584], + [3238755983,1090533387,3236375575,3218993151,1101723621,1075563050], + [3229879871,1096266978,1072489461,3234367144,1081638851,3207288489], + [1092379893,3173797727,3235062930,3219152234,3225710433,1092253874], + [3239152078,1067830050,3219489756,3220253234,3230595095,1090622513], + [3232699836,1077792174,1084528167,1077897024,3238754493,1084939997], + [3214926811,1082632928,1087661789,3243648919,3237538899,3239986215], + [1083876657,3239393670,3231006664,1082188555,1069569254,3231409090], + [1091853468,1075719729,3223840083,3224838218,1078630132,1090674465], + [3235280758,1093700615,1091091945,1074771353,1065639110,1080159715], + [1093161427,3216380788,1076733492,1089131588,3246914254,1093851068], + [1091561334,3244177898,3221830644,1091116799,3235441119,3234084724], + [3240107610,3240227829,1074650393,3215989353,1094679860,1082898585], + [3235153057,1079909632,3240992505,1075404185,1090943813,3201649455], + [1097576189,1092196083,1093794725,3211465573,1082258079,3236560564], + [3203541033,3230574619,1084715279,1076264639,3244399280,1090331377], + [1090578078,1087739535,3221130890,1095864995,3236571021,3229850541], + [1092203617,3215942365,3238158093,1078722637,3250564664,3239414795], + [3230701676,3234770261,1069816296,3242105220,1074498106,1099221816], + [3240297067,3239601348,1085961305,3238327827,1074304017,3200460039], + [1096318138,1086449118,3233561818,3238596605,1076465362,3231696298], + [1095593929,1093801874,3238218603,1099671073,3245453384,1068351271], + [1101589131,3220952187,3231669437,3224345203,3229675838,1082660656], + [3248351821,1089998574,1093561863,1089464475,3244653148,3220879816], + [3232628172,1085928027,3249442221,3239267519,3231009710,1089031876], + [3240670817,1101425217,3244767203,1061333919,3202001415,3208777532], + [1066289027,3206097724,1068605589,3197012659,1094820356,1090641240], + [1066861428,3210221048,1095888007,3209668355,3221537556,3221523831], + [1075983801,1050627424,1095035361,1088860252,3228803631,1075592152], + [3231367893,3214666814,1073561768,3185819914,1073947600,3216689609], + [3233944180,1094879542,1076123102,3225204483,1075704717,3239351952], + [3226823008,3232974131,1078830292,1077911381,3221200653,1052258898], + [3231957824,1090027952,1091427413,1090677010,1095004619,1091906502], + [1090597519,3235664636,3227490267,3245050262,1091661517,1081866196], + [1099512149,1090885051,1093816112,1092965700,3224455262,3222461457], + [3218518488,3238250422,3229935530,1074759205,1099198602,1082418076], + [1075846031,1078938512,1091501553,3225745357,3239314179,3239259628], + [1089753876,3222686808,3238929840,1060914852,1086935549,1072108409], + [3239292808,3240067929,1079194729,1056407399,3229753564,1094913482], + [1075362309,3241875683,1075615306,1067975244,1090803466,1094941533], + [3246619141,3215467845,1088006991,1100270013,3226818160,3238171575], + [3241533056,3239481464,1095664224,3229804895,3241217037,3231062966], + [1100074721,1089095409,3240853172,1089527666,3168353904,1092697689], + [1093211260,1091764226,1083265088,3233241616,1070453657,1068893551], + [3242097056,1095982124,1099768035,1093070352,1055271411,3229595705], + [3241606477,1100200953,1091888868,1083771776,1083183257,1089033663], + [3218084636,3225172507,1096286789,3228813273,3211220140,3225037824], + [1073156162,1088309316,1087319758,3231115948,3247394603,3166983447], + [1083560228,1074065368,1073948251,1084247274,3213351948,3241632009], + [3213512300,3232391117,3230749125,3247233959,1092783442,3226993333], + [3240395514,3206699627,1097546625,1074154174,3234332850,3237445786], + [1067732265,3243855710,1084830688,3231362716,3216723504,3220988999], + [3235595214,3225637680,3229480561,3231659837,1063814100,1093202168], + [1101177894,1090678597,1076256385,3230437699,1094501961,3232318439], + [1085168583,3225391369,1069276569,3222341482,1090757377,3219556767], + [3230756714,3240782568,3232630838,1084277546,1047533170,1086295366], + [3227546122,3236518442,1081933414,1084023826,3205784549,1098480467], + [3231780150,3238938517,3218208002,3241609964,1088182039,3243130916], + [1080717986,1096479500,3240327191,3231726613,3215196432,3238097329], + [3242159253,3234661629,1080424176,3211115057,3245237972,3233596761], + [1092550010,1083201889,3208112965,1090193693,3233179484,3227346678], + [3241893462,1053363614,1089954286,3250900539,3207331188,3232332474], + [3242954294,3230102208,3235962252,3239699006,3238814687,3230086965], + [1089829038,3224506818,1086785421,1070390038,3231257170,1087236952], + [3239536280,1044730087,1080884243,3207424725,1066404382,3235421195], + [1089819675,3249180356,3231640730,1094283469,3217917344,1078758806], + [1063038052,1083596578,1079561232,1082472723,1041638272,3240472663], + [1095559379,1077326605,1071263597,1094795428,1084858738,3231691701], + [1074735032,3225511395,3241527926,1076525680,1096656085,1083828240], + [1090888094,1086759159,1090711040,1101053268,3228147344,3223964847], + [3241070605,1063764815,1082290584,3227668827,1090042758,1078344268], + [1074009470,3238762376,3237694369,3234008622,3238643623,3250015224], + [1071460911,3238594735,1093969714,1093340861,3198127212,3237606524], + [3202589728,3222237941,1092523634,1083101174,1089459947,1090813901], + [1059344569,3232649949,3241074033,1082275827,3240851765,1067445529], + [3196138547,1091074200,3239242400,1094415355,1072761213,1078970190], + [1084785276,1088055917,3211109575,3235565230,1096939262,1089526876], + [1094128157,1078752779,3203933656,3194768781,1087138963,3235855860], + [3224936483,3246927899,1097262069,3206210949,1092313906,1071019362], + [1089047472,3237720583,3230331913,3220036523,1097468186,1096301510], + [3229618473,3217302500,1046159207,1086106001,1092626008,3234046438], + [3221818859,3215151289,1088704927,1087664418,3214727104,1093377573], + [1092952018,3222606212,3240551545,3240152996,3235544796,1086647389], + [1088833172,3187281105,1083522500,1085650759,3242962328,1085900163], + [3243420452,1097343336,3243098270,3215490754,1052980401,1076509486], + [1090812539,3237048756,1095014753,3234131353,1082424892,3246470210], + [3243271933,3248977622,1082872502,1075951343,3170383358,3237551596], + [1097086393,1071284796,1091413692,3239944677,1083974741,1083838250], + [1073271238,1093734842,3227244507,3232774352,1093051422,1055941954], + [1057966409,3233375009,3218149026,3240614577,3237000616,1094474280], + [1068733537,3214295480,3237420985,3243971320,1078316080,3236898375], + [3248948041,3213921090,1081701083,3249433870,3243272312,3239741801], + [3230009899,3238230713,3125788114,3229935610,3240299093,3222192959], + [3233581975,3245931143,3247114638,3233314114,1075832259,3214302024], + [3202204538,1093341357,1081921830,3247192305,1091737275,1005626958], + [1085806409,1084270604,3234572242,1087814717,3233464939,3215475456], + [3240089794,3242580170,3239040649,1092255052,1086164274,3238523054], + [1086801681,3240136603,3234149662,3194154187,3243707340,3225149015], + [3206178744,3247306001,3241429806,3228839822,1093402828,1095981480], + [1086114700,3174674649,3243416294,3230552426,1090614816,3231233158], + [1094976650,1098161082,3235058006,1106937211,1079890148,3211256667] ], "Q_bits": [ - [ - 1091043029, - 3236413372, - 1072960732, - 1091846284, - 3228822993, - 1049260509 - ], - [ - 1089604410, - 3209765174, - 1066995900, - 1096892626, - 3230859161, - 1083373091 - ], - [ - 1089028828, - 3217473928, - 1083471770, - 1081433203, - 3211087247, - 1085197451 - ], - [ - 1092670177, - 3231841514, - 1063598103, - 1103584540, - 3213008072, - 1079116996 - ], - [ - 1090035815, - 3218609985, - 1072967049, - 1088661766, - 3198072896, - 1044746985 - ], - [ - 1046480449, - 3229984088, - 1043699741, - 1091873413, - 3199047408, - 1066277130 - ], - [ - 1065930110, - 3223301471, - 1068698187, - 1101831271, - 3220211818, - 1078129601 - ], - [ - 1102355112, - 3233401852, - 1061806677, - 1094438136, - 3230801565, - 1056751296 - ], - [ - 1071781071, - 3223814534, - 1074126143, - 1087694622, - 3207608420, - 1072523139 - ], - [ - 1100286232, - 3220852743, - 1078104567, - 1091432475, - 3228459147, - 1075102743 - ], - [ - 1093740897, - 3236214117, - 1056326492, - 1092346849, - 3221398009, - 1086728797 - ], - [ - 1100593070, - 3206078492, - 1056922408, - 1091962285, - 3221334560, - 1075874083 - ], - [ - 1057422500, - 3204308046, - 1055447604, - 1086751416, - 3193875437, - 1057501718 - ], - [ - 1102133900, - 3236833396, - 1079289965, - 1096423706, - 3211031308, - 1055779424 - ], - [ - 1102942431, - 3208201125, - 1061303639, - 1082397171, - 3215086253, - 1087148488 - ], - [ - 1099507992, - 3232003671, - 1051027024, - 1094935519, - 3233065923, - 1055457829 - ], - [ - 1089783516, - 3204365081, - 1078144728, - 1100944492, - 3224171091, - 1074309607 - ], - [ - 1077401221, - 3224265922, - 1072518675, - 1101852937, - 3228400449, - 1085998811 - ], - [ - 1083732000, - 3229825614, - 1056330999, - 1099937113, - 3230554319, - 1060965739 - ], - [ - 1051341681, - 3204304237, - 1067216203, - 1099216030, - 3210459266, - 1053140025 - ], - [ - 1107586516, - 3207674385, - 1066156629, - 1094575525, - 3215323668, - 1079011965 - ], - [ - 1069952171, - 3210407666, - 1075530577, - 1100534958, - 3232852127, - 1075204003 - ], - [ - 1099015047, - 3244274170, - 1059425851, - 1095226244, - 3229746387, - 1082150661 - ], - [ - 1076727725, - 3224947536, - 1066836675, - 1075619170, - 3220313669, - 1051532035 - ], - [ - 1078798162, - 3223751731, - 1055999341, - 1064175839, - 3223857090, - 1082095571 - ], - [ - 1093816228, - 3225594692, - 1082125304, - 1068063030, - 3226723980, - 1062858025 - ], - [ - 1103035975, - 3207810284, - 1052807795, - 1091155427, - 3218244981, - 1072712297 - ], - [ - 1079617615, - 3228500725, - 1053610061, - 1080080568, - 3223826525, - 1066614846 - ], - [ - 1108348671, - 3226723032, - 1074538891, - 1091318071, - 3228468606, - 1056778564 - ], - [ - 1093513399, - 3238115488, - 1079684449, - 1084509311, - 3221098552, - 1055546581 - ], - [ - 1060217161, - 3215430168, - 1066270533, - 1081615201, - 3200891155, - 1068640992 - ], - [ - 1094282331, - 3230528647, - 1043740607, - 1080552362, - 3232023744, - 1075864891 - ], - [ - 1094827784, - 3226747240, - 1066322787, - 1086753029, - 3223281656, - 1056962835 - ], - [ - 1098674178, - 3224972798, - 1064467871, - 1070679040, - 3201856793, - 1068261005 - ], - [ - 1098465593, - 3203796861, - 1074175242, - 1090019441, - 3213887196, - 1080081668 - ], - [ - 1094305959, - 3232677516, - 1075737660, - 1096738054, - 3225662719, - 1077618269 - ], - [ - 1101782342, - 3221762042, - 1067527878, - 1101894470, - 3221806563, - 1075476347 - ], - [ - 1092200435, - 3199955949, - 1057000008, - 1100181632, - 3210640283, - 1089607613 - ], - [ - 1091081282, - 3198111811, - 1078294936, - 1098582062, - 3204307983, - 1079911147 - ], - [ - 1098208036, - 3232400609, - 1080540881, - 1094282131, - 3225614473, - 1077628547 - ], - [ - 1099802322, - 3223162142, - 1083027269, - 1094647742, - 3225176036, - 1083150021 - ], - [ - 1111230486, - 3232431131, - 1038786741, - 1054715600, - 3207509192, - 1080923187 - ], - [ - 1098281059, - 3203566345, - 1051946752, - 1096149533, - 3201774998, - 1057171943 - ], - [ - 1097568201, - 3199360717, - 1078218979, - 1095091979, - 3238033375, - 1075879743 - ], - [ - 1052479147, - 3191630079, - 1084367540, - 1086413968, - 3230553481, - 1074501560 - ], - [ - 1075167289, - 3196807132, - 1055156632, - 1089236973, - 3223240571, - 1086189751 - ], - [ - 1093088550, - 3211434454, - 1074061134, - 1098370506, - 3220538954, - 1078670917 - ], - [ - 1103509241, - 3204269792, - 1076097309, - 1098588569, - 3200419693, - 1081669124 - ], - [ - 1102912576, - 3204328820, - 1074401080, - 1101200549, - 3221733624, - 1074125197 - ], - [ - 1097381957, - 3240993366, - 1056752488, - 1066573639, - 3224589810, - 1053479095 - ], - [ - 1091593928, - 3221848363, - 1074958575, - 1109539036, - 3224699448, - 1068515820 - ], - [ - 1102085329, - 3231955557, - 1064572791, - 1095240036, - 3214942400, - 1076182333 - ], - [ - 1097282340, - 3232754826, - 1081825847, - 1074320528, - 3211829949, - 1081317256 - ], - [ - 1093235834, - 3237475214, - 1069358485, - 1089575574, - 3221515398, - 1073593049 - ], - [ - 1095585441, - 3218639642, - 1079942549, - 1088451430, - 3205924932, - 1056898492 - ], - [ - 1101813346, - 3209253481, - 1060102748, - 1102359203, - 3232658413, - 1078667933 - ], - [ - 1090596195, - 3239302235, - 1082622658, - 1099223942, - 3235875623, - 1074915604 - ], - [ - 1082229665, - 3233603221, - 1070119424, - 1091819602, - 3226779304, - 1073850881 - ], - [ - 1084990589, - 3201629230, - 1072981551, - 1096929139, - 3237497239, - 1058112477 - ], - [ - 1088738155, - 3238099732, - 1041776969, - 1099802498, - 3234623119, - 1064436416 - ], - [ - 1098578764, - 3227261602, - 1074532649, - 1101578575, - 3211964151, - 1055196331 - ], - [ - 1100857344, - 3214577377, - 1056814328, - 1100075159, - 3217630961, - 1051774860 - ], - [ - 1092723953, - 3199520085, - 1067079717, - 1111462727, - 3221900379, - 1075102994 - ], - [ - 1095329111, - 3233031670, - 1067114571, - 1092843219, - 3226435042, - 1085803409 - ], - [ - 1100938920, - 3226514717, - 1077465688, - 1105207197, - 3232817693, - 1088959693 - ], - [ - 1093454282, - 3223619523, - 1073714443, - 1099622143, - 3235385805, - 1082416661 - ], - [ - 1098180936, - 3199385064, - 1082897221, - 1077930197, - 3217678318, - 1069677373 - ], - [ - 1077101821, - 3221266930, - 1074170224, - 1108538030, - 3219369158, - 1053097613 - ], - [ - 1097268499, - 3213753194, - 1075531812, - 1069862380, - 3224151522, - 1056396592 - ], - [ - 1104340617, - 3215732934, - 1092296264, - 1092956104, - 3226122821, - 1073841356 - ], - [ - 1088439372, - 3221648463, - 1056942244, - 1088217732, - 3235870930, - 1048059745 - ], - [ - 1103701642, - 3236507915, - 1075005201, - 1096683966, - 3232603705, - 1056001687 - ], - [ - 1101568192, - 3218563316, - 1073738156, - 1073343116, - 3227090505, - 1085679048 - ], - [ - 1085592714, - 3211081045, - 1077967561, - 1104783859, - 3206588190, - 1076358869 - ], - [ - 1101791137, - 3225744941, - 1061464897, - 1101974731, - 3209708481, - 1076226849 - ], - [ - 1099656327, - 3223156863, - 1041381609, - 1103683871, - 3223102136, - 1082778722 - ], - [ - 1102536966, - 3214970135, - 1055379917, - 1100285405, - 3201382906, - 1084716608 - ], - [ - 1107620323, - 3217150008, - 1063998667, - 1091982485, - 3236590348, - 1065155017 - ], - [ - 1090534540, - 3207633065, - 1056230637, - 1069689274, - 3222363161, - 1076296303 - ], - [ - 1099723466, - 3233233367, - 1069207772, - 1076690682, - 3233955385, - 1081954961 - ], - [ - 1076220201, - 3217545521, - 1088650328, - 1074875922, - 3239603715, - 1076543947 - ], - [ - 1106655939, - 3228162236, - 1068725220, - 1082332505, - 3218778715, - 1075067923 - ], - [ - 1098001065, - 3222231517, - 1071902060, - 1109219790, - 3208666549, - 1063807552 - ], - [ - 1099339460, - 3225643643, - 1074343875, - 1104233109, - 3232000594, - 1072134376 - ], - [ - 1082957244, - 3230078757, - 1065302661, - 1098690139, - 3235629725, - 1056541784 - ], - [ - 1084506532, - 3210723662, - 1083985875, - 1076170306, - 3232736965, - 1047285257 - ], - [ - 1103224973, - 3225216826, - 1056233613, - 1101493368, - 3213743097, - 1074966549 - ], - [ - 1080161792, - 3234471232, - 1068413091, - 1080940358, - 3217816440, - 1062284975 - ], - [ - 1093350013, - 3239500404, - 1076687188, - 1080611896, - 3233058191, - 1082820091 - ], - [ - 1063231241, - 3223776879, - 1078180184, - 1067590286, - 3231178451, - 1059646026 - ], - [ - 1100944148, - 3223515100, - 1074450045, - 1059280169, - 3207904617, - 1070019217 - ], - [ - 1102790543, - 3229725155, - 1080281611, - 1100828023, - 3231678084, - 1076078422 - ], - [ - 1094455022, - 3235179122, - 1066222523, - 1083075957, - 3215094194, - 1065558119 - ], - [ - 1087032571, - 3221518535, - 1074550266, - 1092376557, - 3225002817, - 1077281519 - ], - [ - 1071104213, - 3228962128, - 1059898027, - 1105210923, - 3229658854, - 1078765091 - ], - [ - 1092516390, - 3220440852, - 1062117239, - 1102596245, - 3214201943, - 1085658749 - ], - [ - 1086026571, - 3216439392, - 1073346887, - 1099422854, - 3204379426, - 1056920191 - ], - [ - 1092034736, - 3216958217, - 1054127876, - 1067092274, - 3223510484, - 1085434555 - ], - [ - 1100001486, - 3232576427, - 1055704775, - 1105713596, - 3196424194, - 1075592519 - ], - [ - 1099106349, - 3196138306, - 1052055868, - 1089654846, - 3210038815, - 1081340919 - ], - [ - 1094188687, - 3222038690, - 1077144736, - 1097873288, - 3203033027, - 1069162796 - ], - [ - 1103272341, - 3231941231, - 1058033936, - 1096900057, - 3231616364, - 1069965723 - ], - [ - 1086958916, - 3218911365, - 1061248307, - 1095314395, - 3208473479, - 1066913363 - ], - [ - 1085552745, - 3216612362, - 1071184555, - 1093129624, - 3225473466, - 1043939384 - ], - [ - 1095391277, - 3230424650, - 1047785248, - 1084749054, - 3213436778, - 1056708733 - ], - [ - 1102461769, - 3214266904, - 1071553319, - 1096832883, - 3225264004, - 1081993437 - ], - [ - 1075129161, - 3219445619, - 1075434156, - 1083389995, - 3231866667, - 1082618842 - ], - [ - 1082978243, - 3226938936, - 1077217580, - 1064587046, - 3223441662, - 1086959047 - ], - [ - 1086009270, - 3231120470, - 1075132549, - 1101602500, - 3196355265, - 1069201171 - ], - [ - 1079471754, - 3237717403, - 1079132087, - 1093626524, - 3227903069, - 1055054797 - ], - [ - 1083878117, - 3190897726, - 1075647329, - 1092161689, - 3229543087, - 1060084615 - ], - [ - 1095914137, - 3196172703, - 1072892051, - 1097766331, - 3216843463, - 1078271721 - ], - [ - 1086048239, - 3228664541, - 1081137849, - 1099327217, - 3204671138, - 1084334845 - ], - [ - 1082157913, - 3202736730, - 1059499911, - 1090574705, - 3234429547, - 1073372255 - ], - [ - 1103426369, - 3233134738, - 1076870752, - 1083197243, - 3231723761, - 1082531445 - ], - [ - 1101006645, - 3204825680, - 1089314095, - 1094927862, - 3214817556, - 1080369144 - ], - [ - 1088324073, - 3231291731, - 1059952347, - 1092478345, - 3221378065, - 1076542384 - ], - [ - 1091657854, - 3233440452, - 1065562133, - 1090659470, - 3236853590, - 1061309065 - ], - [ - 1084763850, - 3240431199, - 1076389685, - 1090979688, - 3234833666, - 1070589571 - ], - [ - 1086294573, - 3198298188, - 1050874597, - 1099621345, - 3220551889, - 1085247484 - ], - [ - 1075936837, - 3238425004, - 1083603200, - 1081255557, - 3198131371, - 1086077443 - ], - [ - 1090532631, - 3235916948, - 1074506238, - 1090347627, - 3204774072, - 1056230548 - ], - [ - 1088845823, - 3204080906, - 1071148037, - 1101534119, - 3214563346, - 1065501264 - ], - [ - 1092880736, - 3222880087, - 1085061772, - 1078910592, - 3225036010, - 1078832307 - ], - [ - 1094053042, - 3223284226, - 1056467187, - 1107139683, - 3233157689, - 1073246904 - ], - [ - 1099119729, - 3221679601, - 1082122889, - 1092354926, - 3232734778, - 1082476759 - ], - [ - 1082190516, - 3207725434, - 1055054211, - 1102715293, - 3223663795, - 1080833288 - ], - [ - 1092340778, - 3204448135, - 1069923735, - 1092396616, - 3221934601, - 1074520580 - ], - [ - 1100412892, - 3218118037, - 1074005991, - 1090548881, - 3216389450, - 1063341709 - ], - [ - 1091908100, - 3239974398, - 1079964551, - 1098857271, - 3230107148, - 1055400040 - ], - [ - 1094922938, - 3212009411, - 1077532665, - 1101988450, - 3226677962, - 1083881541 - ], - [ - 1077582004, - 3165056772, - 1066782681, - 1084429016, - 3218703736, - 1069983947 - ], - [ - 1080588875, - 3217714886, - 1079084139, - 1086106024, - 3225819434, - 1074185650 - ], - [ - 1108386246, - 3220494206, - 1080833364, - 1099692636, - 3230042836, - 1082999383 - ], - [ - 1051991608, - 3221775905, - 1074722067, - 1103463883, - 3231246988, - 1076398810 - ], - [ - 1105228080, - 3225485685, - 1068692477, - 1102024394, - 3227743438, - 1075652555 - ], - [ - 1088172344, - 3226426203, - 1075095978, - 1086535515, - 3221557375, - 1063575888 - ], - [ - 1097131892, - 3234853388, - 1046393491, - 1091967309, - 3212445299, - 1079104597 - ], - [ - 1099683040, - 3211869746, - 1078476253, - 1100800485, - 3226551431, - 1056186145 - ], - [ - 1100939044, - 3214617936, - 1053102900, - 1077479108, - 3235543599, - 1083822752 - ], - [ - 1087906897, - 3225098356, - 1074487657, - 1090617785, - 3196509175, - 1056222801 - ], - [ - 1082527501, - 3222280945, - 1035606372, - 1082819364, - 3232858649, - 1085833187 - ], - [ - 1087301817, - 3222812008, - 1046988716, - 1052235375, - 3208752935, - 1061371855 - ], - [ - 1103898560, - 3227505172, - 1059609851, - 1066168763, - 3215842988, - 1080765109 - ], - [ - 1099470499, - 3204392713, - 1038688956, - 1101443719, - 3208267177, - 1067358597 - ], - [ - 1092555211, - 3236120663, - 1058341335, - 1100768922, - 3231124168, - 1076268861 - ], - [ - 1067229940, - 3218375224, - 1056951316, - 1085303063, - 3201025831, - 1070459243 - ], - [ - 1067941321, - 3228106924, - 1078101952, - 1108885649, - 3221176472, - 1034668964 - ], - [ - 1094147016, - 3217393975, - 1061589831, - 1087825299, - 3232719370, - 1077341804 - ], - [ - 1092706927, - 3190613654, - 1082069328, - 1085434171, - 3225147055, - 1083956455 - ], - [ - 1092220192, - 3230815729, - 1068659965, - 1089330212, - 3218521874, - 1072508879 - ], - [ - 1101311477, - 3225947118, - 1085438048, - 1084546624, - 3229885141, - 1073374692 - ], - [ - 1084469556, - 3221851437, - 1077301883, - 1101091944, - 3213645893, - 1068992292 - ], - [ - 1078751918, - 3217605226, - 1057867021, - 1102696868, - 3230252236, - 1062522457 - ], - [ - 1076826349, - 3227919673, - 1083901239, - 1091375470, - 3237757563, - 1069947433 - ], - [ - 1082649326, - 3224248441, - 1067344119, - 1075172943, - 3237612959, - 1058645272 - ], - [ - 1099394929, - 3227395350, - 1072411113, - 1077623027, - 3240281497, - 1067134501 - ], - [ - 1089968201, - 3234271370, - 1065331984, - 1094736919, - 3215788807, - 1054156869 - ], - [ - 1101027376, - 3189117113, - 1070589257, - 1077784387, - 3220992744, - 1080806640 - ], - [ - 1099791189, - 3203700268, - 1056641233, - 1078899671, - 3224110234, - 1078608268 - ], - [ - 1093067949, - 3210255723, - 1074401408, - 1087293702, - 3231360362, - 1074683022 - ], - [ - 1073194194, - 3217372611, - 1076535389, - 1104289240, - 3230370845, - 1076896624 - ], - [ - 1092776651, - 3232002747, - 1064155765, - 1091086118, - 3202175352, - 1065358923 - ], - [ - 1100500779, - 3205115912, - 1051305092, - 1084644293, - 3212171960, - 1078678393 - ], - [ - 1095651269, - 3231694699, - 1079241609, - 1084155880, - 3204438510, - 1070389979 - ], - [ - 1101809136, - 3213702714, - 1067973851, - 1098035846, - 3239035357, - 1082541905 - ], - [ - 1100208541, - 3236802613, - 1055338157, - 1099763835, - 3229014298, - 1069218003 - ], - [ - 1100747067, - 3232840562, - 1066473038, - 1074561367, - 3232678423, - 1072822624 - ], - [ - 1095523471, - 3213923549, - 1078258004, - 1084791728, - 3228224221, - 1051790635 - ], - [ - 1106224753, - 3230181371, - 1082504077, - 1067362120, - 3216581984, - 1072666553 - ], - [ - 1046976464, - 3224089407, - 1074529411, - 1085655768, - 3237024489, - 1078341855 - ], - [ - 1099224884, - 3224551183, - 1056049125, - 1104513292, - 3229883557, - 1060539671 - ], - [ - 1100851044, - 3213461492, - 1074408407, - 1088121640, - 3242689837, - 1076121369 - ], - [ - 1091066982, - 3228337582, - 1063312036, - 1102745226, - 3198190444, - 1086350576 - ], - [ - 1100936585, - 3232211388, - 1075379715, - 1098966577, - 3196643957, - 1051361888 - ], - [ - 1104966512, - 3223237684, - 1068478608, - 1099235480, - 3207062257, - 1065784933 - ], - [ - 1104242177, - 3231796459, - 1074491203, - 1108189827, - 3238041728, - 1062176831 - ], - [ - 1110108044, - 3216166509, - 1065745259, - 1084148895, - 3223173793, - 1072494251 - ], - [ - 1109124868, - 3226840298, - 1082347708, - 1098368909, - 3237278907, - 1056160569 - ], - [ - 1092996202, - 3222705547, - 1087907203, - 1099906676, - 3224531708, - 1077463493 - ], - [ - 1101310449, - 3239982722, - 1082734402, - 1073389762, - 3204600194, - 1054768468 - ], - [ - 1076580604, - 3206518913, - 1062375635, - 1056855987, - 3232819499, - 1078633547 - ], - [ - 1077162421, - 3209056984, - 1083908184, - 1065393060, - 3216642055, - 1055700872 - ], - [ - 1085373818, - 3199832705, - 1083336558, - 1097764360, - 3222696841, - 1067155485 - ], - [ - 1091734265, - 3212668618, - 1065740503, - 1054141646, - 3189921283, - 1056958425 - ], - [ - 1094313576, - 3232878926, - 1067537255, - 1085012193, - 3205074944, - 1076035991 - ], - [ - 1086636829, - 3226522931, - 1069458383, - 1087308102, - 3216298376, - 1052484089 - ], - [ - 1092325023, - 3226870003, - 1079693811, - 1099323861, - 3233004503, - 1080339100 - ], - [ - 1099244334, - 3229239659, - 1054875797, - 1105690814, - 3229642921, - 1071577537 - ], - [ - 1108030887, - 3228105602, - 1082518437, - 1101613355, - 3219694546, - 1054416655 - ], - [ - 1077141201, - 3230853459, - 1060815437, - 1084143671, - 3237502280, - 1072158969 - ], - [ - 1085235483, - 3212850550, - 1079793709, - 1085555296, - 3231922878, - 1075910516 - ], - [ - 1098658458, - 3217851566, - 1075461829, - 1072956889, - 3223733464, - 1065047403 - ], - [ - 1099931975, - 3232680002, - 1069714405, - 1068531967, - 3223253090, - 1083254818 - ], - [ - 1084749663, - 3234494283, - 1067172175, - 1078292152, - 3227940883, - 1083273632 - ], - [ - 1107392016, - 3213199104, - 1076769456, - 1108788824, - 3221679379, - 1074426857 - ], - [ - 1102172924, - 3232090953, - 1083758192, - 1089817933, - 3233833476, - 1064326076 - ], - [ - 1108593514, - 3225926362, - 1078069753, - 1098432133, - 3186451118, - 1081403616 - ], - [ - 1101858982, - 3229746414, - 1073327588, - 1093610322, - 3200990837, - 1062599811 - ], - [ - 1102737060, - 3233985576, - 1087081244, - 1101718036, - 3201677679, - 1059730171 - ], - [ - 1102246363, - 3238756685, - 1080315359, - 1092671642, - 3218525247, - 1077464703 - ], - [ - 1076700275, - 3220437220, - 1084175413, - 1088632784, - 3209647715, - 1043312092 - ], - [ - 1082825858, - 3225129599, - 1076303310, - 1091481935, - 3239516413, - 1035358543 - ], - [ - 1092459827, - 3192713773, - 1065959056, - 1093147702, - 3211180987, - 1079121165 - ], - [ - 1070252127, - 3225932807, - 1063371152, - 1108006903, - 3230772495, - 1052773384 - ], - [ - 1101035063, - 3206902014, - 1085019219, - 1083535384, - 3227896070, - 1073813560 - ], - [ - 1078045941, - 3236479681, - 1074608317, - 1091729080, - 3213890531, - 1056113067 - ], - [ - 1095965956, - 3220917680, - 1059537466, - 1092026631, - 3204377509, - 1082081741 - ], - [ - 1109696778, - 3227688709, - 1067632789, - 1090802537, - 3232499768, - 1066695476 - ], - [ - 1094069973, - 3220663385, - 1062896537, - 1082132601, - 3227847815, - 1056623181 - ], - [ - 1091122114, - 3233397454, - 1067147471, - 1093178008, - 3198465317, - 1075607136 - ], - [ - 1087362203, - 3229857075, - 1071624153, - 1092923996, - 3206317416, - 1085644325 - ], - [ - 1092147110, - 3231545350, - 1056891069, - 1102249851, - 3225000463, - 1081139825 - ], - [ - 1090121641, - 3234484566, - 1077358416, - 1092093500, - 3213048369, - 1074325224 - ], - [ - 1102799272, - 3228227953, - 1070574579, - 1066983162, - 3237864933, - 1068528181 - ], - [ - 1101197545, - 3218564125, - 1054507244, - 1099003069, - 3226730646, - 1054287728 - ], - [ - 1102533419, - 3200999848, - 1078087141, - 1111673754, - 3207298617, - 1066715850 - ], - [ - 1103594482, - 3223608501, - 1071839307, - 1100338323, - 3231420875, - 1061301999 - ], - [ - 1098733658, - 3219748011, - 1075940391, - 1080733479, - 3224783033, - 1076247097 - ], - [ - 1100175538, - 3197571771, - 1070895204, - 1060208334, - 3204316014, - 1071088301 - ], - [ - 1098724291, - 3241304285, - 1065702821, - 1102931453, - 3214540210, - 1069408101 - ], - [ - 1074442598, - 3219385772, - 1069971375, - 1091370757, - 3196416818, - 1077555269 - ], - [ - 1104207621, - 3209162621, - 1064412763, - 1103443522, - 3221609438, - 1065778146 - ], - [ - 1084119375, - 3220787332, - 1078980771, - 1085917796, - 3234661695, - 1073921738 - ], - [ - 1099535036, - 3223553783, - 1078727759, - 1109572143, - 3222361102, - 1050917346 - ], - [ - 1101710351, - 3204372895, - 1071982549, - 1087485268, - 3226884973, - 1069116109 - ], - [ - 1083389840, - 3231368287, - 1073984200, - 1094378077, - 3231248893, - 1088673189 - ], - [ - 1081813615, - 3231199737, - 1082621558, - 1101988617, - 3201910364, - 1073923921 - ], - [ - 1050718641, - 3217380471, - 1081169541, - 1092000154, - 3226295422, - 1078866565 - ], - [ - 1071327594, - 3226194873, - 1078368123, - 1091173537, - 3233466906, - 1061462008 - ], - [ - 1056963866, - 3228487365, - 1075887095, - 1103063368, - 3195025532, - 1069556725 - ], - [ - 1093686285, - 3224872461, - 1055549500, - 1095935950, - 3234945715, - 1077798275 - ], - [ - 1102776106, - 3212448011, - 1052536823, - 1056914874, - 3223940542, - 1071691865 - ], - [ - 1084743007, - 3239049023, - 1084828682, - 1057036172, - 3230299988, - 1064228271 - ], - [ - 1097951684, - 3230462276, - 1062077472, - 1078680336, - 3235476125, - 1084185276 - ], - [ - 1089444278, - 3214206458, - 1049463042, - 1095008229, - 3230614098, - 1069164035 - ], - [ - 1081080851, - 3213023249, - 1077242224, - 1096567797, - 3212736134, - 1082223926 - ], - [ - 1101599669, - 3217767092, - 1077661973, - 1100792468, - 3229118837, - 1075846566 - ], - [ - 1097737264, - 3196210901, - 1073681669, - 1094552596, - 3235584059, - 1075338069 - ], - [ - 1104060731, - 3235350934, - 1081095913, - 1074049259, - 3200851169, - 1067813893 - ], - [ - 1099459449, - 3230124123, - 1083322738, - 1094500920, - 3216934447, - 1083929457 - ], - [ - 1103912184, - 3241101344, - 1072786649, - 1085341228, - 3187538501, - 1073886217 - ], - [ - 1105734887, - 3199534446, - 1079675321, - 1100584080, - 3220169913, - 1073928608 - ], - [ - 1082883776, - 3231729096, - 1053860417, - 1093142550, - 3231042042, - 1054462905 - ], - [ - 1069885607, - 3226928354, - 1056898271, - 1101254193, - 3230099886, - 1082960187 - ], - [ - 1079059790, - 3212251523, - 1073796525, - 1104611698, - 3211460357, - 1073132313 - ], - [ - 1109721135, - 3211828680, - 1071462984, - 1110206998, - 3235894852, - 1076565248 - ], - [ - 1090228791, - 3230833633, - 1015357520, - 1090079910, - 3232912114, - 1054839605 - ], - [ - 1093951026, - 3238281056, - 1084792879, - 1093682895, - 3205420366, - 1056565233 - ], - [ - 1051608263, - 3231333579, - 1071616121, - 1107965245, - 3229719259, - 1028319307 - ], - [ - 1094708383, - 3220781572, - 1069903140, - 1096718194, - 3227019269, - 1056835885 - ], - [ - 1100729245, - 3235200856, - 1075612675, - 1100902495, - 3222946945, - 1074907237 - ], - [ - 1095704455, - 3232748962, - 1069309483, - 1056856051, - 3236330957, - 1041536537 - ], - [ - 1056925838, - 3239427686, - 1078848388, - 1088659399, - 3231395378, - 1083970828 - ], - [ - 1095016935, - 3189499995, - 1081523592, - 1090917471, - 3227559870, - 1064838484 - ], - [ - 1103624781, - 3236170807, - 1070582388, - 1115456372, - 3213902237, - 1055592176 - ] + [1091043029,3236413372,1072960732,1091846284,3228822993,1049260509], + [1089604410,3209765174,1066995900,1096892626,3230859161,1083373091], + [1089028828,3217473928,1083471770,1081433203,3211087247,1085197451], + [1092670177,3231841514,1063598103,1103584540,3213008072,1079116996], + [1090035815,3218609985,1072967049,1088661766,3198072896,1044746985], + [1046480449,3229984088,1043699741,1091873413,3199047408,1066277130], + [1065930110,3223301471,1068698187,1101831271,3220211818,1078129601], + [1102355112,3233401852,1061806677,1094438136,3230801565,1056751296], + [1071781071,3223814534,1074126143,1087694622,3207608420,1072523139], + [1100286232,3220852743,1078104567,1091432475,3228459147,1075102743], + [1093740897,3236214117,1056326492,1092346849,3221398009,1086728797], + [1100593070,3206078492,1056922408,1091962285,3221334560,1075874083], + [1057422500,3204308046,1055447604,1086751416,3193875437,1057501718], + [1102133900,3236833396,1079289965,1096423706,3211031308,1055779424], + [1102942431,3208201125,1061303639,1082397171,3215086253,1087148488], + [1099507992,3232003671,1051027024,1094935519,3233065923,1055457829], + [1089783516,3204365081,1078144728,1100944492,3224171091,1074309607], + [1077401221,3224265922,1072518675,1101852937,3228400449,1085998811], + [1083732000,3229825614,1056330999,1099937113,3230554319,1060965739], + [1051341681,3204304237,1067216203,1099216030,3210459266,1053140025], + [1107586516,3207674385,1066156629,1094575525,3215323668,1079011965], + [1069952171,3210407666,1075530577,1100534958,3232852127,1075204003], + [1099015047,3244274170,1059425851,1095226244,3229746387,1082150661], + [1076727725,3224947536,1066836675,1075619170,3220313669,1051532035], + [1078798162,3223751731,1055999341,1064175839,3223857090,1082095571], + [1093816228,3225594692,1082125304,1068063030,3226723980,1062858025], + [1103035975,3207810284,1052807795,1091155427,3218244981,1072712297], + [1079617615,3228500725,1053610061,1080080568,3223826525,1066614846], + [1108348671,3226723032,1074538891,1091318071,3228468606,1056778564], + [1093513399,3238115488,1079684449,1084509311,3221098552,1055546581], + [1060217161,3215430168,1066270533,1081615201,3200891155,1068640992], + [1094282331,3230528647,1043740607,1080552362,3232023744,1075864891], + [1094827784,3226747240,1066322787,1086753029,3223281656,1056962835], + [1098674178,3224972798,1064467871,1070679040,3201856793,1068261005], + [1098465593,3203796861,1074175242,1090019441,3213887196,1080081668], + [1094305959,3232677516,1075737660,1096738054,3225662719,1077618269], + [1101782342,3221762042,1067527878,1101894470,3221806563,1075476347], + [1092200435,3199955949,1057000008,1100181632,3210640283,1089607613], + [1091081282,3198111811,1078294936,1098582062,3204307983,1079911147], + [1098208036,3232400609,1080540881,1094282131,3225614473,1077628547], + [1099802322,3223162142,1083027269,1094647742,3225176036,1083150021], + [1111230486,3232431131,1038786741,1054715600,3207509192,1080923187], + [1098281059,3203566345,1051946752,1096149533,3201774998,1057171943], + [1097568201,3199360717,1078218979,1095091979,3238033375,1075879743], + [1052479147,3191630079,1084367540,1086413968,3230553481,1074501560], + [1075167289,3196807132,1055156632,1089236973,3223240571,1086189751], + [1093088550,3211434454,1074061134,1098370506,3220538954,1078670917], + [1103509241,3204269792,1076097309,1098588569,3200419693,1081669124], + [1102912576,3204328820,1074401080,1101200549,3221733624,1074125197], + [1097381957,3240993366,1056752488,1066573639,3224589810,1053479095], + [1091593928,3221848363,1074958575,1109539036,3224699448,1068515820], + [1102085329,3231955557,1064572791,1095240036,3214942400,1076182333], + [1097282340,3232754826,1081825847,1074320528,3211829949,1081317256], + [1093235834,3237475214,1069358485,1089575574,3221515398,1073593049], + [1095585441,3218639642,1079942549,1088451430,3205924932,1056898492], + [1101813346,3209253481,1060102748,1102359203,3232658413,1078667933], + [1090596195,3239302235,1082622658,1099223942,3235875623,1074915604], + [1082229665,3233603221,1070119424,1091819602,3226779304,1073850881], + [1084990589,3201629230,1072981551,1096929139,3237497239,1058112477], + [1088738155,3238099732,1041776969,1099802498,3234623119,1064436416], + [1098578764,3227261602,1074532649,1101578575,3211964151,1055196331], + [1100857344,3214577377,1056814328,1100075159,3217630961,1051774860], + [1092723953,3199520085,1067079717,1111462727,3221900379,1075102994], + [1095329111,3233031670,1067114571,1092843219,3226435042,1085803409], + [1100938920,3226514717,1077465688,1105207197,3232817693,1088959693], + [1093454282,3223619523,1073714443,1099622143,3235385805,1082416661], + [1098180936,3199385064,1082897221,1077930197,3217678318,1069677373], + [1077101821,3221266930,1074170224,1108538030,3219369158,1053097613], + [1097268499,3213753194,1075531812,1069862380,3224151522,1056396592], + [1104340617,3215732934,1092296264,1092956104,3226122821,1073841356], + [1088439372,3221648463,1056942244,1088217732,3235870930,1048059745], + [1103701642,3236507915,1075005201,1096683966,3232603705,1056001687], + [1101568192,3218563316,1073738156,1073343116,3227090505,1085679048], + [1085592714,3211081045,1077967561,1104783859,3206588190,1076358869], + [1101791137,3225744941,1061464897,1101974731,3209708481,1076226849], + [1099656327,3223156863,1041381609,1103683871,3223102136,1082778722], + [1102536966,3214970135,1055379917,1100285405,3201382906,1084716608], + [1107620323,3217150008,1063998667,1091982485,3236590348,1065155017], + [1090534540,3207633065,1056230637,1069689274,3222363161,1076296303], + [1099723466,3233233367,1069207772,1076690682,3233955385,1081954961], + [1076220201,3217545521,1088650328,1074875922,3239603715,1076543947], + [1106655939,3228162236,1068725220,1082332505,3218778715,1075067923], + [1098001065,3222231517,1071902060,1109219790,3208666549,1063807552], + [1099339460,3225643643,1074343875,1104233109,3232000594,1072134376], + [1082957244,3230078757,1065302661,1098690139,3235629725,1056541784], + [1084506532,3210723662,1083985875,1076170306,3232736965,1047285257], + [1103224973,3225216826,1056233613,1101493368,3213743097,1074966549], + [1080161792,3234471232,1068413091,1080940358,3217816440,1062284975], + [1093350013,3239500404,1076687188,1080611896,3233058191,1082820091], + [1063231241,3223776879,1078180184,1067590286,3231178451,1059646026], + [1100944148,3223515100,1074450045,1059280169,3207904617,1070019217], + [1102790543,3229725155,1080281611,1100828023,3231678084,1076078422], + [1094455022,3235179122,1066222523,1083075957,3215094194,1065558119], + [1087032571,3221518535,1074550266,1092376557,3225002817,1077281519], + [1071104213,3228962128,1059898027,1105210923,3229658854,1078765091], + [1092516390,3220440852,1062117239,1102596245,3214201943,1085658749], + [1086026571,3216439392,1073346887,1099422854,3204379426,1056920191], + [1092034736,3216958217,1054127876,1067092274,3223510484,1085434555], + [1100001486,3232576427,1055704775,1105713596,3196424194,1075592519], + [1099106349,3196138306,1052055868,1089654846,3210038815,1081340919], + [1094188687,3222038690,1077144736,1097873288,3203033027,1069162796], + [1103272341,3231941231,1058033936,1096900057,3231616364,1069965723], + [1086958916,3218911365,1061248307,1095314395,3208473479,1066913363], + [1085552745,3216612362,1071184555,1093129624,3225473466,1043939384], + [1095391277,3230424650,1047785248,1084749054,3213436778,1056708733], + [1102461769,3214266904,1071553319,1096832883,3225264004,1081993437], + [1075129161,3219445619,1075434156,1083389995,3231866667,1082618842], + [1082978243,3226938936,1077217580,1064587046,3223441662,1086959047], + [1086009270,3231120470,1075132549,1101602500,3196355265,1069201171], + [1079471754,3237717403,1079132087,1093626524,3227903069,1055054797], + [1083878117,3190897726,1075647329,1092161689,3229543087,1060084615], + [1095914137,3196172703,1072892051,1097766331,3216843463,1078271721], + [1086048239,3228664541,1081137849,1099327217,3204671138,1084334845], + [1082157913,3202736730,1059499911,1090574705,3234429547,1073372255], + [1103426369,3233134738,1076870752,1083197243,3231723761,1082531445], + [1101006645,3204825680,1089314095,1094927862,3214817556,1080369144], + [1088324073,3231291731,1059952347,1092478345,3221378065,1076542384], + [1091657854,3233440452,1065562133,1090659470,3236853590,1061309065], + [1084763850,3240431199,1076389685,1090979688,3234833666,1070589571], + [1086294573,3198298188,1050874597,1099621345,3220551889,1085247484], + [1075936837,3238425004,1083603200,1081255557,3198131371,1086077443], + [1090532631,3235916948,1074506238,1090347627,3204774072,1056230548], + [1088845823,3204080906,1071148037,1101534119,3214563346,1065501264], + [1092880736,3222880087,1085061772,1078910592,3225036010,1078832307], + [1094053042,3223284226,1056467187,1107139683,3233157689,1073246904], + [1099119729,3221679601,1082122889,1092354926,3232734778,1082476759], + [1082190516,3207725434,1055054211,1102715293,3223663795,1080833288], + [1092340778,3204448135,1069923735,1092396616,3221934601,1074520580], + [1100412892,3218118037,1074005991,1090548881,3216389450,1063341709], + [1091908100,3239974398,1079964551,1098857271,3230107148,1055400040], + [1094922938,3212009411,1077532665,1101988450,3226677962,1083881541], + [1077582004,3165056772,1066782681,1084429016,3218703736,1069983947], + [1080588875,3217714886,1079084139,1086106024,3225819434,1074185650], + [1108386246,3220494206,1080833364,1099692636,3230042836,1082999383], + [1051991608,3221775905,1074722067,1103463883,3231246988,1076398810], + [1105228080,3225485685,1068692477,1102024394,3227743438,1075652555], + [1088172344,3226426203,1075095978,1086535515,3221557375,1063575888], + [1097131892,3234853388,1046393491,1091967309,3212445299,1079104597], + [1099683040,3211869746,1078476253,1100800485,3226551431,1056186145], + [1100939044,3214617936,1053102900,1077479108,3235543599,1083822752], + [1087906897,3225098356,1074487657,1090617785,3196509175,1056222801], + [1082527501,3222280945,1035606372,1082819364,3232858649,1085833187], + [1087301817,3222812008,1046988716,1052235375,3208752935,1061371855], + [1103898560,3227505172,1059609851,1066168763,3215842988,1080765109], + [1099470499,3204392713,1038688956,1101443719,3208267177,1067358597], + [1092555211,3236120663,1058341335,1100768922,3231124168,1076268861], + [1067229940,3218375224,1056951316,1085303063,3201025831,1070459243], + [1067941321,3228106924,1078101952,1108885649,3221176472,1034668964], + [1094147016,3217393975,1061589831,1087825299,3232719370,1077341804], + [1092706927,3190613654,1082069328,1085434171,3225147055,1083956455], + [1092220192,3230815729,1068659965,1089330212,3218521874,1072508879], + [1101311477,3225947118,1085438048,1084546624,3229885141,1073374692], + [1084469556,3221851437,1077301883,1101091944,3213645893,1068992292], + [1078751918,3217605226,1057867021,1102696868,3230252236,1062522457], + [1076826349,3227919673,1083901239,1091375470,3237757563,1069947433], + [1082649326,3224248441,1067344119,1075172943,3237612959,1058645272], + [1099394929,3227395350,1072411113,1077623027,3240281497,1067134501], + [1089968201,3234271370,1065331984,1094736919,3215788807,1054156869], + [1101027376,3189117113,1070589257,1077784387,3220992744,1080806640], + [1099791189,3203700268,1056641233,1078899671,3224110234,1078608268], + [1093067949,3210255723,1074401408,1087293702,3231360362,1074683022], + [1073194194,3217372611,1076535389,1104289240,3230370845,1076896624], + [1092776651,3232002747,1064155765,1091086118,3202175352,1065358923], + [1100500779,3205115912,1051305092,1084644293,3212171960,1078678393], + [1095651269,3231694699,1079241609,1084155880,3204438510,1070389979], + [1101809136,3213702714,1067973851,1098035846,3239035357,1082541905], + [1100208541,3236802613,1055338157,1099763835,3229014298,1069218003], + [1100747067,3232840562,1066473038,1074561367,3232678423,1072822624], + [1095523471,3213923549,1078258004,1084791728,3228224221,1051790635], + [1106224753,3230181371,1082504077,1067362120,3216581984,1072666553], + [1046976464,3224089407,1074529411,1085655768,3237024489,1078341855], + [1099224884,3224551183,1056049125,1104513292,3229883557,1060539671], + [1100851044,3213461492,1074408407,1088121640,3242689837,1076121369], + [1091066982,3228337582,1063312036,1102745226,3198190444,1086350576], + [1100936585,3232211388,1075379715,1098966577,3196643957,1051361888], + [1104966512,3223237684,1068478608,1099235480,3207062257,1065784933], + [1104242177,3231796459,1074491203,1108189827,3238041728,1062176831], + [1110108044,3216166509,1065745259,1084148895,3223173793,1072494251], + [1109124868,3226840298,1082347708,1098368909,3237278907,1056160569], + [1092996202,3222705547,1087907203,1099906676,3224531708,1077463493], + [1101310449,3239982722,1082734402,1073389762,3204600194,1054768468], + [1076580604,3206518913,1062375635,1056855987,3232819499,1078633547], + [1077162421,3209056984,1083908184,1065393060,3216642055,1055700872], + [1085373818,3199832705,1083336558,1097764360,3222696841,1067155485], + [1091734265,3212668618,1065740503,1054141646,3189921283,1056958425], + [1094313576,3232878926,1067537255,1085012193,3205074944,1076035991], + [1086636829,3226522931,1069458383,1087308102,3216298376,1052484089], + [1092325023,3226870003,1079693811,1099323861,3233004503,1080339100], + [1099244334,3229239659,1054875797,1105690814,3229642921,1071577537], + [1108030887,3228105602,1082518437,1101613355,3219694546,1054416655], + [1077141201,3230853459,1060815437,1084143671,3237502280,1072158969], + [1085235483,3212850550,1079793709,1085555296,3231922878,1075910516], + [1098658458,3217851566,1075461829,1072956889,3223733464,1065047403], + [1099931975,3232680002,1069714405,1068531967,3223253090,1083254818], + [1084749663,3234494283,1067172175,1078292152,3227940883,1083273632], + [1107392016,3213199104,1076769456,1108788824,3221679379,1074426857], + [1102172924,3232090953,1083758192,1089817933,3233833476,1064326076], + [1108593514,3225926362,1078069753,1098432133,3186451118,1081403616], + [1101858982,3229746414,1073327588,1093610322,3200990837,1062599811], + [1102737060,3233985576,1087081244,1101718036,3201677679,1059730171], + [1102246363,3238756685,1080315359,1092671642,3218525247,1077464703], + [1076700275,3220437220,1084175413,1088632784,3209647715,1043312092], + [1082825858,3225129599,1076303310,1091481935,3239516413,1035358543], + [1092459827,3192713773,1065959056,1093147702,3211180987,1079121165], + [1070252127,3225932807,1063371152,1108006903,3230772495,1052773384], + [1101035063,3206902014,1085019219,1083535384,3227896070,1073813560], + [1078045941,3236479681,1074608317,1091729080,3213890531,1056113067], + [1095965956,3220917680,1059537466,1092026631,3204377509,1082081741], + [1109696778,3227688709,1067632789,1090802537,3232499768,1066695476], + [1094069973,3220663385,1062896537,1082132601,3227847815,1056623181], + [1091122114,3233397454,1067147471,1093178008,3198465317,1075607136], + [1087362203,3229857075,1071624153,1092923996,3206317416,1085644325], + [1092147110,3231545350,1056891069,1102249851,3225000463,1081139825], + [1090121641,3234484566,1077358416,1092093500,3213048369,1074325224], + [1102799272,3228227953,1070574579,1066983162,3237864933,1068528181], + [1101197545,3218564125,1054507244,1099003069,3226730646,1054287728], + [1102533419,3200999848,1078087141,1111673754,3207298617,1066715850], + [1103594482,3223608501,1071839307,1100338323,3231420875,1061301999], + [1098733658,3219748011,1075940391,1080733479,3224783033,1076247097], + [1100175538,3197571771,1070895204,1060208334,3204316014,1071088301], + [1098724291,3241304285,1065702821,1102931453,3214540210,1069408101], + [1074442598,3219385772,1069971375,1091370757,3196416818,1077555269], + [1104207621,3209162621,1064412763,1103443522,3221609438,1065778146], + [1084119375,3220787332,1078980771,1085917796,3234661695,1073921738], + [1099535036,3223553783,1078727759,1109572143,3222361102,1050917346], + [1101710351,3204372895,1071982549,1087485268,3226884973,1069116109], + [1083389840,3231368287,1073984200,1094378077,3231248893,1088673189], + [1081813615,3231199737,1082621558,1101988617,3201910364,1073923921], + [1050718641,3217380471,1081169541,1092000154,3226295422,1078866565], + [1071327594,3226194873,1078368123,1091173537,3233466906,1061462008], + [1056963866,3228487365,1075887095,1103063368,3195025532,1069556725], + [1093686285,3224872461,1055549500,1095935950,3234945715,1077798275], + [1102776106,3212448011,1052536823,1056914874,3223940542,1071691865], + [1084743007,3239049023,1084828682,1057036172,3230299988,1064228271], + [1097951684,3230462276,1062077472,1078680336,3235476125,1084185276], + [1089444278,3214206458,1049463042,1095008229,3230614098,1069164035], + [1081080851,3213023249,1077242224,1096567797,3212736134,1082223926], + [1101599669,3217767092,1077661973,1100792468,3229118837,1075846566], + [1097737264,3196210901,1073681669,1094552596,3235584059,1075338069], + [1104060731,3235350934,1081095913,1074049259,3200851169,1067813893], + [1099459449,3230124123,1083322738,1094500920,3216934447,1083929457], + [1103912184,3241101344,1072786649,1085341228,3187538501,1073886217], + [1105734887,3199534446,1079675321,1100584080,3220169913,1073928608], + [1082883776,3231729096,1053860417,1093142550,3231042042,1054462905], + [1069885607,3226928354,1056898271,1101254193,3230099886,1082960187], + [1079059790,3212251523,1073796525,1104611698,3211460357,1073132313], + [1109721135,3211828680,1071462984,1110206998,3235894852,1076565248], + [1090228791,3230833633,1015357520,1090079910,3232912114,1054839605], + [1093951026,3238281056,1084792879,1093682895,3205420366,1056565233], + [1051608263,3231333579,1071616121,1107965245,3229719259,1028319307], + [1094708383,3220781572,1069903140,1096718194,3227019269,1056835885], + [1100729245,3235200856,1075612675,1100902495,3222946945,1074907237], + [1095704455,3232748962,1069309483,1056856051,3236330957,1041536537], + [1056925838,3239427686,1078848388,1088659399,3231395378,1083970828], + [1095016935,3189499995,1081523592,1090917471,3227559870,1064838484], + [1103624781,3236170807,1070582388,1115456372,3213902237,1055592176] ] }, "inputs": { "X": { - "shape": [ - 7 - ], + "shape": [7], "dtype": "int32", - "bits": [ - 3, - -1, - 4, - -1, - 5, - -9, - 2 - ] + "bits": [3,-1,4,-1,5,-9,2] }, "F": { - "shape": [ - 2, - 3 - ], + "shape": [2,3], "dtype": "float32", - "bits": [ - 1069547520, - 3196059648, - 1077936128, - 1036831949, - 1088421888, - 3223322624 - ] + "bits": [1069547520,3196059648,1077936128,1036831949,1088421888,3223322624] } } -} \ No newline at end of file +} diff --git a/crates/tract-harvest-probe/fixtures/tiny/shape_fold.onnx b/crates/tract-harvest-probe/fixtures/tiny/shape_fold.onnx new file mode 100644 index 000000000..fe5d25559 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/shape_fold.onnx differ diff --git a/crates/tract-harvest-probe/fixtures/tiny/shape_sym.onnx b/crates/tract-harvest-probe/fixtures/tiny/shape_sym.onnx new file mode 100644 index 000000000..73bcf3131 Binary files /dev/null and b/crates/tract-harvest-probe/fixtures/tiny/shape_sym.onnx differ diff --git a/crates/tract-harvest-probe/src/dr3.rs b/crates/tract-harvest-probe/src/dr3.rs index c5450b920..74b2955f0 100644 --- a/crates/tract-harvest-probe/src/dr3.rs +++ b/crates/tract-harvest-probe/src/dr3.rs @@ -228,6 +228,10 @@ fn dtype_of(dt: DatumType) -> DType { DatumType::F16 => DType::F16, DatumType::I32 => DType::I32, DatumType::I64 => DType::I64, + // Tract's ONNX `Shape` (onnx/src/ops/array/shape.rs) and `Size` emit + // `TDim`, and ops over them keep it; the ONNX type of those values + // is int64 (the spec's output type of `Shape`/`Size`). + DatumType::TDim => DType::I64, DatumType::Bool => DType::Bool, _ => DType::Other(-1), } @@ -240,9 +244,7 @@ pub fn tract_facts( bytes: &[u8], pins: &[(usize, DatumType, Vec)], ) -> Result, Refusal> { - let onnx = tract_onnx::onnx(); - let proto = onnx.proto_model_for_read(&mut &*bytes).map_err(fe)?; - facts_of(onnx.model_for_proto_model(&proto).map_err(fe)?, pins) + Ok(tract_fact_sources(bytes, pins)?.merged()) } /// [`tract_facts`] for a model with external data next to it: the weights @@ -251,19 +253,26 @@ pub fn tract_facts_path( path: &std::path::Path, pins: &[(usize, DatumType, Vec)], ) -> Result, Refusal> { - let onnx = tract_onnx::onnx().with_lazy_external_data(true); - facts_of(onnx.model_for_path(path).map_err(fe)?, pins) + Ok(tract_fact_sources_path(path, pins)?.merged()) } fn fe(e: TractError) -> Refusal { Refusal::Frontend(format!("{e:#}")) } -fn facts_of( - m: InferenceModel, - pins: &[(usize, DatumType, Vec)], -) -> Result, Refusal> { - Ok(fact_sources_of(m, pins)?.merged()) +/// A Tract outlet: node name and output slot. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub struct Outlet { + pub node: String, + pub slot: usize, +} + +/// A Tract fact not admitted for a source value, and why. +#[derive(Debug, Clone, PartialEq)] +pub struct Rejected { + pub stage: &'static str, + pub value: String, + pub reason: String, } /// Facts from both Tract stages, kept apart so they can be compared. @@ -274,6 +283,9 @@ fn facts_of( /// dimension is not a symbol: two of them need not be equal). /// - `typed`: the typed model, complete per outlet but labelling only the /// values that survive typing (2,546 of 7,620 on the Jina graph). +/// +/// A Tract fact is admitted for a source value only through [`admit`]: a +/// matching label alone is not authority. Rejections are kept in `rejected`. #[derive(Debug, Default)] pub struct FactSources { pub inference: HashMap, @@ -282,10 +294,13 @@ pub struct FactSources { /// their own (Tract refuses to type a graph with an undetermined input /// dimension, or more than one symbol through `Flatten`). pub typed_error: Option, + pub rejected: Vec, } impl FactSources { /// Inference facts first; the typed fact fills a missing dtype or shape. + /// Both maps hold only admitted facts, so this never merges two tensors + /// that merely share a label. pub fn merged(&self) -> HashMap { let mut out = self.inference.clone(); for (k, t) in &self.typed { @@ -317,14 +332,120 @@ impl FactSources { } } +/// The Tract outlet each source value must come from, by Tract's own naming +/// rule (`onnx/src/model.rs`): graph inputs and initializers are nodes named +/// after the value; a node takes its source name, else its first output's +/// name. `None` when that identity is not unique (two source nodes with one +/// name) or not reproducible (an unnamed node whose first output is empty, +/// which Tract names by a running counter). +pub fn source_producers(src: &SourceInventory) -> HashMap> { + let mut by_name: HashMap = HashMap::new(); + let node_name = |n: &crate::source::SourceNode| { + if !n.name.is_empty() { + Some(n.name.clone()) + } else { + n.outputs.first().filter(|o| !o.is_empty()).cloned() + } + }; + for n in &src.nodes { + if let Some(name) = node_name(n) { + *by_name.entry(name).or_default() += 1; + } + } + for v in src + .inputs + .iter() + .chain(src.initializers.iter().map(|i| &i.name)) + { + *by_name.entry(v.clone()).or_default() += 1; + } + let mut out: HashMap> = HashMap::new(); + let mut put = |v: &str, o: Option| { + let e = out.entry(v.to_string()).or_insert(o.clone()); + if *e != o { + *e = None; // the value has two producers in the source + } + }; + for v in src + .inputs + .iter() + .chain(src.initializers.iter().map(|i| &i.name)) + { + put( + v, + (by_name[v] == 1).then(|| Outlet { + node: v.clone(), + slot: 0, + }), + ); + } + for n in &src.nodes { + let name = node_name(n); + let unique = name.as_ref().is_some_and(|m| by_name[m] == 1); + for (slot, o) in n.outputs.iter().filter(|o| !o.is_empty()).enumerate() { + put( + o, + name.clone() + .filter(|_| unique) + .map(|node| Outlet { node, slot }), + ); + } + } + out +} + +/// Admit one stage's facts: a fact for source value `v` is kept only if +/// exactly one outlet of the stage carries `v`'s label and that outlet is +/// `v`'s producer (see [`source_producers`]). Labels that name no source +/// value are Tract-internal and ignored. +pub fn admit( + stage: &'static str, + raw: Vec<(String, Outlet, Fact)>, + producers: &HashMap>, +) -> (HashMap, Vec) { + let mut by_label: HashMap> = HashMap::new(); + for (l, o, f) in raw { + by_label.entry(l).or_default().push((o, f)); + } + let (mut ok, mut rej) = (HashMap::new(), Vec::new()); + let mut labels: Vec<_> = by_label.into_iter().collect(); + labels.sort_by(|a, b| a.0.cmp(&b.0)); + for (value, mut outs) in labels { + let Some(want) = producers.get(&value) else { + continue; + }; + let reason = match (outs.len(), want) { + (n, _) if n > 1 => Some(format!("label carried by {n} outlets")), + (_, None) => Some("source producer not unique".to_string()), + (_, Some(w)) if outs[0].0 != *w => Some(format!( + "Tract outlet {:?} is not the source producer {:?}", + outs[0].0, w + )), + _ => None, + }; + match reason { + Some(reason) => rej.push(Rejected { + stage, + value, + reason, + }), + None => { + ok.insert(value, outs.pop().expect("one").1); + } + } + } + (ok, rej) +} + /// Both fact sources for an in-memory model. pub fn tract_fact_sources( bytes: &[u8], pins: &[(usize, DatumType, Vec)], ) -> Result { + let src = SourceInventory::from_bytes(bytes).map_err(|e| Refusal::Frontend(e.to_string()))?; let onnx = tract_onnx::onnx(); let proto = onnx.proto_model_for_read(&mut &*bytes).map_err(fe)?; - fact_sources_of(onnx.model_for_proto_model(&proto).map_err(fe)?, pins) + fact_sources_of(&src, onnx.model_for_proto_model(&proto).map_err(fe)?, pins) } /// Both fact sources for a model with lazy external data. @@ -332,18 +453,24 @@ pub fn tract_fact_sources_path( path: &std::path::Path, pins: &[(usize, DatumType, Vec)], ) -> Result { + let bytes = std::fs::read(path).map_err(|e| Refusal::Frontend(e.to_string()))?; + let src = SourceInventory::from_bytes(&bytes).map_err(|e| Refusal::Frontend(e.to_string()))?; let onnx = tract_onnx::onnx().with_lazy_external_data(true); - fact_sources_of(onnx.model_for_path(path).map_err(fe)?, pins) + fact_sources_of(&src, onnx.model_for_path(path).map_err(fe)?, pins) } -fn label(m: &Graph, o: OutletId) -> String +fn outlet_of(m: &Graph, o: OutletId) -> (Option, Outlet) where F: tract_onnx::prelude::Fact + Clone + 'static, O: std::fmt::Debug + std::fmt::Display + AsRef + AsMut + Clone + 'static, { - m.outlet_label(o) - .map(str::to_string) - .unwrap_or_else(|| m.node(o.node).name.clone()) + ( + m.outlet_label(o).map(str::to_string), + Outlet { + node: m.node(o.node).name.clone(), + slot: o.slot, + }, + ) } fn dim_of(d: &TDim) -> Dim { @@ -352,7 +479,23 @@ fn dim_of(d: &TDim) -> Dim { .unwrap_or_else(|| Dim::Symbol(d.to_string())) } +/// Tract panics on some invalid graphs: a constant `Gather` index outside the +/// gathered dimension indexes out of bounds in `tract-core` +/// (`ops/array/gather.rs`) during `analyse`. A panic of the cold frontend is a +/// frontend refusal, never a crash of resolution. +fn no_panic(stage: &str, f: impl FnOnce() -> T) -> Result { + std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)).map_err(|p| { + let why = p + .downcast_ref::() + .map(String::as_str) + .or_else(|| p.downcast_ref::<&str>().copied()) + .unwrap_or("non-string panic"); + Refusal::Frontend(format!("Tract panicked during {stage}: {why}")) + }) +} + fn fact_sources_of( + src: &SourceInventory, mut m: InferenceModel, pins: &[(usize, DatumType, Vec)], ) -> Result { @@ -360,12 +503,16 @@ fn fact_sources_of( m.set_input_fact(*i, InferenceFact::dt_shape(*dt, shape.clone())) .map_err(fe)?; } + let producers = source_producers(src); let mut out = FactSources::default(); // Inference stage: analyse without stopping at the first node it cannot - // type, then read whatever each labelled outlet determined. + // type, then read whatever each labelled outlet determined. An unlabelled + // outlet is identified by its node's name (Tract labels a single-output + // node's outlet by the node only). let mut inf = m.clone(); - inf.analyse(true).map_err(fe)?; + no_panic("analyse", || inf.analyse(true))?.map_err(fe)?; + let mut raw = Vec::new(); for node in inf.nodes() { for slot in 0..node.outputs.len() { let o = OutletId::new(node.id, slot); @@ -383,32 +530,47 @@ fn fact_sources_of( shape, }; if fact.dtype.is_some() || fact.shape.is_some() { - out.inference.insert(label(&inf, o), fact); + let (label, outlet) = outlet_of(&inf, o); + raw.push((label.unwrap_or_else(|| outlet.node.clone()), outlet, fact)); } } } + let (ok, rej) = admit("inference", raw, &producers); + out.inference = ok; + out.rejected.extend(rej); // Typed stage. - let m = match m.into_typed() { - Ok(m) => m, - Err(e) => { + let m = match no_panic("typing", || m.into_typed()) { + Ok(Ok(m)) => m, + Ok(Err(e)) => { out.typed_error = Some(format!("{e:#}")); return Ok(out); } + Err(Refusal::Frontend(e)) => { + out.typed_error = Some(e); + return Ok(out); + } + Err(e) => return Err(e), }; + let mut raw = Vec::new(); for node in m.nodes() { for slot in 0..node.outputs.len() { let o = OutletId::new(node.id, slot); let f = m.outlet_fact(o).map_err(fe)?; - out.typed.insert( - label(&m, o), + let (label, outlet) = outlet_of(&m, o); + raw.push(( + label.unwrap_or_else(|| outlet.node.clone()), + outlet, Fact { dtype: Some(dtype_of(f.datum_type)), shape: Some(f.shape.iter().map(dim_of).collect()), }, - ); + )); } } + let (ok, rej) = admit("typed", raw, &producers); + out.typed = ok; + out.rejected.extend(rej); Ok(out) } @@ -423,6 +585,87 @@ mod tests { } } + fn outlet(node: &str, slot: usize) -> Outlet { + Outlet { + node: node.into(), + slot, + } + } + + /// A Tract fact is admitted for a source value only from the value's + /// producer outlet, and only if one outlet carries the label: a matching + /// label alone is not authority. Labels that name no source value are + /// ignored. + #[test] + fn a_label_alone_does_not_admit_a_fact() { + let producers: HashMap> = [ + ("a", Some(outlet("n_a", 0))), + ("b", Some(outlet("n_b", 1))), + ("c", Some(outlet("n_c", 0))), + ("d", None), + ] + .into_iter() + .map(|(k, v)| (k.to_string(), v)) + .collect(); + let x = fact(DType::F32, &[2]); + let raw = vec![ + ("a".to_string(), outlet("n_a", 0), x.clone()), + ("b".to_string(), outlet("n_b", 0), x.clone()), + ("c".to_string(), outlet("n_c", 0), x.clone()), + ("c".to_string(), outlet("n_c.1", 0), fact(DType::I64, &[3])), + ("d".to_string(), outlet("n_d", 0), x.clone()), + ("tract_internal".to_string(), outlet("x", 0), x.clone()), + ]; + let (ok, rej) = admit("t", raw, &producers); + assert_eq!(ok.keys().collect::>(), [&"a".to_string()]); + let why: Vec<(&str, &str)> = rej + .iter() + .map(|r| (r.value.as_str(), r.reason.as_str())) + .collect(); + assert_eq!(why.len(), 3, "{why:?}"); + assert!(why[0].0 == "b" && why[0].1.contains("is not the source producer")); + assert_eq!(why[1], ("c", "label carried by 2 outlets")); + assert_eq!(why[2], ("d", "source producer not unique")); + } + + /// Tract's naming: a named node, else its first output; a duplicated + /// name or an unnamed node without a first output has no producer. + #[test] + fn source_producers_follow_tract_naming() { + use crate::source::SourceNode; + let n = |name: &str, outs: &[&str]| SourceNode { + name: name.into(), + op_type: "Abs".into(), + domain: String::new(), + inputs: vec!["x".into()], + outputs: outs.iter().map(|s| s.to_string()).collect(), + attributes: vec![], + attrs: vec![], + }; + let src = SourceInventory { + nodes: vec![ + n("p", &["v1", "", "v2"]), + n("", &["v3"]), + n("dup", &["v4"]), + n("dup", &["v5"]), + n("", &["", "v6"]), + ], + inputs: vec!["x".into()], + model_sha256: String::new(), + ir_version: 8, + opsets: vec![], + initializers: vec![], + outputs: vec![], + }; + let p = source_producers(&src); + assert_eq!(p["x"], Some(outlet("x", 0))); + assert_eq!(p["v1"], Some(outlet("p", 0))); + assert_eq!(p["v2"], Some(outlet("p", 1)), "empty outputs take no slot"); + assert_eq!(p["v3"], Some(outlet("v3", 0))); + assert_eq!((p["v4"].clone(), p["v5"].clone()), (None, None)); + assert_eq!(p["v6"], None); + } + /// `disagreements` reports a dtype or shape conflict and stays silent on /// agreement or on a value only one stage describes; `merged` prefers the /// inference fact and fills only what it lacks from the typed one. diff --git a/crates/tract-harvest-probe/src/fold.rs b/crates/tract-harvest-probe/src/fold.rs new file mode 100644 index 000000000..8228361a2 --- /dev/null +++ b/crates/tract-harvest-probe/src/fold.rs @@ -0,0 +1,263 @@ +//! Cold shape-subgraph folding (D-TRACT-SHAPE-1). +//! +//! An ONNX node is folded, i.e. executed once while the plan is built and its +//! outputs kept as constants, only when its result is **invariant under the +//! sealed source and the resolved input facts**: +//! +//! - every input is a constant value of the source (initializer, `Constant`, +//! or an earlier fold), or +//! - it is `Shape` and its input's shape is fully known in the facts (no +//! symbolic, no undetermined dimension). The plan refuses at run time any +//! input whose dtype, rank or known dimensions differ from the facts +//! (`plan::check_against_fact`), so a dimension folded here is the one the +//! run sees. +//! +//! Everything else stays dynamic. A fold runs the node's registered DO (the +//! same immutable method the run would call), after its WHEN guard is TRUE. + +use std::collections::{BTreeMap, HashMap, HashSet}; + +use crate::registry::{Dim, Fact, NodeView, Registry, K3}; +use crate::source::SourceInventory; +use crate::value::Value; + +/// What a value depends on. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum Dep { + /// Source constants only. + Const, + /// Constants and fully known input shapes. + Shape, + /// An input shape with a symbolic dimension. + Symbolic, + /// An input value, or a shape the facts do not determine. + Data, +} + +fn shape_dep(f: Option<&Fact>) -> Dep { + match f.and_then(|f| f.shape.as_ref()) { + Some(s) if s.iter().all(|d| matches!(d, Dim::Known(_))) => Dep::Shape, + Some(_) => Dep::Symbolic, + None => Dep::Data, + } +} + +/// The dependency class of every value of the graph (by value name), and of +/// every node (the strongest class among its inputs; a `Shape` node takes +/// the class of its input's *shape*). +pub fn classify( + src: &SourceInventory, + facts: &HashMap, +) -> (HashMap, Vec) { + let mut dep: HashMap = HashMap::new(); + for i in &src.initializers { + dep.insert(i.name.clone(), Dep::Const); + } + for i in &src.inputs { + dep.insert(i.clone(), Dep::Data); + } + let mut node_dep = vec![None; src.nodes.len()]; + // Fixed point: the source order need not be topological. + loop { + let mut changed = false; + for (n, node) in src.nodes.iter().enumerate() { + if node_dep[n].is_some() { + continue; + } + let ins: Vec<&String> = node.inputs.iter().filter(|s| !s.is_empty()).collect(); + let d = if node.op_type == "Shape" && node.default_domain() { + match ins.first() { + Some(i) => Some(shape_dep(facts.get(i.as_str()))), + None => Some(Dep::Data), + } + } else { + ins.iter() + .map(|i| dep.get(i.as_str()).copied()) + .try_fold(Dep::Const, |a, d| d.map(|d| a.max(d))) + }; + if let Some(d) = d { + node_dep[n] = Some(d); + for o in node.outputs.iter().filter(|o| !o.is_empty()) { + dep.insert(o.clone(), d); + } + changed = true; + } + } + if !changed { + break; + } + } + // A node whose inputs never resolve (a cycle, or a dangling name) is Data. + ( + dep, + node_dep + .into_iter() + .map(|d| d.unwrap_or(Dep::Data)) + .collect(), + ) +} + +/// Per op type, how many nodes fall in each class `[Const, Shape, Symbolic, Data]`. +pub fn census( + src: &SourceInventory, + node_dep: &[Dep], + ops: &[&str], +) -> BTreeMap { + let mut out = BTreeMap::new(); + for (node, d) in src.nodes.iter().zip(node_dep) { + if ops.contains(&node.op_type.as_str()) { + out.entry(node.op_type.clone()).or_insert([0; 4])[*d as usize] += 1; + } + } + out +} + +/// The result of folding a graph. +#[derive(Debug, Default)] +pub struct Folded { + /// Folded node indices. + pub nodes: HashSet, + /// Values produced by folded nodes. + pub values: HashMap, + /// Foldable by dependency (Const or Shape) but not folded, with the reason. + pub blocked: Vec<(usize, String)>, +} + +impl Folded { + /// Folded values still read by a node that is not folded (or that are + /// graph outputs): the constants the plan must retain. + pub fn retained<'a>(&'a self, src: &'a SourceInventory) -> Vec<(&'a str, &'a Value)> { + let live: HashSet<&str> = src + .nodes + .iter() + .enumerate() + .filter(|(n, _)| !self.nodes.contains(n)) + .flat_map(|(_, node)| node.inputs.iter().map(String::as_str)) + .chain(src.outputs.iter().map(String::as_str)) + .collect(); + let mut v: Vec<_> = self + .values + .iter() + .filter(|(k, _)| live.contains(k.as_str())) + .map(|(k, v)| (k.as_str(), v)) + .collect(); + v.sort_by_key(|(k, _)| *k); + v + } +} + +/// The value `Shape` produces for a known shape, honouring `start`/`end` +/// (opset 15) with ONNX clamping. `None` if a dimension is not known. +pub fn shape_value(shape: &[Dim], node: &crate::source::SourceNode) -> Option { + let dims: Vec = shape + .iter() + .map(|d| { + if let Dim::Known(v) = d { + Some(*v as i64) + } else { + None + } + }) + .collect::>()?; + let out = crate::registry::shape_slice(&dims, node); + Some(Value::I64( + ndarray::ArrayD::from_shape_vec(ndarray::IxDyn(&[out.len()]), out).expect("1-D"), + )) +} + +/// Fold every node whose result is invariant under the source and the +/// facts. Uses the registry's own WHEN and DO; never a separate evaluator. +pub fn fold(src: &SourceInventory, facts: &HashMap, registry: &Registry) -> Folded { + let mut known: HashMap = src + .initializers + .iter() + .filter_map(|i| i.value.clone().map(|v| (i.name.clone(), v))) + .collect(); + let mut out = Folded::default(); + let (_, node_dep) = classify(src, facts); + loop { + let mut changed = false; + for (n, node) in src.nodes.iter().enumerate() { + // Skips work only: a Symbolic or Data node never has every + // input known, so it could not fold anyway (measured: removing + // this test changes no result). + if out.nodes.contains(&n) || node_dep[n] > Dep::Shape { + continue; + } + let args: Vec> = node.inputs.iter().map(|i| facts.get(i)).collect(); + let Some(reg) = registry.lookup(node, &args) else { + continue; + }; + let consts: Vec> = node.inputs.iter().map(|i| known.get(i)).collect(); + if (reg.when)(&NodeView { + node, + args: args.clone(), + consts: consts.clone(), + }) != K3::True + { + continue; + } + let vals = if node.op_type == "Shape" && node.default_domain() { + match node + .inputs + .first() + .and_then(|i| facts.get(i)) + .and_then(|f| f.shape.as_deref()) + { + Some(s) => shape_value(s, node).map(|v| vec![v]), + None => None, + } + } else { + let present: Vec<&Value> = node + .inputs + .iter() + .filter(|i| !i.is_empty()) + .map(|i| known.get(i)) + .collect::>() + .unwrap_or_default(); + if present.len() != node.inputs.iter().filter(|i| !i.is_empty()).count() { + None + } else { + (reg.do_)(&present, node).ok() + } + }; + if let Some(vals) = vals { + for (o, v) in node.outputs.iter().zip(vals) { + known.insert(o.clone(), v.clone()); + out.values.insert(o.clone(), v); + } + out.nodes.insert(n); + changed = true; + } + } + if !changed { + break; + } + } + for (n, node) in src.nodes.iter().enumerate() { + if out.nodes.contains(&n) || node_dep[n] > Dep::Shape || node.op_type == "Constant" { + continue; + } + let args: Vec> = node.inputs.iter().map(|i| facts.get(i)).collect(); + let why = match registry.lookup(node, &args) { + None => format!("unregistered {}", node.op_type), + Some(reg) => { + let consts: Vec> = + node.inputs.iter().map(|i| known.get(i)).collect(); + let missing = node + .inputs + .iter() + .zip(&consts) + .any(|(i, c)| !i.is_empty() && c.is_none()); + if missing && !(node.op_type == "Shape" && node.default_domain()) { + format!("{}: an input is not folded upstream", node.op_type) + } else { + let t = (reg.when)(&NodeView { node, args, consts }); + format!("{}: WHEN {t:?}", node.op_type) + } + } + }; + out.blocked.push((n, why)); + } + out +} diff --git a/crates/tract-harvest-probe/src/lib.rs b/crates/tract-harvest-probe/src/lib.rs index d1b77033b..6ebfbff85 100644 --- a/crates/tract-harvest-probe/src/lib.rs +++ b/crates/tract-harvest-probe/src/lib.rs @@ -19,6 +19,7 @@ //! Execution never calls Tract. No native crate depends on this crate. pub mod dr3; +pub mod fold; pub mod gather; pub mod plan; pub mod provenance; diff --git a/crates/tract-harvest-probe/src/plan.rs b/crates/tract-harvest-probe/src/plan.rs index 43b9dd1ea..e13a07d18 100644 --- a/crates/tract-harvest-probe/src/plan.rs +++ b/crates/tract-harvest-probe/src/plan.rs @@ -36,6 +36,9 @@ pub struct Plan<'r> { pub graph: OnnxGraph, /// In schedule order. pub steps: Vec, + /// Nodes executed once at resolution (see [`crate::fold`]): their outputs + /// are constants of the plan and they are not steps of a run. + pub folded: Vec, registry: &'r Registry, nodes: Vec, frame: Frame, @@ -63,13 +66,28 @@ pub fn revisions(src: &SourceInventory, registry: &Registry, graph: &OnnxGraph) /// Bind every node of `graph` against `registry`, in schedule order. pub fn resolve<'r>( src: &SourceInventory, - graph: OnnxGraph, + mut graph: OnnxGraph, registry: &'r Registry, store: &FrameStore, ) -> Result, Refusal> { let opset = src.default_opset().unwrap_or(0); let order = graph.schedule()?; + // Cold folding over the tile's own facts: a folded output becomes a + // constant before any guard below reads it. + let names: HashMap = graph + .props + .iter() + .cloned() + .zip(graph.facts.iter().cloned()) + .collect(); + let fold = crate::fold::fold(src, &names, registry); + for (p, name) in graph.props.iter().enumerate() { + if let Some(v) = fold.values.get(name) { + graph.consts[p] = Some(v.clone()); + } + } let mut steps = Vec::with_capacity(order.len()); + let mut folded = Vec::new(); for &n in &order { let node = &src.nodes[n]; let args: Vec<_> = graph.args[n] @@ -88,6 +106,23 @@ pub fn resolve<'r>( opset, }); } + let reg_ix = registry + .regs + .iter() + .position(|r| std::ptr::eq(r, reg)) + .expect("from the table"); + let bound = Bound { + node: n, + name: node.name.clone(), + classid: reg.classid, + method: reg.method, + numeric: reg.numeric, + reg: reg_ix, + }; + if fold.nodes.contains(&n) { + folded.push(bound); + continue; + } let consts = graph.args[n] .iter() .map(|p| p.and_then(|p| graph.consts[p].as_ref())) @@ -101,19 +136,7 @@ pub fn resolve<'r>( guard: reg.guard, }); } - let reg_ix = registry - .regs - .iter() - .position(|r| std::ptr::eq(r, reg)) - .expect("from the table"); - steps.push(Bound { - node: n, - name: node.name.clone(), - classid: reg.classid, - method: reg.method, - numeric: reg.numeric, - reg: reg_ix, - }); + steps.push(bound); } if store.revisions() != revisions(src, registry, &graph) { return Err(Refusal::Frame( @@ -123,6 +146,7 @@ pub fn resolve<'r>( Ok(Plan { graph, steps, + folded, registry, nodes: src.nodes.clone(), frame: store.seal(), @@ -192,7 +216,10 @@ impl Plan<'_> { /// Registration coverage of a graph too large for one tile: per op type, how /// many nodes have a registered signature with WHEN TRUE, a registered /// signature with WHEN not TRUE, or no registration. Facts and constants are -/// looked up by value name, as [`OnnxGraph::build`] does. +/// looked up by value name, as [`OnnxGraph::build`] does; the constants +/// include the values [`crate::fold::fold`] computes, so a guard over a +/// folded shape (a `Reshape` target) reads it. A folded node counts as WHEN +/// TRUE. pub fn coverage( src: &SourceInventory, facts: &HashMap, @@ -214,6 +241,10 @@ pub fn coverage( consts.insert(o.as_str(), v); } } + let fold = crate::fold::fold(src, facts, registry); + for (k, v) in &fold.values { + consts.insert(k.as_str(), v); + } let mut out = std::collections::BTreeMap::new(); for node in &src.nodes { let args: Vec<_> = node.inputs.iter().map(|i| facts.get(i)).collect(); diff --git a/crates/tract-harvest-probe/src/registry.rs b/crates/tract-harvest-probe/src/registry.rs index 5be499690..58d2f79f0 100644 --- a/crates/tract-harvest-probe/src/registry.rs +++ b/crates/tract-harvest-probe/src/registry.rs @@ -108,11 +108,18 @@ pub type Polyfill = fn(&[&Value], &SourceNode) -> Result, Refusal>; /// A WHEN guard: truth and the guard's name. pub type Guard = fn(&NodeView) -> K3; -/// A required discrete attribute value, with the ONNX default when absent. +/// A discrete integer attribute the signature admits. +/// +/// - `value: Some(v)`: the attribute must equal `v` (or be absent with +/// `default == Some(v)`); a different value is a different signature. +/// - `value: None`: any integer, read by the DO as a parameter (an axis). +/// Absent is admitted only if `default` is `Some` (the ONNX default the DO +/// applies). The method's behaviour does not depend on which value: the +/// same immutable function reads it. #[derive(Debug, Clone, Copy)] pub struct AttrRule { pub name: &'static str, - pub value: i64, + pub value: Option, pub default: Option, } @@ -209,13 +216,17 @@ fn attrs_match(node: &SourceNode, rules: &[AttrRule]) -> bool { // listed discrete attribute with the required value. let listed = node.attrs.iter().all(|(n, a)| { (n == "value" && matches!(a, Attr::Tensor(_))) - || rules - .iter() - .any(|r| r.name == n && matches!(a, Attr::Int(v) if *v == r.value)) + || rules.iter().any(|r| { + r.name == n && matches!(a, Attr::Int(v) if r.value.is_none_or(|w| *v == w)) + }) + }); + let defaults = rules.iter().all(|r| { + node.attr(r.name).is_some() + || match r.value { + Some(w) => r.default == Some(w), + None => r.default.is_some(), + } }); - let defaults = rules - .iter() - .all(|r| node.attr(r.name).is_some() || r.default == Some(r.value)); listed && defaults } @@ -405,6 +416,181 @@ fn guard_where(v: &NodeView) -> K3 { } } +fn int_attr(n: &SourceNode, name: &str, default: i64) -> i64 { + match n.attr(name) { + Some(Attr::Int(v)) => *v, + _ => default, + } +} + +/// `axis` normalised against `rank`, ONNX range `[-rank, rank)`. +fn norm_axis(axis: i64, rank: usize) -> Option { + let r = rank as i64; + (-r..r).contains(&axis).then(|| axis.rem_euclid(r) as usize) +} + +fn arg_shape<'a>(v: &'a NodeView, i: usize) -> Option<&'a Vec> { + v.args + .get(i) + .copied() + .flatten() + .and_then(|f| f.shape.as_ref()) +} + +/// `Gather` over shape data: the indices must be a constant (their validity +/// is otherwise a run-time fact) inside `[-d, d)` of the gathered dimension. +fn guard_gather(v: &NodeView) -> K3 { + let Some(data) = arg_shape(v, 0) else { + return K3::Unknown; + }; + let Some(axis) = norm_axis(int_attr(v.node, "axis", 0), data.len()) else { + return K3::False; + }; + match v.args.get(1).copied().flatten().and_then(|f| f.dtype) { + Some(DType::I64 | DType::I32) => {} + Some(_) => return K3::False, + None => return K3::Unknown, + } + let Some(idx) = v.consts.get(1).copied().flatten().and_then(Value::i64s) else { + return K3::Unknown; + }; + match data[axis] { + Dim::Known(d) => { + let d = d as i64; + if idx.iter().all(|i| (-d..d).contains(i)) { + K3::True + } else { + K3::False + } + } + Dim::Symbol(_) => K3::Unknown, + } +} + +/// `Unsqueeze` (opset 13: `axes` is an input): constant, unique axes within +/// the output rank. +fn guard_unsqueeze(v: &NodeView) -> K3 { + if v.node.inputs.get(1).is_none_or(|s| s.is_empty()) { + return K3::False; + } + let Some(axes) = v.consts.get(1).copied().flatten().and_then(Value::i64s) else { + return K3::Unknown; + }; + let Some(data) = arg_shape(v, 0) else { + return K3::Unknown; + }; + match unsqueeze_shape(data.len(), &axes) { + Some(_) => K3::True, + None => K3::False, + } +} + +/// The positions of the inserted axes in the output, sorted, or `None` if +/// an axis is out of range or repeated. +fn unsqueeze_shape(rank: usize, axes: &[i64]) -> Option> { + let out = rank + axes.len(); + let mut a: Vec = axes + .iter() + .map(|&x| norm_axis(x, out)) + .collect::>()?; + a.sort_unstable(); + a.windows(2).all(|w| w[0] != w[1]).then_some(a) +} + +/// `Concat`: every input of one rank and one dtype, equal in every dimension +/// but `axis`. +fn guard_concat(v: &NodeView) -> K3 { + let facts: Vec<&Fact> = match v.args.iter().copied().collect::>>() { + Some(f) if !f.is_empty() => f, + _ => return K3::Unknown, + }; + let dt = facts[0].dtype; + if facts.iter().any(|f| f.dtype.is_none()) { + return K3::Unknown; + } + if facts.iter().any(|f| f.dtype != dt) { + return K3::False; + } + let Some(shapes) = facts + .iter() + .map(|f| f.shape.as_ref()) + .collect::>>() + else { + return K3::Unknown; + }; + let rank = shapes[0].len(); + if shapes.iter().any(|s| s.len() != rank) { + return K3::False; + } + let Some(axis) = norm_axis(int_attr(v.node, "axis", i64::MIN), rank) else { + return K3::False; + }; + let mut unknown = false; + for d in (0..rank).filter(|&d| d != axis) { + for s in &shapes[1..] { + match (&shapes[0][d], &s[d]) { + (Dim::Known(a), Dim::Known(b)) if a != b => return K3::False, + (Dim::Known(_), Dim::Known(_)) => {} + (Dim::Symbol(a), Dim::Symbol(b)) if a == b => {} + _ => unknown = true, + } + } + } + if unknown { + K3::Unknown + } else { + K3::True + } +} + +/// The output shape of `Reshape(data, target)` with `allowzero = 0`: `0` +/// copies the input dimension, one `-1` is inferred. `None` if the target +/// is invalid for this input. +fn reshape_target(input: &[usize], target: &[i64]) -> Option> { + let mut out = Vec::with_capacity(target.len()); + let mut infer = None; + for (i, &t) in target.iter().enumerate() { + match t { + 0 => out.push(*input.get(i)?), + -1 if infer.is_none() => { + infer = Some(i); + out.push(1); + } + t if t > 0 => out.push(t as usize), + _ => return None, + } + } + let total: usize = input.iter().product(); + let known: usize = out.iter().product(); + match infer { + Some(i) if known != 0 && total.is_multiple_of(known) => out[i] = total / known, + Some(_) => return None, + None if known != total => return None, + None => {} + } + Some(out) +} + +/// `Reshape` with a constant target over a fully known input shape. +fn guard_reshape(v: &NodeView) -> K3 { + let Some(target) = v.consts.get(1).copied().flatten().and_then(Value::i64s) else { + return K3::Unknown; + }; + let Some(input) = v + .args + .first() + .copied() + .flatten() + .and_then(|f| known(&f.shape)) + else { + return K3::Unknown; + }; + match reshape_target(&input, &target) { + Some(_) => K3::True, + None => K3::False, + } +} + // ── DO polyfills: existing native functions only ─────────────────────────── fn f32_arg<'a>( @@ -555,7 +741,8 @@ fn do_greater_i32(a: &[&Value], n: &SourceNode) -> Result, Refusal> { }]) } -/// `Where(C, X, Y)`, int32, `Y` scalar or of `X`'s shape: `blend_i32`. +/// `Where(C, X, Y)`, int32, `Y` of `X`'s shape: `blend_i32`; `Y` scalar: +/// `blend_i32_scalar` (ndarray #399), which reads the scalar directly. fn do_where_i32(a: &[&Value], n: &SourceNode) -> Result, Refusal> { let Some(Value::Bool { shape, words }) = a.first() else { return Err(Refusal::Runtime { @@ -572,43 +759,245 @@ fn do_where_i32(a: &[&Value], n: &SourceNode) -> Result, Refusal> { }); } let xs: Vec = x.iter().copied().collect(); - let ys: Vec = if y.len() == 1 { - vec![y.iter().copied().next().unwrap_or(0); xs.len()] + let mut out = vec![0i32; xs.len()]; + if let (1, Some(&yv)) = (y.len(), y.iter().next()) { + // Scalar Y: the mask picks X, the rest is the scalar; no Y vector. + ndarray::simd::blend_i32_scalar(words, &xs, yv, &mut out); } else { - y.iter().copied().collect() - }; - if ys.len() != xs.len() { - return Err(Refusal::Runtime { - node: n.name.clone(), - what: "Y does not broadcast to X".into(), - }); + let ys: Vec = y.iter().copied().collect(); + if ys.len() != xs.len() { + return Err(Refusal::Runtime { + node: n.name.clone(), + what: "Y does not broadcast to X".into(), + }); + } + lance_graph_onnx::where_i32(words, &xs, &ys, &mut out); } - let mut out = vec![0i32; xs.len()]; - lance_graph_onnx::where_i32(words, &xs, &ys, &mut out); Ok(vec![Value::I32( ArrayD::from_shape_vec(IxDyn(x.shape()), out).expect("same length"), )]) } +fn i64_arg<'a>( + args: &[&'a Value], + i: usize, + node: &SourceNode, +) -> Result<&'a ArrayD, Refusal> { + match args.get(i) { + Some(Value::I64(a)) => Ok(a), + other => Err(Refusal::Runtime { + node: node.name.clone(), + what: format!("arg {i}: expected i64, got {:?}", other.map(|v| v.dtype())), + }), + } +} + +fn runtime(n: &SourceNode, what: impl Into) -> Refusal { + Refusal::Runtime { + node: n.name.clone(), + what: what.into(), + } +} + +/// `Shape`'s `start`/`end` (opset 15) applied to `dims`, with ONNX clamping. +pub fn shape_slice(dims: &[i64], node: &SourceNode) -> Vec { + let r = dims.len() as i64; + let clamp = |v: i64| (if v < 0 { v + r } else { v }).clamp(0, r) as usize; + let (s, e) = ( + clamp(int_attr(node, "start", 0)), + clamp(int_attr(node, "end", r)), + ); + if s < e { + dims[s..e].to_vec() + } else { + vec![] + } +} + +fn i64_vector(v: Vec) -> Value { + Value::I64(ArrayD::from_shape_vec(IxDyn(&[v.len()]), v).expect("1-D")) +} + +/// `Shape`: the value's shape (no element is read). +fn do_shape(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = a.first().ok_or_else(|| runtime(n, "no input"))?; + let dims: Vec = x.shape().iter().map(|&d| d as i64).collect(); + Ok(vec![i64_vector(shape_slice(&dims, n))]) +} + +/// The same elements in a new shape: an `ndarray` reshape of the standard +/// layout, or the packed mask with its logical shape replaced. +fn reshape_value(v: &Value, shape: &[usize], n: &SourceNode) -> Result { + fn re(a: &ArrayD, s: &[usize]) -> Option> { + a.as_standard_layout() + .into_owned() + .into_shape_with_order(IxDyn(s)) + .ok() + } + let bad = || runtime(n, format!("cannot reshape {:?} to {shape:?}", v.shape())); + Ok(match v { + Value::F32(a) => Value::F32(re(a, shape).ok_or_else(bad)?), + Value::I32(a) => Value::I32(re(a, shape).ok_or_else(bad)?), + Value::I64(a) => Value::I64(re(a, shape).ok_or_else(bad)?), + Value::Bool { shape: s, words } => { + if s.iter().product::() != shape.iter().product::() { + return Err(bad()); + } + Value::Bool { + shape: shape.to_vec(), + words: words.clone(), + } + } + }) +} + +/// `Gather` over `int64` data: `ndarray`'s `select` along `axis`, then the +/// index tensor's shape in place of that axis. An index outside `[-d, d)` +/// refuses. +fn do_gather_i64(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = i64_arg(a, 0, n)?; + let idx_v = a.get(1).ok_or_else(|| runtime(n, "no indices"))?; + let idx = idx_v + .i64s() + .ok_or_else(|| runtime(n, "indices are not integers"))?; + let axis = norm_axis(int_attr(n, "axis", 0), x.ndim()) + .ok_or_else(|| runtime(n, "axis out of range"))?; + let d = x.shape()[axis] as i64; + let ix: Vec = idx + .iter() + .map(|&i| { + (-d..d) + .contains(&i) + .then(|| i.rem_euclid(d) as usize) + .ok_or_else(|| runtime(n, format!("index {i} outside [-{d}, {d})"))) + }) + .collect::>()?; + let sel = x.select(Axis(axis), &ix); + let mut shape = x.shape()[..axis].to_vec(); + shape.extend(idx_v.shape()); + shape.extend(&x.shape()[axis + 1..]); + Ok(vec![reshape_value(&Value::I64(sel), &shape, n)?]) +} + +fn do_unsqueeze(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = a.first().ok_or_else(|| runtime(n, "no input"))?; + let axes = a + .get(1) + .and_then(|v| v.i64s()) + .ok_or_else(|| runtime(n, "no axes"))?; + let at = unsqueeze_shape(x.shape().len(), &axes) + .ok_or_else(|| runtime(n, format!("invalid axes {axes:?}")))?; + let mut shape = x.shape(); + for p in at { + shape.insert(p, 1); + } + Ok(vec![reshape_value(x, &shape, n)?]) +} + +fn do_reshape(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = a.first().ok_or_else(|| runtime(n, "no input"))?; + let target = a + .get(1) + .and_then(|v| v.i64s()) + .ok_or_else(|| runtime(n, "no target shape"))?; + let shape = reshape_target(&x.shape(), &target) + .ok_or_else(|| runtime(n, format!("target {target:?} invalid for {:?}", x.shape())))?; + Ok(vec![reshape_value(x, &shape, n)?]) +} + +/// `Concat` along `axis`: `ndarray::concatenate` over the input views. +fn do_concat(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + fn cat(xs: Vec<&ArrayD>, ax: usize, n: &SourceNode) -> Result, Refusal> { + let views: Vec<_> = xs.iter().map(|x| x.view()).collect(); + ndarray::concatenate(Axis(ax), &views).map_err(|e| runtime(n, e.to_string())) + } + let rank = a + .first() + .ok_or_else(|| runtime(n, "no input"))? + .shape() + .len(); + let ax = norm_axis(int_attr(n, "axis", i64::MIN), rank) + .ok_or_else(|| runtime(n, "axis out of range"))?; + let out = match a[0] { + Value::I64(_) => Value::I64(cat( + (0..a.len()) + .map(|i| i64_arg(a, i, n)) + .collect::>()?, + ax, + n, + )?), + Value::F32(_) => Value::F32(cat( + (0..a.len()) + .map(|i| f32_arg(a, i, n)) + .collect::>()?, + ax, + n, + )?), + other => return Err(runtime(n, format!("concat of {:?}", other.dtype()))), + }; + Ok(vec![out]) +} + +/// `Erf`: `ndarray::hpc::vml::vserf`, at most 2 ULP over `[-5, 5]`. +fn do_erf(a: &[&Value], n: &SourceNode) -> Result, Refusal> { + let x = f32_arg(a, 0, n)?; + let mut out = ArrayD::::zeros(x.raw_dim()); + ndarray::hpc::vml::vserf(x.view(), out.view_mut()); + Ok(vec![Value::F32(out)]) +} + // ── the table ─────────────────────────────────────────────────────────────── const OPSETS: RangeInclusive = 13..=18; const NO_ATTRS: &[AttrRule] = &[]; const CAST_TO_F32: &[AttrRule] = &[AttrRule { name: "to", - value: 1, + value: Some(1), + default: None, +}]; +const CAST_TO_I64: &[AttrRule] = &[AttrRule { + name: "to", + value: Some(7), default: None, }]; const KEEPDIMS_1: &[AttrRule] = &[AttrRule { name: "keepdims", - value: 1, + value: Some(1), default: Some(1), }]; const KEEPDIMS_0: &[AttrRule] = &[AttrRule { name: "keepdims", - value: 0, + value: Some(0), default: None, }]; +const GATHER_AXIS: &[AttrRule] = &[AttrRule { + name: "axis", + value: None, + default: Some(0), +}]; +const CONCAT_AXIS: &[AttrRule] = &[AttrRule { + name: "axis", + value: None, + default: None, +}]; +/// `end` defaults to the rank; `i64::MAX` clamps to it. +const SHAPE_RANGE: &[AttrRule] = &[ + AttrRule { + name: "start", + value: None, + default: Some(0), + }, + AttrRule { + name: "end", + value: None, + default: Some(i64::MAX), + }, +]; +const ALLOWZERO_0: &[AttrRule] = &[AttrRule { + name: "allowzero", + value: Some(0), + default: Some(0), +}]; macro_rules! reg { ($cid:expr, $m:expr, $op:expr, $arg:expr, $dt:expr, $attrs:expr, $when:path, $do:path, $num:expr) => { @@ -629,12 +1018,12 @@ macro_rules! reg { }; } -/// The lab registry: every op the two fixtures use, nothing else. +/// The lab registry: every op the fixtures use, nothing else. pub fn registry() -> Registry { use DType::*; use Numeric::*; Registry { - version: 1, + version: 2, regs: vec![ reg!( 0x0F01, @@ -680,6 +1069,18 @@ pub fn registry() -> Registry { do_identity, Exact ), + // The signature fixes input dtype I64 and `to = INT64`. + reg!( + 0x0F11, + "Cast i64 -> i64: identity", + "Cast", + Some(0), + I64, + CAST_TO_I64, + guard_true, + do_identity, + Exact + ), reg!( 0x0F20, "lance_graph_onnx::add (broadcast checked)", @@ -803,7 +1204,7 @@ pub fn registry() -> Registry { ), reg!( 0x0F51, - "lance_graph_onnx::where_i32 = simd::blend_i32", + "simd::blend_i32 (Y tensor) / simd::blend_i32_scalar (Y scalar)", "Where", Some(1), I32, @@ -812,6 +1213,138 @@ pub fn registry() -> Registry { do_where_i32, Exact ), + reg!( + 0x0F33, + "ndarray::hpc::vml::vserf", + "Erf", + Some(0), + F32, + NO_ATTRS, + guard_true, + do_erf, + Bounded("at most 2 ULP over [-5, 5] (ndarray #399)") + ), + reg!( + 0x0F60, + "the input's shape, start/end clamped", + "Shape", + Some(0), + F32, + SHAPE_RANGE, + guard_true, + do_shape, + Exact + ), + reg!( + 0x0F61, + "the input's shape, start/end clamped", + "Shape", + Some(0), + I64, + SHAPE_RANGE, + guard_true, + do_shape, + Exact + ), + reg!( + 0x0F62, + "the input's shape, start/end clamped", + "Shape", + Some(0), + I32, + SHAPE_RANGE, + guard_true, + do_shape, + Exact + ), + reg!( + 0x0F63, + "the input's shape, start/end clamped", + "Shape", + Some(0), + Bool, + SHAPE_RANGE, + guard_true, + do_shape, + Exact + ), + reg!( + 0x0F64, + "ndarray select along axis, int64 data", + "Gather", + Some(0), + I64, + GATHER_AXIS, + guard_gather, + do_gather_i64, + Exact + ), + reg!( + 0x0F65, + "ndarray reshape: insert unit axes", + "Unsqueeze", + Some(0), + I64, + NO_ATTRS, + guard_unsqueeze, + do_unsqueeze, + Exact + ), + reg!( + 0x0F66, + "ndarray reshape: insert unit axes", + "Unsqueeze", + Some(0), + F32, + NO_ATTRS, + guard_unsqueeze, + do_unsqueeze, + Exact + ), + reg!( + 0x0F67, + "ndarray::concatenate along axis", + "Concat", + Some(0), + I64, + CONCAT_AXIS, + guard_concat, + do_concat, + Exact + ), + reg!( + 0x0F68, + "ndarray::concatenate along axis", + "Concat", + Some(0), + F32, + CONCAT_AXIS, + guard_concat, + do_concat, + Exact + ), + reg!( + 0x0F69, + "ndarray reshape, allowzero 0", + "Reshape", + Some(0), + I64, + ALLOWZERO_0, + guard_reshape, + do_reshape, + Exact + ), + reg!( + 0x0F6A, + "ndarray reshape, allowzero 0", + "Reshape", + Some(0), + F32, + ALLOWZERO_0, + guard_reshape, + do_reshape, + Exact + ), ], } } @@ -832,6 +1365,165 @@ mod tests { Value::F32(ArrayD::from_elem(IxDyn(shape), 2.0)) } + fn node(op: &str, inputs: usize, attrs: Vec<(String, Attr)>) -> SourceNode { + SourceNode { + name: format!("n_{op}"), + op_type: op.into(), + domain: String::new(), + inputs: (0..inputs).map(|i| format!("in{i}")).collect(), + outputs: vec!["out".into()], + attributes: attrs.iter().map(|(n, _)| n.clone()).collect(), + attrs, + } + } + + fn f(shape: Vec) -> Fact { + Fact { + dtype: Some(DType::F32), + shape: Some(shape), + } + } + + /// Concat compares every non-axis dimension. Two different symbols are + /// not proven equal (UNKNOWN, never TRUE); the same symbol is; known + /// dimensions that differ are FALSE. + #[test] + fn concat_never_aliases_two_symbols() { + use Dim::*; + let n = node("Concat", 2, vec![("axis".into(), Attr::Int(1))]); + let check = |a: Vec, b: Vec| { + let (fa, fb) = (f(a), f(b)); + guard_concat(&NodeView { + node: &n, + args: vec![Some(&fa), Some(&fb)], + consts: vec![None, None], + }) + }; + let s = |x: &str| Symbol(x.into()); + assert_eq!( + check(vec![s("N"), Known(2)], vec![s("M"), Known(3)]), + K3::Unknown + ); + assert_eq!( + check(vec![s("N"), Known(2)], vec![s("N"), Known(3)]), + K3::True + ); + assert_eq!( + check(vec![Known(4), Known(2)], vec![Known(5), Known(3)]), + K3::False + ); + assert_eq!( + check(vec![Known(4), Known(2)], vec![Known(4)]), + K3::False, + "rank" + ); + } + + /// Gather over a symbolic dimension cannot prove a constant index in + /// range (UNKNOWN); a known dimension decides it. + #[test] + fn gather_index_range_is_three_valued() { + use Dim::*; + let n = node("Gather", 2, vec![("axis".into(), Attr::Int(0))]); + let idx = Value::I64(ArrayD::from_elem(IxDyn(&[]), 2)); + let ifact = Fact { + dtype: Some(DType::I64), + shape: Some(vec![]), + }; + let check = |d: Dim| { + let data = Fact { + dtype: Some(DType::I64), + shape: Some(vec![d]), + }; + guard_gather(&NodeView { + node: &n, + args: vec![Some(&data), Some(&ifact)], + consts: vec![None, Some(&idx)], + }) + }; + assert_eq!(check(Known(3)), K3::True); + assert_eq!(check(Known(2)), K3::False); + assert_eq!(check(Symbol("N".into())), K3::Unknown); + let data = Fact { + dtype: Some(DType::I64), + shape: Some(vec![Known(3)]), + }; + let no_const = guard_gather(&NodeView { + node: &n, + args: vec![Some(&data), Some(&ifact)], + consts: vec![None, None], + }); + assert_eq!( + no_const, + K3::Unknown, + "a run-time index is not proven in range" + ); + } + + /// `Shape`'s `start`/`end` (opset 15): negative counts from the end, + /// both clamp to `[0, rank]`, an empty range is an empty shape. + #[test] + fn shape_start_end_clamp() { + let at = |a: Vec<(String, Attr)>| shape_slice(&[2, 3, 5, 7], &node("Shape", 1, a)); + assert_eq!(at(vec![]), [2, 3, 5, 7]); + assert_eq!( + at(vec![ + ("start".into(), Attr::Int(1)), + ("end".into(), Attr::Int(-1)) + ]), + [3, 5] + ); + assert_eq!(at(vec![("start".into(), Attr::Int(-10))]), [2, 3, 5, 7]); + assert_eq!(at(vec![("end".into(), Attr::Int(10))]), [2, 3, 5, 7]); + assert_eq!( + at(vec![ + ("start".into(), Attr::Int(3)), + ("end".into(), Attr::Int(1)) + ]), + [] as [i64; 0] + ); + } + + /// `0` copies, one `-1` infers, the element count must match. + #[test] + fn reshape_target_rules() { + assert_eq!(reshape_target(&[2, 3], &[3, -1]), Some(vec![3, 2])); + assert_eq!(reshape_target(&[2, 3], &[0, -1]), Some(vec![2, 3])); + assert_eq!( + reshape_target(&[2, 3], &[4, -1]), + None, + "6 is not a multiple of 4" + ); + assert_eq!(reshape_target(&[2, 3], &[-1, -1]), None); + assert_eq!(reshape_target(&[2, 3], &[5]), None); + assert_eq!( + reshape_target(&[2, 3], &[0, 0, 0]), + None, + "0 past the input rank" + ); + assert_eq!(reshape_target(&[2, 3], &[-2, 3]), None); + } + + /// An attribute rule with `value: None` admits any integer, absent only + /// with a default; a fixed rule keeps a different value a different + /// signature. + #[test] + fn attr_rules_any_and_fixed() { + let any = |a: Vec<(String, Attr)>| attrs_match(&node("Gather", 2, a), GATHER_AXIS); + assert!(any(vec![("axis".into(), Attr::Int(-1))])); + assert!(any(vec![]), "absent axis takes the default 0"); + assert!( + !any(vec![("batch_dims".into(), Attr::Int(0))]), + "an unlisted attribute does not match" + ); + let cat = |a: Vec<(String, Attr)>| attrs_match(&node("Concat", 2, a), CONCAT_AXIS); + assert!(cat(vec![("axis".into(), Attr::Int(3))])); + assert!(!cat(vec![]), "Concat's axis has no default"); + let k = |a: Vec<(String, Attr)>| attrs_match(&node("ReduceMean", 2, a), KEEPDIMS_1); + assert!(k(vec![]) && k(vec![("keepdims".into(), Attr::Int(1))])); + assert!(!k(vec![("keepdims".into(), Attr::Int(0))])); + } + #[test] fn pow2_requires_the_exponent_not_to_grow_the_base() { let node = SourceNode { diff --git a/crates/tract-harvest-probe/tests/jina.rs b/crates/tract-harvest-probe/tests/jina.rs index d46e806b3..0eedbae00 100644 --- a/crates/tract-harvest-probe/tests/jina.rs +++ b/crates/tract-harvest-probe/tests/jina.rs @@ -146,7 +146,21 @@ fn rmsnorm_slice_through_the_graph_against_ort() { .map(|s| (s.name.trim_start_matches(P).to_string(), s.classid)) .collect(); eprintln!("schedule: {steps:?}"); - assert_eq!(steps.len(), 13, "all 13 source nodes, Constants included"); + let folded: Vec<&str> = p + .folded + .iter() + .map(|s| s.name.trim_start_matches(P)) + .collect(); + eprintln!("folded: {folded:?}"); + assert_eq!( + (steps.len(), folded.len()), + (9, 4), + "13 source nodes: the 4 Constants fold at resolution, 9 run" + ); + assert!(p + .folded + .iter() + .all(|b| [0x0F01, 0x0F02, 0x0F03].contains(&b.classid))); let cid = |n: &str| steps.iter().find(|(s, _)| s == n).unwrap().1; assert_eq!( [ @@ -429,9 +443,11 @@ fn new_weight_generation_refuses_the_old_plan() { } /// The full graph (inputs pinned to [1, 10]): the inference stage labels all -/// 7,620 source values with a full fact, the typed stage 2,546, and the two -/// agree wherever both speak. Registry v1 coverage with these facts, pinned -/// exactly so any change forces a re-measure. +/// 7,620 source values with a full fact; the typed stage labels 2,546, of +/// which 1,988 are admitted (a typed fact is admitted only from the outlet +/// that is the value's source producer, `dr3::admit`); the two agree wherever +/// both speak. Registry v2 coverage with these facts, cold folding included, +/// pinned exactly so any change forces a re-measure. #[test] #[ignore = "needs fixtures/jina: python3 tools/fetch_jina_slice.py"] fn full_graph_facts_and_coverage() { @@ -458,7 +474,12 @@ fn full_graph_facts_and_coverage() { }; assert_eq!(values.len(), 7620); assert_eq!(full(&fs.inference), 7620); - assert_eq!(full(&fs.typed), 2546); + assert_eq!(full(&fs.typed), 1988); + assert!( + fs.rejected.iter().all(|r| r.stage == "typed"), + "no inference fact is rejected" + ); + assert_eq!(fs.rejected.len(), 558); assert!( fs.disagreements().is_empty(), "{:?}", @@ -470,6 +491,6 @@ fn full_graph_facts_and_coverage() { .values() .fold([0; 3], |a, c| [a[0] + c[0], a[1] + c[1], a[2] + c[2]]) }; - assert_eq!(tot(&fs.typed), [3751, 169, 3700]); - assert_eq!(tot(&fs.merged()), [4146, 0, 3474]); + assert_eq!(tot(&fs.typed), [3841, 226, 3553]); + assert_eq!(tot(&fs.merged()), [6404, 56, 1160]); } diff --git a/crates/tract-harvest-probe/tests/tiny.rs b/crates/tract-harvest-probe/tests/tiny.rs index 7e1537493..558bcec24 100644 --- a/crates/tract-harvest-probe/tests/tiny.rs +++ b/crates/tract-harvest-probe/tests/tiny.rs @@ -270,7 +270,7 @@ fn incompatible_broadcast_is_when_false() { } } -/// `Erf` has no registration: UNKNOWN, refused by signature. +/// `Acosh` has no registration: UNKNOWN, refused by signature. #[test] fn unregistered_op_is_refused() { let reg = registry(); @@ -278,7 +278,7 @@ fn unregistered_op_is_refused() { Err(Refusal::Unregistered { node, signature }) => { assert_eq!(node, "n_abs"); assert!( - signature.starts_with("ai.onnx:Erf(Some(F32))"), + signature.starts_with("ai.onnx:Acosh(Some(F32))"), "{signature}" ); } @@ -505,3 +505,265 @@ fn undetermined_input_dim_gives_no_shape_and_typing_failure_is_recorded() { other => panic!("{:?}", other.map(|_| ())), } } + +// ── D-TRACT-SHAPE-1: cold shape folding ───────────────────────────────────── + +fn ort_f32(v: &str, out: &str) -> (Vec, Vec) { + let e = &ort()["variants"][v]["ort"][out]; + let shape = e["shape"] + .as_array() + .unwrap() + .iter() + .map(|d| d.as_u64().unwrap() as usize) + .collect(); + (shape, u32s(&e["bits"])) +} + +fn f32_bits(m: &std::collections::HashMap, k: &str) -> (Vec, Vec) { + match &m[k] { + Value::F32(a) => (a.shape().to_vec(), a.iter().map(|v| v.to_bits()).collect()), + other => panic!("{k}: {other:?}"), + } +} + +fn f_input() -> Value { + f32_value(&[2, 3], &u32s(&ort()["inputs"]["F"]["bits"])) +} + +/// Distance in units in the last place between two f32 (NaN only matches NaN). +fn ulp(a: f32, b: f32) -> u32 { + if a.is_nan() || b.is_nan() { + return if a.is_nan() && b.is_nan() { + 0 + } else { + u32::MAX + }; + } + let key = |x: f32| { + let b = x.to_bits() as i64; + if b & 0x8000_0000 != 0 { + 0x8000_0000 - b + } else { + b + } + }; + (key(a) - key(b)).unsigned_abs() as u32 +} + +/// `Concat(Unsqueeze(Gather(Shape(F), 1), [0]), [-1])` depends on F's shape +/// only, which the facts know: it folds at resolution through the registered +/// DOs. `Reshape(F, ·)` and `Unsqueeze(F, ·)` read F's values and run. The run +/// agrees with ORT bit for bit. +#[test] +fn shape_chain_folds_and_the_data_nodes_run_against_ort() { + let reg = registry(); + let (fx, p) = plan("shape_fold", ®); + let p = p.unwrap(); + let folded: Vec<(&str, u16)> = p + .folded + .iter() + .map(|b| (b.name.as_str(), b.classid)) + .collect(); + assert_eq!( + folded, + [ + ("n_shape", 0x0F60), + ("n_gather", 0x0F64), + ("n_unsq_d", 0x0F65), + ("n_concat", 0x0F67) + ] + ); + let steps: Vec<(&str, u16)> = p + .steps + .iter() + .map(|b| (b.name.as_str(), b.classid)) + .collect(); + assert_eq!(steps, [("n_reshape", 0x0F6A), ("n_unsq_f", 0x0F66)]); + let t = p.graph.props.iter().position(|n| n == "t").unwrap(); + assert_eq!( + p.graph.consts[t].as_ref().and_then(Value::i64s), + Some(vec![3, -1]) + ); + let out = p.run(&fx.store, &[("F", f_input())]).unwrap(); + assert_eq!(f32_bits(&out, "R"), ort_f32("shape_fold", "R")); + assert_eq!(f32_bits(&out, "U"), ort_f32("shape_fold", "U")); +} + +/// The folded target `[3, -1]` was computed from F's resolved shape [2, 3]. A +/// run with another F shape is refused before any step, so the fold is never +/// applied to an input it was not computed for. +#[test] +fn folded_shapes_refuse_a_run_with_other_input_dims() { + let reg = registry(); + let (fx, p) = plan("shape_fold", ®); + let p = p.unwrap(); + let f = Value::F32(ArrayD::zeros(IxDyn(&[3, 2]))); + match p.run(&fx.store, &[("F", f)]) { + Err(Refusal::Runtime { node, what }) => { + assert_eq!(node, "F"); + assert!(what.contains("resolved for"), "{what}"); + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// F's shape is fully known, yet `Unsqueeze(F)` and `Reshape(F, ·)` read F's +/// values: their class is Data and neither folds. +#[test] +fn a_node_reading_input_values_never_folds() { + use tract_harvest_probe::fold::{classify, fold, Dep}; + let b = bytes("shape_fold"); + let src = SourceInventory::from_bytes(&b).unwrap(); + let facts = tract_facts(&b, &[]).unwrap(); + let (_, dep) = classify(&src, &facts); + let by: Vec<(&str, Dep)> = src.nodes.iter().map(|n| n.name.as_str()).zip(dep).collect(); + assert_eq!( + by, + [ + ("n_shape", Dep::Shape), + ("n_gather", Dep::Shape), + ("n_unsq_d", Dep::Shape), + ("n_concat", Dep::Shape), + ("n_reshape", Dep::Data), + ("n_unsq_f", Dep::Data) + ] + ); + let f = fold(&src, &facts, ®istry()); + assert!(!f.values.contains_key("R") && !f.values.contains_key("U")); +} + +/// Over `X float[N]` the same chain has a symbolic dimension: nothing folds +/// (the symbol is not a value), the Reshape target is not a constant, and its +/// WHEN is UNKNOWN, which resolution refuses rather than treating as TRUE. +#[test] +fn a_symbolic_shape_is_not_folded_and_unknown_is_not_true() { + use tract_harvest_probe::fold::{classify, fold, Dep}; + let b = bytes("shape_sym"); + let src = SourceInventory::from_bytes(&b).unwrap(); + let facts = tract_facts(&b, &[]).unwrap(); + let (_, dep) = classify(&src, &facts); + assert_eq!(dep[0], Dep::Symbolic, "Shape(X) of X[N]"); + assert!(dep.iter().all(|d| *d >= Dep::Symbolic)); + assert!(fold(&src, &facts, ®istry()).nodes.is_empty()); + let reg = registry(); + match plan("shape_sym", ®).1 { + Err(Refusal::When { + node, + classid, + truth, + .. + }) => { + assert_eq!( + (node.as_str(), classid, truth), + ("n_reshape", 0x0F6A, K3::Unknown) + ); + } + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// `Gather(Shape(F), 5)`: the index is outside `[-2, 2)`. ORT fails the run; +/// Tract panics during `analyse` (`tract-core` `gather.rs`), which the cold +/// frontend turns into a refusal instead of a crash. The guard's own FALSE on +/// this index is `registry::tests::gather_index_range_is_three_valued`. +#[test] +fn an_out_of_range_gather_index_is_refused_not_a_panic() { + assert!(ort()["variants"]["neg_gather_index"]["ort_error"] + .as_str() + .unwrap() + .contains("out of data bounds")); + match tract_facts(&bytes("neg_gather_index"), &[]) { + Err(Refusal::Frontend(e)) => assert!(e.contains("panicked during analyse"), "{e}"), + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// `Concat([2,3], [2,2], axis=0)`: ORT rejects the model and Tract's +/// inference refuses it, so the cold frontend refuses before any guard runs +/// (the guard's own FALSE on this case is a unit test in `registry`). +#[test] +fn a_concat_dimension_mismatch_is_refused_by_the_frontend() { + assert!(ort()["variants"]["neg_concat"]["ort_error"].is_string()); + match tract_facts(&bytes("neg_concat"), &[]) { + Err(Refusal::Frontend(e)) => assert!(e.contains("n_concat"), "{e}"), + other => panic!("{:?}", other.map(|_| ())), + } +} + +/// `Erf` (fixture A with Erf for Abs) runs through `vserf`; the Erf output is +/// within 2 ULP of ORT and the integer branch is exact. +#[test] +fn erf_in_the_graph_against_ort() { + let reg = registry(); + let (fx, p) = plan("erf", ®); + let p = p.unwrap(); + let erf = p.steps.iter().find(|s| s.name == "n_abs").unwrap(); + assert_eq!(erf.classid, 0x0F33); + let out = p.run(&fx.store, &inputs(None)).unwrap(); + let (shape, got) = f32_bits(&out, "A"); + let (oshape, want) = ort_f32("erf", "A"); + assert_eq!(shape, oshape); + let worst = got + .iter() + .zip(&want) + .map(|(&a, &b)| ulp(f32::from_bits(a), f32::from_bits(b))) + .max() + .unwrap(); + assert!(worst <= 2, "Erf vs ORT: {worst} ULP"); + assert_eq!(outputs(&out).0, ort_out("erf").0); +} + +/// Erf over a dense grid of [-6, 6] plus signed zeros, infinities, NaN and +/// subnormals: within 2 ULP of erf in f64 rounded to f32 over [-5, 5] (the +/// ndarray #399 contract), exact on the special values, and the measured +/// distance to ORT pinned. +#[test] +fn erf_sweep_against_f64_and_ort() { + let o = ort(); + let sw = &o["erf_sweep"]; + let xs: Vec = u32s(&sw["X_bits"]) + .into_iter() + .map(f32::from_bits) + .collect(); + let refs: Vec = u32s(&sw["f64_ref_bits"]) + .into_iter() + .map(f32::from_bits) + .collect(); + let orts: Vec = u32s(&sw["ort_bits"]) + .into_iter() + .map(f32::from_bits) + .collect(); + let b = bytes("erf_sweep"); + let src = SourceInventory::from_bytes(&b).unwrap(); + let facts = tract_facts(&b, &[]).unwrap(); + let g = OnnxGraph::build(&src, &facts).unwrap(); + let reg = registry(); + let store = FrameStore::new(revisions(&src, ®, &g)); + let p = resolve(&src, g, ®, &store).unwrap(); + let x = Value::F32(ArrayD::from_shape_vec(IxDyn(&[xs.len()]), xs.clone()).unwrap()); + let out = p.run(&store, &[("X", x)]).unwrap(); + let ys: Vec = f32_bits(&out, "Y") + .1 + .into_iter() + .map(f32::from_bits) + .collect(); + let (mut in5, mut all, mut vs_ort) = (0u32, 0u32, 0u32); + for ((&x, &y), (&r, &t)) in xs.iter().zip(&ys).zip(refs.iter().zip(&orts)) { + let d = ulp(y, r); + if x.abs() <= 5.0 { + in5 = in5.max(d); + } + all = all.max(d); + vs_ort = vs_ort.max(ulp(y, t)); + if x == 0.0 || x.is_infinite() || x.is_nan() { + assert_eq!(y.to_bits(), r.to_bits(), "erf({x:e}) = {y:e}, want {r:e}"); + } + } + eprintln!( + "Erf sweep: {} points, max ULP vs f64 {in5} on [-5,5] ({all} overall), vs ORT {vs_ort}", + xs.len() + ); + assert!(in5 <= 2, "{in5} ULP on [-5, 5]"); + assert!(all <= 2, "{all} ULP overall"); + assert!(vs_ort <= 2, "{vs_ort} ULP vs ORT"); +} diff --git a/crates/tract-harvest-probe/tools/gen_tiny.py b/crates/tract-harvest-probe/tools/gen_tiny.py index 18aa36076..d074f3c9b 100644 --- a/crates/tract-harvest-probe/tools/gen_tiny.py +++ b/crates/tract-harvest-probe/tools/gen_tiny.py @@ -41,7 +41,9 @@ def build(variant="base"): if variant == "neg_where_nonzero": z_val = 1 # ReduceSum(Where(c, X, 1)) is not a masked sum of X if variant == "neg_unsupported": - abs_op = "Erf" # no admitted native method + abs_op = "Acosh" # no admitted native method + if variant == "erf": + abs_op = "Erf" # admitted: ndarray::hpc::vml::vserf (Bounded) if variant == "neg_domain": domain = "com.example" # same op_type in a foreign domain if variant == "div3": @@ -64,12 +66,15 @@ def build(variant="base"): h.make_node("Sqrt", ["A"], ["Y"], name="n_sqrt"), h.make_node("Div", ["Y", "B"], ["Q"], name="n_div"), ] + outs = [h.make_tensor_value_info("S", T.INT32, []), + h.make_tensor_value_info("Q", T.FLOAT, [2, 3])] + if variant == "erf": + outs.append(h.make_tensor_value_info("A", T.FLOAT, [2, 3])) # the Erf output itself g = h.make_graph( nodes, f"tiny_{variant}", [h.make_tensor_value_info("X", T.INT32, ["N"]), h.make_tensor_value_info("F", T.FLOAT, [2, 3])], - [h.make_tensor_value_info("S", T.INT32, []), - h.make_tensor_value_info("Q", T.FLOAT, [2, 3])], + outs, inits) imports = [h.make_opsetid("", opset)] if domain: @@ -78,6 +83,78 @@ def build(variant="base"): return m +def i64(name, v): + return nh.from_array(np.array(v, np.int64), name) + + +def build_shape(variant): + """D-TRACT-SHAPE-1 shape plumbing. + + shape_fold: T = Concat(Unsqueeze(Gather(Shape(F), 1), [0]), [-1]) is + [3, -1], a function of F's (known) shape only, so it folds; + R = Reshape(F, T) and U = Unsqueeze(F, [0]) read F's values and run. + shape_sym: the same chain over X float[N]: Shape(X) has a symbolic + dimension, nothing folds, the Reshape target is not a constant and its + WHEN is UNKNOWN. + neg_gather_index: Gather(Shape(F), 5): index outside [-2, 2). + neg_concat: Concat(F[2,3], G[2,2], axis=0): dimension 1 differs.""" + inits = [i64("one", 1), i64("ax0", [0]), i64("m1", [-1])] + if variant == "shape_fold": + inp = [h.make_tensor_value_info("F", T.FLOAT, [2, 3])] + nodes = [ + h.make_node("Shape", ["F"], ["s"], name="n_shape"), + h.make_node("Gather", ["s", "one"], ["d"], name="n_gather", axis=0), + h.make_node("Unsqueeze", ["d", "ax0"], ["u"], name="n_unsq_d"), + h.make_node("Concat", ["u", "m1"], ["t"], name="n_concat", axis=0), + h.make_node("Reshape", ["F", "t"], ["R"], name="n_reshape"), + h.make_node("Unsqueeze", ["F", "ax0"], ["U"], name="n_unsq_f"), + ] + outs = [h.make_tensor_value_info("R", T.FLOAT, [3, 2]), + h.make_tensor_value_info("U", T.FLOAT, [1, 2, 3])] + elif variant == "shape_sym": + inp = [h.make_tensor_value_info("X", T.FLOAT, ["N"])] + inits.append(i64("zero", 0)) + nodes = [ + h.make_node("Shape", ["X"], ["s"], name="n_shape"), + h.make_node("Gather", ["s", "zero"], ["d"], name="n_gather", axis=0), + h.make_node("Unsqueeze", ["d", "ax0"], ["u"], name="n_unsq_d"), + h.make_node("Concat", ["u", "m1"], ["t"], name="n_concat", axis=0), + h.make_node("Reshape", ["X", "t"], ["R"], name="n_reshape"), + ] + outs = [h.make_tensor_value_info("R", T.FLOAT, ["N", 1])] + elif variant == "neg_gather_index": + inp = [h.make_tensor_value_info("F", T.FLOAT, [2, 3])] + inits.append(i64("five", 5)) + nodes = [ + h.make_node("Shape", ["F"], ["s"], name="n_shape"), + h.make_node("Gather", ["s", "five"], ["d"], name="n_gather", axis=0), + ] + outs = [h.make_tensor_value_info("d", T.INT64, [])] + elif variant == "neg_concat": + inp = [h.make_tensor_value_info("F", T.FLOAT, [2, 3])] + inits.append(nh.from_array(np.zeros((2, 2), np.float32), "G")) + nodes = [h.make_node("Concat", ["F", "G"], ["C"], name="n_concat", axis=0)] + outs = [h.make_tensor_value_info("C", T.FLOAT, [4, 3])] + g = h.make_graph(nodes, f"tiny_{variant}", inp, outs, inits) + return h.make_model(g, opset_imports=[h.make_opsetid("", OPSET)], producer_name="d-tract-shape-1", ir_version=8) + + +def build_erf_sweep(): + g = h.make_graph([h.make_node("Erf", ["X"], ["Y"], name="n_erf")], "erf_sweep", + [h.make_tensor_value_info("X", T.FLOAT, ["N"])], + [h.make_tensor_value_info("Y", T.FLOAT, ["N"])]) + return h.make_model(g, opset_imports=[h.make_opsetid("", OPSET)], producer_name="d-tract-shape-1", ir_version=8) + + +def erf_sweep_inputs(): + """A dense grid over [-6, 6] (the tails saturate past ~3.9), plus signed + zeros, infinities, NaN, subnormals and tiny values.""" + grid = np.linspace(-6.0, 6.0, 4097, dtype=np.float64).astype(np.float32) + special = np.array([0.0, -0.0, np.inf, -np.inf, np.nan, 1e-30, -1e-30, 1e-10, -1e-10, + np.float32(1.4e-45), 3.4e38, -3.4e38, 0.5, 1.0, 2.0, 3.9, 4.0, 5.0], np.float32) + return np.concatenate([grid, special]) + + INPUTS = { "X": np.array([3, -1, 4, -1, 5, -9, 2], np.int32), "F": np.array([[1.5, -0.25, 3.0], [0.1, 7.0, -2.5]], np.float32), @@ -91,10 +168,25 @@ def bits(a): return [int(v) for v in a.ravel()] +def dumps_compact(v, depth=0): + """JSON with one key per line, but every list of scalars on one line, so + a sweep is one line instead of one line per element.""" + pad = " " * (depth + 1) + if isinstance(v, dict): + items = [f"{pad}{json.dumps(k)}: {dumps_compact(x, depth + 1)}" for k, x in v.items()] + return "{\n" + ",\n".join(items) + "\n" + " " * depth + "}" if items else "{}" + if isinstance(v, list) and all(not isinstance(x, (dict, list)) for x in v): + return json.dumps(v, separators=(",", ":")) + if isinstance(v, list): + items = [pad + dumps_compact(x, depth + 1) for x in v] + return "[\n" + ",\n".join(items) + "\n" + " " * depth + "]" if items else "[]" + return json.dumps(v) + + def main(): os.makedirs(OUT, exist_ok=True) manifest = {"ort_version": ort.__version__, "ort_graph_optimization_level": "ORT_DISABLE_ALL", "onnx_version": onnx.__version__, "variants": {}} - for v in ["base", "gen2", "div3", "neg_shape", "neg_where_nonzero", "neg_unsupported", "neg_domain"]: + for v in ["base", "gen2", "div3", "neg_shape", "neg_where_nonzero", "neg_unsupported", "neg_domain", "erf"]: m = build(v) data = m.SerializeToString() path = os.path.join(OUT, f"{v}.onnx") @@ -109,6 +201,34 @@ def main(): except Exception as e: # ORT refusing a variant is itself evidence entry["ort_error"] = type(e).__name__ + ": " + str(e).splitlines()[0][:200] manifest["variants"][v] = entry + for v in ["shape_fold", "shape_sym", "neg_gather_index", "neg_concat"]: + data = build_shape(v).SerializeToString() + with open(os.path.join(OUT, f"{v}.onnx"), "wb") as f: + f.write(data) + entry = {"sha256": hashlib.sha256(data).hexdigest()} + feeds = {"F": INPUTS["F"]} if v != "shape_sym" else {"X": np.arange(6, dtype=np.float32)} + try: + sess = session(data) + outs = sess.run(None, feeds) + entry["ort"] = {o.name: {"shape": list(r.shape), "dtype": str(r.dtype), "bits": bits(r)} + for o, r in zip(sess.get_outputs(), outs)} + except Exception as e: + entry["ort_error"] = type(e).__name__ + ": " + str(e).splitlines()[0][:200] + manifest["variants"][v] = entry + # Erf sweep: ORT's output, and erf in f64 rounded once to f32 (the + # reference the ULP bound is stated against). + import math + data = build_erf_sweep().SerializeToString() + with open(os.path.join(OUT, "erf_sweep.onnx"), "wb") as f: + f.write(data) + xs = erf_sweep_inputs() + ref = np.array([math.erf(float(x)) if not math.isnan(x) else math.nan for x in xs], np.float64).astype(np.float32) + manifest["erf_sweep"] = { + "sha256": hashlib.sha256(data).hexdigest(), + "X_bits": bits(xs), + "ort_bits": bits(session(data).run(["Y"], {"X": xs})[0]), + "f64_ref_bits": bits(ref), + } # div3 sweep: 256 seeded F inputs through ORT, so a property that holds for # some values only (Div vs Mul(recip)) is tested on a population. rng = np.random.default_rng(20261030) @@ -121,7 +241,7 @@ def main(): } manifest["inputs"] = {k: {"shape": list(a.shape), "dtype": str(a.dtype), "bits": bits(a)} for k, a in INPUTS.items()} with open(os.path.join(OUT, "ort.json"), "w") as f: - json.dump(manifest, f, indent=1) + f.write(dumps_compact(manifest) + "\n") print(json.dumps({k: list(v.keys()) for k, v in manifest["variants"].items()}))