Repository navigation
Core Conformance Corpus with Python, TS and JS runners - #3
Merged
Merged
Conversation
One golden fixture and prose were all that said the six ports agree on what a document means. vectors/manifest.ison now indexes executable documents, each with a per-port expectation, and ison-graph/tests/ test_vectors.py runs them as part of the Python suite. A meta-test pins the rules the runners assume: LF-only files ending in one newline, unique ids, existing files, and that a port says 'todo' exactly when it has no runner, so a port cannot be half-admitted. Two vectors to start; the rest follow by group. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Positive, negative, numeric-string and int64-extreme ids round-trip; int/string twins and float/bool ids are refused as 1.4.1 promises. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pins what the wire carries for 1 vs 1.0, -0.0, int64 edges and beyond, bare nan/inf (strings, which ison-py quotes back out), null elision, NFD vs NFC (kept distinct), typed-looking strings and the quoting rules. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… blocks Directed two-way vs undirected mirror pairs no longer share bytes; a mirror pair with conflicting properties is kept as the first mirror; an explicit 'directed true' and an unknown block kind canonicalize away; a MultiGraph document is refused by the simple graph. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A deliberately unsorted document canonicalizes to its sibling, and the ISONL spelling of two existing vectors loads to the same canonical ISON. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Same runner shape as Python, reading the manifest with ison-ts. Setting the ts column from what the port does today records three things worth knowing: it loads a MultiGraph document that every other port refuses, ison-ts writes the string "nan" bare where ison-py quotes it, and ids or values beyond 2^53 come back rounded. Each is in README Findings. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mirror of the TypeScript runner against ison-parser. The js column matches ts everywhere, which is the point: the two ports share one writer design and the corpus now proves it per document. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SHA-256 over the sorted (path, bytes) of vectors/**, so a consumer can state exactly which corpus it was proven against without copying it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A corpus digest says which corpus a port was proven against, not how it did. The receipt adds the port's posture - FULL, LOSSY_KNOWN, KNOWN_DEFECT, PARTIAL or INCONCLUSIVE - derived from the manifest it is bound to, with the counts behind it and full commit SHAs, so a downstream consumer can refuse identity claims through a lossy or defective port instead of discovering the problem later. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
README carries the manifest schema, the expectation grammar and the Findings table the meta-test enforces; the changelog entry names the three divergences the corpus surfaced on day one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review findings. A 'pass:lossy' cell only checked idempotence, so a port that later preserved the input would keep reporting a loss forever; all three runners now also assert the output differs from the file. And the 10^20 vector read as "TS/JS keep integers beyond int64", which is false: that literal is exactly representable in binary64. A companion vector with 10^20 + 1 records the rounding, and the Findings row for the MultiGraph defect now says that TS/JS also drop meta.graph on output. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
vectors/, an executable Core-owned conformance corpus: 32 ISON/ISONL documents indexed byvectors/manifest.ison(plain ISON, parsed by each port's own parser), each with a per-port expectation (pass/pass:canonicalized/pass:lossy/defect/reject:<Error>/todo). Python, TypeScript and JavaScript each get a parametrized runner inside their existing suites, so CI runs the corpus with no new job. A Python meta-test pins the corpus rules: LF-only, exactly one trailing newline, unique ids, files exist,canonicalset iff some port ispass:canonicalized,todoiff that port has no runner (admission is atomic), and everydefecthas a Findings row.scripts/corpus_digest.pyandscripts/corpus_receipt.pyproduce what ISONGraphExt pins: the corpus digest (SHA-256 over sorted path+bytes) and a per-port Core Conformance Receipt with full commit SHAs and a derived posture (FULL/LOSSY_KNOWN/KNOWN_DEFECT/PARTIAL/INCONCLUSIVE). Today: PythonFULL, TS and JSKNOWN_DEFECT, Rust/C#/C++PARTIAL.What it found
Expectations record observed behaviour on 1.4.1; no vector was edited to go green (verified by an independent reviewer per commit).
vectors/README.md§ Findings:ison-graph-multigraph-v1document (treatingkeyas a property and droppingmeta.graphon output, so a round trip turns it into a simple-graph document Python then accepts). Python refuses it. Markeddefect- needs a TS/JS fix.nanbare; ison-py quotes it. Same graph, two canonical byte forms - an ISONCS writer divergence to settle upstream.10^20round-trips;10^20 + 1does not) -pass:lossy, the Semantic/2 JS binding rule.Verification
tsc --noEmitclean; JavaScript 221 → 255check_cross_port_goldens.py: the existing six-port golden fixture still passespass:lossynow asserts the loss is real; inexact beyond-int64 vector added), Minors recorded for follow-upNot in this PR
Rust, C# and C++ runners (columns
todo; admission is atomic per port). Semantic/2 digest values (column reserved). Pinning observed output bytes forlossy/defectcells (corpus_version 2).🤖 Generated with Claude Code