Skip to content

Core Conformance Corpus with Python, TS and JS runners - #3

Merged
maheshvaikri-code merged 11 commits into
mainfrom
feat/conformance-corpus
Nov 5, 2026
Merged

maheshvaikri-code merged 11 commits into
mainfrom
feat/conformance-corpus

Conversation

@maheshvaikri-code

Copy link
Copy Markdown
Contributor

Summary

Adds vectors/, an executable Core-owned conformance corpus: 32 ISON/ISONL documents indexed by vectors/manifest.ison (plain ISON, parsed by each port's own parser), each with a per-port expectation (pass / pass:canonicalized / pass:lossy / defect / reject:<Error> / todo). Python, TypeScript and JavaScript each get a parametrized runner inside their existing suites, so CI runs the corpus with no new job. A Python meta-test pins the corpus rules: LF-only, exactly one trailing newline, unique ids, files exist, canonical set iff some port is pass:canonicalized, todo iff that port has no runner (admission is atomic), and every defect has a Findings row.

scripts/corpus_digest.py and scripts/corpus_receipt.py produce what ISONGraphExt pins: the corpus digest (SHA-256 over sorted path+bytes) and a per-port Core Conformance Receipt with full commit SHAs and a derived posture (FULL / LOSSY_KNOWN / KNOWN_DEFECT / PARTIAL / INCONCLUSIVE). Today: Python FULL, TS and JS KNOWN_DEFECT, Rust/C#/C++ PARTIAL.

What it found

Expectations record observed behaviour on 1.4.1; no vector was edited to go green (verified by an independent reviewer per commit). vectors/README.md § Findings:

  • TS/JS load an ison-graph-multigraph-v1 document (treating key as a property and dropping meta.graph on output, so a round trip turns it into a simple-graph document Python then accepts). Python refuses it. Marked defect - needs a TS/JS fix.
  • ison-ts/ison-js write the string nan bare; ison-py quotes it. Same graph, two canonical byte forms - an ISONCS writer divergence to settle upstream.
  • Integers beyond 2^53 are rounded in TS/JS unless exactly representable in binary64 (10^20 round-trips; 10^20 + 1 does not) - pass:lossy, the Semantic/2 JS binding rule.

Verification

  • Python 498 → 546 passed; TypeScript 221 → 255, tsc --noEmit clean; JavaScript 221 → 255
  • check_cross_port_goldens.py: the existing six-port golden fixture still passes
  • Independent whole-branch review: approve with fixes; both Important findings fixed test-first (pass:lossy now asserts the loss is real; inexact beyond-int64 vector added), Minors recorded for follow-up

Not in this PR

Rust, C# and C++ runners (columns todo; admission is atomic per port). Semantic/2 digest values (column reserved). Pinning observed output bytes for lossy/defect cells (corpus_version 2).

🤖 Generated with Claude Code

maheshvaikri-code and others added 11 commits November 5, 2026 16:17
One golden fixture and prose were all that said the six ports agree on
what a document means. vectors/manifest.ison now indexes executable
documents, each with a per-port expectation, and ison-graph/tests/
test_vectors.py runs them as part of the Python suite. A meta-test pins
the rules the runners assume: LF-only files ending in one newline,
unique ids, existing files, and that a port says 'todo' exactly when it
has no runner, so a port cannot be half-admitted.

Two vectors to start; the rest follow by group.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Positive, negative, numeric-string and int64-extreme ids round-trip;
int/string twins and float/bool ids are refused as 1.4.1 promises.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pins what the wire carries for 1 vs 1.0, -0.0, int64 edges and beyond,
bare nan/inf (strings, which ison-py quotes back out), null elision,
NFD vs NFC (kept distinct), typed-looking strings and the quoting rules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… blocks

Directed two-way vs undirected mirror pairs no longer share bytes; a
mirror pair with conflicting properties is kept as the first mirror; an
explicit 'directed true' and an unknown block kind canonicalize away; a
MultiGraph document is refused by the simple graph.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A deliberately unsorted document canonicalizes to its sibling, and the
ISONL spelling of two existing vectors loads to the same canonical ISON.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Same runner shape as Python, reading the manifest with ison-ts. Setting
the ts column from what the port does today records three things worth
knowing: it loads a MultiGraph document that every other port refuses,
ison-ts writes the string "nan" bare where ison-py quotes it, and ids or
values beyond 2^53 come back rounded. Each is in README Findings.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mirror of the TypeScript runner against ison-parser. The js column
matches ts everywhere, which is the point: the two ports share one
writer design and the corpus now proves it per document.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SHA-256 over the sorted (path, bytes) of vectors/**, so a consumer can
state exactly which corpus it was proven against without copying it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A corpus digest says which corpus a port was proven against, not how it
did. The receipt adds the port's posture - FULL, LOSSY_KNOWN,
KNOWN_DEFECT, PARTIAL or INCONCLUSIVE - derived from the manifest it is
bound to, with the counts behind it and full commit SHAs, so a
downstream consumer can refuse identity claims through a lossy or
defective port instead of discovering the problem later.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
README carries the manifest schema, the expectation grammar and the
Findings table the meta-test enforces; the changelog entry names the
three divergences the corpus surfaced on day one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review findings. A 'pass:lossy' cell only checked idempotence, so a port
that later preserved the input would keep reporting a loss forever; all
three runners now also assert the output differs from the file. And the
10^20 vector read as "TS/JS keep integers beyond int64", which is false:
that literal is exactly representable in binary64. A companion vector
with 10^20 + 1 records the rounding, and the Findings row for the
MultiGraph defect now says that TS/JS also drop meta.graph on output.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@maheshvaikri-code
maheshvaikri-code merged commit ce9d3b7 into main Nov 5, 2026
14 checks passed
@maheshvaikri-code
maheshvaikri-code deleted the feat/conformance-corpus branch November 5, 2026 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant