feat(parity): stream progress, journal every result, resume after interruption

[proggeramlug]

A full sweep reached 4h34m with zero visible output today, and an interruption at any point would have lost all of it. Results were accumulated in a bash array and serialized once, at the very end.

One mechanism, both properties

record_result() was already the single choke point every result path funnels through. It now appends one JSONL line as each test completes, and emits the progress line from the same call — so the checkpoint and the progress signal are the same write.

• Progress — [142/1178] (01:12:33) test_gap_foo … PASS on stderr. stdout is untouched because run_module_parity.sh scrapes the summary from a 2>&1 capture with awk -F': *'. The [i/N] denominator required moving --filter/--shard into a selection pre-pass; shard arithmetic and ordering unchanged.
• Journal — test-parity/reports/journal/parity_<selection>.jsonl, keyed by suite/module/filter/shard so the 8 conformance shards never share a file. A write failure is fatal rather than silently dropping results — the realistic cause is a full disk, and this box hit 11 GiB today.
• Resume — refuses on fingerprint mismatch: SHA-256 over perry and libperry_runtime.a/libperry_stdlib.a, plus selection flags and platform. Content hash, not mtime — mtime's failure mode (binary swapped in place, compares equal) is the unsafe direction, and a stale archive changes behaviour while perry is untouched.
• Pause — SIGINT/SIGTERM, exit 130/143, prints the resume command. An interrupted run deliberately does not publish latest.json: that feeds the gap/threshold/matrix gates, and a partial sweep read as complete would move a gate on tests that never ran.

Two bugs found while testing it

1. A torn final line swallowed the next record. kill -9 mid-write leaves a newline-less line; the next append concatenated onto it, making both unparseable — silently losing a second, innocent test. Caught because results came back 01,02,04,03,…. Fixed by sealing with a newline before appending.

2. tests/test_parity_build_reuse.sh's assertions were inert on macOS. A standalone failing [[ ]] does not trip set -e under bash 3.2 — verified directly:

$ /bin/bash -c 'set -e; [[ 1 -eq 2 ]]; echo REACHED'
REACHED

Every [[ ]] in that file, including pre-existing ones, could not fail a run on a Mac. Converting to explicit || fail immediately surfaced a real header/grep mismatch.

Verification

check	result
streaming w/ timestamps	mock: 8 lines at 1.1s intervals; real compiler: 12:33:00→06 as each test finished
SIGINT + resume	exit 130, latest.json withheld, resumes at [3/8], report identical; also process-group SIGINT and a real-compiler SIGTERM at 6/12
kill -9 + resume	exit 137, torn line tolerated, 0 lost, report identical
different compiler	hard-fails on mutated archive, mutated binary, and mismatched --filter; control still resumes
no new flags	report JSON byte-identical to origin/main over the same filter; both exit 0; run_gap_tests.sh wrapper unaffected
bash -n / shellcheck	clean, at parity with origin/main

Each check was verified capable of failing — four mutations (unseal the torn line, drop toolchain from the identity check, disable streaming, journal only at the end) each turn the suite red.

Left alone

known_failures.json handling, normalize_output, the 80% threshold, and the report emitter — the arrays it consumes are rebuilt from the journal rather than rewriting it, because the format has downstream-depended-upon quirks (the [""]-for-empty-failure-list that run_gap_tests.sh filters). Byte-identical output proves the equivalence.

Summary by CodeRabbit

• New Features

  • Added resumable parity test runs with durable progress journaling.
  • Added live progress updates and automatic recovery from interrupted or partially written journals.
  • Added validation to prevent resuming with mismatched builds or test selections.
  • Reports can now be reconstructed from journaled results, while interrupted runs clearly indicate how to resume.

• Tests

  • Expanded coverage for resume behavior, progress reporting, report reproducibility, and build changes.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c2e26524-45ed-4253-83e7-524c73ee97c5

📥 Commits

Reviewing files that changed from the base of the PR and between 3a26fda and 0080c6f.

📒 Files selected for processing (3)

• changelog.d/7436-parity-streaming-resume.md
• run_parity_tests.sh
• tests/test_parity_build_reuse.sh

---

📝 Walkthrough

Walkthrough

The parity runner adds durable JSONL journaling, resumable execution, selection-aware progress, torn-line recovery, interruption handling, and journal-based report reconstruction. Tests cover fresh runs, resumed runs, report equivalence, runtime changes, and process cleanup.

Changes

Parity streaming and resume

Layer / File(s)	Summary
Journal state and validation run_parity_tests.sh	Adds --journal and --resume, journal fingerprinting, JSONL recovery, identity validation, default journal naming, initialization, and durable result recording.
Selection, progress, cleanup, and interruption run_parity_tests.sh	Precomputes the selected tests, reports progress with a known total, skips journaled tests during resume, stops at test boundaries after interruption, and cleans up run-scoped processes.
Report publication and validation coverage run_parity_tests.sh, tests/test_parity_build_reuse.sh, changelog.d/7436-parity-streaming-resume.md	Rebuilds reports from journal records, preserves latest.json during interruptions, reports resume provenance, and tests journal creation, resume behavior, torn-line recovery, runtime validation, and assertion diagnostics. The changelog documents the behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• PerryTS/perry#6762: Both changes modify parity-runner cleanup and process handling.
• PerryTS/perry#6891: Both changes modify run_parity_tests.sh and parity test selection behavior.
• PerryTS/perry#7064: Both changes modify parity build or artifact validation and related reuse tests.

Suggested labels: parity, tooling, enhancement

Suggested reviewers: thehypnoo

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/parity-resumable

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}