docs(evidence): rebind published evidence to Flow 1.27.1

[abrichr]

The failure

evidence-freshness has failed every scheduled run since 2026-07-28, most recently on 8132e60, which is head:

DRIFT: docs/eval_results/current_flow_v1_24_0_local_20260727: published evidence
is pinned to openadapt-flow 1.24.0 but the current published release is 1.27.1.
Re-run the comparison against 1.27.1 and publish a new evidence set, or the
public numbers describe an engine nobody is running.

The check is right and is not relaxed here. Flow shipped 1.25.0, 1.25.1, 1.26.0, 1.27.0 and 1.27.1 after the evidence was measured, so the published numbers described an engine nobody runs. The fix is to re-measure. No window was widened, no exclusion added, no threshold moved.

What was measured

Both runners re-run against the exact published 1.27.1 wheel (99d8f3ef…4094) bound to release-tagged tracked-clean source ee52def, on the same macOS host as the 1.24.0 evidence, plus an independent replication of the whole comparison. Zero model calls, $0.00, loopback only. Exact commands in REPRODUCE.md.

The comparison is genuinely controlled this time

MockMed ships inside the Flow wheel, so repinning the wheel can repin the target application — that caveat governed the 1.16.1 → 1.24.0 comparison. Between v1.24.0 and v1.27.1 the fixture tree is byte-identical (f0736c7a82ca2aba2beb334feaf461c5f06532a5), and the transaction probe runner is byte-identical (097cc900…40ce8). So the deltas below are attributable to the engine.

Both 1.24.0 defects are closed

	1.24.0	1.27.1	Named cause
clean region_stable over-halt	2/3 in primary and replication	0/3 in primary and replication	flow c068554 fix(compiler): stop a parameter's demonstrated value becoming a pixel invariant (#285), 1.25.0
False-absence claim on uncertain delivery	no_false_absence_claim FAILED 7/16; unverified_delivered_write_needs_reconciliation FAILED 14/23	0 violations across 30 runs	flow 11c115c fix(transaction): require positive evidence of absence for HALTED_BEFORE_EFFECT (#280), 1.25.0

The timeout fault mode — commits the row, then hangs past the client timeout — reported HALTED_BEFORE_EFFECT in all 6 runs on 1.24.0 and reports RECONCILIATION_REQUIRED in all 6 runs here.

Compiled clean end-to-end median also drops 12.888s → 7.408s, because the over-halting trials no longer spend their postcondition retry budget.

Held

Compiled replay 3/3 in every condition in both runs: 18 counted compiled trials, zero over-halts, zero silent incorrect successes, zero wrong actions, zero model calls, $0.00. Identity coverage unchanged at 5/8 armed clicks. The DOM selector controls still fail 0/3 under label drift, loudly, at the first renamed locator and before any mutation.

Reported honestly rather than rounded up

• One invariant is now vacuous, not a pass: no counted run claimed an absence, so no run could exercise it.
• No Flow-versus-zero-shot comparison is claimed. It still needs a paid model and a paid WAA environment.
• The comparison runner digest moved (ac58c0b9…5970f → 68f9e5a2…8a126) for two non-measurement reasons only: 1801027 replaced a hard-coded 1.16.1 reproduce string with a formatted one (12 lines, metadata in an emitted JSON file), and c4b7e9b deleted one blank line during the ruff cleanup. Arms, oracle, classification rules, trial count and retry policy are unchanged. This is stated in REPRODUCE.md rather than left for a reader to discover.

Checks run locally

• scripts/check_published_evidence_freshness.py against PyPI: exit 0, "bound to the current openadapt-flow release 1.27.1".
• --offline half: exit 0.
• ruff check .: clean. uv build + check_source_boundary.py --require-dist: OK.
• pytest tests/ -m "not heavy and not gpu and not vm": 2137 passed, 55 skipped.

1.24.0 is marked superseded, never deleted, and stays reproducible against the wheel it was measured on.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM