Skip to content

provenance: compare the saved record with its source document - #1446

Merged
abrichr merged 4 commits into
mainfrom
verification-coverage/provenance
Oct 21, 2026
Merged

abrichr merged 4 commits into
mainfrom
verification-coverage/provenance

Conversation

@abrichr

@abrichr abrichr commented Oct 20, 2026

Copy link
Copy Markdown
Member

Part 3 of #1434 (docs/VERIFICATION_COVERAGE.md). Stacked on #1445; the base is its branch.

Problem

A record check confirms that the store holds the values it was given. If the referral says 02139 and the extraction read 02193, a faithful save of 02193 passes the delta and the frame. The error happened before the save, and the read-back can't see it.

Change

  • openadapt_flow/provenance.py:
    • openadapt.param-provenance/v1: each value names its source, which is one of: a document (the digest of its bytes, plus the line and label), a worklist cell, or the instruction itself.
    • cross_check: a second extraction by a different method compares value by value, using the screen read-back's kind rules (text, code, date, currency), read exactly. A disagreement, a missing value, a missing document, or the same method twice holds the item.
    • round_trip: compares the record as read back with the second reading of the cited source. It never compares with the params.
    • LabelExtractor (the deterministic intake label parse) and CallableExtractor (a second model or OCR, supplied by the caller). No model calls here.
  • intake writes params.provenance.json beside params.json. Intake already found the line for every field and then dropped it.
  • verify-run --provenance FILE --source DOC:
    • checked.source becomes held, refuted or not_checked from the round trip. Without provenance it stays instruction.
    • Params that differ from their own provenance refute the source part.
    • evidence.source names the fields and extractors, never the values.
  • Guarded save:
    • A task may carry provenance and sources.
    • begin runs the second reading. A value it doesn't confirm sends the save to a person under any approval mode, and the approval request names the fields.
    • confirm runs the round trip.
    • A task whose values differ from its provenance is refused.
  • Docs: VERIFY_RUN.md, GUARDED_SAVE.md, INTAKE.md; a CAPABILITIES.md row.

Safety review

This adds refusals and holds only: a refuted round trip blocks verified, and a disagreement forces a person's approval. Without provenance nothing changes (source: instruction). It touches when verified is issued, so it waits for the coordinator with #1443 and #1445.

Not covered

A source document that is itself wrong, and a value both methods misread the same way. Values from intake:local hold in the guard until a person confirms them, because the only built-in second reader is the same label parse.

Tests

  • tests/test_provenance.py (13):
    • Intake keeps the line.
    • Agreement lets the save go; a misread holds it; the same method twice holds; a missing document holds.
    • The round trip holds for a faithful record and refutes a correctly saved wrong entry.
    • verify-run end to end on MockMed: a misread saved faithfully gives delta: held, source: refuted, so the run is not verified.
    • The guard holds a disagreement for a person even when not gated.
    • A task whose values differ from its provenance is refused.
  • 128 passed locally across provenance, guarded save, verify-run, frame, fault gate, intake and agent guard. ruff; mypy clean on the changed modules.

🤖 Generated with Claude Code

abrichr and others added 3 commits October 20, 2026 13:27
A record check confirms that the store holds the values it was given. A
value misread from the referral and saved faithfully passes every such
check. Now each value can name its source, a second reading by another
method must agree before a guarded save, and a round trip compares the
record as read back with the source document.

- openadapt_flow/provenance.py: openadapt.param-provenance/v1 (document
  line and digest, worklist cell, or the instruction), cross_check (a
  second, independent extraction; same method twice holds), round_trip
  (read-back vs the source, never vs the params), LabelExtractor and
  CallableExtractor.
- intake writes params.provenance.json beside params.json.
- verify-run --provenance/--source: checked.source is held, refuted or
  not_checked by the round trip; 'instruction' without provenance.
  evidence.source names fields and extractors, never values.
- guarded save: a task may carry provenance and sources; a value the
  second reading does not confirm sends the save to a person under any
  approval mode; confirm runs the round trip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from verification-coverage/fault-gate to main October 20, 2026 23:29
@abrichr
abrichr added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit 3d385d2 Oct 21, 2026
39 checks passed
@abrichr
abrichr deleted the verification-coverage/provenance branch October 21, 2026 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant