Repository navigation
provenance: compare the saved record with its source document - #1446
Merged
Merged
Conversation
A record check confirms that the store holds the values it was given. A value misread from the referral and saved faithfully passes every such check. Now each value can name its source, a second reading by another method must agree before a guarded save, and a round trip compares the record as read back with the source document. - openadapt_flow/provenance.py: openadapt.param-provenance/v1 (document line and digest, worklist cell, or the instruction), cross_check (a second, independent extraction; same method twice holds), round_trip (read-back vs the source, never vs the params), LabelExtractor and CallableExtractor. - intake writes params.provenance.json beside params.json. - verify-run --provenance/--source: checked.source is held, refuted or not_checked by the round trip; 'instruction' without provenance. evidence.source names fields and extractors, never values. - guarded save: a task may carry provenance and sources; a value the second reading does not confirm sends the save to a person under any approval mode; confirm runs the round trip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 3 of #1434 (
docs/VERIFICATION_COVERAGE.md). Stacked on #1445; the base is its branch.Problem
A record check confirms that the store holds the values it was given. If the referral says
02139and the extraction read02193, a faithful save of02193passes the delta and the frame. The error happened before the save, and the read-back can't see it.Change
openadapt_flow/provenance.py:openadapt.param-provenance/v1: each value names its source, which is one of: a document (the digest of its bytes, plus the line and label), a worklist cell, or the instruction itself.cross_check: a second extraction by a different method compares value by value, using the screen read-back's kind rules (text, code, date, currency), read exactly. A disagreement, a missing value, a missing document, or the same method twice holds the item.round_trip: compares the record as read back with the second reading of the cited source. It never compares with the params.LabelExtractor(the deterministic intake label parse) andCallableExtractor(a second model or OCR, supplied by the caller). No model calls here.params.provenance.jsonbesideparams.json. Intake already found the line for every field and then dropped it.--provenance FILE --source DOC:checked.sourcebecomesheld,refutedornot_checkedfrom the round trip. Without provenance it staysinstruction.evidence.sourcenames the fields and extractors, never the values.provenanceandsources.beginruns the second reading. A value it doesn't confirm sends the save to a person under any approval mode, and the approval request names the fields.confirmruns the round trip.VERIFY_RUN.md,GUARDED_SAVE.md,INTAKE.md; aCAPABILITIES.mdrow.Safety review
This adds refusals and holds only: a refuted round trip blocks
verified, and a disagreement forces a person's approval. Without provenance nothing changes (source: instruction). It touches whenverifiedis issued, so it waits for the coordinator with #1443 and #1445.Not covered
A source document that is itself wrong, and a value both methods misread the same way. Values from
intake:localhold in the guard until a person confirms them, because the only built-in second reader is the same label parse.Tests
tests/test_provenance.py(13):delta: held, source: refuted, so the run is not verified.🤖 Generated with Claude Code