Skip to content

flow: keep the server's answer to a save, labelled same_screen - #1298

Merged
abrichr merged 12 commits into
mainfrom
flow/save-response-evidence
Oct 20, 2026
Merged

abrichr merged 12 commits into
mainfrom
flow/save-response-evidence

Conversation

@abrichr

@abrichr abrichr commented Oct 18, 2026

Copy link
Copy Markdown
Member

What this does

In a browser, a save usually sends a write request and the server answers it. That answer is now kept as evidence, labelled honestly.

  • PlaywrightBackend logs answers to POST/PUT/PATCH/DELETE (backends/_server_responses.py).
  • The replayer marks the log before a save step's action and attaches what came back to the step: StepResult.server_responses (method, status, endpoint digest without query string, body digest and size, media type). No body, URL, header or value.
  • The receipt counts them: server_responses: {observed, ok, check_level: same_screen}. The production receipt projection is unchanged.

Why same_screen, never a check

The answer comes from the same transaction as the save. A server can say 201 Created and store the wrong value, store it under another record, or roll it back. So on its own it never confirms an effect and never makes a run VERIFIED; the configured verifier still decides. A 500 behind a "Saved" banner is still worth seeing.

Results and receipts without answers serialize byte-for-byte as before (fields omitted when empty).

Tests

tests/test_server_responses.py: only write answers kept, no URL/query/body text; a real Chromium save produces one; a 201 with an unconfirmed read-back still ends RECONCILIATION_REQUIRED, not billable; the receipt counts answers without changing its evidence tier. test_run_receipt.py allow-list gains server_responses (deliberate privacy-boundary edit: counts and a fixed label only). 275 related tests pass locally.

Docs: EFFECT_KIT.md, APPROVAL_AND_VERIFICATION.md (check-level section), claims.yaml + VERIFICATION.md.

Stacked on #1280. Part 4 of 4.

🤖 Generated with Claude Code

abrichr and others added 3 commits October 18, 2026 15:22
When the browser sees the server answer a save request (POST, PUT, PATCH,
DELETE), the replayer keeps the answer on the save step
(StepResult.server_responses) and the receipt counts it (server_responses:
observed, 2xx, check_level same_screen). Kept: method, status, a digest of
the endpoint (no query string), a digest and size of the body, the media
type. Never a body, URL, header or value.

The answer comes from the same transaction as the save, so on its own it
never confirms an effect and never makes a run VERIFIED; the configured
verifier still decides. Results and receipts without answers keep their
exact bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Keep only method, status and media type: no endpoint or body digests
  (a digest of a short path or body can be guessed).
- Never read a response body, so a streaming answer can't hold up a run.
- Count only the page's own fetch, XHR and form writes to its origin, sent
  after the save began; never beacons, other origins, requests already in
  flight, or service-worker answers.
- "Not an error" is a status below 400: a form save often redirects.
- Keep up to 64 answers per step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr

abrichr commented Oct 18, 2026

Copy link
Copy Markdown
Member Author

Full-diff review found 4 issues + 2 minor; fixed in the latest commit and pinned in tests/test_server_responses.py: possible hang on streaming bodies (bodies are never read now); brute-forceable endpoint/body digests (dropped: method, status, media type only); unrelated answers counted (only same-origin fetch/XHR/form writes sent after the mark; no beacons, in-flight requests or service-worker answers); redirects counted as failures (ok = status < 400); per-step cap raised to 64; caveat wording fixed.

abrichr and others added 3 commits October 18, 2026 17:03
…ventory

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	packages/flow/docs/VERIFICATION.md
#	packages/flow/docs/verification.json
#	packages/flow/openadapt_flow/receipt.py
#	packages/flow/public-artifacts.json
abrichr and others added 2 commits October 18, 2026 22:41
# Conflicts:
#	packages/flow/docs/VERIFICATION.md
#	packages/flow/docs/verification.json
#	packages/flow/public-artifacts.json
The page gets no request/response listeners until a save step marks the
log (nothing earlier is kept anyway), so a page that never saves keeps the
exact listener set it had, and the mask-tracking teardown check holds.
close() removes the listeners. Masking is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from flow/desktop-readback to main October 19, 2026 14:00
…idence

# Conflicts:
#	packages/flow/openadapt_flow/backends/playwright_backend.py
#	packages/flow/openadapt_flow/runtime/replayer.py
…idence

# Conflicts:
#	packages/flow/claims.yaml
#	packages/flow/docs/VERIFICATION.md
#	packages/flow/docs/verification.json
#	packages/flow/public-artifacts.json
…idence

# Conflicts:
#	packages/flow/claims.yaml
#	packages/flow/docs/EFFECT_KIT.md
#	packages/flow/docs/VERIFICATION.md
#	packages/flow/docs/verification.json
#	packages/flow/openadapt_flow/runtime/replayer.py
#	packages/flow/public-artifacts.json
…idence

# Conflicts:
#	packages/flow/claims.yaml
#	packages/flow/docs/VERIFICATION.md
#	packages/flow/docs/verification.json
#	packages/flow/openadapt_flow/ir.py
#	packages/flow/public-artifacts.json
@abrichr

abrichr commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

Included in batch #1384, which runs CI once for several reviewed PRs. This PR is marked merged when #1384 merges. Please don't push here unless asked; a new commit leaves it out of the batch.

@abrichr
abrichr merged commit e784cf8 into main Oct 20, 2026
40 checks passed
@abrichr
abrichr deleted the flow/save-response-evidence branch October 20, 2026 02:18
abrichr added a commit that referenced this pull request Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant