[abrichr]

Stacked on #1678 (shadow mode). Large: about 2,700 lines of its own, 1,230 of them tests.

The problem

Many apps write to their own database on every request: the acting user's last_login, an API token's last_seen, a first-login property row for the acting user. The whole-database frame sees those rows move inside every write's window, nothing in the contract explains them, and it refutes every write, correct or not. In the held-out retest, three of four apps had every correct write refuted this way.

The change: learn, then enforce

openadapt-flow learn (also python -m openadapt_flow.learn and openadapt learn):

• collect: runs shadow mode (SQL) with a hook on Shadow.on_checked. Each verifier call adds one line to episodes.jsonl: the kind, the outcome, which parts held, and the frame's moved rows by table and column name. No record values.
• propose: writes proposal.yaml and a readable proposal.md with a digest.
• approve: one command by a person writes the excuses into the contracts as frame.learned, recording the proposal's digest, who approved and when. All contracts of a kind or none. An edited proposal is refused.

Fail closed at every step:

1. Only checks whose declared effects held and whose frame was refuted by database rows alone teach. Every other check is ignored and counted by reason in the proposal.
2. A moved row can be excused only as target (a row the write declares or adds) or actor_row (the users table's row keyed by the actor's id, or a row whose user column holds it; the user columns come from the schema's foreign keys or a person's --user-column). Any other row is listed as "stays a stop", however often it moves.
3. An excuse is {table, columns, change: changed|added, relation, actor_column}. Exactly the columns seen to move. Never a whole table, a wildcard, any row, or a removal. Proposed only when seen in at least K of every N learnable checks of that kind (defaults 3 of 5; each column must meet the same bar). Why: five is the smallest sample in which a majority still allows two checks where a throttled column (a last-used time written at most once a minute) didn't move, and a one-off isn't proposed. The count isn't the safety boundary; the relation is.
4. Enforcement (frame.learned in runtime/effects/learned.py and frame.py): at each check and recheck the relation is derived again from this run's actor and declared rows. A target excuse applies only to rows this write declares; an actor_row excuse only when the actor column holds this run's actor id before and after; the moved columns must all be in the excuse. A row in a table over max_rows, whose baseline didn't keep every column, is never excused. Anything else is collateral, as before. The receipt names the excuses used and the proposal digest. Contracts without the key keep their digests.

The verifier now also reports, for a refuted frame, each collateral row by names only (evidence.frame.moves: table, change, columns, whether it's the write's own row, which columns hold the actor's id). The learner reads only that.

Tests

tests/test_learned_excuses.py, on SQLite, and on MariaDB and PostgreSQL with OPENADAPT_SNAPSHOT_MYSQL / OPENADAPT_SNAPSHOT_POSTGRES:

• before/after on a real write: refuted by the bookkeeping, confirmed once learned;
• a learned actor_row excuse never covers another user's row or token, another column of the user's row, a row moved to the actor, a removal, a row added for another user, an added row of a kind only changes were learned for, a column a large table didn't compare, or the same moves under another run's actor;
• a learned target excuse never covers a look-alike row, another column of the target, or the excused column on a second row (the retest's fail-open, where a column excused for the target was excused on every row);
• learn, propose and approve end to end on real writes, user columns from the live schema's foreign keys on MariaDB and PostgreSQL;
• the learner's rules: rows that are neither target nor actor stay a stop when seen in every check, K of N, columns under the bar left out, every ignored check counted, an edited proposal refused, approval all-or-nothing with --reject;
• collecting through the real shadow proxy's hook;
• the CLI.

A mutation check (scratch, not committed): letting any row count as the actor's fails 6 tests; letting any column count fails 2; letting a row count as the target fails 2.

Real-app evidence

Running.

Docs

packages/flow/docs/LEARNED_EXCUSES.md (new), pointers in VERIFY_RUN.md and SHADOW_MODE.md, the command map and CAPABILITIES.

Note on the diff

This branch merges main into #1678's branch to get the PostgreSQL frame (#1673), so until #1678 is updated from main the diff against it also shows #1672 and #1673.

[abrichr]

Adversarial review (round 1, independent): BLOCKED

Head 68ecef3. CI: green (ci and packages-gate pass; 24 success, 20 skipped by path filter, CodeQL neutral).

I read learn.py, runtime/effects/learned.py, the frame.py and verify_run.py changes, and the tests. I ran tests/test_learned_excuses.py on SQLite, MariaDB 11.4 and PostgreSQL 16 in throwaway capped containers: 70 passed, none skipped. The existing frame tests (tests/test_frame_*.py, SQLite) also pass: 80 passed, 4 skipped. Then I ran my own attack tests on all three engines (scratch, not pushed).

The enforcement side holds up. Each check re-derives the relation from this run's actor and declared rows. I could not make an actor_row excuse cover another user's row, a NULL user column, a removal, another column, a row moved to the actor, or a token the actor issued for someone else. I could not make a target excuse cover a look-alike row. The learner side has one way to end with a wrong write CONFIRMED.

BLOCKER

1. The agent's own systematic wrong write on the target is learned as a target excuse. Files: learn.py relation_of (if move.get("target"): return "target", ""), and frame.py where the stray columns of a declared row go to covers(..., relation="target").

The PR's guard against poisoning is "only checks whose declared effects held teach". A write that sets the declared field correctly, and also changes an undeclared column of the same record, passes that guard. Shadow-mode learning watches the agent's own traffic, so a consistent agent mistake looks the same as the app's own bookkeeping.

Repro, on SQLite, MariaDB and PostgreSQL, using the PR's asset_contract, which says target_fields: [name]:

• Five learning writes each rename asset LT-100 (correct) and also change assigned_to (wrong, undeclared). The delta holds and the frame is refuted by rows only, so all five teach.
• propose gives | E1 | asset | assigned_to, modified_at | changed | the write's own row | 5 of 5 |.
• approve with the defaults writes it, with no --reject.
• A later wrong write (name='Laptop Z', assigned_to=3) ends CONFIRMED, with used: ['learned:asset:changed:target:assigned_to,modified_at'].

A target excuse widens what the declared write may change on its own record, which is the record check itself. The one-step, approve-everything default and the row label "the write's own row" make a column like this easy to wave through. Possible fixes (the author's choice):

• Don't approve target excuses in bulk. Require each one by id (for example --accept E1).
• In proposal.md, flag every target excuse with a warning that a column the agent's own write sets looks the same, so approve only columns the app sets itself, such as timestamps and versions.
• Add the case to Limits in LEARNED_EXCUSES.md.
• Optionally, refuse a target column that the contract's selector or record fields name.

The same mechanism can teach an actor_row excuse from the agent's systematic edits to its own rows. The harm there is limited to the actor's own rows, so I rate that part SHOULD-FIX and cover it with the same warning.

SHOULD-FIX

2. An added actor_row excuse has no row count. File: learned.py covers (an added row is checked for its columns and actor columns only). With the first-login excuse learned (user_property added {property,user_id,value}), a correct note write plus 20 extra user_property rows for the actor, each with any property (for example auth.mfa_required.N = off), ends CONFIRMED on all three engines. The docs cover "any values", but not "any number". The learner can record the most rows per check it saw for each added excuse (one, for a first-login row), and covers can refuse more. This is an exact rule, not a fuzzy one. It also limits the "second token for itself" case.

3. The digest catches accidents, not edits. File: learn.py load_proposal. The digest is an unkeyed sha256 over the body, so an edited proposal with its digest recomputed (learn.proposal_digest) loads and approves. My test added an app_user {email,last_login} excuse that was never seen. At verify time frame.learned.proposal_sha256 is only a format check. That is acceptable, because contracts are trusted config, but LEARNED_EXCUSES.md says "If anyone edited it, its digest no longer matches". Reword it to say the digest catches accidental edits and records provenance, not tampering.

NIT

4. One excuse can allow a column combination that was never seen. Each column must pass the K-of-N bar on its own, so a seen in 3 of 5 and b in 3 of 5 (together once) gives one excuse that lets both move at once. This is harmless as long as the columns are bookkeeping. You could say so in the docs.

5. Shared accounts. The actor's id stands for every session of that app user, so another person's session under a shared login is excused within the learned columns. One line in Limits would cover it.

6. approve is not atomic on disk. It writes the files in sequence, so an I/O error partway through leaves some contracts updated. This is rare. You could write to temp files and then rename them.

Checked and held (all three engines unless noted)

• actor_row:
  • stops a token with user_id = actor and issued_by NULL under a two-column excuse (REFUTED);
  • no non-user column can become an actor column: the candidates are the schema's foreign keys and person-confirmed columns only;
  • canonical keeps 2 and "2" equal, and NULL never matches;
  • the excuse needs every learned actor column on both sides.
• Removals, unnamed columns, rows over max_rows, an actor column that changed, and a moved-to-actor row are never excused.
• The target excuse is limited to declared rows, so a look-alike stays collateral.
• K of N: need = ceil(K·total/N) with total ≥ N. Every ignored episode is counted. Truncated moves (over 50) don't teach.
• --reject of an unknown id is refused. Approval of a kind writes every one of its contract files, or none when one file is invalid.
• Recheck: the actor's later per-request write is excused (CONFIRMED). target excuses don't apply at recheck (no declared rows), which fails closed.
• A contract without frame.learned keeps its digest. The only change it sees is the added evidence.frame.moves on a refuted frame, and the existing frame tests pass.

To clear the block: fix finding 1 (per-excuse opt-in or an explicit warning for target excuses, plus a test where a learned stray column is rejected or must be accepted by id). Findings 2 and 3 can go in a follow-up.

[abrichr]

Adversarial review (round 2, independent): BLOCKED

Head 829f290e0 (the rebase on main; learn.py, learned.py, the frame.py change, the tests and LEARNED_EXCUSES.md are identical to a83834420). CI: green on 829f290e0 (24 pass, including ci and the flow core unit and critical shards; 21 skipped by path filter; none failing).

I read learn.py, runtime/effects/learned.py, the frame.py and verify_run.py changes, the docs and the tests. I ran them on SQLite, MySQL 5.7 and PostgreSQL (local throwaway servers): tests/test_learned_excuses.py 86 passed, none skipped. The existing tests/test_frame_*.py pass on SQLite (198). Then I ran 35 attack tests of my own on all three engines (scratch, not pushed).

Most of round 1's fixes hold. The learner no longer proposes a target excuse, a relation: target entry is refused wherever a contract is loaded, an excuse is one exact column set, and added excuses carry max_rows. One path still learns the written row.

BLOCKER

1. The written row is learned as an actor_row excuse when another actor also wrote it. File: runtime/effects/frame.py, _moves:

"target": bool(c.on_target and not c.other_on_target),

A declared row that another actor also changed in the window is collateral with on_target=True, other_on_target=True. _moves reports it as target: false, so learn.relation_of doesn't stop it as "the written row". It goes on to the actor check, and when the record has a user column that holds the actor's id (an owner or creator column), it is proposed as actor_row. proposal.md then labels it "the acting user's row (owner_id)".

At enforcement the excuse never applies to its source, because declared rows never reach learned_actor_row. It only widens what the actor's other rows may do.

Repro, on SQLite, MySQL and PostgreSQL. The asset table from the PR's tests, plus owner_id (a confirmed user column, 2 on every asset) and modified_by (record_field attribution, server_set):

• Five learning writes. Each sets LT-100's name to the value it already holds, so the delta holds and name isn't among the moved columns. Then an app worker under account 9 sets assigned_to and modified_by on the same row, and the app does its usual per-request bookkeeping.
• Each episode's move is {table: asset, columns: [assigned_to, modified_by], target: false, attribution: other, actor_columns: [owner_id]}.
• propose gives asset changed {assigned_to, modified_by}, actor_columns: [owner_id], alongside the three real bookkeeping excuses. The default approve writes all four.
• A later wrong write renames LT-100 correctly and also reassigns a different asset the actor owns (assigned_to=3, modified_by=2, self-attributed). With only the bookkeeping excuses, this is REFUTED. With the approved proposal, it is CONFIRMED, used: [..., 'learned:asset:changed:actor_row:owner_id:assigned_to,modified_by', ...].

The setup is narrow: the declared field must already hold its value (an idempotent or ensure-style write), and something else must write the record in the window. Neither is malicious, though. An app's auto-assignment rule under a service account is enough. It also breaks the stated rule that the written row is never learned, which was round 1's blocker.

Fix: "target": bool(c.on_target). With that one-line change, the same run proposes only the three bookkeeping excuses, the wrong write is REFUTED, and the PR's own tests still pass (SQLite). Please add the case as a test.

SHOULD-FIX

2. The "a column the agent sends" rule reads request bodies only. File: learn.py record_bodies. Query-string names aren't recorded. Requests of a method outside WRITE_METHODS aren't observed at all, and a body-less write still counts as "recorded". Repro through the real shadow proxy (SQLite): the agent posts /api/notes?email=bot@x.test with a JSON body {title, body}, and the app stores email on the actor's own user row. sent_fields is [body, title]. The proposal is app_user changed {email, last_login} (actor_row), even though the agent sent a field named exactly email. Add the query parameter names of every request through the shadow, of any method, to sent_fields. You could also mark a kind unrecorded when a write in the window had no parsable body.

3. proposal.md gives no warning on actor_row excuses. The Limits section says "it still lets through a column the agent sets under a field of another name", but the page a person approves from doesn't say it. Repro on all three engines: the agent's note request carries contact_address, and the app writes it to the actor's email. That gives app_user changed {email, last_login}, and after the default approval a write that sets the actor's email to attacker@evil.test is CONFIRMED. The exact column set limits this to the actor's own row and to exactly those columns, so I don't block on it. Still, a one-line note per excuse table ("approve only columns the app sets itself, such as times and counters") is cheap. Round 1 asked for the same thing on target.

NIT

4. approve with a recomputed digest accepts any actor_columns (my test: actor_columns: [name]) and any max_rows. The docs now say the contracts are trusted config, so I'm only noting it. approve could check actor_columns against the proposal's users block.

5. An added excuse's max_rows counts per check, and a recheck window can span several requests. On an app that adds a row on every request, a recheck then refutes. That fails closed. One line in Limits would cover it.

Checked and held (SQLite, MySQL, PostgreSQL)

• actor_row, a second row for the actor. A user_property row the agent adds for itself, within max_rows, is CONFIRMED, as documented. These stay REFUTED:
  • a new api_token for the actor (no added excuse);
  • a property row with a NULL value (a different column set);
  • the actor's rows moved to user 3.
• Shared login. Another session under the actor's account moving last_login is CONFIRMED, as documented. An actor id of 02 or 2 matches nothing (REFUTED).
• A user column that happens to hold the actor's id. With an asset changed {modified_at} excuse on a confirmed assigned_to, another asset assigned to the actor may bump modified_at (the documented "creator" limit). These stay REFUTED:
  • that asset renamed as well;
  • that asset reassigned away;
  • a stray modified_at on the target when the target is also assigned to the actor.
• Recheck. The actor's later last_login is CONFIRMED. These are REFUTED: another user's last_login, two added property rows, the actor's email changed with last_login, and an added token. A recheck under another actor is UNCERTAIN.
• Edits to a proposal. An edit without a new digest is refused. A relation: target excuse injected with a recomputed digest is refused at approve, which validates the contract. Rejecting it with --reject approves the rest. columns given as a string is refused.
• relation: target (and Target) is refused at load through Contract.from_dict, Contract.load, verify_run.load_verify_config, verify_run.parse_verify_config and FrameSpec.model_validate, alone or mixed with a valid excuse. Every verify-run and verifier entry point goes through VerifyConfig.model_validate. Qualify drops frame entirely, so it applies no excuse. Nothing builds FrameSpec or LearnedFrame with model_construct.

To clear the block, fix finding 1 and add a test for it. Findings 2 and 3 can go in a follow-up.

[abrichr]

Adversarial review (round 3, independent): APPROVE

Head 5f130132f. I reviewed only the change since round 2's head 829f290e0. The rest of the rebase is unrelated main work (the Windows and macOS agents, and their CAPABILITIES row). The PR's own delta is commit 5f130132f, which changes frame.py _moves, learn.py (query_fields, record_bodies, render_markdown), LEARNED_EXCUSES.md and the tests.

CI on 5f130132f is green: ci, flow core lint, unit shards 1 to 4, critical and agent frameworks all pass. None fail, and the rest are skipped by path filter.

I reran round 2's attack tests against local throwaway PostgreSQL 17 and MySQL 5.7 servers (temporary data directories, since removed) and SQLite, together with tests/test_learned_excuses.py, plus 3 new probes on SQLite: 130 passed, none skipped. tests/test_frame_*.py and tests/test_shadow*.py also pass on SQLite (230).

Round 2 items

1. Blocker 1 (the written row learned when another actor also wrote it) is closed. _moves now sets "target": bool(c.on_target). On all three engines, round 2's repro now gives target: true for each learning episode. The asset {assigned_to, modified_by} move stays a stop ("the written row"), and the proposal holds only the three bookkeeping excuses. The later wrong write (rename plus a self-attributed reassignment of another asset the actor owns) is REFUTED after the default approval. Before this fix it was CONFIRMED.

   • Nothing else is widened. The flag is read in one place, learn.relation_of. FrameVerdict.moves reaches evidence.frame.moves and the learner's episode lines. Shadow receipts copy only database, violations and moved. Enforcement (learned.py) and recheck never read moves or target. Rechecks don't call on_checked, so they don't feed learning. The change can only turn a proposal into a stop. Frame and shadow tests are unchanged and pass.

2. Query-string names are now covered. record_bodies also wraps _forward, so every request through the shadow, of any method, adds its query parameter names. _check adds them to the episode before it is checked. Results:

   • Round 2's repro (POST /api/notes?email=...): email is now in sent_fields, and no excuse names email.
   • New probe, a GET /api/profile?email=... after the write and before the check that sets the actor's email: no excuse is proposed.
   • New probe, a body-less DELETE ?email=... in the window: email isn't excused. Only app_user {last_login} is proposed.
   • New probe, a GET before the write: outside the window, as expected.

   The name set is global and only grows, so it errs toward stopping.

3. The proposal warnings are in place. proposal.md now has a "Warning, for each excuse" list naming each excuse's columns and the "a field of another name isn't caught" limit. Round 2's A1 (contact_address stored as email) still learns app_user {email, last_login} and, once approved, still confirms the email write. That is the documented limit, now stated on the page a person approves from. Same classification as round 2: not a blocker.

New findings

SHOULD-FIX (follow-up). Bracket-style names aren't matched to their column. _names splits only on ., so a query or form field note[email] records note[email] and never email. The form-body half of this predates the PR. The query half comes with the new code. On apps that post nested forms in a[b] style, an email set this way can still be proposed on the actor's row, with the new warning shown. A fix would also split on [/] as a declared exact rule.

NIT. record_bodies._check copies the query names before calling the real check. A GET that arrives during the check's own reads isn't counted for that episode. Checks run only after quiet_s of idle time, so this needs a request in that short gap. It could be closed by adding the names in the collector, after the check returns.

No blockers.