feat: Goal 13 low-latency turn delta sync — trigger journal, twdelta1 watermark, turn.delta RPC (closes #9)

[jerryfane]

Summary

Implements Goal 13: Low-latency turn delta sync (docs/goals/post-release/13-low-latency-turn-delta-sync.md, status urgent) — the cache-only, host-scoped change feed that replaces full turn-list traversal on every herdres sync. Closes #9 (coordination issue).

Design (adjudicated by a two-model panel against your spec)

• Capture: trigger-populated append-only turn_change_journal (v18 migration) — AFTER INSERT/UPDATE/DELETE on turns (op classified by superseded_at: tombstones and deletes emit remove, never upsert) plus a current-revision trigger on turn_content_revisions. Triggers were chosen over writer instrumentation because they are bypass-proof by construction — no present or future Python writer can skip the feed. Journal rows carry no payload and no private identifiers.
• Watermark: twdelta1.* durable token + twdeltac1.* short-lived page cursor, mirroring the existing twsince1/twlist1 HMAC-digest machinery, additionally binding a store epoch (created with the journal) so a rebuilt database invalidates old watermarks instead of silently replaying wrong sequences. Floor-based expiry with bounded, configurable compaction (age + per-host row count).
• Read (turn.delta RPC + tendwire turn delta CLI): one read transaction — validate, freeze the batch ceiling, collapse per turn by MAX(seq), rebuild projections at read time through the same sanitizers as the list read (no canonical content — Goal 05 stays authoritative; oversized projections degrade to descriptor-only so every page fits the 1 MiB frame). Explicit outcomes: invalid/expired/cross-host watermark, invalid/expired cursor, incompatible_schema, bootstrap_too_large, store_unavailable. Resumable bootstrap, checkpoint only on the final batch page, size gate independent of client page size. Nothing in the delta path can authorize a final send — Goal 10 outbox remains the only delivery authority.

Acceptance coverage (tendwire side)

Tests map to the spec's criteria: genuine 10,000-turn seeded bootstrap with every frame verified under 1 MiB; two forced unchanged polls proven to read zero list/content pages via spies on the real readers; single Working mutation → exactly one upsert; tombstone → exactly one remove with no duplicate after later physical retention deletion; capture verified through the real writers (refresh, merge, command claims, sweeper, retention); genuinely interleaved concurrent pagination; token failure matrix incl. store-epoch rebuild; migration chain parametrized from every prior version; warmed 51-sample p95 ≤ 350 ms through the production Unix socket. Full suite: 2,857 passed.

Review process

Same pipeline as #8: two-model design panel → implementation → multi-agent adversarial review (one finding reproduced live) → fix round → mutation-tested verification of every fix. Documented residuals (non-blocking): a pathological oversized descriptor could still exceed the page budget via the first-item path; journal_rows_scanned reports the whole batch range on each page (consumers must not sum across pages); the zero-pages spy exercises the store function rather than daemon dispatch.

Also included

Your latest main (9904056 exact-content-only claim reconciliation — which deletes the fallback tier our Phase-1 review had flagged; we agree with the stricter direction — and 16fb7dc) is merged in; suite green on the combined tree.

Not in this PR: the herdres consumer (watermark persistence, fast/slow lane split) — that lands in the herdres repo next, per the goal doc's Herdres-side requirements.

🤖 Generated with Claude Code