Skip to content

fix(server): history trim stops on activity and publishes what it removed - #264

Open
jayleaton wants to merge 3 commits into
mainfrom
fix/thread-trim-followup-20261117
Open

jayleaton wants to merge 3 commits into
mainfrom
fix/thread-trim-followup-20261117

Conversation

@jayleaton

@jayleaton jayleaton commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #259, covering Randy's post-merge review (#259 (comment)) and his review of this PR. A history trim checked eligibility once per thread, so a thread opened mid-pass still lost rows. Its deletes bypassed the event log: the snapshot sequence stayed the same for the thread and its forks, live clients and the timeline and search caches kept stale rows, and the preview counted ancestors the delete keeps.

Fix

  • New domain event thread.history-trimmed names the source thread and the turn items and nodes removed from it. The source's projection deletes those rows when it applies the event, in the same transaction that appends it. Every live fork of the source (at any fork depth; side chats inherit nothing) gets its own durable copy on its own stream. Each affected thread's sequence advances, so the timeline index and ThreadFind refresh, live web and mobile clients apply the event in client-runtime, and a reconnect replays it. Older clients skip the unknown event type. Trim events don't count as thread activity, so threads keep their place in lists sorted by activity.
  • The "older detail was trimmed" notice is positioned inside the page transaction and carried in the forks' copies. A live fork that shows the notice's run places it where a fresh snapshot does.
  • EventSink.writeBuilt builds a write's events inside the commit transaction. Each page of up to 200 rows rechecks the current setting and the thread's idle eligibility there. A thread that is opened, messaged, run, pinned, snoozed or PR-linked stops its trim; setting trimming to 0 stops the whole pass.
  • Rows are deleted children first. Each page also rechecks references inside its transaction: a row stays if anything outside the page still hangs off it, whether a child item, a child node, an item on a node, or another table's reference to a node. A pass that stops early never orphans a row.
  • Preview and delete share one selection, including kept ancestors and node references, so the counts match.

Tests (ThreadHistoryTrim.test.ts, orchestrationV2Projection.test.ts)

  • Activity during a pass: one table row per kind of activity, run by a hook after the pass has chosen its rows and before it deletes them. All rows fail on main.
  • A pass stopped between pages, with a reasoning child of a removable command placed on a later page: no orphan is left. At the previous head (6940c4a) the child was left pointing at a deleted command.
  • A row that gains a child after the pass chose it is kept. This fails with the in-transaction recheck removed.
  • Reconnect, live replay, paging and search across a pass, for the thread and for an active fork: the sequence advances, replaying from the mid-pass snapshot matches a fresh one, and windows and search report the new sequence. At 6940c4a the fork's sequence stayed at 440.
  • The preview item count matches the delete (main reported one more).
  • Inline outputs (send_to_user, HTML renders, MCP apps, delegate_task) survive a trim. They stay in the snapshot timeline (which transcript export uses), the wire snapshot web and mobile receive, and the rows thread transfer reads. These pass on main too; they guard against regressions.
  • The hooked cases share one migrated database. Each case resets the trim setting and clears what an earlier case left. The file runs in about 26 s, down from about 103 s.

The 30-day default and the opt-in behaviour are unchanged.

vp run check:changed passes: format, lint, knip, typecheck of 37 packages, and the tests beside the changes.

Done by Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

jayleaton and others added 2 commits November 17, 2026 18:22
…oved

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted Author is trusted by repo permissions or the VOUCHED list. size:L labels Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 21.5 KiB 21.5 KiB 0 B (0.0%) 24.4 KiB ✅
Codex Live turn messages 1 1 0 (0.0%) 8 ✅
Claude Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 21.9 KiB 21.9 KiB 0 B (0.0%) 24.4 KiB ✅
Claude Live turn messages 1 1 0 (0.0%) 8 ✅

Other scenarios

All 64 scenario measurements match main.

All 64 scenario measurements
Provider Scenario Metric Main baseline This PR Impact PR ceiling
Codex short turn resume WebSocket decoded 10.5 KiB 10.5 KiB 0 B (0.0%) 12.7 KiB ✅
Codex short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn running, cold open HTTP decoded 281.7 KiB 281.7 KiB 0 B (0.0%) 302.7 KiB ✅
Codex long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Codex long turn running, history page HTTP decoded 92.4 KiB 92.4 KiB 0 B (0.0%) 107.4 KiB ✅
Codex long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Codex long turn finished, cold open HTTP decoded 381.5 KiB 381.5 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, socket cold open WebSocket decoded 381.6 KiB 381.6 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, work page HTTP decoded 186.2 KiB 186.2 KiB 0 B (0.0%) 200.2 KiB ✅
Codex long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Codex idle thread resume WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell snapshot HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Codex fleet shell subscribe WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Codex fleet shell resume check rows 0 0 0 0 ✅
Codex stale shell reload HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex stale shell reload rows 27 27 0 (0.0%) 27 ✅
Codex stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Codex stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex stale todo list cold open HTTP decoded 72.4 KiB 72.4 KiB 0 B (0.0%) 83.0 KiB ✅
Codex stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Codex stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅
Claude short turn resume WebSocket decoded 10.6 KiB 10.6 KiB 0 B (0.0%) 12.7 KiB ✅
Claude short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn running, cold open HTTP decoded 283.3 KiB 283.3 KiB 0 B (0.0%) 302.7 KiB ✅
Claude long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Claude long turn running, history page HTTP decoded 93.4 KiB 93.4 KiB 0 B (0.0%) 107.4 KiB ✅
Claude long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Claude long turn finished, cold open HTTP decoded 384.2 KiB 384.2 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, socket cold open WebSocket decoded 384.3 KiB 384.3 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, work page HTTP decoded 188.1 KiB 188.1 KiB 0 B (0.0%) 200.2 KiB ✅
Claude long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Claude idle thread resume WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell snapshot HTTP decoded 43.0 KiB 43.0 KiB 0 B (0.0%) 53.7 KiB ✅
Claude fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Claude fleet shell subscribe WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Claude fleet shell resume check rows 0 0 0 0 ✅
Claude stale shell reload HTTP decoded 43.0 KiB 43.0 KiB 0 B (0.0%) 53.7 KiB ✅
Claude stale shell reload rows 27 27 0 (0.0%) 27 ✅
Claude stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Claude stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude stale todo list cold open HTTP decoded 73.4 KiB 73.4 KiB 0 B (0.0%) 83.0 KiB ✅
Claude stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Claude stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅

Baseline: de3a810 · PR result: 5363779 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@jayleaton

Copy link
Copy Markdown
Owner Author

Randy reviewed head 6940c4a. The fixes from the #259 review are verified:

  • Each page now checks eligibility and the setting inside a BEGIN IMMEDIATE transaction.
  • Trims on the thread itself publish a revision, refresh the caches and replay correctly.
  • The preview and the delete now use the same selection.
  • The inline-output retention tests cover the timeline, the wire snapshot and transfer.

117 targeted tests passed, and so did the 7 package typechecks and changed-file lint.

Still needed:

  1. Blocking: ThreadHistoryTrim.ts:423. An interrupted pass can delete an ancestor of a row that's kept. To reproduce: 250 reasoning rows, plus a reasoning child of a removable command. Opening the thread before page two stops the trim, but the child is left pointing at a command that's already deleted. Fix: delete children before parents and recheck references inside each page transaction. Extend the interrupted-page test with a dependency that spans pages.
  2. Blocking: ThreadHistoryTrim.ts:398. Forks still change without a durable revision. An active fork's snapshot went from 11 visible rows to 10 while its sequence stayed at 17, and a replay after 17 returned no events. The gap survives a reconnect. Fix: either give affected forks a durable invalidation that forces a fresh snapshot, or send trim events that name their source through the forks' streams. Cover live updates, reconnect and paging.
  3. Nit: ThreadHistoryTrim.test.ts:622. The fixtures are slow. Each activity case takes 8.9–11.1 s, and the file takes 103.5 s. Reuse one migrated database fixture and reset the settings between cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added size:XL and removed size:L labels Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL vouch:trusted Author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant