Skip to content

Batch thinking deltas like text deltas while streaming #155

Closed

Description

@GoDiao

Part of #153.

Problem

Text and tool-input deltas are buffered for 50ms before they update the store, but thinking deltas are not. Each thinking event updates the store at once (desktop/src/stores/chatStore.ts:1309-1328). That creates a new messages array, which:

  • makes MessageList rebuild its whole render model (buildRenderModel, MessageList.tsx:1513);
  • re-renders every tool group through a new toolResultMap.

With fast thinking output this happens dozens of times per second.

What to do

Buffer thinking deltas the same way as text deltas: the 50ms buffer around pendingDeltaBySession (chatStore.ts:272-301, flushed at :1265), or one rAF.

Two ordering rules:

  • Flush pending thinking before the first text delta, a tool call, or turn end, so the order of thinking and text stays correct.
  • Flush on session switch and disconnect, the same way text is flushed today (chatStore.ts:768, :927).

Done when

  • A test shows that several thinking deltas inside one window produce one store update, and that thinking still appears before the text that follows it.
  • The PR includes a before/after React Profiler capture, or a render count, for a reply with a long thinking block.
  • Desktop tests and typecheck pass.

Activity

  1. GoDiao commented on Nov 14, 2026

    @GoDiao
    OwnerAuthor

    #160 was closed after it went quiet, so this is open again. The approach there was right; its review lists what a PR needs on top of it:

    • flush pending thinking at the top of content_start for every block type, not at the first text delta;
    • flush it in the user_message case too, so a message from another client doesn't land before the thinking;
    • an ordering test that sends the real sequence (thinking → content_start text → delta) and checks chatState and activeThinkingId;
    • no comments in new code.

    Comment here if you'd like to take it.

  2. iruzen-dono commented on Nov 16, 2026

    @iruzen-dono
    Contributor

    Taking this one — #258 is open and covers the four points from the #160 review: flush at the top of content_start for every block type, the real ordering sequence with chatState / activeThinkingId assertions, the remote user_message flush, and no comments in the new code. The mirror idle path now folds text before thinking, matching flushPendingThinking.

    Measured on Windows: 10 thinking deltas in one window produced 10 store updates on main, 1 with the branch. chatStore.test.ts 104 pass, lint clean, and the full desktop suite is at 3017 pass / 5 fail — those same 5 fail by name with the source reverted, so they are the pre-existing Windows set from #255, not this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions