Skip to content

perf(flow-chat): optimize long-round streaming projection and scroll rendering - #3778

Open
Raing5Days wants to merge 1 commit into
GCWing:mainfrom
Raing5Days:perf/long-round-scroll-and-render
Open

Raing5Days wants to merge 1 commit into
GCWing:mainfrom
Raing5Days:perf/long-round-scroll-and-render

Conversation

@Raing5Days

Copy link
Copy Markdown
Contributor

Summary

Optimize Web UI flow_chat session scrolling and streaming projection performance during long multi-round turns (10 -> 300+ ModelRounds), and add a categorized benchmark harness (src/web-ui/src/flow_chat/perf/longRoundPerfHarness*) covering the 5 frontend projection/rendering stages.

Key changes:

  • Store & adjacent group projection (roundGroups.ts, lifecycle.ts, groupProjection.ts, modernFlowChatStore.ts):
    • Cache getProjectedModelRoundGroups, getModelRoundFlowGroups, getCachedInlineFlowGroupData, flowGroupLifecycle, and mergeFlowGroupRun via WeakMap so settled ModelRounds do not recompute buildFlowItemGroups or computeExploreStats on every streaming flush.
    • Add prefix-aware turn projection (projectTurnItemsWithPrefixCache, canStartFreshFlowGroupRunAt, reconcileTurnVirtualItems, reconcileProjectedGroupsInPlace) so unchanged settled rounds before the last hard group boundary skip projectAdjacentFlowGroups and preserve exact === VirtualItem and FlowGroupData object identities (97.7% settled VirtualItem reuse, 94.6% settled TimelineBlock reuse).
  • Container group indexing, receive feedback & search (selectors.ts, receiveFeedback.ts, useFlowChatSearch.ts):
    • Cache per-item flow groups (getVirtualItemFlowGroups) and incrementally reuse prefix maps in indexFlowGroups.
    • Replace JSON.stringify([turnId, itemId]) allocations in receiveFeedback.ts with cached per-item keys, skip completed turns in captureFlowGroupFeedbackSnapshot, and short-circuit collectFlowGroupReceiveFeedback when previous.activeMembers.size === 0.
    • Cache pre-folded search sources and per-item matches in WeakMap<VirtualItem, ...>.
  • Retained memory budget & background event guard (resourceBudget.ts, EventHandlerModule.ts):
    • Cache byte estimates for completed ModelRounds and completed-round prefix sums per turn in estimateRetainedBytes, turning the 500ms budget walk on active turns from O(rounds) to O(1) incremental tail evaluation.
    • Ignore missing_state_machine data events for unloaded internal background sessions (e.g. MemoryPhase2) instead of logging warnings and triggering requestRuntimeProjectionRepair hundreds of times per second.
  • Virtualizer & message list (useFlowChatVirtualizer.ts, VirtualMessageList.tsx):
    • Avoid scheduling an extra RAF re-render when synchronous virtualizer commits stay within MAX_SYNCHRONOUS_VIRTUALIZER_COMMITS.
    • Skip scheduling viewportSnapshot RAF and full DOM geometry reads when onViewportSnapshot is not provided, and lazily sample readFlowChatViewportGeometry in native scroll handlers.

Type and Areas

Type:

Performance / refactor / test

Areas:

Web UI (src/web-ui/src/flow_chat, src/web-ui/src/shared/utils/resourceBudget.ts)

Motivation / Impact

During long multi-round agent turns (100–300+ rounds), every 32ms streaming flush previously re-projected all completed rounds in the active turn and allocated fresh VirtualItem / FlowGroupData references. That invalidated downstream WeakMap caches in ConversationDocumentProjection and React.memo on VirtualItemRenderer, causing O(rounds) main-thread work per flush and unnecessary settled-row re-renders.

Measured benchmark improvements (src/flow_chat/perf/longRoundPerfHarness.test.tsx, 10 -> 300 rounds, 20 flushes):

  • Category 1 (bench:store-projection): 111.79 ms -> 7.82 ms (~14x faster), settled VirtualItem reference reuse 6.8% -> 97.7%.
  • Category 2 (bench:timeline-document): 17.16 ms -> 3.12 ms (5.5x faster), settled TimelineBlock reference reuse 62.8% -> 94.6%.
  • Category 3 (bench:group-state-search): 14.12 ms -> 3.74 ms (3.8x faster).
  • Category 4 (bench:budget-persistence): estimateRetainedBytes per-call at 300 rounds 1.56 ms -> 0.01 ms (100x+ faster).
  • Category 5A (bench:react-streaming, 120 rounds, 10 tail chunks): VirtualMessageList commits 23 -> 14, wasted settled-row renders 805 -> 2.
  • Category 6 (bench:e2e-pipeline): Total 5-stage pipeline time 139.67 ms -> 12.38 ms (avg 0.39~0.62 ms/flush).

Verification

Automated checks run locally (all passed):

  • pnpm --dir src/web-ui exec tsc --noEmit
  • pnpm --dir src/web-ui exec eslint src/flow_chat/grouping/roundGroups.ts src/flow_chat/grouping/lifecycle.ts src/flow_chat/grouping/groupProjection.ts src/flow_chat/grouping/selectors.ts src/flow_chat/grouping/receiveFeedback.ts src/flow_chat/store/modernFlowChatStore.ts src/flow_chat/components/modern/useFlowChatSearch.ts src/flow_chat/components/modern/useFlowChatVirtualizer.ts src/flow_chat/components/modern/VirtualMessageList.tsx src/shared/utils/resourceBudget.ts src/flow_chat/services/flow-chat-manager/EventHandlerModule.ts src/flow_chat/perf/longRoundPerfHarness.ts src/flow_chat/perf/longRoundPerfHarness.test.tsx
  • pnpm --dir src/web-ui exec vitest run src/flow_chat/perf/longRoundPerfHarness.test.tsx
  • pnpm --dir src/web-ui exec vitest run src/flow_chat/store/modernFlowChatStore.test.ts src/flow_chat/grouping/activityGrouping.test.ts src/flow_chat/grouping/receiveFeedback.test.ts src/flow_chat/timeline/document.test.ts src/shared/utils/resourceBudget.test.ts src/flow_chat/services/flow-chat-manager/EventHandlerModule.test.ts src/flow_chat/components/modern/useFlowChatSearch.test.ts
  • pnpm --dir src/web-ui exec vitest run src/flow_chat/components/modern/useFlowChatVirtualizer.test.ts src/flow_chat/components/modern/useFlowChatVirtualizer.cascade.test.tsx src/flow_chat/components/modern/useFlowChatVirtualizer.measurement.test.tsx src/flow_chat/components/modern/useFlowChatVirtualizer.follow.test.tsx src/flow_chat/components/modern/useFlowChatVirtualizer.initial-window.test.tsx src/flow_chat/components/modern/useFlowChatVirtualizer.aim.test.tsx src/flow_chat/components/modern/useFlowChatViewportAnchor.test.tsx src/flow_chat/components/modern/useFlowChatViewportOwner.test.tsx src/flow_chat/components/modern/useFlowChatFollowOutput.test.ts src/flow_chat/components/modern/useFlowChatFollowOutput.hook.test.tsx src/flow_chat/components/modern/useFlowChatFollowOutput.test.tsx src/flow_chat/components/modern/useFlowChatLeadingExtent.test.tsx src/flow_chat/components/modern/VirtualMessageList.session-boundary.test.tsx src/flow_chat/components/modern/VirtualMessageList.resize.test.tsx src/flow_chat/components/modern/VirtualMessageList.history-prepend.test.tsx src/flow_chat/components/modern/VirtualMessageList.timeline-prepend.test.tsx src/flow_chat/components/modern/VirtualMessageList.edgeFade.test.ts src/flow_chat/components/modern/VirtualMessageList.layout.test.ts src/flow_chat/components/modern/modelRoundItemMemo.test.ts
  • pnpm run check:repo-hygiene
  • node scripts/check-git-object-sizes.mjs --base origin/main --head HEAD

Remote scenarios & manual checks:

  • Exercised in local unit/DOM fixtures; no wire protocol, persisted schema, or remote surface table changes. Manual native WebView scrolling checks in FLOWCHAT_VERIFICATION.md remain pending human confirmation.

Reviewer Notes

  • AI-assisted PR; fully tested against the FlowChat unit and virtualizer/scroll-stability test suites (22 test files, 575 tests passing).

Checklist

  • This PR is focused and does not include secrets, temporary prompts, generated scratch files, or unrelated artifacts.
  • Relevant verification is recorded above, or skipped checks are explained.
  • User-facing strings, docs, and locales are updated where applicable.

…rendering

Keep settled ModelRound projections, FlowGroupData references, group indexes, search sources, and retained-byte estimates cached across streaming flushes, avoid redundant virtualizer RAF budget commits and unwatched viewport snapshot reads, and add categorized long-round benchmarks.
@Raing5Days
Raing5Days force-pushed the perf/long-round-scroll-and-render branch from e4130df to e7f9534 Compare November 6, 2026 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant