Skip to content

[v2] Show nodes waiting for a session slot instead of running - #389

Merged
EricWvi merged 5 commits into
ora-space:mainfrom
zhangxinhao:feat/1224-session-slot-wait
Nov 17, 2026
Merged

EricWvi merged 5 commits into
ora-space:mainfrom
zhangxinhao:feat/1224-session-slot-wait

Conversation

@zhangxinhao

Copy link
Copy Markdown
Contributor

Refs ora-space/circus#1224

v2,不涉及 specs.

What it does

Since #382 a run keeps fewer than 8 unfinished sessions on its checkout, and an Agent attempt beyond the free slots waits in the bridge. Until now that attempt read as 运行中 with no session and no hint, so a queued node looked stuck. This marks the wait and shows it.

  • v2-server (workflowhost.beginSession): the first time an attempt finds no free slot it merges session_slot_wait: {since, limit} into its row's payload and publishes. It drops the marker before its Start once it takes a slot, and also when it gives up (cancel, Loop close, resume generation, shutdown), so a cancelled row never keeps reading as queued. An attempt admitted at once writes nothing; the writes happen outside the run's lock. Two store methods in store/workflow_session_slots.go: mark (only while the row is running) and clear (any status).
  • frontend-v2: like retry_wait, the adapter maps a running row carrying the marker under a live run to the display state session_waiting: a still hourglass mark and a muted info badge 等待会话名额 instead of the running spinner and breathing ring, plus the limit in the inspector. The stage focus, branch projection, tasks stage and conversation turns treat it as live work, the same as a waiting retry.
  • README: the queueing paragraph names the marker and when it clears.

Trade-offs

  • A payload marker rather than a new typed field in the run read API: it follows the existing retry_wait pattern, so the API shape and the generated client are unchanged, and updates ride the existing run events. A new node status was not considered worth it: the node status drives the engine, resume and retry.
  • If the process exits while an attempt queues, the marker stays on the row; the boot sweep fails that run, and the view only shows the hint on a running row under a live run.

Verification

  • gofmt -l . clean, go vet ./..., go test -race -count=1 ./... with PostgreSQL: pass.
  • TestSiblingsBeyondTheSessionLimitWaitForASlot (limit 1) now also asserts the waiting sibling's row carries the marker while the executing one does not, and that it is gone once the sibling starts; TestCancelDropsASiblingWaitingForASlot waits for the marker before cancelling and asserts no row keeps it. Removing the mark call fails both; removing the clear call fails both.
  • server-run-adapter.test.ts: queued → session_waiting with the limit; running once the slot is taken; never waiting under a failed / cancelled run or on a non-running row. Removing the status mapping fails the first case.
  • npm run check: pass. npm run e2e: 71 passed, 14 skipped; 7 failed in the full run, of which 6 pass when rerun on their own (load-related). The remaining one, settings.spec.ts:139 (plugin install lifecycle), fails the same way locally on main 1cf476bf.
  • UI (real Node, fixture model; local build only with the limit lowered to 1 and the fixture slowed 12 s per call): with Start → 写方案 / 写测试, one node is 运行中 and the other 等待会话名额 with no session; the API shows the marker on the waiting row only, and once the first finishes the second starts as 运行中 with the marker gone.

An attempt that finds its run's session slots full records session_slot_wait in its row's payload until it takes a slot or gives up, so the run view can tell it from a turn that is executing. An attempt admitted at once writes nothing.
A running row marked session_slot_wait under a live run reads as session_waiting: a still hourglass mark and a muted 等待会话名额 badge instead of the running spinner, with the limit in the inspector. The stage and branch projection keep treating it as live work.

@wanglongan587 wanglongan587 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审查 commit:827ae730fc678abbd927e73ad84bc8149eb16ac5。

[P2] 已绑定会话时应忽略残留的排队标记

位置:frontend-v2/src/features/chat/workflow-run/server-run-adapter.ts:124–126;v2-server/internal/workflowhost/agents.go:362–369。

clearSessionSlotWait 的 SQL 出错只记日志,beginSession 随后仍返回已占名额的 attempt,继续 exec.Start 和 BindWorkflowNodeSession;后者只写 session_id,不会移除标记。因此一次清除失败(例如该 SQL 超时、随后数据库恢复),就能留下 status=running、sessionId 非空、payload 仍含 session_slot_wait 的有效行。适配器不核对 sessionId,会把已经执行中的节点持续显示成“等待会话名额”。交互节点的 summary 合并保留标记,后续轮次也可能继续误显。

复现:直接执行当前提交的 nodeState/projectNodeStatus,将现有 queued fixture 的 sessionId 设为 session-a、保留标记,得到 session_waiting;清除标记后才返回 running。请至少在投影中要求 sessionId 为 null 才显示等待,或在成功绑定时原子清除标记,并补“已经绑定会话但残留标记”的回归用例。

其余核查:16 个改动文件均在 v2 目录且为交付物;Refs #1224、phase:1/v2 豁免说明与排队/开始截图符合要求。正常获取名额、取消、Loop 关闭、换代和关停路径的锁与清理结构未发现其他阻塞问题。

验证:逐个核对 head 文件 blob SHA;用 Node 内置 TypeScript 擦除执行原始投影代码,验证排队、开始、残留标记、取消、终止运行共 5 种输入。CI 44034602323 的 server(PostgreSQL + go test -race)、frontend、e2e 全部成功。本地完整门禁未重跑:无可用 vitest 依赖,Go race 环境未启用 cgo。

@ObsisMc ObsisMc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审查 commit:827ae730fc678abbd927e73ad84bc8149eb16ac5(Request changes)

同意 @wanglongan587 的两点,需要处理后重新送审:

  1. [P2] 已绑定会话时应忽略残留的 session_slot_wait(frontend-v2/src/features/chat/workflow-run/server-run-adapter.ts:122-126、:146)。clearSessionSlotWait 写库失败只记日志,attempt 照常 Start、BindWorkflowNodeSession,标记就留在一个真在执行的 running 行上。我在 head 上临时加了一个用例(queued({ sessionId: 'session-a' })),得到的是 session_waiting 而不是 running,复现成立。建议投影里要求 node.sessionId === null 才显示等待(成本最低,也能兜住交互节点 summary 合并带回标记的情况),并把这个用例补进 attempts queued for a session slot。
  2. 合并冲突:GitHub 显示与 main 冲突。main 在 base 1cf476bf 之后合入了 #383,它也改了 v2-server/README.md、workflowhost/agents.go、host_test.go 和 translations.ts。rebase 后请重跑 go test -race 和前端单测。

其余核查,没发现阻塞问题:

  • payload 标记的取舍:进程在排队时退出,boot sweep 的 failInterruptedNodes 只去掉 retry_wait,标记会留下,但行被置为 failed(被吸收的 Iteration 成员也是 failed);resume 会软删除被清理节点的 attempt 并新建行,不会把旧标记带回 running 行。前端要求 live && node.status === 'running',所以这条路径不会误显。
  • 退出路径:awaitSessionSlot 返回后,只要 waited 为真,beginSession 就会清标记。拿到名额(在 abandoned 检查和 Start 之前)、取消(CancelRun 设 cancelling 并唤醒)、Loop 关闭(loop_exit.go 唤醒)、换代、关停(h.ctx.Done())都走这一处。清标记用 WithoutCancel 加 stopTimeout,并且在 dispatched-- 和 workers.Done() 之前执行,所以 Close 和 forgetFinished 都会等它完成。
  • 锁外写库:mark 和 clear 在同一个 goroutine 里先后执行,同一 attempt 不会出现先清后写;mark 期间释放了名额,回到锁内后 continue 会立刻重查,不会漏掉唤醒。clear 不限状态,mark 只写 running 行,被取消的行不会被重新写上。直接拿到名额的节点 waited=false,确实不写库。
  • 前端:session_waiting 只在运行未结束、行是 running、带合法标记时出现。run-focus 的 ACTIVE_STATUSES、branch-projection、conversation-turns、run-tasks-stage 都把它当作进行中。isNodeWorking 不含它,所以没有转圈和呼吸样式。检查面板写明了上限。对照截图:全图里写方案显示运行中,写测试显示「等待会话名额」(沙漏);名额到手后变为运行中。

可选的小问题,不阻塞:

  • 排队节点的卡片显示开始时间(截图里写测试是「14:40:52 — —」),started_at 在派发时就写了。retry_waiting 在 run-overview-node.tsx 和 run-theater-act-card.tsx 里隐藏了时间范围,session_waiting 也可以照做,否则排队时间看起来像运行时间。
  • types.ts 新增的 WorkflowNodeSessionSlotWait 插在 GraphWorkflowNodeStatus 的文档注释和类型定义之间,那段注释现在落到了接口上。建议把接口挪到注释前面。

验证(本地,PR head 827ae730):

  • GOMAXPROCS=2 go test -p 1 -race -count=1 ./internal/workflowhost/ ./internal/store/(PostgreSQL 17):全部通过(workflowhost 83s,store 47s)。
  • vitest run server-run-adapter.test.ts src/lib/workflows/runtime:7 个文件、78 个用例通过。
  • 反向检查:
    • 前端去掉 session_waiting 映射后,shows a queued attempt as waiting 失败;
    • 后端分别去掉 mark 调用和 clear 调用:两个名额用例在两种改法下都失败(未写标记时等待超时;未清标记时报「a node that took its session slot still reads as waiting」和「cancelled node … still reads as waiting」)。改动已还原。
  • CI 在 827ae730 上的结果:server、frontend、e2e 全部通过(run 44034602323)。

Binding a session drops session_slot_wait in the same statement, and the run view reads the marker only on a row without a session, so a failed clear in the bridge no longer leaves an executing node shown as queued.
@zhangxinhao

Copy link
Copy Markdown
Contributor Author

Thanks — the P2 is right: a failed clear left a bound, executing attempt carrying session_slot_wait, and the view did not look at sessionId. Fixed on both sides in 5012f667:

  • Server: BindWorkflowNodeSession drops session_slot_wait in the same UPDATE that writes session_id, so a bound attempt can never keep the marker (this also covers the interactive summary merge, which runs after binding). Regression test TestBindingASessionDropsALeftoverSlotWait in store/workflow_session_slots_test.go: mark → bind → no marker; it fails when the CASE is removed.
  • View: the adapter reads the marker only on a running row with sessionId === null. New case "shows an attempt with a bound session as running even if the marker was left behind"; it fails when the sessionId check is removed.
  • README notes both.

The README conflict is resolved by merging main (b395cafb) instead of rebasing, so the commits you reviewed stay as they were; the merged tree is identical to a rebase onto 5416335a.

Gates on 5012f667: go test -race -count=1 ./... pass (./demo timed out once while another job held the shared test database; rerun alone it passes in 30 s); npm run check pass; npm run e2e 78 passed, 14 skipped, on isolated ports and database. Screenshots re-taken on this commit (same local-only build: limit 1, slowed fixture) are on ora-space/circus#1224.

A queued node's row carries its dispatch instant as started_at; like a waiting retry, the stage card, inspector and overview no longer show it as a running time range. Also moves WorkflowNodeSessionSlotWait above the status type's doc comment, which it had split.
@zhangxinhao

Copy link
Copy Markdown
Contributor Author

@ObsisMc thanks. Both blocking points were already addressed in 5012f667 (see the reply above: the bind clears the marker, the view requires sessionId === null, regression tests on both sides; the conflict with #383 is resolved by merging main, and go test -race plus the frontend gates were re-run on the merged tree).

Both optional points are taken in f0d44807:

  • A node in session_waiting no longer shows its dispatch instant as a running time range, on the stage card, the inspector and the overview node, like retry_waiting.
  • WorkflowNodeSessionSlotWait moved above the GraphWorkflowNodeStatus doc comment it had split.

npm run check passes on f0d44807 (941 tests). New screenshots on ora-space/circus#1224 show the queued node without a start time.

@ObsisMc ObsisMc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复审 commit:f0d44807cb46781ba7ec94036c7e073505fa4504(Approve)

上一轮的意见都已改好:

  • [P2] 残留标记:后端和前端两处都修了。
    • 后端:BindWorkflowNodeSession 在写 session_id 的同一条 UPDATE 里去掉 session_slot_wait。
    • 前端:sessionSlotWait 只在 sessionId === null 时读取标记(WorkflowNodeRun.sessionId 的类型是 string | null,没有 undefined 的情况)。
    • 两边都补了回归用例。
  • 冲突:用 merge main(b395cafb)解决,第二个父提交是当时的 main 5416335a。我逐行比对过:5416335a → b395cafb 的改动和 1cf476bf → 827ae730 完全一致,README 里两段都保留了,merge 没有夹带别的改动。
  • 两个可选项:
    • 舞台卡片、检查面板、全图节点都不再给排队节点显示开始时间;
    • types.ts 的注释位置也改对了。
    • 新截图对得上。

验证(本地,head f0d44807):

  • GOMAXPROCS=2 go test -p 1 -race -count=1 ./internal/workflowhost/ ./internal/store/(PostgreSQL 17):通过。
  • vitest run src/features/chat/workflow-run src/lib/workflows/runtime:198 个用例里有 1 个失败,是 run-flow.test.tsx 的分析用例。当时 Go 测试正在并行跑,单独重跑这个文件 26 个用例全部通过,判断是负载导致的偶发失败。
  • 反向检查(都已还原):
    • bind 不去掉标记:TestBindingASessionDropsALeftoverSlotWait 失败;
    • 前端去掉 sessionId 判断:「bound session … left behind」用例失败;
    • 去掉写标记、去掉清标记:两个名额用例仍然都失败。
  • CI 在 f0d44807 上:server、frontend、e2e 全部通过(run 44048911348)。

不阻塞的建议:

  • 舞台的自动聚焦可能落到排队节点上。pickPrimaryAmongActive 在活动节点里选 startedAt 最新的,而排队节点的 startedAt 是派发时刻:如果排队的兄弟节点比正在运行的那个晚派发几毫秒,舞台主卡片就是排队节点(「Agent 尚未启动」)。我用临时用例验证过:running a(派发于 …52.310Z)和 session_waiting b(派发于 …52.329Z)同时在跑时,resolveTheaterFocus(run, null).primaryId 是 b。retry_waiting 没有 startedAt,所以正在运行的节点会排在它前面;建议 session_waiting 也照这样处理,可以另开 PR 跟进。
  • 排队节点拿到名额后,卡片上的开始时间仍是派发时刻:截图里写测试显示 16:40:34,实际开始是 16:41:12 左右。这是 started_at 原有的语义,记一笔就好。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants