Repository navigation
test(benchmark): inject timeout after native session identity is observed - #6920
Conversation
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
loopx-agent
left a comment
There was a problem hiding this comment.
Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
动机
维护发布资格与原生会话连续性的开发者。 原测试把“启动后 1 秒内发出 native identity”当作前提;负载下进程尚未发任何事件就超时,产品正确拒绝未确认身份,但测试没走到自己要验证的已观察会话恢复。 现在先由真实子进程发出 thread.started,再在同一次 communicate 注入 TimeoutExpired;原 worker 清理与 Session owner 仍负责观察、保存和下一次 resume。 67 tests across the three complete resume/usage/UTF8 modules passed, including the installed native CLI against a disposable local Responses endpoint. Immutable old test fails with a 1.2-second startup delay; repaired test passes with the same production code. 本 PR 不修改生产超时、会话接受规则、评分、反馈、提交策略或试验预算,也不启动 benchmark 或收费模型。
改动思路
把测试的故障注入点固定在实际 native event 之后,保留生产进程、输入、stdout、回收、身份观察和持久化路径。原失败里 stdout 为空,故产品拒绝未确认会话符合合同;改变该拒绝或放宽生产 deadline 都会修错 owner。
具体改动
精确 head 5760c67ef01ed6d90dbf8445a83ecab9767a197e,相对 b93f65be0f43389a2ed7b45ff316df8ee55c5609 一个既有测试 +34/-1。规范 docs/development/testing-and-quality.md,spec_revision=b93f65be0f43389a2ed7b45ff316df8ee55c5609。Observed identity 与 Timeout recovery 分别由真实 child event、原超时回执和下一次 wake 同 ID 验证。
关键内容讲解
局部 wrapper 仍调用真实 child_process。communicate 接收原 input 和 timeout,断言 timeout=1,把输入交给真实合成客户端,等一行完整 thread.started 后记录其真实 ID,再抛出 TimeoutExpired(command,1)。原 context 的进程组回收和 Session owner 的 observe/persist 继续执行。wrapper 不生成 receipt、不写 session state,也不修改生产超时。随后恢复普通 child_process,保留 second.ok、action=resume 和两次 session_id 一致的原断言,并新增独立观察 ID 与第一回执相等的检查。等待事件有单独有界的 fixture 安全限时;它不是新增试验预算。
对主干的风险
三个完整模块 67 项通过、0 失败、0 跳过,30.69 秒。包含真实安装 Codex 对隔离本地 Responses fixture 的历史延续,无收费模型。另将合成客户端首事件延迟 1.2 秒,同一生产代码下,immutable 旧测试按预期因无身份失败,新测试观察真实 ID 后注入超时并恢复成功。Ruff、精确 scope change-quality、标准 premerge 的所有 direct/6 项 selected 检查通过。
原 canary 的 benchmark_sensitive manual hold=1 原样保留;本次显式人工审查予以解除:只有测试修改,无 runner、评分、反馈、提交、权限或预算变化,无 benchmark job。完整发布与实际模型失败没有被此局部检查盖过。本次 wait_for_ci=false,未查询、轮询或等待 CI。
最强反例是 wrapper 替生产 owner 制造成功:它只从实际 child stdout 读取独立 ID 并抛出传输故障,receipt 与 binding 仍由原 owner 产生,下一次 wake 走真实普通路径。原身份缺失、fork/home/profile 负例也仍在完整模块中。远端服务可用性不在这个本地用例的验证范围。
我的整体评价
在既有 case 的真实子进程事件后注入故障,实际 session owner 和清理路径继续运行;不为测试新增生产时钟接口。 仅修复一个有独立反例的测试,不改变 runner、评分、反馈、提交或预算。 APPROVE,限 5760c67ef01ed6d90dbf8445a83ecab9767a197e。 bounded_refactor=applied:保留一个用例,在本地注入点改进确定性,未为测试扩充生产接口。long_horizon=preserved,user_experience=preserved,实际恢复/权限/用户步骤不变。
English verdict: APPROVE — 5760c67ef01ed6d90dbf8445a83ecab9767a197e. Inject the timeout after the real synthetic child emits its native identity, then use the unchanged worker/session owner and ordinary next wake. All 67 related tests, Ruff and native automatic checks passed; the benchmark manual hold is explicitly resolved for this test-only diff. Delayed startup fails the old test and passes this one. Production deadlines, scoring, authority and budgets are unchanged; no paid model or benchmark job was launched.
A release sweep terminated the synthetic native client before it emitted any session identity: this recovery test assumed interpreter startup always finished within one second. The production owner correctly refused an unconfirmed session.
Inject TimeoutExpired after the real child publishes its thread.started event, then retain the original timeout receipt, stored identity and next-wake resume assertions. The request still passes its original one-second timeout to the child wrapper; production deadlines, cleanup and identity refusal are unchanged.
Validation: complete resume and Session UTF-8/usage modules; real installed Codex against a disposable local Responses endpoint (no hosted-model calls); Ruff; native premerge automatic checks. A deterministic 1.2-second startup counterfactual fails under the old test and passes under the repaired test with the same production worker. The generic benchmark-sensitive manual hold is reviewed as test-only: no runner, scoring, feedback, submission or execution budget changes, and no benchmark job launched.