Conversation
Share device leases, durable goals, evidence, and workbench components across Codex, WorkBuddy, and DSH while keeping host state and installations isolated. Use host models for planning and screenshot decisions; disable user model configuration in the production task flow and preserve legacy controls. Add native workbench resources, conversation-bound handoff, upgrade guards, and first-frame and task lifecycle regression coverage. Validation: all three host check commands, Codex viewer decoding test, and DSH 0.1.1-rc.2 package install/host/API/client compatibility check passed. Candidate checkpoint only: actual Codex native MCP loading and handoff remain unverified. DSH visual localization and remaining physical-device acceptance gates are documented; this commit does not claim release readiness.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 75eb5c834f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (native) this.runtime.viewers.allowNativeEmbedding(preview.viewerId, 'workbench-preview:' + deviceId, native) | ||
| send(200, preview) | ||
| } | ||
| else if (route === 'manage') send(200, await (this.runtime.goals.tasks.has(String(args.taskId)) ? this.runtime.goals : this.runtime).manage(String(args.taskId), String(args.action), typeof args.text === 'string' ? args.text : undefined)) |
There was a problem hiding this comment.
当同一后台服务同时服务多个宿主会话时，state 会把 goals.list() 和 runtime.list() 的全部任务及其 ID 返回给每个会话，而此处调用 manage 时又没有传入已从 session capability 解析出的 owner。因此任一会话的工作台都能使用泄露的 ID 停止、恢复或修改另一会话的任务；events 和 evidence 路由同样省略了 owner 校验。应按当前 owner 过滤状态列表，并把 owner 传给所有任务读取和变更接口。
Useful? React with 👍 / 👎.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Ref DMY-59 Codex, WorkBuddy, and DSH submit phone tasks to one user-level service that executes with the configured model. Disconnecting a host no longer cancels accepted work. Legacy journals can be archived without copying credentials, and a launch agent plist can be installed without embedding the service token.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e62322b139
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| export async function validateSourceBoundary(hostRoot) { | ||
| const host = await realpath(resolve(hostRoot, 'src')) | ||
| const shared = await Promise.all(['device-runtime', 'phone-agent', 'workbench'].map(name => realpath(resolve(core, '..', name, 'src')))) |
There was a problem hiding this comment.
plugins/opengui 和 workbuddy-plugin 现在都直接导入 packages/task-service，但这里的共享源码白名单以及第 122 行的 staging 循环都只包含另外三个包。我检查了两个隔离导出目录，导出后均不存在 packages/task-service，且直接运行 validateSourceBoundary 时三个宿主都会因这些导入越界而失败；因此 Codex 和 WorkBuddy 工作流在隔离目录执行构建时无法解析模块。请同时把该包加入校验边界和 staging 输入。
Useful? React with 👍 / 👎.
|
|
||
| interface Reply { result?: unknown; error?: string } | ||
|
|
||
| const entry = fileURLToPath(new URL('../../../plugins/opengui/src/task-service-main.ts', import.meta.url)) |
There was a problem hiding this comment.
在开发树中此相对 URL 能找到 Codex 源文件，但该模块发布时会被打包到各宿主的 lib 文件中，届时 import.meta.url 指向安装目录，而 Codex staging、WorkBuddy npm 包和 DSH 包都不包含 plugins/opengui/src/task-service-main.ts。因此首次任务触发自动启动时，分离的 Node 进程会立即因入口不存在而退出，调用方等待 15 秒后报错。应构建并随包发布独立服务入口，再相对于已安装的 lib 路径解析它。
Useful? React with 👍 / 👎.
| const existing = await lstat(endpoint).catch(() => undefined) | ||
| if (existing) { | ||
| if (!existing.isSocket()) throw new Error('task service endpoint is not a socket') | ||
| await rm(endpoint) |
There was a problem hiding this comment.
当两个宿主几乎同时首次调用任务服务时，它们都可能在 alive() 尚未成功的窗口内启动子进程；后启动者看到前一个进程刚创建的 socket 后会在未确认其是否存活的情况下直接删除并重新绑定。原服务及其已接收任务随后仍运行但不再可达，而新请求进入另一个运行时，形成分裂状态。这里需要启动锁或对现有端点完成存活/所有权确认，只有确认是陈旧 socket 后才能删除。
Useful? React with 👍 / 👎.
Classify emulator and tcp adb rows, label them, and keep the same machine lease and adb -s execution path for emulator serials. Workbench copy now includes a running Android emulator beside USB phones.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ed17c8c99
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| instructions: [ | ||
| 'Use the current Codex model to plan and decide phone tasks. OpenGUI executes validated actions and stores evidence; do not configure another model.', | ||
| 'Call opengui_open_workbench for the native panel in this conversation. Keep it open for progress and evidence. If the host cannot render its resource, use open_in_codex for its URL. Submit a chat goal with opengui_run_task and a stable requestId. The shared service executes it with the configured model. Never submit the same goal twice and never call next/decide.', | ||
| 'Continue next/decide until terminal or waiting for user help. Read each returned imagePath with the host image viewer before deciding. Preserve observation IDs, coordinate contracts, device leases, and action serialization.', |
There was a problem hiding this comment.
共享任务服务现在由 plugins/opengui/src/task-service-main.ts 注入自配置的 phoneExecutor，而 TaskHost.call 会明确拒绝 next 和 decide；但这里紧接着“never call next/decide”的说明，又要求每个 Codex 会话持续调用它们。模型若遵循这一句，任务管理调用会稳定返回 Host decision bridge unavailable，使新的 Codex 手机任务流程无法可靠推进或汇报。请删除遗留的宿主驱动说明，并让整段 MCP instructions 只描述共享服务的 status/steer/resume/stop 流程。
Useful? React with 👍 / 👎.
| try { | ||
| agent.followup(createUserMessage({ | ||
| source: { kind: 'plugin', plugin: 'dsh-coremate-mobile' }, | ||
| content: [{ type: 'text', text: `Continue the user's accepted OpenGUI phone task ${id}. Do not submit a duplicate task. Use opengui_manage_task next/decide with taskId ${id} to plan and execute with your current host model. Inspect each returned image and follow its coordinate and completion contracts. Continue until terminal or waiting for user help. Use status for this task to verify the result. Never substitute shell/ADB or configure another model.` }], |
There was a problem hiding this comment.
当用户通过 /opengui <goal> 或 continue 路径提交任务时，这条 follow-up 会要求 DSH 模型使用 next/decide 接管执行；然而共享服务已经切换为配置模型自运行，公开工具说明也明确声明这两个操作会被拒绝。因此每次该入口成功提交后都会额外触发一轮必然失败的宿主调用，并可能让用户误以为仍在后台正常执行的任务已经失败。请改为仅查询状态，或在自运行模式下完全省略宿主接管消息。
Useful? React with 👍 / 👎.
| if (maintenance) throw new Error('upgrade_in_progress: runtime is draining; no new work accepted') | ||
| if (message.method === 'prepare_upgrade') { | ||
| if (!installerConnection) throw new Error('Upgrade requires an installer connection') | ||
| if (cleanups.size > 0 || service.hasActiveSessions() || service.hasPersistentMirrors() || !tasks.prepareMaintenance()) { |
There was a problem hiding this comment.
sharedPhoneTasks 的 activeCount 和 watching 初始均为零，只有调用 snapshot() 才会从共享服务刷新；普通 call() 不更新这两个缓存，而 WorkBuddy 代码中也没有任何 snapshot() 调用。因此即使共享服务仍有活动任务或打开的工作台，这里的升级检查也会把 tasks.prepareMaintenance() 判为 true，关闭 broker 并允许替换安装。请在判定前查询共享服务的实时状态，或直接调用服务端的 __prepare_upgrade__ 检查。
Useful? React with 👍 / 👎.
| task.phase = running ? 'stopping' : 'cancelled' | ||
| await this.record(task, 'stop_requested') |
There was a problem hiding this comment.
当用户在详情页停止一个尚未开始的排队分支时，running 为空，因此该分支会直接变为 cancelled，但这里不会调用 goals.reconcile(task.parentId)。如果这是父目标唯一的分支，例如设备正被另一个任务占用，之后也没有执行完成回调来触发汇总，父目标就会永久显示为 running，直到有人显式请求父目标的 status。请在非运行分支进入终态后同步重新汇总父目标并记录 settled 状态。
Useful? React with 👍 / 👎.
| paths: | ||
| - 'packages/device-runtime/**' | ||
| - 'plugins/opengui/**' |
There was a problem hiding this comment.
宿主现在直接编译并打包 packages/phone-agent、packages/workbench 和 packages/task-service，但路径过滤器只新增了 packages/device-runtime/**；WorkBuddy 工作流同样如此，DSH 工作流则没有任何共享目录过滤项。因此只修改其余三个共享包的 PR 不会触发对应宿主的构建、测试或隔离边界校验，破坏性改动可以在所有发布检查之外合入。请将所有被宿主导入或 staged 的共享目录加入各宿主 CI 的 push 和 pull_request 路径。
Useful? React with 👍 / 👎.
Add a skipped-by-default live test that submits from the workbench, Codex, WorkBuddy, and DSH, stores a real emulator screenshot, and stops the task. The planner is scripted so the run does not call a model.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 939a2bbc36
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| await writeFile(tokenFile, token, { mode: 0o600 }) | ||
| } | ||
| const runtime = new PhoneRuntime({ | ||
| root: dataPath(options.root), host: 'shared', hardware: options.hardware, |
There was a problem hiding this comment.
共享服务把 PhoneRuntime.options.host 固定为 shared，但工作台仍用该字段判断宿主：Workbench 的原生入口只接受 codex/workbuddy，导致 MCP App 添加 mcpApp 查询参数后页面路由变成未知路径并返回 404；TaskHost 也只在 host 为 dsh 时允许 embedOrigin，因此 DSH iframe 会被 frame-ancestors 'none' 阻止。三个宿主切换到 sharedPhoneTasks 后，其主要工作台入口都会失效；应把服务端已经认证的请求 host 传入工作台打开逻辑，而不是依赖这个固定值。
Useful? React with 👍 / 👎.
| const allowed = await (this.options.confirmAction ?? confirmPhoneAction)(String(effect), `目标：${task.goal}\n手机：${task.deviceName}\n动作：${JSON.stringify(input)}`, signal) | ||
| signal.throwIfAborted() | ||
| await this.record(task, allowed ? 'confirmation_allowed' : 'confirmation_declined') | ||
| if (!allowed) { declined = true; throw new Error('user_declined: 此操作未获确认，请报告受阻。') } |
There was a problem hiding this comment.
当 send、purchase 或 delete 等确认对话框停留期间手机画面发生变化时，这里在用户允许后仍直接沿用确认前的 last.observationId 执行动作。共享服务使用的 LocalAdbPhoneHost 没有提供 PhoneController.sampleFrame，所以控制器的可选派发前画面比较也不会运行；最多等待 60 秒的确认结束后，坐标或 Enter 键可能作用于完全不同的界面。应在批准后强制获取新观察并重新确认目标有效，再允许派发该副作用操作。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b51e272aea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| assert(client.getInstructions()?.includes('Use host-driven phone tasks by default'), 'Packed MCP must advertise the host-driven task contract') | ||
| assert(client.getInstructions()?.includes('opengui_manage_task next/decide'), 'Packed MCP must explain how the host advances accepted tasks') |
There was a problem hiding this comment.
startMcp 当前明确声明任务由共享服务中配置的模型执行，并要求不要调用 next/decide，因此生成的 MCP instructions 不可能包含这里断言的两个旧宿主驱动字符串。workbuddy-plugin-ci.yml 的 packaged-startup 以及发布工作流都会运行 npm run smoke:packed，所以成品成功启动后会在此处稳定失败，阻断所有 WorkBuddy 候选构建和发布；应改为断言当前共享服务契约。
Useful? React with 👍 / 👎.
| const path = join(this.runtime.options.root, 'workbench-draft-v1.json') | ||
| await writeFile(path + '.tmp', JSON.stringify({ goal: value }), { mode: 0o600 }) | ||
| await rename(path + '.tmp', path) | ||
| this.draft = value |
There was a problem hiding this comment.
当多个宿主会话通过各自的 /session/<token>/ 打开共享工作台时，任一会话保存草稿都会写入同一个 workbench-draft-v1.json 并覆盖单例 this.draft，而 state 又会把该值返回给任意 owner。因此会话 B 能看到并覆盖会话 A 尚未提交的目标文本；应将草稿按已解析的 owner 分区存储和读取。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1208c3132a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!tasks.prepareMaintenance()) throw new Error('upgrade_blocked: stop phone tasks and close the workbench first') | ||
| send({ result: { stopping: true } }) | ||
| setImmediate(() => { void close() }) |
There was a problem hiding this comment.
任务服务会并发处理不同 socket，而这里仅检查一次 prepareMaintenance()，随后到 setImmediate(close) 之间仍会接受普通 opengui_run_task 请求。若宿主恰在升级停机窗口提交任务，该任务可以在检查后被持久化并开始规划，紧接着又被 tasks.close() 中断为 unknown，但升级方已经收到 stopping: true 并会继续替换安装。应在返回停机成功前设置服务级 maintenance/draining 状态以拒绝所有新业务请求，并等待已进入的请求排空后再关闭运行时。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
OpenGUI/plugins/opengui/src/cli.ts
Lines 40 to 44 in e4a7c7b
在升级切换前关闭旧共享任务服务
当共享任务服务已由任一宿主自动启动后再升级 Codex 时，此处只向 Codex daemon 发送 __shutdown__，从未关闭独立的 task service；该服务没有空闲退出机制，而 ensureTaskService() 会持续复用仍存活的 socket。因此配置切到新插件后，新任务仍会由旧安装中的 task-service.js 执行，修复和协议变更要等到重启或手工杀进程才生效。请在确认共享服务无活动任务/工作台后调用其已实现的认证 __shutdown__，并等待服务端点退出后再切换安装。
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e8fd0c953c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (finished || ++calls > 120) throw new Error('operation_budget_exhausted') | ||
| const value = action ? await run.act(action) : await run.observe() | ||
| return [ | ||
| { type: 'text', text: JSON.stringify({ observationId: value.observationId, width: value.width, height: value.height, foregroundPackage: value.foregroundPackage }) }, |
There was a problem hiding this comment.
当手机截图的长边超过 2048 像素时，encodeCodexPhoneScreenshot 会缩小实际发送给模型的 JPEG，但这里仍把原始屏幕的 value.width/value.height 声明为图像尺寸；而 PhoneController 按 observation.image.width/height 校验并换算 targetBBox 和滑动坐标。因此共享服务的自主执行器会让模型针对错误的像素空间生成坐标，常见的 1080×2400 等设备上会出现越界拒绝或点击、滑动到错误位置。这里应向模型暴露 value.image.width 和 value.image.height，与 HostExecutor 已采用的坐标契约保持一致。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 355b9a7d9b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| try { return await request(root, '__ping__', {}, 'ping', 'codex') as { activeTasks: number; watching: boolean } } | ||
| catch (error) { if (['ENOENT', 'ECONNREFUSED'].includes(String((error as NodeJS.ErrnoException).code))) return undefined; throw error } |
There was a problem hiding this comment.
共享服务以 detached/launchd 进程跨宿主存活，但我检查了 Codex 的 --check-upgrade、WorkBuddy 的 prepareUpgrade 和 DSH 更新路径，它们都不会调用服务端已有的 __prepare_upgrade__/__shutdown__。因此安装新版本后，旧进程仍继续提供旧代码；一旦 TASK_SERVICE_PROTOCOL 升级，旧服务会返回 task service protocol mismatch，而这里仅把 ENOENT/ECONNREFUSED 当作可重启状态并直接抛出协议错误，导致新客户端永远不会启动新服务，所有任务调用持续失败，直到用户手动终止旧进程或重启系统。请让升级流程在替换包前完成维护检查并关闭共享服务，或为协议不兼容提供受控的旧服务替换路径。
Useful? React with 👍 / 👎.
gofenix
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 90082d056d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return connection | ||
| } | ||
| // HTTP inspectors frequently close their MCP sessions. That must not interrupt phone tasks. | ||
| const shared: ToolConnection = { call: async (...args) => (await broker()).call(...args), close: () => {} } |
There was a problem hiding this comment.
当原生 HTTP 端点同时承载多个 MCP 会话且调用没有 hostContext（broker 明确支持的非 hook 路径）时，这里让所有会话复用同一个 BrokerClient socket；而 broker.ts 将任务 owner 回退为该 socket 唯一的 connectionOwner。因此这些原本独立的会话会被任务服务视为同一 owner，任一会话都能通过 opengui_list_tasks 获得其他会话的任务，并用 opengui_manage_task 停止或修改它们。应把会话身份随调用传递，或为每个 HTTP MCP 会话建立独立的 broker 所有权上下文，同时仍可独立保持底层进程存活。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2d661f225d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -1,8 +1,10 @@ | |||
| # OpenGUI | |||
|
|
|||
| > Current candidate phone execution is host-driven: the current DSH conversation model plans and inspects screenshots through `opengui_manage_task next/decide`. The workbench has no separate model configuration. `/opengui` opens the workbench; `/opengui <goal>` submits once and hands the accepted task back to the host. Embedded homepage handoff uses `/opengui continue <taskId>`. Closing the page does not stop execution, but execution cannot continue without host decisions. New phone tasks currently support macOS only. Dedicated-model and router/subagent instructions below describe legacy/browser paths, not the new phone workbench. See the [current Chinese flow](README.zh.md). Real DSH host/model acceptance remains pending. | |||
There was a problem hiding this comment.
当用户按照此处的新手说明操作时，流程会稳定失败：最终实现由 plugins/opengui/src/task-service-main.ts 向共享服务注入 phoneExecutor，需要先在工作台配置视觉模型，而 packages/phone-agent/src/host.ts 会拒绝该模式下的 next/decide；这里却声明无需独立模型并要求持续调用这两个操作。Codex 和 WorkBuddy README 中新增的 Host-driven 章节也描述了同一旧契约，请统一改为当前的配置模型及 status/steer/resume/stop 流程，以免安装者无法启动任务。 CLAUDE.mdL15-L18
Useful? React with 👍 / 👎.
The recurrence came from a capped reconnect loop and a replay burst that reset WebCodecs before its first output. Replaying the bounded GOP, preserving the decoder queue, and renewing the receipt at first media make cold entry and reconnect self-healing.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4a835aec35
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Persist before dispatch: an RPC timeout cannot prove that the host did not receive it. | ||
| sessionStorage.setItem(key,'attempted'); | ||
| try{ | ||
| const delivered=await request('ui/message',{role:'user',content:[{type:'text',text:'Continue the accepted OpenGUI phone task '+task.id+'. Do not submit a duplicate task. Use opengui_manage_task next/decide with this taskId and your current host model. Inspect each returned image and follow its coordinate and completion contracts. Continue until terminal or waiting for user help. Verify the result with status. Keep the existing native workbench open; it already displays progress and evidence. Do not call present_files or open another workbench/preview tab, including in the final response. Do not substitute shell/ADB or configure another model.'}]}); |
There was a problem hiding this comment.
当用户在 Codex 或 WorkBuddy 原生工作台提交或继续任务时，这条 ui/message 会明确要求宿主调用 next/decide，但共享任务服务使用自主 phoneExecutor，TaskHost.call 会以 Host decision bridge unavailable 拒绝这两个操作，因此每次 handoff 都会额外触发一轮必然失败的宿主执行并可能误报仍在后台运行的任务。相较于已有的 MCP/DSH 评论，新的证据是这里存在一条独立的原生工作台唤醒路径；应改为只查询 status，或在自主执行模式下不发送接管消息。
Useful? React with 👍 / 👎.
| const candidates = [...branch.eligibleDeviceIds].sort((a, b) => this.load(a) - this.load(b) || a.localeCompare(b)) | ||
| if (!candidates.length) throw new Error('No eligible phone') | ||
| const child = await this.runtime.submit({ requestId: task.id + ':' + index, parentId: task.id, | ||
| goal: branch.goal, successCriteria: branch.successCriteria, deviceId: candidates[0]!, modelProfileId: task.modelProfile.id }, task.owner, task.modelProfile) |
There was a problem hiding this comment.
当规划器为分支返回多个候选设备，而排序后的第一个设备正被另一宿主进程的旧式控制会话占用时，load() 只统计当前共享运行时中的任务，因此仍会把该设备固定给子任务；随后 acquireDeviceLease 抛出 device_busy，分支直接进入 blocked，即使其他 eligibleDeviceIds 中存在空闲手机也不会尝试。应在分配时结合跨进程租约可用性，或在获取租约失败后依次尝试其余候选设备。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0ee3278424
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const effect = input.externalSideEffect ?? 'none' | ||
| if (!['none', 'send', 'publish', 'purchase', 'delete'].includes(String(effect))) throw new Error('Invalid external side effect') | ||
| if (effect !== 'none') { |
There was a problem hiding this comment.
当视觉模型误判或受屏幕内容诱导时，它可以把发送、发布、购买或删除按钮的动作声明为 externalSideEffect: "none"；这里完全信任该字段（甚至缺失时也默认为 none），随后直接派发动作，因此本应由本机对话框保护的操作会在没有用户确认的情况下执行。敏感操作确认属于运行时安全边界，不能仅依靠同一个不受信任模型如实标注副作用；应采用独立的判定/确认策略，并至少对缺失或无法可信分类的动作采取失败关闭。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 036f03a27e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| function drawRows(target,list){target.replaceChildren();if(!list.length){target.append(node('p','暂无任务','muted'));return}for(const t of list){const row=node('article',undefined,'task-row'),body=node('div'),actions=node('div',undefined,'row');body.append(link(t.goal,'task/'+t.id));body.firstChild.className='task-title';body.append(node('div',source(t)+' · '+t.deviceName+' · '+new Date(t.createdAt).toLocaleString('zh-CN'),'meta'));actions.append(node('span',phaseLabel(t),'pill'),link('查看','task/'+t.id));if(active(t))actions.append(button('停止',async()=>{await api('manage',{taskId:t.id,action:'stop'});await refresh()},'danger'));row.append(body,actions);target.append(row)}} | ||
| function drawWorkspaceRows(target,list){target.replaceChildren();for(const t of list){const card=link('', 'task/'+t.id);card.className='workspace-task-card';card.dataset.task=t.id;const header=node('div',undefined,'row spread');header.append(node('span',phaseLabel(t),'pill'),node('time',new Date(t.createdAt).toLocaleTimeString('zh-CN',{hour:'2-digit',minute:'2-digit'}),'muted'));card.append(header,node('strong',t.goal,'task-title'),node('small',t.deviceName));target.append(card)}if(!list.length)target.append(node('p','暂无任务','muted'))} | ||
| function drawDevices(target,list){target.replaceChildren();if(!list.length){target.append(node('p','暂无设备','muted'));return}for(const d of list){const card=node('article',undefined,'device'),head=node('div',undefined,'row spread'),title=node('h3'),preview=node('div',undefined,'device-preview');title.append(link(d.name,'device/'+d.id));head.append(title,node('span',deviceLabel(d),'pill'));if(d.connected&&d.authorized&&cardStreams.has(d.id)){preview.className='card-stream';const frame=node('iframe');frame.title=d.name+' 实时画面';frame.src=cardStreams.get(d.id);preview.append(frame)}else preview.append(node('p',!d.connected?'已断开':!d.authorized?'待授权':'正在连接画面…'));card.append(head,preview);const task=tasks.find(t=>t.deviceId===d.id&&active(t));if(task){const foot=node('div',undefined,'device-foot');foot.append(link(task.goal,'task/'+task.id));card.append(foot)}target.append(card)}} | ||
| function syncStreams(){const ready=new Set(phones.filter(d=>d.connected&&d.authorized).map(d=>d.id));for(const id of cardStreams.keys())if(!ready.has(id))cardStreams.delete(id);if(previewDevice&&!ready.has(previewDevice)){previewUrl='';$('deviceViewer').removeAttribute('src')}for(const id of ready){if(cardStreams.has(id)||previewPending.has(id))continue;previewPending.add(id);void api('preview',{deviceId:id}).then(value=>{if(phones.some(d=>d.id===id&&d.connected&&d.authorized)){cardStreams.set(id,value.url);if(previewDevice===id)previewUrl=value.url;deviceSignature='';render()}}).catch(()=>{}).finally(()=>previewPending.delete(id))}} |
There was a problem hiding this comment.
当连接五台或更多已授权设备时，syncStreams 会立即为每台设备创建预览 iframe，并让前四台长期占满 ScrcpyVideoStreams 默认的四个 source 槽位；此后若任务分配到第五台设备，其任务 viewer 会收到 stream_capacity_wait，无法提交首帧回执并在 30 秒后以 display_timeout 结束，即使该设备本身完全可用。应优先保留活动任务的流，并限制或回收空闲设备预览，而不是对 ready 集合中的所有设备常驻订阅。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b10ce543e0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| name: tool.name, title: 'OpenGUI Phone Tasks', description: tool.description, | ||
| inputSchema: { type: 'object', additionalProperties: false, properties: tool.properties, required: tool.required }, | ||
| outputSchema: { type: 'object' }, | ||
| annotations: { readOnlyHint: tool.name === 'opengui_list_tasks', destructiveHint: tool.name === 'opengui_run_task', idempotentHint: true, openWorldHint: tool.name === 'opengui_run_task' }, |
There was a problem hiding this comment.
idempotentHint: true 同样应用于 opengui_manage_task，但该工具的 steer 操作会把指令追加到排队任务或再次发送给正在运行的执行器，并不具备幂等性。当 MCP 宿主依据此提示在响应丢失或超时后安全重试时，同一补充指令可能被执行两次并导致重复手机操作。应对管理工具使用保守的非幂等提示，或将只读状态查询与变更操作拆分。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b3bfcb8f8c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
| async call(name: string, args: Record<string, unknown>, owner: string): Promise<unknown> { | ||
| if (process.platform !== 'darwin') throw new Error('Autonomous phone tasks currently support macOS only') | ||
| const runtime = await this.load() |
There was a problem hiding this comment.
在 .github/workflows/opengui-codex-ci.yml 的 ubuntu-latest 作业中，pnpm check 会执行未按平台跳过的 phone-agent.spec.ts 和 task-service.spec.ts；这些测试使用假硬件调用 TaskHost.call()，其中 startTaskService() 还会在监听套接字前调用该方法做 bootstrap。这个无条件平台检查会让所有这些调用立即抛错，因此 Linux CI 会稳定失败，甚至无法启动测试服务。应把平台限制放到真实生产适配器/客户端边界，或为假硬件测试提供显式绕过并相应跳过仅限 macOS 的用例。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 06012039c5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| phoneLifetimes.delete(bound.id) | ||
| }, { once: true }) | ||
| } | ||
| sendFrame(socket, { id, result: await tasks.call(message.name, args, owner) }); return |
There was a problem hiding this comment.
当 WorkBuddy 会话在 tasks.call() 等待共享服务响应期间被中止或超时时，这个分支在 requests.set(...) 之前直接调用并返回，因此 BrokerClient 随后发送的 cancel 帧无法在 broker 的 requests 映射中找到该请求。共享服务仍可能在调用方已经收到取消结果后接受 opengui_run_task，并让这个持久任务继续操作手机；应像其他工具调用一样登记控制器并向任务服务传播取消，或至少把结果明确标记为未知并提供恢复查询。
Useful? React with 👍 / 👎.
| name: tool.name, title: 'OpenGUI Phone Tasks', description: tool.description, | ||
| inputSchema: { type: 'object', additionalProperties: false, properties: tool.properties, required: tool.required }, | ||
| outputSchema: { type: 'object' }, | ||
| annotations: { readOnlyHint: tool.name === 'opengui_list_tasks', destructiveHint: tool.name === 'opengui_run_task', idempotentHint: true, openWorldHint: tool.name === 'opengui_run_task' }, |
There was a problem hiding this comment.
这里仅把 opengui_run_task 标为 destructive，导致 opengui_manage_task 的 destructiveHint 为 false，但其 stop 操作会不可逆地取消正在运行的任务，steer/resume 也会改变后续执行。依赖 MCP 注解决定风险提示的宿主可能因此把这些操作当成安全读取；应保守地将管理工具标为 destructive，或把 status 与变更操作拆成不同工具。
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 91c56cbacb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| entry: { | ||
| ...Object.fromEntries(readdirSync('src').filter(name => name.endsWith('.ts')) | ||
| .map(name => [name.slice(0, -3), `src/${name}`])), | ||
| 'task-service': '../plugins/opengui/src/task-service-main.ts', |
There was a problem hiding this comment.
WorkBuddy 的 CI 和发布工作流先用 packages/device-runtime/build.mjs stage workbuddy 创建隔离目录，该目录只包含 workbuddy-plugin 和共享包，不包含 plugins/opengui；因此 npm run build 执行 tsdown 时无法解析这个新入口。该问题与已有评论指出的 packages/task-service 未复制相互独立：即使补齐共享包，这条跨宿主路径仍然不存在，所有隔离的 WorkBuddy 构建都会失败。请在 WorkBuddy 或共享包内提供入口，并从导出树中的路径引用它。
Useful? React with 👍 / 👎.
Use one shared package list for staging, hashing and source validation so new core packages cannot be omitted from isolated builds. Add red-green stage regression coverage for all three adapters and give WorkBuddy its own task-service entry with shared startup wiring instead of importing the Codex adapter.
The web profile disables HMR while its CLI requires registerConfig, and permissive upstream ranges now resolve a provider that removed that API. Restore exact-path HMR with providers compatible with the current SDK. Use the DSH adapter for the task-service entry, reject leftover package imports, and exercise installed task-service requests and browser registration from isolated CI exports.
2 participants
Problem and resulting behavior
Ref DMY-59. OpenGUI needs a local Web workbench that can configure a model, submit phone tasks, show automatic live previews, inspect steps/evidence and stop execution. This branch serves a direct loopback URL without a browser token, stores credentials in the local keychain, and shares device admission, durable task records and the task-service process across adapters. HTTP and HTTPS model addresses enter the same probe/save flow; URL syntax and HTTPS certificate checks remain enabled.
The independent Web execution path is implemented. Existing host-model tool paths remain available, but selecting host/BYOK modes in the shared Web service is not connected yet. Phone/browser mixed cards and the remaining PRD Session decisions are design work, not completed capabilities in this PR.
Build and DSH compatibility corrections
Isolated exports now use the same shared-package list as hashing and source validation, including task-service. WorkBuddy and DSH provide their own task-service adapter entries rather than importing Codex source. DSH CI builds and tests the isolated export, including its required public marketplace metadata.
DSH web profiles disabled HMR while their CLI required user-patch watching. Upstream HMR 1.0.18/1.0.19 also removed registerConfig within the accepted dependency range. The plugin restores the HMR entry with exact-path watchers and pins HMR 1.0.16/timer 1.1.3, compatible with the existing SDK. It does not suppress host errors or change the startup acceptance checks.
The legacy task-service entry reproduced four unresolved Pi/tar imports and failed when run without those external dependencies. The DSH-owned entry bundles them; validation rejects any remaining non-builtin task-service imports. Compatibility tests now start the installed task-service bundle in a private fixture and require protocol-2 ping/list responses before checking the host and browser client.
Ubuntu test-contract corrections
Fake-device task/workbench and broker tests now explicitly inject the supported host platform. Production still defaults to the actual OS and rejects unsupported autonomous phone hosts; new regressions verify Linux/Windows rejection before runtime initialization and actual-platform defaults. Shared-service fixtures forward that platform explicitly. Fake ADB and mirror lifecycle fixtures no longer select real executable assets from the test runner's architecture. No tests or assertions were broadly disabled.
Codex and WorkBuddy CI path filters now include all shared packages. WorkBuddy's original production audit exposed independent moderate advisories after its test failure was fixed; the lockfile updates only fast-uri 3.1.7 → 3.1.8 and ip-address 10.7.0 → 10.7.2, without changing the audit threshold.
Validation
On macOS arm64 with Node 22.23.3 and Chrome, using isolated source exports and installed archives:
Remaining acceptance and CI
Keep this PR Draft. Complete installed-host handoff, shared installer/upgrade matrix and remaining product acceptance before merge; do not treat the mixed-layout design or shared Web dual-mode proposal as implemented.
CI was queried once at the start of this repair, without watch. At head 62d179c, Codex macOS arm64/Intel and WorkBuddy macOS packaging passed; Codex Ubuntu had 11 failures and WorkBuddy Node 24 had 1 failure at the macOS-only TaskHost guard (Node 22.19 was cancelled). Those test-contract failures are corrected and locally verified above. New-head GitHub CI results have not been collected; this is not a claim that the remote matrix is green.
Local DSH compatibility validation covers macOS arm64/Node 22.23.3 only. The latest adapter repair additionally covers real Ubuntu arm64 containers, not the GitHub x64 runner. Windows, macOS Intel, exact Node 22.19.0/22.23.2 and installed-host/device execution were not repeated locally. No release was published; no model credentials or unrelated research/cache files were committed. This PR does not close DMY-59.