Conversation
added 14 commits
September 13, 2026 11:24
…nits (uninstalled)
…stop in run_one - _newest_score globbed */*.score.json but runs write <run>/score.json, so every nightly result fell through to _missing_score.json and was recorded 'failed' regardless of outcome (49/49 false failures on 2026-09-14). - The dispatch loop queued all futures at window open, so the WINDOW_STOP check never bit and workers drained past 07:30. run_one now defers any queued task that starts after the window closes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR body (used with
gh pr create --body-fileafter fork push is authorized)Title:
feat: DSH (DeepSeek Harness) backend + nightly window schedulerAdds a fifth agent harness — DSH / DeepSeek Harness — alongside OpenClaw, Claude Code, Codex, and Hermes. Unlike the OpenRouter-backed harnesses, the DSH backend runs its solver against a self-hosted LiteLLM gateway, and ships with an opt-in nightly-window scheduler for running the full 60-task suite in a fixed low-traffic window with a resumable ledger.
What's in it
src/agents/dsh/—DshAgentimplementingBaseAgent: starts the standard task container, writes a per-task DSH config patch (--patch, pinning provider/model/reasoning rung), runsdsh --profile benchheadless one-shot per task, collects usage from the session transcript. No gateway process (expects_gateway=False), same background-exec/timeout pattern as the other backends.src/agents/dsh/compat_transcript.py— converts DSH's compressed session events into the OpenClaw transcript format the existing grading pipeline already consumes (same precedent as the Hermes adapter). Grading and judge code are untouched.docker/Dockerfile.dsh+docker/dsh-settings.yaml+docker/verify_image.sh— solver image on top ofwildclawbench-ubuntu:v1.3(pins the dsh CLI, materializes a headlessbenchprofile, bakes the gateway provider definition) plus an acceptance gate that builds, checks profile/rung/model, probes the gateway, and does one live round-trip.script/nightly_lib.py— window-bounded (host-local start/stop), resumable per-task ledger (atomic writes, failed rows retried next night), bounded parallelism, container sweep at exit.WCB_BACKEND=openclawreuses the same machinery for matched-control runs against the same endpoint viamy_api.lb.json.dshinrun.sh/run_batch.py/ cli choices; README section;.env.examplekeys.tests/— 25 unit tests (transcript conversion against a captured fixture, rung validation, patch generation, dispatch wiring, nightly ledger/window/commands). All green; upstream tests untouched.Evidence
docker/verify_image.sh).01_Productivity_Flow_task_1_arxiv_digest): score.json graded in-container with the stock pipeline,usage.jsoncollected real token counts (request_count=39, 873k tokens), compat transcript consumed by the judge path. Write-up:docs/superpowers/plans/m0-results.md.Design notes
usage.jsoncarriesself_hosted: trueandcost_usd: 0.0(matching the existing self-hosted precedent rows — no dollar figure is derivable from a private gateway).--thinkingacceptsoff|low|medium|xhigh(defaultxhigh) and is validated before any container side effects; unsupported values fail fast.host.docker.internalautomatically; both addresses are configurable via.env.systemd/are examples and are not installed by anything in this PR.Not included