feat(v1): persist harnesses over native runtime processes

[hallerite]

Summary

This is the native-process counter-proposal to #2116 and #2141:

• add rollout-scoped HarnessSession ownership to the harness lifecycle
• add a small RuntimeProcess contract: byte-stream stdout/stderr, stdin, wait, terminate, and kill
• implement it for subprocess, Docker, Modal, and Prime runtimes
• keep one RLM ACP runner alive across interaction turns over framed stdio
• keep sandboxed tool installation scratch space and uv caches on the runtime workdir instead of a VM's small /tmp tmpfs
• remove the ACP sidecar, Unix socket, probes, background service, and their custom networking lifecycle

Design

Rollout
  └─ HarnessSession
       └─ RuntimeProcess
            └─ ACP runner
                 └─ RLM ACP session

Persistence is a runtime capability rather than a harness-specific network service. The default HarnessSession adapts existing launch() / resume() implementations, so only stateful harnesses need a specialized session.

Prime live processes are deliberately gated to runtime.prime.vm=true. Container sandboxes fail with an actionable error because Prime containers are being deprecated.

Dependency

Depends on PrimeIntellect-ai/prime#819 for AsyncSandboxClient.open_process(). Until that SDK change is released, the Prime adapter reports that the installed SDK is too old; the other runtimes work independently.

Validation

• uv run pytest tests/v1/ -q
• full contributor suite passed after rerunning the sandbox-restricted socket/semaphore modules with normal host permissions
• uv run ruff check --fix .
• uv run pre-commit run --all-files
• real Prime VM SDK acceptance covers streaming stdout/stderr, delayed stdin, exit codes, signals, and clean shutdown
• live rlm-acp-in-prime-vm E2E passes: one ACP/RLM process survives two turns, calls its MCP tool on the resumed turn, and tears down cleanly

Note

Persist harness sessions across turns using live native runtime processes

• Introduces HarnessSession in harness.py as a rollout-scoped abstraction that replaces per-turn harness.run() calls in rollout.py; default implementation wraps existing launch/resume semantics.
• Adds RuntimeProcess abstract class and Runtime.open_process() to runtimes/base.py, with concrete implementations for Docker, Modal, Prime (VM-only), and subprocess runtimes.
• Rewrites acp/runner.py with a LiveACPSession that keeps a single ACP agent process alive across turns, communicating via a framed 8-byte size-prefixed stdio packet protocol; one-shot runs use a separate run_once path.
• Overrides RLMHarness.session() in harnesses/rlm/harness.py to return an ACPHarnessSession backed by a persistent rlm --acp process; per-rollout state is isolated under a trace-specific RLM_HOME.
• Risk: open_process is unsupported on Prime without a VM sandbox or a recent SDK version, raising SandboxError explicitly; Modal falls back to terminating the entire sandbox if PID acquisition times out.

Changes since #2249 opened

• Added Harness.run() coroutine method for one-shot segment execution and refactored result validation into Harness._check_result() helper method, with HarnessSession.turn() updated to delegate to the new validation helper [7b91006]
• Modified RLMHarness.session() to conditionally use base Harness.session() implementation when runtime does not support live processes [7b91006]
• Added supports_live_processes property to Runtime base class and PrimeRuntime to indicate live process capability support [7b91006]
• Added prime-sandboxes package as a git-based dependency source [e69250e]
• Added E2E test execution step for Prime VM persistent RLM scenario [e69250e]
• Reordered CI test execution steps to run Prime VM persistent RLM E2E before live v1 E2Es [14ef360]
• Removed CI step executing the Prime VM persistent RLM end-to-end test [0475b23]
• Added end-to-end test step for Kuhn Poker with RLM in Prime VMs to GitHub Actions workflow [c583beb]
• Added new CI workflow step for modeled user simulation with RLM and MCP [adb6bd6]
• Updated Kuhn Poker RLM evaluation step configuration and validation [adb6bd6]
• Modified Kuhn Poker test step validation logic and execution conditions [efe7ff6]
• Added unconditional execution to modeled user simulation test step [efe7ff6]
• Removed the 'Run Kuhn Poker with RLM in Prime VMs' step from the GitHub Actions workflow, including its evaluation invocation and Python validation [65d809b]
• Modified validation assertions in the 'Run modeled user simulation with RLM and MCP' step to enforce stricter success criteria and validate specific node content [65d809b]
• Removed GitHub Actions workflow step for modeled user simulation [17338f2]
• Renamed VerifiersClient class to VerifiersACPClient and updated all instantiation and usage sites [af19253]
• Converted private utility functions to public functions by removing underscore prefixes [af19253]

^{Macroscope summarized 5f7dc2d.}

---

Note

High Risk
This changes core rollout/harness lifecycle and long-lived sandbox process teardown across Docker, Modal, and Prime VMs; mis-handled sessions could leak processes or drop scoreable trajectories on close failures.

Overview
Rollouts now open a rollout-scoped HarnessSession and drive turns via turn() instead of spawning a fresh harness process each segment. The default session still maps to launch() / resume(); stateful transports override Harness.session() to keep live state.

RuntimeProcess and Runtime.open_process() add stdin/stdout streaming with graceful shutdown across subprocess, Docker, Modal, and Prime (VM-only). ACP gains a long-lived stream runner with framed stdio packets plus ACPHarnessSession, while one-shot segments use once.

RLM switches to rlm-harness with --acp, using a persistent ACP session when the runtime supports live processes and falling back otherwise. Sandbox tool installs on Prime VMs use the runtime workdir for source/build/uv cache instead of small /tmp tmpfs. prime-sandboxes is pinned from git for open_process. E2E coverage adds RLM ACP resume on Docker and Prime VM.

^{Reviewed by Cursor Bugbot for commit af19253. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces a new feature for persistent harness sessions across runtime processes, including new abstractions (HarnessSession, RuntimeProcess), a packet-based communication protocol, implementations across multiple runtimes, and changes to the rollout loop. The scope and complexity of these runtime behavior changes warrant human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[hallerite]

Integration verification: prime-sandboxes is temporarily pinned to Prime main commit 892bea7, which contains AsyncSandboxClient.open_process.

The targeted rlm-acp-in-prime-vm E2E passed against a real Prime VM and live model in Actions run 31001845264 (commit 14ef360b7). It exercised two ACP interaction segments over the persistent RLM process and retained tool access after resume.

The temporary workflow step was removed after the successful run. The immutable SDK pin remains until prime-sandboxes 0.2.35 is released; then it can be replaced with prime-sandboxes>=0.2.35 and a registry lock.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit efe7ff6. Configure here.}

[hallerite]

Real-workload validation update (the temporary workflow steps were removed afterward):

• Kuhn Poker, both seats RLM on Prime VMs: Actions run 31005016449. The diagnostic step passed 3/3 hands and 6/6 traces with clean user_closed termination, no trace errors/forfeits, and zero-sum payouts. Histories were check-bet-call (seed 0) and check-bet-fold (seeds 1–2). The same harness sessions resumed after opponent turns; one player used six sampled RLM nodes across its decisions. An earlier run's seed-0 failure was separately confirmed as the model spending all eight internal RLM turns deriving a CFR strategy (rollout stopped: max_turns), not a process/session failure.
• Modeled user + colocated MCP, assistant RLM on a Prime VM: Actions run 31005761673. The diagnostic step passed: the modeled user made the request and then returned ###DONE###; the assistant executed three internal RLM steps, invoked await echo_back.run(message="hello world") through persistent IPython, observed the hidden hello world [ok-7f3] stamp, earned reward 1.0, and both traces closed cleanly with user_closed.

Together these cover environment-level interleaving/session resume (Kuhn) and generic modeled-user + nested MCP composition (user-sim) end to end over the pinned native Prime process API. The PR no longer contains the temporary CI diagnostics.