feat(automator): CONDUCTOR_WORKER_ISOLATION=thread worker isolation mode

[kowser-orkes]

Problem

Inside Firecracker microVM guests (and similar minimal environments), multiprocessing's two
fork+exec bootstraps reliably fail:

• a bare Process() under spawn exits 255 before its target runs;
• a bare Queue() breaks the resource_tracker handshake (silent corruption at 1 vCPU,
  BrokenPipeError [Errno 32] in resource_tracker._send at 4 vCPUs).

TaskHandler hard-wires both (Queue() in __init__, one Process per worker), so every worker
dies at bootstrap and tool tasks are never polled (pollCount: 0 forever). Ruled out by direct
reproduction: jailer/seccomp (bare-Firecracker repro), memory (256 MB→1 GB identical), musl/Alpine
(chroot repro passes), PID-1 reaping (unshare --pid repro passes). Identical work on
threading.Thread succeeds every time.

Fix

A first-class worker isolation mode, selected by env var:

CONDUCTOR_WORKER_ISOLATION	Behavior
process (default / unset)	exactly today's: multiprocessing.Process under spawn
thread	every worker a threading.Thread; log relay a plain queue.Queue

• The existing Windows-only Process→Thread shim moves from
  conductor.ai.agents.runtime.worker_manager to a new
  conductor.client.automator.worker_isolation (the layer that owns the worker bootstrap, and
  usable by plain TaskHandler deployments without the agents extra). It gains the half Windows
  never needed: task_handler.Queue → queue.Queue.
• TaskHandler.__init__ applies it as its first statement (must precede _setup_logging_queue(),
  which creates the Queue and starts the logger Process).
• _patch_conductor_use_threads_on_windows remains as a delegating alias; the
  platform.system() == "Windows" call site is unchanged.
• Invalid values warn once and fall back to process (fail-open).

This is not a start-method change. spawn stays the sole start method (per the removal of
CONDUCTOR_MP_START_METHOD in 1b03c812 — fork's macOS SIGSEGV / held-lock hazards). Fork also
wouldn't fix this: resource_tracker is fork+exec'd regardless of start method. Thread mode
bypasses multiprocessing entirely.

Tradeoffs (thread mode only, documented in the module docstring)

No per-worker force-kill (terminate/kill are no-ops, shutdown cooperative — in the target
environment the microVM is the isolation boundary and the platform hard-kills at VM level);
CPU-bound workers share the GIL (tool workers are I/O-bound); signal.signal no-ops off the main
thread instead of raising. Known cosmetic issue: duplicate console log lines in thread mode
(double handlers once the relay is same-process) — not a correctness problem.

Testing

• 16 new unit tests (tests/unit/automator/test_worker_isolation.py + alias/gate tests in
  tests/unit/ai/test_worker_manager.py); all patched globals snapshot/restored per test.
• Full suites green: tests/unit 2582 passed / 1 skipped; tests/backwardcompatibility +
  tests/serdesertest 1077 passed.
• E2E in the AgentSpan Firecracker stack via a local wheel, with the previous agent-level
  monkeypatch removed and only the env var active: tool polled + executed, workflow COMPLETED.
  Negative control (env var removed): the hang reproduces — the switch is the active ingredient.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
src/conductor/ai/agents/runtime/worker_manager.py	80.00% <100.00%> (+14.64%)	⬆️
src/conductor/client/automator/task_handler.py	83.23% <100.00%> (+4.53%)	⬆️
src/conductor/client/automator/worker_isolation.py	100.00% <100.00%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.