Skip to content

mwm and RRSI: refuse a learner backend other than Codex or Claude Code at setup - #92

Draft
iamziyuzhao wants to merge 1 commit into
mainfrom
fix/mwm-rrsi-backend-check
Draft

iamziyuzhao wants to merge 1 commit into
mainfrom
fix/mwm-rrsi-backend-check

Conversation

@iamziyuzhao

Copy link
Copy Markdown
Collaborator

Change

evolver.base: pi loads for every AgentEvolver-based method, but two of them cannot drive a Pi session:

  • mwm builds its learner's session settings without provider (MwmEvolver._settings), so a Pi session ran
    without its bound model. Its texts, homes and sign-ins are Codex's or Claude Code's.
  • RRSI runs its proposer on the bound model, but keeps its roles read-only through Codex's own sandbox mode
    (codex_sandbox: read-only) or Claude Code's tool settings (allowed_tools, text_only). Pi reads none of these,
    so on Pi the roles ran with all of Pi's tools.

Both now refuse an evolver.params.backend other than codex or claude in setup, before the first step.

Found while adding DeepSeek Harness (#90), whose dsh backend these methods cannot drive either, and split out of
it so that #90 changes only dsh. #90 refuses dsh on the same two lines; whichever merges second keeps this check,
which covers both.

Validation

  • tests/methods/mwm/test_mwm_evolver.py::test_a_learner_that_is_not_codex_or_claude_code_is_refused_at_setup and
    tests/methods/rrsi_adapter/test_rrsi_adapter.py::test_setup_checks_what_every_step_needs (the rrsi dependency
    group installed): backend: pi is refused at setup.
  • Each test fails with its check undone (each undone in turn in a copy of the checkout).
  • Full suite, Linux container (Ubuntu 24.04) as root, --integration, with the rrsi group: 2 failed, 1230
    passed, 80 skipped. On main the same run gives 2 failed, 1229 passed, 80 skipped: the same two tests, which fail
    as root (test_isolate::test_a_rollout_sees_what_is_declared_and_nothing_else,
    test_ahe_hook_runner::test_logs_that_cannot_be_written_lose_the_line_not_the_call).

Review

  • Tests and documentation cover the affected behavior, where applicable. Tests: see Validation. Documentation:
    unchanged, already right: docs/methods/mwm.md names its learner as Codex or Claude Code, and
    docs/methods/rrsi_adapter.md its proposer and roles as codex or claude sessions.
  • Commands, defaults, and public interfaces are consistent. Codex and Claude Code configs are unchanged; a Pi
    learner for these two methods is refused with the backends they take.
  • Shared logs and examples contain no credentials or private data.
  • Documentation-only: N/A.
  • Core changes: none; two methods' setup.
  • Failures reported accurately: a backend the method cannot drive fails the run at setup with a message
    naming the backends it takes, instead of running each step without its model or its read-only roles.
  • Test checklist: each test is in its method's group, named for the behaviour, and fails without its check.
  • This description: TBD.
  • Independent review (post-PR audit): pending.

🤖 Generated with Claude Code

…e at setup

mwm ran a Pi learner without its bound model (its session settings carry no provider), and
RRSI's roles, kept read-only through Codex's own sandbox mode or Claude Code's tool settings,
ran with all of Pi's tools. Both now refuse an evolver.params.backend other than codex or
claude before the first step. Each test fails without its check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant