fix(swe-smith): parse pytest statuses that include ANSI color

[Xiang Chucheng (xccElephant)]

Summary

SmithAgent grades SWE-smith rollouts by regex-matching PASSED/FAILED in pytest -rA output. When pytest emits ANSI color (a repo addopts = --color=yes, PY_COLORS=1, or a TTY), those codes glue themselves to the status token:

ESC[32mPASSEDESC[0m tests/test_demo.py::test_ok

parse_test_statuses() never sees a bare PASSED, so a fully green suite is scored 0/N. This is a false negative in the reward, not a failed test.

This PR:

• strips ANSI escape sequences before parsing status lines
• runs pytest with --color=no so the evaluator does not depend on TTY or repo color settings

This does not change SWE-bench Verified scoring, which uses the official swebench harness rather than this parser.

Validation

PYTHONPATH=. python -m pytest tests/examples/test_swe_smith_agent.py -q
# 27 passed
uvx ruff check examples/swe_smith/agents/smith_agent.py tests/examples/test_swe_smith_agent.py
uvx ruff format --check examples/swe_smith/agents/smith_agent.py tests/examples/test_swe_smith_agent.py

Reproduced locally on cantools SWE-smith instances whose pytest log showed 185 passed while the parser reported FAIL_TO_PASS 0/60. After stripping ANSI, all 185 nodes parse as PASSED.

AI assistance: Cursor Grok 4.6 helped investigate, implement, and validate this change. I reviewed the diff and ran the checks above.

[Copilot]

🟢 Approval recommended

The reviewed changes address the parsing issue and include regression coverage.

Pull request overview

Fixes SWE-smith pytest status parsing when ANSI color codes surround test results.

Changes:

• Strip ANSI escape sequences before parsing statuses.
• Force pytest color output off during evaluation.
• Add regression tests.

File summaries

File	Description
tests/examples/test_swe_smith_agent.py	Tests ANSI parsing and pytest command construction.
examples/swe_smith/agents/smith_agent.py	Implements ANSI stripping and disables pytest colors.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Xiang Chucheng (xccElephant)]

@microsoft-github-policy-service agree

[Zhiyuan He (hzy46)]

Thank you for this contribution.

Please remove the test here as we have already had a lot of tests.

I am checking with our team member about this issue. If confirmed, we will merge your PR.

[Xiang Chucheng (xccElephant)]

Thanks for the review. I have removed the extra tests as requested.

The remaining change is only in examples/swe_smith/agents/smith_agent.py: strip ANSI before parsing pytest statuses, and run pytest with --color=no.

[Ldemon (ldemon2333)]

accept pr for --color=no