Skip to content

fix(eval): sut-platynui fixture requires Python 3.12 - #19

Merged
manykarim merged 1 commit into
mainfrom
fix/platynui-fixture-python
Nov 16, 2026
Merged

manykarim merged 1 commit into
mainfrom
fix/platynui-fixture-python

Conversation

@manykarim

Copy link
Copy Markdown
Collaborator

Summary

eval/fixtures/sut-platynui/pyproject.toml declared requires-python = ">=3.10", but robotframework-PlatynUI==0.12.0.dev330 (and platynui-native) need Python 3.12+. uv resolves for every Python version the project allows, so it fails:

No solution found when resolving dependencies for split (markers: python_full_version >= '3.10' and python_full_version < '3.12')

In the GLM re-baseline, agents that ran uv run in the PlatynUI workspace hit this error and spent turns working around it. Found while answering robotcodedev/robotcode#689.

Changes

  • Raise the fixture's floor to >=3.12.
  • Add test_platynui_fixture_python_floor_matches_platynui to tests/eval/test_eval_content.py.

Testing

  • uv lock on a copy of the fixture: fails before, passes after. After the fix, uv run python -c "import PlatynUI.BareMetal" works.
  • pytest tests/eval/test_eval_content.py -k "platynui or spec_only or readme_layout": 11 passed. ruff check: clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FHEmzL3ErAhGQykKWDx2wP

PlatynUI 0.12.0.dev330 needs Python 3.12+, but the fixture declared
requires-python >=3.10. uv then can't resolve the project, so `uv run`
in an eval workspace fails and agents can't use the fixture's own
environment. Raise the floor to 3.12 and add a test for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHEmzL3ErAhGQykKWDx2wP
@github-actions

Copy link
Copy Markdown

Skill Evaluation: rf-desktop (narrow, treatment, N=3, z-ai/glm-5.3-flash)

Model: z-ai/glm-5.3-flash (provider openrouter, turn-limit factor 1.5)

OpenRouter spend

USD
Key spend during this job (includes concurrent jobs) 0.05
Harness estimate (OpenRouter list prices) 0.03
Credit remaining on the key 7.063268768999997

Gate

gate: REBASELINE-NEEDED (1 task(s) checked)
- [rebaseline-needed] narrow-platynui-calculator-01: task definition or fixture changed

rf-skill-eval report

Runs: 3 · pass: 3 · fail: 0 · incomplete: 0 · skipped checks: 0 · estimated cost (OpenRouter list prices): $0.03

Pass rate = runs whose gate result is pass / runs attempted. Deltas are treatment − baseline over outcome checks only.

Tasks

Task Skill Tier Arm Runs Pass rate Outcome pass Incomplete Input tok Output tok Turns Duration s Cost $ Flaky
narrow-platynui-calculator-01 rf-platynui narrow treatment 3 100% 100% 0 331526 3910 18.0 36.6 0.0107 (est.) no
narrow-platynui-calculator-01 rf-platynui narrow baseline (stored (rebaseline-needed)) 6 100% 100% 0 192866 3103 12.7 30.3 0.0064 (est.) –
narrow-platynui-calculator-01 rf-platynui narrow Δ T−B +0 pp +138660 +807 +5.3 +6.3 +0.0043

Skills

Skill Tasks Treatment outcome pass Baseline outcome pass Δ outcome pass Δ input tok Δ turns Δ cost $
rf-platynui 1 100% 100% +0 pp +138660 +5.3 +0.0043

Skipped / errored checks

None.

Incomplete runs

None.

Process checks (treatment arm, not in deltas)

  • narrow-platynui-calculator-01 loaded_rf_platynui_skill: 3/3

Adversarial tier (reported only, never gates)

None.

@manykarim
manykarim merged commit 56547f0 into main Nov 16, 2026
32 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant