Repository navigation
Conversation
Adds a skill, a Mutant Must Fail resource and a stdlib kill ledger so a new or changed test is shown to fail when its check is broken. An empty ledger reports rate 0.0, never a full kill. Verified: pytest plugins/rf-core/tests/test_test_quality_skill.py, 4 passed; with the ledger broken, 3 failed. build-catalog --check, governance, validate-skills, plugin references and docs voice pass.
Add the rf-core skill rf-run-digest: read a short digest of a Robot Framework run (summary, failures grouped by signature) before output.xml. Ships digest_listener.py and run_digest.py (stdlib only), a pytest that runs the selftest and checks summary and grouped failures on a tiny output.xml, a trigger query set and one narrow eval task. Bundle regenerated with build-catalog.py. Verified: rf-core pytest 332 passed; the new summary test failed with run_digest.py broken; build-catalog --check, check-governance, validate-skills --channel all, check-plugin-references, check-docs-voice and check_content_rules all exit 0.
The digest listener rewrote output.xml in place with rebot --removekeywords PASSED on every run, undocumented. It now does so only when given the shrink listener argument (digest_listener.py:shrink). SKILL.md documents the argument and what it removes. Both skill copies are identical. Verified: pytest -q plugins/rf-core/tests/test_run_digest_skill.py (log pytest-blocker1.log in the session scratchpad): 4 passed, 1 failed. The failing test is the body-redaction test fixed in the next commit; the two listener tests failed before this change.
The failures view printed the last request body (sent:) and response body (body:) of each failed test, so tokens and passwords landed in digest.txt. Bodies are no longer printed; the request line and status stay. Both skill copies are identical. Verified: pytest -q plugins/rf-core/tests/test_run_digest_skill.py (log pytest-blocker2.log in the session scratchpad): 5 passed. The body test failed before this change with the token in the output.
failures and trace now print a screenshot: line for failed tests, taken from the last <img src> the test logged (Browser, SeleniumLibrary, AppiumLibrary, MaestroLibrary). norm() drops the Playwright "Call log:" tail, cuts absolute file paths to their basename and replaces 32+ hex session ids with <id> before grouping. SKILL.md names the libraries. The bundles/rf-agentskills copies match the plugin files (cmp). Verified (log: scratchpad/digest-pytest.log, written this run): - pytest plugins/rf-core/tests/test_run_digest_skill.py: 6 new tests red before the change, then 12 passed - run_digest.py selftest: ok - failures on a real Browser run: 1 TimeoutError group with its FAILURE_SCREENSHOT_1.png line. The Appium fixture is built from the library's message format, not a live run.
Mutant Must Fail now counts a kill only on an assertion failure; a
missing keyword, missing variable or wrong argument count is a survivor.
kill_rate keeps the latest ledger row per test and operator. SKILL.md
mutates the value the test checks, imports the resource through
${CLAUDE_SKILL_DIR} and runs scripts with uv run. Trigger set grows to
8 positive and 8 negative queries with the sibling and look-alike
quotas; the narrow task targets tests/example.robot and adds robot_pass.
Tests use 3-of-4 data and assert the exact rate.
Verified: uv run --frozen python -m pytest -q
plugins/rf-core/tests/test_test_quality_skill.py
tests/eval/test_trigger_sets.py: red 6 failed, 29 passed (../red.log),
green 35 passed (../green.log). selftest ok. Plugin and bundle copies
identical (cmp).
SKILL.md calls the scripts through uv run and ${CLAUDE_SKILL_DIR}.
run_digest.py drops the fingerprint, body capture and unwrap helpers,
uses argparse, and moves the selftest to digest_selftest.py. The
listener reads DIGEST_MAX_PER_GROUP inside a try. Trigger set meets
the 8/8 and sibling quotas; the narrow task targets tests/example.robot
and adds a robot_pass grader.
Verified (logs in scratchpad mksq/digest-logs): red.txt 3 failed,
11 passed before the fix; green.txt 42 passed for
test_run_digest_skill.py and tests/eval/test_trigger_sets.py; selftest
ok; failures output on cs2369-r1 output.xml matches before.txt apart
from the removed fp tag.
3 of 6 tasks
This was referenced Nov 15, 2026
…ndled parser The skill now points to robotcode results summary/show/stats/log/diff instead of its own output.xml parser and listener. Removes the scripts, their tests and the shrink option; updates the bundle copy, the command test map, the trigger set and the eval task.
…okup, JSON grouping)
Stack on rf-test-quality so the two merge one after the other: union of the skill lists and counts in the installer tests, robotcode declared in the skill compatibility field, skill version kept at the plugin version.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the rf-run-digest skill. After a run, the agent reads a short digest instead of output.xml or log.html.
Verified: plugin tests, 42 passed. The digest selftest passes on bare Python. The digest of a real Browser run matches its log. Catalog, governance, plugin references and docs voice checks pass.
What this changes
The digest gives the summary and groups failures by message. It reads Browser, SeleniumLibrary, AppiumLibrary, MaestroLibrary and RequestsLibrary failures. A listener writes digest.txt when the run ends. Its opt-in
shrinkargument rewrites output.xml without the keywords of passed tests.Review checklist
Tick each item, or write "n/a: " after it. Reviewers don't approve a PR
with an unticked item that has no reason (see CONTRIBUTING.md, "Review").
catalog.yamllists the libraries and version ranges the content was checked against, and the Robot Framework range. n/a: no new library,catalog.yamlis unchanged.robotexamples passrobot --dryrun; Python examples run.toolsand skillallowed-toolschanges are listed below; the security gate passes, and any new baseline entry has a reason. n/a: scanners not installed locally, and the CI security job waits for maintainer approval on fork PRstoolslist; noallowed-toolsentry grants unrestricted shell access. n/a: no subagent, and the skill sets noallowed-toolsSecurity-relevant changes
scripts/run_digest.py(new): stdlib only. Reads output.xml. Forshrinkonly, runspython -m robot.rebotwith an argument list, never a shell. No network.scripts/digest_listener.py(new): Robot listener, writes digest.txt to the output dir. Withshrink, rewrites output.xml in place. No network.scripts/digest_selftest.py(new): selftest on built-in samples.