Skip to content

feat(studio): a suggested fix and a reason under a failed see or tap step - #527

Open
pbertsch wants to merge 6 commits into
mainfrom
feat/studio-failure-triage-main
Open

pbertsch wants to merge 6 commits into
mainfrom
feat/studio-failure-triage-main

Conversation

@pbertsch

@pbertsch pbertsch commented Nov 5, 2026

Copy link
Copy Markdown
Member

Summary

  • A failed see or tap on a text or an id shows, under the step in Results and in studio_run_state, a rule-based suggestion: for a text, the wait until "<text>" appears line to try before the step, plus the reason and the time the step ran.
  • The rules run without AI and send nothing over the network. Steps that are not see or tap are not covered.
  • The runner's step event says whether a selector is an id, so an id selector gets a reason only.
  • The triage rows join the test's rows, so the failure-first grouping from the run-scope change keeps them with their test.
  • CHANGELOG, the Studio page and the MCP table are updated.

Test plan

  • go vet ./... and go test ./... in studio, including the triage tests
  • go vet and go test ./internal/mcp/... ./internal/runner/... at the repo root, including the step-event selector test
  • Frontend: npm run build, typecheck:device-view, test:device-view (116), test:probescript (11), test:run-scope (10), check:probescript, and the triage tests in triage-tests (3)
  • CI green on this PR
  • On a device: a failed see on a text shows a wait until suggestion and a reason under its step

🤖 Generated with Claude Code

Results can explain a failed see or tap on a text or an id without AI. The rules are
deterministic and need no network: a failed see or tap on a quoted text gets the line
wait until "<text>" appears as its fix and a reason with the time the step ran; an id
selector gets a reason only; any other failure text is shown as the reason. Steps that
are not see or tap are not covered.

The suggestion is carried on RunStep.Triage, so the Results pane and studio_run_state
read the same data.
The finished steps of studio_run_state now carry the rule-based triage of a failed
selector step. The tool description says so, because agents read the description to
know the field is there.
…lector step

The Results pane shows the triage of a failed see or tap below its error: a suggested
fix line when the rules have one, then the reason. The rows come from triageRows (a
pure function, tested in triage-tests) so the rendering rules have a test without a DOM.

The frontend suites pass: device-view 116, probescript 11, triage 3. The frontend
builds with tsc and vite.
…e MCP table

The Unreleased changelog, the Studio Results section and the Studio row of the MCP tool table
now say that a failed see or tap carries a rule-based suggested fix and a reason, that it runs
without AI, and which steps it does not cover.
The step text quotes both a text and an id, so a consumer cannot tell them apart from the
text. StepEvent.SelectorIsID is true for an action step whose selector is an id. The CLI
ignores it; the Studio triage uses it to avoid suggesting a text wait for an id.
…eason only

The runner writes an id selector in the same quoted form as a text, so the triage parsed
an id as a text and suggested wait until "<id>" appears, a wait that would never match.
The triage now takes the step event's id flag: an id gets the reason alone, and a text
keeps the wait line. Tests cover both, and the RunStep wiring with the flag.
@pbertsch
pbertsch requested a review from a team as a code owner November 5, 2026 04:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant