Skip to content

fix(report): a step skipped by its when: is reported skipped - #275

Open
luka-vladika-happening-xyz wants to merge 1 commit into
devicelab-dev:mainfrom
luka-vladika-happening-xyz:fix/when-skipped-is-skipped
Open

luka-vladika-happening-xyz wants to merge 1 commit into
devicelab-dev:mainfrom
luka-vladika-happening-xyz:fix/when-skipped-is-skipped

Conversation

@luka-vladika-happening-xyz

Copy link
Copy Markdown
Contributor

A runFlow or runScript whose when: is false does not run, but the report said passed, the same as a step that ran, so a reader of the report (the HTML report, or a tool reading the JSON) could not tell a skipped branch from one that ran. Maestro reports such a step as skipped: runFlowCommand and runScriptCommand in Orchestra.kt throw CommandSkipped, and its console prints SKIPPED.

The step is now skipped in the report, at the top level and inside a runFlow, retry or repeat, and counts as skipped instead of passed.

The console still prints it with a check mark: OnStepComplete now gets passed for any step that did not fail, so a skipped step is not printed as a failure. That also applies to a step skipped by its platform:, which was printed with a cross.

TestWhenFalseIsReportedSkipped runs a flow with a false when: on a runScript and a runFlow, at the top level and inside a runFlow, and checks each command's status in the flow's JSON and the flow's step counts. On main it fails: every step is passed, 5 passed and 0 skipped. Checked on an Android emulator: maestro-runner and Maestro 2.11.0 now mark the same steps skipped. go test -race ./... passes apart from the CDP package, which hangs launching Chrome on this Mac and also does on main; golangci-lint v2.13.2: 0 issues.

A runFlow or runScript whose when: was false did not run, but the report
said passed, the same as a step that ran, so a reader of the report could
not tell the two apart. Maestro reports such a step as skipped (it throws
CommandSkipped in Orchestra.kt's runFlowCommand and runScriptCommand).
The step is now skipped in the report, at the top level and inside a
runFlow, retry or repeat, and counts as skipped instead of passed.

The console still prints it with a check mark: OnStepComplete gets passed
for any step that did not fail, so a skipped step is not printed as a
failure. That also applies to a step skipped by its platform:, which was
printed with a cross.
@codecov

codecov Bot commented Nov 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants