Skip to content

GitHub Actions: per-test results from the JUnit reports, and shorter action references - #194

Merged
drowaudio merged 2 commits into
v2from
feature/actions-junit-summary
Oct 13, 2026
Merged

drowaudio merged 2 commits into
v2from
feature/actions-junit-summary

Conversation

@drowaudio

@drowaudio drowaudio commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #193.

Per-test results

The validate and acceptance actions wrote a JUnit report per plugin / config but only reported pass / fail from the exit code.

  • common/junit.awk (POSIX awk, no dependencies) reads each report for the test count and the name and message of every failed test.
  • Job summary: the table gains a "Tests" column (17, or 2 of 17 failed), followed by a "Failed tests" list grouped by plugin / config.
  • Annotations: the one error annotation per failed plugin / config now lists its failed tests and messages. Still one per item, as GitHub caps error annotations at 10 per step.
  • When there is no report (an array config, or a crash before one is written) the reason shown is the action's own message or the exit code.
  • New report-dir input on both actions, so the reports can be written inside the workspace.
  • README: where results appear, plus how to publish every test as its own PR check with a reporter action (needs checks: write, so not available to fork PRs).

Pass / fail is still decided by the exit code only.

Shorter references

The actions are now Tracktion/pluginval/validate@v2 and Tracktion/pluginval/acceptance@v2 (were Tracktion/pluginval/.github/actions/...). Only each action.yml moved to the repo root (validate/, acceptance/); the scripts, common/ and the README stay in .github/actions/ and are found relative to $GITHUB_ACTION_PATH. Nothing has been released against the old paths.

Testing

  • All local cases (pass, wrong reference, array config, missing reference with and without recording, empty dir, no plugin, plugin glob, missing plugin, empty list) run under macOS bash 3.2 / BSD awk.
  • The dogfood steps in acceptance.yml now use ./validate and ./acceptance, so they exercise the new locations and the parser on Linux, macOS and Windows. They don't assert on the summary text.
  • The reporter-action example in the README hasn't been run.

@drowaudio drowaudio changed the title GitHub Actions: show per-test results from the JUnit reports GitHub Actions: per-test results from the JUnit reports, and shorter action references Oct 13, 2026
@drowaudio
drowaudio merged commit 0175cbf into v2 Oct 13, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant