Skip to content

feat: expose the analysis outcome and make reporting configurable - #3

Draft
javier-godoy wants to merge 7 commits into
masterfrom
feature/wip-action-required
Draft

feat: expose the analysis outcome and make reporting configurable#3
javier-godoy wants to merge 7 commits into
masterfrom
feature/wip-action-required

Conversation

@javier-godoy

@javier-godoy javier-godoy commented Sep 9, 2026

Copy link
Copy Markdown
Member

Refs FlowingCode/AddonsInternal#99

WIP commits are a valid, expected state of an unfinished branch, but they were reported
exactly like a genuinely invalid commit message, so the same red ❌ meant two different
things. Rather than settle that here, this branch exposes what the analysis found and lets
the caller decide how to report it (see FlowingCode/GithubActions#7).

Rebased on master

#7 landed the three commit-retrieval fixes, the typescript devDependency and the bundle
rebuild that this branch also carried. It has been rebased onto master, where those five
commits dropped out as already applied. The diff against master is now only what this branch
adds:

 README.md             | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++
 action.yml            | 17 +++++++++++
 dist/main/index.js    | 50 ++++++++++++++++++++++++++-----
 src/extractCommits.ts |  5 ++--
 src/main.ts           | 54 ++++++++++++++++++++++++++++-----

Changes

feat: expose the result of the analysis through outputs The action reported its findings only through the log and its own outcome, so a caller could not tell what was found. It now sets results, describing every commit as JSON.
feat: report invalid commit messages as annotations core.error instead of core.info for an invalid message, so the offending commit and the reason are visible on the pull request itself rather than only in the job log. Only the header is included, so the annotation is not filled with the body of the message.
feat: add enforce input for callers that report the outcome themselves An invalid commit message and a WIP commit both fail the check with the same red ❌, though only one of them is a defect, and a step cannot block a merge without looking like a failure. enforce: false makes the action a pure analyzer — it produces results, exits successfully and writes no annotations — so that a caller can report WIP commits as a check run of their own, which can conclude action_required and be both.
build: rebuild the dist bundle Rebuilt from the sources above.
feat: include the commit digest in the error annotations The annotation named the offending commit by its header alone, which does not identify it when the same message appears twice and leaves nothing to look the commit up by. The commits now carry their sha — it was discarded when they were mapped to their message — and the abbreviated digest precedes the header.
feat: expose the commit digest in the results output A caller that renders the outcome itself builds its annotations from results, so the digest is part of every result too, not only of the annotations this action writes.
build: rebuild the dist bundle Rebuilt from the current sources.

Interface

Name Type Description
results output The result for every commit, as a JSON array of {sha, header, level, reason}, where level is valid, wip or invalid.
enforce input Whether the action reports the outcome and fails on what it found. Default true.
invalid message WIP commit annotations
enforce: true (default) ❌ fails ❌ fails yes
enforce: false analysed only analysed only none

A value other than true or false fails the action rather than being read as the default,
so a misspelling is not silently a policy. enforce is one boolean rather than a mode
enumeration because there are only two behaviours left to choose between: whether the action
reports, or the caller does.

results is the only output: has-errors and has-wip are derivable from it, and the
Semantic Versioning level is already exported as the SEMVER_LEVEL environment variable. It
is set before the check fails, so it can be read from an if: always() step, and on every
path — including the one where the commits cannot be retrieved, where it is an empty array.
That path still fails even with enforce: false, because not being able to analyse anything is
a failure of the action itself rather than a finding about the commits.

The default changes nothing. With enforce: true a WIP commit still fails, exactly as on
master, so merging this branch moves no existing caller. FlowingCode/AddonsInternal#99 asks for
a pull request that is blocked and not shown as a failure, and a step cannot be both: its
conclusion comes from its exit code. Only a check run can, by concluding action_required, and
only a caller can create one. So the answer to #99 is enforce: false plus that check run —
which is what FlowingCode/GithubActions#7 does — and not a default that would leave a caller
without one merging unfinished code behind a green tick.

README.md gains a Work in Progress section documenting both, with a caller snippet that
reports WIP commits as a separate check run with the action_required conclusion.

Verification

npm test passes (3/3). The built bundle was also run against synthetic push payloads:

payload enforce outcome
one valid, one WIP unset (default) ::error::🚧 Work-in-Progress (WIP) commits found. They must be squashed before rebasing or merging., exit 1
one valid, one WIP false no annotation, exit 0
valid + invalid + WIP true ::error::🚩 9f8e7d6 broken message : The commit header is not formatted according to Conventional Commits., then ::error::🚫 According to the…, exit 1, SEMVER_LEVEL=2, and the sha of all three commits in results
valid + invalid + WIP false the same line as plain log output, no ::error::, exit 0
one valid, one WIP yes ::error::🚫 The enforce input must be true or false, not "yes"., exit 1

End to end in javier-godoy/test-repo

The test pull requests there run
FlowingCode/GithubActions/.github/workflows/check-commits.yml@feature/wip-action-required
the caller in FlowingCode/GithubActions#7, which calls this branch with enforce: false and
renders every conclusion itself. check-commits / check-commits is the job's own check run,
which GitHub creates and which carries no message; semver-alignment and wip-commits are
created by the caller and carry the verdict.

The annotations below carry the abbreviated digest ahead of the header: the caller reads sha
from results, so the commit that must be amended is named rather than only described.

results being ordered and per-commit is what lets the caller go further than this action
could. It tells a branch that may be squashed at merge time — exactly one non-WIP commit
followed by WIP commits — apart from one the author has to consolidate, purely from the order
and the levels, and the action knows nothing about that policy. Compare
javier-godoy/test-repo#11 with javier-godoy/test-repo#14 below.

javier-godoy/test-repo#13 — valid commit message — ✅
Check Conclusion Title
check-commits / check-commits ✅ success
semver-alignment ✅ success Version 1.2.3-SNAPSHOT is consistent with a PATCH change
wip-commits ✅ success No Work-in-Progress (WIP) commits

No annotations.

javier-godoy/test-repo#12 — invalid commit message — ❌
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ✅ success Version 1.2.3-SNAPSHOT is consistent with a NONE change
wip-commits ✅ success No Work-in-Progress (WIP) commits
[failure] 1 commit message does not follow the guidelines
[failure] 3d91269 broken commit message without a type : The commit header is not formatted according to Conventional Commits.
javier-godoy/test-repo#11 — a WIP commit, squashable at merge time — ⚠️
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ✅ success Version 1.2.3-SNAPSHOT is consistent with a NONE change
wip-commits ⚠️ action_required 🚧 Must squash WIP commit
[failure] 1 Work-in-Progress (WIP) commit must be squashed on merge
[warning] 🚧 A Work-in-Progress (WIP) commit was found. The branch can be squashed on merge, so the author does not need to consolidate it.

The action itself did not fail for the WIP commit: the warning is the action's, the failure is
the caller's, and wip-commits is what says which problem it was. The history here is one
non-WIP commit followed by a single WIP one, so the caller asks for a Squash and Merge rather
than sending the author back to rebase — contrast javier-godoy/test-repo#14, which has two
non-WIP commits and therefore cannot be collapsed at merge time.

javier-godoy/test-repo#14 — breaking change without a version bump — ❌ on two checks
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ❌ failure 🚫 Version 1.2.3-SNAPSHOT contains breaking changes
wip-commits ❌ failure 🚧 Must consolidate WIP commit
[failure] 1 Work-in-Progress (WIP) commit must be consolidated by the author — 🚫 Version 1.2.3-SNAPSHOT contains breaking changes
[warning] 🚧 A Work-in-Progress (WIP) commit was found. It must be consolidated by the author before merging.

Two independent problems, each on its own check run, and the job's failure message names both.

javier-godoy/test-repo#17 — every commit message invalid — ❌, level unknown
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ⚪ neutral The semantic versioning level is unknown
wip-commits ✅ success No Work-in-Progress (WIP) commits
[failure] 2 commit messages do not follow the guidelines
[failure] 3b62ec4 note what this branch is for : The commit header is not formatted according to Conventional Commits.
[failure] c577a04 point check-commits at the feature branch : The commit header is not formatted according to Conventional Commits.

The action exits 0 under enforce: false and the caller fails the job. No message could be
parsed, so the level is unknown rather than NONE and the version is not verified against it.

From a fork

javier-godoy/test-repo#19, javier-godoy/test-repo#20 and javier-godoy/test-repo#21 are the same
cases opened from FlowingCode/test-repo2. They confirm the one limitation of leaving the
reporting to the caller: a pull_request from a fork gets a read-only GITHUB_TOKEN whatever
the permissions: block asks for, so checks.create answers 403 there and only the job's own
check run exists. The annotations are workflow commands and still appear, and the job still
fails, so the outcome is reported either way — only the checks that carry it separately are
missing.

javier-godoy/test-repo#19 — fork, invalid commit message — ❌
Check Conclusion Title
check-commits / check-commits ❌ failure
[failure] 1 commit message does not follow the guidelines
[failure] b3bf7fb fork pr with an invalid commit message : The commit header is not formatted according to Conventional Commits.
javier-godoy/test-repo#20 — fork, WIP commits only — ❌
Check Conclusion Title
check-commits / check-commits ❌ failure
[failure] 2 Work-in-Progress (WIP) commits must be squashed on merge
[warning] 🚧 2 Work-in-Progress (WIP) commits were found. The branch can be squashed on merge, so the author does not need to consolidate it.
javier-godoy/test-repo#21 — fork, valid commit messages — ✅
Check Conclusion Title
check-commits / check-commits ✅ success

No annotations.

🤖 Generated with Claude Code

@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch 9 times, most recently from 890ad04 to 0f2f7a0 Compare September 10, 2026 12:33
@javier-godoy javier-godoy changed the title feat: add fail-on-wip input and has-wip output feat: expose the analysis outcome and make reporting configurable Sep 10, 2026
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch from 0f2f7a0 to 9cc1e86 Compare September 10, 2026 17:42
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch 2 times, most recently from b59c6d6 to df3ebd4 Compare September 10, 2026 17:58
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch 5 times, most recently from fd3bd27 to 03b07fc Compare September 11, 2026 16:09
The action reported its findings only through the log and its own outcome,
so a caller could not tell what was found. It now also sets results, which
describes every commit as JSON: its header, whether it is valid, WIP or
invalid, and the reason when it is invalid.

results is set before the check fails, so that it can be read from a step
that runs on failure, and on the path where the commits cannot be retrieved,
where it is empty. It is the only output: has-errors and has-wip are
derivable from results, and the Semantic Versioning level is already
exported as the SEMVER_LEVEL environment variable.
core.error creates an annotation on the check run, so the offending commit
and the reason why it is invalid are visible on the pull request itself,
instead of only in the job log. Only the header is included, so that the
annotation is not filled with the body of the commit message.
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch 3 times, most recently from b9ac13c to b0a1feb Compare September 11, 2026 23:06
The action reported the outcome the only way it knew, and a caller could not ask
for anything else: an invalid commit message and a Work-in-Progress commit both
failed the check with the same red X, though only one of them is a defect.

A WIP commit is now a warning rather than a failure, and is listed in the results
output, so that a caller can report it as a check of its own and the red X is
left to mean what it used to mean alone. An invalid commit message still fails.

enforce says whether the action reports at all. With false it becomes a pure
analyzer: it produces the result for every commit, exits successfully and writes
no annotations, so that the caller can render the conclusions, the wording and
the annotations itself, without those being compiled into this action. A value
other than true or false fails the action rather than being read as the default,
so that a misspelling is not silently a policy.
The bundle is rebuilt from the current sources. It also picks up newer patch
versions of the bundled dependencies, which is why the diff is larger than
the changes that produced it.
The annotation named the offending commit by its header alone, which does not
identify it when the same message appears more than once in a pull request, and
leaves nothing to look the commit up by. The abbreviated digest now precedes the
header.

Reading it needs the commits to carry it: the identifier was discarded when the
commits of a pull request were mapped to their message, and the commits of a
push carry it as id. Both are now mapped to a message and a sha.
A caller that renders the outcome itself builds its annotations from results,
which named every commit by its header alone. The digest is now part of each
result, so that such a caller can identify the commit and link to it, and the
annotations this action writes are not the only ones that can carry it.
The bundle is rebuilt from the current sources.
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch from b0a1feb to 1d75761 Compare September 11, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant