Repository navigation
A Scorecard publish check: an action that sees a failed publish and runs Scorecard once more - #199
Merged
Merged
Conversation
…uns Scorecard once more scorecard-action reports a failed publish to scorecard.dev as a warning on a green run. On 2026-11-09 a publish from askalf/pitlane was answered 500 three times, scorecard.dev served "not found" for the repository for fifteen hours, and the README badge read "invalid repo path" with no run red. actions/scorecard-publish-check runs on workflow_run of a repository's Scorecard workflow. For a run that succeeded on the default branch of a public repository it reads what scorecard.dev serves, four times at most. A result dated no earlier than the run's start is published. Otherwise it dispatches the Scorecard workflow once; when the unpublished run was itself a dispatch it fails the job, so there is no loop and a publish that fails twice is a red run. It is the fifth action the pin bump moves. No repository calls it yet: a caller pins a commit of main that has it.
askalf
marked this pull request as ready for review
November 10, 2026 02:35
…t carry a time CodeQL flagged the event file's data going into the dispatch request. The script no longer reads the event file: each fact is an input of the action that defaults to the workflow_run event's own field, and a workflow_run that arrives with no run id fails the step. Redline's review: a date with no time of day cannot say a result is this run's. The results this action publishes are dated to the second (the API served 2026-11-09T10:25:29Z and 2026-11-10T01:38:45Z for askalf/pitlane), and a date-only value is OpenSSF's weekly scan, so it is now never taken for a run's publish. Tests use the dates the API served.
The plain-name pattern on the default branch read a valid name such as release+stable as none, and a run on it was then skipped with no read of scorecard.dev. The branch is kept as given: it goes into the dispatch as a JSON string and nowhere else. The README states the one-minute skew allowance the script applies.
The test built a pattern from each input's event field, escaping only its dots, which CodeQL reads as incomplete escaping. Each input's block is now found by splitting the action file and compared as a string.
sprayberry-redline
approved these changes
Nov 10, 2026
sprayberry-redline
left a comment
Collaborator
There was a problem hiding this comment.
Verdict: approve. The check bounds its polling, dispatches one retry for an unpublished run, and fails an unpublished dispatch without starting another. The tests cover freshness, skipped runs, request failures, dispatch handling and input wiring; the new action is also included in pin updates and the test job.
This was referenced Nov 10, 2026
This was referenced Nov 10, 2026
askalf
pushed a commit
to askalf/pitlane
that referenced
this pull request
Nov 10, 2026
Moves this repository's askalf/ci pins (its reusable workflow calls, their redline-ref inputs and any action pins) to askalf/ci@50ee9df, the main commit that last changed the review, the fix or an action (main 2026-11-10, askalf/ci#199). The scripts are fetched at that same commit. The pins are the only change. --------- Co-authored-by: askalf-pin-bump[bot] <352119504+askalf-pin-bump[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes, and why
scorecard-actionreports a failed publish to scorecard.dev as a warning, and the run stays green. On 2026-11-09 the scheduled Scorecard run of askalf/pitlane had its publish answered 500 three times. scorecard.dev then served "not found" for the repository for about fifteen hours, the README badge read "invalid repo path", and no run was red. One clean publish cleared it.This adds the check for that, as a fifth shared action:
scripts/scorecard-publish/check.mjs, run byactions/scorecard-publish-check(a JavaScript action on the runner's Node, likeactions/fleet-status) onworkflow_runof a repository's Scorecard workflow.https://api.scorecard.dev/projects/github.com/<owner>/<name>, the address the badge reads, up to four times, 15 s apart.2026-11-09T10:25:29Zand2026-11-10T01:38:45Zfor askalf/pitlane's publishes of those days, and the tests use those values. A date with no time of day is OpenSSF's own weekly scan, which no run of ours wrote, so it is never taken for a run's publish, whatever its day.schedule,pushorbranch_protection_rulerun: the Scorecard workflow is dispatched on the default branch once, with a warning.workflow_dispatchrun: the step fails. A dispatch is never answered with another, so one failed publish costs at most one extra run, and a publish that fails twice is a red run.workflow_runevent's own field (github.event.workflow_run.idand so on), so a caller passes none and the script reads no file. Aworkflow_runthat arrives with no run id fails the step. The workflow file must be a plain file name under.github/workflows; the default branch is used as GitHub names it and is sent only as a JSON string in the dispatch body.pin.mjslists the action inACTIONSandredline-pin-bump.ymlwatches its two directories, so its pins move with the other actions'. The README has the caller workflow and a section on the check;scripts/scorecard-publish/README.mdhas the rules.Two things this relies on and does not prove:
github.event.*, as it does fromgithub.token. If it does not, the first run fails on the missing run id, which is why that is an error and not a skip; a caller can then pass the inputs itself.workflow_runfor the check in turn. If it does not, the retry still happens and only the second check, the one that turns a twice-failed publish red, is missing.How it is tested:
node test-all.mjspasses (all 12; the new one isscripts/scorecard-publish/check.test.mjs, also added to thetestjob). It covers which runs are checked, a default branch with a+or a letter outside ASCII, the dating rule with the dates the API served and with date-only values, andmainagainst a stubbed scorecard.dev and GitHub: a published run, a result that arrives on a later read, the 2026-11-09 case (404 on every read, one dispatch ofscorecard.ymlonmainwith the job token), an older result, only the weekly scan's result, no answer, an unpublished dispatch failing with nothing sent, a dry run, a refused dispatch, a missing run id, a default branch with a+and quotes sent as a JSON string, and a workflow path that is not a plain file name. The action's entry is run for real for the paths that make no request, and a test holds each input's default to its event field.Not tested: the action on a runner against the real scorecard.dev. No repository calls it yet.
Callers: no caller runs anything new until it adds the workflow.
pin.mjsand the pin bump's watch list change, so the next pin bump moves every caller's existing pins to this commit. The callers' ownscorecard-publish.ymlfiles come after this merges, pinned at the merged commit.