Prevent duplicate workflow failure issues - #350
Closed
trask wants to merge 19 commits into
Closed
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
This PR updates the shared “workflow-failure-issue” automation to avoid duplicate tracking issues and to prevent cancelled runs from modifying existing workflow-failure issues.
Changes:
- Document cancellation behavior and concurrency serialization in the README.
- Add workflow-level concurrency to serialize updates.
- Replace
gh issue list --searchwith a paginated REST issues lookup + exact title-prefix matching.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| workflow-failure-issue/README.md | Documents “no-op on cancellation” and serialized updates; clarifies success input behavior. |
| .github/workflows/workflow-failure-issue.yml | Adds concurrency + skips on cancellation; switches issue lookup to paginated REST listing with exact prefix match to reduce races/search lag. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Copilot comment: Using `--paginate` over all open issues can become expensive in repositories with large numbers of open issues (runtime + API rate-limit consumption). A more scalable approach is to narrow the REST query (e.g., apply a dedicated label to tracking issues and query with `labels=...`, or otherwise constrain the candidate set) so the workflow doesn’t have to scan every open issue on each run. Analysis: The workflow creates tracking issues through GITHUB_TOKEN, so GitHub records github-actions[bot] as their creator. Filtering the REST request by that creator keeps the immediate, exact API lookup while excluding open issues created by people and other apps. Upsides: Repositories with many open issues scan only GitHub Actions-created candidates. The exact title-prefix check and compatibility with existing tracking issues remain intact. Downsides: Repositories with many unrelated open issues created by github-actions[bot] may still need more than one page. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot comment:
The wording suggests callers may only need to pass `success` for non-cancelled runs, but the input is still marked `Required: yes` (so callers must pass something regardless). Consider clarifying that callers should always provide `success` (e.g., `success: ${{ success() }}` / combined expression), and that cancelled runs are ignored because the job is skipped via `if: ... && !cancelled()`.
Analysis: The documentation now states that every caller must provide success. It gives success() and needs-based expressions as options, and separately explains that the reusable workflow ignores the value when cancellation skips its issue-update job.
Upsides: Callers can distinguish the required input contract from the reusable workflow's cancellation behavior.
Downsides: The README adds a short explanatory paragraph.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pull request dashboard statusClosed · refreshed 2026-09-04 00:45 UTC Status above doesn't look right?
|
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
Copilot comment: This approach paginates through all open issues created by `github-actions[bot]` and filters client-side, which can be expensive in repos with many open bot-created issues (more API calls + more JSON processing). If possible, narrow the server-side result set (e.g., by applying a dedicated label to tracking issues and adding `labels=...` to the REST query, or otherwise constraining the listing criteria) to reduce pagination and improve reliability under rate limits. Analysis: The workflow now derives a stable label from the caller workflow name and queries open issues by that label. On the first run for a caller, it scans the older bot-created candidates once and labels an exact match. Every issue created afterward receives the label. Upsides: Steady-state lookups return only tracking issues for one caller workflow. Existing open tracking issues migrate without creating duplicates. Downsides: Each monitored workflow adds one generated label to the calling repository. Its first run still scans open issues created by github-actions[bot]. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
Copilot comment: This collects *all* matching open issue numbers but then truncates to only the first one. That contradicts the documented behavior in the README (“On success, any open tracking issue is closed.”) when duplicates already exist (historical races, manual edits, etc.). Consider iterating over all values in `numbers` for actions that should apply to every open tracking issue (at least the success/close path), and/or explicitly closing or consolidating extras so the workflow converges back to a single tracking issue. Analysis: The workflow now keeps every exact matching issue number. A successful run closes all matches. A failed run comments on the first match and closes later matches as duplicates. The one-time label migration also labels every matching legacy issue before processing it. Upsides: Historical duplicates converge to zero open issues on success or one canonical open issue on failure. The documented close behavior now covers every matching issue. Downsides: Runs with historical duplicates make one close request per extra issue. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
…nistic Copilot comment: Migration of pre-label issues only runs when the label does not exist. If the label exists but there are still matching open tracking issues without the label (e.g., label was created manually, prior run partially migrated, or automation was interrupted), the workflow will ignore those issues and may create a new one—reintroducing duplicates. A more robust approach is to always include a fallback lookup (e.g., creator/title-prefix) and union the results with the label-based lookup, then ensure any unlabelled matches get the tracking label. Copilot comment: The canonical issue chosen for commenting/consolidation is implicitly whichever issue appears first in the GitHub API response. That ordering can be non-obvious (and may change with API defaults), which can cause the ‘primary’ tracking issue to flip over time. Consider sorting the collected issue numbers (e.g., numerically) and consistently choosing the oldest (lowest number) as the canonical issue before commenting and closing duplicates. Copilot comment: Migration of pre-label issues only runs when the label does not exist. If the label exists but there are still matching open tracking issues without the label (e.g., label was created manually, prior run partially migrated, or automation was interrupted), the workflow will ignore those issues and may create a new one—reintroducing duplicates. A more robust approach is to always include a fallback lookup (e.g., creator/title-prefix) and union the results with the label-based lookup, then ensure any unlabelled matches get the tracking label. Copilot comment: The canonical issue chosen for commenting/consolidation is implicitly whichever issue appears first in the GitHub API response. That ordering can be non-obvious (and may change with API defaults), which can cause the ‘primary’ tracking issue to flip over time. Consider sorting the collected issue numbers (e.g., numerically) and consistently choosing the oldest (lowest number) as the canonical issue before commenting and closing duplicates. Analysis: Every run now unions the label-based REST results with a narrow issue-search fallback for exact unlabelled title matches. It labels fallback matches, removes duplicate numbers, and sorts them numerically before choosing the first issue. Upsides: Manual label creation, interrupted migration, and removed labels cannot hide an existing tracking issue. The oldest issue number remains the stable canonical issue across API response order changes. Downsides: Each run makes one narrow search request in addition to the label-based REST request. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
Copilot comment: `gh issue close --reason` does not support a `duplicate` reason for GitHub Issues (supported reasons are typically limited to values like `completed` / `not planned`). This is likely to fail at runtime and prevent consolidation. If you want to mark duplicates, either (1) close with a supported reason (or default) and leave the explanatory comment, or (2) use the GitHub API/GraphQL "mark as duplicate" capability to formally mark the issue as a duplicate of the primary issue. Analysis: Duplicate tracking issues now close with the broadly supported not planned reason. The closing comment still links each extra issue to the canonical issue. Upsides: Consolidation no longer depends on support for the newer duplicate reason value in the GitHub CLI and API versions available to the runner. Downsides: GitHub records the issue as not planned rather than as a formal duplicate, although the closing comment identifies the canonical issue. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Same-named caller workflows are conflated, and the README recommends an unsupported input expression.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
workflow-failure-issue/README.md:55
success()is a status-check function forif:conditionals, butjobs.<job_id>.withpermits no special functions; callers following this recommendation can fail workflow validation. Keep the documentedneeds.<job>.resultpattern (or another explicit boolean expression) instead.
.github/workflows/workflow-failure-issue.yml:14
github.workflowis the caller's display name, and GitHub does not require workflow display names to be unique within a repository. Distinct same-named caller files therefore share this concurrency group; because GitHub retains only one pending run per group, one caller's update can replace another's pending update. Key both concurrency and tracking from a unique caller workflow path/ref or an explicit stable caller key.
group: shared-workflow-failure-issue-${{ github.workflow }}
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
Review finding:
The new guidance tells callers to use `success: ${{ success() }}`, but a status check function is not available in a reusable-workflow call's `with:` block. GitHub's context availability table lists `jobs.<job_id>.with.<with_id>` with special functions `None`, while `always, cancelled, success, failure` are listed only for `jobs.<job_id>.if` and `jobs.<job_id>.steps.if`. A consumer who copies this line gets a workflow that fails to parse with an unrecognized-function error, so the README hands them broken YAML. Fix: drop the `success()` suggestion and keep the `needs.<job>.result` expression the README already demonstrates, or point callers at a preceding step output.
Analysis: GitHub's context availability table gives the "Special functions" column the value None for jobs.<job_id>.with.<with_id>, and lists always, cancelled, success, and failure only for jobs.<job_id>.if and jobs.<job_id>.steps.if. The runner rejects a status check function outside an if condition with "Unrecognized function: 'success'", so a caller that copies the suggested line gets a workflow that never parses. The paragraph now points callers at the needs results the example above it already uses, and says in one sentence why success() is not an option, so a reader does not try it and hit the parse error.
Upsides: A consumer who follows the documentation gets a workflow that runs. The note about status check functions answers the obvious next question instead of leaving the reader to discover the restriction from a parse error.
Downsides: The paragraph is one line longer.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Review finding: The Behavior list is extended with cancellation and serialization, but it never mentions that the workflow now creates a label named `workflow-failure-<16 hex chars>` in the calling repository and applies it to every tracking issue. That label is visible in the consumer's issue list and label settings, and its name is an opaque hash of the workflow name, so a maintainer who finds it has no way to tell what created it or whether it is safe to delete. The README is the only consumer documentation for this reusable workflow. Fix: add a Behavior bullet stating that a per-workflow tracking label is created in the calling repository and applied to the tracking issues, and that deleting or renaming it breaks the lookup. Analysis: The workflow creates the label in the calling repository, not in shared-workflows, and applies it to new issues through gh issue create --label and to pre-existing tracking issues through gh issue edit --add-label. The label name ends in a truncated SHA-256 of the workflow name, so it reads as noise to anyone who meets it in the repository's label settings. This README is the only consumer documentation for the reusable workflow, and this pull request is already editing its Behavior list to record the new behavior, so the new label belongs in that same list. The added bullet says what the label is for and that the workflow creates it, which also tells a maintainer that removing it costs nothing permanent. Upsides: A consumer who finds the label knows what created it and why. The Behavior list now covers every repository resource the workflow touches. Downsides: No material downside identified. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot comment: Hashing the caller's display name does not produce a unique per-workflow label because two workflow files may legally use the same `name:`. Their label and title prefix then coincide, so a success from one workflow can close the other workflow's failure issue. Derive identity from a unique caller path/ref or explicit stable key, while retaining the display name only for human-readable text. Analysis: The concurrency group and tracking label now use GITHUB_WORKFLOW_REF, which includes the caller repository, workflow path, and ref. Label-based lookup no longer checks the display-name title. The legacy fallback accepts an exact embedded workflow ref or verifies the issue's original Actions run before applying the new label. Upsides: Workflows with the same display name cannot share tracking issues. Existing unlabelled issues still migrate when their original run proves that they belong to the current caller. Display names remain in issue titles, links, and comments for readers. Downsides: Migrating an unlabelled legacy issue can require an extra Actions API request. A caller that changes refs receives a new tracking label. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
Copilot comment: The label hash is truncated to 16 hex chars (~64 bits). While collisions are unlikely, a collision would cause unrelated workflows to share a tracking label and incorrectly merge/close issues. Increasing the prefix length (e.g., 24–32 chars) materially reduces collision risk while still staying well within GitHub label length limits. Analysis: The tracking label now keeps 32 hexadecimal SHA-256 characters, which provides 128 bits of workflow identity. The resulting label is 49 characters long, within GitHub's 50-character label-name limit. Upsides: The longer hash makes an accidental label collision negligible while preserving the existing deterministic label format. Downsides: Existing 16-character labels migrate through the legacy lookup once more, and generated label names are 16 characters longer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
Copilot comment: The workflow identity reconstruction hardcodes `@refs/heads/<branch>`. If the original run executed from a tag or other non-branch ref, this comparison can never match `$GITHUB_WORKFLOW_REF`, preventing legitimate legacy issues from being migrated/labeled. A more robust approach is to parse `$GITHUB_WORKFLOW_REF` and compare repository + workflow path + ref type/value accordingly, rather than forcing `refs/heads`. Analysis: Legacy migration now splits GITHUB_WORKFLOW_REF into the caller workflow identity and ref. It compares the candidate run's repository and path first, then matches its reported ref name as a branch or tag. Pull request refs match by the candidate run's pull request number. Upsides: Legacy issues can migrate for branch, tag, and pull request runs without treating every candidate as a branch. Downsides: The migration check reads two additional fields from the existing workflow-run API response and adds ref-specific matching branches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot comment: The `--body` string includes indentation spaces on the second line (`See ...`) due to YAML indentation being inside the quoted literal. This will render with leading spaces in the issue body. Consider building the body without embedded indentation (e.g., using a heredoc or `$'...'` with `\\n`) so the `See ...` line starts at column 0. Analysis: YAML already strips the run block's common indentation, but the source layout makes that behavior easy to misread. The workflow now constructs the complete issue body with `printf -v` and an explicit newline before passing it to `gh issue create`. Upsides: The generated body makes the column-zero second line explicit and no longer relies on readers recognizing YAML literal-block indentation rules. Downsides: The shell code adds one local variable and separate format arguments for the issue body. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Legacy-run verification lacks the required actions: read permission for private repositories.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
Copilot comment: This workflow calls the Actions API to verify legacy issues, but its job grants only `issues: write`; once any explicit permissions are set, `actions` is `none`. The workflow-run endpoint requires `actions: read` for private repositories, so these lookups fail, legacy issues are skipped, and a duplicate issue can be created. Add `actions: read` here and require it in the caller job/README as well, since a reusable workflow cannot elevate permissions omitted by its caller. Analysis: The reusable job now requests actions: read for workflow-run lookups. Its in-repository caller grants the same permission, and the consumer example and guidance require callers to grant it alongside issues: write. Upsides: Legacy issue migration can verify workflow runs in private repositories. Callers receive a complete permission contract instead of a lookup that silently lacks access. Downsides: Callers adopting this version must add actions: read to jobs that currently grant only issues: write. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Legacy matching can adopt spoofed issues or associate issues with the wrong workflow ref.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
.github/workflows/workflow-failure-issue.yml:98
- These alternatives do not establish the candidate run's actual ref type. A
pull_requestrun exposes both its sourcehead_branchand a PR number, so the same legacy issue is accepted for bothrefs/heads/<source>andrefs/pull/<number>/...; workflows triggered on both push and PR can therefore migrate, reuse, or close the other ref's issue. Include the run event/exact triggering-ref information and allow only the corresponding ref form instead of OR-ing every representation.
elif [[ "$caller_ref" == "refs/heads/$candidate_ref_name" ||
"$caller_ref" == "refs/tags/$candidate_ref_name" ||
(-n "$candidate_pr_number" &&
"$caller_ref" == refs/pull/"$candidate_pr_number"/*) ]]; then
- Files reviewed: 3/3 changed files
- Comments generated: 1
- Review effort level: Balanced
Copilot comment: The fallback trusts issue titles/bodies from any author. In a public repository, anyone can create this title and include the public workflow-ref marker or a URL for a matching run; the workflow will then adopt that issue and suppress creation of the real failure issue. Restrict migration candidates to issues created by the GitHub Actions app, since both legacy and new issues are created with `GITHUB_TOKEN`. This issue also appears on line 95 of the same file. Analysis: The fallback search now uses the author:app/github-actions qualifier and also checks that each returned issue's login is github-actions[bot] before trusting its marker or run link. Upsides: A repository visitor cannot make the workflow adopt a crafted issue or suppress creation of the real tracking issue. The server-side qualifier also narrows the search result set. Downsides: A legacy tracking issue created manually instead of through GITHUB_TOKEN will not migrate automatically. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Queued updates can be discarded, and refs containing @ are misparsed during migration.
Review details
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
.github/workflows/workflow-failure-issue.yml:15
cancel-in-progress: falsestill allows only one pending run by default, so a third invocation cancels the previous pending update. If the replacement caller is then cancelled, that dropped successful or failed run never updates the issue, leaving state from an older run. Addqueue: maxso these updates are actually queued and serialized.
.github/workflows/workflow-failure-issue.yml:77- These expansions split at the last
@, but valid branch and tag names may contain@. For a ref such asrefs/heads/feature@candidate, this produces an identity ending inrefs/heads/featureand a ref of onlycandidate, so the legacy issue cannot be verified and a duplicate can be created. UseGITHUB_REFas the exact suffix instead.
- Files reviewed: 3/3 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Review finding: This bullet tells consumers that serialization stops new duplicates from appearing, but the Behavior list never says what the workflow now does to tracking issues that are already open. The script adds two user-visible behaviors that the list omits. First, on failure with more than one matching open issue it comments only on the lowest-numbered one and closes every other one with `--reason "not planned"` and a `Superseded by #N` comment. Second, it adopts pre-existing issues created by earlier versions of this workflow by adding the per-workflow label to them, after confirming the run linked in the issue body belongs to the same caller. A maintainer of a consumer repository sees their existing failure issues get closed as not planned and relabelled by automation, with the README as the only place that documents this workflow. The PR description already states both behaviors, so the consumer documentation is the part that is missing them. Fix: extend the Behavior list with a bullet saying that on failure any extra open tracking issues are closed as not planned and superseded by the one that receives the comment, and a bullet saying that issues opened by earlier versions of this workflow are adopted and labelled once their linked run identifies the same caller workflow. Analysis: On failure the script comments on the lowest-numbered matching issue and closes every other match with --reason "not planned" and a "Superseded by #N" comment, but the Behavior list still described the old single-issue path. It also adds the per-workflow label to issues opened by earlier versions once the run linked in the issue body identifies the same caller workflow file and ref. CONTRIBUTING.md states that a workflow's README is the contract between the workflow and the repositories that use it, and this README is the only consumer documentation for the reusable workflow. A maintainer whose repository already holds several failure issues sees automation close some of them as not planned and relabel the rest, so the contract has to say so. The subsequent-failure bullet now names which issue keeps the comment and what happens to the others, and the label bullet now records the condition under which an older issue is adopted. Upsides: The Behavior list matches what the script does to issues that are already open. A consumer can tell in advance which of their existing failure issues survives and which ones close. Downsides: No material downside identified. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove generated labels and legacy workflow-run migration while retaining serialization and duplicate convergence. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Restore the existing issue lookup and handling so this change only serializes workflow updates. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5ed70e46-7212-49db-9233-8783a74f7d85
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Serializes workflow-failure issue updates by workflow name so concurrent runs cannot both observe that no tracking issue is open and create duplicates.
No issue lookup or handling behavior changes.