Skip to content

ci: sweep Roadmap board Status every 5 minutes - #623

Merged
navareamol merged 8 commits into
mainfrom
ci/project-review-status
Oct 28, 2026
Merged

navareamol merged 8 commits into
mainfrom
ci/project-review-status

Conversation

@navareamol

@navareamol navareamol commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

What

Adds a sweep that sets Status on the ExtendDB org Project board from the real
state of each open issue and PR. CI-only; no product code changes.

File Purpose
.github/workflows/project-status-sweep.yml Runs the sweep about every 5 minutes, on issue assigned/unassigned, and on demand (dry_run input).
.github/scripts/project-status-sweep.js The rules and the sweep.
.github/scripts/project-status-sweep.test.js 46 unit tests (node --test), no dependencies.
.github/workflows/scripts-test.yml Runs those tests on PRs touching .github/scripts/.

Also removes project-review-status.yml, added in this branch's first commits.

How it works

flowchart TD
    A[Schedule, issue assigned/unassigned, or manual run; always from main] --> B[Mint short-lived GitHub App token]
    B --> C[Read all board items, 50 per page, with retries]
    C --> D{For each item}
    D -->|archived, draft, closed, unreadable| S[Skip]
    D -->|Status not Todo / In Progress / Review| S
    D -->|Auto status = Off| S
    D -->|person moved it, nothing changed since| S
    D --> E{Issue or PR?}
    E -->|PR| F[Status from draft flag, human reviews, review requests]
    E -->|Issue| G[Status from assignees and trusted open PRs]
    F --> H{Same as now?}
    G --> H
    H -->|yes| S
    H -->|no| I[Set Status, retried on error]
    I --> J[Write details to the private 'Board sweep:' card]
    S --> J
    J --> K[Public log: one line of totals]
Loading

Cases it handles

# Situation Card goes to
1 New issue, nobody on it Todo
2 Issue gets an assignee In Progress (within a minute: issue events trigger a run)
3 Last assignee removed, no open PR Todo
4 Maintainer opens a PR with Closes #N PR: In Progress; issue #N: In Progress
5 Outside contributor's PR says Closes #N PR: In Progress; issue #N: unchanged
6 Same, but a maintainer links it from the issue sidebar Issue #N follows the PR
7 Draft PR, even with reviews In Progress
8 PR author replies to comments No change
9 Bot or code-scanning reviews No change
10 A person reviews (comment, approve) Review; linked issue too
11 Any current reviewer requests changes In Progress
12 Author asks that reviewer to review again Review
13 That reviewer approves Review
14 Reviewer's review is dismissed, no other verdict Review (a person did look)
15 PR merged or closed Done (built-in workflow; sweep skips it)
16 Card in Done, Blocked or any other status Never touched
17 Card's Auto status field set to Off Never touched
18 Person moves a card by hand Kept until the issue/PR (or the issue's trusted PR) changes again
19 Archived card or board-only draft Never touched

Privacy

This repo is public, so the workflow's Actions logs are public; the board is
private. The sweep keeps the board's contents out of anything public:

  • The log is one line of totals, e.g. Sweep ok: 3 card(s) moved, 0 problem(s), query cost 54.
    No issue or PR numbers, statuses, draft counts or board settings.
  • Every detail (each move, each problem, what was skipped) goes to one board-only
    draft card, Board sweep: ..., which only org members can see.
  • No issues or comments are posted anywhere.
  • A test fails if any issue number, status, draft title or problem detail reaches the log.

Failure modes and how they are handled

# Failure Handling
1 App key expired/rotated, App uninstalled Run fails (red); GitHub emails the workflow's owner. The health card goes stale.
2 Hand move undone by any later activity Auto status = Off holds a card for good. Hand moves still hold until the work changes.
3 Re-requested review after changes requested Reviewer present in reviewRequests no longer blocks; card goes to Review.
4 Status field or option renamed Card title says NEEDS ATTENTION with the detail; other cards still move; run stays green.
5 Scheduled runs delayed by GitHub Issue assign/unassign also triggers a run; the card shows the last run time.
6 Built-in board workflows also set Status Sweep reads the board's workflows and flags any that conflict on the card.
7 API errors on read or write 3 tries with backoff; a write that still fails marks the run failed; other cards still move.
8 Rate limit runs low Stops reading early, applies what it has, flags it on the card. About 54 points per sweep.
9 Issue with many linked PRs First 5 in the page query; the rest fetched for that issue. If that fails, the card is left as is and flagged.
10 Cards from another repo Token covers every repo the App is installed on; unreadable cards are flagged.
11 deployment: false and the branch rule Secrets stay limited by the environment's branch policy (docs).
12 Health card cannot be written Run fails (red), since the card is the only detailed report.
13 Schedule switched off after 60 days without repo activity The card goes stale; GitHub also emails the workflow's last editor.

Why

The board's built-in workflows have no "review requested/approved" trigger, and
"Pull request linked to issue" moves cards for any PR, including outside ones.
Every trigger here runs from the default branch, so no fork can start it, and
it needs no pull_request_target (off by default for public repos from 2026-12-02).

Addresses the earlier review on this PR: no pull_request_target; no secret
reachable from a non-main ref (environment limited to main); GitHub App token,
not a personal PAT; actions pinned to SHAs; ubuntu-latest; author and bot
reviews ignored; Closes # from untrusted PRs ignored; unknown statuses,
archived cards and missing options left alone and flagged, not a red run.

No linked issue.

Setup (done)

  1. GitHub App owned by ExtendDB with Organization → Projects read/write and
    Repository → Issues, Pull requests, Metadata read; no webhook; installed on extenddb.
  2. Environment project-board, deployment branches main only, with variable
    PROJECT_APP_CLIENT_ID and secret PROJECT_APP_PRIVATE_KEY.
  3. Board workflows that set Status, other than "Item closed" and "Pull request merged", are off.
  4. Auto status field (single select, option Off) is on the board.
  5. The old PROJECT_TOKEN repo secret is deleted.

After merge: run the workflow once with dry_run: true and read the board's
Board sweep: DRY RUN ... card.

Testing done

  • 46 unit tests in project-status-sweep.test.js covering every case, failure
    mode and the privacy rule above, with a fake GitHub client. Checked that they
    fail when the re-request, author-reply, trust, retry, hold, rate-limit,
    card-failure and log-privacy logic is broken. CI runs them with Node.
  • Ran the sweep's exact read queries against the board and replayed them through
    the sweep in dry-run mode: query cost 54 per sweep, 0 problems, and the public
    log line contained only totals.
  • Independent code review of an earlier revision; its findings are addressed.
  • Not run end-to-end from Actions yet: that needs the workflow on main.

Checklist

  • I have read CONTRIBUTING.md
  • n/a: cargo test, cargo fmt, cargo clippy (no Rust changes)
  • I have added tests for new functionality (.github/scripts/project-status-sweep.test.js)
  • Documentation: setup, rules, privacy and failure handling are in the workflow and script headers
  • n/a: breaking changes (CI-only)
  • n/a: RFC (no wire protocol, Storage, auth model, on-disk format or CLI change)

ADR / RFC: n/a


By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.

Signed-off-by: navareamol <28789438+navareamol@users.noreply.github.com>
…k on changes requested

Signed-off-by: navareamol <28789438+navareamol@users.noreply.github.com>

@robinnsc robinnsc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question first: has an Actions policy allowing pull_request_target been set up for this repo or this workflow? GitHub is rolling out a default rule that disables pull_request_target on public repos without an allowing event policy, enforced from December 2 (changelog, docs). No workflow on main uses it today, so unless a policy is added, the opened, reopened, ready_for_review and review_requested runs would start failing then, leaving only the issues and same-repo review runs. We couldn't read the org or repo policy to confirm. A scheduled sweep from main would be one alternative that needs no fork-triggered runs.

Otherwise, the script doesn't interpolate any payload text into the script or the GraphQL query, and it never checks out PR code, so there's no injection or code-execution path from a fork. Two things about the description:

  • It's behind the second commit. Drafts are no longer skipped (they move to In Progress on open, review request and any review), a card in Review does move back on changes requested, the issues assigned/unassigned trigger isn't mentioned, and neither is the header's instruction to turn off the board's built-in "Code changes requested" workflow.
  • PROJECT_TOKEN is already set, and pull_request_review runs the workflow from the PR's merge ref, so reviews on this PR (including this one) run the script against the live board before merge. It should do little here since the PR closes no issues, but it's worth knowing those are the first end-to-end runs.

Comment thread .github/workflows/project-review-status.yml Outdated
Comment thread .github/workflows/project-review-status.yml Outdated
Comment thread .github/workflows/project-review-status.yml Outdated
Comment thread .github/workflows/project-review-status.yml Outdated
Comment thread .github/workflows/project-review-status.yml Outdated
if: >-
github.event_name != 'pull_request_review' ||
github.event.pull_request.head.repo.full_name == github.repository
runs-on: extenddb_ubuntu-2404_4-core

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On main the 4-core pool is only used for the Rust build and test jobs, and every lightweight job runs on ubuntu-latest, which is free for a public repo. This job makes a few GraphQL calls on every issue assign/unassign and every PR event, and with pull_request_target forks can start it without approval, so ubuntu-latest would keep it out of the queue the merge checks wait on. Unassign events that still leave assignees could also be filtered in the job's if: (for example with toJSON(github.event.issue.assignees) != '[]') so they don't start a runner at all.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both jobs now run on ubuntu-latest (L52, L103), so the merge-check pool is untouched.

On filtering unassign events: forks can't start runs any more, and only people with triage access can assign or unassign. Each run sweeps the whole board, so an extra run just catches up. I left it unfiltered on purpose, so the issue-triggered run also works as a fast catch-up when scheduled runs are delayed. Happy to add toJSON(github.event.issue.assignees) != '[]' if you'd still prefer it, so I'll leave this thread open for you.

Comment thread .github/workflows/project-review-status.yml Outdated
Addresses review on #623: no pull_request_target (off by default from
2026-12-02), runs only from main via a main-only environment, uses a
GitHub App token, pins actions to SHAs, runs on ubuntu-latest, ignores
author/bot reviews, ignores Closes # from untrusted PRs, and leaves
archived, unknown-status and hand-moved cards alone.

Signed-off-by: navareamol <28789438+navareamol@users.noreply.github.com>
- Read hand moves from the issue/PR timeline's last status-change event
  on this board; Status.creator is the first creator and never changes.
- An issue's hand move is released by its trusted linked PR changing too.
- Use latestOpinionatedReviews for verdicts so an old changes-requested
  is never lost past the last 30 reviews.
- Warn when linked-PR lists are truncated.
- environment deployment: false; step names; separate dry-run
  concurrency group; setup notes name the right built-in workflows.

Signed-off-by: navareamol <28789438+navareamol@users.noreply.github.com>
@navareamol

Copy link
Copy Markdown
Collaborator Author

In the next revision, will push the tests for this change along with some failure modes which are being tested.

@navareamol navareamol changed the title ci: move Roadmap board cards to Review when a PR is picked up for review ci: sweep Roadmap board Status every 5 minutes Oct 28, 2026
@navareamol
navareamol requested a review from robinnsc October 28, 2026 03:23
- Health: rewrite a 'Board sweep: ...' draft card on the board each run,
  and a report job opens one alert issue on failed/degraded runs and
  closes it on the next clean run.
- Re-requested review after changes requested moves a PR to Review.
- 'Auto status = Off' board field holds a card permanently.
- Retry GraphQL calls with backoff; stop early on a low rate limit.
- Fetch full linked-PR lists when the first page is truncated.
- Report renamed status options, a missing Status field, conflicting
  built-in board workflows, and cards the App cannot read.
- Run on issue assigned/unassigned too, for faster issue updates.
- Token covers the repos the App is installed on, not just extenddb.
- Add node:test unit tests and a Scripts test workflow.

Signed-off-by: navareamol <28789438+navareamol@users.noreply.github.com>
@navareamol

Copy link
Copy Markdown
Collaborator Author

@robinnsc thanks for the thorough review. I reworked the PR around your pull_request_target point:

  • pull_request_target / Dec 2 policy: removed. All triggers run from main (schedule, issue assign/unassign, dispatch), so no Actions policy is needed.
  • Stale description: rewritten for d301027, with a flow chart, 19 cases and a failure-mode table.
  • Pre-merge runs against the live board: the old event workflow is deleted, and the sweep only runs from main, so nothing on this branch touches the board.

Tests are in: 44 node:test cases, run by the new Scripts test workflow and green on d301027. Each inline thread has a reply with links. I resolved the six that are fixed and left the runner/unassign thread open for your call. Could you take another look?

robinnsc
robinnsc previously approved these changes Oct 28, 2026
Scheduled and issue-triggered runs only happen while the repo variable
PROJECT_SWEEP_ENABLED is 'true'. Manual runs always work, so the sweep
can be dry-run after merge before it moves any card, and paused later
by deleting the variable.

Signed-off-by: navareamol <28789438+navareamol@users.noreply.github.com>
@navareamol

navareamol commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

Added an on/off switch in 5d29fb2. Reverted in 70ae5df, so the code is identical to d301027 again. The sweep runs on its schedule once merged.

This reverts commit 5d29fb2. The sweep runs on its schedule as soon as
it is merged, as in d301027.

Signed-off-by: navareamol <28789438+navareamol@users.noreply.github.com>
The repo is public, so Actions logs are public; the board is not.

- Log one line of totals only: no issue/PR numbers, statuses, draft
  counts or board settings.
- Put every detail (moves, problems, skipped) on the private
  'Board sweep: ...' health card; dry runs write only that card.
- Remove the report job: it opened a public issue describing the
  private board.
- Fail the run when the health card cannot be written, since it is
  now the only alert channel.
- Tests: public log leaks nothing, card has the details, card write
  failure fails the run.

Signed-off-by: navareamol <28789438+navareamol@users.noreply.github.com>
@navareamol

Copy link
Copy Markdown
Collaborator Author

Pushed 09a6b77: privacy fix. This repo is public, so Actions logs are public while the board is not. The sweep now logs one line of totals only. Every detail goes to a board-only Board sweep: ... draft card, and the job that opened a public alert issue is removed. A run goes red (and GitHub emails) when card updates fail or the card can't be written. 70ae5df also reverted the on/off switch from 5d29fb2. Description updated; 46 tests.

@navareamol
navareamol requested a review from robinnsc October 28, 2026 06:31
@navareamol
navareamol added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit cf321f0 Oct 28, 2026
28 checks passed
@navareamol
navareamol deleted the ci/project-review-status branch October 28, 2026 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants