Skip to content

ci: deliver scheduled-run reports somewhere that notifies #2030

Open

Description

@JarryShaw

Is your feature request related to a problem? Please describe.
Scheduled runs (ci-timing, cron-vendor, cron-conda, cron-python-versions, codeql-analysis, ...) have no PR to comment on. Their reports reach nobody: a step summary sends no notification, and ci-timing only signals by failing, which GitHub emails to whoever last edited its cron line.

Describe the solution you'd like
Recommended: Discussions (already enabled).

  • Add a "CI reports" announcement-type category, with one thread per workflow.
  • A separate job with discussions: write comments only when the report changed or crossed a threshold. It uses gh api graphql and GITHUB_TOKEN, with no third-party action.
  • You get email through normal GitHub notifications if you watch the category.
  • It stays off the issue board, so the board-clear goal and state labels are untouched.
  • Start with ci-timing and cron-vendor.

Describe alternatives you've considered

Additional context
The reusable unit-tests.yml stays without write permissions, since its callers grant contents: read. Decision needed: Discussions, a rolling issue, or neither; and which workflows report first.

Activity

  1. added
    enhancementIssues requesting a new capability (set by the feature request template)
    ciPull requests that change CI or workflow configuration (ci: subject prefix)
    needs: decisionWaiting on the maintainer to decide — not blocked by other work
    on Oct 22, 2026
  2. moved this to Needs decision in PyPCAPKiton Oct 22, 2026
  3. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    Go with Discussions. I think we create one new discussion to hold all these reports. and for each thread, it holds one job's reports - each job will just add a comment to the same thread instead of creating new threads.

  4. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    Starts with the two but will cover all with this issue.

  5. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    Taking Discussions, as I read your two comments: one category holds all the reports, with one thread per job. Each run adds a comment to its job's thread rather than opening a new one. ci-timing and cron-vendor go first, then every other scheduled workflow, all under this issue.

    • util/discussion_report.py finds the job's thread by title and creates it only on first use, so no ids are hard-coded. It then adds one comment, but only when the report changed or crossed a threshold. It uses gh api graphql and GITHUB_TOKEN in a job holding only discussions: write.
    • PR 1: the script plus ci-timing and cron-vendor, stacked on ci: fix shellcheck findings and trim workflow comments (#1967) #2023 since it edits both files. The remaining workflows follow in later PRs.

    Needs you: the API cannot create a category (GraphQL has no mutation for it), so it has to be made in Settings → Discussions. My recommendation: a category called CI reports, in the Announcement format so only maintainers and the bot can start threads. The script reads the category name from its input and the tests stub the API, so the work starts now.

  6. added
    wipWork in flight - a covering PR is open or an agent is actively on it
    on Oct 22, 2026
  7. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    Category created.

  8. removed
    needs: decisionWaiting on the maintainer to decide — not blocked by other work
    on Oct 22, 2026
  9. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    Confirmed: the API now lists CI reports (slug ci-reports). The first PR uses that name.

  10. moved this from Needs decision to WIP in PyPCAPKiton Oct 22, 2026
  11. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    I think this applies not only to scheduled ones but also those run on branches - anything that generates a report but cannot post to a PR's comment should go there.

  12. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    Scope widened, as I read it: every report that has no PR to comment on goes to its job's thread, whether it comes from a scheduled run or a branch push. A report from a PR run keeps commenting on the PR. Plan:

    • PR 1 (in progress): util/discussion_report.py plus ci-timing and cron-vendor.
    • PR 2: push runs of Unit Tests. unit-tests.yml is reusable and must keep contents: read, so posting goes through coverage-comment.yml's workflow_run relay, which today only handles pull_request runs. It gains a branch for other events that posts to the thread instead of a PR.
    • Later PRs: experimental.yml (its non-blocking warnings), python-compatibility.yml, lint.yml and the remaining crons, one workflow each.

    Needs you: how often a push run posts. Pushes to main land several times an hour during a merge wave.
    My recommendation: post only when the report differs from the last one in the thread, or when a non-blocking leg warned or failed, and never for an unchanged green run. The alternative is every run. I'm building the recommended behaviour now; it's one flag to change if you'd rather have every run.

  13. 21 remaining items

  14. moved this from Needs decision to WIP in PyPCAPKiton Oct 22, 2026
  15. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    Lint and Pages reporting is up in #2056. Three workflows need a call first:

    1. CodeQL: what counts as "warned"? Its findings go to Code Scanning, not annotations, and recent main runs all end success. Recommended: the status is analyze's result. The report lists open code-scanning alerts by severity, which needs security-events: read on the posting job only, and a rise in that count is a warning.
    2. Experimental: skipped runs. On main, a superseded or docs-only push skips every leg, which would post a "back to green" with nothing tested. Recommended: follow Unit Tests' rule: job conclusions plus annotations, with an empty by-design list. Post nothing when current or changes.code is false.
    3. Compat: lands after build!: require Python 3.8+ and drop the BPC conversion (#2051) #2053. build!: require Python 3.8+ and drop the BPC conversion (#2051) #2053 drops the 3.7 leg whose warnings are on every run today. Recommended: after build!: require Python 3.8+ and drop the BPC conversion (#2051) #2053, move the check-suite reporter out of coverage-comment.yml into a shared util/ script, rather than copy it into a third workflow, and use an empty by-design list.
  16. added
    needs: decisionWaiting on the maintainer to decide — not blocked by other work
    on Oct 22, 2026
  17. moved this from WIP to Needs decision in PyPCAPKiton Oct 22, 2026
  18. JarryShaw commented on Oct 22, 2026

    @JarryShaw
    OwnerAuthor

    go with your recommendations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciPull requests that change CI or workflow configuration (ci: subject prefix)enhancementIssues requesting a new capability (set by the feature request template)needs: decisionWaiting on the maintainer to decide — not blocked by other workwipWork in flight - a covering PR is open or an agent is actively on it

    Projects

    • Status
      Needs decision

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions