-
Notifications
You must be signed in to change notification settings - Fork 15
Add issue-opener when wheels are almost dead #175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
55e36ec
b2fd76c
55a2560
ad04c64
32929d1
52e7484
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this will be a nice short action to reuse elsewhere, too. So ultimately, I think it would make sense to actually document/mention all of the actions, suitable for direct downstream usage, in the README. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| name: Report a failing workflow | ||
|
|
||
| # Called by the scheduled workflows in this repository when they fail. A cron job | ||
| # that quietly stops working can go unnoticed for months, which is the very thing | ||
| # this repository nags other projects about. | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| title: | ||
| description: 'Title of the issue to open, and to reuse while it stays open' | ||
| required: true | ||
| type: string | ||
|
|
||
| jobs: | ||
| report: | ||
|
|
||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| issues: write | ||
|
|
||
| steps: | ||
| - name: Open or update an issue | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| TITLE: ${{ inputs.title }} | ||
| RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} | ||
| run: | | ||
| body="\`${GITHUB_WORKFLOW}\` failed, most recently in [this run](${RUN_URL}). | ||
|
|
||
| Until it is fixed, whatever that workflow keeps an eye on is unattended." | ||
|
|
||
| # Only ever match our own issues, so the default page of 30 cannot hide the | ||
| # one we are looking for. Without --search this is the REST creator filter, | ||
| # which wants the literal login: "app/github-actions" matches nothing here. | ||
| number=$(gh issue list --repo "${GITHUB_REPOSITORY}" --state open \ | ||
| --author "github-actions[bot]" --json number,title \ | ||
| --jq 'map(select(.title == env.TITLE)) | .[0].number // ""') | ||
|
|
||
| # Rewrite rather than comment, so a workflow that fails every day does not | ||
| # collect a comment a day | ||
| if [ -n "${number}" ]; then | ||
| gh issue edit "${number}" --repo "${GITHUB_REPOSITORY}" --body "${body}" | ||
| else | ||
| gh issue create --repo "${GITHUB_REPOSITORY}" --title "${TITLE}" --body "${body}" | ||
| fi |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| name: Report stale wheels | ||
|
|
||
| on: | ||
| # Run daily at 0:23 UTC, an hour before "Remove old wheels" | ||
| schedule: | ||
| - cron: '23 0 * * *' | ||
| workflow_dispatch: | ||
| inputs: | ||
| dry_run: | ||
| description: 'Report findings without opening or closing any issues' | ||
| type: boolean | ||
| default: true | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| report: | ||
|
|
||
| runs-on: ubuntu-latest | ||
| if: github.repository_owner == 'scientific-python' | ||
| # Needed to open an issue here if we cannot reach a project's own tracker | ||
| permissions: | ||
| issues: write | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
|
|
||
| - name: Install the latest version of uv | ||
| uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 | ||
| with: | ||
| python-version: '3.14' | ||
|
|
||
| - name: Check for stale wheels | ||
| env: | ||
| ISSUE_OPENER_TOKEN: ${{ secrets.ISSUE_OPENER_TOKEN }} | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we have this token yet?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep I added it! |
||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| uv run --with-requirements requirements.txt tools/check_stale_wheels.py ${{ inputs.dry_run && '--dry-run' || '' }} | ||
|
|
||
| report-failure: | ||
| needs: [report] | ||
| if: failure() && github.event_name == 'schedule' | ||
| permissions: | ||
| issues: write | ||
| uses: ./.github/workflows/report-failure.yml | ||
| with: | ||
| title: 'The stale nightly wheel check is failing' | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,5 @@ | ||
| # pixi environments | ||
| .pixi | ||
| *.egg-info | ||
| __pycache__ | ||
| .pytest_cache |
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I haven't reviewed or even read this file. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # Notes for agents | ||
|
|
||
| Guidance for automated contributors working in this repository. | ||
| Written for someone who can read the code but has no way to know the conventions or the reasoning behind them. | ||
|
|
||
| ## What is here | ||
|
|
||
| `action.yml` and `upload_wheels.sh` are the published action itself, which projects call from their own workflows to upload nightly wheels. | ||
| Everything else exists to look after the [`scientific-python-nightly-wheels`](https://anaconda.org/scientific-python-nightly-wheels) channel those uploads land on. | ||
|
|
||
| `.github/workflows/remove-wheels.yml` deletes wheels beyond the retention policy, daily at 01:23 UTC. | ||
| `tools/check_stale_wheels.py` warns projects before that deletion strands them, daily at 00:23 UTC. | ||
| The hour between the two is deliberate: a package must be flagged before it can be deleted, and a fully deleted package disappears from the channel listing where we would no longer see it at all. | ||
|
|
||
| `.github/workflows/report-failure.yml` is a reusable workflow every scheduled workflow calls when it fails. | ||
| This repository nags other projects about silently broken automation, so its own cron jobs must not fail quietly. | ||
| It is called as a job rather than used as a composite action because `ci.yml` checks out to `_action_path` and `remove-wheels.yml` does not check out at all. | ||
|
|
||
| ## Conventions | ||
|
|
||
| Pin third-party actions to a full commit SHA with a `# vX.Y.Z` comment; Dependabot updates them monthly as a single group. | ||
|
|
||
| Run the Python tools with `uv run --with-requirements requirements.txt <script>`, which is how the workflows invoke them; the scripts carry no PEP 723 header, so that their dependencies are declared in exactly one place. | ||
| That file pins exact versions rather than a `uv.lock`, because Dependabot has no `uv` ecosystem and an unmaintainable lock would only rot, while it does understand `requirements.txt` and proposes updates monthly. | ||
| `scientific-python/issue-from-pytest-log-action` is worth reading for the rest of the house style, though it locks its scripts individually where this repository does not. | ||
|
|
||
| Pin versions exactly, as `pixi.toml` and `requirements.txt` both do. | ||
| Do not add a `[tool.uv] exclude-newer` window to a script whose dependencies Dependabot pins: uv then refuses to resolve any pin newer than the cutoff, leaving every Dependabot pull request unresolvable until the release ages past it. | ||
| The window in `pixi.toml` is fine, because those versions are bumped by hand. | ||
|
|
||
| Format Python with `ruff format --line-length 100`; there is no ruff configuration in the repository yet, so pass the length explicitly. | ||
|
|
||
| ## Things that will catch you out | ||
|
|
||
| Pull requests from forks *and* from Dependabot receive no secrets, so a job keyed on `github.event.pull_request.head.repo.fork` still breaks for Dependabot. | ||
| Gate on the secret instead, as `ci.yml` does with `HAS_TOKEN: ${{ secrets.UPLOAD_TOKEN != '' }}`, and have the tokenless path assert the action refuses to upload. | ||
|
|
||
| A called workflow's token is capped by the calling job's, so a job that calls `report-failure.yml` must itself declare `permissions: issues: write`. | ||
|
|
||
| The anaconda.org API's `latest_version` is not the most recently uploaded version, because nightly version strings do not sort consistently across projects. | ||
| Take the maximum `upload_time` across every version instead. | ||
|
|
||
| That API carries no repository URL for these packages — `home`, `dev_url`, and `source_git_url` are all null — so `check_stale_wheels.py` resolves the target repository from PyPI `Project-URL` metadata, with `PYPI_MAP` covering the one package that has none. | ||
|
|
||
| PyGithub serializes every request through an internal connection lock, so threading does not speed up the GitHub half of a run. | ||
| A full stale wheel check takes about two minutes, which is fine for a daily job. | ||
|
|
||
| ## Working on the stale wheel check | ||
|
|
||
| Run it with `ISSUE_OPENER_TOKEN` and `GITHUB_TOKEN` set, and keep `--dry-run` on unless you intend to open issues in other projects' trackers. | ||
| Dry runs still authenticate and still read issues; they only skip writes. | ||
|
|
||
| The thresholds are constants at the top of the script rather than command line options, by request: add an option only when something actually needs to vary. | ||
| `RETENTION_DAYS` must stay in step with the 30 days in `remove-wheels.yml` and the policy section of `README.md`. | ||
|
|
||
| Tests live in `tests/test_check_stale_wheels.py` and run with `uv run --frozen tests/test_check_stale_wheels.py`. | ||
| They stub the network, so they are fast and safe to run anywhere. | ||
| Every case in them is a real package whose metadata would break a naive implementation; add to that table rather than replacing it when the resolution logic changes. | ||
|
|
||
| Issues opened in other projects are posted by [@scientific-python-bot](https://github.com/scientific-python-bot) using a classic token with the `public_repo` scope, stored as the `ISSUE_OPENER_TOKEN` secret. | ||
| Be conservative with anything that posts outward: those messages arrive unsolicited in other maintainers' inboxes. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| See [AGENTS.md](AGENTS.md). |
Uh oh!
There was an error while loading. Please reload this page.