Skip to content

feat(util): have bump_version.py keep CITATION.cff in step with the bump - #625

Merged
JarryShaw merged 1 commit into
mainfrom
fix/bump-version-citation-cff
Sep 22, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/bump-version-citation-cff

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Sep 22, 2026

Copy link
Copy Markdown
Owner

Requested by the owner: "we have version cited in CITATION.cff, might need to have the version_bump.py handle that as well". No issue exists for this, so there is no Fixes line.

Two notes on the ask as phrased: the script is util/bump_version.py, not version_bump.py, and it rewrote two files rather than one — pcapkit/__init__.py and, from the working directory, conda/build.

The problem

CITATION.cff landed in #615 carrying version: 1.5.0b4 and date-released: '2026-09-20'. Nothing in the repository maintains it. A repo-wide grep for CITATION outside .git hits only changelog prose — no workflow, hook, Makefile, setup.py or pyproject.toml mentions it, and the file has exactly two commits in its history. So every version bump left it naming the previous release, which matters more than a stale docs file usually would: GitHub renders it as the Cite this repository button, and citation managers, Zenodo and dependency inventories read it directly, so a wrong number propagates into papers and bills of materials.

What the release flow actually does, and what that settles

bump_version.py is invoked from exactly one place — the Bump Version step of .github/workflows/cron-vendor.yml:78, between the registry crawl and the git commit. Completion of that workflow is itself the workflow_run trigger of create-release.yml, which reads pcapkit.__version__ back out of the tree, tags v<version>, and publishes to PyPI and Anaconda with no manual approval gate (environment: release is commented out at create-release.yml:234).

So in this repository the bump is not a preparatory step that a release later follows — the bump is what causes the release. Measured over the thirty most recent releases:

median max ≤1 h >24 h
bump commit → PyPI upload 3 min 2.4 h 24/25 0, ever

and the measurement that actually governs a date field: the bump commit's UTC calendar date equals the PyPI upload's UTC calendar date 30 times out of 30, with zero midnight crossings — the weekly cron fires at 10:05 UTC and publishes at 10:07, twelve hours clear of a date boundary.

Recommendation on date-released: bump it, in UTC

Both fields move. They have the same standing — the file's own header says version and date-released both "describe the newest published release", so at bump time both are equally anticipatory and there is no principled line that moves one and pins the other. Moving only version would assert something plainly false and leave it there indefinitely: "1.5.0b5, released on the day 1.5.0b4 was". A stale date is wrong in every case where the release does happen, which is eleven times in twelve.

UTC, not date.today(): seven of those thirty bumps were made late evening in US-Eastern, where a naive local date is a day behind the publish it describes.

The residual risk, stated rather than hidden. Roughly one bump in twelve historically never reached PyPI at all — eight consecutive weekly bumps between 2024-04-13 and 2024-06-15 (1.3.1.post10post17, all tagged), and 1.5.0b1 last week. For those this writes an optimistic date for a release that never happened. That is an argument for eventually anchoring both fields to create-release.yml, which runs only when a publish is actually going out — not for leaving the field stale. .github/ was outside this change's scope, so it is a recommendation rather than a commit.

And the script is no longer the main way versions move. Of the 159 commits that have moved __version__ on main, 40 are hand-authored and so never ran this script — a quarter over the project's life, and a rising share lately: 11 of the most recent 25. (Two different windows, which is worth stating plainly: 25% lifetime, 44% recent. 1.4.0, 1.4.1, 1.5.0a1 and the 1.5.0b1b3 run are all in the recent set.) That gap is why the change also adds a gate: RepositoryCitationTests fails when the committed CITATION.cff has drifted from pcapkit/__init__.py, so a hand-made bump that forgets the file is caught in CI rather than shipped. Proven non-vacuous — it catches drift in either direction, does not latch onto cff-version, and survives a quoted value.

The absent-file case: reported on stderr, then skipped

Deliberately not fatal, and deliberately not silent either.

Silence is out for the reason the whole change exists — a bump that quietly does not update a file it maintains is how this staleness starts. But hard failure is wrong for this script specifically, because of where it runs: the Bump Version step executes under bash -e, before the workflow's git commit. A non-zero exit there does not merely skip the citation update; it fails the step, discards the entire vendor crawl the run existed to produce, and the release that would have followed never happens. Trading a published release and a week of registry updates for a missing documentation file is the wrong way round.

A file that is present and carries no top-level version field is a different case and does raise CitationFieldError. There the script is in exactly the situation it was written for, and finding nothing to update means the file's shape changed underneath it — an outcome indistinguishable from success if it were allowed to pass. Unlike an absent file, that cannot happen by accident of packaging. The check runs before anything is written, so the fatal case leaves the tree untouched rather than half-bumped; plan_citation takes text and returns text, so it has nothing to write with.

date-released is optional in CFF 1.2.0, so a file without one keeps not having one — the absence is reported and no date is invented.

How the rewrite works, and what it refuses to disturb

Line-oriented, not a yaml load-and-dump. A round trip through a YAML library would reorder keys, normalise quoting and drop the sixteen comment lines explaining why the file omits doi and orcid — none of which anyone asked for.

  • Anchored at column zero. cff-version: 1.2.0 shares the version: suffix but must never move, and a references entry's own version is indented and belongs to that reference. Both are covered by tests.
  • Each field keeps the quoting it had. The committed file spells one of each — version: 1.5.0b4 bare, date-released: '2026-09-20' quoted, the latter deliberately so a validator reads a string rather than casting it to a YAML date. Preserving the style rather than normalising it keeps the bot's diff down to the value that moved. An unquoted version is safe because every string bump() returns carries a .devN/aN/bN/rcN/.postN suffix, so none can be read as a YAML number the way a bare 2.0 would be — asserted by test_every_bumped_version_carries_a_non_numeric_suffix.
  • Line terminators and a missing final newline survive; CRLF in, CRLF out.
  • One acknowledged limit: an inline # comment after either value would not survive. Neither line carries one, all of the file's commentary being whole-line, and parsing far enough to know the difference is the YAML round trip this avoids. The loss would be visible in the diff.

On the real file the rewrite changes exactly two lines:

-version: 1.5.0b4
-date-released: '2026-09-20'
+version: 1.5.0b5
+date-released: '2026-09-22'

CITATION.cff itself is not edited by this PR. Maintaining it is the script's job; a hand-edit now would be the bug this change removes.

Two other things this touches

The script gains a main() guard. It previously ran the entire bump at module scope, so importing it rewrote two files as a side effect — which is why it had no testable surface at all and why variant A below cannot even be collected.

A latent bug fixed while here. The import pcapkit fallback in the version reader ran line.split('=')[1].strip(" '"), which stops at the trailing newline and so returned "1.5.0b4'\n" — closing quote and newline included — which packaging rejects outright:

packaging.version.InvalidVersion: Invalid version: "1.5.0b4'\n"

That path had never worked. It went unnoticed because the only caller installs the package first, so import pcapkit always succeeds and the fallback is unreachable in CI. Now .strip().strip('\'"').

Verification

The tests fail without the change

Three variants of util/bump_version.py, same test module, same interpreter — a venv with pytest and packaging but no pcapkit, so nothing can reach the real checkout. Exit codes read from files, not from a pipe.

variant exit result
Aorigin/main verbatim 2 collection error: InvalidVersion: "1.5.0b4'\n"
B — new file, CFF machinery removed + old strip restored 1 22 failed, 9 passed, 1 skipped
C — as committed 0 31 passed, 1 skipped (32 passed where PyYAML is present)

Variant A proves the old script had no testable surface: merely importing it runs the bump, and it dies.

tests/project/test_bump_version.py:99: in _load_script
    spec.loader.exec_module(module)
util/bump_version.py:24: in <module>
    ver_obj = Version(version)
E   packaging.version.InvalidVersion: Invalid version: "1.5.0b4'\n"
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!

Variant B isolates the new behaviour from the refactor. The 22 failures are exactly the 21 citation tests plus the fallback-strip test; the 9 that pass are the behaviour-preservation set — bump(), the __init__.py rewrite, and the repository gate:

E   AttributeError: module 'bump_version' has no attribute 'plan_citation'
    def test_the_fallback_reader_strips_the_closing_quote_and_newline(self) -> None:
>       self.assertEqual(version, '1.5.0b4')
E       AssertionError: "1.5.0b4'\n" != '1.5.0b4'
E       - 1.5.0b4'
E       ?        -
E       + 1.5.0b4

And with the change:

$ python -m pytest -q tests/project/test_bump_version.py
................................                                         [100%]
32 passed in 0.08s

The produced CFF is valid — through the real validator

cffconvert is not in any extra here, so it was installed into a throwaway venv rather than the repo's. --validate on the rewritten real file:

$ cffconvert --validate -i CITATION.cff
Citation metadata are valid according to schema version 1.2.0.

$ cffconvert -i CITATION.cff -f apalike
Shaw J. (2026). PyPCAPKit (version 1.5.0b5). URL: https://jarryshaw.github.io/PyPCAPKit/

The test fixture validates both before and after the rewrite, so the round trip is proven on a document the schema accepts rather than on a fragment. A PyYAML parse of the result confirms the field types, which is the point of the quoting:

'cff-version'   = '1.2.0'       type=str
'version'       = '1.5.0b5'     type=str
'date-released' = '2026-09-22'  type=str      # not a datetime.date

Existing behaviour is unchanged

Three proofs, all run from outside the checkout with the real tree's sha verified unchanged before and after:

  1. The pcapkit/__init__.py rewrite is byte-identical. The old rewrite block and the new rewrite_init were each run on a copy of the real file: same SHA-256 (4f0782de…), same length (4479 → 4479 bytes), tail __version__ = '1.5.0b5'\n.
  2. End to end on the primary path (pcapkit importable, via a stub package): old and new produce byte-identical pcapkit/__init__.py and conda/build (0). conda/build is still written relative to the working directory, exactly as before.
  3. The fallback path: old exits 1 and leaves the version untouched; new exits 0 and bumps correctly.

Scoped test runs (never the whole tree — a coverage run -m pytest tests/ here reached 41 GB RSS):

tests/project                      96 passed, 469 subtests   (was 64 before the 32 added)
tests/test_tier_guard.py           25 passed, 19 subtests
tests/test_docstring_contract.py    7 passed, 12 subtests

Measured against the worktree, not the stale main checkout — pcapkit.__file__ asserted to be inside this branch's tree before anything else was imported.

Cross-review

Reviewed by an independent agent on a different model (Sonnet; the change was authored on Opus). Verdict: GOOD TO GO, no blocking changes. It independently re-derived the byte-identical __init__.py rewrite (same SHA-256), audited every write in the new script, attacked plan_citation with its own adversarial fixtures and validated the outputs with cffconvert, confirmed the bash -e step ordering in cron-vendor.yml, and spot-checked the never-published versions against PyPI.

One disagreement, resolved against the reviewer. It measured the hand-authored share as 10 of the most recent 25 rather than 11, and called the "quarter" figure wrong. Re-measured directly — git log -G'__version__ = ' -- pcapkit/__init__.py over origin/main — gives 159 commits, 119 bot, 40 human: 25.2% lifetime and 11 of the most recent 25 (44%). Both original figures were right; the reviewer's window was "commits touching pcapkit/__init__.py", which is not the same set. But the criticism landed anyway: the prose put a 25% claim next to an 11-of-25 figure without saying they measured different windows, which reads as self-contradictory. Reworded here, in the script docstring, in the test docstring and in the changelog entry.

Two non-blocking notes it raised were also acted on: the trailing-inline-#-comment limit and the version : space-before-colon refusal now each have a dedicated test (the two added since the first push), and test_the_release_date_defaults_to_today_in_utc no longer has a one-in-86400 flake at a UTC midnight boundary — it accepts either bound around the call, and additionally asserts the local date was not used whenever the runner's local date differs from UTC, which it does on the machine this was developed on (local 2026-09-21, UTC 2026-09-22).

Found and deliberately not fixed

  • conda/build is still resolved relative to the working directory, where CITATION.cff is resolved relative to the repository root (the spelling util/changelog_md.py documents and uses, so that the script answers the same from anywhere). The two disagree and the root-relative form is the better one, but changing conda/build is a behaviour change for every existing caller and belongs in its own review.
  • CITATION.cff is still not in the sdist. MANIFEST.in has no include CITATION.cffdocs: add CITATION.cff and bring the copyright notice up to date #615 flagged this and docs(readme): trim to a landing page and convert to Markdown #619 did not add it. It does not affect the GitHub button, which reads the repository, but it does mean RepositoryCitationTests skips when run from an unpacked sdist, which is why it is guarded.
  • docs/source/conf.py starts its copyright at 2017 where LICENSE starts at 2018. Inherited from docs: add CITATION.cff and bring the copyright notice up to date #615's own "not changed" list; still true, still the owner's call.
  • PyPI's info.version for this project reports 1.4.1.post2, not 1.5.0b4, because 1.5.0b4 is a prerelease and that field reports the latest stable. Anything that ever validates CITATION.cff against "latest on PyPI" through that field will disagree with the checked-in value.
  • Eight consecutive bumps in 2024 were tagged but never published (1.3.1.post10post17). Why is not established — I worked from commit, tag and PyPI metadata, not from Actions run logs — but it is the concrete reason date-released at bump time is a prediction rather than a fact.
  • packaging is an undeclared test dependency now. util/bump_version.py imports it, and this is the first test module to load that script, so the suite now needs packaging where it did not before. It is present on every path that can run pytest at all — pip show pytest gives Requires: iniconfig, packaging, pluggy, pygments — but it is nowhere in pyproject.toml. Adding it to the test extra would make that explicit; pyproject.toml was outside this change's scope.

@JarryShaw
JarryShaw force-pushed the fix/bump-version-citation-cff branch from d4b725b to 586e5b6 Compare September 22, 2026 03:42
Requested by the owner: "we have version cited in CITATION.cff, might need to
have the version_bump.py handle that as well".

* `util/bump_version.py` now rewrites `version` and `date-released` in
  `CITATION.cff` alongside `__version__` in `pcapkit/__init__.py`. Nothing else
  in the repository maintains that file, so every bump left it naming the
  previous release -- and GitHub renders it as the "Cite this repository" button.
* The rewrite is line-oriented, so the comment header, key ordering and each
  field's existing quoting survive; `cff-version` and a `references` entry's own
  `version` are anchored out at column zero. An absent file is reported on stderr
  and skipped rather than failing the vendor cron before its commit; a file
  present with no `version` field raises, before anything is written.
* The script gains a `main()` guard. It previously ran the bump at import, which
  is why it had no testable surface at all.
* Fixed while here: the `import pcapkit` fallback in the version reader returned
  `"1.5.0b4'\n"`, quote and newline included, which `packaging` rejects. That
  path had never worked; CI never reaches it because the package is installed
  first.
* New `tests/project/test_bump_version.py`, 32 tests against a temporary CFF
  fixture -- the repository's own file is only ever read, by a gate asserting it
  still names the packaged version.

`tests/project` 96 passed, tier guard and docstring contract green. The rewrite of
`pcapkit/__init__.py` and `conda/build` is byte-for-byte identical to before, and
the produced CFF passes `cffconvert --validate`.
@JarryShaw
JarryShaw force-pushed the fix/bump-version-citation-cff branch from 586e5b6 to 3ad3096 Compare September 22, 2026 03:55
@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-review (independent agent, Sonnet — the change was authored on Opus): GOOD TO GO, no blocking changes.

Two non-blocking notes acted on: dedicated tests for the trailing-inline-comment limit and for the version : space-before-colon refusal, and the UTC-midnight flake removed from the date-default test.

One disagreement, resolved against the reviewer: its 10-of-25 hand-authored figure used a different commit set. git log -G'__version__ = ' -- pcapkit/__init__.py over origin/main gives 40 of 159 lifetime (25%) and 11 of the most recent 25 (44%), so both original figures were right — but the prose juxtaposed those two windows without saying so, which read as self-contradictory, and has been reworded here and in the changelog entry.

Full write-up in the PR description.

@JarryShaw
JarryShaw merged commit a411377 into main Sep 22, 2026
12 of 24 checks passed
@JarryShaw
JarryShaw deleted the fix/bump-version-citation-cff branch September 22, 2026 04:31
JarryShaw added a commit that referenced this pull request Sep 22, 2026
Requested by the owner; no issue is open for it.

* `MANIFEST.in` had `include` lines for `README.md`, `LICENSE` and
  `CHANGELOG.md` but none for `CITATION.cff`, and its two `global-include`
  patterns are `*.rst` and `*.py`, neither of which matches a `.cff`. The file
  was therefore in the repository and in no source distribution.
* The gap is invisible from the web UI, since GitHub renders the "Cite this
  repository" button from the repository. Citation managers, Zenodo and
  dependency inventories read the published artifact, which is the surface that
  was missing it.
* It matters now because #625 has just taught `util/bump_version.py` to keep
  that file's `version` and `date-released` in step with the bump, so a release
  exercising that path would publish an sdist omitting the artefact under test.
  #615 flagged the omission when it added the file; #619 did not address it.
* The new line carries a comment, as its neighbours do, saying why it is
  load-bearing rather than belt-and-braces.
* Changelog bullet added; `CHANGELOG.md` regenerated with
  `util/changelog_md.py`, and `--check` exits 0.

Measured both ways with `python -m build --sdist`: `tar tzf | grep -i citation`
found nothing before and `pypcapkit-1.5.0b4/CITATION.cff` after; the two archive
listings differ by that one entry and nothing else, 860 against 861; the shipped
copy is byte-identical to the repository's; and `twine check --strict` reports
PASSED on both. No tests were run -- one line of packaging configuration,
nothing under `pcapkit/`.
JarryShaw added a commit that referenced this pull request Sep 22, 2026
Requested by the owner; no issue is open for it.

* `MANIFEST.in` had `include` lines for `README.md`, `LICENSE` and
  `CHANGELOG.md` but none for `CITATION.cff`, and its two `global-include`
  patterns are `*.rst` and `*.py`, neither of which matches a `.cff`. The file
  was therefore in the repository and in no source distribution.
* Nor was there a default to fall back on. Removing all three `include` lines
  and rebuilding shows `README.md` and `LICENSE` shipping regardless --
  setuptools adds the latter from `license_files`, logging `adding license file
  'LICENSE'` and recording `License-File: LICENSE` in `PKG-INFO` -- while
  `CHANGELOG.md` disappears. Of the three only `CHANGELOG.md` is load-bearing,
  and a citation file, which nothing outside `MANIFEST.in` knows about, is in
  the same position.
* The gap is invisible from the web UI, since GitHub renders the "Cite this
  repository" button from the repository. Citation managers, Zenodo and
  dependency inventories read the published artifact, which is the surface that
  was missing it.
* It matters now because #625 has just taught `util/bump_version.py` to keep
  that file's `version` and `date-released` in step with the bump, so a release
  exercising that path would publish an sdist omitting the artefact under test.
  #615 flagged the omission when it added the file; #619 did not address it.
* Changelog bullet added; `CHANGELOG.md` regenerated with
  `util/changelog_md.py`, and `--check` exits 0.

Measured both ways with `python -m build --sdist`: `tar tzf | grep -i citation`
found nothing before and `pypcapkit-1.5.0b4/CITATION.cff` after; the two archive
listings differ by that one entry and nothing else, 860 against 861; the shipped
copy is byte-identical to the repository's; and `twine check --strict` reports
PASSED on both. No tests were run -- one line of packaging configuration,
nothing under `pcapkit/`.
JarryShaw added a commit that referenced this pull request Sep 22, 2026
Requested by the owner; no issue is open for it.

* `MANIFEST.in` had `include` lines for `README.md`, `LICENSE` and
  `CHANGELOG.md` but none for `CITATION.cff`, and its two `global-include`
  patterns are `*.rst` and `*.py`, neither of which matches a `.cff`. The file
  was therefore in the repository and in no source distribution.
* Nor was there a default to fall back on. Removing all three `include` lines
  and rebuilding shows `README.md` and `LICENSE` shipping regardless --
  setuptools adds the latter from `license_files`, logging `adding license file
  'LICENSE'` and recording `License-File: LICENSE` in `PKG-INFO` -- while
  `CHANGELOG.md` disappears. Of the three only `CHANGELOG.md` is load-bearing,
  and a citation file, which no packaging default covers at all, is in the same
  position.
* The gap is invisible from the web UI, since GitHub renders the "Cite this
  repository" button from the repository. Citation managers, Zenodo and
  dependency inventories read the published artifact, which is the surface that
  was missing it.
* It matters now because #625 has just taught `util/bump_version.py` to keep
  that file's `version` and `date-released` in step with the bump, so a release
  exercising that path would publish an sdist omitting the artefact under test.
  #615 flagged the omission when it added the file; #619 did not address it.
* It also lets `RepositoryCitationTests` in `tests/project/test_bump_version.py`
  -- the gate #625 added for the hand-authored bumps that never run the script --
  execute against an unpacked sdist, where today it skips itself with
  "CITATION.cff is not shipped in the source distribution".
* Changelog bullet added; `CHANGELOG.md` regenerated with
  `util/changelog_md.py`, and `--check` exits 0.

Rebased onto `cfb81d3f6` after #630 merged; the changelog conflict was resolved
keeping both bullets and regenerating `CHANGELOG.md` rather than merging it.

Measured both ways with `python -m build --sdist` against that base: `tar tzf |
grep -i citation` found nothing before and `pypcapkit-1.5.0b4/CITATION.cff`
after; the listings differ by that one entry and nothing else, 860 against 861;
the shipped copy is byte-identical to the repository's; and `twine check
--strict` reports PASSED on both. No tests were run -- one line of packaging
configuration, nothing under `pcapkit/`.
@JarryShaw JarryShaw added the feat Pull requests that add a new capability (feat: subject prefix) label Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat Pull requests that add a new capability (feat: subject prefix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant