Limit the multiversion docs build to the newest minor versions

[laughingman7743]

WHAT

• docs/conf.py computes smv_tag_whitelist from the git tags: the latest patch release of each of the SMV_MINOR_VERSIONS = 3 newest minor versions (currently v3.36.0, v3.35.4, v3.34.0), plus the latest tag of the previous major version when those do not include it and the tag contains docs/conf.py (v2 tags have no Sphinx docs, so nothing is added today; after v4.2.0 the set is v4.2.0, v4.1.0, v4.0.0, v3.36.x). master is still built. Only the top-level sphinx-multiversion process uses the whitelist; where conf.py is evaluated outside a git repository (each version's exported tree), the selection is empty and unused.
• New docs/_root/404.html, copied to the site root by just docs build, is the GitHub Pages custom 404. It redirects /vX.Y.Z/<path> (with query and hash) to /master/<path> when the page is missing, so links to versions that are no longer built land on the current docs. A page missing on master shows a plain not-found page with a link to master; the path no longer matches, so it does not loop.
• The version switcher (docs/_templates/versioning.html) lists master first, then tags from newest to oldest by numeric version, through a versions_newest_first context value added in docs/conf.py. sphinx-multiversion's own versions lists tags in ref name order (oldest first, and v4.10.0 before v4.9.0) followed by master.
• docs/testing.md describes what just docs build builds.

The Docs workflow already drops unreleased tags before the build, so only released tags are selected there. The version switcher lists only the built versions.

WHY

Closes #868. In Docs workflow run 36324334450 (19 minutes), Sphinx spent about 16 minutes rebuilding all 53 released tags and 46 seconds on master, and the time grows with every release; the docs-lint PR check runs the same build.

TEST

Tested commit 3e82d85; 6feb71c only qualifies the previous-major sentence in docs/testing.md.

• Tag selection in a scratch git repository (v2.25.2 without docs/conf.py, later tags with it), calling _select_documented_tags(3):
  • v3.34.0, v3.35.4, v3.36.0 → v3.36.0, v3.35.4, v3.34.0 (v2.25.2 not added: no Sphinx docs)
  • • v4.0.0 → v4.0.0, v3.36.0, v3.35.4; + v3.36.1 → v4.0.0, v3.36.1, v3.35.4
  • • v4.1.0, v4.2.0 → v4.2.0, v4.1.0, v4.0.0, v3.36.1
  • • v4.9.0, v4.10.0 → v4.10.0, v4.9.0, v4.2.0, v3.36.1; + v5.0.0 → v5.0.0, v4.10.0, v4.9.0
  • v3.37.0rc1 and foo are ignored; outside a git repository the result is [].
• In this repository, smv_tag_whitelist is ^(v3\.36\.0|v3\.35\.4|v3\.34\.0)$.
• just docs build → succeeded in 99 s locally with 4 Sphinx builds; output contains master/, v3.34.0/, v3.35.4/, v3.36.0/, index.html, 404.html, CNAME, .nojekyll. The dropdown on master/index.html and v3.34.0/index.html lists master, v3.36.0, v3.35.4, v3.34.0 in that order.
• 404 redirect (tested on 9e3235e; 404.html has only a comment change since), with a local server that serves 404.html for missing paths, in Chromium via Playwright:
  • /v3.5.0/usage.html?x=1#connection → /master/usage.html?x=1#connection
  • /v3.5.0 → /master/
  • /v3.5.0/removed-page.html → /master/removed-page.html, 404 page shown, no further redirect
• just lint (including license headers) and just docs lint passed.
• CI docs-lint build on cca5ad2 (run 36331604190): 4 Sphinx builds, 3 min 30 s; on 588085d: 3 min 14 s. Recent docs-lint runs on other PRs took 19–20 min.
• Not verified before merge: GitHub Pages serving the root 404.html on pyathena.dev, and the Docs deploy workflow duration. Check both after merge. No AWS resources are involved.

🤖 Generated with Claude Code

[laughingman7743]

Self-review round one (implementation behavior): CLEAN

Scope: git diff 645e04ebf9f818dfcc5336bd8af66adf1587f33a..9e3235e81fb5ed256e88f53cbe2ecec679ef54c1. Files: docs/conf.py, docs/_root/404.html, justfile, docs/testing.md.

Checked:

• Where the whitelist is read (sphinx-multiversion 0.2.4 main.py): smv_tag_whitelist comes only from the top-level load_sphinx_config(confdir_absolute), which runs inside working_dir(confdir), so git tag runs in the checkout. conf.py is loaded again from each ref's exported tree and in each per-version sphinx-build subprocess, but those runs are not in a git repository. git tag raises CalledProcessError there and the function returns []. The extension only registers the whitelist value and never reads it (sphinx.py), so those runs are unaffected.
• Tag selection: the vX.Y.Z regex keeps the tag filter the old whitelist used. Versions are compared as integer tuples, so v3.9 < v3.10. The newest-first sort plus setdefault keeps the latest patch of each minor. After a 4.0.0 release, patch releases from the 3.x maintenance branch still sort correctly (e.g. 4.0.x, 3.36.1, 3.35.4).
• Released tags only: in docs.yaml, the "Drop unreleased version tags" step runs before just docs build. The docs-lint PR check skips that step, so it could pick up an in-flight tag during a release window, but it never deploys.
• Build output: .html is not in source_suffix (.rst/.md), so Sphinx ignores docs/_root/. A local build produced no master/_root. docs-lint.yaml triggers on docs/** and justfile, so the PR check covers these files.
• 404 redirect: the regex matches /vX.Y.Z with or without a trailing path. The redirect keeps the query and hash. A page missing on master no longer matches, so there is no loop (Chromium check recorded in the PR TEST section).
• Simplicity: the change follows the existing get_version() subprocess pattern and adds no new dependency. No tests are needed; this is build configuration, and the build plus --dump-equivalent checks exercise it.

Findings: none.

[laughingman7743]

Repair record (588085d), round one and round two applied to the repair

Correction to round one: I wrote that each per-version sphinx-build subprocess runs outside a git repository. That is wrong. The subprocess gets -c <invoking checkout>/docs, and Sphinx's eval_config_file runs with chdir(filename.parent), so git tag succeeds there. The whitelist it computes is registered but never read (sphinx_multiversion/sphinx.py), so behavior is unchanged. Only per-ref config reads from exported trees (main.py load_sphinx_config(confpath), inside working_dir(confpath)) lack git. The independent review found this.

• Implementation perspective: the repair is docstring-only (git diff cca5ad2..588085d). just lint passed, and the whitelist is still ^(v3\.36\.0|v3\.35\.4|v3\.34\.0)$.
• Claims perspective: the docstring now names the exported-tree config reads as the git-less case and says only the invoking checkout's selection is used. The PR body's WHAT bullet was corrected to match. No other prose repeats the old claim.

[laughingman7743]

Self-review round two (claims and operational behavior): FINDINGS, repaired

Scope: full pass over git diff 645e04ebf9f818dfcc5336bd8af66adf1587f33a..9e3235e81fb5ed256e88f53cbe2ecec679ef54c1, the PR body, the commit message, and issue #868. The repair was then checked at cca5ad2e6d3a8a8940d8c52ffebdcfbdac3d44b8.

Claims checked:

• "17 of its 19 minutes rebuilding all 53 released tags" (PR WHY): inaccurate, corrected. The log of run 36324334450 shows 54 Sphinx builds totaling 1010 s. master took 45.6 s, so the 53 tags took about 964 s (16 min), plus about 80 s between builds. The PR body now says "about 16 minutes rebuilding all 53 released tags and 46 seconds on master". Issue Limit the multiversion docs build to the latest patch of the 3 newest minor versions #868 already stated "about 17 minutes of Sphinx time" and "master alone took about 46 seconds", which is accurate.
• "The docs-lint PR check runs the same build": confirmed. docs-lint.yaml runs just docs build with fetch-depth: 0. It has no unreleased-tag drop step, as noted in round one.
• Measured effect: confirmed on CI. The docs-lint build job for this PR (run 36331604190, head cca5ad2) ran 4 Sphinx builds (master, v3.34.0, v3.35.4, v3.36.0) and finished in 3 min 30 s. Recent docs-lint runs on other PRs took 19–20 min (36330087767, 36320484140, 36308386844). The Docs deploy workflow runs the same recipe, but its post-merge duration is not measured yet.
• "Exported trees" (docstring): confirmed. sphinx_multiversion/git.py:copy_tree extracts git archive output, so the per-version trees have no .git.
• "The version switcher lists only the built versions": confirmed. The dropdown in the local build's master/index.html has exactly 4 options.
• 404 comment: imprecise, repaired in cca5ad2. The old comment said "Versions that are no longer built redirect...". The script redirects any missing page under a /vX.Y.Z/ path, including a missing page of a built version, so the comment now says that. The PR body's WHAT bullet was aligned the same way.
• Documentation reader: docs/testing.md is the only prose that describes just docs build. git grep finds no hard-coded pyathena.dev/vX.Y.Z links in the repository.
• Operator: the Pages deploy replaces the whole site, so the removed version directories disappear on the next deploy. The root 404.html covers them. Its status is still 404, so search engines drop the old URLs. This PR does not verify that GitHub Pages serves the root 404.html on the custom domain; that is listed as a post-merge check in TEST.

Other findings: none.

[laughingman7743]

Independent review (relayed): CLEAN; one non-actionable observation repaired

• Reviewer: OpenAI Codex CLI 0.157.1, model gpt-6-astra, reasoning effort high, sandbox read-only, session 01a0e39d-54a1-7f50-9842-f0a8fd9f3194.
• Scope: base 645e04ebf9f818dfcc5336bd8af66adf1587f33a, head cca5ad2e6d3a8a8940d8c52ffebdcfbdac3d44b8, on a detached snapshot.
• Supplied: the literal diff, sphinx-multiversion 0.2.4 sources, and repository conventions. Not supplied: the PR text or author conclusions.
• This was a static review: no edits, builds, tests, or GitHub access. The snapshot and the PR worktree were unchanged afterwards.

Covered: all four changed files; initial config loading, exported per-ref config, per-version subprocesses, and the switcher metadata; tag ordering, multiple majors, maintenance releases, non-matching tags, and an empty tag set; 404 placement, custom-domain paths, query/hash preservation, and loop prevention; the local build, docs-lint, deploy filtering, and the changed docs.

Verdict CLEAN. Non-actionable observations:

1. The docstring here was imprecise about per-version builds: those subprocesses evaluate conf.py in the invoking checkout, where git is available. I checked Sphinx eval_config_file (with chdir(filename.parent)) and confirmed this. Repaired in 588085d; see the repair reply on the round-one thread.
2. just docs build does not prune existing docs/_build/html output, so a repeated local build can keep directories for versions that are no longer selected. Not changed: this is pre-existing recipe behavior, and CI builds from fresh checkouts.

Independent follow-up on the repair: same reviewer and settings, session 01a0e3a2-a5b8-7871-9024-8fb8c6156ff8, repair diff cca5ad2..588085ddfab82be4b36561766cbbee52b70c71aa. Verdict CLEAN: the new docstring matches the exported-tree and -c build paths, and the repair introduces no other issue.

[laughingman7743]

Self-review round one, expanded scope (implementation behavior): CLEAN

Scope: the full diff 645e04ebf9f818dfcc5336bd8af66adf1587f33a..3e82d85cf60e7b43dbca47769232b8c605e841e1. The maintainer added two requirements: keep the previous major version's latest release, and list versions newest first. The contract grew, so this pass reviews the whole diff again.

Checked:

• Hook ordering: Sphinx EventManager.emit sorts listeners by ascending priority (sorted(..., key=attrgetter('priority'))). sphinx-multiversion sets versions in its html-page-context handler at the default 500. This handler (priority 600) runs after it. conf.py's setup() also runs after all extensions are set up (application.py).
• Switcher context: versions.branches + versions.tags sorted by _parse_version_tag(item.name). Tags always parse because the whitelist admits only vX.Y.Z. item == current_version compares Version namedtuples built from the same metadata; the local build renders "(current)" on the right entry on both the master and v3.34.0 pages. A plain sphinx-build (the working-tree check in testing.md) has no versions, so the switcher is omitted, as before.
• Previous major: the first tag whose major is lower than the newest one is the latest release of the previous major. It is added only when it is not already selected and git cat-file -e <tag>:docs/conf.py succeeds. That check runs only after git tag has succeeded in the invoking checkout. Exported trees return early.
• Selection: a scratch-repository simulation (PR TEST section) covered the current tags, 4.0.0, a 3.36.1 maintenance patch, 4.1/4.2, 4.9/4.10 numeric order, 5.0.0, rc/non-version tags, and a directory without git.
• Simplicity: _parse_version_tag is shared by the selection and the switcher order. There are no new dependencies and no changes to the workflow or the recipe.

Findings: none.

[laughingman7743]

Self-review round two, expanded scope (claims and operational behavior): CLEAN

Claims checked against evidence at 3e82d85cf60e7b43dbca47769232b8c605e841e1:

• "Tags before v3.5.0, including all v2 tags, have no docs/conf.py" (docstring): confirmed. git cat-file -e <tag>:docs/conf.py over all 183 v* tags in version order gives 130 without the file, then 53 with it, and the first with it is v3.5.0.
• "sphinx-multiversion's own versions lists tags in ref name order ... followed by master": confirmed. git.get_all_refs reads git for-each-ref (sorted by refname), and VersionInfo.__iter__ yields tags then branches. The previous build rendered v3.34.0, v3.35.4, v3.36.0, master. "v4.10.0 before v4.9.0" follows from string order.
• PR body example "after v4.2.0 the set is v4.2.0, v4.1.0, v4.0.0, v3.36.x": confirmed by the simulation.
• "nothing is added today": confirmed. The whitelist in this repository is unchanged: ^(v3\.36\.0|v3\.35\.4|v3\.34\.0)$, with 4 Sphinx builds.
• docs/testing.md wording matches the implemented rule.
• Issue Limit the multiversion docs build to the latest patch of the 3 newest minor versions #868 has a comment recording the maintainer's scope update.
• Operator: each deploy now also builds at most one extra version (the previous major's latest), about 40 s. Deploy-time filtering of unreleased tags applies to the previous-major candidate too, because it comes from the same git tag output.

Findings: none.

[laughingman7743]

Independent review, expanded scope (relayed): FINDINGS (1 × P3), repaired; follow-up CLEAN

• Reviewer: OpenAI Codex CLI 0.157.1, model gpt-6-astra, reasoning effort high, sandbox read-only, session 01a0e51e-72a0-70f2-a62b-04205fb7ce18.
• Scope: full diff from base 645e04ebf9f818dfcc5336bd8af66adf1587f33a to head 3e82d85cf60e7b43dbca47769232b8c605e841e1, on a detached snapshot.
• Supplied: sphinx-multiversion 0.2.4 and Sphinx 8.2.3 events.py sources. Not supplied: the PR text or author conclusions.
• This was a static review; the snapshot and the PR worktree were unchanged afterwards.

Covered: whitelist consumption, per-ref config loading and per-version subprocesses; numeric ordering, minor grouping, previous-major eligibility, maintenance releases, non-matching tags, and no tags; event priority, template resolution, switcher ordering, URLs, and current-version selection; the 404 redirect, the recipe, both workflows, and the changed text.

Finding (P3), this line: the sentence promised an unconditional previous-major build. With today's tags the candidate is v2.25.2, which has no docs/conf.py and is correctly excluded. Verified and repaired in 6feb71c4320c90a566d3fd9f48ea654d70f8ae43, which adds "if that tag contains docs/conf.py".

No functional defects. Non-actionable observations: local rebuilds keep stale output directories (pre-existing recipe behavior; CI builds from fresh checkouts). Local builds and docs-lint select from all matching tags; only the Docs deploy drops unpublished tags (noted in round one).

Repair, checked from both self-review perspectives:

• Implementation: text-only; just lint and just docs lint pass.
• Claims: the sentence now matches _select_documented_tags and _has_sphinx_docs, and the PR body already stated the condition.

Independent follow-up: same reviewer and settings, session 01a0e520-7b51-7e22-8d44-b6f77f854c3f, repair diff 3e82d85..6feb71c. Verdict CLEAN.