Skip to content

release.sh: a merge to main during the bump PR's CI wait strands the release #1451

Open

Description

@JacobStephens2

This was generated by AI during triage.

What happened

A merge to main while scripts/release.sh waits for CI on the bump PR makes the release refuse, and rerunning the same version can never succeed. On 2026-10-27, release.sh 0.16.6 cut release-0.16.6 from main at 662f223 and opened #1442. PR #1437 merged at 10:27:40 while #1442's CI ran (10:22 to 10:41), and the script refused with:

refusing to release 0.16.6: main contains content absent from the measured Candidate; choose a fresh release version to measure current main

Recovery took closing #1442 and deleting release-0.16.6 on origin by hand, then rerunning 0.16.6 from current main. That cost about 40 minutes of CI waits, and a second merge during the rerun would have stranded it again.

Why it happens

wait_and_merge calls validate_main_content after CI passes. It requires bump_tree origin/main to equal the bump PR's tree, so any new content on main fails it. That check came in with #977 to stop a release from publishing content its Experiment didn't measure, which is correct when an Experiment ran. Two things make it worse than it needs to be:

  1. A rerun can't recover. With the bump PR still OPEN, find_earlier_run resumes it, and wait_and_merge refuses again every time. The message says to pick a fresh version, but it doesn't say the stale PR still has to be closed.
  2. The refusal applies even with no Experiment. The release Experiment is now skipped by default ("use --with-experiment to run it"), so there is no measured Candidate to protect, but the script still treats main moving as fatal.

#976 fixed the same race for the CI wait on main before the bump, by following main's new tip. This is the same race one step later.

Possible directions (for triage)

  • When no Experiment ran: on main moving, close the stale bump PR, delete its branch, and re-cut the bump from main's new tip, with a progress line naming the move, as release.sh: keep waiting on main's new tip when main moves during the CI wait #976 did.
  • When an Experiment ran: keep refusing, but name the stale PR and the cleanup in the message (close the PR, delete release-<version> on origin), or close it as part of the refusal so rerunning the same version starts over.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds-triageMaintainer needs to evaluate this issue

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions