[backspace]

Background and Goal

The Percy nonce had the run attempt in it, so "Re-run failed jobs" made a new Percy build that held only the snapshots of the shards that ran again. That build showed "N missing snapshots found", or "Visual review requested but no snapshots were uploaded" when the re-run shard takes no snapshots. On main, such a build became the baseline, and later PR builds then showed 50 to 118 snapshots with no baseline. RunsOn starts these re-runs automatically after a spot interruption, so this happened often. The attempt was added to the nonce in #1333, because attempt 1 always sealed its build and a re-run could not add to a sealed build.

This PR keeps one Percy build for each run:

1. The nonce is ${{ github.run_id }}.
2. Finalize seals the build only when every host shard passed. A run with a failed shard leaves the build open, and a notice says so.
3. "Re-run failed jobs" uses the same nonce, so the re-run shards add their snapshots to the open build. The finalize job runs again because it depends on host-test, and it seals the build when every shard has passed.
4. The finalize job reads the jobs API to learn which host shards ran in this attempt and whether an earlier attempt sealed the build. A shard that ran in this attempt counts only this attempt's tally; a shard carried over from an earlier attempt counts its latest one; after a seal, no earlier tally counts.
5. On main, a build that a re-run of failed shards completed is rejected when a re-run shard took snapshots in an earlier attempt, or left no tally there. A shard that failed before any snapshot does not cause a reject.
6. Finalize is skipped when no host shard ran in this attempt and an earlier attempt sealed the build, so re-running the finalize job alone does not make an empty build.

Key decisions and non-obvious mechanics

• Verified on a test branch (run 39424977100, Percy build 54755379). Attempt 1 failed shards 4, 6 and 9. Finalize was skipped, and the build stayed pending with 58 snapshots. After "Re-run failed jobs", the same build was sealed with 60 snapshots and no diffs.
• A shard that uploaded snapshots and then failed uploads them again when it re-runs. Percy logs "Ignored duplicate snapshot … this name already exists in the build" and keeps the first copy, and the shard passes. So a re-run cannot replace a snapshot taken in a failed attempt.
• "Re-run all jobs" after the build was sealed uses the same nonce, and Percy starts a new build for it. Verified on the test branch: attempt 3 of run 39424977100 reused the nonce of sealed build 54755379, and Percy created build 54787532 with all 60 snapshots.
• A job that "Re-run failed jobs" carries over keeps its earlier started_at, and its run_attempt field reports the new attempt. So the finalize job compares started_at with the attempt's run_started_at to find the shards that ran. If the jobs API cannot answer, it falls back to counting each shard's latest tally and treating every later attempt as a re-run.
• A run that nobody re-runs leaves its build open, and its Percy status stays "processing". A new push starts a new run and a new build. We have not measured how long Percy keeps an open build before it times out.
• The reject on a host-test failure and the partial re-run check from Keep partial Percy builds out of the main baseline #6854 are replaced. A build with a failed shard is no longer sealed. Percy keeps the first copy of each snapshot name, so a build that a re-run completed can hold a snapshot that a re-run shard took in the attempt where its test failed; on main that build is rejected (item 5). Pull requests keep it for review. The reject on a tally shortfall on main stays, because a shard can pass and still lose an upload.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Jtn9kCHY8C4FfiQkzHPAgr

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-21T08:15:03.496288Z	42857de	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 42857dee86

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

Host Test Results

    1 files  ±0      1 suites  ±0   1h 28m 17s ⏱️ - 2m 28s
5 842 tests +1  5 831 ✅ +2  11 💤 ±0  0 ❌ ±0 
5 857 runs  +1  5 846 ✅ +3  11 💤 ±0  0 ❌  - 1 

Results for commit 01e4e1c. ± Comparison against earlier commit 270db6c.

Realm Server Test Results

    1 files  ±0    359 suites  ±0   2h 6m 5s ⏱️ + 5m 12s
5 972 tests ±0  5 972 ✅ ±0  0 💤 ±0  0 ❌ ±0 
6 035 runs  ±0  6 035 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 7bfa986. ± Comparison against earlier commit 7dba379.

[richardhjtan]

[Claude Code 🤖] Reviewed the Percy nonce, seal-marker and finalize/reject logic for how a build spanning several attempts reaches main's baseline.

One blocking issue: on main, a build sealed after "Re-run failed jobs" now becomes the baseline carrying the failed attempt's snapshots, with no reject left to stop it. Detail inline on the reject step's condition.