fix(gates): gap suite could neither pass nor fail on macOS - #7568
Merged
Conversation
added 2 commits
August 7, 2026 05:36
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe gap test runner now falls back to the shared baseline when an implicit platform snapshot is missing. Explicit missing snapshots remain errors with exit code 2. Version metadata and the changelog are updated. ChangesGap snapshot fallback and release metadata
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: Suggested reviewers: ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
run_gap_tests.shselectstest-parity/gap_snapshot.${platform}.jsonfor non-Linux hosts. There is nogap_snapshot.macos.jsonin the repo, so every local macOS run did all ~490 tests' worth of work and then died in aFileNotFoundError— an exit 1 indistinguishable from "here is your regression list".This is the fifth gate pathology found this campaign, and a notable one: the useful per-test results were only recoverable by reading the log by hand, which is literally how this session's verification runs were read.
A missing platform baseline now says so and falls back to the shared baseline — the one required CI uses, so a local run is measured against the same bar. Announced on stderr rather than applied silently, because the platforms genuinely differ (macOS carries known host-local failures) and quietly comparing against another platform's expectations would be its own kind of lie.
An explicitly-passed
GAP_SNAPSHOT=that does not exist stays a hard error (exit 2) — the fallback is for the unset default, not for a typo.Found by an agent whose gap run failed this way while validating #7565.
Summary by CodeRabbit
Bug Fixes
Documentation
Chores