feat(Preflight): refuse a campaign before it burns compute - #34
Open
sotashimozono wants to merge 1 commit into
Open
feat(Preflight): refuse a campaign before it burns compute#34sotashimozono wants to merge 1 commit into
sotashimozono wants to merge 1 commit into
Conversation
Ported out of FiniteTemperature.jl, where it was written after the same class
of defect had cost a campaign four separate times. The shape is always the
same: declared in one place, relied on in another, checked nowhere.
[datavault] float_format = "auto" the config asked; the path builder ignored it
beta_quench ⊆ beta_targets required in a comment
a stage knowing its upstream's required in a comment
discretisation
the checkpoint directory carrying not carried at all
the parameter distinguishing runs
Each is decidable by comparing strings before any physics runs. What lands here
is the part that is not specific to one study:
Finding / PreflightReport / launchable / n_errors / n_warns / show
a report grouped by layer, so a refusal names its own cause
check_injective! distinct parameter points must get distinct paths
check_opens! a vault that will not open is a finding, not a stack trace
on_grid is this value a multiple of the step the loop takes
representative_keys one key per point; expand() gives (point × sample)
check_injective! is the one worth having. Under a content-blind float rendering,
h = 0.002 and 0.004 both render "h0.00", so two points share a directory — and
not only their observables. They share the STATUS MARKER, so the second point is
reported complete without ever running. A 168-point sweep finished as 132, with
no error anywhere. Pass status paths as well as data paths: checking only the
data half catches the overwrite and misses the skipped work, which is worse.
What stays with the study: which values must lie on which grid, and whether the
artefact a later stage LOOKS for is the one an earlier stage WRITES. That last
one is decisive and cannot be generic, because only the study knows what its
stages hand each other — but it is always built the same way, and the docstring
says so: generate the path from BOTH sides and compare the strings, rather than
trusting a comment that says they agree.
Tests carry a positive control per check. A gate that has only ever returned
"ok" is indistinguishable from one that cannot fail. Two of them are worth
naming:
* the collision message is asserted DETERMINISTIC across repeats — Set
iteration order is not, and a CI failure that names a different path each
run is unreproducible.
* representative_keys is shown against its own absence: counting over all
keys reports n_samples-1 false collisions per point, and the test asserts
the naive version fails where the correct one passes.
Minor bump: additive, but it widens the public surface.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Ported out of
FiniteTemperature.jl, where it was written after the same class of defect had cost a campaign four separate times. The shape is always the same: declared in one place, relied on in another, checked nowhere.[datavault] float_format = "auto"beta_quench ⊆ beta_targetsEach is decidable by comparing strings before any physics runs.
Type of Change
enhancement)Proposed Changes
What lands here is the part that is not specific to one study:
Finding/PreflightReport/launchable/n_errors/n_warns/showcheck_injective!check_opens!on_gridrepresentative_keysexpand()gives (point × sample)check_injective!is the one worth having. Under a content-blind float rendering,h = 0.002and0.004both renderh0.00, so two points share a directory — and not only their observables. They share the status marker, so the second point is reported complete without ever running. A 168-point sweep finished as 132, with no error anywhere.Pass status paths as well as data paths: checking only the data half catches the overwrite and misses the skipped work, which is the worse of the two. The docstring says so.
What deliberately stays with the study
Which values must lie on which grid, and whether the artefact a later stage looks for is the one an earlier stage writes. That last one is decisive and cannot be generic — only the study knows what its stages hand each other. But it is always built the same way, and the docstring records the recipe: generate the path from both sides and compare the strings, rather than trusting a comment that says they agree.
Usage or Results
The tests carry a positive control per check
A gate that has only ever returned
okis indistinguishable from one that cannot fail. Two are worth naming:Setiteration order is not, and a CI failure that names a different path each run is unreproducible.representative_keysis shown against its own absence. Counting over all keys reportsn_samples - 1false collisions per point; the test asserts the naive version fails exactly where the correct one passes.check list
🤖 Generated with Claude Code