Skip to content

feat(Preflight): refuse a campaign before it burns compute - #34

Open
sotashimozono wants to merge 1 commit into
mainfrom
feat/preflight
Open

feat(Preflight): refuse a campaign before it burns compute#34
sotashimozono wants to merge 1 commit into
mainfrom
feat/preflight

Conversation

@sotashimozono

Copy link
Copy Markdown
Member

Description

Ported out of FiniteTemperature.jl, where it was written after the same class of defect had cost a campaign four separate times. The shape is always the same: declared in one place, relied on in another, checked nowhere.

declared relied on checked
[datavault] float_format = "auto" the path builder ✗ — it ignored it
beta_quench ⊆ beta_targets a stage finding its initial state ✗ — a comment
a stage knowing its upstream's discretisation the checkpoint directory name ✗ — a comment
the checkpoint dir carrying the distinguishing parameter resuming the right run ✗ — not carried at all

Each is decidable by comparing strings before any physics runs.

Type of Change

  • Feature (enhancement)

Proposed Changes

What lands here is the part that is not specific to one study:

Finding / PreflightReport / launchable / n_errors / n_warns / show a report grouped by layer, so a refusal names its own cause
check_injective! distinct parameter points must get distinct paths
check_opens! a vault that will not open is a finding, not a stack trace out of the checker
on_grid is this value a multiple of the step the loop actually takes
representative_keys one key per point — expand() gives (point × sample)

check_injective! is the one worth having. Under a content-blind float rendering, h = 0.002 and 0.004 both render h0.00, so two points share a directory — and not only their observables. They share the status marker, so the second point is reported complete without ever running. A 168-point sweep finished as 132, with no error anywhere.

Pass status paths as well as data paths: checking only the data half catches the overwrite and misses the skipped work, which is the worse of the two. The docstring says so.

What deliberately stays with the study

Which values must lie on which grid, and whether the artefact a later stage looks for is the one an earlier stage writes. That last one is decisive and cannot be generic — only the study knows what its stages hand each other. But it is always built the same way, and the docstring records the recipe: generate the path from both sides and compare the strings, rather than trusting a comment that says they agree.

Usage or Results

fs = Finding[]
check_injective!(fs, :injective, "sweep.toml", "obs", obs_paths)
check_injective!(fs, :injective, "sweep.toml", "done", done_paths)   # ← don't skip this half
r = PreflightReport(fs)
launchable(r) || (show(r); exit(1))
Preflight | 42  42  1.4s

The tests carry a positive control per check

A gate that has only ever returned ok is indistinguishable from one that cannot fail. Two are worth naming:

  • The collision message is asserted deterministic across repeats. Set iteration order is not, and a CI failure that names a different path each run is unreproducible.
  • representative_keys is shown against its own absence. Counting over all keys reports n_samples - 1 false collisions per point; the test asserts the naive version fails exactly where the correct one passes.

check list

  • test駆動をしたか — 42/42, positive control per check
  • Project.tomlのバージョンを上げたか — 0.4.5 → 0.5.0 (additive, but widens the public surface)

🤖 Generated with Claude Code

Ported out of FiniteTemperature.jl, where it was written after the same class
of defect had cost a campaign four separate times. The shape is always the
same: declared in one place, relied on in another, checked nowhere.

  [datavault] float_format = "auto"   the config asked; the path builder ignored it
  beta_quench ⊆ beta_targets          required in a comment
  a stage knowing its upstream's      required in a comment
    discretisation
  the checkpoint directory carrying   not carried at all
    the parameter distinguishing runs

Each is decidable by comparing strings before any physics runs. What lands here
is the part that is not specific to one study:

  Finding / PreflightReport / launchable / n_errors / n_warns / show
      a report grouped by layer, so a refusal names its own cause

  check_injective!   distinct parameter points must get distinct paths
  check_opens!       a vault that will not open is a finding, not a stack trace
  on_grid            is this value a multiple of the step the loop takes
  representative_keys  one key per point; expand() gives (point × sample)

check_injective! is the one worth having. Under a content-blind float rendering,
h = 0.002 and 0.004 both render "h0.00", so two points share a directory — and
not only their observables. They share the STATUS MARKER, so the second point is
reported complete without ever running. A 168-point sweep finished as 132, with
no error anywhere. Pass status paths as well as data paths: checking only the
data half catches the overwrite and misses the skipped work, which is worse.

What stays with the study: which values must lie on which grid, and whether the
artefact a later stage LOOKS for is the one an earlier stage WRITES. That last
one is decisive and cannot be generic, because only the study knows what its
stages hand each other — but it is always built the same way, and the docstring
says so: generate the path from BOTH sides and compare the strings, rather than
trusting a comment that says they agree.

Tests carry a positive control per check. A gate that has only ever returned
"ok" is indistinguishable from one that cannot fail. Two of them are worth
naming:

  * the collision message is asserted DETERMINISTIC across repeats — Set
    iteration order is not, and a CI failure that names a different path each
    run is unreproducible.
  * representative_keys is shown against its own absence: counting over all
    keys reports n_samples-1 false collisions per point, and the test asserts
    the naive version fails where the correct one passes.

Minor bump: additive, but it widens the public surface.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the enhancement New feature or request label Aug 31, 2026
@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant