Stamp bin/ interpreters with the usermod commits they were built from, so a gate can refuse a binary older than the code it certifies #27

Activity

1. bdbarnett

   added

   housekeepingThe 2026-09-22 sweep: workable from a desk, a build, or a board hereThe 2026-09-22 sweep: workable from a desk, a build, or a board here

   on Sep 22, 2026
2. bdbarnett commented on Sep 22, 2026

   

   bdbarnett

   on Sep 22, 2026

   ContributorAuthor

   More actions

   Half 1 is in, on main (f812d35). Half 2 — wiring the gates — is not, and I say why at the bottom.

   build_interpreters.sh now writes bin/<name>.provenance beside every binary it installs: each usermod in cmods with its commit and describe, whether that tree was dirty, the overlay patches for the port, the binary's sha256, and whether it was an --install-only copy rather than a build. scripts/provenance.py check is what a gate calls; it refuses, and names the rebuild command.

   $ python3 scripts/provenance.py check bin/micropython --source audiodsp
   micropython: provenance OK (1 source(s) match the checkout)
   

   The stamp for the binary sitting in bin/ right now:

   audiodsp         v0.0.3-252-g95e58b3
   audioif          v0.1.0-1-g3c418f4
   cameraif         a798eed
   circuitpython    v1.25.0-24997-gd897c15-dirty (dirty)
   cmods            391403c-dirty (dirty)
   displayif        e2e74fa
   lvgl-micropython 126d760
   micropython      v1.29.0-dirty (dirty)
   pygraphics       v0.0.32-76-g7f4d1ab
   usbif            30583bc
   

   Shown refusing, each against that real binary — a checker nobody has seen fail is not evidence:

   the stamp says audiodsp is 5 commits back   REFUSED  "audiodsp: the binary is 5 commits behind
                                                         the checkout. Built from v0.0.3-247-g02b1841,
                                                         the tree is at v0.0.3-252-g95e58b3."   exit 1
   the binary changed after stamping            REFUSED  sha256 named, "Something replaced it
                                                         without stamping it."                  exit 1
   no stamp beside the binary                   REFUSED  "may predate anything."                exit 1
   the real thing                               OK, 10 sources match                            exit 0
   

   And it caught something live on its first bare run. audioif is two commits ahead of this binary already — the T-Embed agent is working in that tree as I write — so bin/micropython is right now an interpreter that cannot contain audioif e2e4e1f. That is exactly the 2026-09-03 shape, visible for the first time.

   Two design calls worth your eye:

   • A dirty tree is a note, not a refusal. Dirty is the normal state in this workspace, and a check that is red every day stops being read — which is the disease, not the cure. It is recorded, because a dirty build is not any commit and nobody can reconstruct it later.
   • The usermod list is derived, never maintained: every directory in cmods with a micropython.mk, because py.mk globs for exactly that, so a module that drifts into the build appears in the stamp without anyone editing a list. The first version of that got it wrong — it listed wasmbridge and three .cpws_* scratch trees as usermods, each reporting cmods' own HEAD under a different name. A usermod is now a sibling repository, checked by git toplevel.

   Not done, and it is the half that closes this: run_instruments_parity.py (audiocomponents), verify_*.py (audioif) and capture_render_reference.py (micropython-vst3) do not call it yet. Those are three other repositories, two of them other agents' working trees this sitting. The call is one line each and the script is stable: python3 <cmods>/scripts/provenance.py check <interpreter> --source audiodsp. Leaving this open for that.

   bin/micropython was rebuilt as this change's proof (mp-unix), sha256 f1eb4d33457dbe82d1f2dc5e40d75fecc0700a422c60e53b8c3c997812cde79a.
3. bdbarnett commented on Sep 22, 2026

   

   bdbarnett

   on Sep 22, 2026

   Last edited by bdbarnett

   ContributorAuthor

   More actions

   Both halves are in. The stamps landed in f812d35/d02da69; this closes the second half — the three gates that run a cmods/bin/ interpreter now refuse a binary whose stamp does not match the tree it certifies.

   Wiring

   gate	asks	commit
   audiocomponents tests/parity/run_instruments_parity.py	does the binary contain AUDIODSP_PIN	072bdd6
   audiocomponents test_effect_kit_11_20's cross-interpreter digest	the same	same
   audioif tests/run_probes.sh (new: the CI job's shape, locally)	does the binary contain audioif HEAD	227e79f
   mpvst tools/check-engine-provenance.py	is the engine behind vstaudio, vstui or audiodsp	0102428

   scripts/provenance.py check grew two things doing it (a635b6c), and both came out of the wiring rather than from planning:

   1. --contains SOURCE=REV. You wrote "audioif HEAD for audioif's own gates; AUDIOIF_PIN for audiocomponents'", and that is exactly right: audiocomponents pins its core, so what it must refuse is a binary built before the pin, not one whose checkout has moved since.
   2. A usermod is a directory, not a repository. mpvst carries vstaudio and vstui inside a repo that is mostly C++ and docs. A repo-wide HEAD comparison would have declared the engine stale on a README, and a gate that is red every day stops being read — which is how the week of green gates this issue is about got its hiding place. So when the checkout has moved, the check now asks whether anything under the usermod's own directory moved with it.

   Proof — each gate shown refusing a planted stale stamp, and passing on the real one

   audiocomponents, with 9018e35 (two commits before the pin) planted in bin/micropython.provenance:

   parity runner  exit 1  "audiodsp: the binary is 2 commits behind the pin it must contain"
                          refused before rendering anything
   kit test       FAILED  same message
   

   and with the stamp put back: parity runner "provenance OK", ran, 0 failures; kit test OK, cpython and micropython both FNV a3e0b411.

   audioif — this one found a real stale binary rather than a planted one:

   cmods/bin/micropython      REFUSED  "2 commits behind the checkout"
   

   and those two commits are 95a6905 and e2e4e1f, both of which touch _audioif.c. It is genuinely stale and nothing in the workspace knew. Against cmods/bin/micropython.exe, whose stamp does name audioif HEAD, the runner ran: three planted faults refused, sink digest f423175c4608d981 on both sides, busio all legs PASS, 32328 bytes out of CircuitPython's docstring example. Planting v0.1.0 in the exe's stamp: REFUSED, "3 commits behind".

   mpvst — the engine was rebuilt (scripts/build-micropython-engine.sh --port unix) so that it has a stamp; three states:

   the unstamped engine that was there   REFUSED  "an engine without one predates the
                                                   stamp itself, so it is older than
                                                   the tree by construction"
   vstaudio planted 64 commits back      REFUSED  "64 commits behind the checkout
                                                   (usermods/vstaudio changed)"
   the repo moved, usermods did not      note, and OK -- "nothing under
                                                   usermods/vstaudio moved, so it
                                                   carries this module's current code"
   the real stamp                        OK, 3 sources
   

   That third row is the control. A suite of only refusals proves a checker always refuses.

   One thing left undone, deliberately. The optional runtime half — exposing the stamp as sys.implementation._pydevices so a board or a REPL can answer the same question — is not here. Every gate that needed it can read the file beside the binary, and a board's firmware is a different artifact from a bin/ interpreter, so it wants its own answer rather than this one. Filed as cmods#36 if you want it.
4. bdbarnett

   closed this as completedon Sep 22, 2026