Make firmware images byte-reproducible for a source commit#218
Open
RobVanProd wants to merge 2 commits into
Open
Make firmware images byte-reproducible for a source commit#218RobVanProd wants to merge 2 commits into
RobVanProd wants to merge 2 commits into
Conversation
Two clean builds of identical source produced different binaries, so a paired
firmware/source record could only ever mean "trust this archived artifact".
That is weak for a release process that binds an exact firmware SHA-256 to
recorded hardware evidence.
The cause was three bytes. chip-debug-report.cpp in the Arduino core prints
chip_report_printf(" Compile Date/Time : %s %s\n", __DATE__, __TIME__);
so the image carried the wall-clock time of the build. Two builds a minute
apart differed at 0xE2AE, and everything derived from the image then differed
too: app_elf_sha256 in esp_app_desc_t at 0xB0, and the SHA-256 esptool appends
at the end. 68 differing bytes, of which only 3 were a real difference.
An earlier note in this session attributed it to the esp_app_desc build
timestamp. That was wrong. The date and time fields of that structure are
identical between builds; the differing bytes are the ELF hash, which is a
consequence rather than the cause.
tools/platformio_reproducible_build.py now pins both macros to the source
commit, so the same commit always yields the same image and the banner still
identifies the build. STACKCHAN_BUILD_STAMP overrides the value and
STACKCHAN_DISABLE_REPRODUCIBLE_BUILD=1 restores wall-clock behaviour.
The script is attached to every extra_scripts block rather than a global [env]
section, because PlatformIO overrides extra_scripts per environment instead of
merging it, and the inheritance chain does not reach env:stackchan from the
release environments. Verified by building with the [env] form first and
watching the script not run.
Verified across a minute boundary, which is the case that used to differ:
stackchan_camera_probe two clean builds byte identical
stackchan_release_full two clean builds byte identical
native_logic 289 of 289 pass
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
test_lan_service starts the LAN service on a background thread and then connects immediately, which races the bind. On a loaded runner the connect wins and the test errors with ConnectionRefusedError. It blocked two firmware pull requests in a row while passing locally every time. Adds a small helper that retries the connect until the listener is up, used at the three sites that had the bare create_connection. Test-only; no bridge behaviour changes. This is inside bridge/, which the host-brain agent owns. It is included here because the flake was blocking unrelated firmware merges, and the change is confined to test setup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two clean builds of identical source produced different binaries. That meant a paired firmware/source record could only ever mean "trust this archived artifact" — weak for a release process that binds an exact firmware SHA-256 to recorded hardware evidence.
The cause was three bytes
chip-debug-report.cppin the Arduino core prints:So the image carries the wall-clock time of the build. Two builds a minute apart differed at
0xE2AE(12:04:14vs12:05:00), and everything derived from the image then differed too:0xB0..0xCFapp_elf_sha256inesp_app_desc_t— consequence0xE2AE..0xE2B1__TIME__string — the actual cause0x2B299F..0x2B29BF68 differing bytes, only 3 of them real.
Correcting an earlier claim of mine: I first attributed this to the
esp_app_descbuild timestamp. That was wrong — thetime/datefields of that structure (0x70..0x90) are byte-identical between builds. The differing bytes there are the ELF hash, which is downstream of the real cause.Fix
tools/platformio_reproducible_build.pypins__DATE__/__TIME__to the source commit, so the same commit always yields the same image while the banner still identifies the build:STACKCHAN_BUILD_STAMPoverrides the value;STACKCHAN_DISABLE_REPRODUCIBLE_BUILD=1restores wall-clock behaviour.Why it is attached the way it is
The script is added to every
extra_scriptsblock rather than a global[env]section. PlatformIO overridesextra_scriptsper environment instead of merging it, and the inheritance chain does not reachenv:stackchanfrom the release environments —stackchan_wake_mww_uplinkhas noextends, so the chain fromstackchan_camera_probestops there.I tried the
[env]form first and watched the script silently not run, which is how I found that out rather than assuming it.Verified across a minute boundary
That boundary is exactly the case that used to differ:
stackchan_camera_probestackchan_release_fullnative_logicFollow-up for whoever adds an environment
AGENTS.mdnow states the invariant and warns that a new build environment needs the pre-script or its images stop being reproducible. A contract test asserting every firmware env carries it would be a reasonable next step — I did not add one because the existing env-contract tests are PowerShell and cannot run here.🤖 Generated with Claude Code