Repository navigation
Bound publication memory and recover saved forecast publishing - #47
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The October 28 forecasts completed and passed integrity validation, but dashboard publication was killed while the CSV exporter retained the entire historical instrument log. This change streams export rows into a disk-backed stable sort, preserving CSV bytes and ordering, and retains only the selected day/protocol for policy and capability views. Validation hashes logs in chunks.
Publication builds each component in a fresh process with a 2048 MiB address-space limit and logs elapsed time and peak RSS.
airo statusnow shows the latest receipts by start time so forecast failures are visible. The deployment guide documents recovery from saved forecasts without rerunning paid calls.Validation: byte-for-byte CSV sorting regression coverage; a generated 160 MiB export under a 96 MiB memory limit; existing publication failure, integrity, panel selection, and golden-output tests. All four required CI checks passed on bb169fe: tests on Python 3.11, 3.12, and 3.13, plus reproduction. Local publication/export/golden, conditional/instrument/dispatch, cache, and memory regression tests passed. Production-data profiling reduced policy/capability view peak RSS from about 1.8 GiB to 462 MiB; the original CSV exporter exceeded a 2.5 GiB limit while loading history. The fixed production-sized export completed at 33 MiB peak RSS (870,294 forecasts, 156,277 rationales, 1,004 debriefs), including the download bundle, under a 1 GiB address-space limit.
After merge, deploy with
airo deploy --publish-datato validate and publish the already-saved forecasts. No new forecasting calls are required.