Repository navigation
Validate released compilers with realistic isolated scenarios - #9
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The previous compiler fixtures mostly exercised one-case contracts. Add twelve practical programs with 72 independent input/output/exit-code oracles, deliberate compiler/runtime/wrong-answer controls, exact unused/uncovered anchors and real authenticated compiler jobs. A hosted workflow tests the immutable published 0.4.2 source and current source separately, then verifies and exercises the actual published Linux frozen service across process restart and encrypted report replay.
Reports record exact runtime/validation source, worker image, compiler versions and evidence hashes. Guards refuse real execution outside the owned hosted Linux environment. Docker, the installed app and accounts on the owner's PC are unchanged. C# expected nonzero collector coverage remains explicitly partial; the original source/oracles and runtime implementations were not altered to hide that limitation.
Validation: final main
49f70acd752c45c7e7ee4cd45d32798a56c76dcdpassed all four workflows and eleven jobs, including 70 acceptance scenarios per source lane, 72 released-service cases and twelve restored reports, 345 Python tests plus 28 subtests, 85.01% measured coverage against the unchanged 80% floor, browser/original worker/fuzz/training checks, native packages for all three operating systems and CodeQL with zero open main alerts. All 59 focused local evidence/protocol regressions and Ruff passed. The stopped broad Windows unit attempt is not claimed as passed. Only small synthetic reports have three-day artifact retention; no credentials, real account data or executables are uploaded.Observed source-bound results and tool-specific limitations: validation report.