Wasi build - #79
Draft
fredbi wants to merge 4 commits into
Draft
Conversation
fredbi
force-pushed
the
wasi-build
branch
2 times, most recently
from
August 3, 2026 09:27
7c9b8bb to
88b7156
Compare
packages.Load resolves a package graph by running `go list`. That binds a
scan to an installed toolchain, and rules out any environment without one —
a WebAssembly guest, where there is no process model at all, being the case
that forced the issue.
internal/packages does the same job in pure Go: patterns to directories,
build-constraint selection through go/build, parse, type-check. Its entry
point mirrors the original and its vocabulary is aliased to it, so a caller
moves between the two by changing the call and nothing else. Across the
fixture corpus the two agree byte for byte.
Three things a caller could not previously say, each of which changes the
emitted spec rather than merely how it is produced:
- GOOS/GOARCH, because the platform a scan is built for was silently the
platform codescan was running on. Inside a guest that is wasip1, which
drops every _linux.go file without a word.
- FS, a filesystem to read the sources from, which is what lets a scan
cover a tree that was never written to disk. It also selects the loader:
`go list` can only ever see the real filesystem, so asking for a virtual
one already says which loader has to run.
- StubStdlib, which synthesizes the standard library from the names the
scanned code selects through it rather than reading GOROOT. Far smaller
and faster, and not failsafe: a synthesized type has no fields and no
method set, so json.RawMessage stops rendering as a byte array and a
type is no longer seen to implement encoding.TextMarshaler.
Synthesis is not silent. Every import fabricated this way raises a
scan.synthesized-import diagnostic on the import that caused it — a hint
when it was asked for, a warning when the import merely could not be found,
which is usually a mounting mistake. Without it the loss surfaces only as
the wreckage of a value-position use, reading as an error in the scanned
code rather than as a dependency that was never there.
Function bodies are type-checked despite codescan reading only
declarations: an annotated type may be declared inside one, and skipping
bodies leaves no entry for it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
genspec writes a specification to standard output and depends on nothing beyond the library, so it cross-compiles to wasip1/wasm and runs under any WASI runtime with no toolchain present and no subprocess. -loader picks how packages are resolved, defaulting to whichever is possible: on wasm the go command can never run, so the choice makes itself. Every boolean scan option has a flag, named after its field, and a test fails when one arrives without either a flag or a recorded reason not to have one. Coverage is decided by which field an entry writes to rather than by deriving a name from the field, since no mechanical rendering knows that JSONify is one word and HTTPServer is two. Reading dependencies from the export data the compiler already produced, instead of parsing and type-checking them, is where nearly all of a scan's time goes: on the petstore under wasmtime, 7.3 s and 681 MB becomes 1.0 s and 138 MB, with GOROOT never mounted. It costs no fidelity, the types being the compiler's own — and unlike withholding the standard library entirely it keeps method sets and interface identity. genexportdata produces the data, natively, because generating it needs the toolchain the consumer does not have; a module of its own states what a published bundle covers. The exportdata build tag carries the result inside the binary, so an artifact needs nothing mounted but the sources being scanned. A package whose meaning lives in comments cannot go in. strfmt declares its formats with swagger:strfmt, and export data holds types, not comments, so it would come back structurally intact and semantically empty with nothing erroring. Those are detected, skipped and named at generation time. The go/packages loader is excluded from WebAssembly builds by tag rather than guarded at runtime, so the go command plumbing is never linked. What the artifact asks of a host drops to twenty-one WASI functions, every one of which reads, and a test pins that: it also rejects a second import module, and any import that is not a function, since an imported memory would mean SharedArrayBuffer and the headers that rule out static hosting. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
A Svelte front-end around the wasip1 artifact: pick a module, watch the specification it produces. No server, no toolchain, and the files never leave the browser. hack/browser is the conformance probe that established this is possible at all, kept because the properties it checks are easy to lose. Under @bjorn3/browser_wasi_shim a 200-package tree yields 6000 definitions in 3.5 s; compiling the module costs 22 ms once and instantiating 4-5 ms per run, flat in the size of the workload. poll_oneoff was the risk worth checking, the shim implementing it by spinning rather than yielding — it never fires, Go reaching it only through the netpoller and only once the scheduler runs out of work. The scan runs in a worker, and each one gets a fresh instance: the artifact is a WASI command that ends at proc_exit, and Go's wasip1 target emits no reactor form. Compiling is the expensive half and is kept between runs. Opening a module replaces the tree rather than merging into it — otherwise the previous one stays behind and the next scan sees two modules at once — and re-roots on the outermost go.mod, since a directory pick may sit above the module or below it. Test files are skipped and vendor is kept, being the only way a third-party import resolves with no module cache to read. The standard library is synthesized rather than read, which is fast and needs no asset but loses structural detail. Pointing it at published export data instead is one argument, once it is settled which packages that data covers. The front-end is a prototype: no highlighting, no cross-references, no chrome. It lags the terminal UI, which has had a great deal more polish. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change type
Please select: 🆕 New feature or enhancement|🔧 Bug fix'|📃 Documentation update
Short description
Fixes
Full description
Checklist