Measure the privileged kernel surface, fix what it found, align the repo - #2
Merged
Conversation
The rename to marstack-container left this repo without what the other three carry: a description in the house form, topics, a homepage, and a usage guide on GitHub Pages. The theme is vendored rather than copied by a build step. marstack-cloud, -secrets and -access all run `make site` to lift meridian.css out of internal/console/assets, but this runtime is a CLI with no console and no Makefile, so site/meridian.css and site/site.css are committed byte-identical to the copies those repos ship. A workflow step diffs them against marstack-access on every run, because vendoring trades a build-time guarantee for drift that nothing would otherwise catch. pages.yml keeps the three guards the other repos use — every referenced asset present, every anchor pointing at a section that exists, every theme token the stylesheet uses actually defined — since a broken link on a page nobody rebuilds is invisible until someone clicks it.
The integration suite has never passed in CI. 96 of 118 assertions failed with the same line: ./tests/run-integration.sh: line 84: target/debug/mars: No such file run_in() enters the bundle directory before invoking the runtime: (cd "$dir" && "$MARS" run "$@") CI passes MARS=target/debug/mars, and a relative path cannot survive that cd. The auto-detection above it has the same defect, since every candidate it tries is relative too, so a local run only worked when the caller happened to pass an absolute path. Resolve MARS to an absolute path once, and refuse up front if it is not executable rather than reporting the same missing-file error ninety-six times. This is what has been hiding the seccomp fix from CI: the suite covers seccomp errno mapping and no_new_privs, and none of those assertions could run.
With the runtime path fixed the suite finally ran in CI, and hung for nineteen minutes on "memory.max and OOM kill" until the job was cancelled. The test sets memory.limit but not memory.swap, so memory.swap.max keeps its default of max. The awk loop writes each page once and never touches it again, so the kernel satisfies memory.max by pushing those cold pages to swap instead of killing anything. Caught in the act on a host with swap: memory.current 33296384 memory.max 33554432 memory.swap.max max memory.events oom 0 oom_kill 0 SwapFree fell by ~353MB and kept falling The awk process had been running for over ten minutes at that point. GitHub runners have swap; the Lima dev VM has none, which is why this only ever hung in CI. Setting memory.swap equal to memory.limit makes mars write memory.swap.max=0, so the cgroup has nowhere to spill and the kill is deterministic. It also means the second assertion — that the runtime names memory.max as the cause from memory.events — is exercised for the first time. A first attempt to reproduce this with a C malloc loop was misleading: it rewrites every page, so the pages stay hot, and it OOMs before swap is ever reached. The allocation pattern is the whole mechanism. timeout-minutes on the integration job so the next hang costs twenty minutes rather than the six-hour default. Full suite on a host with swap: 128 passed, 0 failed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three things, in the order they happened.
Measure the creation path
scripts/hap-bench.shcounts the distinct host kernel functions a runtime traverses while it holdsroot — the window where every privileged operation happens, and the one axis published runtime
comparisons do not cover.
docs/attack-surface.mdcarries the method and the results.runvolexecttymarscrun1.14.1runc1.5.1Not because
marsskips work: namespaces,pivot_root, cgroup setup and capabilities all appear atparity. Of the 1000 functions
runcreaches andmarsdoes not, 10 are thread or schedulerfunctions and 301 are file and
/proctraversal.Three methodology faults each produced a plausible wrong answer before the assertions caught them —
per-PID trace filters lose
crun's child, global tracing charges a runtime for its neighbours, and aruntime that exits non-zero still yields a number. The script asserts rather than trusts.
Fix what it found
Widening the workload to cover seccomp exposed a blocker:
marsrefused the profile Podman andCRI-O ship, needing 59 syscall names removed first where
runcandcrunremoved none.The failing names all look absent on aarch64, so the first reading was an unresolvable-syscall
problem. It was not.
libseccompreturnsEACCESfor a rule whose action equals the filter'sdefault action, and these profiles deny by default and then spell out denials — every such rule is
redundant by construction.
marsnow toleratesEACCESonly when the rule's verdict equals thedefault, which cannot change what the filter permits. A deny rule under an allow-by-default filter
differs from the default and never takes that path; a test asserts it.
Align the repo after the rename
Description, topics, homepage and a Pages usage guide, matching
marstack-cloud,-secretsand-access. The theme is vendored rather than lifted bymake site, because this runtime is a CLIwith no console and no Makefile — so a workflow step diffs both stylesheets against
marstack-accessto catch the drift that vendoring invites.Still missing next to the other three:
CHANGELOG.md,CONTRIBUTING.md,SECURITY.md,CODE_OF_CONDUCT.md,CODEOWNERS,Makefile,release.yml.Verified
cargo test --lib106 passed,cargo clippy --all-targetswith-D warningsclean,cargo fmt --checkclean,shellcheck -S warningclean, andmarsaccepting the unmodified 442-name profile.All on aarch64 — this PR is the first time CI sees any of it.