Enforce process-backend quotas instead of silently dropping them - #286
Merged
Conversation
Supervisor.spawn accepts cpu_ms, wall_time_ms, open_files_max,
network_ops_max, output_bytes_max, child_work_max and numa_node, but
_spawn_process forwarded only mem_bytes. Every other quota was accepted
and discarded with no error and no warning, so
iso.spawn("x", backend="process", cpu_ms=50, wall_time_ms=100)
ran an unbounded loop to completion, while the same arguments on the
sub-interpreter backend raise WallTimeExceeded. The weaker backend enforced
limits that the one documented as the security boundary ignored.
Forward what this backend can actually enforce:
* cpu_ms -> RLIMIT_CPU, mem_bytes -> RLIMIT_AS, open_files_max ->
RLIMIT_NOFILE, all applied in the child before guest code runs and
recorded in the confinement report.
* wall_time_ms -> a supervisor-side timer, armed when an operation is
dispatched and disarmed by the completion frame. This cannot be an
rlimit: a guest blocked on I/O burns no CPU, so RLIMIT_CPU never fires.
Reject what it cannot. network_ops_max, output_bytes_max, child_work_max
and numa_node are in-process counters in the thread backend with no
equivalent across an address-space boundary, so they now raise
NotImplementedError naming the parameter rather than pretending to apply.
Two details that are easy to get wrong:
* RLIMIT_CPU has one-second granularity, so a sub-second cpu_ms rounds up.
Rounding up silently would recreate the bug in miniature, so the weaker
effective limit is logged.
* RLIMIT_NOFILE reserves headroom for stdio and the supervisor channel;
without it a small open_files_max stops the child reporting confinement.
* The wall-clock timer and the reader thread race to observe a dying guest.
Termination now surfaces exactly one error, the specific one, and the
kill completes before it is raised so a caller catching WallTimeExceeded
does not find the guest still running.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ebvMQ3vLxdK3joymz6Feg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Supervisor.spawnacceptscpu_ms,wall_time_ms,open_files_max,network_ops_max,output_bytes_max,child_work_max, andnuma_node._spawn_processforwarded onlymem_bytes. Everything else was accepted and discarded — no error, no warning. Measured onmain:The sub-interpreter backend — the one documented as not a security boundary — enforced limits that the process backend ignored. SECURITY.md §2 meanwhile claimed "Resource quotas (CPU/RAM/I/O) are enforced by
rlimitand cgroup v2 controls." A limit that looks configured and does nothing is the worst failure shape for this project.The change
Forward what this backend can enforce.
cpu_msRLIMIT_CPUin the child, before guest code runsmem_bytesRLIMIT_AS(already worked)open_files_maxRLIMIT_NOFILEwall_time_msWall time can't be an rlimit: a guest blocked on I/O burns no CPU, so
RLIMIT_CPUnever fires. The timer is armed when an operation is dispatched and disarmed by the child's completion frame, re-arming while operations are still pending.Reject what it cannot.
network_ops_max,output_bytes_max,child_work_max, andnuma_nodeare in-process counters in the thread backend with no equivalent across an address-space boundary. They now raiseNotImplementedErrornaming the parameter and pointing at the alternatives, rather than pretending to apply.Verified against the original repro:
Three details worth review
RLIMIT_CPUhas one-second granularity. A sub-secondcpu_msrounds up — silently applying a 20× weaker limit than requested would recreate this bug in miniature, so the effective value is logged at WARNING. Precise sub-second CPU needs cgroupcpu.max(not in this PR).RLIMIT_NOFILEreserves headroom (_NOFILE_CHANNEL_HEADROOM = 16) for stdio and the supervisor channel. Without it, a smallopen_files_maxstops the child from even reporting its confinement.WallTimeExceededinto the queue. Termination now surfaces exactly one error — the specific one — and the kill completes before it is raised, so a caller catchingWallTimeExceededdoesn't find the guest still burning CPU. That ordering is asserted in the test.Tests
12 new tests: wall-clock kills a runaway guest (and the guest is dead when the error lands), in-budget work is untouched and the timer is disarmed, the timer re-arms across sequential ops, each rlimit is observable from inside the guest, sub-second
cpu_msrounds up and warns, and each unenforceable quota is refused.Full suite 517 passed / 6 skipped (was 505). Wall-clock tests re-run 5× for flakiness — stable. flake8 clean.
SECURITY.md §2 now states the quota position precisely instead of the blanket claim.
Scope
Depends on nothing; environment scrubbing (#285) and the fail-open filesystem default are separate branches. Both touch
process_backend.pyin different regions.Generated by Claude Code