Skip to content

feat: every scan job runs under the per-host budget (nuclei, httpx, katana, path discovery) - #408

Merged
0xmanhnv merged 12 commits into
mainfrom
feat/host-budget
Nov 17, 2026
Merged

0xmanhnv merged 12 commits into
mainfrom
feat/host-budget

Conversation

@0xmanhnv

ghost commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

sdk-go #509, #510 and #511 are merged on main. go.mod pins sdk-go v0.51.1-0.20261117233817-f8315b68975c, the #511 merge commit on main (its tree is identical to the #511 branch head d2bdb4c this PR pinned before). The pin change is go.mod and go.sum only.

Why

A scan must not flood one target. #404 lowered the default rate, but one target host could still get far more than the level says:

  • a nuclei job over eight hosts runs at 150 requests per second, and with the tool sandbox on, the forwarder's per-host ceiling was that same 150: if seven hosts are slow or dead, the eighth gets it all;
  • any rate_limit a person set (150 in a scan's or the organization's settings) went to one host unchanged;
  • httpx over 500 URLs of one host ran at 50 targets per second against that host, and httpx and katana had no per-host bound at the forwarder;
  • on HTTPS nothing saw a 429 or 503, so nothing backed off.

The trust rule: the lowest limit wins

The per-host budget of a job (sdk-go #509) is, for each number, the lowest of the sensor's level (SENSOR_PACING_LEVEL, default Standard: 20 requests per second and 8 in flight per target host), the job's signed budget, the operator's ceiling (rate.max_host_rps, rate.max_host_inflight) and the built-in ceiling (200 and 50). A job can only lower it. To run faster on hosts you own, raise the level on the sensor. The result says which limit set each number (budget.won_by).

Which tools (code and docs now agree; a test keeps them so)

internal/toolrun/traffic.go lists every compiled-in tool by what it sends to its targets. TestEveryToolHasATrafficKind fails when a tool is missing or contradicts its manifest's network class, and the runner refuses to start a tool that sends requests to its targets without a budget (TestToolThatSendsToTargetsIsNotStartedWithoutABudget). The SDK's tool host now refuses the same for any tool whose manifest says network: targets (sdk-go #509), so tools the operator installs in an adapter directory are covered too: they get the scan's budget through the SDK's compat scanner.

tools sends to the targets paced by
nuclei, nuclei re-verification and retests (nuclei-validate), httpx, katana, webguess, webshot, svcdetect requests the per-host budget (bold: new in this revision)
a tool installed in an adapter directory with network: targets whatever it sends the per-host budget of the scan (the default one when the scan carries none); its rate key is lowered to it
naabu port probes not the request budget: packets per second of the budget's level (-rate 100, 300, 1000); a chosen rate replaces it under rate.max_rps. It is started with the budget and as a port scan (RunOptions.PortProbes); the runner refuses any other tool that asks for that
dnsx, subfinder nothing (DNS to the sensor's resolvers; passive sources) not applicable
asn, rdap (internal/lookup), trivy, semgrep, codeql nothing by design, but their manifests say network: egress-proxy (any public host, the targets included) the scan's budget (third revision): the runner and the SDK's tool host refuse them without one, and the forwarder holds every host they reach to it
opengrep, betterleaks nothing (a local checkout, no network) not applicable

What is enforced where

sandbox on (default) SENSOR_SANDBOX=off
per-host rate and connections the task forwarder, per host and per address, for every connection, every request on plain HTTP, and every request over HTTP/1.1 on HTTPS including kept-alive connections. Names of one address share its rate and connection slots and nothing else: a block page, a 429 or a Retry-After on one name slows the address and never pauses or refuses the other names there. A connection the slots refuse makes its host a coverage gap the tools' flags only: one host's budget for the whole run, whatever its target list
HTTP/2 on HTTPS counted for a client that writes each request in a TLS record of its own (the scan tools); not for a client that batches frames (a browser: webshot), nor streams in flight. There the flags are the enforcement flags only
back-off on what the tool's own traffic shows; on HTTPS the canary halves the rate none: nothing reads the answers
canary a fixed GET / (the sensor's own User-Agent, nothing of the job's identification), on the forwarder's dial path; it slows, it never pauses a host or stops a scan by itself; two rate-limit answers to it in a row are reported as a coverage gap throttled marked observed, which stops nothing none
result budget, won_by, clamped, back-offs, breaker openings with the evidence that opened them the same budget fields and pacing_limited: no_forwarder

Flags under the budget (Standard):

tool flags
nuclei -rate-limit 20 for each host worked at once (at most 8, never above the ceilings), -c 8, -bs hosts up to 25, -mhe 15, -timeout 10, -retries 1, -ss template-spray over several hosts
nuclei re-verification and retests -rate-limit 20, never above the budget's rate (5 at Gentle)
httpx -rl counts targets: a list of one host -rl 10 -threads 8 with the favicon probe on (two requests a target); several hosts their budgets in all, at most the sweep -rl 50 -threads 25
katana -rl 20 -c 8 for each host crawled at once
webguess rate per origin, at most 20
svcdetect rate, at most 20 for each host probed at once

A setting above the budget is no longer lowered silently (finding 9): a rate_limit, concurrency, threads or rate a person set (settings layers, the scanner config, the scan's own request) above the budget is recorded in metadata.target_rate.budget.clamped with what was asked and what applies, and the job's log has a line for it, on every path (tool host, tool child, in-process).

Second round of fixes (this revision)

what was wrong fix test
the sensor reported pacing.budget@1 also when it verifies no job signatures, and then ignored the signed budget advertiseHostBudget in main.go: the capability is reported only when a job-signing key or root is pinned (Kit.VerifiesJobs) TestHostBudgetCapabilityNeedsVerifiedJobs
retests ran under the sensor's own budget, whatever the retest command's signed budget said ValidatingCommandExecutor.retestBudget: the command's budget, which the kit's retest command passes on the context; never above the sensor's own TestRetestRunsUnderItsCommandsBudget (a signed Gentle retest: -rate-limit 5)
the SDK's tool host now refuses a network: targets tool without a budget naabu is started with the budget and PortProbes; the runner's guard checks both TestToolThatSendsToTargetsIsNotStartedWithoutABudget
a rate or concurrency of zero would have reached nuclei as "no limit" (not reachable through the settings, which are validated) Scanner.pacedLimits TestPacedLimitsNeverPassAZeroRate

Everything about shared addresses, connection slots, the observed throttle and adapter tools is in sdk-go #509.

Third round of fixes (this revision)

A further security review of sdk-go #509 found that a tool declaring network: egress-proxy skipped the budget guard. The SDK side (and the other findings: open connections bounded by the slots, a slot refusal reported as the sensor-side gap slots_full instead of rate_floor, the fallback budget, vendor hosts that are targets) is sdk-go #510. Here:

what was wrong fix test
the runner's guard (toolrun.requireBudget) let a tool of network egress-proxy start without a budget refused without a valid budget, whatever its traffic kind or whether it is listed TestEgressProxyToolIsNotStartedWithoutABudget (Run, RunRaw, RunTask; listed and unlisted)
trivy, semgrep, codeql, rdap and asn (all egress-proxy) were started without a budget, which the SDK now refuses started with the scan's budget (toolrun.Budget(opts)) the existing scanner tests run through the guard
a scan handed no budget ran at Standard even on a Gentle sensor toolrun.Budget falls back to adapt.FallbackBudget (the lowest of the sensor's level, Standard and the local policy's per-host ceiling) sdk-go #510

Sixth round (this revision)

sdk-go #511 changed the SDK only; this pull request re-pins go.mod to its head (ee3f722) and updates the docs:

  • a kept-alive connection is held for 30 seconds of quiet only on its first request (on HTTPS the host's last bytes may be its session tickets); once the host has answered on it and the tool asked again, it is closed for a waiting connection after 1 second of quiet, or 5 seconds when the tool offered HTTP/2;
  • only an HTTPS connection closed on its first request leaves a slots_full gap; keep-alive reuse leaves none, also for tools that offer HTTP/2 (webshot, Go-based tools);
  • a waiting connection whose tool gives up stops at once: it closes no other connection, opens none to the target and leaves no gap;
  • a target that sends a byte now and then keeps its slot until the tool or the task ends it; a connection to another name of its address waits at most 45 seconds and is refused with a visible slots_full gap;
  • the comment of toolrun.Budget now names the operator's per-host ceiling in the fallback.

Fifth round

A follow-up on sdk-go #510 changed the SDK only; this pull request re-pins go.mod to the new sdk-go #510 head (4f41626) and updates docs/REFERENCE.md:

  • a kept-alive connection whose answer the forwarder cannot be sure ended (its first request, or a client that offered HTTP/2) is closed for a waiting connection only after 30 seconds of quiet; the waiting connection waits meanwhile, and open connections still never exceed the slots;
  • only such a close leaves a slots_full gap, so ordinary keep-alive reuse (sure idle connections, closed at once) makes no gap and no partial run.

Fourth round

A fourth review of sdk-go #510 changed the SDK only; this pull request re-pins go.mod to the new sdk-go #510 head (e100eb8) and updates the docs:

  • the gap reason is now slots_full (the platform uses host_busy for a task waiting on a host another task holds);
  • a slots_full gap no longer makes the target unfinished: the target keeps its tool's outcome and the gap alone marks the host not covered, so the platform does not re-scan the host at once in a follow-up command;
  • a tunnel closed for a waiting connection while its first answer (or an HTTP/2 answer) may still have been arriving makes its host a slots_full gap;
  • the fallback budget also stays under the local policy's rate.max_host_rps and rate.max_host_inflight;
  • docs/REFERENCE.md states the cost for asn, rdap, trivy, semgrep and codeql: their own services (registries, rule and pack hosts, RDAP and ASN servers) get the per-host budget, so on Gentle two connections per host, and long parallel downloads may see slots_full; and that a vendor host that is only a DNS alias of a target is not matched as a target.

No sensor code referred to the old reason name, so nothing else changes here.

Measured with the real tools

internal/scanners/budget_real_test.go runs nuclei v3.11.1, httpx v1.12.0 and katana v1.8.0 with the command lines the sensor builds, against a server in the test process on 127.0.0.1 that speaks HTTPS, keeps connections alive, offers HTTP/2 and counts every request and connection. Build box, in a slot.

tool as built, one host, Standard (a person asked for 150/s) what the server counted
nuclei, 30 templates of 5 requests 150 requests in 7.0 s (21.3/s), most in any 1 s 25 to 32, in any 5 s 108 to 113, 8 in flight at most, 33 to 39 connections, 0 over HTTP/2
katana, a crawl of 150 linked pages 153 requests in 8.0 s (19.1/s), most in any 1 s 40, in any 5 s 120, 8 in flight, 9 connections (kept alive), 0 over HTTP/2
httpx, 150 URLs, favicon and JARM off 150 requests in 7.0 s (21.3/s), most in any 1 s 28, 8 in flight, 0 over HTTP/2
httpx, 30 URLs, default probes (favicon and JARM), no forwarder 60 requests and 480 connections in 29 s (16.5 connections/s), most in any 1 s 30 connections
the same with a signed Fast on a Standard sensor the same numbers: the budget stays Standard
httpx through the forwarder, three hosts, 150 of 160 URLs on one host (-rl 30 -threads 24) that host: 300 requests in 17.7 s (17.0/s), most in any 1 s 23, 2 in flight; the other two got their 10 each

The tools refill their rate once a second, so the stated tolerance for "any 1 s" is one refill (40); the forwarder's is a bucket of one token.

What this test found: httpx's -rl counts targets, not requests. With the default probes one target is two requests (the page and the favicon), and where httpx dials itself a JARM fingerprint is fourteen connections more. At -rl 20 one host received 42 requests a second (300 requests for 150 URLs in 7.2 s). The rate is now the budget divided by what a target costs (httpx.Scanner.requestsPerTarget).

Scan time

case before this PR now factor
One host, nothing chosen, sandbox on, a tool that opens a connection per request forwarder at 20/s forwarder at 20/s and 8 connections 1.02 measured (200 requests: 12.5 s to 12.7 s at 20 ms answers, 12.8 s to 13.0 s at 300 ms)
The same, a tool that keeps connections alive (katana, nuclei in part) the forwarder paced the connections only (a test client on 8 kept-alive connections reached 189 requests/s at a budget of 20); the tool's own -rl 20 was the only bound the forwarder holds 20/s too 1.00 for a tool that keeps to its flag
A host that takes 1 s per answer 20/s with unbounded connections 8 in flight, so 8/s 1.95 measured: the in-flight bound doing its job
httpx over many URLs of one host -rl 50 -threads 25 (100 requests/s with the favicon) -rl 10 -threads 8 (20 requests/s) 5 slower, by design
A chosen rate_limit of 150 on one host 150/s 20/s (recorded as clamped) 7.5 slower, by design; SENSOR_PACING_LEVEL=fast gives 100/s on hosts you own
SENSOR_SANDBOX=off, nuclei over 8 hosts 150/s in all 20/s in all up to 7.5 slower, by design
SENSOR_SANDBOX=off, httpx with JARM on, a list of one host 20 targets/s (320 connections/s to that host) 1 target/s (16 connections/s) 20 slower in that mode, by design; turn JARM off or use the sandbox
webshot, svcdetect, nuclei re-verification not under the budget under it a page with many sub-resources on one host loads at the budget's rate

Threat model

  • Job-supplied values cannot raise limits. Settings layers and the command's config only lower what a tool sends; a signed budget only lowers the sensor's level (TestBudgetBoundsChosenValues: an organization asking for 5000/200 runs at 20/8; a signed Fast, or a signed budget above every ceiling, on a Standard sensor runs 20/8).
  • No tool reaches a target unpaced because its adapter forgot the budget: the runner refuses it, and so does the SDK's tool host for every tool that can connect to its targets (targets, egress-proxy, and vendor with a target among its configured vendor hosts).
  • The platform is not told a signed budget is honoured when it is not: pacing.budget@1 is reported only by a sensor that verifies job signatures.
  • One name of a shared address cannot stop the scan of the others (sdk-go #509).
  • Nothing new is executed from job data: the budget is two numbers and a level name; the flags are built from them by the sensor.
  • No evasion: nothing rotates addresses or changes the identification after a block.
  • Contract versions are unchanged (v1). tool.yaml files are unchanged.

Residual, honestly

  • Requests a browser packs into one TLS record over HTTP/2 (webshot) are bounded in bytes and connections, not counted: a page load is not held to the request rate as exactly as the scan tools are.
  • naabu is paced in packets, not by the request budget, and a chosen rate replaces the level's.
  • SENSOR_SANDBOX=off: flags only, no reaction to what the targets answer.
  • A sensor with no pinned job-signing key runs every job at its own level and does not report pacing.budget@1.
  • Unconfined webguess probes its origins one after the other at the budget's rate, so one host with several ports still gets one budget at a time; no change was needed there.
  • The real-tool tests need the tools' binaries (NUCLEI_REAL_BIN, HTTPX_REAL_BIN, KATANA_REAL_BIN); CI skips them, the numbers above are from the build box.
  • Redirects httpx follows on the same host are extra requests the rate flag does not count (the forwarder does, with the sandbox on).
  • The vendor traffic of the egress-proxy tools is held to the target budget (see Fourth round); a vendor host that is only a DNS alias of a target is not matched as one.

Tests

  • internal/toolrun/traffic_test.go, tool_traffic_test.go: the tool registry and the start guard (requests, port probes, nothing).
  • host_budget_capability_test.go: the capability with and without verified jobs.
  • internal/executor/retest_test.go: the retest's budget.
  • internal/scanners/nuclei/pacing_test.go: the command line by budget, host count and confinement; chosen values bounded and recorded; the re-verification rate under the budget.
  • internal/recon/httpx/budget_args_test.go, katana/, naabu/: flags by budget; the cost of a target; clamps.
  • internal/recon/politeness_test.go, internal/scanners/recon_limits_test.go: the budget over one and many hosts, with and without a forwarder.
  • internal/scanners/budget_real_test.go: the measurements above.

Checks

Final revision, against sdk-go main f8315b6 (#511 merged), build box: go mod verify exit 0; go build ./... exit 0; go vet ./... exit 0; golangci-lint run ./... (v2.14.0) 0 issues; go test -count=1 ./... (the whole suite) exit 0.

Sixth revision, against sdk-go #511 head ee3f722 (see the pin), build box: go build ./... exit 0; go vet ./... exit 0; golangci-lint run ./... (v2.14.0) 0 issues; go test -count=1 ./... (the whole suite) exit 0; gofmt -l lists only the four files named below that this pull request does not touch.

Fifth revision, against sdk-go #510 head 4f41626 (see the pin), build box: go mod tidy exit 0; go build ./... exit 0; go vet ./... exit 0; golangci-lint run ./... (v2.14.0) exit 0; go test -count=1 ./... (the whole suite) exit 0.

Fourth revision, against sdk-go #510 head e100eb8 (see the pin): go mod tidy exit 0; go build ./... exit 0; go vet ./... exit 0; golangci-lint run ./... (v2.14.0) 0 issues; go test -race -count=1 on the root package, toolrun, lookup, scanners, recon and executor exit 0; go test -count=1 ./... (the whole suite) exit 0. gofmt -l lists four files this pull request does not touch (tool_detect.go, policy_cmd_test.go, tools_cmd_betterleaks_codeql_test.go, internal/scanners/nuclei/validate_real_test.go).

Third revision, against sdk-go #510 head (see the pin): go mod tidy exit 0; go build ./... exit 0; go vet ./... exit 0; go test -race -count=1 on the root package, toolrun, recon, scanners, executor, webguess, webshot, svcdetect and lookup exit 1 on the box under load: two job-file privacy tests (TestListsArePrivateToTheirJob in recon/dnsx, TestPathReportIsPrivateToItsJob in scanners/internal/report) failed once; both packages then passed three times in a row alone, at this pin and at the previous one, and neither touches this change; go test -count=1 ./... (the whole suite) exit 0; golangci-lint run ./... (v2.14.0) exit 0, 0 issues.

Second revision, against sdk-go aa27239: go mod verify exit 0; go build ./... exit 0; go build -tags platform ./... exit 0; go vet ./... exit 0; go test -race -count=1 on the root package, toolrun, recon, scanners, executor, webguess, webshot and svcdetect exit 0; go test -count=1 ./... (the whole suite) exit 0; golangci-lint run ./... (v2.14.0) exit 0, 0 issues; the vocabulary check exit 0. The confined nuclei tests (RealNucleiKeepsToWebScope, RealNucleiThroughThePool, nuclei v3.11.1, in the container image CI uses): 2 passed. The real-tool measurements above were not run again for this revision; the flags the sensor builds are unchanged. deps-on-main is red while go.mod pins the sdk-go branch: that is the draft state described at the top.

Docs

docs/REFERENCE.md ("Per-host budget": the tool list with adapter tools, what is enforced where, names that share an address, the capability and signed jobs, the canary's observed throttle), docs/LOCAL_POLICY.md, docs/RUN.md, docs/sensor-policy.example.yaml, changelog fragment.

Public docs (docs sensors/tool-settings.md, "How hard a scan works one host") need a follow-up for this revision; no docs pull request is opened here. The sentences to change:

  1. "Names that resolve to one address share that address's budget." becomes: "Names that resolve to one address share that address's rate and connections. A block page, a 429 or a Retry-After on one of them slows that address for a while and never pauses or refuses the other names there."
  2. "every new connection, and the number of connections open to a host;" stays, and gains: "A connection a tool keeps open but does not use is closed when another connection to that host needs its place. A connection that finds no free place for 15 seconds is refused, and the run reports its host as a coverage gap slots_full: the sensor's own limit, not the host; the scan of that host is not repeated at once, and the host's findings are not held for it. A connection that is closed for another one while its first answer may still be arriving also makes its host such a gap;"
  3. In "Which tools", the first row gains: "and any tool you install in an adapter directory that connects to its targets", and the sentence after the table gains: "Re-verifications and retests run under their own job's budget." The third row changes: "DNS (dnsx), passive discovery (subfinder), code scans (opengrep, betterleaks) | They send nothing to the target hosts", and a new row: "Lookups (asn, rdap), code scans that fetch their own data (trivy, semgrep, codeql), and any tool whose network class is egress-proxy | The per-host budget: such a tool may reach any public host, so every host it reaches is held to the budget".
  4. "On its own it does nothing more: it never pauses a host and never stops a scan, so a target cannot end its own scan by answering that one request differently." becomes: "On its own it never pauses a host and never stops a scan, so a target cannot end its own scan by answering that one request differently. When it is answered with a rate limit twice in a row, the run reports the host as a coverage gap throttled (marked as observed) and carries on at the lower rate."
  5. In "The lowest limit wins", after item 2: "A sensor reads the signed budget only when it verifies job signatures (a job-signing key is pinned); a sensor that does not runs at its own level, and the platform sends it no budget."
  6. In "Which tools", the row for lookups and code scans that fetch their own data gains: "Their own services (advisory databases, image registries, rule and pack hosts, RDAP and ASN servers) are held to the same budget: on Gentle two connections per host, so long parallel downloads can make the run report slots_full."

…s what a tool sends to one host

nuclei, httpx, katana and path discovery derive their flags from the
budget of the job (Standard: 20 requests per second and 8 in flight per
target host) and hand it to the tool host, whose forwarder holds each
host to it. A rate or concurrency a person chose above the budget is
lowered to it. Without a forwarder a tool whose rate flag is global gets
one host budget for the whole run.

SENSOR_PACING_LEVEL sets the level of jobs without a signed budget; the
local policy rate.max_host_rps and rate.max_host_inflight are the ceiling
no job passes. The sensor reports pacing.budget@1.

go.mod pins the head of sdk-go feat/host-budget (sdk-go #509).

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
@github-actions github-actions Bot added the security-sensitive Changes a security-sensitive path (.github/security-paths.yml); set by the security gate label Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Security gate

PASS: security-sensitive, acknowledged, no open HIGH.

Why this is security-sensitive

  • tool-content-trust: internal/recon/tool.go

Diff review

No high-risk pattern on the added lines.

Code scanning

No new high or critical alert.

Security checks on the head commit

  • CodeQL: success
  • CodeQL Analysis: success
  • Secret Scanning: success
  • Smoke semgrep: success
  • Smoke trivy: success
  • Snyk Security Scan: skipped
  • Trivy: neutral
  • Trivy Vulnerability Scanner: success

Acknowledge

A maintainer reviews the change and adds the security-ack label. The gate re-runs on the label. Status: security-ack added by 0xmanhnv.

Path list: .github/security-paths.yml · how the gate works

@0xmanhnv
0xmanhnv marked this pull request as ready for review November 17, 2026 09:34
…get, a setting above it is recorded, httpx counts what a target costs

Security review of the per-host budget:

- internal/toolrun/traffic.go lists every compiled-in tool by what it
  sends to its targets. The runner refuses to start a tool that sends
  requests to its targets without a budget, and a test fails when a tool
  is missing from the list or contradicts its manifest.
- Under the budget now: nuclei re-verification and retests, webshot and
  svcdetect, besides nuclei, httpx, katana and webguess. The port scan is
  paced in packets per second by the level of the budget; DNS, passive
  lookups and code scans send nothing to the targets.
- A rate or concurrency a person set above the budget is recorded with
  what was asked and what applies (target_rate.budget.clamped) and logged,
  on every path: the tool host, the tool child and the in-process run.
- httpx -rl counts targets, and a target costs its host two requests with
  the favicon probe on, and fourteen connections more with JARM where
  httpx dials itself: the rate is the budget divided by that cost.
  Measured with the real binary: at -rl 20 one host received 42 requests
  a second.
- Without a forwarder a run gets one budget of one host, whatever its
  target list, also for an httpx sweep.
- A job can only lower the budget (sdk-go): the tests and the docs say a
  signed Fast on a Standard sensor stays Standard.
- internal/scanners/budget_real_test.go runs nuclei, httpx and katana as
  built against a local HTTPS server with keep-alive and HTTP/2 offered
  and asserts what the server counts.
- docs/REFERENCE.md: the tool list, what is enforced where and what is
  not, the canary.

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
For CI while this pull request is a draft. Before it leaves draft it is
re-pinned to the merged commit on sdk-go main.

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
@0xmanhnv
0xmanhnv marked this pull request as draft November 17, 2026 13:11
@0xmanhnv
0xmanhnv marked this pull request as ready for review November 17, 2026 13:29
…ir command's budget, the port scan is started with the budget

The sensor reported pacing.budget@1 whenever it ran commands, also with no
job-signing key pinned: it then ran every job at its own level while the
platform believed a signed budget was read. The capability is now reported
only when the sensor verifies job signatures.

A retest ran under the sensor's own budget whatever its command's signed
budget said. It now runs under its command's budget, as a re-verification
does; the lowest limit still wins.

The tool host refuses to start a tool that connects to its targets without
a budget, so the port scan is started with the scan's budget and as a port
scan (it stays paced in packets per second). The runner refuses any other
tool that asks for that pacing.

A rate or concurrency of zero never reaches nuclei as a flag.

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
@0xmanhnv
0xmanhnv marked this pull request as draft November 17, 2026 16:20
@0xmanhnv
0xmanhnv marked this pull request as ready for review November 17, 2026 16:47
…-go to the head of its review-fix branch (0a69d86)

The runner refuses a tool of network egress-proxy (any public host, its
targets included) without a per-host budget, and trivy, semgrep, codeql,
rdap and asn are started with the scan's budget. A scan handed no budget
runs under adapt.FallbackBudget. REFERENCE and the changelog entry say
which tools are held and what the host_busy gap means.

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
@0xmanhnv
0xmanhnv marked this pull request as draft November 17, 2026 19:21
- go.mod pins the sdk-go review branch head e100eb8 (still a branch
  commit: this pull request stays a draft until sdk-go #510 merges).
- docs/REFERENCE.md and the changelog fragment: the sensor-side gap reason
  is now slots_full; the target keeps its tool's outcome, so no follow-up
  scans the host again at once; a tunnel closed while its answer may still
  have been arriving also leaves the gap; the vendor traffic of the
  egress-proxy tools (asn, rdap, trivy, semgrep, codeql) gets the per-host
  budget (two connections per host on Gentle); a vendor host that is only
  a DNS alias of a target is not matched as one.

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
The SDK no longer closes a connection whose answer it cannot be sure
ended (its first request, or HTTP/2) for a waiting connection until it
has been quiet for 30 seconds; only such a close leaves a slots_full gap,
so ordinary keep-alive reuse leaves none. docs/REFERENCE.md says so.

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
…nnection is a gap

- go.mod pins sdk-go ee3f722 (the follow-up pull request's head) until it merges.
- docs/REFERENCE.md: a connection is held 30 s only on its first request, 1 s (5 s when
  the tool offered HTTP/2) once the host has answered on it; only an HTTPS connection
  closed on its first request is a slots_full gap; a waiting connection whose tool gave
  up stops at once; a target that trickles bytes keeps its slot (bounded, visible gap).
- toolrun.Budget: the fallback is also bounded by the operator's per-host ceiling.

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
…wording

An HTTP/2-offering connection is closed for a waiting one after the same
30 seconds of quiet as an unsure one, not after 5 seconds.

Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
Signed-off-by: Nguyen Manh <18032641+0xmanhnv@users.noreply.github.com>
@0xmanhnv
0xmanhnv marked this pull request as ready for review November 17, 2026 23:41
@0xmanhnv 0xmanhnv added the security-ack Maintainer reviewed the security-sensitive change (maintainer only; never automation) label Nov 17, 2026
@0xmanhnv
0xmanhnv merged commit 6198417 into main Nov 17, 2026
36 of 40 checks passed
@0xmanhnv
0xmanhnv deleted the feat/host-budget branch November 17, 2026 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

security-ack Maintainer reviewed the security-sensitive change (maintainer only; never automation) security-sensitive Changes a security-sensitive path (.github/security-paths.yml); set by the security gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant