Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions .github/audit/_preamble.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,29 @@
# Shared preamble — every audit subagent

Read `SECURITY.md` first. Your scope is exactly the sections named in your own
file — ignore every other section, including its `FAIL IF` lines; another agent
owns them.
Read `docs/specs/security.md` first: it states the guarantees, what is not
defended, and the known gaps, and names the spec each domain audits. Your scope
is exactly the spec files listed under **Scope** in your own file — ignore every
other spec's `FAIL IF` lines; another agent owns them. `docs/specs/security-audit.md`
is the contract this run executes.

For each `FAIL IF` in your scope, run the mechanical check (`gh api`, grep,
file read, or a script) and record PASS or FAIL with concrete evidence: file
path and line number, API response excerpt, or command output. A `FAIL IF`
bullet may assert several properties in one sentence; **each clause gets its
own verdict and its own evidence**. Never satisfy a bullet in bulk.
own verdict and its own evidence**. Never satisfy a bullet in bulk. A `FAIL IF`
that ends `(rationale)` has its evidence in the paired `<spec>.rationale.md`
under the same heading; read it when the rule alone is not enough to judge.

Then do the qualitative pass described for your domain, rating findings
BLOCKER / WARNING / INFO. Report what you can prove. Use `UNVERIFIABLE` only
for a check you could not determine — a transient network error, or an area you
ran out of room to reach — and say which it was. It is never a substitute for a
check you could have run.

Where `docs/specs/security.md` says a risk is accepted ("What is not defended")
or a gap is known ("Known gaps"), do not re-report it as a finding — report
only if the situation has changed or is worse than described.

Write your findings to the file named in your own prompt. **Its very first
line must be literally `VERDICT: PASS` or `VERDICT: FAIL`** — nothing else on
that line. The reporting step greps for it, so it is the one part of your
Expand Down
29 changes: 17 additions & 12 deletions .github/audit/application-security.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
# Domain: application-security

**Scope — these sections, and no others:**
**Scope — these specs, and no others:**

`## Remote Control`
`## Loopback Listeners`
- `docs/specs/security-local.md`
- `docs/specs/security-remote.md`

**Output file:** `audit-application.md`

This is a code-and-specs audit of the product's own remote control stack. You
need no GitHub API access and no PAT — do not use one.
This is a code-and-specs audit of the product's own boundaries — the remote
control stack, and the local application. You need no GitHub API access and no
PAT — do not use one.

Read, at minimum: `docs/specs/remote-security-model.md` **and its paired
`docs/specs/remote-security-model.rationale.md`**, `docs/specs/server.md`,
Expand Down Expand Up @@ -42,6 +43,14 @@ states the rule — then each listener it names. Derive the set of listeners by
searching the shipped trees yourself; the section's own list is a description of
today's tree, not the scope.

For the rest of `docs/specs/security-local.md`, read each section's owner first
— `docs/specs/terminal-escapes.md`, `docs/specs/dor-browser.md`,
`docs/specs/dor-cli.md`, `docs/specs/vscode.md` -> "Webview message
authentication", `docs/specs/standalone.md` -> "Persistence" — then the parser,
the iframe shim, the control-socket code, and the persistence path they point
at. The attacker there is a program printing to the terminal, a page in a
browser pane, or another local account, never the network.

## Qualitative pass

Be adversarial, and go past the `FAIL IF` list. Ask specifically:
Expand Down Expand Up @@ -103,13 +112,9 @@ Be adversarial, and go past the `FAIL IF` list. Ask specifically:
and `server.md`'s Relay and E2E framing. `scripts/e2e-lint.mjs` mechanizes the
structural half of that ("one suite, no negotiation, no plaintext path, no
legacy discriminant") — check that each of its rules still names a real
`SECURITY.md` line and that `scripts/e2e-lint-selftest.mjs` still proves every
rule load-bearing, then look for what a *textual* lint cannot see.

Where the section says a risk is accepted (the setup password's hardening) or a
gap is known (revocation, the audit trail, the two `workflow-audit` window
evasions), do not re-report it as a finding — report only if the situation has
changed or is worse than described.
`docs/specs/security-remote.md` line and that
`scripts/e2e-lint-selftest.mjs` still proves every rule load-bearing, then
look for what a *textual* lint cannot see.

You are also the **catch-all** domain, and this is defined by subtraction, not
by a list: you own everything in the repository that `supply-chain.md` and
Expand Down
13 changes: 5 additions & 8 deletions .github/audit/ci-and-secrets.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
# Domain: ci-and-secrets

**Scope — these sections, and no others:**
**Scope — these specs, and no others:**

`## GitHub Actions Policies`
`## Automated Maintainer (tend)`
`## VS Code Extension Releases`
`## Desktop Releases`
`## Reporting a Vulnerability`
`## CI Validation Contract`
- `docs/specs/security.md`
- `docs/specs/security-ci.md`
- `docs/specs/security-audit.md`

**Output file:** `audit-ci-secrets.md`

Expand All @@ -24,7 +21,7 @@ GH_TOKEN=$AUDIT_PAT gh api repos/$GITHUB_REPOSITORY/rulesets/16757376
`$AUDIT_PAT` is a fine-grained, read-only PAT covering Administration +
Secrets + Environments, guaranteed present by an earlier step. If a prefixed
call still returns 403, record FAIL with the note "PAT scope drifted from
SECURITY.md".
docs/specs/security-audit.md".

**Check effective permissions, not declared ones.** A job with no
`permissions:` block inherits the repository default, so read
Expand Down
6 changes: 3 additions & 3 deletions .github/audit/orchestrator.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Security audit — orchestrator

You are the orchestrator of this repository's nightly security audit.
`SECURITY.md` is the document you audit against: its `FAIL IF` lines are
concrete mechanical checks, and it also says that list is not exhaustive, so
each domain gets a qualitative pass too.
The security specs (`docs/specs/security*.md`) are what you audit against:
their `FAIL IF` lines are concrete mechanical checks, and `docs/specs/security-audit.md`
says that list is not exhaustive, so each domain gets a qualitative pass too.

**Audit nothing yourself.** Fan the work out to three subagents with disjoint
scopes, then merge what they return. The domains are genuinely different
Expand Down
4 changes: 2 additions & 2 deletions .github/audit/supply-chain.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Domain: supply-chain

**Scope — these sections, and no others:**
**Scope — these specs, and no others:**

`## Dependency Supply Chain`
- `docs/specs/security-supply-chain.md`

**Output file:** `audit-supply-chain.md`

Expand Down
2 changes: 1 addition & 1 deletion .github/renovate.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"Security fixes must not wait for the Monday window. Everything else here is batched to one day a week, which is right for routine bumps and wrong for a known-vulnerable dependency, so this overrides the schedule only.",
"`minimumReleaseAge` is set here EXPLICITLY, and must stay that way. Renovate\u0027s own default for this block is `minimumReleaseAge: null`, force-applied before lookup — so omitting it does not inherit the cooldown from packageRules, it DROPS the cooldown entirely. Stating it is the only way to keep it.",
"Keeping it is the deliberate choice: the cooldown guards the opposite threat — a compromised release that gets yanked within a day — and a reviewer reading a Renovate diff cannot detect a supply-chain compromise the way the ecosystem\u0027s own yank process can. Nothing here auto-merges, and the Dependabot alert already makes the vulnerability visible the moment it is published, so what the cooldown costs is a day before the remediation PR appears, not a day before anyone knows.",
"See SECURITY.md, Dependency Supply Chain."
"See docs/specs/security-supply-chain.md, Cooldown and alerts."
],
"enabled": true,
"schedule": [],
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# The disclosed dependency snapshots are a SECURITY.md `FAIL IF`
# (Dependency Supply Chain), but until this step existed the only thing
# The disclosed dependency snapshots are a `FAIL IF` in docs/specs/security-supply-chain.md
# (Disclosure), but until this step existed the only thing
# that ever ran the generator was the nightly security audit — strictly
# after the fact, and only if it reached that check. Two prod-dependency
# bumps shipped undisclosed that way (`ws` via vscode-ext, `hono` via a
Expand All @@ -48,7 +48,7 @@ jobs:
# zsh is not on the ubuntu runner image, and it is the default shell on
# macOS — our primary platform. Without it, `standalone/sidecar`'s
# shell-integration suite silently covered only bash, which is half of
# what stands behind an emit-side security fix (SECURITY.md's OSC 633
# what stands behind an emit-side security fix (docs/specs/terminal-escapes.md's OSC 633
# rules; the emitters are the boundary, since the parser cannot defend
# against a terminator that arrives inside a directory name). The suite
# names the shells it covered on every run, so a future image change that
Expand Down
29 changes: 15 additions & 14 deletions .github/workflows/security-audit.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: security-audit

# Audits this repo against SECURITY.md. Runs nightly via the schedule
# Audits this repo against the security specs (docs/specs/security*.md). Runs nightly via the schedule
# trigger and on-demand via workflow_dispatch. release.yml dispatches it
# on the release tag and gates publishing on the result — it dispatches
# rather than calling this workflow with `uses:` because a tag `push`
Expand Down Expand Up @@ -72,11 +72,11 @@ jobs:
run: |
[ -n "$AUDIT_PAT" ] && exit 0
echo "FAIL" > audit-status.txt
echo "**FAIL** — \`AUDIT_PAT\` is not present in the \`security-audit\` environment. See [SECURITY.md > CI Validation Contract](https://github.com/$GITHUB_REPOSITORY/blob/main/SECURITY.md#ci-validation-contract) for provisioning." > audit-report.md
echo "**FAIL** — \`AUDIT_PAT\` is not present in the \`security-audit\` environment. See [security-audit.md > Environment and AUDIT_PAT](https://github.com/$GITHUB_REPOSITORY/blob/main/docs/specs/security-audit.md#environment-and-audit_pat) for provisioning." > audit-report.md
echo "::error::AUDIT_PAT secret is not set."
exit 1

- name: Audit against SECURITY.md
- name: Audit against the security specs
uses: anthropics/claude-code-action@833fb0f8c9f6686b33d963a8bae0a94f4936ab2a # v1
env:
# `claude-code-action` resets `GH_TOKEN` to its own internal
Expand Down Expand Up @@ -133,16 +133,17 @@ jobs:
claude_args: >-
--allowed-tools "Read,Write,Edit,Bash,Grep,Glob,Task,Agent"
--disallowed-tools "Workflow"
--agents '{"supply-chain":{"description":"Audits the Dependency Supply Chain section of SECURITY.md.","prompt":"Read `.github/audit/_preamble.md` and then `.github/audit/supply-chain.md`, and follow them exactly."},"ci-and-secrets":{"description":"Audits the CI, tend, release, and audit-contract sections of SECURITY.md.","prompt":"Read `.github/audit/_preamble.md` and then `.github/audit/ci-and-secrets.md`, and follow them exactly."},"application-security":{"description":"Audits the Remote Control section of SECURITY.md, and sweeps the rest of the product code.","prompt":"Read `.github/audit/_preamble.md` and then `.github/audit/application-security.md`, and follow them exactly.","model":"opus"}}'
--agents '{"supply-chain":{"description":"Audits docs/specs/security-supply-chain.md.","prompt":"Read `.github/audit/_preamble.md` and then `.github/audit/supply-chain.md`, and follow them exactly."},"ci-and-secrets":{"description":"Audits docs/specs/security.md, docs/specs/security-ci.md, and docs/specs/security-audit.md.","prompt":"Read `.github/audit/_preamble.md` and then `.github/audit/ci-and-secrets.md`, and follow them exactly."},"application-security":{"description":"Audits docs/specs/security-local.md and docs/specs/security-remote.md, and sweeps the rest of the product code.","prompt":"Read `.github/audit/_preamble.md` and then `.github/audit/application-security.md`, and follow them exactly.","model":"opus"}}'
# The prompts live in `.github/audit/`, not inline here: they
# are long enough to need real diffs in review, they must be
# runnable locally against the same text CI uses
# (`scripts/security-audit-local.sh`), and the `FAIL IF` that
# requires every `## ` heading to be owned by exactly one
# domain is only a grep when those headings sit in markdown.
# Inline, YAML block-scalar wrapping split
# `## Automated Maintainer (tend)` across two lines and the
# check matched nothing.
# (`scripts/security-audit-local.sh`), and each domain's scope
# — the bullet list of security specs under its `**Scope` line
# — is what `scripts/spec-lint.mjs` (check 16) reads to prove
# every `docs/specs/security*.md` has exactly one owner. That
# lint cannot read YAML block scalars, and when the scopes were
# inline here the ownership check was a grep that wrapping
# once broke silently.
prompt: |
You are the orchestrator of this repository's nightly
security audit.
Expand Down Expand Up @@ -200,7 +201,7 @@ jobs:
TRANSCRIPT="$RUNNER_TEMP/claude-execution-output.json"
# Every sink that the archive step below publishes has to be in
# the argument list, and the fail-closed `rm -f` has to delete
# the same set — SECURITY.md names `audit-report.md`, the three
# the same set — docs/specs/security-audit.md names `audit-report.md`, the three
# per-domain fragments, and the transcript. The fragments are
# agent output like any other: a subagent that shells out and
# prints an environment lands the value in its own fragment,
Expand Down Expand Up @@ -308,7 +309,7 @@ jobs:
# write no status file at all in that case — which lands on the
# MISSING branch below, not on FAIL — but a prompt is not a
# control: without this check the matching `FAIL IF` in
# SECURITY.md is enforced only by the document being audited,
# docs/specs/security-audit.md is enforced only by the document being audited,
# which is circular. Downgrade to MISSING rather than FAIL for the same
# reason the three outcomes exist above — this is an audit that
# did not finish, not a security finding — and MISSING already
Expand Down Expand Up @@ -457,7 +458,7 @@ jobs:
HEADLINE="Audit failed at $DATE."
else
TITLE="[security-audit] INCONCLUSIVE on $(date -u +%Y-%m-%d)"
HEADLINE="Audit reached no usable verdict at $DATE."
HEADLINE="Audit reached no usable verdict at $DATE. This is not a security finding: the run ended without deciding."
fi

{
Expand All @@ -483,7 +484,7 @@ jobs:
} > audit-comment.md
rm -f "$NOTES"

# Truncate before posting, non-fatally. See SECURITY.md.
# Truncate before posting, non-fatally. See docs/specs/security-audit.md -> "Outcomes and reporting".
node scripts/clamp-issue-body.mjs audit-comment.md \
--note "The untruncated \`audit-report.md\` is in this run's \`audit-transcript\` artifact${TRANSCRIPT_URL:+ ([download]($TRANSCRIPT_URL))}." \
|| echo "clamp-issue-body.mjs failed; posting audit-comment.md unclamped." >&2
Expand Down
16 changes: 8 additions & 8 deletions .github/workflows/workflow-audit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ name: workflow-audit
#
# Both classifiers fail open: any error, ambiguity, or unparseable input
# reports the commit. A silent run is the healthy steady state and keeps
# the SECURITY.md 48-hour liveness check green — that check keys on a
# the 48-hour liveness check in docs/specs/security-ci.md green — that check keys on a
# successful *run*, not on an issue existing.
#
# Deliberately not deduped by (branch, file-set): that would let a
Expand Down Expand Up @@ -85,7 +85,7 @@ jobs:
echo "Auditing commits since: $SINCE"

# .github/audit/ is in the window because it holds the security
# audit's own prompts — which domain owns which SECURITY.md section,
# audit's own prompts — which domain owns which security spec,
# what each one is told to check, and the rule that keeps the
# orchestrator from ending its turn. A bot that edits those changes
# what gets audited without touching a single workflow file, which is
Expand All @@ -100,14 +100,14 @@ jobs:
# folder — the same persistence-on-checkout shape a workflow gives,
# reached without touching .github/.
#
# SECURITY.md is deliberately NOT in the window, though it was
# briefly. Two reasons, and the second is why the first is not enough
# The security specs (docs/specs/security*.md) are deliberately NOT in
# the window, though SECURITY.md was briefly. Two reasons, and the second is why the first is not enough
# on its own. What this job watches is code that *executes from any
# branch*: a workflow runs on a bot-pushed branch, and a folderOpen
# task runs when someone checks one out. SECURITY.md is inert until
# task runs when someone checks one out. A security spec is inert until
# it is merged to main, which is admin-gated — so the branch-wide
# watch buys nothing PR review does not already give. And it changes
# in nearly every security commit, so including it would report a
# watch buys nothing PR review does not already give. And they change
# in nearly every security commit, so including them would report a
# commit on almost every such PR; a control that cries wolf on
# routine work is one people learn to skim.
#
Expand Down Expand Up @@ -175,7 +175,7 @@ jobs:
# still repoint `actions/checkout` at `evil/action`, which is why the
# action name is compared. The residual is a ref selected by Renovate
# within that action's own repo — the same trust Renovate bumps already
# rest on (see "GitHub Actions Policies" in SECURITY.md).
# rest on (see docs/specs/security-ci.md -> "Automated Maintainer (tend)").
is_renovate_pin_bump() {
local sha="$1" login pr_authors diff changed removed added

Expand Down
Loading