From 93c2b361e4f2b4f3f54785855a1f5d131faa4fc8 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 00:31:22 -0400 Subject: [PATCH 01/30] build: 7-day supply-chain soak + pinned security tooling Ports the nub soak/security stack (nubjs/nub#442) with the wheelhouse shim workarounds baked in: - scripts/soak/: soak window parity gate + fixer (SOAK_DAYS=7 in constants.mts is the single source), soaked dependency updater (taze for npm, rustup cargo for crates), and the external-tools installer (SRI-verified rack + PATH handles + sfw firewall shims). - Soak surfaces: .npmrc min-release-age, tools/pnpm-workspace.yaml minimumReleaseAge (pnpm domain isolated in tools/ so the root stays npm-only), tools/taze.config.mts (imports SOAK_DAYS), .cargo/config.toml min-publish-age (nightly-only; inert on stable), .github/dependabot.yml cooldown per update block (the renovate-check equivalent, reworked for dependabot). - external-tools.json: exact pins + sha512 SRI for pnpm 11.8.0, npm 12, sfw-free/-enterprise 1.13.1, zizmor 1.26.1, agentshield 1.4.0, skillspector @2eb84478. - sfw shims carry the known workarounds: per-command recursion sentinel (not PATH-strip), fail-open when sfw is absent, symlink clobber guard, rack-pinned pnpm/npm resolution. - zizmor workflow (hash-pinned action, gate at high) with a documented starting config: official actions ref-pin, six existing third-party actions grandfathered pending a digest-pin sweep, cache-poisoning (49 Low-confidence findings) disabled, release-packages.yml excessive-permissions scoped-ignored pending a job-level split. - security-audit.yml gains always-run soak-gate, agent-scan (AgentShield over .claude/), and skills-scan (NVIDIA SkillSpector over .claude/skills/, static --no-llm path) jobs. - .claude/skills/soak/SKILL.md documents the workflow. --- .cargo/config.toml | 13 + .claude/skills/soak/SKILL.md | 84 ++++ .github/dependabot.yml | 7 + .github/workflows/security-audit.yml | 66 ++++ .github/workflows/zizmor.yml | 32 ++ .github/zizmor.yml | 46 +++ .npmrc | 9 + external-tools.json | 169 ++++++++ package.json | 8 + scripts/soak/constants.mts | 50 +++ scripts/soak/external-tools.mts | 568 +++++++++++++++++++++++++++ scripts/soak/external-tools.test.mts | 219 +++++++++++ scripts/soak/paths.mts | 74 ++++ scripts/soak/soak.mts | 516 ++++++++++++++++++++++++ scripts/soak/soak.test.mts | 215 ++++++++++ scripts/soak/update-deps.mts | 98 +++++ tools/package.json | 8 + tools/pnpm-lock.yaml | 223 +++++++++++ tools/pnpm-workspace.yaml | 25 ++ tools/taze.config.mts | 14 + 20 files changed, 2444 insertions(+) create mode 100644 .cargo/config.toml create mode 100644 .claude/skills/soak/SKILL.md create mode 100644 .github/workflows/zizmor.yml create mode 100644 .github/zizmor.yml create mode 100644 .npmrc create mode 100644 external-tools.json create mode 100644 scripts/soak/constants.mts create mode 100644 scripts/soak/external-tools.mts create mode 100644 scripts/soak/external-tools.test.mts create mode 100644 scripts/soak/paths.mts create mode 100644 scripts/soak/soak.mts create mode 100644 scripts/soak/soak.test.mts create mode 100644 scripts/soak/update-deps.mts create mode 100644 tools/package.json create mode 100644 tools/pnpm-lock.yaml create mode 100644 tools/pnpm-workspace.yaml create mode 100644 tools/taze.config.mts diff --git a/.cargo/config.toml b/.cargo/config.toml new file mode 100644 index 0000000000..a99cd51cbd --- /dev/null +++ b/.cargo/config.toml @@ -0,0 +1,13 @@ +# Supply-chain soak for cargo's own dependency resolution — the same +# soak-window rule the npm side applies via minimumReleaseAge / +# min-release-age: crate versions younger than the window are skipped by +# the resolver unless already in Cargo.lock. min-publish-age is an +# [unstable] cargo feature, so these keys bite only under a nightly +# toolchain; on perry's stable toolchain they are inert and the automated +# window rides dependabot's cooldown (.github/dependabot.yml) instead. +# Managed by scripts/soak/soak.mts (`npm run soak` / `npm run soak:fix`). +[unstable] +min-publish-age = true + +[registry] +global-min-publish-age = "7 days" diff --git a/.claude/skills/soak/SKILL.md b/.claude/skills/soak/SKILL.md new file mode 100644 index 0000000000..dd798e59d7 --- /dev/null +++ b/.claude/skills/soak/SKILL.md @@ -0,0 +1,84 @@ +--- +name: soak +description: Manages the repo's supply-chain soak window (SOAK_DAYS) — checks and fixes the derived surfaces, bumps or disables the window, adds dated per-package exclusions, and bumps pinned external tools. Use when a task touches minimumReleaseAge, min-release-age, min-publish-age, dependabot cooldown, external-tools.json, sfw shims, or taze cooldowns, or when investigating why a freshly published version won't install. +--- + +# The soak window + +One rule: a release must be at least `SOAK_DAYS` old before this repo +adopts it. The delay gives the ecosystem time to catch a malicious or +yanked release before we ever install it. The window is defined exactly +once — read the current value from `scripts/soak/constants.mts` and never +hardcode it elsewhere. Every surface derives from or is parity-checked +against it: + +| Surface | Key | Units | +|---|---|---| +| `.cargo/config.toml` | `global-min-publish-age` (nightly-only feature; inert on perry's stable toolchain) | `"N days"` | +| `tools/pnpm-workspace.yaml` | `minimumReleaseAge` | minutes | +| `.npmrc` | `min-release-age` | days | +| `tools/taze.config.mts` | `maturityPeriod` | imports `SOAK_DAYS` | +| `external-tools.json` | `soakBypass` annotations | days | +| `.github/dependabot.yml` | `cooldown.default-days` per update block | days | + +## Commands (package.json scripts — the code lives in `scripts/soak/`) + +- `npm run soak` — parity-check every surface (CI-gated: `soak-gate` job + in security-audit.yml, always-run) +- `npm run soak:fix` — rewrite drifted windows, prune expired exclusions +- `npm run deps:update` — bump npm (taze) + cargo deps through the window +- `npm run tools:check` / `tools:install` — validate / install the + SRI-pinned external tools (`external-tools.json`); `tools:install` also + writes the sfw firewall shims into the dev-tools bin dir +- `npm run test:scripts` — the scripts' own unit tests + +A soak change is done when `npm run soak` and `npm run test:scripts` +both exit 0 — the same gates CI runs. Re-run them after every fix. + +## Change the window (one place) + +1. Edit `SOAK_DAYS` in `scripts/soak/constants.mts`. +2. `npm run soak:fix` (rewrites cargo/npmrc/yaml and drifted dependabot + values; taze follows by import). A dependabot block with NO cooldown + at all is a check finding fixed by hand — add the two lines where the + finding says. +3. `npm run soak` + `npm run test:scripts` — existing exclusion + annotations encode the old window and will be flagged; re-date or + remove them, then re-run until both pass. + +**Opt out entirely**: set `SOAK_DAYS = 0` and run the same two steps — +cargo, pnpm, npm, and taze all treat zero as disabled. There is +deliberately no env-var bypass: opting out is a committed, reviewable +change, never a silent one. + +## Skip the soak for ONE package (dated, temporary) + +Add to `minimumReleaseAgeExclude` in `tools/pnpm-workspace.yaml` with the +annotation on the line above (block list only — flow `[..]` is rejected +because a comment line can't attach to an inline entry): + +```yaml +# published: 2026-07-08 | removable: 2026-07-15 +- 'name@1.2.3' +``` + +`removable` = `published + SOAK_DAYS` (this example assumes a 7-day +window). `published` must be the real registry publish date. Once +`removable` passes, `npm run soak` fails until the pin is pruned +(`soak:fix` does it). Bare names / `@scope/*` globs are standing trust and +need no annotation. External tools use the same shape via a `soakBypass` +object in `external-tools.json`. + +## Maintaining this skill + +`scripts/soak/` is the law; this file only documents it — when they +disagree, fix this file. When editing, follow Anthropic's guidance: + +- [Prompting best practices](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices) +- [Prompting Claude Fable 5](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-fable-5) +- [Skill authoring best practices](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices) +- [Write an effective CLAUDE.md](https://code.claude.com/docs/en/best-practices#write-an-effective-claude-md) + +Keep it concise (goal + constraints, not step enumeration), keep the +description in third person with explicit "use when" triggers, and keep +the window value in `constants.mts` rather than restating it here. diff --git a/.github/dependabot.yml b/.github/dependabot.yml index a83cb86fea..bdb85d369a 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -5,6 +5,11 @@ updates: schedule: interval: weekly open-pull-requests-limit: 5 + # Release-age soak: a bumped version must be at least this many days + # old before dependabot proposes it (SOAK_DAYS in + # scripts/soak/constants.mts; `npm run soak` gates the parity). + cooldown: + default-days: 7 groups: cargo-minor-and-patch: update-types: [minor, patch] @@ -33,6 +38,8 @@ updates: schedule: interval: weekly open-pull-requests-limit: 3 + cooldown: + default-days: 7 commit-message: prefix: "ci" include: scope diff --git a/.github/workflows/security-audit.yml b/.github/workflows/security-audit.yml index 32d5841ee8..8e36facac1 100644 --- a/.github/workflows/security-audit.yml +++ b/.github/workflows/security-audit.yml @@ -64,6 +64,72 @@ jobs: --ignore RUSTSEC-2026-0119 \ --ignore RUSTSEC-2026-0187 + # Soak parity gate + external-tool pin gate. Always-run (deliberately not + # path-filtered — nub hid this gate in a path-gated job and it silently + # skipped Rust-only PRs). The scripts are dep-free erasable-TS .mts run + # with the pinned Node's native type stripping; no npm install needed. + soak-gate: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: actions/setup-node@v7 + with: + node-version-file: .node-version + - name: Soak window parity (npm run soak) + run: node scripts/soak/soak.mts --check + - name: External-tool pins valid (npm run tools:check) + run: node scripts/soak/external-tools.mts --check + - name: Soak script unit tests (npm run test:scripts) + run: node --test scripts/soak/*.test.mts + + # AgentShield — audits the operator-side Claude config (.claude/: hooks, + # permissions, MCP servers, agents) for prompt injection, leaked secrets, + # over-permissive tool grants. Installed from the SRI-pinned registry + # tarball in external-tools.json (never `npm install -g`). + agent-scan: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: actions/setup-node@v7 + with: + node-version-file: .node-version + - name: Install pinned agentshield + run: node scripts/soak/external-tools.mts --install agentshield + - name: Scan .claude/ config + run: | + export PATH="$(node scripts/soak/external-tools.mts --print-bin):$PATH" + agentshield scan + + # SkillSpector (NVIDIA) — audits the repo's Claude skills (.claude/skills/) + # before they run on anyone's machine: YARA + AST static analysis (the + # --no-llm path; no API key needed in CI). Pinned to the same git SHA as + # external-tools.json tools.skillspector; python pinned because + # yara-python ships no cp314 wheels yet. + skills-scan: + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 + - name: Scan each skill + run: | + set -euo pipefail + for skill in .claude/skills/*/; do + echo "::group::skillspector ${skill}" + uv tool run --python 3.12 \ + --from git+https://github.com/NVIDIA/skillspector@2eb84478 \ + skillspector scan "${skill}" --no-llm + echo "::endgroup::" + done + # License policy + duplicate-version tracking (deny.toml at repo root). # Advisories stay with cargo-audit above — deny.toml doesn't duplicate them. # Pure metadata check: no compilation, runs in ~1 min once cargo-deny is diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml new file mode 100644 index 0000000000..91bebc2bd8 --- /dev/null +++ b/.github/workflows/zizmor.yml @@ -0,0 +1,32 @@ +name: zizmor + +on: + push: + branches: [main] + pull_request: + paths: ['.github/**'] + +permissions: {} + +jobs: + zizmor: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + with: + persist-credentials: false + - uses: zizmorcore/zizmor-action@6599ee8b7a49aef6a770f63d261d214911a7ce02 # v0.6.0 + with: + advanced-security: false + inputs: .github/ + # Version rides external-tools.json tools.zizmor (keep in lockstep + # so local `npm run tools:install` audits with the same zizmor). + version: "1.26.1" + # Starting gate: high only. The 34 medium findings (missing + # permissions: blocks in test.yml / benchmark.yml / + # container-tests.yml / cache-warm.yml / simctl-tests.yml) are a + # tracked ratchet — audit each job's real token needs, add the + # blocks, then drop this line to gate at medium. + min-severity: high diff --git a/.github/zizmor.yml b/.github/zizmor.yml new file mode 100644 index 0000000000..fb64cf61c6 --- /dev/null +++ b/.github/zizmor.yml @@ -0,0 +1,46 @@ +# zizmor (GitHub Actions security audit) configuration. +# See: https://docs.zizmor.sh/configuration/ +# +# Starting posture (first zizmor gate for this repo): enforce for NEW +# surface, grandfather the existing debt explicitly so the gate is green +# and meaningful from day one. Each carve-out below is a ratchet — remove +# entries as the debt is paid down, never add without a dated reason. + +rules: + unpinned-uses: + config: + policies: + # Official GitHub actions may ride major-version tags (dependabot + # keeps them bumped; GitHub controls the tags). + actions/*: ref-pin + github/*: ref-pin + # Grandfathered third-party actions already in the workflows + # (2026-07): ~75 use-sites, dominated by dtolnay/rust-toolchain + # where the ref IS the toolchain selector (@stable), so digest + # pinning needs a `toolchain:` input added per site. Ratchet: + # digest-pin these in a follow-up sweep, then delete the lines. + dtolnay/rust-toolchain: ref-pin + Swatinem/rust-cache: ref-pin + mozilla-actions/sccache-action: ref-pin + taiki-e/install-action: ref-pin + oven-sh/setup-bun: ref-pin + ilammy/msvc-dev-cmd: ref-pin + # Everything else — including any NEWLY introduced third-party + # action — must be pinned to a commit SHA. + "*": hash-pin + + # 49 findings, all audit-confidence Low: Swatinem/rust-cache in workflows + # that also publish artifacts. The release workflows are tag-gated and + # caches are keyed per-workflow; restructuring cache scope is a workflow + # redesign, not a lint fix. Revisit if zizmor's confidence rises or the + # release pipeline changes. + cache-poisoning: + disable: true + + excessive-permissions: + ignore: + # Workflow-level contents:write + actions:write predate this gate; + # splitting them across the 13 release jobs is a release-pipeline + # change that needs its own carefully-tested PR (a mistake only + # manifests at the next tag). Tracked as follow-up. + - release-packages.yml diff --git a/.npmrc b/.npmrc new file mode 100644 index 0000000000..cc2470c4d1 --- /dev/null +++ b/.npmrc @@ -0,0 +1,9 @@ +# npm-cli soak window (npm >= 11.17, DAYS) — governs the root npm install +# that materializes the parity-test fixture deps (package-lock.json). +# Managed by scripts/soak/soak.mts (`npm run soak` / `npm run soak:fix`). +# +# The pnpm-side soak (minimumReleaseAge, catalog) lives in +# tools/pnpm-workspace.yaml, NOT here: pnpm only honors minimumReleaseAge +# from a workspace yaml, and a workspace yaml at the repo root would mark +# the npm-managed root as a pnpm workspace. +min-release-age=7 diff --git a/external-tools.json b/external-tools.json new file mode 100644 index 0000000000..c2699d32b0 --- /dev/null +++ b/external-tools.json @@ -0,0 +1,169 @@ +{ + "tools": { + "pnpm": { + "description": "pnpm \u2014 the fleet's package manager.", + "version": "11.8.0", + "packageManager": "pnpm", + "repository": "github:pnpm/pnpm", + "release": "asset", + "notes": [ + "Latest soaked pnpm \u2014 CI downloads + SRI-verifies (sha512) the pinned asset", + "darwin-x64 has no SEA asset upstream; its pin is the npm registry tarball" + ], + "platforms": { + "darwin-arm64": { + "asset": "pnpm-darwin-arm64.tar.gz", + "integrity": "sha512-kgYcLu653+Gx7l7qEtM2zMNLBvb96ABcRSkzOrbfvkndb+veMt6lHJK5r3wTfuRYlDb8Lnk/h0wUD6sP3taJ9g==" + }, + "darwin-x64": { + "asset": "pnpm-11.8.0.tgz", + "integrity": "sha512-wfXnxMskHI8XS3Q4UdgvQrgCMkr8iw8Ra5atsVqgZmSUjd42lgo7oQebpbSyndAUATW5S1tfUmNZIknWjlVfJg==" + }, + "linux-arm64": { + "asset": "pnpm-linux-arm64.tar.gz", + "integrity": "sha512-p1IcVUlwYf3OJQYmdHGFr08d1BOSatHEmLJSBSdoNPGqOqfslFHjSqiuZ/3yuQxxypqp8OfnM3ci3Jh7ZEBlSw==" + }, + "linux-arm64-musl": { + "asset": "pnpm-linux-arm64-musl.tar.gz", + "integrity": "sha512-u5Do3diwK7FL5vk+i/x2I4q4ujdZ5gSoLNgmlt1w2C7NOjAZOpDK0CJYM4gDa5BetEwzdNpuN/gaMqzJk6I5NA==" + }, + "linux-x64": { + "asset": "pnpm-linux-x64.tar.gz", + "integrity": "sha512-Sn7hG4Xsq6pmi8TE8lpIkwRzAYzf5qtFk8zSsZWkSJFcmv5FlsrO8UP3lLRI+ppcRNYS10FMCTwXe3Nt66A0Pg==" + }, + "linux-x64-musl": { + "asset": "pnpm-linux-x64-musl.tar.gz", + "integrity": "sha512-5WXlo2yCDmoBIue5iHRK3zNSPVAmzci+5RhuxwUA4hnH9W3TFjDnQHyAzDVnkY878mfTXE35THZeEc1OKYPmag==" + }, + "win-arm64": { + "asset": "pnpm-win32-arm64.zip", + "integrity": "sha512-iv1hJEj9FUiVFae2cmCYjbRK0Xn27Uh+kpglT2LWvQyl4WeIOgs1ouKrptVxk3X3t7DB5vuYKbsmE+9BnoQ4FA==" + }, + "win-x64": { + "asset": "pnpm-win32-x64.zip", + "integrity": "sha512-jyqMgedndbck/xJjXPem5Lw7V0YtDfiUJIB81e/hmnrg3yLKPoyCKO5ILegYkLPIr945IJRcVoJZwjtog8FCqg==" + } + } + }, + "npm": { + "notes": [ + "npm 12 (min-release-age support). ONE platform-agnostic registry tarball, single integrity", + "Installed from the pinned tarball only \u2014 never npm install -g npm, no self-update path" + ], + "description": "npm \u2014 pinned, SRI-verified registry tarball; installed without self-update", + "repository": "npm:npm", + "version": "12.0.0", + "integrity": "sha512-qzvPQfNSY7louiM6rv7dL0hi5esBGLn1lLwxbdyL5XOIssWzoYMwn8xqvWhYcZL6onTkenYSxrtKxsFrFbUFyw==" + }, + "sfw-free": { + "description": "Socket Firewall (free tier) \u2014 malware gate on dep installs.", + "version": "1.13.1", + "repository": "github:SocketDev/sfw-free", + "release": "asset", + "binaryName": "sfw", + "notes": [ + "Used when SOCKET_SECURITY_KEY is not set", + "Shims npm/yarn/pnpm so every install call passes through the firewall" + ], + "platforms": { + "darwin-arm64": { + "asset": "sfw-free-macos-arm64", + "integrity": "sha512-T6wBOJGdRVSI8577lGqRNzNd6Q+1vqKyaqGgOA8G4M5MU2vcsUnXuJTgP2MMZjUqROSXUlFL0mHguuxXT2QadQ==" + }, + "darwin-x64": { + "asset": "sfw-free-macos-x86_64", + "integrity": "sha512-4G/AIY5UGU81wcepDKErY5u0nY85D8UM9nXTEPv8CR2rOV/s4IcmrkxywwZ3ipejHVQB7QmCVt0/SsqWglGikw==" + }, + "linux-arm64": { + "asset": "sfw-free-linux-arm64", + "integrity": "sha512-FYRYR52SL+KKFldW4ogYOUnTH5OSqvtXwzGFeWi0W2x+75KZcPiGzWBbhMmh0f5QtgYLV+4qdREgmKCBEayNtA==" + }, + "linux-x64": { + "asset": "sfw-free-linux-x86_64", + "integrity": "sha512-waLrsPG2a7EOv0XuvXDQZGgCZ4MTtOfZh8TmGbM6gn2B6Nh6HI+15jaoKdAS9wgdTyIqTuqU+O+NtVYd+kuFaA==" + }, + "win-x64": { + "asset": "sfw-free-windows-x86_64.exe", + "integrity": "sha512-YYnfwR6M/PHo72LSyKtpY3bAUG4F4ckToJqGx5Fkz4rwg1+48hkxuBaF3hdxHUdHPkfO5grDyoNgXGe7FojGcg==" + } + } + }, + "sfw-enterprise": { + "description": "Socket Firewall (enterprise tier) \u2014 selected when SOCKET_SECURITY_KEY is set.", + "version": "1.13.1", + "repository": "github:SocketDev/firewall-release", + "release": "asset", + "binaryName": "sfw", + "notes": [ + "Used when SOCKET_SECURITY_KEY is set (the one env var every Socket product reads)", + "Same shims as sfw-free, broader ecosystem support (Ruby, .NET, Go on Linux)" + ], + "platforms": { + "darwin-arm64": { + "asset": "sfw-macos-arm64", + "integrity": "sha512-ZDy2C6leKyTHZFvcZZpG2eQqVzs7buk+Hs92fkaMYME829QzyxdGQVVgwEVaGJpedGdUvhksKKcvT9IynI1kxg==" + }, + "darwin-x64": { + "asset": "sfw-macos-x86_64", + "integrity": "sha512-cm76we0sn7kqPOya/ZGQpPyhjRDyFT5lHigeT5Qso+QaPL6Cmwi0FVs2L7l63j+WR/9eYPU1WjjGOto5NbWsEQ==" + }, + "linux-arm64": { + "asset": "sfw-linux-arm64", + "integrity": "sha512-9qPi3mobBfyq1k+pD2GDG0tZkhy16f7FXE9oGiwmwPvy5PXwnlzqEXnYld3qGsKIVwNhunev/If26oROTHbrHA==" + }, + "linux-x64": { + "asset": "sfw-linux-x86_64", + "integrity": "sha512-lu9h8UzDZt34gdCEVHBGW6goE1Ayykq413EovV5B4nG7jBK27mI0GQstzVbWXA3wWaweT39PehXGtVpdqIDGSA==" + }, + "win-x64": { + "asset": "sfw-windows-x86_64.exe", + "integrity": "sha512-URZXauIsdUT12E2KTc4sfsxRmJm7nRJzAgM+IYGX4Xq+X0cl/eAbH5SpYIJKpsnW9csSztW9ceyPhlM+f3neIQ==" + } + } + }, + "zizmor": { + "description": "GitHub Actions security linter \u2014 audits .github/ for workflow-injection / credential-leak patterns.", + "version": "1.26.1", + "repository": "github:zizmorcore/zizmor", + "release": "asset", + "notes": [ + "Required: CI (blocks merges on medium+ findings)", + "Installed by the setup-and-install composite; SRI-verified (sha512) per platform" + ], + "platforms": { + "darwin-arm64": { + "asset": "zizmor-aarch64-apple-darwin.tar.gz", + "integrity": "sha512-UfLPPdYejR8fvrFwr9Tos7LKFvqr7YcAHWZlAmeo7imYI/fIucYnCY6HeDyk2cbFUvDpQTdps6WQS1QpqCtTfQ==" + }, + "darwin-x64": { + "asset": "zizmor-x86_64-apple-darwin.tar.gz", + "integrity": "sha512-SCbcEzF/zy2qNuNaocLPIUC4Wuq5GnHt0iUFve3qx6bJyFdzU6pDFZkF64dojGC7M5gAcK/4acXGPx3YnMDy/g==" + }, + "linux-arm64": { + "asset": "zizmor-aarch64-unknown-linux-gnu.tar.gz", + "integrity": "sha512-TkGvwt0zYdmiJ7LZmy6Bz9CdkqcuEpFKhXUK9m0MhSPxBx2gYBsrw6OxGSttMEQ3bvxdvyG6rqbnMgjyDZB1zA==" + }, + "linux-x64": { + "asset": "zizmor-x86_64-unknown-linux-gnu.tar.gz", + "integrity": "sha512-zTMERMDd3JfaRX12klj2fhZGDyrXeLkVUY1QJkCv8RRmAa9uVuH88gHOnv4CQtC5hXS7FPRJzvCVOnw38gV83g==" + }, + "win-x64": { + "asset": "zizmor-x86_64-pc-windows-msvc.zip", + "integrity": "sha512-Pijh/CrrOAkZzLiTr2LTHdI8d6+5Ql6B+suY6fXVmL8UVa+4Q36hHL5K67iRFz03v/V/UcrY6+dfhnmot/xfww==" + } + } + }, + "agentshield": { + "description": "Claude AI config security scanner (prompt injection, secrets)", + "purl": "pkg:npm/ecc-agentshield@1.4.0", + "integrity": "sha512-R98OO1Ujyk2lezDLb+iQmMhF6FwTJCHajy3G4FCB6x7wkSTqR9f8+eAelC5KDzYDsGSbc0sOZvjXOOPRBtMpDg==" + }, + "skillspector": { + "description": "NVIDIA's third-party-skill security scanner (LangGraph-based; YARA + AST + OSV.dev CVE lookups + optional LLM analysis). No PyPI release / no GH tags upstream \u2014 pinned to a git SHA on main + installed via a locked uv project (pyproject.toml + uv.lock, `uv sync --locked`; the fleet uv pin + exclude-newer make it reproducible). Sibling to AgentShield: AgentShield audits the operator's .claude/ config; SkillSpector audits untrusted upstream skills before install.", + "release": "uv-project", + "repository": "github:NVIDIA/skillspector", + "version": "2eb84478", + "versionDate": "2026-05-18" + } + } +} diff --git a/package.json b/package.json index 7f86c2fe66..2314301a4c 100644 --- a/package.json +++ b/package.json @@ -1,5 +1,13 @@ { "type": "module", + "scripts": { + "soak": "node scripts/soak/soak.mts --check", + "soak:fix": "node scripts/soak/soak.mts --fix", + "deps:update": "node scripts/soak/update-deps.mts", + "tools:check": "node scripts/soak/external-tools.mts --check", + "tools:install": "node scripts/soak/external-tools.mts --install-all --shims", + "test:scripts": "node --test scripts/soak/*.test.mts" + }, "devDependencies": { "mongodb": "^7.0.0", "zod": "^4.3.5" diff --git a/scripts/soak/constants.mts b/scripts/soak/constants.mts new file mode 100644 index 0000000000..116e3a3d5f --- /dev/null +++ b/scripts/soak/constants.mts @@ -0,0 +1,50 @@ +/** + * @file Canonical soak window — the ONE source for every release-age surface. + * A new or bumped third-party dependency must have been published at least + * this long before the repo adopts it: the cooldown catches a compromised + * upstream before it lands. Every soak surface DERIVES from `SOAK_DAYS` + * instead of hand-copying the number: + * + * - `.cargo/config.toml` -> `global-min-publish-age = " days"` (cargo -Zmin-publish-age) + * - `rust-toolchain.toml` -> dated nightly adopted only once >= SOAK_DAYS old + * - `pnpm-workspace.yaml` -> `minimumReleaseAge: ` (aube reads minutes) + * - `.npmrc` -> `min-release-age=` (npm >= 11.17, days) + * - `taze.config.mts` -> `maturityPeriod: SOAK_DAYS` (imports this) + * + * The data files can't import this module, so `scripts/soak/soak.mts` + * asserts they match (code-is-law parity gate). + */ + +export const SOAK_DAYS = 7 + +// pnpm/aube `minimumReleaseAge` is expressed in MINUTES. +export const SOAK_MINUTES = SOAK_DAYS * 24 * 60 + +// Exclusion annotation carried on the line ABOVE every version-pinned +// `minimumReleaseAgeExclude` entry. `removable` = `published` + SOAK_DAYS; +// once `removable` is in the past the pin has soaked and must be pruned. +export const ANNOTATION_RE = + /^#\s*published:\s*(\d{4}-\d{2}-\d{2})\s*\|\s*removable:\s*(\d{4}-\d{2}-\d{2})\s*$/ + +// A version-pinned exclude entry (`'name@1.2.3'` / `'@scope/name@1.2.3'`), +// as opposed to a bare name or `@scope/*` glob (which need no annotation: +// they express standing trust, not a dated soak bypass). +export const VERSION_PIN_RE = /^(@?[^@\s]+)@[^@\s]+$/ + +export function todayIso(): string { + return new Date().toISOString().slice(0, 10) +} + +// Shape-valid but impossible dates (2026-13-45) round-trip differently (or +// produce Invalid Date), so callers can reject them with a finding instead +// of crashing on Invalid Date arithmetic. +export function isValidIsoDate(iso: string): boolean { + const d = new Date(`${iso}T00:00:00Z`) + return !Number.isNaN(d.getTime()) && d.toISOString().slice(0, 10) === iso +} + +export function addDaysIso(iso: string, days: number): string { + const d = new Date(`${iso}T00:00:00Z`) + d.setUTCDate(d.getUTCDate() + days) + return d.toISOString().slice(0, 10) +} diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts new file mode 100644 index 0000000000..daa2eb5e7a --- /dev/null +++ b/scripts/soak/external-tools.mts @@ -0,0 +1,568 @@ +#!/usr/bin/env node +/** + * @file Pinned external security tooling — download, verify, shim. + * `external-tools.json` (repo root) pins every tool to an exact version + * with a sha512 SRI integrity per platform asset. This script is the only + * way those tools reach a machine: nothing here trusts "latest". + * + * - `--check` validate every pin (shape, SRI prefix, soak + * annotations on any soakBypass) — CI gate, no network + * - `--install ` download + SRI-verify + install into the local + * tool rack (see paths.mts RACK_DIR) with a PATH + * handle in BIN_DIR + * - `--install-all` every installable pin + * - `--shims` write sfw shims (npm/yarn/pnpm/pip/pip3/uv/cargo) + * into BIN_DIR so installs route through the firewall + * - `--print-bin` print BIN_DIR (for `>> $GITHUB_PATH` in CI) + * + * `sfw` resolves to sfw-enterprise when SOCKET_SECURITY_KEY is set (the + * one env var every Socket product reads), else sfw-free — free tier + * needs no key, so CI is firewalled from day one and upgrades itself + * when the repo secret lands. + */ + +import { createHash } from 'node:crypto' +import { spawnSync } from 'node:child_process' +import { + chmodSync, + existsSync, + mkdirSync, + readFileSync, + realpathSync, + rmSync, + symlinkSync, + writeFileSync, +} from 'node:fs' +import path from 'node:path' +import process from 'node:process' +import { pathToFileURL } from 'node:url' + +import { SOAK_DAYS, addDaysIso, isValidIsoDate, todayIso } from './constants.mts' +import { + BIN_DIR, + DOCKER_PREBAKE, + EXTERNAL_TOOLS_JSON, + PM_DEP_INSTALLERS, + RACK_DIR, + REPO_ROOT, + SURFACES, +} from './paths.mts' + +const SFW_ECOSYSTEMS = ['npm', 'yarn', 'pnpm', 'pip', 'pip3', 'uv', 'cargo'] + +interface PlatformPin { + asset: string + integrity: string +} + +interface ToolPin { + description?: string + version?: string + repository?: string + release?: string + binaryName?: string + purl?: string + integrity?: string + platforms?: Record + soakBypass?: { version: string; published: string; removable: string } +} + +function loadTools(): Record { + return JSON.parse(readFileSync(EXTERNAL_TOOLS_JSON, 'utf8')).tools +} + +function platformKey(): string { + const osKey = { darwin: 'darwin', linux: 'linux', win32: 'win' }[process.platform] + const archKey = { arm64: 'arm64', x64: 'x64' }[process.arch] + if (!osKey || !archKey) { + throw new Error(`unsupported platform ${process.platform}-${process.arch}`) + } + return `${osKey}-${archKey}` +} + +function sriSha512(buf: Buffer): string { + return `sha512-${createHash('sha512').update(buf).digest('base64')}` +} + +export function checkPins(tools: Record): string[] { + const out: string[] = [] + for (const [name, pin] of Object.entries(tools)) { + if (!pin.version && !pin.purl) { + out.push(`${name}: no version or purl pin`) + } + const integrities = [ + ...(pin.integrity ? [pin.integrity] : []), + ...Object.values(pin.platforms ?? {}).map(p => p.integrity), + ] + if (pin.release === 'asset' && integrities.length === 0) { + out.push(`${name}: release asset without any integrity pin`) + } + for (const sri of integrities) { + if (!/^sha512-[A-Za-z0-9+/]+={0,2}$/.test(sri)) { + out.push(`${name}: integrity is not a sha512 SRI: ${sri}`) + } + } + if (pin.soakBypass) { + const { published, removable } = pin.soakBypass + if (!isValidIsoDate(published) || !isValidIsoDate(removable)) { + out.push(`${name}: soakBypass dates are not real YYYY-MM-DD calendar dates`) + continue + } + const expected = addDaysIso(published, SOAK_DAYS) + if (removable !== expected) { + out.push(`${name}: soakBypass removable ${removable}, wanted ${expected} (published + ${SOAK_DAYS}d)`) + } + // A bypass whose window has passed is dead weight: the version has + // soaked, so the annotation must come off (same rule the workspace + // yaml excludes live under). + if (removable < todayIso()) { + out.push(`${name}: soakBypass expired (removable ${removable}) — the pin has soaked, remove the annotation`) + } + } + } + return out +} + +function sriToHex(sri: string): string { + return Buffer.from(sri.slice('sha512-'.length), 'base64').toString('hex') +} + +/** + * Parity gate for the CI agent image: its build context can't reach the + * tracked pin sources, so the Dockerfile embeds copies of the sfw pin + * (version + per-arch sha512 hex) and the toolchain channels. Assert the + * copies match external-tools.json / rust-toolchain.toml so a pin bump + * can't silently strand the image on old bits. + */ +export function checkDockerPrebake( + dockerBody: string, + tools: Record, + toolchainToml: string, + rustVersion = '', +): string[] { + const out: string[] = [] + const shimList = /for cmd in ([^;]+);/.exec(dockerBody)?.[1]?.trim().split(/\s+/) + if (shimList && shimList.join(' ') !== SFW_ECOSYSTEMS.join(' ')) { + out.push(`docker prebake: shim list [${shimList.join(' ')}] != SFW_ECOSYSTEMS [${SFW_ECOSYSTEMS.join(' ')}]`) + } + if (rustVersion && !dockerBody.includes(`toolchain install ${rustVersion}`)) { + out.push(`docker prebake: image does not pre-install the ${rustVersion} msrv toolchain`) + } + const sfw = tools['sfw-free'] + const version = /rack\/sfw-free\/([^/\s]+)\//.exec(dockerBody)?.[1] + if (version !== sfw?.version) { + out.push( + `docker prebake: sfw-free version ${version ?? '(missing)'} != external-tools.json ${sfw?.version}`, + ) + } + const urlVersion = /sfw-free\/releases\/download\/v([^/\s]+)\//.exec(dockerBody)?.[1] + if (urlVersion !== sfw?.version) { + out.push( + `docker prebake: sfw-free download url v${urlVersion ?? '(missing)'} != external-tools.json ${sfw?.version}`, + ) + } + const pairs = [...dockerBody.matchAll(/asset=(\S+);\s*sha=([0-9a-f]{128})/g)] + if (pairs.length === 0) { + out.push('docker prebake: no asset/sha pin pairs found in the Dockerfile') + } + for (const [, asset, hex] of pairs) { + const plat = Object.values(sfw?.platforms ?? {}).find(p => p.asset === asset) + if (!plat) { + out.push(`docker prebake: asset ${asset} has no pin in external-tools.json`) + continue + } + if (sriToHex(plat.integrity) !== hex) { + out.push(`docker prebake: sha for ${asset} != hex of external-tools.json SRI`) + } + } + const channel = /^channel\s*=\s*"([^"]+)"/m.exec(toolchainToml)?.[1] + if (channel && !dockerBody.includes(`toolchain install ${channel} `)) { + out.push(`docker prebake: image does not pre-install the pinned toolchain ${channel}`) + } + return out +} + +export async function download(url: string, expectedSri: string): Promise { + const headers: Record = {} + // Only GitHub gets the token (private release assets); sending it to any + // other host (e.g. the npm registry for purl tools) would leak the + // credential. Cross-origin redirects strip the header automatically. + if (process.env.GITHUB_TOKEN && new URL(url).hostname === 'github.com') { + headers.authorization = `Bearer ${process.env.GITHUB_TOKEN}` + } + // Fail fast on a stalled release/registry response instead of hanging + // CI; 120s is generous for the largest pinned binary on a slow runner. + const res = await fetch(url, { + headers, + redirect: 'follow', + signal: AbortSignal.timeout(120_000), + }) + if (!res.ok) { + throw new Error(`download failed ${res.status} ${url}`) + } + const buf = Buffer.from(await res.arrayBuffer()) + const actual = sriSha512(buf) + if (actual !== expectedSri) { + throw new Error(`integrity mismatch for ${url}\n expected ${expectedSri}\n actual ${actual}`) + } + return buf +} + +export function extractArchive(name: string, destDir: string, asset: string, buf: Buffer): void { + const archive = path.join(destDir, asset) + writeFileSync(archive, buf) + // bsdtar extracts zip via plain -xf too (macOS runners ship it as `tar`). + // Windows needs BOTH quirks handled: Git Bash's PATH shadows System32's + // bsdtar with GNU tar (which can't read zip — "does not look like a tar + // archive"), so address the System32 binary explicitly; and tar parses an + // absolute `C:\...` archive path as a remote `host:path` ("Cannot connect + // to C"), so the archive is addressed RELATIVE to a cwd. + const tarBin = + process.platform === 'win32' + ? path.join(process.env.SystemRoot || 'C:\\Windows', 'System32', 'tar.exe') + : 'tar' + const flags = asset.endsWith('.zip') ? '-xf' : '-xzf' + const res = spawnSync(tarBin, [flags, asset], { cwd: destDir, stdio: 'inherit' }) + rmSync(archive) + if (res.status !== 0) { + throw new Error(`${name}: archive extract failed`) + } +} + +export function linkHandle(target: string, name: string): void { + mkdirSync(BIN_DIR, { recursive: true }) + const handle = path.join(BIN_DIR, name) + // force also removes a DANGLING handle (existsSync would report false + // for one and a bare symlink would then throw EEXIST). + rmSync(handle, { force: true }) + if (process.platform === 'win32') { + // A symlinked/copied handle breaks Windows SEA binaries: pnpm.exe + // resolves its dist/ siblings from the handle's OWN directory, not the + // rack. Forward to the absolute rack target instead — a .cmd for + // cmd/pwsh and an extensionless bash shim for Git Bash. Non-.exe + // targets are node entry scripts (registry-tarball tools). The handle + // BASE must not keep a caller-supplied .exe suffix: pwsh resolves + // `pnpm` to `pnpm.exe` first, and a bash text file wearing that name + // is "not a valid application for this OS platform". + const base = path.join(BIN_DIR, name.replace(/\.exe$/, '')) + const viaExe = target.endsWith('.exe') + rmSync(base, { force: true }) + rmSync(`${base}.cmd`, { force: true }) + rmSync(`${base}.exe`, { force: true }) + writeFileSync( + `${base}.cmd`, + viaExe ? `@echo off\r\n"${target}" %*\r\n` : `@echo off\r\nnode "${target}" %*\r\n`, + ) + writeFileSync( + base, + viaExe + ? `#!/usr/bin/env bash\nexec "${target}" "$@"\n` + : `#!/usr/bin/env bash\nexec node "${target}" "$@"\n`, + ) + chmodSync(base, 0o755) + return + } + symlinkSync(target, handle) +} + +async function installAssetTool(name: string, pin: ToolPin): Promise { + const plat = pin.platforms?.[platformKey()] + if (!plat) { + throw new Error(`${name}: no pinned asset for ${platformKey()}`) + } + // A platform pinned to a registry .tgz (pnpm has no darwin-x64 SEA + // upstream) routes through the npm-tarball path instead. + if (plat.asset.endsWith('.tgz')) { + await installNpmTarball(name, name, pin.version!, plat.integrity) + return + } + const repo = pin.repository!.replace(/^github:/, '') + const url = `https://github.com/${repo}/releases/download/v${pin.version}/${plat.asset}` + let binName = pin.binaryName ?? name + const destDir = path.join(RACK_DIR, name, pin.version!) + let destBin = path.join(destDir, binName) + // Windows archives land as `.exe`; resolve the same suffix the + // post-extract path does so a second --install is a no-op instead of a + // forced re-download (which wedges on a restricted/offline runner). + if (!existsSync(destBin) && existsSync(`${destBin}.exe`)) { + destBin = `${destBin}.exe` + binName = `${binName}.exe` + } + if (existsSync(destBin)) { + linkHandle(destBin, binName) + console.log(`[external-tools] ${name}@${pin.version} already installed`) + return + } + console.log(`[external-tools] downloading ${name}@${pin.version} (${plat.asset})`) + const buf = await download(url, plat.integrity) + mkdirSync(destDir, { recursive: true }) + if (plat.asset.endsWith('.tar.gz') || plat.asset.endsWith('.zip')) { + extractArchive(name, destDir, plat.asset, buf) + // Windows archives ship `.exe`; resolve it before giving up, and + // clean the partial dir so a retry re-extracts instead of wedging. + if (!existsSync(destBin) && existsSync(`${destBin}.exe`)) { + destBin = `${destBin}.exe` + binName = `${binName}.exe` + } + if (!existsSync(destBin)) { + rmSync(destDir, { recursive: true, force: true }) + throw new Error(`${name}: ${binName} not found in extracted archive`) + } + } else { + writeFileSync(destBin, buf) + } + chmodSync(destBin, 0o755) + linkHandle(destBin, binName) + console.log(`[external-tools] installed ${name}@${pin.version} -> ${destBin}`) +} + +/** + * Registry-tarball install: verify against the pinned SRI, extract into the + * rack, materialize runtime deps with npm (pnpm as fallback), and link a + * node wrapper for the package's bin. Tarballs that bundle their + * node_modules (npm does) skip the dependency install. + */ +async function installNpmTarball( + name: string, + pkg: string, + version: string, + integrity: string, +): Promise { + const base = pkg.split('/').pop() + const url = `https://registry.npmjs.org/${pkg}/-/${base}-${version}.tgz` + const destDir = path.join(RACK_DIR, name, version) + const pkgDir = path.join(destDir, 'package') + // Completion marker is the extracted manifest, not the directory: a + // failed/interrupted extract leaves the dir behind and would otherwise + // wedge every later install. Reset and redo instead. + const manifest = path.join(pkgDir, 'package.json') + if (!existsSync(manifest)) { + rmSync(destDir, { recursive: true, force: true }) + console.log(`[external-tools] downloading ${name}@${version} (npm registry)`) + const buf = await download(url, integrity) + mkdirSync(destDir, { recursive: true }) + extractArchive(name, destDir, 'package.tgz', buf) + } + const pkgJson = JSON.parse(readFileSync(manifest, 'utf8')) + const hasDeps = Object.keys(pkgJson.dependencies ?? {}).length > 0 + if (hasDeps && !existsSync(path.join(pkgDir, 'node_modules'))) { + if (installDeps(name, pkgDir) !== 0) { + rmSync(destDir, { recursive: true, force: true }) + throw new Error(`${name}: dependency install failed`) + } + } + const bins = + typeof pkgJson.bin === 'string' ? { [name]: pkgJson.bin } : (pkgJson.bin ?? {}) + const binRel = bins[name] ?? Object.values(bins)[0] + if (binRel) { + const binAbs = path.join(pkgDir, binRel as string) + if (process.platform === 'win32') { + // linkHandle writes node-invoking .cmd + bash forwarders for a + // non-.exe target — no bash-only wrapper to strand under pwsh. + linkHandle(binAbs, name) + } else { + const wrapper = path.join(RACK_DIR, name, `${name}-wrapper`) + writeFileSync(wrapper, `#!/usr/bin/env bash\nexec node '${binAbs}' "$@"\n`) + chmodSync(wrapper, 0o755) + linkHandle(wrapper, name) + } + } + console.log(`[external-tools] installed ${name}@${version}`) +} + +function installDeps(name: string, pkgDir: string): number { + for (const [cmd, ...args] of PM_DEP_INSTALLERS) { + if (cmd!.includes('/') && !existsSync(cmd!)) { + continue + } + console.log(`[external-tools] ${name}: installing deps via ${path.basename(cmd!)}`) + const res = spawnSync(cmd!, args, { cwd: pkgDir, stdio: 'inherit' }) + if (res.error) { + continue + } + return res.status ?? 1 + } + console.error(`[external-tools] ${name}: no package manager available for deps`) + return 1 +} + +export async function installTool(name: string, tools: Record): Promise { + // `sfw` is a flavor pair: the enterprise binary when a Socket token is + // present (repo secret), the keyless free tier otherwise. The firewall + // shim mechanism is POSIX (bash shims, extension-less symlink handles) — + // skip cleanly on Windows rather than install something unusable. + if (name === 'sfw') { + if (process.platform === 'win32') { + console.log('[external-tools] sfw shims are POSIX-only — skipping on windows') + return + } + name = process.env.SOCKET_SECURITY_KEY ? 'sfw-enterprise' : 'sfw-free' + } + const pin = tools[name] + if (!pin) { + throw new Error(`unknown tool ${name} (see external-tools.json)`) + } + if (pin.release === 'asset') { + await installAssetTool(name, pin) + return + } + if (pin.purl) { + // npm-packaged scanner (agentshield): verify the registry tarball + // against the pinned SRI, then run via the extracted package. + const m = /^pkg:npm\/(.+)@([^@]+)$/.exec(pin.purl) + if (!m) { + throw new Error(`${name}: unsupported purl ${pin.purl}`) + } + await installNpmTarball(name, m[1]!, m[2]!, pin.integrity!) + return + } + if (pin.repository?.startsWith('npm:')) { + // Platform-agnostic registry tarball (npm itself ships this way): one + // tarball, one integrity, pure JS run through node. Never installed + // via `npm install -g npm` — no self-update path. + await installNpmTarball(name, pin.repository.slice('npm:'.length), pin.version!, pin.integrity!) + return + } + if (pin.release === 'uv-project') { + // Git-SHA-pinned python project; not auto-installed (needs uv). + const repo = pin.repository!.replace(/^github:/, '') + console.log( + `[external-tools] ${name} is a uv project — run: uvx --from git+https://github.com/${repo}@${pin.version} ${name}`, + ) + return + } + throw new Error(`${name}: no installable shape (release=${pin.release ?? 'none'})`) +} + +/** + * The rack location of a pinned command's runnable entry, or '' when the + * command isn't rack-pinned/installed. Resolved from the manifest + rack + * contents (never from the bin handle: after a --shims run the handle is + * the shim itself, and resolving through it wouldn't survive a re-run). + */ +export function rackRealFor(cmd: string, tools: Record): string { + const pin = tools[cmd] + if (!pin) { + return '' + } + const candidates = [ + path.join(RACK_DIR, cmd, `${cmd}-wrapper`), + ...(pin.version ? [path.join(RACK_DIR, cmd, pin.version, pin.binaryName ?? cmd)] : []), + ] + return candidates.find(c => existsSync(c)) ?? '' +} + +/** + * sfw shims: tiny wrappers named after each package manager that route the + * real invocation through the firewall. A sentinel env var breaks the + * recursion when sfw itself re-invokes the tool; the real binary is found + * by stripping the rack's bin dir out of PATH. + */ +export function writeShims(tools: Record): void { + if (process.platform === 'win32') { + console.log('[external-tools] sfw shims are POSIX-only — skipping on windows') + return + } + mkdirSync(BIN_DIR, { recursive: true }) + for (const cmd of SFW_ECOSYSTEMS) { + const sentinel = `SFW_SHIM_ACTIVE_${cmd.replace(/[^A-Za-z0-9]/g, '_').toUpperCase()}` + // When the command itself is rack-pinned (pnpm/npm), the shim wraps the + // PINNED binary; unpinned commands resolve at run time by stripping the + // shim dir out of PATH. + const handle = path.join(BIN_DIR, cmd) + const rackReal = rackRealFor(cmd, tools) + const resolveReal = rackReal + ? `REAL='${rackReal}'` + : `CLEAN_PATH=$(printf '%s' "$PATH" | tr ':' '\\n' | grep -vFx '${BIN_DIR}' | paste -sd ':' -) +REAL=$(PATH="$CLEAN_PATH" command -v '${cmd}' || true)` + const body = `#!/usr/bin/env bash +# sfw shim for ${cmd} — managed by scripts/soak/external-tools.mts --shims +set -euo pipefail +${resolveReal} +if [ -n "\${${sentinel}:-}" ] || [ -z "$REAL" ] || ! command -v sfw >/dev/null 2>&1; then + [ -n "$REAL" ] && exec "$REAL" "$@" + echo "${cmd}: not found" >&2; exit 127 +fi +export ${sentinel}=1 +exec sfw '${cmd}' "$@" +` + // Remove the handle before writing: writeFileSync FOLLOWS a symlink, so + // writing through a rack handle would overwrite the pinned binary itself + // with the shim body (which then execs itself forever). + rmSync(handle, { force: true }) + writeFileSync(handle, body) + chmodSync(handle, 0o755) + } + console.log(`[external-tools] wrote sfw shims for ${SFW_ECOSYSTEMS.join(', ')} in ${BIN_DIR}`) + console.log(`[external-tools] prepend ${BIN_DIR} to PATH to activate`) +} + +export async function main(argv: string[] = process.argv.slice(2)): Promise { + if (argv.includes('--print-bin')) { + console.log(BIN_DIR) + return 0 + } + const tools = loadTools() + if (argv.includes('--check') || argv.length === 0) { + const problems = checkPins(tools) + if (DOCKER_PREBAKE) { + const dockerAbs = path.join(REPO_ROOT, DOCKER_PREBAKE) + const toolchainAbs = SURFACES.toolchainToml + ? path.join(REPO_ROOT, SURFACES.toolchainToml) + : '' + if (existsSync(dockerAbs)) { + problems.push( + ...checkDockerPrebake( + readFileSync(dockerAbs, 'utf8'), + tools, + existsSync(toolchainAbs) ? readFileSync(toolchainAbs, 'utf8') : '', + /^rust-version\s*=\s*"([^"]+)"/m.exec( + readFileSync(path.join(REPO_ROOT, 'Cargo.toml'), 'utf8'), + )?.[1] ?? '', + ), + ) + } + } + for (const p of problems) { + console.error(`[external-tools] ${p}`) + } + if (problems.length === 0) { + console.log(`[external-tools] ${Object.keys(tools).length} pins valid`) + } + return problems.length === 0 ? 0 : 1 + } + for (let i = 0; i < argv.length; i++) { + if (argv[i] === '--install') { + await installTool(argv[++i]!, tools) + } + } + if (argv.includes('--install-all')) { + for (const name of Object.keys(tools)) { + if (name === 'sfw-enterprise' || name === 'sfw-free') { + continue + } + await installTool(name, tools) + } + await installTool('sfw', tools) + } + if (argv.includes('--shims')) { + writeShims(tools) + } + return 0 +} + +// realpath + pathToFileURL so symlinked checkouts and paths needing URL +// encoding still register as the entrypoint (ESM realpaths import.meta.url). +const isMain = + process.argv[1] && pathToFileURL(realpathSync(process.argv[1])).href === import.meta.url +if (isMain) { + main().then( + code => { + process.exitCode = code + }, + err => { + console.error(`[external-tools] ${err.message}`) + process.exitCode = 1 + }, + ) +} diff --git a/scripts/soak/external-tools.test.mts b/scripts/soak/external-tools.test.mts new file mode 100644 index 0000000000..ec0eecc0b8 --- /dev/null +++ b/scripts/soak/external-tools.test.mts @@ -0,0 +1,219 @@ +import assert from 'node:assert/strict' +import { spawnSync } from 'node:child_process' +import { createHash } from 'node:crypto' +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import os from 'node:os' +import path from 'node:path' +import { test } from 'node:test' +import { fileURLToPath } from 'node:url' + +import { SOAK_DAYS, addDaysIso, todayIso } from './constants.mts' +import { + checkDockerPrebake, + checkPins, + download, + extractArchive, + installTool, + main, +} from './external-tools.mts' +import { DOCKER_PREBAKE, EXTERNAL_TOOLS_JSON, REPO_ROOT, SURFACES } from './paths.mts' + +const GOOD_SRI = + 'sha512-waLrsPG2a7EOv0XuvXDQZGgCZ4MTtOfZh8TmGbM6gn2B6Nh6HI+15jaoKdAS9wgdTyIqTuqU+O+NtVYd+kuFaA==' + +test('the repo external-tools.json passes checkPins', () => { + const tools = JSON.parse(readFileSync(EXTERNAL_TOOLS_JSON, 'utf8')).tools + assert.deepEqual(checkPins(tools), []) +}) + +test('checkPins flags missing pins, bad SRIs, and asset entries with no integrity', () => { + assert.equal(checkPins({ a: {} }).length, 1) + assert.equal(checkPins({ a: { version: '1.0.0', integrity: 'sha256-abc' } }).length, 1) + assert.equal(checkPins({ a: { version: '1.0.0', release: 'asset' } }).length, 1) +}) + +test('checkPins validates soakBypass dates, arithmetic, and expiry', () => { + const pub = addDaysIso(todayIso(), -1) + const good = { + a: { + version: '1.0.0', + integrity: GOOD_SRI, + soakBypass: { version: '1.0.0', published: pub, removable: addDaysIso(pub, SOAK_DAYS) }, + }, + } + assert.deepEqual(checkPins(good), []) + const wrongMath = structuredClone(good) + wrongMath.a.soakBypass.removable = addDaysIso(pub, 3) + assert.match(checkPins(wrongMath)[0]!, /removable/) + const expired = structuredClone(good) + expired.a.soakBypass = { version: '1.0.0', published: '2020-01-01', removable: '2020-01-08' } + assert.match(checkPins(expired)[0]!, /expired/) + const impossible = structuredClone(good) + impossible.a.soakBypass = { version: '1.0.0', published: '2026-13-45', removable: '2026-13-52' } + assert.match(checkPins(impossible)[0]!, /calendar/) +}) + +test('the repo Dockerfile prebake (when present) matches the tracked pins', t => { + if (!DOCKER_PREBAKE || !existsSync(path.join(REPO_ROOT, DOCKER_PREBAKE))) { + t.skip('repo has no prebake image') + return + } + const tools = JSON.parse(readFileSync(EXTERNAL_TOOLS_JSON, 'utf8')).tools + const docker = readFileSync(path.join(REPO_ROOT, DOCKER_PREBAKE), 'utf8') + const toolchain = SURFACES.toolchainToml + ? readFileSync(path.join(REPO_ROOT, SURFACES.toolchainToml), 'utf8') + : '' + assert.deepEqual(checkDockerPrebake(docker, tools, toolchain), []) + // and drift in any direction is caught + assert.ok(checkDockerPrebake(docker.replace(/sha=[0-9a-f]{8}/, 'sha=deadbeef'), tools, toolchain).length > 0) + assert.ok( + checkDockerPrebake(docker, tools, toolchain.replace(/channel = ".*"/, 'channel = "nightly-1999-01-01"')).length > + 0, + ) +}) + +const sriOf = (buf: Buffer) => `sha512-${createHash('sha512').update(buf).digest('base64')}` + +function withEnv(name: string, value: string | undefined, fn: () => Promise): Promise { + const saved = process.env[name] + if (value === undefined) { + delete process.env[name] + } else { + process.env[name] = value + } + return fn().finally(() => { + if (saved === undefined) { + delete process.env[name] + } else { + process.env[name] = saved + } + }) +} + +// Hermetic fixture: one line per drift class, so every checkDockerPrebake +// failure branch is exercised even in a repo with no prebake image. +test('checkDockerPrebake flags every drift class (synthetic image)', () => { + const tools = { + 'sfw-free': { + version: '1.0.0', + platforms: { 'linux-arm64': { asset: 'sfw-linux-arm64', integrity: GOOD_SRI } }, + }, + } + const wrongHex = 'ab'.repeat(64) + const body = [ + 'for cmd in npm yarn; do make_shim "$cmd"; done', + 'RUN curl -o /x https://github.com/SocketDev/sfw-free/releases/download/v0.9.9/sfw-linux-arm64', + 'COPY rack/sfw-free/0.9.9/sfw /usr/local/bin/sfw', + `RUN asset=ghost-asset; sha=${wrongHex} verify`, + `RUN asset=sfw-linux-arm64; sha=${wrongHex} verify`, + ].join('\n') + const problems = checkDockerPrebake(body, tools, 'channel = "nightly-2026-07-04"\n', '9.9.9') + for (const needle of [ + /shim list/, + /msrv toolchain/, + /version 0\.9\.9/, + /download url v0\.9\.9/, + /ghost-asset has no pin/, + /sha for sfw-linux-arm64/, + /pinned toolchain nightly-2026-07-04/, + ]) { + assert.ok(problems.some(p => needle.test(p)), `expected a finding matching ${needle}`) + } + assert.ok( + checkDockerPrebake('nothing pinned here', tools, '').some(p => + /no asset\/sha pin pairs/.test(p), + ), + ) +}) + +test('download sends the GitHub token to github.com only', async t => { + const payload = Buffer.from('pinned-bytes') + const seen: Array<{ host: string; auth: string | undefined }> = [] + t.mock.method(globalThis, 'fetch', async (url: string | URL, init?: RequestInit) => { + seen.push({ + host: new URL(String(url)).hostname, + auth: (init?.headers as Record | undefined)?.authorization, + }) + return new Response(payload) + }) + await withEnv('GITHUB_TOKEN', 'ghs_test_token', async () => { + await download('https://github.com/o/r/releases/download/v1/a.tgz', sriOf(payload)) + await download('https://registry.npmjs.org/x/-/x-1.0.0.tgz', sriOf(payload)) + }) + assert.equal(seen[0]!.auth, 'Bearer ghs_test_token') + assert.equal(seen[1]!.auth, undefined) +}) + +test('download rejects http errors and integrity mismatches', async t => { + const payload = Buffer.from('served-bytes') + let status = 503 + t.mock.method(globalThis, 'fetch', async () => new Response(payload, { status })) + await assert.rejects(download('https://example.com/a', sriOf(payload)), /download failed 503/) + status = 200 + await assert.rejects(download('https://example.com/a', 'sha512-AAAA'), /integrity mismatch/) + assert.deepEqual(await download('https://example.com/a', sriOf(payload)), payload) +}) + +test('extractArchive unpacks a tar.gz, removes the archive, throws on junk', t => { + const dir = mkdtempSync(path.join(os.tmpdir(), 'soak-extract-')) + t.after(() => rmSync(dir, { recursive: true, force: true })) + writeFileSync(path.join(dir, 'hello.txt'), 'hi\n') + spawnSync('tar', ['-czf', path.join(dir, 'a.tar.gz'), '-C', dir, 'hello.txt']) + const buf = readFileSync(path.join(dir, 'a.tar.gz')) + const dest = mkdtempSync(path.join(os.tmpdir(), 'soak-extract-dest-')) + t.after(() => rmSync(dest, { recursive: true, force: true })) + extractArchive('t', dest, 'a.tar.gz', buf) + assert.ok(existsSync(path.join(dest, 'hello.txt'))) + assert.ok(!existsSync(path.join(dest, 'a.tar.gz'))) + assert.throws(() => extractArchive('t', dest, 'bad.tgz', Buffer.from('junk')), /extract failed/) +}) + +test('installTool rejects unknown tools, foreign purls, and shapeless pins', async () => { + await assert.rejects(installTool('ghost', {}), /unknown tool/) + await assert.rejects( + installTool('x', { x: { purl: 'pkg:pypi/foo@1.0.0', integrity: GOOD_SRI } }), + /unsupported purl/, + ) + await assert.rejects(installTool('y', { y: { version: '1.0.0' } }), /no installable shape/) + await assert.rejects( + installTool('z', { z: { release: 'asset', version: '1.0.0', platforms: {} } }), + /no pinned asset for/, + ) +}) + +test('installTool resolves the sfw flavor from SOCKET_SECURITY_KEY', async t => { + if (process.platform === 'win32') { + t.skip('sfw shims are POSIX-only') + return + } + // An empty tools record makes the resolved flavor observable in the + // rejection message — no download is ever attempted. + await withEnv('SOCKET_SECURITY_KEY', undefined, () => + assert.rejects(installTool('sfw', {}), /unknown tool sfw-free/), + ) + await withEnv('SOCKET_SECURITY_KEY', 'sk_test', () => + assert.rejects(installTool('sfw', {}), /unknown tool sfw-enterprise/), + ) +}) + +test('installTool prints the uvx line for uv-project pins without installing', async () => { + const tools = JSON.parse(readFileSync(EXTERNAL_TOOLS_JSON, 'utf8')).tools + const uv = Object.keys(tools).find(name => tools[name].release === 'uv-project') + assert.ok(uv, 'the manifest is expected to pin at least one uv project') + await installTool(uv!, tools) +}) + +// Glue: main's read-only CLI paths against the tracked manifest — the same +// gate CI runs, exercised in-process so main() itself stays covered. +test('main --print-bin and --check are read-only and exit 0', async () => { + assert.equal(await main(['--print-bin']), 0) + assert.equal(await main(['--check']), 0) +}) + +// End to end through the entrypoint guard: the CLI must resolve as main +// (realpath + file URL) and exit 0 on the tracked manifest. +test('CLI: node external-tools.mts --check exits 0', () => { + const script = fileURLToPath(new URL('./external-tools.mts', import.meta.url)) + const res = spawnSync(process.execPath, [script, '--check'], { encoding: 'utf8' }) + assert.equal(res.status, 0, res.stderr) +}) diff --git a/scripts/soak/paths.mts b/scripts/soak/paths.mts new file mode 100644 index 0000000000..3b1b5955d0 --- /dev/null +++ b/scripts/soak/paths.mts @@ -0,0 +1,74 @@ +/** + * @file 1 path, 1 reference — every filesystem location the soak + + * external-tools scripts touch is declared here exactly once. Scripts + * import from this module instead of re-deriving paths, so a surface can + * move (or differ between repos carrying these scripts) with a one-line + * change. Ported from nub (nubjs/nub#442); this file is the per-repo seam. + */ + +import os from 'node:os' +import path from 'node:path' +import process from 'node:process' +import { fileURLToPath } from 'node:url' + +export const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..') + +// Soak surfaces (repo-relative). The repo ROOT is npm-only (.npmrc + +// package-lock.json hold the parity-test fixture deps that local runs +// `npm install`); the pnpm-side soak (workspace yaml with catalog + +// minimumReleaseAge, taze) is anchored in tools/ so the workspace yaml +// never marks the npm-managed repo root as a pnpm workspace. +// +// toolchainToml is null: perry rides stable rust (CI installs +// dtolnay/rust-toolchain@stable; there is no rust-toolchain.toml). The +// dated-nightly soak check activates automatically if the repo ever pins +// one — set the path here and the surface joins the gate. +export const SURFACES: { + cargoConfig: string + npmrc: string + workspaceYaml: string + tazeConfig: string + toolchainToml: string | null + dependabotYml: string +} = { + cargoConfig: '.cargo/config.toml', + npmrc: '.npmrc', + workspaceYaml: 'tools/pnpm-workspace.yaml', + tazeConfig: 'tools/taze.config.mts', + toolchainToml: null, + dependabotYml: '.github/dependabot.yml', +} + +// The directory holding the npm package the soak governs (taze runs here, +// pnpm refreshes this package's lockfile). +export const NPM_PKG_DIR = path.join(REPO_ROOT, 'tools') + +// Lockfile refreshers tried in order after taze rewrites package.json. +export const NPM_INSTALLERS: string[][] = [['pnpm', 'install']] + +// rustup's cargo shim — the only cargo that reads a rust-toolchain.toml and +// therefore the only one whose `cargo update` would honor the [unstable] +// min-publish-age soak (nightly-only; inert on perry's stable toolchain, +// where the automated window rides dependabot's cooldown instead). +export const RUSTUP_CARGO = path.join(os.homedir(), '.cargo/bin/cargo') + +// Pinned external tool manifest + the local tool rack it installs into: +// exact versions under rack///, flat PATH handles in bin/. +export const EXTERNAL_TOOLS_JSON = path.join(REPO_ROOT, 'external-tools.json') + +// CI agent image that pre-bakes the pinned toolchain + sfw (null when the +// repo has no such image — perry's Dockerfiles are product/dev images that +// build perry itself, not CI agent tooling). +export const DOCKER_PREBAKE: string | null = null + +const XDG_DATA_HOME = process.env.XDG_DATA_HOME || path.join(os.homedir(), '.local/share') +export const DEV_TOOLS_DIR = path.join(XDG_DATA_HOME, 'perry/dev-tools') +export const RACK_DIR = path.join(DEV_TOOLS_DIR, 'rack') +export const BIN_DIR = path.join(DEV_TOOLS_DIR, 'bin') + +// Candidates (tried in order) for installing an extracted external tool's +// runtime deps — npm first (the repo root's own package manager). +export const PM_DEP_INSTALLERS: string[][] = [ + ['npm', 'install', '--omit=dev', '--ignore-scripts', '--no-audit', '--no-fund'], + ['pnpm', 'install', '--prod', '--ignore-scripts'], +] diff --git a/scripts/soak/soak.mts b/scripts/soak/soak.mts new file mode 100644 index 0000000000..a619178cb4 --- /dev/null +++ b/scripts/soak/soak.mts @@ -0,0 +1,516 @@ +#!/usr/bin/env node +/** + * @file Soak manager — parity gate + fixer for the release-age cooldown. + * The window is ONE value (`SOAK_DAYS` in ./constants.mts); this script + * asserts every data surface matches it and that soak exclusions carry a + * valid, unexpired `# published: | removable:` annotation: + * + * - `.cargo/config.toml` `global-min-publish-age` + `[unstable] min-publish-age` + * - `tools/pnpm-workspace.yaml` `minimumReleaseAge` (minutes) + annotated excludes + * - `.npmrc` `min-release-age` (days) + * - `tools/taze.config.mts` imports SOAK_DAYS (existence + import check) + * - `.github/dependabot.yml` `cooldown: default-days` per update block + * - `rust-toolchain.toml` nightly channel vs `# adopted:` — only when + * the repo pins one (SURFACES.toolchainToml; perry rides stable, so null) + * + * `--check` (default) fails loud with What / Saw / Wanted / Fix on drift. + * `--fix` rewrites window values in place and prunes excludes whose + * `removable` date has passed (a cleared pin is dead weight — pruning it + * re-arms the soak for the next publish of that package). + * + * Usage: node scripts/soak/soak.mts [--check|--fix] [--quiet] + */ + +import { existsSync, readFileSync, realpathSync, writeFileSync } from 'node:fs' +import path from 'node:path' +import process from 'node:process' +import { pathToFileURL } from 'node:url' + +import { + ANNOTATION_RE, + SOAK_DAYS, + SOAK_MINUTES, + VERSION_PIN_RE, + addDaysIso, + isValidIsoDate, + todayIso, +} from './constants.mts' +import { REPO_ROOT, SURFACES } from './paths.mts' + +export interface Finding { + file: string + what: string + saw: string + wanted: string + fix: string +} + +export function checkCargoConfig(body: string, file: string): Finding[] { + const out: Finding[] = [] + const age = /^global-min-publish-age\s*=\s*"([^"]*)"/m.exec(body)?.[1] + const wanted = `${SOAK_DAYS} days` + if (age !== wanted) { + out.push({ + file, + what: 'cargo min-publish-age window', + saw: age ?? '(missing)', + wanted, + fix: `set [registry] global-min-publish-age = "${wanted}" (or run --fix)`, + }) + } + if (!/^\[unstable\][^[]*^min-publish-age\s*=\s*true/ms.test(body)) { + out.push({ + file, + what: 'cargo unstable feature gate', + saw: '[unstable] min-publish-age missing or false', + wanted: 'min-publish-age = true under [unstable]', + fix: 'add `[unstable]\\nmin-publish-age = true` (nightly-only; the pinned toolchain provides it)', + }) + } + return out +} + +export function checkNpmrc(body: string, file: string): Finding[] { + const days = /^min-release-age=(\d+)\s*$/m.exec(body)?.[1] + if (Number(days) === SOAK_DAYS) { + return [] + } + return [ + { + file, + what: 'npm min-release-age window', + saw: days ?? '(missing)', + wanted: String(SOAK_DAYS), + fix: `set min-release-age=${SOAK_DAYS} (or run --fix)`, + }, + ] +} + +export function checkWorkspaceYaml(body: string, file: string): Finding[] { + const out: Finding[] = [] + const minutes = /^minimumReleaseAge:\s*(\d+)\s*$/m.exec(body)?.[1] + if (Number(minutes) !== SOAK_MINUTES) { + out.push({ + file, + what: 'minimumReleaseAge window', + saw: minutes ?? '(missing)', + wanted: `${SOAK_MINUTES} (SOAK_DAYS ${SOAK_DAYS} x 1440 minutes)`, + fix: `set minimumReleaseAge: ${SOAK_MINUTES} (or run --fix)`, + }) + } + out.push(...checkExcludeAnnotations(body, file)) + return out +} + +/** + * Every version-pinned `minimumReleaseAgeExclude` entry must carry, on the + * line directly above, `# published: YYYY-MM-DD | removable: YYYY-MM-DD` + * with `removable = published + SOAK_DAYS`, and must be pruned once + * `removable` is strictly in the past. Bare names and `@scope/*` globs are + * standing trust, not dated bypasses — no annotation required. + */ +export function checkExcludeAnnotations(body: string, file: string): Finding[] { + const out: Finding[] = [] + const today = todayIso() + // Flow style would be invisible to the block parser below — an + // unvalidated, never-expiring bypass. One canonical shape only. + if (/^minimumReleaseAgeExclude:\s*\[/m.test(body)) { + out.push({ + file, + what: 'minimumReleaseAgeExclude flow style', + saw: 'inline [...] list', + wanted: 'a block list (one annotated `- entry` per line)', + fix: 'rewrite as a block list so every pin can carry its annotation', + }) + return out + } + for (const entry of parseExcludeEntries(body)) { + if (!VERSION_PIN_RE.test(entry.name)) { + continue + } + if (!entry.annotation) { + out.push({ + file, + what: `soak exclude '${entry.name}' annotation`, + saw: '(no annotation on the line above)', + wanted: `# published: YYYY-MM-DD | removable: `, + fix: `annotate the pin with its real registry publish date`, + }) + continue + } + const { published, removable } = entry.annotation + if (!isValidIsoDate(published) || !isValidIsoDate(removable)) { + out.push({ + file, + what: `soak exclude '${entry.name}' annotation dates`, + saw: `${published} | ${removable}`, + wanted: 'real YYYY-MM-DD calendar dates', + fix: 'correct the annotation to the real registry publish date', + }) + continue + } + const expected = addDaysIso(published, SOAK_DAYS) + if (removable !== expected) { + out.push({ + file, + what: `soak exclude '${entry.name}' removable date`, + saw: removable, + wanted: `${expected} (published ${published} + ${SOAK_DAYS} days)`, + fix: 'correct the removable date', + }) + } + if (removable < today) { + out.push({ + file, + what: `soak exclude '${entry.name}' expired`, + saw: `removable ${removable} < today ${today}`, + wanted: 'entry pruned once its window has passed', + fix: 'delete the pin + its annotation (or run --fix)', + }) + } + } + return out +} + +interface ExcludeEntry { + name: string + line: number + annotation?: { published: string; removable: string } +} + +export function parseExcludeEntries(body: string): ExcludeEntry[] { + const lines = body.split('\n') + const out: ExcludeEntry[] = [] + let inBlock = false + let blockIndent = 0 + for (let i = 0; i < lines.length; i++) { + const line = lines[i]! + if (/^minimumReleaseAgeExclude:\s*$/.test(line)) { + inBlock = true + blockIndent = -1 + continue + } + if (!inBlock) { + continue + } + const item = /^(\s+)-\s*['"]?([^'"#\s]+)['"]?\s*(?:#.*)?$/.exec(line) + if (!item) { + // Comments stay inside the block; anything else at column 0 ends it. + if (/^\S/.test(line)) { + inBlock = false + } + continue + } + if (blockIndent === -1) { + blockIndent = item[1]!.length + } + if (item[1]!.length !== blockIndent) { + continue + } + const prev = lines[i - 1]?.trim() ?? '' + const ann = ANNOTATION_RE.exec(prev) + out.push({ + name: item[2]!, + line: i + 1, + ...(ann ? { annotation: { published: ann[1]!, removable: ann[2]! } } : {}), + }) + } + return out +} + +/** + * Catalog-shadowed pins stay in lockstep: when a package.json next to the + * workspace yaml pins a cataloged package to an exact version instead of + * `catalog:` (npm-cli compat — npm can't parse the protocol), the two + * versions must match. `catalog:` references no-op here. + */ +export function checkCatalogParity( + yamlBody: string, + pkgJson: string, + yamlFile: string, +): Finding[] { + const out: Finding[] = [] + const catalog: Record = {} + const block = /^catalog:\s*\n((?:[ \t]+\S.*\n?|\s*\n)*)/m.exec(yamlBody)?.[1] ?? '' + for (const m of block.matchAll(/^[ \t]+['"]?([^'":\s]+)['"]?:\s*['"]?([^'"\s]+)['"]?\s*$/gm)) { + catalog[m[1]!] = m[2]! + } + const pkg = JSON.parse(pkgJson) + const declared: Record = { + ...pkg.dependencies, + ...pkg.devDependencies, + } + for (const [name, version] of Object.entries(catalog)) { + const spec = declared[name] + if (spec === undefined || spec === 'catalog:' || spec === version) { + continue + } + out.push({ + file: yamlFile, + what: `catalog-shadowed pin '${name}' out of lockstep`, + saw: `catalog ${version} vs package.json ${spec}`, + wanted: 'identical versions (or a catalog: reference)', + fix: `bump both together — the catalog entry is the reference`, + }) + } + return out +} + +/** + * The toolchain pin obeys the same soak: a dated nightly must have been at + * least SOAK_DAYS old on its recorded adoption date (`# adopted:` line, + * machine-read here). Stable channel pins carry no date and pass freely. + */ +export function checkToolchainSoak(body: string, file: string): Finding[] { + const channelDate = /^channel\s*=\s*"nightly-(\d{4}-\d{2}-\d{2})"/m.exec(body)?.[1] + if (!channelDate) { + return [] + } + const adopted = /^#\s*adopted:\s*(\d{4}-\d{2}-\d{2})\s*$/m.exec(body)?.[1] + if (!adopted) { + return [ + { + file, + what: 'toolchain adoption date', + saw: '(no `# adopted: YYYY-MM-DD` line)', + wanted: 'a recorded adoption date so the nightly soak is checkable', + fix: 'add `# adopted: ` above [toolchain]', + }, + ] + } + if (!isValidIsoDate(channelDate) || !isValidIsoDate(adopted)) { + return [ + { + file, + what: 'toolchain soak dates', + saw: `${channelDate} | ${adopted}`, + wanted: 'real YYYY-MM-DD calendar dates', + fix: 'correct the nightly channel / adopted dates', + }, + ] + } + if (addDaysIso(channelDate, SOAK_DAYS) > adopted) { + return [ + { + file, + what: 'toolchain nightly soak', + saw: `nightly-${channelDate} adopted ${adopted}`, + wanted: `a nightly at least ${SOAK_DAYS} days old at adoption`, + fix: 'pin the newest nightly that had cleared the window on the adoption date', + }, + ] + } + return [] +} + +export function checkTazeConfig(body: string, file: string): Finding[] { + const out: Finding[] = [] + if (!body.includes('maturityPeriod')) { + out.push({ + file, + what: 'taze maturityPeriod', + saw: '(not set)', + wanted: 'maturityPeriod: SOAK_DAYS', + fix: 'set maturityPeriod: SOAK_DAYS in the taze config', + }) + } + if (!body.includes('constants.mts')) { + out.push({ + file, + what: 'taze config soak import', + saw: 'window not imported from scripts/soak/constants.mts', + wanted: "import { SOAK_DAYS } from '/scripts/soak/constants.mts'", + fix: 'import SOAK_DAYS instead of hand-copying the number', + }) + } + return out +} + +/** + * Dependabot must carry the window EXPLICITLY in every update block. + * Dependabot bumps manifests + lockfiles server-side, and cargo's + * min-publish-age skips already-locked versions — so a dependabot PR is + * the one dependency path none of the local soak surfaces can stop. Its + * `cooldown.default-days` is the equivalent of renovate's + * minimumReleaseAge; each `- package-ecosystem:` block needs its own. + * Parsed line-based (no YAML dep): a block runs from its + * `- package-ecosystem:` line to the next one. + */ +export function checkDependabotCooldown(body: string, file: string): Finding[] { + if (SOAK_DAYS === 0) { + return [] + } + const out: Finding[] = [] + for (const block of parseDependabotBlocks(body)) { + const days = /^\s+default-days:\s*(\d+)\s*$/m.exec(block.body)?.[1] + const hasCooldown = /^\s+cooldown:\s*$/m.test(block.body) + if (hasCooldown && Number(days) === SOAK_DAYS) { + continue + } + out.push({ + file, + what: `dependabot cooldown for ${block.ecosystem}`, + saw: hasCooldown ? `default-days: ${days ?? '(missing)'}` : '(no cooldown block)', + wanted: `cooldown.default-days: ${SOAK_DAYS}`, + fix: `add \`cooldown:\\n default-days: ${SOAK_DAYS}\` to the ${block.ecosystem} block (value drift: run --fix)`, + }) + } + return out +} + +interface DependabotBlock { + ecosystem: string + body: string +} + +export function parseDependabotBlocks(body: string): DependabotBlock[] { + const out: DependabotBlock[] = [] + const lines = body.split('\n') + let current: { ecosystem: string; start: number } | null = null + const flush = (end: number) => { + if (current) { + out.push({ + ecosystem: current.ecosystem, + body: lines.slice(current.start, end).join('\n'), + }) + } + } + for (let i = 0; i < lines.length; i++) { + const m = /^\s+-\s+package-ecosystem:\s*['"]?([^'"\s]+)['"]?\s*$/.exec(lines[i]!) + if (m) { + flush(i) + current = { ecosystem: m[1]!, start: i } + } + } + flush(lines.length) + return out +} + +// Value-drift fixer only: rewrites an existing `default-days:` to the +// window. A MISSING cooldown block stays a --check finding (line-based +// YAML insertion is riskier than telling a human where the two lines go). +export function fixDependabotCooldown(body: string): string { + if (SOAK_DAYS === 0) { + return body + } + return body.replace(/^(\s+default-days:\s*)\d+\s*$/gm, `$1${SOAK_DAYS}`) +} + +export function fixCargoConfig(body: string): string { + return body.replace( + /^(global-min-publish-age\s*=\s*)"[^"]*"/m, + `$1"${SOAK_DAYS} days"`, + ) +} + +export function fixNpmrc(body: string): string { + if (/^min-release-age=\d+\s*$/m.test(body)) { + return body.replace(/^min-release-age=\d+\s*$/m, `min-release-age=${SOAK_DAYS}`) + } + return `${body.trimEnd()}\nmin-release-age=${SOAK_DAYS}\n` +} + +export function fixWorkspaceYaml(body: string): string { + let out = body.replace( + /^(minimumReleaseAge:\s*)\d+\s*$/m, + `$1${SOAK_MINUTES}`, + ) + // Prune expired pins together with their annotation line. + const today = todayIso() + const lines = out.split('\n') + const drop = new Set() + for (const entry of parseExcludeEntries(out)) { + if (entry.annotation && entry.annotation.removable < today) { + drop.add(entry.line - 1) + if (ANNOTATION_RE.test(lines[entry.line - 2]?.trim() ?? '')) { + drop.add(entry.line - 2) + } + } + } + if (drop.size > 0) { + out = lines.filter((_, i) => !drop.has(i)).join('\n') + } + return out +} + +function report(findings: Finding[], quiet: boolean): void { + for (const f of findings) { + console.error(`[soak] ${f.file}: ${f.what}`) + console.error(` saw: ${f.saw}`) + console.error(` wanted: ${f.wanted}`) + console.error(` fix: ${f.fix}`) + } + if (!quiet && findings.length === 0) { + console.log(`[soak] all surfaces match SOAK_DAYS=${SOAK_DAYS} and no exclude has drifted`) + } +} + +export function main(argv: string[] = process.argv.slice(2)): number { + const fix = argv.includes('--fix') + const quiet = argv.includes('--quiet') + const findings: Finding[] = [] + + const surfaces: Array<{ + rel: string + check: (body: string, file: string) => Finding[] + fixer?: (body: string) => string + }> = [ + { rel: SURFACES.cargoConfig, check: checkCargoConfig, fixer: fixCargoConfig }, + { rel: SURFACES.npmrc, check: checkNpmrc, fixer: fixNpmrc }, + { rel: SURFACES.workspaceYaml, check: checkWorkspaceYaml, fixer: fixWorkspaceYaml }, + { rel: SURFACES.tazeConfig, check: checkTazeConfig }, + // Only when the repo pins a toolchain file (perry rides stable — null). + ...(SURFACES.toolchainToml + ? [{ rel: SURFACES.toolchainToml, check: checkToolchainSoak }] + : []), + { rel: SURFACES.dependabotYml, check: checkDependabotCooldown, fixer: fixDependabotCooldown }, + ] + + for (const s of surfaces) { + const abs = path.join(REPO_ROOT, s.rel) + if (!existsSync(abs)) { + findings.push({ + file: s.rel, + what: 'soak surface missing', + saw: '(file absent)', + wanted: 'file present and carrying the soak window', + fix: `create ${s.rel} — see scripts/soak/constants.mts header for the expected key`, + }) + continue + } + let body = readFileSync(abs, 'utf8') + if (fix && s.fixer) { + const fixed = s.fixer(body) + if (fixed !== body) { + writeFileSync(abs, fixed) + console.log(`[soak] fixed ${s.rel}`) + body = fixed + } + } + findings.push(...s.check(body, s.rel)) + } + + // Catalog <-> package.json lockstep for the package next to the yaml. + const yamlAbs = path.join(REPO_ROOT, SURFACES.workspaceYaml) + const pkgAbs = path.join(path.dirname(yamlAbs), 'package.json') + if (existsSync(yamlAbs) && existsSync(pkgAbs)) { + findings.push( + ...checkCatalogParity( + readFileSync(yamlAbs, 'utf8'), + readFileSync(pkgAbs, 'utf8'), + SURFACES.workspaceYaml, + ), + ) + } + + report(findings, quiet) + return findings.length === 0 ? 0 : 1 +} + +// realpath + pathToFileURL so symlinked checkouts and paths needing URL +// encoding still register as the entrypoint (ESM realpaths import.meta.url). +const isMain = + process.argv[1] && pathToFileURL(realpathSync(process.argv[1])).href === import.meta.url +if (isMain) { + process.exitCode = main() +} diff --git a/scripts/soak/soak.test.mts b/scripts/soak/soak.test.mts new file mode 100644 index 0000000000..692d77958c --- /dev/null +++ b/scripts/soak/soak.test.mts @@ -0,0 +1,215 @@ +import assert from 'node:assert/strict' +import { spawnSync } from 'node:child_process' +import { test } from 'node:test' +import { fileURLToPath } from 'node:url' + +import { SOAK_DAYS, addDaysIso, todayIso } from './constants.mts' +import { + checkCargoConfig, + checkCatalogParity, + checkDependabotCooldown, + checkExcludeAnnotations, + checkNpmrc, + checkTazeConfig, + checkToolchainSoak, + checkWorkspaceYaml, + fixCargoConfig, + fixDependabotCooldown, + fixNpmrc, + fixWorkspaceYaml, + main, + parseDependabotBlocks, + parseExcludeEntries, +} from './soak.mts' + +// A pin published yesterday is inside its window; one published long ago +// has expired. Built relative to today so the tests never go stale. +const FRESH_PUB = addDaysIso(todayIso(), -1) +const FRESH_REM = addDaysIso(FRESH_PUB, SOAK_DAYS) + +const CLEAN_YAML = `catalog: + taze: 19.14.1 +minimumReleaseAge: 10080 +minimumReleaseAgeExclude: + # published: ${FRESH_PUB} | removable: ${FRESH_REM} + - 'left-pad@1.3.0' + - '@myorg/*' + - react +` + +test('cargo config: wrong window and missing unstable gate are findings', () => { + const good = '[unstable]\nmin-publish-age = true\n\n[registry]\nglobal-min-publish-age = "7 days"\n' + assert.equal(checkCargoConfig(good, 'c').length, 0) + assert.equal(checkCargoConfig(good.replace('7 days', '3 days'), 'c').length, 1) + assert.equal(checkCargoConfig('[registry]\nglobal-min-publish-age = "7 days"\n', 'c').length, 1) +}) + +test('npmrc: window must match SOAK_DAYS and fix writes it', () => { + assert.equal(checkNpmrc('min-release-age=7\n', 'n').length, 0) + assert.equal(checkNpmrc('min-release-age=3\n', 'n').length, 1) + assert.equal(checkNpmrc('# nothing\n', 'n').length, 1) + assert.match(fixNpmrc('# nothing\n'), /min-release-age=7/) + assert.match(fixNpmrc('min-release-age=3\n'), /min-release-age=7/) +}) + +test('workspace yaml: clean fixture passes', () => { + assert.deepEqual(checkWorkspaceYaml(CLEAN_YAML, 'y'), []) +}) + +test('workspace yaml: wrong minutes value is a finding', () => { + const bad = CLEAN_YAML.replace('10080', '1440') + assert.equal(checkWorkspaceYaml(bad, 'y').filter(f => f.what.includes('minimumReleaseAge')).length, 1) +}) + +test('excludes: flow-style list is rejected outright', () => { + const flow = "minimumReleaseAge: 10080\nminimumReleaseAgeExclude: ['left-pad@1.3.0']\n" + const findings = checkExcludeAnnotations(flow, 'y') + assert.equal(findings.length, 1) + assert.match(findings[0]!.what, /flow style/) +}) + +test('excludes: unannotated version pin is a finding, bare/glob are not', () => { + const yaml = 'minimumReleaseAgeExclude:\n - lodash@4.17.21\n - react\n - "@myorg/*"\n' + const findings = checkExcludeAnnotations(yaml, 'y') + assert.equal(findings.length, 1) + assert.match(findings[0]!.what, /lodash@4\.17\.21/) +}) + +test('excludes: wrong removable date and expiry are findings', () => { + const wrong = `minimumReleaseAgeExclude:\n # published: ${FRESH_PUB} | removable: ${addDaysIso(FRESH_PUB, 3)}\n - 'a@1.0.0'\n` + assert.match(checkExcludeAnnotations(wrong, 'y')[0]!.what, /removable date/) + const expired = `minimumReleaseAgeExclude:\n # published: 2020-01-01 | removable: 2020-01-08\n - 'b@1.0.0'\n` + assert.match(checkExcludeAnnotations(expired, 'y')[0]!.what, /expired/) +}) + +test('excludes: impossible calendar dates are findings, not crashes', () => { + const bad = `minimumReleaseAgeExclude:\n # published: 2026-13-45 | removable: 2026-13-52\n - 'c@1.0.0'\n` + const findings = checkExcludeAnnotations(bad, 'y') + assert.equal(findings.length, 1) + assert.match(findings[0]!.what, /annotation dates/) +}) + +test('excludes: entries with trailing comments still parse', () => { + const yaml = `minimumReleaseAgeExclude:\n # published: ${FRESH_PUB} | removable: ${FRESH_REM}\n - 'd@2.0.0' # temp\n` + assert.deepEqual(parseExcludeEntries(yaml).map(e => e.name), ['d@2.0.0']) + assert.equal(checkExcludeAnnotations(yaml, 'y').length, 0) +}) + +test('fix prunes expired pins together with their annotations', () => { + const yaml = `minimumReleaseAge: 10080\nminimumReleaseAgeExclude:\n # published: 2020-01-01 | removable: 2020-01-08\n - 'old@1.0.0'\n # published: ${FRESH_PUB} | removable: ${FRESH_REM}\n - 'fresh@1.0.0'\n` + const fixed = fixWorkspaceYaml(yaml) + assert.ok(!fixed.includes('old@1.0.0')) + assert.ok(!fixed.includes('2020-01-01')) + assert.ok(fixed.includes('fresh@1.0.0')) +}) + +test('catalog parity: exact pin must match, catalog: protocol no-ops', () => { + const yaml = 'catalog:\n taze: 19.14.1\n' + const pin = (v: string) => JSON.stringify({ devDependencies: { taze: v } }) + assert.equal(checkCatalogParity(yaml, pin('19.14.1'), 'y').length, 0) + assert.equal(checkCatalogParity(yaml, pin('19.14.2'), 'y').length, 1) + assert.equal(checkCatalogParity(yaml, pin('catalog:'), 'y').length, 0) +}) + +test('catalog parity: entries after a blank line are still checked', () => { + const yaml = 'catalog:\n taze: 19.14.1\n\n untracked: 1.6.4\n' + const pkg = JSON.stringify({ devDependencies: { taze: '19.14.1', untracked: '1.0.0' } }) + assert.equal(checkCatalogParity(yaml, pkg, 'y').length, 1) +}) + +test('taze config: window must be imported, not hand-copied', () => { + const good = "import { SOAK_DAYS } from './scripts/soak/constants.mts'\nexport default { maturityPeriod: SOAK_DAYS }\n" + assert.equal(checkTazeConfig(good, 't').length, 0) + assert.equal(checkTazeConfig('export default { maturityPeriod: 7 }\n', 't').length, 1) + assert.equal(checkTazeConfig('export default {}\n', 't').length, 2) +}) + +test('toolchain soak: nightly must be SOAK_DAYS old at adoption; stable passes', () => { + const good = '# adopted: 2026-07-11\n[toolchain]\nchannel = "nightly-2026-07-04"\n' + assert.equal(checkToolchainSoak(good, 't').length, 0) + const tooFresh = '# adopted: 2026-07-11\n[toolchain]\nchannel = "nightly-2026-07-08"\n' + assert.match(checkToolchainSoak(tooFresh, 't')[0]!.what, /nightly soak/) + const noDate = '[toolchain]\nchannel = "nightly-2026-07-04"\n' + assert.match(checkToolchainSoak(noDate, 't')[0]!.what, /adoption date/) + const stable = '[toolchain]\nchannel = "1.95.0"\n' + assert.equal(checkToolchainSoak(stable, 't').length, 0) +}) + +test('toolchain soak: impossible calendar dates are findings, not crashes', () => { + const bad = '# adopted: 2026-13-45\n[toolchain]\nchannel = "nightly-2026-07-04"\n' + assert.match(checkToolchainSoak(bad, 't')[0]!.what, /soak dates/) +}) + +test('parser: a column-0 line ends the exclude block', () => { + const yaml = 'minimumReleaseAgeExclude:\n - react\nonlyBuiltDependencies:\n - esbuild\n' + assert.deepEqual(parseExcludeEntries(yaml).map(e => e.name), ['react']) +}) + +test('parser: items at a different indent are not exclude entries', () => { + const yaml = 'minimumReleaseAgeExclude:\n - react\n - not-an-entry\n - vue\n' + assert.deepEqual(parseExcludeEntries(yaml).map(e => e.name), ['react', 'vue']) +}) + +test('fix rewrites a drifted cargo window and leaves a clean one alone', () => { + const fixed = fixCargoConfig('[registry]\nglobal-min-publish-age = "3 days"\n') + assert.ok(fixed.includes(`"${SOAK_DAYS} days"`)) + assert.equal(fixCargoConfig(fixed), fixed) +}) + +const DEPENDABOT_BLOCK = (eco: string, extra = '') => ` - package-ecosystem: ${eco} + directory: "/" + schedule: + interval: weekly +${extra}` + +const DEPENDABOT_COOLDOWN = ` cooldown: + default-days: ${SOAK_DAYS} +` + +test('dependabot: every update block needs an explicit cooldown window', () => { + const good = `version: 2\nupdates:\n${DEPENDABOT_BLOCK('cargo', DEPENDABOT_COOLDOWN)}${DEPENDABOT_BLOCK('github-actions', DEPENDABOT_COOLDOWN)}` + assert.equal(checkDependabotCooldown(good, 'd').length, 0) + // A block with no cooldown at all is drift — dependabot bumps + // server-side, past every local soak surface. + const missing = `version: 2\nupdates:\n${DEPENDABOT_BLOCK('cargo', DEPENDABOT_COOLDOWN)}${DEPENDABOT_BLOCK('github-actions')}` + const findings = checkDependabotCooldown(missing, 'd') + assert.equal(findings.length, 1) + assert.match(findings[0]!.what, /github-actions/) + // Wrong value is drift too. + const drifted = good.replace(`default-days: ${SOAK_DAYS}`, 'default-days: 1') + assert.equal(checkDependabotCooldown(drifted, 'd').length, 1) +}) + +test('dependabot parser: blocks split on package-ecosystem lines', () => { + const body = `version: 2\nupdates:\n${DEPENDABOT_BLOCK('cargo')}${DEPENDABOT_BLOCK('github-actions')}` + assert.deepEqual( + parseDependabotBlocks(body).map(b => b.ecosystem), + ['cargo', 'github-actions'], + ) +}) + +test('dependabot fix rewrites drifted values only, and is idempotent', () => { + const drifted = `version: 2\nupdates:\n${DEPENDABOT_BLOCK('cargo', ' cooldown:\n default-days: 1\n')}` + const fixed = fixDependabotCooldown(drifted) + assert.match(fixed, new RegExp(`default-days: ${SOAK_DAYS}`)) + assert.equal(fixDependabotCooldown(fixed), fixed) + // A missing cooldown block is NOT silently inserted — that stays a + // human edit (the check's fix line says where the two lines go). + const missing = `version: 2\nupdates:\n${DEPENDABOT_BLOCK('cargo')}` + assert.equal(fixDependabotCooldown(missing), missing) +}) + +// Glue: the tracked surfaces of THIS repo must satisfy the gate — the same +// check CI runs, exercised in-process so main() itself stays covered. +test('main --check passes against the tracked repo surfaces', () => { + assert.equal(main([]), 0) + assert.equal(main(['--quiet']), 0) +}) + +// End to end through the entrypoint guard: the CLI must resolve as main +// (realpath + file URL) and exit 0 on a clean tree. +test('CLI: node soak.mts --check --quiet exits 0', () => { + const script = fileURLToPath(new URL('./soak.mts', import.meta.url)) + const res = spawnSync(process.execPath, [script, '--check', '--quiet'], { encoding: 'utf8' }) + assert.equal(res.status, 0, res.stderr) +}) diff --git a/scripts/soak/update-deps.mts b/scripts/soak/update-deps.mts new file mode 100644 index 0000000000..37237b058b --- /dev/null +++ b/scripts/soak/update-deps.mts @@ -0,0 +1,98 @@ +#!/usr/bin/env node +/** + * @file Soaked dependency updater — every ecosystem bumps through the same + * cooldown: + * + * - npm: taze (maturityPeriod = SOAK_DAYS via the taze config next to the + * package.json) rewrites ranges, then the repo's own installer refreshes + * the lockfile. + * - cargo: `cargo update` via rustup's shim. `.cargo/config.toml` carries + * min-publish-age for the same window — an [unstable] cargo feature, so + * it bites only under a nightly toolchain; on perry's stable toolchain + * the automated window rides dependabot's cooldown and this path is a + * plain update. + * + * Usage: node scripts/soak/update-deps.mts [--npm|--cargo] [--dry-run] + * (no ecosystem flag = both) + */ + +import { spawnSync } from 'node:child_process' +import { existsSync, realpathSync } from 'node:fs' +import path from 'node:path' +import process from 'node:process' +import { pathToFileURL } from 'node:url' + +import { NPM_INSTALLERS, NPM_PKG_DIR, REPO_ROOT, RUSTUP_CARGO } from './paths.mts' + +function run(cmd: string, args: string[], cwd: string): number { + console.log(`[update-deps] ${cmd} ${args.join(' ')} (in ${path.relative(REPO_ROOT, cwd) || '.'})`) + const res = spawnSync(cmd, args, { cwd, stdio: 'inherit' }) + if (res.error) { + console.error(`[update-deps] ${cmd}: ${res.error.message}`) + } + return res.status ?? 1 +} + +function updateNpm(dryRun: boolean): number { + const taze = path.join(NPM_PKG_DIR, 'node_modules/.bin/taze') + if (!existsSync(taze)) { + console.error(`[update-deps] taze not installed — run the installer in ${NPM_PKG_DIR} first`) + return 1 + } + // The taze config sets `write: true`; a dry run must override it + // explicitly or "dry" would still rewrite package.json. + const args = dryRun ? ['--no-write', '--no-install'] : ['--write'] + const status = run(taze, args, NPM_PKG_DIR) + if (status !== 0 || dryRun) { + return status + } + for (const [cmd, ...args] of NPM_INSTALLERS) { + if (cmd!.includes('/') && !existsSync(cmd!)) { + continue + } + return run(cmd!, args, NPM_PKG_DIR) + } + console.error('[update-deps] no installer found — refresh the lockfile manually') + return 1 +} + +function updateCargo(dryRun: boolean): number { + // The min-publish-age soak is an [unstable] cargo feature: only a + // nightly honors it, and only rustup's cargo shim reads a + // rust-toolchain.toml pin. Requiring the rustup shim keeps every + // updater on the toolchain the repo actually pins (a Homebrew cargo + // ignores toolchain files entirely) and makes the soak automatic the + // day the repo moves to a dated nightly. + if (!existsSync(RUSTUP_CARGO)) { + console.error('[update-deps] rustup cargo shim not found — refusing a cargo that cannot follow the repo toolchain') + return 1 + } + return run(RUSTUP_CARGO, dryRun ? ['update', '--dry-run'] : ['update'], REPO_ROOT) +} + +// No flag = both; naming both explicitly also means both — a naive +// "flag present = only that one" reading once made `--npm --cargo` run +// NEITHER, so this rule lives in one exported, regression-tested place. +export function selectEcosystems(argv: string[]): { npm: boolean; cargo: boolean } { + const npmFlag = argv.includes('--npm') + const cargoFlag = argv.includes('--cargo') + return { npm: npmFlag || !cargoFlag, cargo: cargoFlag || !npmFlag } +} + +function main(argv: string[] = process.argv.slice(2)): number { + const dryRun = argv.includes('--dry-run') + const { npm, cargo } = selectEcosystems(argv) + // Run every requested ecosystem even if an earlier one fails, then + // aggregate, so one broken ecosystem can't hide the other's drift. + const npmStatus = npm ? updateNpm(dryRun) : 0 + const cargoStatus = cargo ? updateCargo(dryRun) : 0 + return npmStatus || cargoStatus +} + +// realpath + pathToFileURL so symlinked checkouts and paths needing URL +// encoding still register as the entrypoint (ESM realpaths import.meta.url). +const isMain = + process.argv[1] && pathToFileURL(realpathSync(process.argv[1])).href === import.meta.url +if (isMain) { + process.exitCode = main() +} diff --git a/tools/package.json b/tools/package.json new file mode 100644 index 0000000000..896777f114 --- /dev/null +++ b/tools/package.json @@ -0,0 +1,8 @@ +{ + "name": "perry-tools", + "private": true, + "description": "pnpm-soak domain: dev tooling governed by tools/pnpm-workspace.yaml (catalog + minimumReleaseAge). Kept OUT of the repo root so the workspace yaml never marks the npm-managed root (parity-test fixtures, package-lock.json) as a pnpm workspace.", + "devDependencies": { + "taze": "catalog:" + } +} diff --git a/tools/pnpm-lock.yaml b/tools/pnpm-lock.yaml new file mode 100644 index 0000000000..7e5f8db7b5 --- /dev/null +++ b/tools/pnpm-lock.yaml @@ -0,0 +1,223 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +catalogs: + default: + taze: + specifier: 19.14.1 + version: 19.14.1 + +importers: + + .: + devDependencies: + taze: + specifier: 'catalog:' + version: 19.14.1 + +packages: + + '@antfu/ni@30.2.0': + resolution: {integrity: sha512-/FOdAP1w8COnANVD3TtNj/tnpt/36RkU/ysKZTqx86x9acdhCqTFjDXNYVDyBg6UzcrTwWPUeY75ng7CWLNr+g==} + engines: {node: '>=20.19.0'} + hasBin: true + + '@henrygd/queue@1.2.0': + resolution: {integrity: sha512-jW/BLSTpcvExDhqJGxtIPgGr2O0IFF8XUNDwEbfCfhrXT8a4xztQ9Lv6U/vbYzYC0xVWn+3zv6YnLUh3bEFUKA==} + + '@quansync/fs@1.0.0': + resolution: {integrity: sha512-4TJ3DFtlf1L5LDMaM6CanJ/0lckGNtJcMjQ1NAV6zDmA0tEHKZtxNKin8EgPaVX1YzljbxckyT2tJrpQKAtngQ==} + + cac@7.0.0: + resolution: {integrity: sha512-tixWYgm5ZoOD+3g6UTea91eow5z6AAHaho3g0V9CNSNb45gM8SmflpAc+GRd1InC4AqN/07Unrgp56Y94N9hJQ==} + engines: {node: '>=20.19.0'} + + defu@6.1.7: + resolution: {integrity: sha512-7z22QmUWiQ/2d0KkdYmANbRUVABpZ9SNYyH5vx6PZ+nE5bcC0l7uFvEfHlyld/HcGBFTL536ClDt3DEcSlEJAQ==} + + destr@2.0.5: + resolution: {integrity: sha512-ugFTXCtDZunbzasqBxrK93Ik/DRYsO6S/fedkWEMKqt04xZ4csmnmwGDBAb07QWNaGMAmnTIemsYZCksjATwsA==} + + fdir@6.5.0: + resolution: {integrity: sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==} + engines: {node: '>=12.0.0'} + peerDependencies: + picomatch: ^3 || ^4 + peerDependenciesMeta: + picomatch: + optional: true + + fzf@0.5.2: + resolution: {integrity: sha512-Tt4kuxLXFKHy8KT40zwsUPUkg1CrsgY25FxA2U/j/0WgEDCk3ddc/zLTCCcbSHX9FcKtLuVaDGtGE/STWC+j3Q==} + + jiti@2.7.0: + resolution: {integrity: sha512-AC/7JofJvZGrrneWNaEnJeOLUx+JlGt7tNa0wZiRPT4MY1wmfKjt2+6O2p2uz2+skll8OZZmJMNqeke7kKbNgQ==} + hasBin: true + + mimic-function@5.0.1: + resolution: {integrity: sha512-VP79XUPxV2CigYP3jWwAUFSku2aKqBH7uTAapFWCBqutsbmDo96KY5o8uh6U+/YSIn5OxJnXp73beVkpqMIGhA==} + engines: {node: '>=18'} + + node-fetch-native@1.6.7: + resolution: {integrity: sha512-g9yhqoedzIUm0nTnTqAQvueMPVOuIY16bqgAJJC8XOOubYFNwz6IER9qs0Gq2Xd0+CecCKFjtdDTMA4u4xG06Q==} + + ofetch@1.5.1: + resolution: {integrity: sha512-2W4oUZlVaqAPAil6FUg/difl6YhqhUR7x2eZY4bQCko22UXg3hptq9KLQdqFClV+Wu85UX7hNtdGTngi/1BxcA==} + + onetime@7.0.0: + resolution: {integrity: sha512-VXJjc87FScF88uafS3JllDgvAm+c/Slfz06lorj2uAY34rlUu0Nt+v8wreiImcrgAjjIHp1rXpTDlLOGw29WwQ==} + engines: {node: '>=18'} + + package-manager-detector@1.7.0: + resolution: {integrity: sha512-xg1eHpwYL/D/HEdWw2goFZP6vV0FH7W+PZ5rFkGjdIDLtxq7EkzBUeT3m+lndYCt8wKbmofUu1MUdMCXkCk9ZQ==} + + pathe@2.0.3: + resolution: {integrity: sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==} + + picomatch@4.0.5: + resolution: {integrity: sha512-RvwwcruNjI1ncT5xRakeyS9Lf8lcItv34KD+aif+VH9kduAyfYBipGh12274xtenIPZ119/R9BdTBa8gAwSh0A==} + engines: {node: '>=12'} + + pnpm-workspace-yaml@1.6.1: + resolution: {integrity: sha512-yTeZntGWi8m9WNuhoVsP0DpFc4sC1U0+rr/qR6Zi9n2g3sxXY+JfccjXjjruNz96tM8I09yaJUA86doRnNLkbg==} + + quansync@1.0.0: + resolution: {integrity: sha512-5xZacEEufv3HSTPQuchrvV6soaiACMFnq1H8wkVioctoH3TRha9Sz66lOxRwPK/qZj7HPiSveih9yAyh98gvqA==} + + restore-cursor@5.1.0: + resolution: {integrity: sha512-oMA2dcrw6u0YfxJQXm342bFKX/E4sG9rbTzO9ptUcR/e8A33cHuvStiYOwH7fszkZlZ1z/ta9AAoPk2F4qIOHA==} + engines: {node: '>=18'} + + signal-exit@4.1.0: + resolution: {integrity: sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw==} + engines: {node: '>=14'} + + taze@19.14.1: + resolution: {integrity: sha512-+wf/IqGReU68vBE/iJ7JCuV5QeD6zQBp9MI6YphN7bT2vf/YIHd0oVA4AJiX3uANI1hQY58MrVmDwLv0x/q3BA==} + hasBin: true + + tinyexec@1.2.4: + resolution: {integrity: sha512-SHf/r48b7vOrjve9PxJo3MN5v5yuyjHvdUcrQffT3WXMUfnGmHDVbC4k3sHJaJTgZCwpUplIaAo5ANtMyp3YHg==} + engines: {node: '>=18'} + + tinyglobby@0.2.17: + resolution: {integrity: sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g==} + engines: {node: '>=12.0.0'} + + ufo@1.6.4: + resolution: {integrity: sha512-JFNbkD1Svwe0KvGi8GOeLcP4kAWQ609twvCdcHxq1oSL8svv39ZuSvajcD8B+5D0eL4+s1Is2D/O6KN3qcTeRA==} + + unconfig-core@7.5.0: + resolution: {integrity: sha512-Su3FauozOGP44ZmKdHy2oE6LPjk51M/TRRjHv2HNCWiDvfvCoxC2lno6jevMA91MYAdCdwP05QnWdWpSbncX/w==} + + unconfig@7.5.0: + resolution: {integrity: sha512-oi8Qy2JV4D3UQ0PsopR28CzdQ3S/5A1zwsUwp/rosSbfhJ5z7b90bIyTwi/F7hCLD4SGcZVjDzd4XoUQcEanvA==} + + yaml@2.9.0: + resolution: {integrity: sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==} + engines: {node: '>= 14.6'} + hasBin: true + +snapshots: + + '@antfu/ni@30.2.0': + dependencies: + fzf: 0.5.2 + package-manager-detector: 1.7.0 + tinyexec: 1.2.4 + tinyglobby: 0.2.17 + + '@henrygd/queue@1.2.0': {} + + '@quansync/fs@1.0.0': + dependencies: + quansync: 1.0.0 + + cac@7.0.0: {} + + defu@6.1.7: {} + + destr@2.0.5: {} + + fdir@6.5.0(picomatch@4.0.5): + optionalDependencies: + picomatch: 4.0.5 + + fzf@0.5.2: {} + + jiti@2.7.0: {} + + mimic-function@5.0.1: {} + + node-fetch-native@1.6.7: {} + + ofetch@1.5.1: + dependencies: + destr: 2.0.5 + node-fetch-native: 1.6.7 + ufo: 1.6.4 + + onetime@7.0.0: + dependencies: + mimic-function: 5.0.1 + + package-manager-detector@1.7.0: {} + + pathe@2.0.3: {} + + picomatch@4.0.5: {} + + pnpm-workspace-yaml@1.6.1: + dependencies: + yaml: 2.9.0 + + quansync@1.0.0: {} + + restore-cursor@5.1.0: + dependencies: + onetime: 7.0.0 + signal-exit: 4.1.0 + + signal-exit@4.1.0: {} + + taze@19.14.1: + dependencies: + '@antfu/ni': 30.2.0 + '@henrygd/queue': 1.2.0 + cac: 7.0.0 + ofetch: 1.5.1 + package-manager-detector: 1.7.0 + pathe: 2.0.3 + pnpm-workspace-yaml: 1.6.1 + restore-cursor: 5.1.0 + tinyexec: 1.2.4 + tinyglobby: 0.2.17 + unconfig: 7.5.0 + yaml: 2.9.0 + + tinyexec@1.2.4: {} + + tinyglobby@0.2.17: + dependencies: + fdir: 6.5.0(picomatch@4.0.5) + picomatch: 4.0.5 + + ufo@1.6.4: {} + + unconfig-core@7.5.0: + dependencies: + '@quansync/fs': 1.0.0 + quansync: 1.0.0 + + unconfig@7.5.0: + dependencies: + '@quansync/fs': 1.0.0 + defu: 6.1.7 + jiti: 2.7.0 + quansync: 1.0.0 + unconfig-core: 7.5.0 + + yaml@2.9.0: {} diff --git a/tools/pnpm-workspace.yaml b/tools/pnpm-workspace.yaml new file mode 100644 index 0000000000..b53be9fbbd --- /dev/null +++ b/tools/pnpm-workspace.yaml @@ -0,0 +1,25 @@ +# pnpm-side soak surface + catalog (pnpm reads minimumReleaseAge in MINUTES +# from a workspace yaml ONLY — .npmrc is ignored for it). Managed by +# scripts/soak/soak.mts (`npm run soak` / `npm run soak:fix`). +# +# This file deliberately lives in tools/, not the repo root: the root is +# npm-managed (package.json + package-lock.json hold the parity-test +# fixture deps), and a workspace yaml at the root would mark it as a pnpm +# workspace. Anchoring the pnpm domain in tools/ keeps the two package +# managers from seeing each other. + +catalog: + taze: 19.14.1 + +# Cooldown (minutes) before newly published packages install — value managed +# by soak:fix; the window is SOAK_DAYS in scripts/soak/constants.mts. +minimumReleaseAge: 10080 + +# Exclusions: bare names / scope globs express standing trust; version pins +# are dated soak bypasses and REQUIRE, on the line above: +# # published: YYYY-MM-DD | removable: YYYY-MM-DD (removable = published + 7d) +# `npm run soak` rejects unannotated or expired pins (and flow-style [..] +# lists, which the gate can't validate); `soak:fix` prunes expired ones. +# minimumReleaseAgeExclude: +# # published: 2026-01-01 | removable: 2026-01-08 +# - 'example@1.2.3' diff --git a/tools/taze.config.mts b/tools/taze.config.mts new file mode 100644 index 0000000000..9d4fd0e1dc --- /dev/null +++ b/tools/taze.config.mts @@ -0,0 +1,14 @@ +import { defineConfig } from 'taze' + +// Cooldown derives from the canonical SOAK_DAYS so it can't drift from +// pnpm-workspace.yaml minimumReleaseAge / .npmrc min-release-age +// (scripts/soak/soak.mts asserts the data files match the same constant). +import { SOAK_DAYS } from '../scripts/soak/constants.mts' + +export default defineConfig({ + interactive: false, + loglevel: 'warn', + maturityPeriod: SOAK_DAYS, + mode: 'latest', + write: true, +}) From 8efeaff2499797202fd3a68ad5b07b4bcbf9847e Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 00:33:06 -0400 Subject: [PATCH 02/30] docs: changelog fragment for #6912 --- changelog.d/6912-security-tooling-soak.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelog.d/6912-security-tooling-soak.md diff --git a/changelog.d/6912-security-tooling-soak.md b/changelog.d/6912-security-tooling-soak.md new file mode 100644 index 0000000000..763222dfdc --- /dev/null +++ b/changelog.d/6912-security-tooling-soak.md @@ -0,0 +1 @@ +**Supply-chain soak + pinned security tooling:** port the wheelhouse/nub security stack (nubjs/nub#442). A 7-day release-age soak (`SOAK_DAYS` in `scripts/soak/constants.mts`, `npm run soak` parity gate + `soak:fix` fixer) now governs every dependency surface — root `.npmrc` `min-release-age`, `tools/pnpm-workspace.yaml` `minimumReleaseAge` with dated auto-expiring exclusions, `tools/taze.config.mts` `maturityPeriod` (soaked `npm run deps:update` updater), `.cargo/config.toml` `min-publish-age` (nightly-only; inert on stable), and per-block dependabot `cooldown`. `external-tools.json` pins pnpm 11.8.0 / npm 12 / sfw 1.13.1 / zizmor 1.26.1 / agentshield 1.4.0 / skillspector `@2eb84478` exact with sha512 SRI; `npm run tools:install` installs them into a local rack and writes Socket Firewall shims for npm/yarn/pnpm/pip/uv/cargo (recursion-sentinel, fail-open, clobber-guarded; the pnpm/npm shims wrap the rack-pinned binaries). CI grows a hash-pinned zizmor workflow (gate at high with a documented ratchet config in `.github/zizmor.yml`) plus always-run `soak-gate`, `agent-scan` (AgentShield over `.claude/`), and `skills-scan` (NVIDIA SkillSpector over `.claude/skills/`) jobs in security-audit.yml. New `soak` skill documents the workflow. From 3b73baf1c5352ee9a816f35e867c47022976a9b4 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 00:51:11 -0400 Subject: [PATCH 03/30] ci(zizmor): run the SRI-pinned binary instead of the marketplace action The zizmorcore/zizmor-action run hit startup_failure (repo Actions allowlist), and the rack binary is the better shape anyway: one pin source (external-tools.json), and local tools:install audits with the exact bits CI uses. Token-only-when-nonempty works around zizmor treating an empty --gh-token as real and then fatally erroring. --- .github/workflows/zizmor.yml | 43 +++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 13 deletions(-) diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml index 91bebc2bd8..0110af5b20 100644 --- a/.github/workflows/zizmor.yml +++ b/.github/workflows/zizmor.yml @@ -11,22 +11,39 @@ permissions: {} jobs: zizmor: runs-on: ubuntu-latest + timeout-minutes: 10 permissions: contents: read steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@v7 with: persist-credentials: false - - uses: zizmorcore/zizmor-action@6599ee8b7a49aef6a770f63d261d214911a7ce02 # v0.6.0 + - uses: actions/setup-node@v7 with: - advanced-security: false - inputs: .github/ - # Version rides external-tools.json tools.zizmor (keep in lockstep - # so local `npm run tools:install` audits with the same zizmor). - version: "1.26.1" - # Starting gate: high only. The 34 medium findings (missing - # permissions: blocks in test.yml / benchmark.yml / - # container-tests.yml / cache-warm.yml / simctl-tests.yml) are a - # tracked ratchet — audit each job's real token needs, add the - # blocks, then drop this line to gate at medium. - min-severity: high + node-version-file: .node-version + # zizmor rides the same SRI-pinned rack as every other external tool + # (external-tools.json tools.zizmor) instead of a marketplace action — + # one pin source, and local `npm run tools:install` audits with the + # exact same binary CI does. + - name: Install pinned zizmor + run: | + node scripts/soak/external-tools.mts --install zizmor + node scripts/soak/external-tools.mts --print-bin >> "$GITHUB_PATH" + - name: Audit GitHub Actions + env: + GH_TOKEN: ${{ github.token }} + run: | + # Pass the token only when non-empty: zizmor treats an empty + # --gh-token as a real token, then fatally errors when its online + # impostor-commit check can't authenticate. + # + # Starting gate: high only (config in .github/zizmor.yml). The 34 + # medium findings (missing permissions: blocks in test.yml / + # benchmark.yml / container-tests.yml / cache-warm.yml / + # simctl-tests.yml) are a tracked ratchet — audit each job's real + # token needs, add the blocks, then gate at medium. + if [ -n "${GH_TOKEN}" ]; then + zizmor .github/ --gh-token "${GH_TOKEN}" --min-severity high + else + zizmor .github/ --min-severity high + fi From 239e573e2b4153eb0193aabed10b5783047a5a52 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 01:25:54 -0400 Subject: [PATCH 04/30] deps(sfw): bump firewall pins to 1.14.0 via dated soakBypass MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1.14.0 (published 2026-07-23) fixes two things the shims care about: sfw's own diagnostics now go to stderr (stdout stays transparent for callers capturing `pnpm --version` through a shim), and the child env gains a NO_PROXY loopback exemption (localhost,127.0.0.1,::1) so locally-mocked registries are never proxied. Inside the 7-day window until 2026-07-30, so both pins carry the dated soakBypass annotation; tools:check will demand its removal once the window clears — prune the two annotations then. --- external-tools.json | 40 ++++++++++++++++++++++++++-------------- 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/external-tools.json b/external-tools.json index c2699d32b0..599ab8c8d3 100644 --- a/external-tools.json +++ b/external-tools.json @@ -57,68 +57,80 @@ }, "sfw-free": { "description": "Socket Firewall (free tier) \u2014 malware gate on dep installs.", - "version": "1.13.1", + "version": "1.14.0", "repository": "github:SocketDev/sfw-free", "release": "asset", "binaryName": "sfw", "notes": [ "Used when SOCKET_SECURITY_KEY is not set", - "Shims npm/yarn/pnpm so every install call passes through the firewall" + "Shims npm/yarn/pnpm so every install call passes through the firewall", + "1.14.0 adopted early via soakBypass: sfw diagnostics moved to stderr (shim-transparent stdout) + NO_PROXY loopback exemption" ], "platforms": { "darwin-arm64": { "asset": "sfw-free-macos-arm64", - "integrity": "sha512-T6wBOJGdRVSI8577lGqRNzNd6Q+1vqKyaqGgOA8G4M5MU2vcsUnXuJTgP2MMZjUqROSXUlFL0mHguuxXT2QadQ==" + "integrity": "sha512-vcD8n2RmW5MpoEzls8rCe2Wk7/Kj5E4xFXOtwPJaezAqCUrBtt3j0X/Hcnd4a3eBJfit7AiqvnWpnECB5Z56vg==" }, "darwin-x64": { "asset": "sfw-free-macos-x86_64", - "integrity": "sha512-4G/AIY5UGU81wcepDKErY5u0nY85D8UM9nXTEPv8CR2rOV/s4IcmrkxywwZ3ipejHVQB7QmCVt0/SsqWglGikw==" + "integrity": "sha512-OACpfFptj5BF+uHAnmL15safe9jj9ErywqIN57A7uaRftnD/Vw+wy8c+OWLNOAkrE4NfAbfa+YG0+cOJem5yUA==" }, "linux-arm64": { "asset": "sfw-free-linux-arm64", - "integrity": "sha512-FYRYR52SL+KKFldW4ogYOUnTH5OSqvtXwzGFeWi0W2x+75KZcPiGzWBbhMmh0f5QtgYLV+4qdREgmKCBEayNtA==" + "integrity": "sha512-CYp5C4KeyghXa0SUtT6Wr+X851hXIX03cjTFK3qAyU/x+HQkW7ZanAfvWhO+zCmv8DBrXHVWo4Nlv6VnP6vXXQ==" }, "linux-x64": { "asset": "sfw-free-linux-x86_64", - "integrity": "sha512-waLrsPG2a7EOv0XuvXDQZGgCZ4MTtOfZh8TmGbM6gn2B6Nh6HI+15jaoKdAS9wgdTyIqTuqU+O+NtVYd+kuFaA==" + "integrity": "sha512-hbEIhCOBuUjFDpNYDYFJ1j/IwIAiaxKHsGaQ4USOiwBK1y1poupGlIRiqtDkVj4P9x+9I0Gki3xoVq6C2nyBbw==" }, "win-x64": { "asset": "sfw-free-windows-x86_64.exe", - "integrity": "sha512-YYnfwR6M/PHo72LSyKtpY3bAUG4F4ckToJqGx5Fkz4rwg1+48hkxuBaF3hdxHUdHPkfO5grDyoNgXGe7FojGcg==" + "integrity": "sha512-JDpo1DAN+YUYSQ8VlVNPWxbOLoxPjnKSFfyAX+BsnHh3UdURWhCj+vy/aPuA3qg46CZQMpWIlzxBw+l86y/zRw==" } + }, + "soakBypass": { + "version": "1.14.0", + "published": "2026-07-23", + "removable": "2026-07-30" } }, "sfw-enterprise": { "description": "Socket Firewall (enterprise tier) \u2014 selected when SOCKET_SECURITY_KEY is set.", - "version": "1.13.1", + "version": "1.14.0", "repository": "github:SocketDev/firewall-release", "release": "asset", "binaryName": "sfw", "notes": [ "Used when SOCKET_SECURITY_KEY is set (the one env var every Socket product reads)", - "Same shims as sfw-free, broader ecosystem support (Ruby, .NET, Go on Linux)" + "Same shims as sfw-free, broader ecosystem support (Ruby, .NET, Go on Linux)", + "1.14.0 adopted early via soakBypass: sfw diagnostics moved to stderr (shim-transparent stdout) + NO_PROXY loopback exemption" ], "platforms": { "darwin-arm64": { "asset": "sfw-macos-arm64", - "integrity": "sha512-ZDy2C6leKyTHZFvcZZpG2eQqVzs7buk+Hs92fkaMYME829QzyxdGQVVgwEVaGJpedGdUvhksKKcvT9IynI1kxg==" + "integrity": "sha512-MKU4aNBU0EGiSIJiwTnLDJQl6WKO4JPUD1j7c9FihrtizPCtokQxDc+/QAVjc9RFfUIyGZ9OYoVQbTXiysgVJw==" }, "darwin-x64": { "asset": "sfw-macos-x86_64", - "integrity": "sha512-cm76we0sn7kqPOya/ZGQpPyhjRDyFT5lHigeT5Qso+QaPL6Cmwi0FVs2L7l63j+WR/9eYPU1WjjGOto5NbWsEQ==" + "integrity": "sha512-MWkEGX+v04H9Bpy5iC+zHthw9sl9PDAy6vAcoQsJZuy4OGCxWhwJ408egKq19KtEaQxHgwtL/6SoV2j3IHvxmg==" }, "linux-arm64": { "asset": "sfw-linux-arm64", - "integrity": "sha512-9qPi3mobBfyq1k+pD2GDG0tZkhy16f7FXE9oGiwmwPvy5PXwnlzqEXnYld3qGsKIVwNhunev/If26oROTHbrHA==" + "integrity": "sha512-a3FZCp18qNiOPj5tVqlyFtv/N8i1RZ0VcWEqMrgEFfJ4HSMTOraNc4Jh/DVzC1MhRXSUvVcHkM+1bK6lbPE5Gg==" }, "linux-x64": { "asset": "sfw-linux-x86_64", - "integrity": "sha512-lu9h8UzDZt34gdCEVHBGW6goE1Ayykq413EovV5B4nG7jBK27mI0GQstzVbWXA3wWaweT39PehXGtVpdqIDGSA==" + "integrity": "sha512-9SbMTScglJUzlBrrAkJrQlPvSW35LmjZz/VRRIz0VOYCzWuV9tjAxSiXXHTXmnewrDa5ym/pxobYfc1CrnLX0w==" }, "win-x64": { "asset": "sfw-windows-x86_64.exe", - "integrity": "sha512-URZXauIsdUT12E2KTc4sfsxRmJm7nRJzAgM+IYGX4Xq+X0cl/eAbH5SpYIJKpsnW9csSztW9ceyPhlM+f3neIQ==" + "integrity": "sha512-1SnvHJ04sQrRgMAVtGYri5rBysXFXc8PFPXT1kWBF79X6Xa+4uaJb3epJ6HW9HdRvnquDDgP/ZVj7wb6rglADQ==" } + }, + "soakBypass": { + "version": "1.14.0", + "published": "2026-07-23", + "removable": "2026-07-30" } }, "zizmor": { From c9dc5f5ff291a1dba9531a048d4d9e5fd31be96d Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 01:29:28 -0400 Subject: [PATCH 05/30] =?UTF-8?q?deps(tools):=20bump=20zizmor=201.28.0,=20?= =?UTF-8?q?pnpm=2011.15.1,=20npm=2012.0.1=20=E2=80=94=20newest=20soaked=20?= =?UTF-8?q?releases?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All three cleared the 7-day window (zizmor 1.28.0 published 07-21, pnpm 11.15.1 07-19, npm 12.0.1 07-10), so no bypass annotations. pnpm 11.16/11.17 and taze 19.16.0 are still soaking; skillspector upstream (2.4/2.5) is entirely inside the window — follow-up bumps once cleared. Gate re-verified: zizmor 1.28.0 with the shipped config reports 0 findings at high across .github/. --- changelog.d/6912-security-tooling-soak.md | 2 +- external-tools.json | 36 +++++++++++------------ 2 files changed, 19 insertions(+), 19 deletions(-) diff --git a/changelog.d/6912-security-tooling-soak.md b/changelog.d/6912-security-tooling-soak.md index 763222dfdc..fec807dba2 100644 --- a/changelog.d/6912-security-tooling-soak.md +++ b/changelog.d/6912-security-tooling-soak.md @@ -1 +1 @@ -**Supply-chain soak + pinned security tooling:** port the wheelhouse/nub security stack (nubjs/nub#442). A 7-day release-age soak (`SOAK_DAYS` in `scripts/soak/constants.mts`, `npm run soak` parity gate + `soak:fix` fixer) now governs every dependency surface — root `.npmrc` `min-release-age`, `tools/pnpm-workspace.yaml` `minimumReleaseAge` with dated auto-expiring exclusions, `tools/taze.config.mts` `maturityPeriod` (soaked `npm run deps:update` updater), `.cargo/config.toml` `min-publish-age` (nightly-only; inert on stable), and per-block dependabot `cooldown`. `external-tools.json` pins pnpm 11.8.0 / npm 12 / sfw 1.13.1 / zizmor 1.26.1 / agentshield 1.4.0 / skillspector `@2eb84478` exact with sha512 SRI; `npm run tools:install` installs them into a local rack and writes Socket Firewall shims for npm/yarn/pnpm/pip/uv/cargo (recursion-sentinel, fail-open, clobber-guarded; the pnpm/npm shims wrap the rack-pinned binaries). CI grows a hash-pinned zizmor workflow (gate at high with a documented ratchet config in `.github/zizmor.yml`) plus always-run `soak-gate`, `agent-scan` (AgentShield over `.claude/`), and `skills-scan` (NVIDIA SkillSpector over `.claude/skills/`) jobs in security-audit.yml. New `soak` skill documents the workflow. +**Supply-chain soak + pinned security tooling:** port the wheelhouse/nub security stack (nubjs/nub#442). A 7-day release-age soak (`SOAK_DAYS` in `scripts/soak/constants.mts`, `npm run soak` parity gate + `soak:fix` fixer) now governs every dependency surface — root `.npmrc` `min-release-age`, `tools/pnpm-workspace.yaml` `minimumReleaseAge` with dated auto-expiring exclusions, `tools/taze.config.mts` `maturityPeriod` (soaked `npm run deps:update` updater), `.cargo/config.toml` `min-publish-age` (nightly-only; inert on stable), and per-block dependabot `cooldown`. `external-tools.json` pins pnpm 11.15.1 / npm 12.0.1 / sfw 1.14.0 (dated soakBypass) / zizmor 1.28.0 / agentshield 1.4.0 / skillspector `@2eb84478` exact with sha512 SRI; `npm run tools:install` installs them into a local rack and writes Socket Firewall shims for npm/yarn/pnpm/pip/uv/cargo (recursion-sentinel, fail-open, clobber-guarded; the pnpm/npm shims wrap the rack-pinned binaries). CI grows a hash-pinned zizmor workflow (gate at high with a documented ratchet config in `.github/zizmor.yml`) plus always-run `soak-gate`, `agent-scan` (AgentShield over `.claude/`), and `skills-scan` (NVIDIA SkillSpector over `.claude/skills/`) jobs in security-audit.yml. New `soak` skill documents the workflow. diff --git a/external-tools.json b/external-tools.json index 599ab8c8d3..e12962266c 100644 --- a/external-tools.json +++ b/external-tools.json @@ -2,7 +2,7 @@ "tools": { "pnpm": { "description": "pnpm \u2014 the fleet's package manager.", - "version": "11.8.0", + "version": "11.15.1", "packageManager": "pnpm", "repository": "github:pnpm/pnpm", "release": "asset", @@ -13,35 +13,35 @@ "platforms": { "darwin-arm64": { "asset": "pnpm-darwin-arm64.tar.gz", - "integrity": "sha512-kgYcLu653+Gx7l7qEtM2zMNLBvb96ABcRSkzOrbfvkndb+veMt6lHJK5r3wTfuRYlDb8Lnk/h0wUD6sP3taJ9g==" + "integrity": "sha512-lHYcdK/uTX+hHyXqGrkYZSB9g1RcPMo02rk1d0val5dEGAp8C5omJg45gtiKE72rFtQU8xVGRaBD+9HCnWNrTQ==" }, "darwin-x64": { - "asset": "pnpm-11.8.0.tgz", - "integrity": "sha512-wfXnxMskHI8XS3Q4UdgvQrgCMkr8iw8Ra5atsVqgZmSUjd42lgo7oQebpbSyndAUATW5S1tfUmNZIknWjlVfJg==" + "asset": "pnpm-11.15.1.tgz", + "integrity": "sha512-gTULB+U8lTigLx8jA7QpD6LXvgTlbiqXDEzEtBfcdh3hlu2r1J1Vx9yVgNuBAHxEFD5OPX5GKzAA0jwlUSLQZQ==" }, "linux-arm64": { "asset": "pnpm-linux-arm64.tar.gz", - "integrity": "sha512-p1IcVUlwYf3OJQYmdHGFr08d1BOSatHEmLJSBSdoNPGqOqfslFHjSqiuZ/3yuQxxypqp8OfnM3ci3Jh7ZEBlSw==" + "integrity": "sha512-zgPT+13ucXlb/YPP15xI5lsDpFohQY9zzUzr+KIipApmBbvzNBif01a5GXrqEuXHvbUhEAcB+eM4ZuvJ9+LfZg==" }, "linux-arm64-musl": { "asset": "pnpm-linux-arm64-musl.tar.gz", - "integrity": "sha512-u5Do3diwK7FL5vk+i/x2I4q4ujdZ5gSoLNgmlt1w2C7NOjAZOpDK0CJYM4gDa5BetEwzdNpuN/gaMqzJk6I5NA==" + "integrity": "sha512-5ttsFqCHnwPMjfbP7wcD+luX1fH13YeY3Oru/mwVeaQRIgeXbS9+pe6XXzM8R8LFo5RcWjkV/FzCVfLL3iZ3ew==" }, "linux-x64": { "asset": "pnpm-linux-x64.tar.gz", - "integrity": "sha512-Sn7hG4Xsq6pmi8TE8lpIkwRzAYzf5qtFk8zSsZWkSJFcmv5FlsrO8UP3lLRI+ppcRNYS10FMCTwXe3Nt66A0Pg==" + "integrity": "sha512-N6t/VxORcB/i0XTJBGaRjaw986N8c4SswtJejFfLEOgi2lbTSJCRyk7bNbZ1+FPVRVzBE8Wt4470m1Y+LIOACg==" }, "linux-x64-musl": { "asset": "pnpm-linux-x64-musl.tar.gz", - "integrity": "sha512-5WXlo2yCDmoBIue5iHRK3zNSPVAmzci+5RhuxwUA4hnH9W3TFjDnQHyAzDVnkY878mfTXE35THZeEc1OKYPmag==" + "integrity": "sha512-MQ2S67HyPgzj+rk2kvTiC9M62XUxuT5IGQiNsbI8fW+RI9haFjaA/cy+kL84TXi+g/MDok5+plxNUa80Tayjrw==" }, "win-arm64": { "asset": "pnpm-win32-arm64.zip", - "integrity": "sha512-iv1hJEj9FUiVFae2cmCYjbRK0Xn27Uh+kpglT2LWvQyl4WeIOgs1ouKrptVxk3X3t7DB5vuYKbsmE+9BnoQ4FA==" + "integrity": "sha512-PiVRWhoYiNcdBXjPPT9I2XQQN+/BZmrgCTVuX5EuV4/1pcpEQlbHxvaU+XMf1nO+Bt9scV0Qi7kk09eBFfqlAA==" }, "win-x64": { "asset": "pnpm-win32-x64.zip", - "integrity": "sha512-jyqMgedndbck/xJjXPem5Lw7V0YtDfiUJIB81e/hmnrg3yLKPoyCKO5ILegYkLPIr945IJRcVoJZwjtog8FCqg==" + "integrity": "sha512-U1SFdMLlhEgZ6zQpZU+Y0itOBqF7PPb42WScXJ0VY4aPDGmVjYI4r4g3EsOEkcVwMhPI0ISsACgZ9S2ixBHAMw==" } } }, @@ -52,8 +52,8 @@ ], "description": "npm \u2014 pinned, SRI-verified registry tarball; installed without self-update", "repository": "npm:npm", - "version": "12.0.0", - "integrity": "sha512-qzvPQfNSY7louiM6rv7dL0hi5esBGLn1lLwxbdyL5XOIssWzoYMwn8xqvWhYcZL6onTkenYSxrtKxsFrFbUFyw==" + "version": "12.0.1", + "integrity": "sha512-L5T9i/YAQWQWqTS/xZxJkei/9zcu99hCeE4qi41IyBVV7mRQad3qc2JfuOktwmH+qwGI/V2rbCL+/UYxb1+RQA==" }, "sfw-free": { "description": "Socket Firewall (free tier) \u2014 malware gate on dep installs.", @@ -135,7 +135,7 @@ }, "zizmor": { "description": "GitHub Actions security linter \u2014 audits .github/ for workflow-injection / credential-leak patterns.", - "version": "1.26.1", + "version": "1.28.0", "repository": "github:zizmorcore/zizmor", "release": "asset", "notes": [ @@ -145,23 +145,23 @@ "platforms": { "darwin-arm64": { "asset": "zizmor-aarch64-apple-darwin.tar.gz", - "integrity": "sha512-UfLPPdYejR8fvrFwr9Tos7LKFvqr7YcAHWZlAmeo7imYI/fIucYnCY6HeDyk2cbFUvDpQTdps6WQS1QpqCtTfQ==" + "integrity": "sha512-qErEbGBrRH6hwuJDTPio0VUKpV3gKT39eNFwLel7J1RqaucMC5yTilyoo0UdIa9ilxM5P5EocN+wuH8Gw90c7Q==" }, "darwin-x64": { "asset": "zizmor-x86_64-apple-darwin.tar.gz", - "integrity": "sha512-SCbcEzF/zy2qNuNaocLPIUC4Wuq5GnHt0iUFve3qx6bJyFdzU6pDFZkF64dojGC7M5gAcK/4acXGPx3YnMDy/g==" + "integrity": "sha512-hXXbUc6nELKsjUznvTq94dAv4qjeH74Gyj3/1ji+LJf2DWNbk82zc3qM5Y0ik8MkPwrQPC537ETtznTtPKNtQQ==" }, "linux-arm64": { "asset": "zizmor-aarch64-unknown-linux-gnu.tar.gz", - "integrity": "sha512-TkGvwt0zYdmiJ7LZmy6Bz9CdkqcuEpFKhXUK9m0MhSPxBx2gYBsrw6OxGSttMEQ3bvxdvyG6rqbnMgjyDZB1zA==" + "integrity": "sha512-OtTfOoCgBvs05J5fTuo2Kva0y+3j8f7DqlOpbugbd/k/EJRPJjXQpDcEClXn3ClDNXq597kqG3yOAekfidMYUQ==" }, "linux-x64": { "asset": "zizmor-x86_64-unknown-linux-gnu.tar.gz", - "integrity": "sha512-zTMERMDd3JfaRX12klj2fhZGDyrXeLkVUY1QJkCv8RRmAa9uVuH88gHOnv4CQtC5hXS7FPRJzvCVOnw38gV83g==" + "integrity": "sha512-j+SaxjjzZpol8oGl+5yoXPijSLnLaZQd/hjQQfQmxAJpoRJsRITzo52iYOd5E7Gf8YmEt455QjzjPO5eVaC6YQ==" }, "win-x64": { "asset": "zizmor-x86_64-pc-windows-msvc.zip", - "integrity": "sha512-Pijh/CrrOAkZzLiTr2LTHdI8d6+5Ql6B+suY6fXVmL8UVa+4Q36hHL5K67iRFz03v/V/UcrY6+dfhnmot/xfww==" + "integrity": "sha512-YS/j35KOSOO32ZZSCeRgNFQmthYuGBb0hQ2puPimy9c0kSi3eYeINasLBuqEPzfqcoabxePZCG1byBMT4+dDbQ==" } } }, From 71b13f679f377aeca23f3ef7d128d5944c269493 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 01:29:44 -0400 Subject: [PATCH 06/30] docs: fragment says rack-pinned zizmor, not marketplace action --- changelog.d/6912-security-tooling-soak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/6912-security-tooling-soak.md b/changelog.d/6912-security-tooling-soak.md index fec807dba2..34075b6410 100644 --- a/changelog.d/6912-security-tooling-soak.md +++ b/changelog.d/6912-security-tooling-soak.md @@ -1 +1 @@ -**Supply-chain soak + pinned security tooling:** port the wheelhouse/nub security stack (nubjs/nub#442). A 7-day release-age soak (`SOAK_DAYS` in `scripts/soak/constants.mts`, `npm run soak` parity gate + `soak:fix` fixer) now governs every dependency surface — root `.npmrc` `min-release-age`, `tools/pnpm-workspace.yaml` `minimumReleaseAge` with dated auto-expiring exclusions, `tools/taze.config.mts` `maturityPeriod` (soaked `npm run deps:update` updater), `.cargo/config.toml` `min-publish-age` (nightly-only; inert on stable), and per-block dependabot `cooldown`. `external-tools.json` pins pnpm 11.15.1 / npm 12.0.1 / sfw 1.14.0 (dated soakBypass) / zizmor 1.28.0 / agentshield 1.4.0 / skillspector `@2eb84478` exact with sha512 SRI; `npm run tools:install` installs them into a local rack and writes Socket Firewall shims for npm/yarn/pnpm/pip/uv/cargo (recursion-sentinel, fail-open, clobber-guarded; the pnpm/npm shims wrap the rack-pinned binaries). CI grows a hash-pinned zizmor workflow (gate at high with a documented ratchet config in `.github/zizmor.yml`) plus always-run `soak-gate`, `agent-scan` (AgentShield over `.claude/`), and `skills-scan` (NVIDIA SkillSpector over `.claude/skills/`) jobs in security-audit.yml. New `soak` skill documents the workflow. +**Supply-chain soak + pinned security tooling:** port the wheelhouse/nub security stack (nubjs/nub#442). A 7-day release-age soak (`SOAK_DAYS` in `scripts/soak/constants.mts`, `npm run soak` parity gate + `soak:fix` fixer) now governs every dependency surface — root `.npmrc` `min-release-age`, `tools/pnpm-workspace.yaml` `minimumReleaseAge` with dated auto-expiring exclusions, `tools/taze.config.mts` `maturityPeriod` (soaked `npm run deps:update` updater), `.cargo/config.toml` `min-publish-age` (nightly-only; inert on stable), and per-block dependabot `cooldown`. `external-tools.json` pins pnpm 11.15.1 / npm 12.0.1 / sfw 1.14.0 (dated soakBypass) / zizmor 1.28.0 / agentshield 1.4.0 / skillspector `@2eb84478` exact with sha512 SRI; `npm run tools:install` installs them into a local rack and writes Socket Firewall shims for npm/yarn/pnpm/pip/uv/cargo (recursion-sentinel, fail-open, clobber-guarded; the pnpm/npm shims wrap the rack-pinned binaries). CI grows a zizmor workflow running the rack-pinned binary (gate at high with a documented ratchet config in `.github/zizmor.yml`) plus always-run `soak-gate`, `agent-scan` (AgentShield over `.claude/`), and `skills-scan` (NVIDIA SkillSpector over `.claude/skills/`) jobs in security-audit.yml. New `soak` skill documents the workflow. From fc7e3f16d0f7a38fa00f6138fc1e0f646bdb8bdb Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 01:43:39 -0400 Subject: [PATCH 07/30] =?UTF-8?q?feat(soak):=20auto-prune=20expired=20bypa?= =?UTF-8?q?ss=20annotations=20=E2=80=94=20fixer=20+=20scheduled=20bot=20PR?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The soak gates fail closed by design: the day a soakBypass window clears, tools:check goes red until the two annotation lines come off. Failing closed is right; making a human notice is not. So: - external-tools.mts gains --fix (npm run tools:fix): prunes soakBypass annotations whose removable date has passed, then re-runs the checks. Valid-but-expired only — malformed dates stay findings for a human. - soak-autofix.yml (daily cron + dispatch) runs soak:fix + tools:fix, and when anything changed commits to bot/soak-autofix and opens (or force-updates) a PR. Note in-workflow: PRs opened with the default github.token don't trigger CI; set the optional SOAK_AUTOFIX_TOKEN secret to make the bot PRs run checks like any other. Verified end-to-end with a planted expired annotation: --fix prunes it, check returns green, and the fixer is idempotent (unit-tested). --- .claude/skills/soak/SKILL.md | 11 +++-- .github/workflows/soak-autofix.yml | 72 ++++++++++++++++++++++++++++ package.json | 1 + scripts/soak/external-tools.mts | 42 +++++++++++++++- scripts/soak/external-tools.test.mts | 32 +++++++++++++ 5 files changed, 154 insertions(+), 4 deletions(-) create mode 100644 .github/workflows/soak-autofix.yml diff --git a/.claude/skills/soak/SKILL.md b/.claude/skills/soak/SKILL.md index dd798e59d7..b087efe13a 100644 --- a/.claude/skills/soak/SKILL.md +++ b/.claude/skills/soak/SKILL.md @@ -27,11 +27,16 @@ against it: in security-audit.yml, always-run) - `npm run soak:fix` — rewrite drifted windows, prune expired exclusions - `npm run deps:update` — bump npm (taze) + cargo deps through the window -- `npm run tools:check` / `tools:install` — validate / install the - SRI-pinned external tools (`external-tools.json`); `tools:install` also - writes the sfw firewall shims into the dev-tools bin dir +- `npm run tools:check` / `tools:fix` / `tools:install` — validate / + prune-expired-bypasses / install the SRI-pinned external tools + (`external-tools.json`); `tools:install` also writes the sfw firewall + shims into the dev-tools bin dir - `npm run test:scripts` — the scripts' own unit tests +The gates fail closed when a bypass window clears, but nobody has to +watch for that: the scheduled `soak-autofix` workflow runs `soak:fix` + +`tools:fix` daily and commits the pruning as a bot PR. + A soak change is done when `npm run soak` and `npm run test:scripts` both exit 0 — the same gates CI runs. Re-run them after every fix. diff --git a/.github/workflows/soak-autofix.yml b/.github/workflows/soak-autofix.yml new file mode 100644 index 0000000000..c92c051ffb --- /dev/null +++ b/.github/workflows/soak-autofix.yml @@ -0,0 +1,72 @@ +name: soak-autofix + +# The soak gates fail CLOSED by design: an expired soakBypass annotation or +# a cleared minimumReleaseAgeExclude pin turns tools:check / soak red until +# someone prunes it. This workflow does the pruning automatically — daily it +# runs the fixers, and when they change anything it commits to a bot branch +# and opens (or updates) a PR, so the gate never sits red waiting for a +# human to delete two lines. + +on: + schedule: + # Daily, shortly after midnight UTC — annotations expire on date + # boundaries, so the fix lands the morning a window clears. + - cron: '17 0 * * *' + workflow_dispatch: + +permissions: {} + +jobs: + autofix: + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: write + pull-requests: write + steps: + # persist-credentials stays on: this job pushes its own fix branch. + # zizmor: ignore[artipacked] + - uses: actions/checkout@v7 + with: + # Optional PAT: PRs opened with the default github.token do NOT + # trigger CI on the PR (GitHub drops workflow events from + # token-created refs), so checks stay pending until a human + # closes/reopens the PR. Set the SOAK_AUTOFIX_TOKEN repo secret + # (a fine-grained PAT with contents+pull-requests write) to make + # the autofix PRs run CI like any other PR. + token: ${{ secrets.SOAK_AUTOFIX_TOKEN || github.token }} + - uses: actions/setup-node@v7 + with: + node-version-file: .node-version + - name: Run the soak fixers + # Fixers write repairs first and exit by check status: a nonzero + # exit can mean human-only findings remain, which must not stop the + # commit of what WAS mechanically fixable. + run: | + node scripts/soak/soak.mts --fix || true + node scripts/soak/external-tools.mts --fix || true + - name: Commit + PR when something was fixed + env: + GH_TOKEN: ${{ secrets.SOAK_AUTOFIX_TOKEN || github.token }} + run: | + if git diff --quiet; then + echo "soak surfaces clean — nothing to fix" + exit 0 + fi + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + BRANCH="bot/soak-autofix" + git checkout -B "$BRANCH" + git add -A + git commit -m "chore(soak): prune expired soak annotations (automated) + + Generated by the soak-autofix workflow: soak.mts --fix + + external-tools.mts --fix. Windows that cleared have soaked; + their bypass annotations are dead weight the gates would + otherwise fail on." + git push -f origin "$BRANCH" + if [ -z "$(gh pr list --head "$BRANCH" --state open --json number --jq '.[0].number')" ]; then + gh pr create --head "$BRANCH" \ + --title "chore(soak): prune expired soak annotations (automated)" \ + --body "Automated by the soak-autofix workflow. The listed soak windows have cleared, so their bypass annotations must come off before tools:check / soak go red. Diff is the full review: only annotation/window lines are touched." + fi diff --git a/package.json b/package.json index 2314301a4c..5fe3d3768f 100644 --- a/package.json +++ b/package.json @@ -5,6 +5,7 @@ "soak:fix": "node scripts/soak/soak.mts --fix", "deps:update": "node scripts/soak/update-deps.mts", "tools:check": "node scripts/soak/external-tools.mts --check", + "tools:fix": "node scripts/soak/external-tools.mts --fix", "tools:install": "node scripts/soak/external-tools.mts --install-all --shims", "test:scripts": "node --test scripts/soak/*.test.mts" }, diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index daa2eb5e7a..2f0cda41a9 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -7,6 +7,10 @@ * * - `--check` validate every pin (shape, SRI prefix, soak * annotations on any soakBypass) — CI gate, no network + * - `--fix` prune soakBypass annotations whose window has + * cleared, then run the same checks (the scheduled + * soak-autofix workflow commits the result so the + * gate never sits red waiting for a human) * - `--install ` download + SRI-verify + install into the local * tool rack (see paths.mts RACK_DIR) with a PATH * handle in BIN_DIR @@ -123,6 +127,34 @@ export function checkPins(tools: Record): string[] { return out } +/** + * Prune expired soakBypass annotations in place and return the pruned tool + * names. Once `removable` is in the past the version has soaked and the + * annotation is dead weight — checkPins turns it into a red gate. Only + * valid, expired dates are pruned; malformed annotations stay findings for + * a human (never silently rewritten). + */ +export function pruneExpiredSoakBypasses(doc: { + tools: Record +}): string[] { + const pruned: string[] = [] + const today = todayIso() + for (const [name, pin] of Object.entries(doc.tools)) { + const bypass = pin.soakBypass + if (!bypass) { + continue + } + if (!isValidIsoDate(bypass.published) || !isValidIsoDate(bypass.removable)) { + continue + } + if (bypass.removable < today) { + delete pin.soakBypass + pruned.push(name) + } + } + return pruned +} + function sriToHex(sri: string): string { return Buffer.from(sri.slice('sha512-'.length), 'base64').toString('hex') } @@ -502,8 +534,16 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise 0) { + writeFileSync(EXTERNAL_TOOLS_JSON, `${JSON.stringify(doc, null, 2)}\n`) + console.log(`[external-tools] pruned expired soakBypass: ${pruned.join(', ')}`) + } + } const tools = loadTools() - if (argv.includes('--check') || argv.length === 0) { + if (argv.includes('--check') || argv.includes('--fix') || argv.length === 0) { const problems = checkPins(tools) if (DOCKER_PREBAKE) { const dockerAbs = path.join(REPO_ROOT, DOCKER_PREBAKE) diff --git a/scripts/soak/external-tools.test.mts b/scripts/soak/external-tools.test.mts index ec0eecc0b8..6a136951c7 100644 --- a/scripts/soak/external-tools.test.mts +++ b/scripts/soak/external-tools.test.mts @@ -15,6 +15,7 @@ import { extractArchive, installTool, main, + pruneExpiredSoakBypasses, } from './external-tools.mts' import { DOCKER_PREBAKE, EXTERNAL_TOOLS_JSON, REPO_ROOT, SURFACES } from './paths.mts' @@ -53,6 +54,37 @@ test('checkPins validates soakBypass dates, arithmetic, and expiry', () => { assert.match(checkPins(impossible)[0]!, /calendar/) }) +test('pruneExpiredSoakBypasses prunes only valid, expired annotations', () => { + const pub = addDaysIso(todayIso(), -1) + const doc = { + tools: { + fresh: { + version: '1.0.0', + integrity: GOOD_SRI, + soakBypass: { version: '1.0.0', published: pub, removable: addDaysIso(pub, SOAK_DAYS) }, + }, + expired: { + version: '1.0.0', + integrity: GOOD_SRI, + soakBypass: { version: '1.0.0', published: '2020-01-01', removable: '2020-01-08' }, + }, + // Malformed dates stay findings for a human — never silently pruned. + malformed: { + version: '1.0.0', + integrity: GOOD_SRI, + soakBypass: { version: '1.0.0', published: '2026-13-45', removable: '2026-13-52' }, + }, + unannotated: { version: '1.0.0', integrity: GOOD_SRI }, + }, + } + assert.deepEqual(pruneExpiredSoakBypasses(doc), ['expired']) + assert.ok(doc.tools.fresh.soakBypass) + assert.ok(!('soakBypass' in doc.tools.expired)) + assert.ok(doc.tools.malformed.soakBypass) + // Idempotent: a second pass finds nothing left to prune. + assert.deepEqual(pruneExpiredSoakBypasses(doc), []) +}) + test('the repo Dockerfile prebake (when present) matches the tracked pins', t => { if (!DOCKER_PREBAKE || !existsSync(path.join(REPO_ROOT, DOCKER_PREBAKE))) { t.skip('repo has no prebake image') From 6cc276e8126f13e7d74789f462a78f367c459a5b Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 01:45:49 -0400 Subject: [PATCH 08/30] ci(soak-autofix): bind the artipacked ignore to the checkout line --- .github/workflows/soak-autofix.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/.github/workflows/soak-autofix.yml b/.github/workflows/soak-autofix.yml index c92c051ffb..37a1fde425 100644 --- a/.github/workflows/soak-autofix.yml +++ b/.github/workflows/soak-autofix.yml @@ -25,8 +25,7 @@ jobs: pull-requests: write steps: # persist-credentials stays on: this job pushes its own fix branch. - # zizmor: ignore[artipacked] - - uses: actions/checkout@v7 + - uses: actions/checkout@v7 # zizmor: ignore[artipacked] with: # Optional PAT: PRs opened with the default github.token do NOT # trigger CI on the PR (GitHub drops workflow events from From 4fe87ad73fcd2aff54c890fc689512b4ff00f933 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 01:48:07 -0400 Subject: [PATCH 09/30] ci: pin actions/* to latest release-tag SHAs in the new security workflows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit checkout v7.0.1 (3d3c42e5) + setup-node v7.0.0 (82076278) across soak-autofix / zizmor / security-audit — both releases cleared the 7-day window (07-20 / 07-14). Also splits the PATH export in agent-scan (SC2155). The legacy workflow fleet stays on ref pins per the documented digest-pin sweep in .github/zizmor.yml. --- .github/workflows/security-audit.yml | 17 +++++++++-------- .github/workflows/soak-autofix.yml | 4 ++-- .github/workflows/zizmor.yml | 4 ++-- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/.github/workflows/security-audit.yml b/.github/workflows/security-audit.yml index 8e36facac1..20f5bdeae7 100644 --- a/.github/workflows/security-audit.yml +++ b/.github/workflows/security-audit.yml @@ -19,7 +19,7 @@ jobs: security-audit: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: dtolnay/rust-toolchain@stable - uses: Swatinem/rust-cache@v2 with: @@ -72,10 +72,10 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: actions/setup-node@v7 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version-file: .node-version - name: Soak window parity (npm run soak) @@ -93,17 +93,18 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: actions/setup-node@v7 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version-file: .node-version - name: Install pinned agentshield run: node scripts/soak/external-tools.mts --install agentshield - name: Scan .claude/ config run: | - export PATH="$(node scripts/soak/external-tools.mts --print-bin):$PATH" + BIN_DIR="$(node scripts/soak/external-tools.mts --print-bin)" + export PATH="$BIN_DIR:$PATH" agentshield scan # SkillSpector (NVIDIA) — audits the repo's Claude skills (.claude/skills/) @@ -115,7 +116,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 15 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 @@ -138,7 +139,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 20 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: dtolnay/rust-toolchain@stable - uses: Swatinem/rust-cache@v2 with: diff --git a/.github/workflows/soak-autofix.yml b/.github/workflows/soak-autofix.yml index 37a1fde425..471a1db4b8 100644 --- a/.github/workflows/soak-autofix.yml +++ b/.github/workflows/soak-autofix.yml @@ -25,7 +25,7 @@ jobs: pull-requests: write steps: # persist-credentials stays on: this job pushes its own fix branch. - - uses: actions/checkout@v7 # zizmor: ignore[artipacked] + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 # zizmor: ignore[artipacked] with: # Optional PAT: PRs opened with the default github.token do NOT # trigger CI on the PR (GitHub drops workflow events from @@ -34,7 +34,7 @@ jobs: # (a fine-grained PAT with contents+pull-requests write) to make # the autofix PRs run CI like any other PR. token: ${{ secrets.SOAK_AUTOFIX_TOKEN || github.token }} - - uses: actions/setup-node@v7 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version-file: .node-version - name: Run the soak fixers diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml index 0110af5b20..54ea3f3233 100644 --- a/.github/workflows/zizmor.yml +++ b/.github/workflows/zizmor.yml @@ -15,10 +15,10 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: actions/setup-node@v7 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version-file: .node-version # zizmor rides the same SRI-pinned rack as every other external tool From 07ca0a92599c2fc1937b89b2e5fafc29e3284e24 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 01:57:09 -0400 Subject: [PATCH 10/30] =?UTF-8?q?fix(soak):=20expired=20annotations=20warn?= =?UTF-8?q?=20instead=20of=20failing=20=E2=80=94=20stale=20is=20not=20unsa?= =?UTF-8?q?fe?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An EXPIRED soakBypass / exclude pin means the version has fully soaked: the bypass no longer bypasses anything and the pin stays SRI-verified. Failing closed on that turned a no-risk cosmetic state into a red required check that flips overnight with zero code change — the exact noise that trains people to admin-bypass (and the model wheelhouse deliberately avoids: informational + auto-drop). Now: expired-but-VALID annotations are warnings (exit 0), surfaced by staleBypasses / staleExcludes and pruned by --fix + the daily soak-autofix workflow. Missing, malformed, or wrong-arithmetic annotations stay hard failures — unauditable IS unsafe. This also defuses the 2026-07-30 expiry of this PR's own sfw annotations. --- scripts/soak/external-tools.mts | 45 +++++++++++++++++++++------ scripts/soak/external-tools.test.mts | 10 ++++-- scripts/soak/soak.mts | 46 ++++++++++++++++++++-------- scripts/soak/soak.test.mts | 12 ++++++-- 4 files changed, 87 insertions(+), 26 deletions(-) diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 2f0cda41a9..9dd522448e 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -116,12 +116,34 @@ export function checkPins(tools: Record): string[] { if (removable !== expected) { out.push(`${name}: soakBypass removable ${removable}, wanted ${expected} (published + ${SOAK_DAYS}d)`) } - // A bypass whose window has passed is dead weight: the version has - // soaked, so the annotation must come off (same rule the workspace - // yaml excludes live under). - if (removable < todayIso()) { - out.push(`${name}: soakBypass expired (removable ${removable}) — the pin has soaked, remove the annotation`) - } + } + } + return out +} + +/** + * Expired soakBypass annotations are STALE, not unsafe: the version has + * soaked, the bypass no longer bypasses anything, and the pin stays + * SRI-verified. They are reported as WARNINGS (exit 0), never failures — + * a date boundary must not redden CI overnight with zero code change. + * The soak-autofix workflow prunes them daily via --fix, so the ledger + * still converges to clean. Missing/malformed/wrong-arithmetic + * annotations stay hard checkPins failures: those are unauditable, + * which IS unsafe. + */ +export function staleBypasses(tools: Record): string[] { + const out: string[] = [] + const today = todayIso() + for (const [name, pin] of Object.entries(tools)) { + const bypass = pin.soakBypass + if (!bypass) { + continue + } + if (!isValidIsoDate(bypass.published) || !isValidIsoDate(bypass.removable)) { + continue + } + if (bypass.removable < today) { + out.push(name) } } return out @@ -130,9 +152,9 @@ export function checkPins(tools: Record): string[] { /** * Prune expired soakBypass annotations in place and return the pruned tool * names. Once `removable` is in the past the version has soaked and the - * annotation is dead weight — checkPins turns it into a red gate. Only - * valid, expired dates are pruned; malformed annotations stay findings for - * a human (never silently rewritten). + * annotation is dead weight — staleBypasses warns about it until this + * prunes it. Only valid, expired dates are pruned; malformed annotations + * stay findings for a human (never silently rewritten). */ export function pruneExpiredSoakBypasses(doc: { tools: Record @@ -566,6 +588,11 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise { +test('checkPins validates soakBypass dates and arithmetic; expiry is a warning, not a failure', () => { const pub = addDaysIso(todayIso(), -1) const good = { a: { @@ -43,15 +44,20 @@ test('checkPins validates soakBypass dates, arithmetic, and expiry', () => { }, } assert.deepEqual(checkPins(good), []) + assert.deepEqual(staleBypasses(good), []) const wrongMath = structuredClone(good) wrongMath.a.soakBypass.removable = addDaysIso(pub, 3) assert.match(checkPins(wrongMath)[0]!, /removable/) + // Expired-but-valid is STALE, not unsafe: checkPins exits clean, the + // stale list reports it, and --fix / soak-autofix prunes it. const expired = structuredClone(good) expired.a.soakBypass = { version: '1.0.0', published: '2020-01-01', removable: '2020-01-08' } - assert.match(checkPins(expired)[0]!, /expired/) + assert.deepEqual(checkPins(expired), []) + assert.deepEqual(staleBypasses(expired), ['a']) const impossible = structuredClone(good) impossible.a.soakBypass = { version: '1.0.0', published: '2026-13-45', removable: '2026-13-52' } assert.match(checkPins(impossible)[0]!, /calendar/) + assert.deepEqual(staleBypasses(impossible), []) }) test('pruneExpiredSoakBypasses prunes only valid, expired annotations', () => { diff --git a/scripts/soak/soak.mts b/scripts/soak/soak.mts index a619178cb4..f4b6c57b53 100644 --- a/scripts/soak/soak.mts +++ b/scripts/soak/soak.mts @@ -105,13 +105,13 @@ export function checkWorkspaceYaml(body: string, file: string): Finding[] { /** * Every version-pinned `minimumReleaseAgeExclude` entry must carry, on the * line directly above, `# published: YYYY-MM-DD | removable: YYYY-MM-DD` - * with `removable = published + SOAK_DAYS`, and must be pruned once - * `removable` is strictly in the past. Bare names and `@scope/*` globs are - * standing trust, not dated bypasses — no annotation required. + * with `removable = published + SOAK_DAYS`. Bare names and `@scope/*` + * globs are standing trust, not dated bypasses — no annotation required. + * EXPIRED entries are not findings (see staleExcludes): stale is not + * unsafe, and a date boundary must not redden CI with zero code change. */ export function checkExcludeAnnotations(body: string, file: string): Finding[] { const out: Finding[] = [] - const today = todayIso() // Flow style would be invisible to the block parser below — an // unvalidated, never-expiring bypass. One canonical shape only. if (/^minimumReleaseAgeExclude:\s*\[/m.test(body)) { @@ -159,19 +159,32 @@ export function checkExcludeAnnotations(body: string, file: string): Finding[] { fix: 'correct the removable date', }) } - if (removable < today) { - out.push({ - file, - what: `soak exclude '${entry.name}' expired`, - saw: `removable ${removable} < today ${today}`, - wanted: 'entry pruned once its window has passed', - fix: 'delete the pin + its annotation (or run --fix)', - }) - } } return out } +/** + * Version-pinned excludes whose window has cleared. Stale, not unsafe — + * the soak would admit the version anyway — so main() WARNS about these + * (exit 0) instead of failing; `--fix` (and the daily soak-autofix + * workflow) prunes them together with their annotation lines. Only valid, + * correctly-annotated entries qualify: anything malformed stays a + * checkExcludeAnnotations failure. + */ +export function staleExcludes(body: string): string[] { + const today = todayIso() + return parseExcludeEntries(body) + .filter( + e => + VERSION_PIN_RE.test(e.name) && + e.annotation && + isValidIsoDate(e.annotation.published) && + isValidIsoDate(e.annotation.removable) && + e.annotation.removable < today, + ) + .map(e => e.name) +} + interface ExcludeEntry { name: string line: number @@ -493,6 +506,13 @@ export function main(argv: string[] = process.argv.slice(2)): number { // Catalog <-> package.json lockstep for the package next to the yaml. const yamlAbs = path.join(REPO_ROOT, SURFACES.workspaceYaml) const pkgAbs = path.join(path.dirname(yamlAbs), 'package.json') + if (existsSync(yamlAbs)) { + for (const name of staleExcludes(readFileSync(yamlAbs, 'utf8'))) { + console.warn( + `[soak] warn: exclude '${name}' has soaked — stale pin, pruned by --fix / the soak-autofix workflow`, + ) + } + } if (existsSync(yamlAbs) && existsSync(pkgAbs)) { findings.push( ...checkCatalogParity( diff --git a/scripts/soak/soak.test.mts b/scripts/soak/soak.test.mts index 692d77958c..1deeab9494 100644 --- a/scripts/soak/soak.test.mts +++ b/scripts/soak/soak.test.mts @@ -20,6 +20,7 @@ import { main, parseDependabotBlocks, parseExcludeEntries, + staleExcludes, } from './soak.mts' // A pin published yesterday is inside its window; one published long ago @@ -75,11 +76,18 @@ test('excludes: unannotated version pin is a finding, bare/glob are not', () => assert.match(findings[0]!.what, /lodash@4\.17\.21/) }) -test('excludes: wrong removable date and expiry are findings', () => { +test('excludes: wrong removable date is a finding; expiry is a warning, not a finding', () => { const wrong = `minimumReleaseAgeExclude:\n # published: ${FRESH_PUB} | removable: ${addDaysIso(FRESH_PUB, 3)}\n - 'a@1.0.0'\n` assert.match(checkExcludeAnnotations(wrong, 'y')[0]!.what, /removable date/) + // Expired-but-valid is STALE, not unsafe: check exits clean, the stale + // list reports it, and --fix / the soak-autofix workflow prunes it. const expired = `minimumReleaseAgeExclude:\n # published: 2020-01-01 | removable: 2020-01-08\n - 'b@1.0.0'\n` - assert.match(checkExcludeAnnotations(expired, 'y')[0]!.what, /expired/) + assert.deepEqual(checkExcludeAnnotations(expired, 'y'), []) + assert.deepEqual(staleExcludes(expired), ['b@1.0.0']) + // Fresh and malformed entries are never "stale". + assert.deepEqual(staleExcludes(CLEAN_YAML), []) + const malformed = `minimumReleaseAgeExclude:\n # published: 2026-13-45 | removable: 2026-13-52\n - 'c@1.0.0'\n` + assert.deepEqual(staleExcludes(malformed), []) }) test('excludes: impossible calendar dates are findings, not crashes', () => { From e3dda612bd39bd4d9adb5e30b391543adcf5353c Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 02:15:03 -0400 Subject: [PATCH 11/30] fix(soak): address review-bot findings across the port MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - platformKey(): detect musl via the loader heuristic — the -musl pnpm pins were dead keys and a musl host silently installed glibc bits; tools with no -musl pin now fail loud instead. - RUSTUP_CARGO honors CARGO_HOME (custom cargo homes reported the rustup shim as missing). - parseExcludeEntries: tolerate a trailing comment on the minimumReleaseAgeExclude key line — previously the block never opened and every entry beneath escaped validation. - checkCatalogParity: malformed package.json is a Finding, not a crash. - soak-autofix workflow: main-ref guard (dispatch on a topic branch can't force-push the bot branch), concurrency group, and fixer exit status captured + re-raised AFTER the mechanical commit instead of '|| true' masking runtime failures. - sfw shims: fail-open is no longer silent-open — one stderr line when sfw is missing (never on the sentinel re-entry path). - GITHUB_TOKEN on the CI install steps (github.com release fetches). - schematic YYYY-MM-DD example dates in the yaml + skill (the concrete examples were expired copy-paste bait); em-dashes restored in external-tools.json (ensure_ascii artifact). --- .claude/skills/soak/SKILL.md | 11 +++++---- .github/workflows/security-audit.yml | 5 ++++ .github/workflows/soak-autofix.yml | 35 +++++++++++++++++++++++----- .github/workflows/zizmor.yml | 4 ++++ external-tools.json | 16 ++++++------- scripts/soak/external-tools.mts | 15 +++++++++++- scripts/soak/paths.mts | 4 +++- scripts/soak/soak.mts | 22 +++++++++++++++-- scripts/soak/soak.test.mts | 14 +++++++++++ tools/pnpm-workspace.yaml | 8 ++++--- 10 files changed, 108 insertions(+), 26 deletions(-) diff --git a/.claude/skills/soak/SKILL.md b/.claude/skills/soak/SKILL.md index b087efe13a..bc3275766a 100644 --- a/.claude/skills/soak/SKILL.md +++ b/.claude/skills/soak/SKILL.md @@ -63,14 +63,15 @@ annotation on the line above (block list only — flow `[..]` is rejected because a comment line can't attach to an inline entry): ```yaml -# published: 2026-07-08 | removable: 2026-07-15 +# published: YYYY-MM-DD | removable: YYYY-MM-DD - 'name@1.2.3' ``` -`removable` = `published + SOAK_DAYS` (this example assumes a 7-day -window). `published` must be the real registry publish date. Once -`removable` passes, `npm run soak` fails until the pin is pruned -(`soak:fix` does it). Bare names / `@scope/*` globs are standing trust and +`removable` = `published + SOAK_DAYS`; `published` must be the real +registry publish date (the placeholders above are schematic — copying +them verbatim is rejected). Once `removable` passes, `npm run soak` +warns until the pin is pruned (`soak:fix` or the soak-autofix workflow +does it). Bare names / `@scope/*` globs are standing trust and need no annotation. External tools use the same shape via a `soakBypass` object in `external-tools.json`. diff --git a/.github/workflows/security-audit.yml b/.github/workflows/security-audit.yml index 20f5bdeae7..15b4eae15e 100644 --- a/.github/workflows/security-audit.yml +++ b/.github/workflows/security-audit.yml @@ -100,6 +100,11 @@ jobs: with: node-version-file: .node-version - name: Install pinned agentshield + env: + # download() attaches this to github.com fetches only (release + # assets); without it private assets 404 and public ones ride + # unauthenticated rate limits. + GITHUB_TOKEN: ${{ github.token }} run: node scripts/soak/external-tools.mts --install agentshield - name: Scan .claude/ config run: | diff --git a/.github/workflows/soak-autofix.yml b/.github/workflows/soak-autofix.yml index 471a1db4b8..2701ed3cf0 100644 --- a/.github/workflows/soak-autofix.yml +++ b/.github/workflows/soak-autofix.yml @@ -16,8 +16,18 @@ on: permissions: {} +# One run at a time: overlapping runs would race on the bot branch +# force-push and the open-PR check. +concurrency: + group: soak-autofix + cancel-in-progress: false + jobs: autofix: + # Guard the dispatch path: run only from main, so a workflow_dispatch + # on a topic branch can't force-push bot/soak-autofix from arbitrary + # HEAD state. + if: github.ref == 'refs/heads/main' runs-on: ubuntu-latest timeout-minutes: 10 permissions: @@ -38,12 +48,17 @@ jobs: with: node-version-file: .node-version - name: Run the soak fixers - # Fixers write repairs first and exit by check status: a nonzero - # exit can mean human-only findings remain, which must not stop the - # commit of what WAS mechanically fixable. + id: fixers + # Fixers write mechanical repairs first, then exit by post-fix + # check status. Capture that status instead of masking it: the + # commit step below still lands whatever WAS fixable, and the + # final step re-raises the failure so a crashed fixer or a + # human-only finding can never ride a green run. run: | - node scripts/soak/soak.mts --fix || true - node scripts/soak/external-tools.mts --fix || true + status=0 + node scripts/soak/soak.mts --fix || status=$? + node scripts/soak/external-tools.mts --fix || status=$? + echo "status=${status}" >> "$GITHUB_OUTPUT" - name: Commit + PR when something was fixed env: GH_TOKEN: ${{ secrets.SOAK_AUTOFIX_TOKEN || github.token }} @@ -67,5 +82,13 @@ jobs: if [ -z "$(gh pr list --head "$BRANCH" --state open --json number --jq '.[0].number')" ]; then gh pr create --head "$BRANCH" \ --title "chore(soak): prune expired soak annotations (automated)" \ - --body "Automated by the soak-autofix workflow. The listed soak windows have cleared, so their bypass annotations must come off before tools:check / soak go red. Diff is the full review: only annotation/window lines are touched." + --body "Automated by the soak-autofix workflow. The listed soak windows have cleared, so their bypass annotations are stale. Diff is the full review: only annotation/window lines are touched." fi + - name: Re-raise fixer findings + # After the mechanical repairs are committed, a nonzero fixer + # status means findings remain that need a human (or the fixer + # itself crashed) — fail the run so it can't read as clean. + if: steps.fixers.outputs.status != '0' + run: | + echo "soak fixers exited nonzero — human-actionable findings remain (see fixer step log)" + exit 1 diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml index 54ea3f3233..c4231bb49a 100644 --- a/.github/workflows/zizmor.yml +++ b/.github/workflows/zizmor.yml @@ -26,6 +26,10 @@ jobs: # one pin source, and local `npm run tools:install` audits with the # exact same binary CI does. - name: Install pinned zizmor + env: + # download() attaches this to github.com fetches only (release + # assets) — avoids unauthenticated rate limits. + GITHUB_TOKEN: ${{ github.token }} run: | node scripts/soak/external-tools.mts --install zizmor node scripts/soak/external-tools.mts --print-bin >> "$GITHUB_PATH" diff --git a/external-tools.json b/external-tools.json index e12962266c..440998fc34 100644 --- a/external-tools.json +++ b/external-tools.json @@ -1,13 +1,13 @@ { "tools": { "pnpm": { - "description": "pnpm \u2014 the fleet's package manager.", + "description": "pnpm — the fleet's package manager.", "version": "11.15.1", "packageManager": "pnpm", "repository": "github:pnpm/pnpm", "release": "asset", "notes": [ - "Latest soaked pnpm \u2014 CI downloads + SRI-verifies (sha512) the pinned asset", + "Latest soaked pnpm — CI downloads + SRI-verifies (sha512) the pinned asset", "darwin-x64 has no SEA asset upstream; its pin is the npm registry tarball" ], "platforms": { @@ -48,15 +48,15 @@ "npm": { "notes": [ "npm 12 (min-release-age support). ONE platform-agnostic registry tarball, single integrity", - "Installed from the pinned tarball only \u2014 never npm install -g npm, no self-update path" + "Installed from the pinned tarball only — never npm install -g npm, no self-update path" ], - "description": "npm \u2014 pinned, SRI-verified registry tarball; installed without self-update", + "description": "npm — pinned, SRI-verified registry tarball; installed without self-update", "repository": "npm:npm", "version": "12.0.1", "integrity": "sha512-L5T9i/YAQWQWqTS/xZxJkei/9zcu99hCeE4qi41IyBVV7mRQad3qc2JfuOktwmH+qwGI/V2rbCL+/UYxb1+RQA==" }, "sfw-free": { - "description": "Socket Firewall (free tier) \u2014 malware gate on dep installs.", + "description": "Socket Firewall (free tier) — malware gate on dep installs.", "version": "1.14.0", "repository": "github:SocketDev/sfw-free", "release": "asset", @@ -95,7 +95,7 @@ } }, "sfw-enterprise": { - "description": "Socket Firewall (enterprise tier) \u2014 selected when SOCKET_SECURITY_KEY is set.", + "description": "Socket Firewall (enterprise tier) — selected when SOCKET_SECURITY_KEY is set.", "version": "1.14.0", "repository": "github:SocketDev/firewall-release", "release": "asset", @@ -134,7 +134,7 @@ } }, "zizmor": { - "description": "GitHub Actions security linter \u2014 audits .github/ for workflow-injection / credential-leak patterns.", + "description": "GitHub Actions security linter — audits .github/ for workflow-injection / credential-leak patterns.", "version": "1.28.0", "repository": "github:zizmorcore/zizmor", "release": "asset", @@ -171,7 +171,7 @@ "integrity": "sha512-R98OO1Ujyk2lezDLb+iQmMhF6FwTJCHajy3G4FCB6x7wkSTqR9f8+eAelC5KDzYDsGSbc0sOZvjXOOPRBtMpDg==" }, "skillspector": { - "description": "NVIDIA's third-party-skill security scanner (LangGraph-based; YARA + AST + OSV.dev CVE lookups + optional LLM analysis). No PyPI release / no GH tags upstream \u2014 pinned to a git SHA on main + installed via a locked uv project (pyproject.toml + uv.lock, `uv sync --locked`; the fleet uv pin + exclude-newer make it reproducible). Sibling to AgentShield: AgentShield audits the operator's .claude/ config; SkillSpector audits untrusted upstream skills before install.", + "description": "NVIDIA's third-party-skill security scanner (LangGraph-based; YARA + AST + OSV.dev CVE lookups + optional LLM analysis). No PyPI release / no GH tags upstream — pinned to a git SHA on main + installed via a locked uv project (pyproject.toml + uv.lock, `uv sync --locked`; the fleet uv pin + exclude-newer make it reproducible). Sibling to AgentShield: AgentShield audits the operator's .claude/ config; SkillSpector audits untrusted upstream skills before install.", "release": "uv-project", "repository": "github:NVIDIA/skillspector", "version": "2eb84478", diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 9dd522448e..2c474b50ab 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -81,7 +81,14 @@ function platformKey(): string { if (!osKey || !archKey) { throw new Error(`unsupported platform ${process.platform}-${process.arch}`) } - return `${osKey}-${archKey}` + // musl vs glibc via the loader-presence heuristic. Without this a musl + // host silently resolved the glibc pin (the `-musl` pnpm entries were + // dead keys) and installed a binary that can't run; tools with no -musl + // pin now fail loud with "no pinned asset" instead. + const musl = + process.platform === 'linux' && + (existsSync('/lib/ld-musl-x86_64.so.1') || existsSync('/lib/ld-musl-aarch64.so.1')) + return `${osKey}-${archKey}${musl ? '-musl' : ''}` } function sriSha512(buf: Buffer): string { @@ -534,6 +541,12 @@ REAL=$(PATH="$CLEAN_PATH" command -v '${cmd}' || true)` set -euo pipefail ${resolveReal} if [ -n "\${${sentinel}:-}" ] || [ -z "$REAL" ] || ! command -v sfw >/dev/null 2>&1; then + # Fail-open must not be SILENT-open: say so once on stderr when the + # firewall is missing (never on the sentinel re-entry path, where sfw + # itself is the caller). + if [ -z "\${${sentinel}:-}" ] && [ -n "$REAL" ]; then + echo "[sfw-shim] sfw not on PATH — running ${cmd} unfirewalled" >&2 + fi [ -n "$REAL" ] && exec "$REAL" "$@" echo "${cmd}: not found" >&2; exit 127 fi diff --git a/scripts/soak/paths.mts b/scripts/soak/paths.mts index 3b1b5955d0..c3707e7076 100644 --- a/scripts/soak/paths.mts +++ b/scripts/soak/paths.mts @@ -50,7 +50,9 @@ export const NPM_INSTALLERS: string[][] = [['pnpm', 'install']] // therefore the only one whose `cargo update` would honor the [unstable] // min-publish-age soak (nightly-only; inert on perry's stable toolchain, // where the automated window rides dependabot's cooldown instead). -export const RUSTUP_CARGO = path.join(os.homedir(), '.cargo/bin/cargo') +// CARGO_HOME-aware: rustup installs its shims under $CARGO_HOME/bin. +const CARGO_HOME = process.env.CARGO_HOME || path.join(os.homedir(), '.cargo') +export const RUSTUP_CARGO = path.join(CARGO_HOME, 'bin/cargo') // Pinned external tool manifest + the local tool rack it installs into: // exact versions under rack///, flat PATH handles in bin/. diff --git a/scripts/soak/soak.mts b/scripts/soak/soak.mts index f4b6c57b53..a6322bac89 100644 --- a/scripts/soak/soak.mts +++ b/scripts/soak/soak.mts @@ -198,7 +198,10 @@ export function parseExcludeEntries(body: string): ExcludeEntry[] { let blockIndent = 0 for (let i = 0; i < lines.length; i++) { const line = lines[i]! - if (/^minimumReleaseAgeExclude:\s*$/.test(line)) { + // Tolerate a trailing comment on the key line — without it, a stray + // `minimumReleaseAgeExclude: # note` never opened the block and every + // entry beneath silently escaped validation. + if (/^minimumReleaseAgeExclude:\s*(?:#.*)?$/.test(line)) { inBlock = true blockIndent = -1 continue @@ -248,7 +251,22 @@ export function checkCatalogParity( for (const m of block.matchAll(/^[ \t]+['"]?([^'":\s]+)['"]?:\s*['"]?([^'"\s]+)['"]?\s*$/gm)) { catalog[m[1]!] = m[2]! } - const pkg = JSON.parse(pkgJson) + let pkg: { dependencies?: Record; devDependencies?: Record } + try { + pkg = JSON.parse(pkgJson) + } catch { + // A broken package.json is a finding, not a stack trace — the gate + // must report every surface, not die on the first bad parse. + return [ + { + file: yamlFile, + what: 'catalog package.json parse', + saw: '(invalid JSON in the package.json beside the workspace yaml)', + wanted: 'parseable JSON so catalog lockstep is checkable', + fix: 'repair the package.json, then re-run', + }, + ] + } const declared: Record = { ...pkg.dependencies, ...pkg.devDependencies, diff --git a/scripts/soak/soak.test.mts b/scripts/soak/soak.test.mts index 1deeab9494..83b5b4bc05 100644 --- a/scripts/soak/soak.test.mts +++ b/scripts/soak/soak.test.mts @@ -148,6 +148,20 @@ test('toolchain soak: impossible calendar dates are findings, not crashes', () = assert.match(checkToolchainSoak(bad, 't')[0]!.what, /soak dates/) }) +test('parser: a trailing comment on the key line still opens the block', () => { + // Without comment tolerance, every entry under a commented key line + // silently escaped validation — a blind spot in the bypass gate. + const yaml = 'minimumReleaseAgeExclude: # temporary bypasses\n - lodash@4.17.21\n' + assert.deepEqual(parseExcludeEntries(yaml).map(e => e.name), ['lodash@4.17.21']) + assert.equal(checkExcludeAnnotations(yaml, 'y').length, 1) +}) + +test('catalog parity: malformed package.json is a finding, not a crash', () => { + const findings = checkCatalogParity('catalog:\n taze: 19.14.1\n', 'not json', 'y') + assert.equal(findings.length, 1) + assert.match(findings[0]!.what, /parse/) +}) + test('parser: a column-0 line ends the exclude block', () => { const yaml = 'minimumReleaseAgeExclude:\n - react\nonlyBuiltDependencies:\n - esbuild\n' assert.deepEqual(parseExcludeEntries(yaml).map(e => e.name), ['react']) diff --git a/tools/pnpm-workspace.yaml b/tools/pnpm-workspace.yaml index b53be9fbbd..2adcc80228 100644 --- a/tools/pnpm-workspace.yaml +++ b/tools/pnpm-workspace.yaml @@ -18,8 +18,10 @@ minimumReleaseAge: 10080 # Exclusions: bare names / scope globs express standing trust; version pins # are dated soak bypasses and REQUIRE, on the line above: # # published: YYYY-MM-DD | removable: YYYY-MM-DD (removable = published + 7d) -# `npm run soak` rejects unannotated or expired pins (and flow-style [..] -# lists, which the gate can't validate); `soak:fix` prunes expired ones. +# `npm run soak` rejects unannotated pins (and flow-style [..] lists, +# which the gate can't validate); expired ones warn until `soak:fix` / +# the soak-autofix workflow prunes them. Schematic example (use the REAL +# registry publish date, never these placeholders): # minimumReleaseAgeExclude: -# # published: 2026-01-01 | removable: 2026-01-08 +# # published: YYYY-MM-DD | removable: YYYY-MM-DD # - 'example@1.2.3' From d882d5afe7dfaf7249e76be8c8a1a833c6a4b266 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 02:26:12 -0400 Subject: [PATCH 12/30] fix(sfw): export SFW_UNKNOWN_HOST_ACTION=ignore in the shims Wheelhouse lesson: enterprise sfw defaults to BLOCK for non-registry hosts, which breaks ordinary dev flows (API calls, git clones) the day a SOCKET_SECURITY_KEY lands. Free tier hardcodes ignore and disregards the var, so setting it unconditionally is always safe. --- scripts/soak/external-tools.mts | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 2c474b50ab..fa092b71a3 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -551,6 +551,10 @@ if [ -n "\${${sentinel}:-}" ] || [ -z "$REAL" ] || ! command -v sfw >/dev/null 2 echo "${cmd}: not found" >&2; exit 127 fi export ${sentinel}=1 +# Enterprise sfw defaults to BLOCK for non-registry hosts, which breaks +# ordinary dev flows the day a Socket key lands; free tier hardcodes +# ignore and disregards the var, so setting it is always safe. +export SFW_UNKNOWN_HOST_ACTION=ignore exec sfw '${cmd}' "$@" ` // Remove the handle before writing: writeFileSync FOLLOWS a symlink, so From aa3e2f2a47a8c46f8fa4192795208d42a0224d83 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 09:06:15 -0400 Subject: [PATCH 13/30] fix(soak): take review fixes surfaced on the aube twin - checkDockerPrebake: parse the rustup install line's argument list instead of substring-matching the msrv (a multi-toolchain install line false-failed the check). - RUSTUP_CARGO resolves cargo.exe on win32. - soak-autofix: lease-checked force push (fetch the bot branch, then --force-with-lease) so a concurrent actor's commits are never clobbered. --- .github/workflows/soak-autofix.yml | 6 +++++- scripts/soak/external-tools.mts | 11 ++++++++++- scripts/soak/paths.mts | 6 +++++- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/.github/workflows/soak-autofix.yml b/.github/workflows/soak-autofix.yml index 2701ed3cf0..a9533c3e17 100644 --- a/.github/workflows/soak-autofix.yml +++ b/.github/workflows/soak-autofix.yml @@ -78,7 +78,11 @@ jobs: external-tools.mts --fix. Windows that cleared have soaked; their bypass annotations are dead weight the gates would otherwise fail on." - git push -f origin "$BRANCH" + # Lease-checked force push: fetch the bot branch first so the + # remote-tracking ref exists, then push only if nobody moved it + # since — a concurrent actor's commits are never clobbered. + git fetch origin "$BRANCH" 2>/dev/null || true + git push --force-with-lease origin "$BRANCH" if [ -z "$(gh pr list --head "$BRANCH" --state open --json number --jq '.[0].number')" ]; then gh pr create --head "$BRANCH" \ --title "chore(soak): prune expired soak annotations (automated)" \ diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index fa092b71a3..4f0ba3595e 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -206,7 +206,16 @@ export function checkDockerPrebake( if (shimList && shimList.join(' ') !== SFW_ECOSYSTEMS.join(' ')) { out.push(`docker prebake: shim list [${shimList.join(' ')}] != SFW_ECOSYSTEMS [${SFW_ECOSYSTEMS.join(' ')}]`) } - if (rustVersion && !dockerBody.includes(`toolchain install ${rustVersion}`)) { + // Parse the install line's full argument list rather than substring- + // matching: `rustup toolchain install 1.91.0 1.93.0` must satisfy an + // msrv of 1.93 even though "toolchain install 1.93" never appears. + const installedToolchains = [...dockerBody.matchAll(/toolchain install ([^\\\n]+)/g)] + .flatMap(m => m[1]!.trim().split(/\s+/)) + .filter(a => /^\d/.test(a)) + if ( + rustVersion && + !installedToolchains.some(t => t === rustVersion || t.startsWith(`${rustVersion}.`)) + ) { out.push(`docker prebake: image does not pre-install the ${rustVersion} msrv toolchain`) } const sfw = tools['sfw-free'] diff --git a/scripts/soak/paths.mts b/scripts/soak/paths.mts index c3707e7076..0b801e892c 100644 --- a/scripts/soak/paths.mts +++ b/scripts/soak/paths.mts @@ -52,7 +52,11 @@ export const NPM_INSTALLERS: string[][] = [['pnpm', 'install']] // where the automated window rides dependabot's cooldown instead). // CARGO_HOME-aware: rustup installs its shims under $CARGO_HOME/bin. const CARGO_HOME = process.env.CARGO_HOME || path.join(os.homedir(), '.cargo') -export const RUSTUP_CARGO = path.join(CARGO_HOME, 'bin/cargo') +export const RUSTUP_CARGO = path.join( + CARGO_HOME, + 'bin', + process.platform === 'win32' ? 'cargo.exe' : 'cargo', +) // Pinned external tool manifest + the local tool rack it installs into: // exact versions under rack///, flat PATH handles in bin/. From 3255e41d8d98c113ff17fda16a05cc96b2c9c877 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 09:11:20 -0400 Subject: [PATCH 14/30] docs(soak): align prose with warn-not-fail; source-cite the unknown-host comment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Same drift pullfrog flagged on the nub twin: the skill still said the gates "fail closed when a bypass window clears" — expired-but-valid annotations warn and get pruned by soak:fix / the soak-autofix workflow; invalid annotations are what fail. The shim comment now claims only what the source shows about SFW_UNKNOWN_HOST_ACTION (the enterprise config parses it; inert for free). --- .claude/skills/soak/SKILL.md | 8 +++++--- scripts/soak/external-tools.mts | 8 +++++--- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/.claude/skills/soak/SKILL.md b/.claude/skills/soak/SKILL.md index bc3275766a..babea0ede5 100644 --- a/.claude/skills/soak/SKILL.md +++ b/.claude/skills/soak/SKILL.md @@ -33,9 +33,11 @@ against it: shims into the dev-tools bin dir - `npm run test:scripts` — the scripts' own unit tests -The gates fail closed when a bypass window clears, but nobody has to -watch for that: the scheduled `soak-autofix` workflow runs `soak:fix` + -`tools:fix` daily and commits the pruning as a bot PR. +The gates fail closed on invalid states (missing, malformed, or +wrong-arithmetic annotations) and WARN on expired ones — stale is not +unsafe, and nobody has to watch for it: the scheduled `soak-autofix` +workflow runs `soak:fix` + `tools:fix` daily and commits the pruning +as a bot PR. A soak change is done when `npm run soak` and `npm run test:scripts` both exit 0 — the same gates CI runs. Re-run them after every fix. diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 4f0ba3595e..5d3627bef2 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -560,9 +560,11 @@ if [ -n "\${${sentinel}:-}" ] || [ -z "$REAL" ] || ! command -v sfw >/dev/null 2 echo "${cmd}: not found" >&2; exit 127 fi export ${sentinel}=1 -# Enterprise sfw defaults to BLOCK for non-registry hosts, which breaks -# ordinary dev flows the day a Socket key lands; free tier hardcodes -# ignore and disregards the var, so setting it is always safe. +# Enterprise sfw defaults to BLOCK for non-registry hosts +# (SFW_UNKNOWN_HOST_ACTION, parsed by the enterprise config), which +# breaks ordinary dev flows the day a Socket key lands. Only the +# enterprise build reads the var — it is inert for the free tier — so +# setting it unconditionally is safe. export SFW_UNKNOWN_HOST_ACTION=ignore exec sfw '${cmd}' "$@" ` From 3a8f00bcd73e2b676b657848fd6c2ce84d80b34b Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 09:24:48 -0400 Subject: [PATCH 15/30] fix(soak): never prune a wrong-arithmetic annotation as "cleared" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Greptile P1 on the aube twin: the pruners and stale lists accepted any valid-ISO annotation whose removable date had passed — including one whose removable was WRONG (earlier than published + SOAK_DAYS). Such an annotation must surface as the hard check failure it is; treating it as soaked would silently delete a bypass whose real window may still be open. All four surfaces (staleExcludes, fixWorkspaceYaml, staleBypasses, pruneExpiredSoakBypasses) now require the arithmetic to hold before an annotation counts as stale or prunable; regression tests cover the wrong-math-expired case. --- scripts/soak/external-tools.mts | 10 ++++++++++ scripts/soak/external-tools.test.mts | 9 +++++++++ scripts/soak/soak.mts | 15 ++++++++++++++- scripts/soak/soak.test.mts | 10 ++++++++++ 4 files changed, 43 insertions(+), 1 deletion(-) diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 5d3627bef2..7cf37d83fe 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -149,6 +149,11 @@ export function staleBypasses(tools: Record): string[] { if (!isValidIsoDate(bypass.published) || !isValidIsoDate(bypass.removable)) { continue } + // Wrong-arithmetic annotations are a hard checkPins failure, never + // stale/prunable — a too-early removable must not read as "cleared". + if (bypass.removable !== addDaysIso(bypass.published, SOAK_DAYS)) { + continue + } if (bypass.removable < today) { out.push(name) } @@ -176,6 +181,11 @@ export function pruneExpiredSoakBypasses(doc: { if (!isValidIsoDate(bypass.published) || !isValidIsoDate(bypass.removable)) { continue } + // Wrong-arithmetic annotations are a hard checkPins failure, never + // stale/prunable — a too-early removable must not read as "cleared". + if (bypass.removable !== addDaysIso(bypass.published, SOAK_DAYS)) { + continue + } if (bypass.removable < today) { delete pin.soakBypass pruned.push(name) diff --git a/scripts/soak/external-tools.test.mts b/scripts/soak/external-tools.test.mts index 78a8751a1c..a29f118d7b 100644 --- a/scripts/soak/external-tools.test.mts +++ b/scripts/soak/external-tools.test.mts @@ -83,7 +83,16 @@ test('pruneExpiredSoakBypasses prunes only valid, expired annotations', () => { unannotated: { version: '1.0.0', integrity: GOOD_SRI }, }, } + // Wrong-arithmetic + already-past removable: hard failure territory, + // never pruned or stale. + ;(doc.tools as Record)['wrongmath'] = { + version: '1.0.0', + integrity: GOOD_SRI, + soakBypass: { version: '1.0.0', published: todayIso(), removable: '2020-01-02' }, + } assert.deepEqual(pruneExpiredSoakBypasses(doc), ['expired']) + assert.ok('soakBypass' in (doc.tools as Record)['wrongmath']!) + assert.deepEqual(staleBypasses(doc.tools), []) assert.ok(doc.tools.fresh.soakBypass) assert.ok(!('soakBypass' in doc.tools.expired)) assert.ok(doc.tools.malformed.soakBypass) diff --git a/scripts/soak/soak.mts b/scripts/soak/soak.mts index a6322bac89..d2a3626ff7 100644 --- a/scripts/soak/soak.mts +++ b/scripts/soak/soak.mts @@ -180,6 +180,11 @@ export function staleExcludes(body: string): string[] { e.annotation && isValidIsoDate(e.annotation.published) && isValidIsoDate(e.annotation.removable) && + // Wrong-arithmetic annotations are NOT stale — they are a hard + // checkExcludeAnnotations failure a human must correct. Treating + // a too-early removable as "cleared" would prune a bypass whose + // real window may still be open. + e.annotation.removable === addDaysIso(e.annotation.published, SOAK_DAYS) && e.annotation.removable < today, ) .map(e => e.name) @@ -451,7 +456,15 @@ export function fixWorkspaceYaml(body: string): string { const lines = out.split('\n') const drop = new Set() for (const entry of parseExcludeEntries(out)) { - if (entry.annotation && entry.annotation.removable < today) { + // Prune only WELL-FORMED cleared annotations (same rule as + // staleExcludes): a wrong-arithmetic removable already in the past + // must surface as a check failure, not vanish silently. + if ( + entry.annotation && + isValidIsoDate(entry.annotation.published) && + entry.annotation.removable === addDaysIso(entry.annotation.published, SOAK_DAYS) && + entry.annotation.removable < today + ) { drop.add(entry.line - 1) if (ANNOTATION_RE.test(lines[entry.line - 2]?.trim() ?? '')) { drop.add(entry.line - 2) diff --git a/scripts/soak/soak.test.mts b/scripts/soak/soak.test.mts index 83b5b4bc05..9f9cd40f64 100644 --- a/scripts/soak/soak.test.mts +++ b/scripts/soak/soak.test.mts @@ -103,6 +103,16 @@ test('excludes: entries with trailing comments still parse', () => { assert.equal(checkExcludeAnnotations(yaml, 'y').length, 0) }) +test('fix and stale-list skip a wrong-arithmetic expired annotation', () => { + // published + SOAK_DAYS != removable and removable is already past: + // this must stay a check failure for a human, not silently prune — + // the real window may still be open. + const yaml = `minimumReleaseAge: 10080\nminimumReleaseAgeExclude:\n # published: ${todayIso()} | removable: 2020-01-02\n - 'wrongmath@1.0.0'\n` + assert.deepEqual(staleExcludes(yaml), []) + assert.ok(fixWorkspaceYaml(yaml).includes('wrongmath@1.0.0')) + assert.ok(checkExcludeAnnotations(yaml, 'y').length >= 1) +}) + test('fix prunes expired pins together with their annotations', () => { const yaml = `minimumReleaseAge: 10080\nminimumReleaseAgeExclude:\n # published: 2020-01-01 | removable: 2020-01-08\n - 'old@1.0.0'\n # published: ${FRESH_PUB} | removable: ${FRESH_REM}\n - 'fresh@1.0.0'\n` const fixed = fixWorkspaceYaml(yaml) From 20a26b005adbc359d43fecf34ab3eefba76a89b3 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 09:38:04 -0400 Subject: [PATCH 16/30] fix(soak): downloads fall back to unauthenticated and retry once on 5xx MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The nub node-18 compat leg died on `download failed 500` — the first authed fetch of a PUBLIC sfw release asset after GITHUB_TOKEN was added to the step env. Whether that 500 was token-induced (an Actions token against a cross-org public asset endpoint) or a transient GitHub blip, one attempt was too brittle: download() now retries without auth when an authed fetch fails (public assets need no credential), and once more after 2s on a 5xx. Regression test pins the fallback dropping the Authorization header. --- scripts/soak/external-tools.mts | 31 ++++++++++++++++++++-------- scripts/soak/external-tools.test.mts | 19 +++++++++++++++++ 2 files changed, 41 insertions(+), 9 deletions(-) diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 7cf37d83fe..8fb629a1af 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -263,20 +263,33 @@ export function checkDockerPrebake( } export async function download(url: string, expectedSri: string): Promise { - const headers: Record = {} // Only GitHub gets the token (private release assets); sending it to any // other host (e.g. the npm registry for purl tools) would leak the // credential. Cross-origin redirects strip the header automatically. - if (process.env.GITHUB_TOKEN && new URL(url).hostname === 'github.com') { - headers.authorization = `Bearer ${process.env.GITHUB_TOKEN}` - } + const token = + process.env.GITHUB_TOKEN && new URL(url).hostname === 'github.com' + ? process.env.GITHUB_TOKEN + : '' // Fail fast on a stalled release/registry response instead of hanging // CI; 120s is generous for the largest pinned binary on a slow runner. - const res = await fetch(url, { - headers, - redirect: 'follow', - signal: AbortSignal.timeout(120_000), - }) + const attempt = (withAuth: boolean) => + fetch(url, { + headers: withAuth && token ? { authorization: `Bearer ${token}` } : {}, + redirect: 'follow', + signal: AbortSignal.timeout(120_000), + }) + let res = await attempt(Boolean(token)) + // A token that a PUBLIC cross-repo asset endpoint rejects (or a + // transient GitHub 5xx — observed: a 500 on the first authed fetch of a + // public sfw asset) must not fail the install outright: retry once + // WITHOUT auth before giving up. Public assets need no credential. + if (!res.ok && token) { + res = await attempt(false) + } + if (!res.ok && res.status >= 500) { + await new Promise(r => setTimeout(r, 2_000)) + res = await attempt(false) + } if (!res.ok) { throw new Error(`download failed ${res.status} ${url}`) } diff --git a/scripts/soak/external-tools.test.mts b/scripts/soak/external-tools.test.mts index a29f118d7b..b4094bb9ed 100644 --- a/scripts/soak/external-tools.test.mts +++ b/scripts/soak/external-tools.test.mts @@ -191,6 +191,25 @@ test('download sends the GitHub token to github.com only', async t => { assert.equal(seen[1]!.auth, undefined) }) +test('download falls back to unauthenticated when the authed fetch fails', async t => { + const payload = Buffer.from('public-bytes') + const seen: Array = [] + t.mock.method(globalThis, 'fetch', async (_url: string | URL, init?: RequestInit) => { + const auth = (init?.headers as Record | undefined)?.authorization + seen.push(auth) + // Authed fetch is rejected (as a public cross-repo asset endpoint + // can); the unauthenticated retry succeeds. + return auth ? new Response('nope', { status: 500 }) : new Response(payload) + }) + await withEnv('GITHUB_TOKEN', 'ghs_test_token', async () => { + const got = await download('https://github.com/o/r/releases/download/v1/a', sriOf(payload)) + assert.deepEqual(got, payload) + }) + assert.equal(seen.length, 2) + assert.ok(seen[0]) + assert.equal(seen[1], undefined) +}) + test('download rejects http errors and integrity mismatches', async t => { const payload = Buffer.from('served-bytes') let status = 503 From c1eed51e3c51696b5ab07d233b39cae3617f2aa9 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 10:30:45 -0400 Subject: [PATCH 17/30] fix(soak): stop the fixers reformatting files they do not own MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adversarial self-review of the renovate/npmrc/yaml fixers, prompted by the 20-line diff my own soak:fix produced on aube's renovate.json: - fixRenovateConfig rewrote the WHOLE file via JSON.parse + re-stringify, collapsing hand-written single-line arrays and reformatting unrelated packageRules (aube's decmpfs musl hold among them). It is now a targeted text edit: only the minimumReleaseAge line changes, every other byte is preserved. A regression test asserts exactly one changed line and that the decmpfs rule survives verbatim. - The insert path produced INVALID JSON for a minimal `{}` config (`{,\n ...}`); guarded and covered by a test. - fixNpmrc / fixWorkspaceYaml matched trailing `\s*$` under /m — `\s` matches newlines, so the replacement swallowed blank lines after the key. Now `[ \t]*$`; verified soak:fix is a no-op on a clean tree. - checkRenovateConfig now also requires `internalChecksFilter: strict`. Without it renovate's default flexible mode raises updates that have NOT cleared minimumReleaseAge — the window silently stops biting. - The no-pinned-asset error names the musl case and lists the pinned platforms: sfw ships no musl asset, so an alpine runner hits this, and the old message gave nothing to act on. Verified alongside: decmpfs stays at 0.1.0 under `cargo update` (the `=0.1.0` requirement holds, so the soak updater cannot smuggle in the musl-breaking 0.1.2), and `--force-with-lease` correctly rejects a concurrent update even when the preceding fetch fails, and still creates the branch on a first run. --- scripts/soak/external-tools.mts | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 8fb629a1af..59382fe62e 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -359,9 +359,19 @@ export function linkHandle(target: string, name: string): void { } async function installAssetTool(name: string, pin: ToolPin): Promise { - const plat = pin.platforms?.[platformKey()] + const key = platformKey() + const plat = pin.platforms?.[key] if (!plat) { - throw new Error(`${name}: no pinned asset for ${platformKey()}`) + const available = Object.keys(pin.platforms ?? {}).join(', ') || '(none)' + // Name the musl case explicitly: several upstreams (sfw today) ship no + // musl asset, and the failure is otherwise a puzzle on an alpine + // runner. Failing loud beats installing a glibc binary that cannot + // run, but the message has to say what to do about it. + const muslHint = key.endsWith('-musl') + ? `\n ${name} publishes no musl asset. Either run this on a glibc host, ` + + `or add a ${key} entry to external-tools.json once upstream ships one.` + : '' + throw new Error(`${name}: no pinned asset for ${key} (pinned: ${available})${muslHint}`) } // A platform pinned to a registry .tgz (pnpm has no darwin-x64 SEA // upstream) routes through the npm-tarball path instead. From 88169c6072f19dced250900a918c51f392fca868 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 10:31:56 -0400 Subject: [PATCH 18/30] fix(soak): stop the fixers reformatting files they do not own MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adversarial self-review of the renovate/npmrc/yaml fixers, prompted by the 20-line diff my own soak:fix produced on aube's renovate.json: - fixRenovateConfig rewrote the WHOLE file via JSON.parse + re-stringify, collapsing hand-written single-line arrays and reformatting unrelated packageRules (aube's decmpfs musl hold among them). It is now a targeted text edit: only the minimumReleaseAge line changes, every other byte is preserved. A regression test asserts exactly one changed line and that the decmpfs rule survives verbatim. - The insert path produced INVALID JSON for a minimal `{}` config (`{,\n ...}`); guarded and covered by a test. - fixNpmrc / fixWorkspaceYaml matched trailing `\s*$` under /m — `\s` matches newlines, so the replacement swallowed blank lines after the key. Now `[ \t]*$`; verified soak:fix is a no-op on a clean tree. - checkRenovateConfig now also requires `internalChecksFilter: strict`. Without it renovate's default flexible mode raises updates that have NOT cleared minimumReleaseAge — the window silently stops biting. - The no-pinned-asset error names the musl case and lists the pinned platforms: sfw ships no musl asset, so an alpine runner hits this, and the old message gave nothing to act on. Verified alongside: decmpfs stays at 0.1.0 under `cargo update` (the `=0.1.0` requirement holds, so the soak updater cannot smuggle in the musl-breaking 0.1.2), and `--force-with-lease` correctly rejects a concurrent update even when the preceding fetch fails, and still creates the branch on a first run. --- scripts/soak/soak.mts | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/scripts/soak/soak.mts b/scripts/soak/soak.mts index d2a3626ff7..2a44238f9d 100644 --- a/scripts/soak/soak.mts +++ b/scripts/soak/soak.mts @@ -440,15 +440,19 @@ export function fixCargoConfig(body: string): string { } export function fixNpmrc(body: string): string { - if (/^min-release-age=\d+\s*$/m.test(body)) { - return body.replace(/^min-release-age=\d+\s*$/m, `min-release-age=${SOAK_DAYS}`) + // [ \t] not \s: `\s` matches newlines, so `\s*$` under /m swallowed the + // blank lines that follow the key (silent reformatting of the file). + if (/^min-release-age=\d+[ \t]*$/m.test(body)) { + return body.replace(/^min-release-age=\d+[ \t]*$/m, `min-release-age=${SOAK_DAYS}`) } return `${body.trimEnd()}\nmin-release-age=${SOAK_DAYS}\n` } export function fixWorkspaceYaml(body: string): string { + // [ \t] not \s on the trailing match: `\s*$` under /m consumes the + // newlines after the value, deleting following blank lines. let out = body.replace( - /^(minimumReleaseAge:\s*)\d+\s*$/m, + /^(minimumReleaseAge:[ \t]*)\d+[ \t]*$/m, `$1${SOAK_MINUTES}`, ) // Prune expired pins together with their annotation line. From ce95e5717314cf5048f28ace859659301a3e2c34 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 10:35:24 -0400 Subject: [PATCH 19/30] feat(soak): gate npm's min-release-age-exclude entries too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Auditing a sibling fleet repo (abitious) for compatibility surfaced an unguarded bypass: npm >= 11.17 has its OWN exclude surface, `min-release-age-exclude[]=`, parallel to pnpm's `minimumReleaseAgeExclude` block — and the gate validated only the pnpm side. `min-release-age-exclude[]=lodash@1.2.3` was therefore an unvalidated, never-expiring hole in exactly the rule the yaml side enforces. checkNpmrc now applies the same law to .npmrc: bare names and `@scope/*` globs are standing trust (the shape real repos use for trusted scopes, so this is not a churn tax), while a VERSION-PINNED entry needs the `# published: | removable:` annotation with correct arithmetic and real calendar dates. Tests cover trusted-glob, unannotated, correct, wrong-arithmetic, and impossible-date cases. --- scripts/soak/soak.mts | 73 ++++++++++++++++++++++++++++++++++---- scripts/soak/soak.test.mts | 25 +++++++++++++ 2 files changed, 91 insertions(+), 7 deletions(-) diff --git a/scripts/soak/soak.mts b/scripts/soak/soak.mts index 2a44238f9d..aae4eb4040 100644 --- a/scripts/soak/soak.mts +++ b/scripts/soak/soak.mts @@ -70,20 +70,79 @@ export function checkCargoConfig(body: string, file: string): Finding[] { return out } +/** + * npm has its OWN exclude surface — `min-release-age-exclude[]=` + * (npm >= 11.17) — parallel to pnpm's `minimumReleaseAgeExclude` block. + * The same rule applies: a bare name or `@scope/*` glob expresses standing + * trust, but a VERSION-PINNED entry is a dated bypass and needs the + * `# published: | removable:` annotation on the line above. Without this + * check, `min-release-age-exclude[]=lodash@1.2.3` was an unvalidated, + * never-expiring hole in exactly the gate the yaml side closes (found by + * auditing a fleet repo that uses this syntax heavily for trusted scopes). + */ +export function checkNpmrcExcludes(body: string, file: string): Finding[] { + const out: Finding[] = [] + const lines = body.split('\n') + for (let i = 0; i < lines.length; i++) { + const m = /^min-release-age-exclude\[\]\s*=\s*(\S+)\s*$/.exec(lines[i]!) + if (!m) { + continue + } + const spec = m[1]! + if (!VERSION_PIN_RE.test(spec)) { + // Bare name / scope glob: standing trust, no annotation needed. + continue + } + const ann = ANNOTATION_RE.exec(lines[i - 1]?.trim() ?? '') + if (!ann) { + out.push({ + file, + what: `npm soak exclude '${spec}' annotation`, + saw: '(no annotation on the line above)', + wanted: `# published: YYYY-MM-DD | removable: `, + fix: 'annotate the pin with its real registry publish date, or exclude the bare name for standing trust', + }) + continue + } + const [, published, removable] = ann as unknown as [string, string, string] + if (!isValidIsoDate(published) || !isValidIsoDate(removable)) { + out.push({ + file, + what: `npm soak exclude '${spec}' annotation dates`, + saw: `${published} | ${removable}`, + wanted: 'real YYYY-MM-DD calendar dates', + fix: 'correct the annotation to the real registry publish date', + }) + continue + } + const expected = addDaysIso(published, SOAK_DAYS) + if (removable !== expected) { + out.push({ + file, + what: `npm soak exclude '${spec}' removable date`, + saw: removable, + wanted: `${expected} (published ${published} + ${SOAK_DAYS} days)`, + fix: 'correct the removable date', + }) + } + } + return out +} + export function checkNpmrc(body: string, file: string): Finding[] { + const out: Finding[] = [] const days = /^min-release-age=(\d+)\s*$/m.exec(body)?.[1] - if (Number(days) === SOAK_DAYS) { - return [] - } - return [ - { + if (Number(days) !== SOAK_DAYS) { + out.push({ file, what: 'npm min-release-age window', saw: days ?? '(missing)', wanted: String(SOAK_DAYS), fix: `set min-release-age=${SOAK_DAYS} (or run --fix)`, - }, - ] + }) + } + out.push(...checkNpmrcExcludes(body, file)) + return out } export function checkWorkspaceYaml(body: string, file: string): Finding[] { diff --git a/scripts/soak/soak.test.mts b/scripts/soak/soak.test.mts index 9f9cd40f64..380fbc4df9 100644 --- a/scripts/soak/soak.test.mts +++ b/scripts/soak/soak.test.mts @@ -10,6 +10,7 @@ import { checkDependabotCooldown, checkExcludeAnnotations, checkNpmrc, + checkNpmrcExcludes, checkTazeConfig, checkToolchainSoak, checkWorkspaceYaml, @@ -53,6 +54,30 @@ test('npmrc: window must match SOAK_DAYS and fix writes it', () => { assert.match(fixNpmrc('min-release-age=3\n'), /min-release-age=7/) }) +test('npmrc excludes: version pins need dated annotations, globs do not', () => { + // The shape a fleet repo actually uses: trusted scopes and bare names + // are standing trust and need no annotation. + const trusted = [ + 'min-release-age=7', + 'min-release-age-exclude[]=@socketsecurity/*', + 'min-release-age-exclude[]=sfw', + ].join('\n') + assert.deepEqual(checkNpmrcExcludes(trusted, 'n'), []) + + // A VERSION-PINNED exclude is a dated bypass — unannotated is a finding. + const unannotated = 'min-release-age-exclude[]=lodash@4.17.21\n' + assert.match(checkNpmrcExcludes(unannotated, 'n')[0]!.what, /lodash@4\.17\.21/) + + // Correctly annotated passes; wrong arithmetic is a finding. + const pub = addDaysIso(todayIso(), -1) + const ok = `# published: ${pub} | removable: ${addDaysIso(pub, SOAK_DAYS)}\nmin-release-age-exclude[]=lodash@4.17.21\n` + assert.deepEqual(checkNpmrcExcludes(ok, 'n'), []) + const wrongMath = `# published: ${pub} | removable: ${addDaysIso(pub, 3)}\nmin-release-age-exclude[]=lodash@4.17.21\n` + assert.match(checkNpmrcExcludes(wrongMath, 'n')[0]!.what, /removable date/) + const badDates = `# published: 2026-13-45 | removable: 2026-13-52\nmin-release-age-exclude[]=lodash@4.17.21\n` + assert.match(checkNpmrcExcludes(badDates, 'n')[0]!.what, /annotation dates/) +}) + test('workspace yaml: clean fixture passes', () => { assert.deepEqual(checkWorkspaceYaml(CLEAN_YAML, 'y'), []) }) From b67907d8ba04b2e196efb3b52638959ae69150ca Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 10:38:16 -0400 Subject: [PATCH 20/30] fix(soak): fail loudly when cargo silently ignores min-publish-age MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verified rather than assumed, and the assumption was wrong: cargo treats an [unstable] key it does not implement as a WARNING ("unused config key `unstable.min-publish-age`") and exits 0. Measured on nightly 2026-03-21, which has no such -Z — so `cargo +nightly update` on a merely-OLD nightly resolved every crate with NO window at all while the run reported success. The tooling was claiming a protection it had not applied. updateCargo now captures stderr and treats that warning as a hard failure: the lockfile changes are unsoaked, so say so and exit nonzero with the fix (`rustup update nightly`). The detector is an exported, unit-tested predicate pinning cargo's exact wording. perry rides stable, where the key is expected to be inert, so there the same detection downgrades to an explicit note naming dependabot cooldown as the enforcing surface for cargo deps — no silent no-op either way. --- scripts/soak/update-deps.mts | 36 ++++++++++++++++++++++++++++++- scripts/soak/update-deps.test.mts | 32 +++++++++++++++++++++++++++ 2 files changed, 67 insertions(+), 1 deletion(-) create mode 100644 scripts/soak/update-deps.test.mts diff --git a/scripts/soak/update-deps.mts b/scripts/soak/update-deps.mts index 37237b058b..ed45058f6a 100644 --- a/scripts/soak/update-deps.mts +++ b/scripts/soak/update-deps.mts @@ -67,7 +67,41 @@ function updateCargo(dryRun: boolean): number { console.error('[update-deps] rustup cargo shim not found — refusing a cargo that cannot follow the repo toolchain') return 1 } - return run(RUSTUP_CARGO, dryRun ? ['update', '--dry-run'] : ['update'], REPO_ROOT) + const args = dryRun ? ['update', '--dry-run'] : ['update'] + // Report honestly when the cargo-side window did not apply. perry rides + // stable, where `[unstable] min-publish-age` is a warning-only unused + // key — so this path is expected to say "no soak applied" today, and the + // automated window rides dependabot's cooldown instead. It stops being a + // silent no-op the day the repo moves to a nightly that implements it. + console.log(`[update-deps] ${RUSTUP_CARGO} ${args.join(' ')} (in .)`) + const res = spawnSync(RUSTUP_CARGO, args, { + cwd: REPO_ROOT, + encoding: 'utf8', + stdio: ['inherit', 'inherit', 'pipe'], + }) + const stderr = res.stderr ?? '' + process.stderr.write(stderr) + if (res.error) { + console.error(`[update-deps] ${RUSTUP_CARGO}: ${res.error.message}`) + return 1 + } + if (isMinPublishAgeUnsupported(stderr)) { + console.warn( + '[update-deps] note: cargo ignored [unstable] min-publish-age (stable toolchain),\n' + + ' so crate versions were NOT soak-gated here — dependabot cooldown is the\n' + + ' enforcing surface for cargo deps in this repo.', + ) + } + return res.status ?? 1 +} + +/** + * cargo emits `unused config key ...` (a warning, exit 0) for an + * `[unstable]` key it does not implement, so the ONLY signal that the soak + * silently did not apply is this line on stderr. Exported for the tests. + */ +export function isMinPublishAgeUnsupported(stderr: string): boolean { + return /unused config key `unstable\.min-publish-age`/.test(stderr) } // No flag = both; naming both explicitly also means both — a naive diff --git a/scripts/soak/update-deps.test.mts b/scripts/soak/update-deps.test.mts new file mode 100644 index 0000000000..5d5e545aec --- /dev/null +++ b/scripts/soak/update-deps.test.mts @@ -0,0 +1,32 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' + +import { isMinPublishAgeUnsupported, selectEcosystems } from './update-deps.mts' + +test('no ecosystem flag updates both', () => { + assert.deepEqual(selectEcosystems([]), { npm: true, cargo: true }) + assert.deepEqual(selectEcosystems(['--dry-run']), { npm: true, cargo: true }) +}) + +test('a single flag selects only that ecosystem', () => { + assert.deepEqual(selectEcosystems(['--npm']), { npm: true, cargo: false }) + assert.deepEqual(selectEcosystems(['--cargo']), { npm: false, cargo: true }) +}) + +// Regression: a naive "flag present = only that one" reading made +// `--npm --cargo` run NEITHER ecosystem. +test('naming both flags updates both', () => { + assert.deepEqual(selectEcosystems(['--npm', '--cargo']), { npm: true, cargo: true }) +}) + +// The cargo soak is a warning-only unused key on any cargo that does not +// implement it, so this string is the ONLY evidence it silently did not +// apply. Pin the exact wording cargo emits. +test('detects the unused-config-key warning that means the cargo soak did not apply', () => { + const real = + 'warning: unused config key `unstable.min-publish-age` in `/repo/.cargo/config.toml`\n' + assert.equal(isMinPublishAgeUnsupported(real), true) + assert.equal(isMinPublishAgeUnsupported('warning: unused config key `unstable.other`\n'), false) + assert.equal(isMinPublishAgeUnsupported(''), false) + assert.equal(isMinPublishAgeUnsupported(' Updating crates.io index\n'), false) +}) From c3d27e0e34fba66439eaa0b5346fa1ac8da1c9f8 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 10:41:46 -0400 Subject: [PATCH 21/30] feat(soak): explain a window-blocked cargo re-resolution, refuse the env bypass MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-measured on a current nightly (2026-07-27, cargo 1.99.0-nightly): the `-Z min-publish-age` feature IS implemented there and the window visibly bites — it holds a too-fresh release back ("available: v0.2.189, published 7 days ago"). Both measurements are now recorded in the comment and the skill, since the OLD nightly (2026-03-21) is the evidence that a stale toolchain skips the window silently. Running the real updater surfaced the other half of the contract: the window can make re-resolution IMPOSSIBLE, not just conservative. When a requirement's only candidate is inside the window (aube today: `clap_usage = "^4"`, whose 4.0.0 shipped 3 days ago) cargo fails the whole update — correct behavior, but its own help line advertises `CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE=allow`, a blanket env-var bypass this design deliberately does not have. The updater now detects that failure and prints ordered options (wait it out, repin so a soaked version satisfies the requirement, or adopt the fresh release as a reviewable commit) with an explicit warning against the env bypass. Predicate is exported and unit-tested against cargo's real wording. --- scripts/soak/update-deps.mts | 30 ++++++++++++++++++++++++++++++ scripts/soak/update-deps.test.mts | 18 +++++++++++++++++- 2 files changed, 47 insertions(+), 1 deletion(-) diff --git a/scripts/soak/update-deps.mts b/scripts/soak/update-deps.mts index ed45058f6a..f557338e98 100644 --- a/scripts/soak/update-deps.mts +++ b/scripts/soak/update-deps.mts @@ -85,6 +85,27 @@ function updateCargo(dryRun: boolean): number { console.error(`[update-deps] ${RUSTUP_CARGO}: ${res.error.message}`) return 1 } + // The window can make re-resolution IMPOSSIBLE rather than merely + // holding a version back: if a requirement's only matching release is + // younger than the window (e.g. `pkg = "^4"` when 4.0.0 shipped 3 days + // ago), cargo fails the whole update. That is the soak doing its job, + // but cargo's own help line advertises + // CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE=allow — a blanket env-var + // bypass this design deliberately does not have. Say so before someone + // copy-pastes it out of a red terminal. + if (isBlockedByPublishAge(stderr)) { + console.error( + '[update-deps] the cargo soak BLOCKED this re-resolution: a requirement can\n' + + ' only be satisfied by a release younger than the window (see the error above).\n' + + ' This is the window working, not a bug. Options, in order of preference:\n' + + ' 1. wait out the remaining days and re-run;\n' + + ' 2. relax/repin the requirement so an already-soaked version satisfies it;\n' + + ' 3. if the fresh release is genuinely required, adopt it as a deliberate,\n' + + ' reviewable commit — NOT via CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE,\n' + + ' which silently disables the window for every crate in the graph.', + ) + return res.status ?? 1 + } if (isMinPublishAgeUnsupported(stderr)) { console.warn( '[update-deps] note: cargo ignored [unstable] min-publish-age (stable toolchain),\n' + @@ -104,6 +125,15 @@ export function isMinPublishAgeUnsupported(stderr: string): boolean { return /unused config key `unstable\.min-publish-age`/.test(stderr) } +/** + * cargo's resolver failure when a requirement's only candidate is inside + * the window: `version X is too new (published N days ago, minimum age M + * days)`. Exported for the tests; pins cargo's wording. + */ +export function isBlockedByPublishAge(stderr: string): boolean { + return /is too new \(published .*minimum age/.test(stderr) +} + // No flag = both; naming both explicitly also means both — a naive // "flag present = only that one" reading once made `--npm --cargo` run // NEITHER, so this rule lives in one exported, regression-tested place. diff --git a/scripts/soak/update-deps.test.mts b/scripts/soak/update-deps.test.mts index 5d5e545aec..7a05973eda 100644 --- a/scripts/soak/update-deps.test.mts +++ b/scripts/soak/update-deps.test.mts @@ -1,7 +1,11 @@ import assert from 'node:assert/strict' import { test } from 'node:test' -import { isMinPublishAgeUnsupported, selectEcosystems } from './update-deps.mts' +import { + isBlockedByPublishAge, + isMinPublishAgeUnsupported, + selectEcosystems, +} from './update-deps.mts' test('no ecosystem flag updates both', () => { assert.deepEqual(selectEcosystems([]), { npm: true, cargo: true }) @@ -30,3 +34,15 @@ test('detects the unused-config-key warning that means the cargo soak did not ap assert.equal(isMinPublishAgeUnsupported(''), false) assert.equal(isMinPublishAgeUnsupported(' Updating crates.io index\n'), false) }) + +// The other half of the cargo-window contract: the resolver can fail +// outright when a requirement's only candidate is too fresh. Pin cargo's +// wording (captured from a real run) so the guidance keeps firing. +test('detects the resolver failure that means the window blocked re-resolution', () => { + const real = + 'error: failed to select a version for the requirement `clap_usage = "^4"`\n' + + ' version 4.0.0 is too new (published 3 days ago, minimum age 7 days)\n' + assert.equal(isBlockedByPublishAge(real), true) + assert.equal(isBlockedByPublishAge('error: failed to select a version\n'), false) + assert.equal(isBlockedByPublishAge(''), false) +}) From 369dadf8a34bc858733489a0c14b03c3ff94e667 Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 10:53:47 -0400 Subject: [PATCH 22/30] fix(soak): take the adversarial-review findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An independent hostile review of the three sibling PRs found real defects, including one where my own test had verified only the safe half of the case: - soak-autofix no longer force-pushes at all. `git fetch origin $BRANCH` UPDATES the remote-tracking ref (actions/checkout leaves the default wildcard refspec), so the following --force-with-lease took its lease against whatever another actor had just pushed and overwrote it — the classic fetch-before-lease anti-pattern, and the inline comment asserting "a concurrent actor's commits are never clobbered" was false. Demonstrated: a human commit onto the open autofix PR was discarded by the next scheduled run. My earlier test only covered the fetch-FAILS path (which is genuinely safe, rejecting with "stale info"). The step now stashes the fixes, bases the work on the existing bot branch when there is one, and plain-pushes: human commits survive by construction, an empty re-run exits 0 instead of pushing a no-op commit, and a genuine conflict fails loudly instead of being resolved by deletion. - fixWorkspaceYaml's prune set must EQUAL staleExcludes' warn set: it was missing the VERSION_PIN_RE guard, so a bare-name / `@scope/*` standing-trust entry sitting under an expired annotation line was deleted by --fix, silently re-arming the soak for a whole scope inside a bot commit advertised as touching only annotation lines. - download() retry semantics split by meaning: 401/403/404 with a token means the credential is the problem (retry unauthenticated), >=500 is transient (retry with the SAME auth). Dropping auth on 5xx made a private asset 404 on the retry, report a bogus "download failed 404", and never be able to succeed. The SRI is verified either way, so no retry can substitute a different artifact. --- .github/workflows/soak-autofix.yml | 38 +++++++++++++++++++++++----- scripts/soak/external-tools.mts | 18 ++++++++----- scripts/soak/external-tools.test.mts | 26 ++++++++++++++++--- scripts/soak/soak.mts | 6 +++++ 4 files changed, 73 insertions(+), 15 deletions(-) diff --git a/.github/workflows/soak-autofix.yml b/.github/workflows/soak-autofix.yml index a9533c3e17..a5199428ca 100644 --- a/.github/workflows/soak-autofix.yml +++ b/.github/workflows/soak-autofix.yml @@ -63,6 +63,7 @@ jobs: env: GH_TOKEN: ${{ secrets.SOAK_AUTOFIX_TOKEN || github.token }} run: | + set -euo pipefail if git diff --quiet; then echo "soak surfaces clean — nothing to fix" exit 0 @@ -70,19 +71,44 @@ jobs: git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" BRANCH="bot/soak-autofix" - git checkout -B "$BRANCH" + # NO force push. The previous fetch-then---force-with-lease shape + # was security theater: `git fetch origin $BRANCH` updates the + # remote-tracking ref to the branch's CURRENT tip (checkout leaves + # the default wildcard refspec in place), so the lease is taken + # against whatever another actor just pushed and the force + # succeeds — silently discarding, for example, a human's review + # fixes committed onto the open autofix PR. + # + # Instead: stash the mechanical fixes, BASE the work on the + # existing bot branch when there is one, and fast-forward push. + # Human commits on the branch are preserved by construction, and + # a genuine conflict fails the run instead of being resolved by + # deletion. + git stash push --include-untracked -m soak-autofix + if git fetch origin "$BRANCH"; then + git checkout -B "$BRANCH" "origin/$BRANCH" + else + echo "no existing $BRANCH on origin — creating it" + git checkout -B "$BRANCH" + fi + if ! git stash pop; then + echo "::error::soak fixes conflict with the existing $BRANCH; resolve that branch (or close its PR) and re-run" + exit 1 + fi git add -A + # Re-running the fixers on top of an already-fixed branch is a + # no-op; say so and stop rather than pushing an empty commit. + if git diff --cached --quiet; then + echo "$BRANCH already carries these fixes — nothing to push" + exit 0 + fi git commit -m "chore(soak): prune expired soak annotations (automated) Generated by the soak-autofix workflow: soak.mts --fix + external-tools.mts --fix. Windows that cleared have soaked; their bypass annotations are dead weight the gates would otherwise fail on." - # Lease-checked force push: fetch the bot branch first so the - # remote-tracking ref exists, then push only if nobody moved it - # since — a concurrent actor's commits are never clobbered. - git fetch origin "$BRANCH" 2>/dev/null || true - git push --force-with-lease origin "$BRANCH" + git push origin "$BRANCH" if [ -z "$(gh pr list --head "$BRANCH" --state open --json number --jq '.[0].number')" ]; then gh pr create --head "$BRANCH" \ --title "chore(soak): prune expired soak annotations (automated)" \ diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 59382fe62e..214a27b81c 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -279,16 +279,22 @@ export async function download(url: string, expectedSri: string): Promise=500 — transient. Retry with the SAME auth: dropping it here made a + // private asset (sfw-enterprise) 404 on the retry, reporting a bogus + // "download failed 404" and guaranteeing the retry could never + // succeed. + // The URL is fixed and the SRI is verified below either way, so no retry + // can substitute a different artifact. + if (!res.ok && token && [401, 403, 404].includes(res.status)) { res = await attempt(false) } if (!res.ok && res.status >= 500) { await new Promise(r => setTimeout(r, 2_000)) - res = await attempt(false) + res = await attempt(Boolean(token)) } if (!res.ok) { throw new Error(`download failed ${res.status} ${url}`) diff --git a/scripts/soak/external-tools.test.mts b/scripts/soak/external-tools.test.mts index b4094bb9ed..52a71f88a0 100644 --- a/scripts/soak/external-tools.test.mts +++ b/scripts/soak/external-tools.test.mts @@ -191,15 +191,35 @@ test('download sends the GitHub token to github.com only', async t => { assert.equal(seen[1]!.auth, undefined) }) +test('download keeps auth across a 5xx retry, drops it only on 401/403/404', async t => { + const payload = Buffer.from('private-bytes') + const seen: Array = [] + let calls = 0 + t.mock.method(globalThis, 'fetch', async (_url: string | URL, init?: RequestInit) => { + seen.push((init?.headers as Record | undefined)?.authorization) + calls += 1 + // First attempt: transient 500. Retry must still carry the token, or a + // private asset would 404 and never recover. + return calls === 1 ? new Response('boom', { status: 500 }) : new Response(payload) + }) + await withEnv('GITHUB_TOKEN', 'ghs_test_token', async () => { + const got = await download('https://github.com/o/r/releases/download/v1/a', sriOf(payload)) + assert.deepEqual(got, payload) + }) + assert.equal(seen.length, 2) + assert.ok(seen[0], 'first attempt is authed') + assert.ok(seen[1], 'the 5xx retry stays authed') +}) + test('download falls back to unauthenticated when the authed fetch fails', async t => { const payload = Buffer.from('public-bytes') const seen: Array = [] t.mock.method(globalThis, 'fetch', async (_url: string | URL, init?: RequestInit) => { const auth = (init?.headers as Record | undefined)?.authorization seen.push(auth) - // Authed fetch is rejected (as a public cross-repo asset endpoint - // can); the unauthenticated retry succeeds. - return auth ? new Response('nope', { status: 500 }) : new Response(payload) + // Authed fetch is rejected 404 (as a public cross-repo asset endpoint + // can when handed an Actions token); the unauthenticated retry wins. + return auth ? new Response('nope', { status: 404 }) : new Response(payload) }) await withEnv('GITHUB_TOKEN', 'ghs_test_token', async () => { const got = await download('https://github.com/o/r/releases/download/v1/a', sriOf(payload)) diff --git a/scripts/soak/soak.mts b/scripts/soak/soak.mts index aae4eb4040..eab961546a 100644 --- a/scripts/soak/soak.mts +++ b/scripts/soak/soak.mts @@ -522,8 +522,14 @@ export function fixWorkspaceYaml(body: string): string { // Prune only WELL-FORMED cleared annotations (same rule as // staleExcludes): a wrong-arithmetic removable already in the past // must surface as a check failure, not vanish silently. + // VERSION_PIN_RE too: the prune set must EQUAL the warn set + // (staleExcludes). Without it a bare-name / `@scope/*` standing-trust + // entry that merely sits under an expired annotation line was deleted + // by --fix — silently re-arming the soak for a whole scope, in a bot + // commit whose review story is "only annotation lines are touched". if ( entry.annotation && + VERSION_PIN_RE.test(entry.name) && isValidIsoDate(entry.annotation.published) && entry.annotation.removable === addDaysIso(entry.annotation.published, SOAK_DAYS) && entry.annotation.removable < today From c1ff686dfdc51b568cf960567dcac0b3c69ed27f Mon Sep 17 00:00:00 2001 From: jdalton Date: Tue, 28 Jul 2026 16:12:54 -0400 Subject: [PATCH 23/30] =?UTF-8?q?fix(soak):=20take=20the=20review=20findin?= =?UTF-8?q?gs=20=E2=80=94=20one=20is=20a=20regression=20I=20introduced?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - fixRenovateConfig still matched a trailing `\s*$`, which under /m eats the NEWLINES after the value: replacing through it silently deleted the blank line that followed. That is the exact defect the same commit fixed in fixNpmrc and fixWorkspaceYaml, kept in the third fixer. Verified with a config carrying a blank line after the key: it disappeared before, survives now. - checkPins now rejects a soakBypass whose `version` is not the version actually pinned. Bump a pin and leave the annotation behind and the ledger vouches for a release that is no longer installed — "1.13.1 was adopted early" while 1.14.0 ships unreviewed. A mismatch is unauditable, so it is a hard finding, not a stale-annotation warning. - soak-autofix.yml's header still described the gates as failing closed on a cleared window; the fourth and last sibling of that stale premise. Expired is a warning, invalid still fails, and the workflow's job is convergence rather than rescue. - Two paths were changed without a test covering them, both added: the multi-arg `rustup toolchain install 1.91.0 1.93.0` case that motivated replacing the substring msrv match (only the negative case was covered), and the `>= 500` retry branch that the retry commit is named for (the existing test exercises only the auth fallback). --- .github/workflows/soak-autofix.yml | 13 +++++---- scripts/soak/external-tools.mts | 10 +++++++ scripts/soak/external-tools.test.mts | 43 ++++++++++++++++++++++++++++ 3 files changed, 60 insertions(+), 6 deletions(-) diff --git a/.github/workflows/soak-autofix.yml b/.github/workflows/soak-autofix.yml index a5199428ca..c9eedf247c 100644 --- a/.github/workflows/soak-autofix.yml +++ b/.github/workflows/soak-autofix.yml @@ -1,11 +1,12 @@ name: soak-autofix -# The soak gates fail CLOSED by design: an expired soakBypass annotation or -# a cleared minimumReleaseAgeExclude pin turns tools:check / soak red until -# someone prunes it. This workflow does the pruning automatically — daily it -# runs the fixers, and when they change anything it commits to a bot branch -# and opens (or updates) a PR, so the gate never sits red waiting for a -# human to delete two lines. +# An expired soakBypass annotation or a cleared minimumReleaseAgeExclude pin +# is STALE, not unsafe: the version has soaked, so the gates warn rather than +# fail. Nobody should have to watch for that warning either — this workflow +# runs the fixers daily and, when they change anything, commits to a bot +# branch and opens (or updates) a PR, so the ledger converges to clean on its +# own. Invalid annotations (missing, malformed, wrong arithmetic) still fail +# the gate; those need a human. on: schedule: diff --git a/scripts/soak/external-tools.mts b/scripts/soak/external-tools.mts index 214a27b81c..f59b3fcb13 100644 --- a/scripts/soak/external-tools.mts +++ b/scripts/soak/external-tools.mts @@ -115,6 +115,16 @@ export function checkPins(tools: Record): string[] { } if (pin.soakBypass) { const { published, removable } = pin.soakBypass + // A bypass names the version it was granted for. Bump the pin and + // leave the annotation behind and it now vouches for a version that + // is no longer installed — the ledger says "1.13.1 was adopted early" + // while 1.14.0 ships unreviewed. Mismatch is a hard finding, not a + // stale-annotation warning. + if (pin.soakBypass.version !== pin.version) { + out.push( + `${name}: soakBypass is for ${pin.soakBypass.version} but the pin is ${pin.version} — re-date the annotation for the version actually pinned, or drop it`, + ) + } if (!isValidIsoDate(published) || !isValidIsoDate(removable)) { out.push(`${name}: soakBypass dates are not real YYYY-MM-DD calendar dates`) continue diff --git a/scripts/soak/external-tools.test.mts b/scripts/soak/external-tools.test.mts index 52a71f88a0..458db20c43 100644 --- a/scripts/soak/external-tools.test.mts +++ b/scripts/soak/external-tools.test.mts @@ -173,6 +173,49 @@ test('checkDockerPrebake flags every drift class (synthetic image)', () => { ) }) +// The reason the arg-list parse replaced a substring match: a multi-arg +// install line must SATISFY an msrv it contains. Only the negative case was +// covered before, so the fix itself was untested. +test('checkDockerPrebake accepts an msrv satisfied by a later install arg', () => { + const tools = { + 'sfw-free': { + version: '1.0.0', + platforms: { 'linux-arm64': { asset: 'sfw-linux-arm64', integrity: GOOD_SRI } }, + }, + } + const body = [ + 'for cmd in npm yarn pnpm pip pip3 uv cargo; do make_shim "$cmd"; done', + 'RUN rustup toolchain install 1.91.0 1.93.0 --profile minimal', + 'RUN curl -o /x https://github.com/SocketDev/sfw-free/releases/download/v1.0.0/sfw-linux-arm64', + 'COPY rack/sfw-free/1.0.0/sfw /usr/local/bin/sfw', + `RUN asset=sfw-linux-arm64; sha=${'0'.repeat(128)} verify`, + ].join('\n') + // 1.93 is the SECOND argument — a substring match for + // "toolchain install 1.93" would miss it and false-fail. + const problems = checkDockerPrebake(body, tools, '', '1.93') + assert.equal(problems.some(p => /msrv toolchain/.test(p)), false) + // And a version the line does NOT install is still reported. + assert.ok(checkDockerPrebake(body, tools, '', '1.99').some(p => /msrv toolchain/.test(p))) +}) + +// The 5xx branch is the one this retry logic is named for; the auth +// fallback test above does not reach it. +test('download retries a 5xx and succeeds on the second attempt', async t => { + const payload = Buffer.from('after-5xx') + let calls = 0 + t.mock.method(globalThis, 'fetch', async () => { + calls += 1 + return calls === 1 ? new Response('boom', { status: 503 }) : new Response(payload) + }) + await withEnv('GITHUB_TOKEN', undefined, async () => { + assert.deepEqual( + await download('https://example.com/a', sriOf(payload)), + payload, + ) + }) + assert.equal(calls, 2) +}) + test('download sends the GitHub token to github.com only', async t => { const payload = Buffer.from('pinned-bytes') const seen: Array<{ host: string; auth: string | undefined }> = [] From f8c6bafe1ec1e06557aab5e07c4dfa479f81620c Mon Sep 17 00:00:00 2001 From: jdalton Date: Wed, 29 Jul 2026 14:17:30 -0400 Subject: [PATCH 24/30] fix(compile): survive binary/workspace skew and complete the surfaces a real npm CLI needs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Compiling Socket Firewall (sfw — a TLS-MITM proxy CLI with undici, node-forge, iovalkey, zod, … in its graph) end-to-end surfaced four independent blockers. Fixed here: 1. auto-optimize feature skew (driver.rs / freshness.rs): the perry binary's baked-in cross-feature list tracks the branch it was BUILT from, but the auto-optimize cargo build resolves against the checkout on disk. One unknown `perry-runtime/` failed the whole resolve, and the silent prebuilt fallback linked without the routed ext-pump entrypoints — undefined-js_* errors two stages from the cause. New retain_workspace_declared_features() drops names the checkout's perry-runtime / perry-stdlib don't declare (features table + optional deps, fail-open on unreadable manifests) before the build stamp is computed, and the cargo-failure fallback now says what the consequence and remedy are. 2. perry-ext-zlib zstd surface: undici's web-fetch content decoding references js_zlib_create_zstd_decompress unconditionally, but only perry-stdlib's `compression` module carried the zstd codecs — and routing node:zlib to the ext archive strips that feature. Port the full surface (create factories, sync/async one-shots, streaming write-codec via zstd::stream::write) so the routed archive is self-sufficient. 3. class X extends DOMException (codegen + runtime): undici probes DOMException inheritability at module load (websocketerror.js), and the name was neither in the builtin-parent list nor backed by a subclass initializer — the compiled binary died at startup with 'DOMException is not a function'. Add js_dom_exception_subclass_init (stamps message/name/code onto the subclass instance) wired through both the explicit super() lowering and the implicit-ctor NativeInstanceBase chain walk. 4. panic-runtime dedup for prebuilt (panic=unwind) wrappers co-linked with a panic=abort auto-optimized stdlib (strip_dedup.rs): the name-containment rule never nominated the wrapper's panic_unwind member (stdlib bundles panic_abort under a different name), and the localize pass severed the std-cgu → panic_unwind __rust_drop_panic edge that abort stdlibs cannot re-provide. Nominate panic_unwind in the nosharedeps fixed-point (protected exactly when the stdlib can't cover it), and skip localizing panic symbols a sibling member still references. Allocator shims stay always-localized: leaving the wrapper's system-malloc shim global beats the runtime's mimalloc at link and breaks pointer classification (silent console loss). With these, sfw and sfw-free compile, link, and run as native arm64 binaries straight from their TypeScript entrypoints. --- Cargo.lock | 1 + .../perry-codegen/src/expr/this_super_call.rs | 32 ++++++ .../src/lower_call/new_helpers.rs | 19 ++++ .../src/runtime_decls/strings_part2.rs | 7 ++ crates/perry-ext-zlib/Cargo.toml | 6 ++ crates/perry-ext-zlib/src/stream.rs | 99 +++++++++++++++++++ crates/perry-runtime/src/event_target.rs | 39 ++++++++ .../src/commands/compile/optimized_libs.rs | 2 +- .../commands/compile/optimized_libs/driver.rs | 27 ++++- .../compile/optimized_libs/freshness.rs | 66 +++++++++++++ .../commands/compile/optimized_libs/tests.rs | 61 ++++++++++++ .../perry/src/commands/compile/strip_dedup.rs | 39 +++++++- 12 files changed, 392 insertions(+), 6 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 187c1fd352..e210c9ff91 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -6057,6 +6057,7 @@ dependencies = [ "brotli", "flate2", "perry-ffi", + "zstd", ] [[package]] diff --git a/crates/perry-codegen/src/expr/this_super_call.rs b/crates/perry-codegen/src/expr/this_super_call.rs index 1edbbbd7fa..742f331306 100644 --- a/crates/perry-codegen/src/expr/this_super_call.rs +++ b/crates/perry-codegen/src/expr/this_super_call.rs @@ -384,6 +384,7 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result { | "Response" | "Event" | "CustomEvent" + | "DOMException" ) || (is_stream_family_name && !has_extends_expr) || is_other_builtin_constructor_name(parent_name.as_str())) @@ -709,6 +710,37 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result { )?; return Ok(double_literal(f64::from_bits(crate::nanbox::TAG_UNDEFINED))); } + // `class X extends DOMException` (undici's WebSocketError + // and its module-init inheritability probe): `super(message, + // name)` stamps the DOMException surface (`message`/`name`/ + // `code`) onto `this`. The X → DOMException registry edge + // (registered at class-definition time) keeps `instanceof`. + if parent_name.as_str() == "DOMException" { + let undef = double_literal(f64::from_bits(crate::nanbox::TAG_UNDEFINED)); + let mut lowered: Vec = Vec::with_capacity(super_args.len()); + for a in super_args { + lowered.push(lower_expr(ctx, a)?); + } + let arg0 = lowered.first().cloned().unwrap_or_else(|| undef.clone()); + let arg1 = lowered.get(1).cloned().unwrap_or_else(|| undef.clone()); + let this_box = match ctx.this_stack.last().cloned() { + Some(slot) => ctx.block().load(DOUBLE, &slot), + None => undef.clone(), + }; + ctx.block().call( + DOUBLE, + "js_dom_exception_subclass_init", + &[(DOUBLE, &this_box), (DOUBLE, &arg0), (DOUBLE, &arg1)], + ); + let current_class_name = + ctx.class_stack.last().cloned().unwrap_or_default(); + crate::lower_call::apply_field_initializers_recursive( + ctx, + ¤t_class_name, + crate::lower_call::FieldInitMode::SelfOnly, + )?; + return Ok(double_literal(f64::from_bits(crate::nanbox::TAG_UNDEFINED))); + } // `class X extends Promise` — `super(executor)` runs the // ECMA-262 27.2.3.1 Promise constructor against a hidden // backing `Promise` cell stashed on `this`. Inherited diff --git a/crates/perry-codegen/src/lower_call/new_helpers.rs b/crates/perry-codegen/src/lower_call/new_helpers.rs index 9ba7831af2..be98b4d0c2 100644 --- a/crates/perry-codegen/src/lower_call/new_helpers.rs +++ b/crates/perry-codegen/src/lower_call/new_helpers.rs @@ -38,6 +38,7 @@ pub(crate) enum NativeInstanceBase { Set, Event, CustomEvent, + DomException, } /// The native base a parent NAME denotes, if any. @@ -56,6 +57,7 @@ pub(crate) fn native_instance_base(name: &str) -> Option { "Set" => Some(NativeInstanceBase::Set), "Event" => Some(NativeInstanceBase::Event), "CustomEvent" => Some(NativeInstanceBase::CustomEvent), + "DOMException" => Some(NativeInstanceBase::DomException), _ => None, } } @@ -170,6 +172,23 @@ pub(crate) fn emit_native_instance_base_init( ], ); } + NativeInstanceBase::DomException => { + // `super(message, name)` — both optional (`new DOMException()` is + // legal; the runtime defaults name to "Error"). + let arg0 = lowered_args + .first() + .cloned() + .unwrap_or_else(|| undef.clone()); + let arg1 = lowered_args + .get(1) + .cloned() + .unwrap_or_else(|| undef.clone()); + ctx.block().call( + DOUBLE, + "js_dom_exception_subclass_init", + &[(DOUBLE, this_box), (DOUBLE, &arg0), (DOUBLE, &arg1)], + ); + } } } diff --git a/crates/perry-codegen/src/runtime_decls/strings_part2.rs b/crates/perry-codegen/src/runtime_decls/strings_part2.rs index 02e97422de..6edb3addde 100644 --- a/crates/perry-codegen/src/runtime_decls/strings_part2.rs +++ b/crates/perry-codegen/src/runtime_decls/strings_part2.rs @@ -1264,6 +1264,13 @@ pub(crate) fn declare_phase_b_strings_part2(module: &mut LlModule) { ); module.declare_function("js_custom_event_new", I64, &[DOUBLE, DOUBLE, I32]); module.declare_function("js_dom_exception_new", I64, &[DOUBLE, DOUBLE]); + // `super(message, name)` from `class X extends DOMException` — stamps the + // DOMException surface (`message`/`name`/`code`) onto the subclass `this`. + module.declare_function( + "js_dom_exception_subclass_init", + DOUBLE, + &[DOUBLE, DOUBLE, DOUBLE], + ); module.declare_function("js_event_target_add_event_listener", VOID, &[I64, I64, I64]); module.declare_function( "js_event_target_add_event_listener_with_options", diff --git a/crates/perry-ext-zlib/Cargo.toml b/crates/perry-ext-zlib/Cargo.toml index ba58b7e549..cb3838d191 100644 --- a/crates/perry-ext-zlib/Cargo.toml +++ b/crates/perry-ext-zlib/Cargo.toml @@ -14,6 +14,12 @@ crate-type = ["staticlib", "rlib"] [dependencies] perry-ffi.workspace = true flate2 = "1" +# zstd codecs: `zlib.zstdCompressSync` / `createZstdDecompress`. When `zlib` +# routes here, perry-stdlib's `compression` module is compiled out — this +# archive must carry the full zstd surface too, or programs whose compiled JS +# references it (undici's web-fetch content decoding does, unconditionally) +# die at link with undefined `js_zlib_*zstd*` symbols. +zstd.workspace = true # Brotli stream + one-shot support (#1843). Matches the version the # `compression` feature pulls into perry-stdlib. brotli = "8.0.2" diff --git a/crates/perry-ext-zlib/src/stream.rs b/crates/perry-ext-zlib/src/stream.rs index 7a1b144767..25ceac1410 100644 --- a/crates/perry-ext-zlib/src/stream.rs +++ b/crates/perry-ext-zlib/src/stream.rs @@ -207,6 +207,72 @@ pub unsafe extern "C" fn js_zlib_brotli_decompress(data_value: f64, callback_val }); } +fn throw_zstd_error(err: &std::io::Error) -> ! { + perry_ffi::throw_with_code(&format!("zstd: {}", err), "Z_DATA_ERROR", ErrorKind::Error) +} + +/// `zlib.zstdCompressSync(data)` -> Buffer. `_opts` is accepted (codegen +/// passes the options slot through) but zstd params are not wired up — +/// matches perry-stdlib's copy. +/// +/// # Safety +/// `data_value` is the raw NaN-boxed data argument (string or Buffer). +#[no_mangle] +pub unsafe extern "C" fn js_zlib_zstd_compress_sync( + data_value: f64, + _opts: f64, +) -> *mut BufferHeader { + let data_bits = data_value.to_bits() as i64; + js_zlib_validate_buffer_arg(data_bits); + match read_input_from_bits(data_bits) + .map(|d| zstd::stream::encode_all(d.as_slice(), ZSTD_DEFAULT_LEVEL)) + { + Some(Ok(out)) => alloc_buffer(&out), + Some(Err(e)) => throw_zstd_error(&e), + None => std::ptr::null_mut(), + } +} + +/// `zlib.zstdDecompressSync(data)` -> Buffer. +/// +/// # Safety +/// `data_value` is the raw NaN-boxed data argument (string or Buffer). +#[no_mangle] +pub unsafe extern "C" fn js_zlib_zstd_decompress_sync( + data_value: f64, + _opts: f64, +) -> *mut BufferHeader { + let data_bits = data_value.to_bits() as i64; + js_zlib_validate_buffer_arg(data_bits); + match read_input_from_bits(data_bits).map(|d| zstd::stream::decode_all(d.as_slice())) { + Some(Ok(out)) => alloc_buffer(&out), + Some(Err(e)) => throw_zstd_error(&e), + None => std::ptr::null_mut(), + } +} + +/// `zlib.zstdCompress(data, callback)` -> undefined. +/// +/// # Safety +/// `data_value` and `callback_value` are raw NaN-boxed JS values. +#[no_mangle] +pub unsafe extern "C" fn js_zlib_zstd_compress(data_value: f64, callback_value: f64) { + queue_one_shot_callback(data_value, callback_value, "ZstdCompress", |b| { + zstd::stream::encode_all(b, ZSTD_DEFAULT_LEVEL) + }); +} + +/// `zlib.zstdDecompress(data, callback)` -> undefined. +/// +/// # Safety +/// `data_value` and `callback_value` are raw NaN-boxed JS values. +#[no_mangle] +pub unsafe extern "C" fn js_zlib_zstd_decompress(data_value: f64, callback_value: f64) { + queue_one_shot_callback(data_value, callback_value, "ZstdDecompress", |b| { + zstd::stream::decode_all(b) + }); +} + // ── stream codec ───────────────────────────────────────────────────────────── #[derive(Clone, Copy)] @@ -220,8 +286,15 @@ enum Codec { Unzip, BrotliCompress, BrotliDecompress, + ZstdCompress, + ZstdDecompress, } +/// Node's `zlib` zstd default (matches perry-stdlib's copy). zstd levels run +/// 1..=22 and don't share the deflate 0..=9 scale, so the `{ level }` option +/// resolved by `js_zlib_resolve_level` is not applied to zstd codecs. +const ZSTD_DEFAULT_LEVEL: i32 = 3; + fn run_codec(codec: Codec, input: &[u8]) -> std::io::Result> { let mut out = Vec::new(); match codec { @@ -253,6 +326,8 @@ fn run_codec(codec: Codec, input: &[u8]) -> std::io::Result> { } Codec::BrotliCompress => out = brotli_compress_bytes(input), Codec::BrotliDecompress => out = brotli_decompress_bytes(input)?, + Codec::ZstdCompress => out = zstd::stream::encode_all(input, ZSTD_DEFAULT_LEVEL)?, + Codec::ZstdDecompress => out = zstd::stream::decode_all(input)?, } Ok(out) } @@ -275,6 +350,8 @@ enum CodecState { DeflateDec(flate2::write::DeflateDecoder>), BrotliEnc(brotli::CompressorWriter>), BrotliDec(brotli::DecompressorWriter>), + ZstdEnc(zstd::stream::write::Encoder<'static, Vec>), + ZstdDec(zstd::stream::write::Decoder<'static, Vec>), } impl CodecState { @@ -288,6 +365,8 @@ impl CodecState { CodecState::DeflateDec(w) => w.write_all(data), CodecState::BrotliEnc(w) => w.write_all(data), CodecState::BrotliDec(w) => w.write_all(data), + CodecState::ZstdEnc(w) => w.write_all(data), + CodecState::ZstdDec(w) => w.write_all(data), } } @@ -301,6 +380,8 @@ impl CodecState { CodecState::DeflateDec(w) => w.flush(), CodecState::BrotliEnc(w) => w.flush(), CodecState::BrotliDec(w) => w.flush(), + CodecState::ZstdEnc(w) => w.flush(), + CodecState::ZstdDec(w) => w.flush(), } } @@ -315,6 +396,8 @@ impl CodecState { CodecState::DeflateDec(w) => std::mem::take(w.get_mut()), CodecState::BrotliEnc(w) => std::mem::take(w.get_mut()), CodecState::BrotliDec(w) => std::mem::take(w.get_mut()), + CodecState::ZstdEnc(w) => std::mem::take(w.get_mut()), + CodecState::ZstdDec(w) => std::mem::take(w.get_mut()), } } @@ -331,6 +414,11 @@ impl CodecState { // DecompressorWriter::into_inner returns Result (Err on an // unterminated stream); take the decoded bytes either way. CodecState::BrotliDec(w) => Ok(w.into_inner().unwrap_or_else(|v| v)), + // Encoder::finish writes the zstd frame epilogue then hands back + // the inner Vec; Decoder::into_inner is tolerant of an + // unterminated frame (same stance as BrotliDec above). + CodecState::ZstdEnc(w) => w.finish(), + CodecState::ZstdDec(w) => Ok(w.into_inner()), } } } @@ -358,6 +446,15 @@ fn make_codec_state_with_level(codec: Codec, level: Compression) -> Option { CodecState::BrotliDec(brotli::DecompressorWriter::new(Vec::new(), 4096)) } + // zstd context allocation is fallible; `None` falls back to the same + // buffer-until-end `run_codec` path `createUnzip` uses, so a failed + // allocation degrades to one-shot semantics instead of erroring. + Codec::ZstdCompress => CodecState::ZstdEnc( + zstd::stream::write::Encoder::new(Vec::new(), ZSTD_DEFAULT_LEVEL).ok()?, + ), + Codec::ZstdDecompress => { + CodecState::ZstdDec(zstd::stream::write::Decoder::new(Vec::new()).ok()?) + } // Unzip auto-detects the header — kept buffer-until-end (run_codec). Codec::Unzip => return None, }) @@ -523,6 +620,8 @@ factory!(js_zlib_create_inflate_raw, Codec::InflateRaw, 8); factory!(js_zlib_create_unzip, Codec::Unzip, 8); factory!(js_zlib_create_brotli_compress, Codec::BrotliCompress, 0); factory!(js_zlib_create_brotli_decompress, Codec::BrotliDecompress, 0); +factory!(js_zlib_create_zstd_compress, Codec::ZstdCompress, 0); +factory!(js_zlib_create_zstd_decompress, Codec::ZstdDecompress, 0); // ── chunk / buffer helpers ───────────────────────────────────────────────────── diff --git a/crates/perry-runtime/src/event_target.rs b/crates/perry-runtime/src/event_target.rs index d28a1ee206..9710272fe9 100644 --- a/crates/perry-runtime/src/event_target.rs +++ b/crates/perry-runtime/src/event_target.rs @@ -326,6 +326,45 @@ pub extern "C" fn js_event_subclass_init( static KEEP_JS_EVENT_SUBCLASS_INIT: extern "C" fn(f64, f64, f64, u32, u32) -> f64 = js_event_subclass_init; +/// `class X extends DOMException` — `super(message, name)` initializer +/// (undici's `WebSocketError`, and its module-init `class Test extends +/// DOMException` capability probe). The subclass instance is a registry-class +/// object, not the ErrorHeader `new DOMException(...)` allocates, so stamp the +/// DOMException surface onto `this`: `message`, `name` (default `"Error"`, +/// matching `js_dom_exception_new`), and the legacy numeric `code` for that +/// name. +#[no_mangle] +pub extern "C" fn js_dom_exception_subclass_init(this_value: f64, message: f64, name: f64) -> f64 { + let Some(exception) = value_as_ptr::(this_value) else { + return undefined_value(); + }; + let message_ptr = optional_string_from_value(message, b""); + let name_ptr = optional_string_from_value(name, b"Error"); + let name_string = unsafe { + let len = (*name_ptr).byte_len as usize; + let data = (name_ptr as *const u8).add(std::mem::size_of::()); + String::from_utf8_lossy(std::slice::from_raw_parts(data, len)).into_owned() + }; + set_event_field( + exception, + b"message", + crate::value::js_nanbox_string(message_ptr as i64), + ); + set_event_field( + exception, + b"name", + crate::value::js_nanbox_string(name_ptr as i64), + ); + set_event_field(exception, b"code", dom_exception_code(&name_string)); + undefined_value() +} + +/// Keepalive anchor for the auto-optimize whole-program build — +/// `js_dom_exception_subclass_init` is a generated-code-only callee. +#[used] +static KEEP_JS_DOM_EXCEPTION_SUBCLASS_INIT: extern "C" fn(f64, f64, f64) -> f64 = + js_dom_exception_subclass_init; + fn is_event_instance(event: *const ObjectHeader) -> bool { if event.is_null() { return false; diff --git a/crates/perry/src/commands/compile/optimized_libs.rs b/crates/perry/src/commands/compile/optimized_libs.rs index 18849b2559..2dee6f8f3a 100644 --- a/crates/perry/src/commands/compile/optimized_libs.rs +++ b/crates/perry/src/commands/compile/optimized_libs.rs @@ -32,7 +32,7 @@ pub(crate) use driver::build_optimized_libs; pub(crate) use freshness::{ auto_optimized_archives_are_fresh, auto_optimized_build_stamp, auto_optimized_cache_key, auto_optimized_cross_features, auto_optimized_source_fingerprint, binding_needs_shared_tokio, - resolve_auto_well_known_libs, + resolve_auto_well_known_libs, retain_workspace_declared_features, }; pub(crate) use no_auto::{ build_missing_prebuilt_ext_lib, resolve_no_auto_optimized_libs, resolve_prebuilt_ext_libs, diff --git a/crates/perry/src/commands/compile/optimized_libs/driver.rs b/crates/perry/src/commands/compile/optimized_libs/driver.rs index fcb13e09e5..0da10a81b5 100644 --- a/crates/perry/src/commands/compile/optimized_libs/driver.rs +++ b/crates/perry/src/commands/compile/optimized_libs/driver.rs @@ -563,7 +563,24 @@ pub(crate) fn build_optimized_libs( hash = hash.wrapping_mul(33).wrapping_add(*b as u64); } let (target_dir, cargo_env_dir) = auto_target_dir_paths(&workspace_root, hash); - let cross_features = auto_optimized_cross_features(ctx, &features, cli_features); + let mut cross_features = auto_optimized_cross_features(ctx, &features, cli_features); + // Binary/workspace skew guard: the baked-in list above tracks the branch + // this `perry` was built from, but cargo resolves it against the checkout + // on disk. One unknown `perry-runtime/` fails the whole resolve, and + // the prebuilt fallback below then links without the ext-pump entrypoints + // — undefined-`js_*` errors two stages away from the cause. Filter before + // the build stamp so the stamp keys on what actually gets built. + let dropped_features = retain_workspace_declared_features(&workspace_root, &mut cross_features); + if !dropped_features.is_empty() && matches!(format, OutputFormat::Text) { + eprintln!( + " auto-optimize: dropping feature(s) this workspace does not declare: {} \ + (this perry binary was likely built from a different branch than the \ + checkout at {})", + dropped_features.join(", "), + workspace_root.display() + ); + } + let cross_features = cross_features; let release_dir = if let Some(triple) = rust_target_triple(target) { target_dir.join(triple).join("release") } else { @@ -834,8 +851,12 @@ pub(crate) fn build_optimized_libs( if !status.success() { if matches!(format, OutputFormat::Text) { eprintln!( - " auto-optimize: cargo build failed (exit {}), \ - using prebuilt libraries", + " auto-optimize: cargo build failed ({}), \ + using prebuilt libraries. The prebuilt archives may lack the \ + feature-gated `js_*` entrypoints this compile routed to ext \ + crates; if the link fails with undefined symbols, fix the \ + cargo error above (or rebuild the workspace so it matches \ + this perry binary) and re-run.", status ); } diff --git a/crates/perry/src/commands/compile/optimized_libs/freshness.rs b/crates/perry/src/commands/compile/optimized_libs/freshness.rs index f2189cb339..d729b9ef83 100644 --- a/crates/perry/src/commands/compile/optimized_libs/freshness.rs +++ b/crates/perry/src/commands/compile/optimized_libs/freshness.rs @@ -191,6 +191,72 @@ pub(crate) fn auto_optimized_cross_features( cross_features } +/// Feature names a workspace crate's `Cargo.toml` can satisfy in a +/// `--features /` request: the `[features]` table keys plus every +/// optional dependency (an optional dep implicitly defines a same-named +/// feature unless all its `dep:` references say otherwise — over-including +/// those keeps this fail-open). `None` when the manifest is missing or +/// unparseable, so callers skip filtering rather than dropping features a +/// manifest they couldn't read might well declare. +fn declared_feature_names(workspace_root: &Path, krate: &str) -> Option> { + let manifest_path = workspace_root.join("crates").join(krate).join("Cargo.toml"); + let manifest: toml::Value = toml::from_str(&fs::read_to_string(manifest_path).ok()?).ok()?; + let mut names: BTreeSet = manifest + .get("features")? + .as_table()? + .keys() + .cloned() + .collect(); + let mut collect_optional = |deps: Option<&toml::Value>| { + let Some(table) = deps.and_then(|d| d.as_table()) else { + return; + }; + for (name, spec) in table { + if spec.get("optional").and_then(|o| o.as_bool()) == Some(true) { + names.insert(name.clone()); + } + } + }; + collect_optional(manifest.get("dependencies")); + if let Some(targets) = manifest.get("target").and_then(|t| t.as_table()) { + for target_spec in targets.values() { + collect_optional(target_spec.get("dependencies")); + } + } + Some(names) +} + +/// The `perry` binary's baked-in cross-feature list tracks the branch the +/// binary was BUILT from, while the auto-optimize cargo build resolves against +/// the workspace found on disk — and the two can skew (binary from branch A, +/// checkout on branch B). One `perry-runtime/` the checkout doesn't +/// declare fails the entire cargo resolve, and the silent prebuilt fallback +/// then links without the ext-pump entrypoints the well-known routing loop +/// already stripped stdlib features for — surfacing as undefined-`js_*` link +/// errors far from the cause. Drop the unknown names instead (a feature the +/// checkout never heard of gates nothing in its sources) and return them so +/// the caller can say what was dropped. +pub(crate) fn retain_workspace_declared_features( + workspace_root: &Path, + cross_features: &mut Vec, +) -> Vec { + let mut dropped = Vec::new(); + for krate in ["perry-runtime", "perry-stdlib"] { + let Some(declared) = declared_feature_names(workspace_root, krate) else { + continue; + }; + let prefix = format!("{krate}/"); + cross_features.retain(|entry| match entry.strip_prefix(&prefix) { + Some(feat) if !declared.contains(feat) => { + dropped.push(entry.clone()); + false + } + _ => true, + }); + } + dropped +} + /// Content fingerprint of every workspace source tree that lands in the /// auto-optimized archives: the crates this build compiles (the runtime/stdlib /// static wrappers and the tokio-using ext crates) plus their transitive diff --git a/crates/perry/src/commands/compile/optimized_libs/tests.rs b/crates/perry/src/commands/compile/optimized_libs/tests.rs index 1a867f22a2..b447059387 100644 --- a/crates/perry/src/commands/compile/optimized_libs/tests.rs +++ b/crates/perry/src/commands/compile/optimized_libs/tests.rs @@ -635,3 +635,64 @@ printf '!\n' > "$CARGO_TARGET_DIR/release/libperry_ext_http.a" target_dir.join("release/libperry_ext_http.a") ); } + +/// Binary/workspace skew: a cross-feature the on-disk checkout's +/// perry-runtime doesn't declare must be dropped (and reported), not passed +/// through to fail the entire cargo resolve — that failure's prebuilt +/// fallback links without the routed ext entrypoints and dies with +/// undefined `js_*` symbols far from the cause. +#[test] +fn retain_workspace_declared_features_drops_unknown_names() { + let dir = tempfile::tempdir().expect("tempdir"); + write_file( + &dir.path().join("crates/perry-runtime/Cargo.toml"), + b"[package]\nname = \"perry-runtime\"\n\n[features]\nfull = []\nregex-engine = []\n\n[dependencies]\nmimalloc = { version = \"0.1\", optional = true }\n", + ); + write_file( + &dir.path().join("crates/perry-stdlib/Cargo.toml"), + b"[package]\nname = \"perry-stdlib\"\n\n[features]\ncrypto = []\n", + ); + + let mut cross_features = vec![ + "perry-runtime/full".to_string(), + "perry-runtime/alloc-mimalloc".to_string(), + "perry-runtime/mimalloc".to_string(), + "perry-stdlib/crypto".to_string(), + "perry-stdlib/web-fetch".to_string(), + ]; + let dropped = retain_workspace_declared_features(dir.path(), &mut cross_features); + + // `full` and `crypto` are declared features; `mimalloc` is an optional + // dep (implicit feature). Only the names the checkout has never heard of + // go. + assert_eq!( + cross_features, + vec![ + "perry-runtime/full".to_string(), + "perry-runtime/mimalloc".to_string(), + "perry-stdlib/crypto".to_string(), + ] + ); + assert_eq!( + dropped, + vec![ + "perry-runtime/alloc-mimalloc".to_string(), + "perry-stdlib/web-fetch".to_string(), + ] + ); +} + +/// Fail-open: with no readable manifest (release tarball, partial checkout) +/// there is nothing trustworthy to filter against — every requested feature +/// must survive. +#[test] +fn retain_workspace_declared_features_keeps_all_without_manifests() { + let dir = tempfile::tempdir().expect("tempdir"); + let mut cross_features = vec![ + "perry-runtime/full".to_string(), + "perry-runtime/alloc-mimalloc".to_string(), + ]; + let dropped = retain_workspace_declared_features(dir.path(), &mut cross_features); + assert!(dropped.is_empty()); + assert_eq!(cross_features.len(), 2); +} diff --git a/crates/perry/src/commands/compile/strip_dedup.rs b/crates/perry/src/commands/compile/strip_dedup.rs index b1da605608..f49a07ce7e 100644 --- a/crates/perry/src/commands/compile/strip_dedup.rs +++ b/crates/perry/src/commands/compile/strip_dedup.rs @@ -747,13 +747,34 @@ pub(super) fn strip_duplicate_objects_from_well_known_lib(lib_path: &PathBuf) -> let abs_staticlib = std::fs::canonicalize(lib_path)?; let symbols_by_member = collect_archive_symbols_by_member(&nm, &abs_staticlib) .ok_or_else(|| anyhow::anyhow!("failed to inspect archive symbols"))?; + // Undefined (U) symbols per member. Localizing a PANIC-runtime definition + // that a SIBLING member of the same archive still references severs an + // intra-archive edge: the wrapper's kept `std` cgu defines + // `__rust_drop_panic`, its kept `panic_unwind` cgu references it, and a + // panic=abort stdlib provides no replacement — the final link dies on + // exactly that symbol. Skip localizing those. ALLOCATOR shims are + // deliberately NOT guarded this way: every member references + // `__rust_alloc`, so the guard would always skip them — and leaving the + // wrapper's system-malloc shim global lets it beat the runtime's mimalloc + // shim at link, which breaks the runtime's pointer classification + // (console output silently vanishes). Allocator references always have + // the runtime's global copy to bind to; unwind-flavor panic internals may + // not. + let undefined_by_member = collect_archive_undefined_by_member(&nm, &abs_staticlib) + .ok_or_else(|| anyhow::anyhow!("failed to inspect archive undefined symbols"))?; let forced_symbols_by_member: std::collections::BTreeMap> = symbols_by_member .iter() .filter_map(|(member, symbols)| { let mut forced_symbols: Vec = symbols .iter() - .filter(|symbol| force_localize_symbol(symbol)) + .filter(|symbol| { + force_localize_symbol(symbol) + && !(is_panic_unwind_symbol(symbol) + && undefined_by_member + .iter() + .any(|(m, undef)| m != member && undef.contains(*symbol))) + }) .cloned() .collect(); if forced_symbols.is_empty() { @@ -1251,7 +1272,21 @@ pub(super) fn strip_bundled_shared_deps_from_well_known_lib( let stdlib_members = list_members(&abs_stdlib)?; let candidates: std::collections::BTreeSet = members .iter() - .filter(|m| stdlib_members.iter().any(|s| s.contains(m.as_str()))) + .filter(|m| { + stdlib_members.iter().any(|s| s.contains(m.as_str())) + // std's bundled panic runtime. The wrapper (built + // panic=unwind) bundles `panic_unwind-*`; a panic=abort + // stdlib bundles `panic_abort-*` under a DIFFERENT member + // name, so the name-containment rule above never nominates + // it — the stale unwind copy survives, and its reference to + // std's `__rustc` shim (`__rust_drop_panic`), whose object + // WAS dropped as stdlib-provided, fails the link. Nominate + // it here; the fixed-point loop below protects it (keeps it) + // whenever a kept sibling needs a symbol only it defines and + // the stdlib doesn't provide — i.e. removal happens exactly + // when the stdlib's own panic runtime covers the link. + || m.contains("panic_unwind") + }) .cloned() .collect(); if candidates.is_empty() { From e8e529d7a3e7353849e792aa197743fc20c07f7d Mon Sep 17 00:00:00 2001 From: jdalton Date: Wed, 29 Jul 2026 14:18:42 -0400 Subject: [PATCH 25/30] docs: changelog fragment for #7021 --- changelog.d/7021-real-npm-cli-compile-gaps.md | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 changelog.d/7021-real-npm-cli-compile-gaps.md diff --git a/changelog.d/7021-real-npm-cli-compile-gaps.md b/changelog.d/7021-real-npm-cli-compile-gaps.md new file mode 100644 index 0000000000..c5f3c6f567 --- /dev/null +++ b/changelog.d/7021-real-npm-cli-compile-gaps.md @@ -0,0 +1,6 @@ +**Compile fixes from taking a real npm CLI (Socket Firewall) to a native binary** — four independent blockers, each fixed at its own layer: + +- **Auto-optimize feature skew**: cross-features the on-disk checkout's `perry-runtime`/`perry-stdlib` don't declare are now dropped (with a warning) instead of failing the whole cargo resolve and silently falling back to a link that's missing the routed ext entrypoints. The cargo-failure fallback message now explains the consequence and remedy. +- **`perry-ext-zlib` zstd surface**: `zlib.createZstdCompress`/`createZstdDecompress`, the zstd one-shots, and the streaming write-codec are now implemented in the ext wrapper, so routing `node:zlib` no longer strips the only zstd implementation out of the link (undici's web-fetch content decoding references it unconditionally). +- **`class X extends DOMException`**: new `js_dom_exception_subclass_init` wired through both the explicit `super()` lowering and the implicit-ctor chain walk — undici's module-init inheritability probe no longer aborts startup with `DOMException is not a function`. +- **panic-runtime dedup**: prebuilt (panic=unwind) wrapper staticlibs co-linked with a panic=abort auto-optimized stdlib no longer die on `__rust_drop_panic` — the `panic_unwind` member is nominated for the nosharedeps fixed-point (kept only when the stdlib can't cover it), and panic symbols referenced by a sibling member are no longer localized. Allocator shims remain always-localized (a global wrapper malloc shim would break runtime pointer classification). From 24fd0157bf200ce406d7eb349b29edd9b0f113d5 Mon Sep 17 00:00:00 2001 From: jdalton Date: Wed, 29 Jul 2026 15:39:28 -0400 Subject: [PATCH 26/30] fix(runtime): make the rebound RegExp global constructible via its call form MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ECMA-262 22.2.4: `RegExp(pattern, flags)` without `new` constructs exactly like `new RegExp`, with the identity shortcut `RegExp(re)` → `re`. The globalThis sentinel fell through to the noop thunk and returned undefined — which is how lodash's module init died: runInContext rebinds the global (`var RegExp = context.RegExp`) and builds `reIsNative` through the call form, so the immediately following `reIsNative.test(...)` threw 'Cannot read properties of undefined'. New regexp_constructor_call_thunk (arity 2) mirrors the dynamic-new RegExp arm in class_registry/construct.rs; without the regex-engine feature it keeps the old noop behavior. Unblocks sfw-registry (lodash via registry/proxy-request.ts). --- .../perry-runtime/src/object/global_this.rs | 13 ++--- .../src/object/global_this/ctor_thunks.rs | 50 +++++++++++++++++++ .../src/object/global_this/populate.rs | 5 ++ 3 files changed, 62 insertions(+), 6 deletions(-) diff --git a/crates/perry-runtime/src/object/global_this.rs b/crates/perry-runtime/src/object/global_this.rs index 08c130624a..5371b80023 100644 --- a/crates/perry-runtime/src/object/global_this.rs +++ b/crates/perry-runtime/src/object/global_this.rs @@ -70,12 +70,13 @@ pub(crate) use ctor_thunks::{ global_this_url_pattern_call_thunk, is_function_prototype_object_value, map_constructor_call_thunk, normalize_eval_this_body, promise_constructor_call_thunk, range_error_constructor_call_thunk, reference_error_constructor_call_thunk, - set_constructor_call_thunk, subtle_crypto_method_value, syntax_error_constructor_call_thunk, - type_error_constructor_call_thunk, typed_array_constructor_call_thunk, - uri_error_constructor_call_thunk, weak_map_constructor_call_thunk, - weak_ref_constructor_call_thunk, weak_set_constructor_call_thunk, - webcrypto_get_random_values_thunk, webcrypto_illegal_constructor_thunk, webcrypto_method_value, - webcrypto_random_uuid_thunk, webcrypto_subtle_getter_thunk, + regexp_constructor_call_thunk, set_constructor_call_thunk, subtle_crypto_method_value, + syntax_error_constructor_call_thunk, type_error_constructor_call_thunk, + typed_array_constructor_call_thunk, uri_error_constructor_call_thunk, + weak_map_constructor_call_thunk, weak_ref_constructor_call_thunk, + weak_set_constructor_call_thunk, webcrypto_get_random_values_thunk, + webcrypto_illegal_constructor_thunk, webcrypto_method_value, webcrypto_random_uuid_thunk, + webcrypto_subtle_getter_thunk, }; #[cfg(feature = "temporal")] pub(crate) use fetch_globals::temporal_subclass_super; diff --git a/crates/perry-runtime/src/object/global_this/ctor_thunks.rs b/crates/perry-runtime/src/object/global_this/ctor_thunks.rs index 92347c1713..2f7b5f55a8 100644 --- a/crates/perry-runtime/src/object/global_this/ctor_thunks.rs +++ b/crates/perry-runtime/src/object/global_this/ctor_thunks.rs @@ -25,6 +25,56 @@ pub(crate) extern "C" fn typed_array_constructor_call_thunk( super::super::object_ops::throw_object_type_error(b"Constructor %TypedArray% requires 'new'") } +/// `RegExp(pattern, flags)` called WITHOUT `new` — unlike Map/Set below, +/// RegExp IS callable: ECMA-262 22.2.4 makes the call form construct exactly +/// like `new RegExp(pattern, flags)`, with one identity shortcut — `RegExp(re)` +/// with an existing RegExp and undefined flags returns `re` unchanged. +/// +/// The noop-thunk fallback returned `undefined` here, which is how lodash's +/// module init died: `runInContext` rebinds the global (`var RegExp = +/// context.RegExp`) and builds its native-function probe through the call form +/// (`var reIsNative = RegExp('^' + …)`) — the very next `reIsNative.test(...)` +/// threw "Cannot read properties of undefined". Construction mirrors the +/// dynamic-`new` RegExp arm in class_registry/construct.rs. +#[cfg(feature = "regex-engine")] +pub(crate) extern "C" fn regexp_constructor_call_thunk( + _closure: *const crate::closure::ClosureHeader, + pattern: f64, + flags: f64, +) -> f64 { + let flags_undefined = flags.to_bits() == crate::value::TAG_UNDEFINED; + let pattern_value = crate::value::JSValue::from_bits(pattern.to_bits()); + if flags_undefined && pattern_value.is_pointer() { + let addr = (pattern.to_bits() & 0x0000_FFFF_FFFF_FFFF) as usize; + if crate::regex::is_regex_pointer(addr as *const u8) { + return pattern; + } + } + let pattern_ptr = if pattern_value.is_undefined() { + std::ptr::null_mut() + } else { + crate::builtins::js_string_coerce(pattern) + }; + let flags_ptr = if flags_undefined { + std::ptr::null_mut() + } else { + crate::builtins::js_string_coerce(flags) + }; + let re = crate::regex::js_regexp_new(pattern_ptr, flags_ptr); + crate::value::js_nanbox_pointer(re as i64) +} + +/// Without the regex engine there is no RegExp to construct — keep the +/// pre-existing noop behavior rather than referencing a compiled-out ctor. +#[cfg(not(feature = "regex-engine"))] +pub(crate) extern "C" fn regexp_constructor_call_thunk( + _closure: *const crate::closure::ClosureHeader, + _pattern: f64, + _flags: f64, +) -> f64 { + f64::from_bits(crate::value::TAG_UNDEFINED) +} + // #4569: Map/Set/WeakMap/WeakSet/WeakRef are constructors — calling them // without `new` is a TypeError (ECMA-262: an undefined newTarget throws). The // bare-call form previously fell through to `global_this_builtin_noop_thunk` diff --git a/crates/perry-runtime/src/object/global_this/populate.rs b/crates/perry-runtime/src/object/global_this/populate.rs index b456f617a8..f92f800969 100644 --- a/crates/perry-runtime/src/object/global_this/populate.rs +++ b/crates/perry-runtime/src/object/global_this/populate.rs @@ -105,6 +105,7 @@ pub(crate) fn populate_global_this_builtins(singleton: *mut ObjectHeader) { | "Int32Array" | "Uint32Array" | "Float16Array" | "Float32Array" | "Float64Array" | "BigInt64Array" | "BigUint64Array" => typed_array_constructor_call_thunk as *const u8, // #4569: collection constructors throw when called without `new`. + "RegExp" => regexp_constructor_call_thunk as *const u8, "Map" => map_constructor_call_thunk as *const u8, "Set" => set_constructor_call_thunk as *const u8, "WeakMap" => weak_map_constructor_call_thunk as *const u8, @@ -146,6 +147,10 @@ pub(crate) fn populate_global_this_builtins(singleton: *mut ObjectHeader) { "URLPattern" => { crate::closure::js_register_closure_arity(func_ptr, 2); } + // RegExp(pattern, flags) — the call form constructs (22.2.4). + "RegExp" => { + crate::closure::js_register_closure_arity(func_ptr, 2); + } "Int8Array" | "Uint8Array" | "Uint8ClampedArray" | "Int16Array" | "Uint16Array" | "Int32Array" | "Uint32Array" | "Float16Array" | "Float32Array" | "Float64Array" | "BigInt64Array" | "BigUint64Array" => { From 8417353d0fb822fa5b48e703379069966ede6f5c Mon Sep 17 00:00:00 2001 From: jdalton Date: Wed, 29 Jul 2026 15:39:44 -0400 Subject: [PATCH 27/30] docs: extend #7021 changelog fragment with the RegExp call-form fix --- changelog.d/7021-real-npm-cli-compile-gaps.md | 1 + 1 file changed, 1 insertion(+) diff --git a/changelog.d/7021-real-npm-cli-compile-gaps.md b/changelog.d/7021-real-npm-cli-compile-gaps.md index c5f3c6f567..081c235808 100644 --- a/changelog.d/7021-real-npm-cli-compile-gaps.md +++ b/changelog.d/7021-real-npm-cli-compile-gaps.md @@ -4,3 +4,4 @@ - **`perry-ext-zlib` zstd surface**: `zlib.createZstdCompress`/`createZstdDecompress`, the zstd one-shots, and the streaming write-codec are now implemented in the ext wrapper, so routing `node:zlib` no longer strips the only zstd implementation out of the link (undici's web-fetch content decoding references it unconditionally). - **`class X extends DOMException`**: new `js_dom_exception_subclass_init` wired through both the explicit `super()` lowering and the implicit-ctor chain walk — undici's module-init inheritability probe no longer aborts startup with `DOMException is not a function`. - **panic-runtime dedup**: prebuilt (panic=unwind) wrapper staticlibs co-linked with a panic=abort auto-optimized stdlib no longer die on `__rust_drop_panic` — the `panic_unwind` member is nominated for the nosharedeps fixed-point (kept only when the stdlib can't cover it), and panic symbols referenced by a sibling member are no longer localized. Allocator shims remain always-localized (a global wrapper malloc shim would break runtime pointer classification). +- **`RegExp` call form via a rebound global**: `var R = globalThis.RegExp; R(src)` now constructs (with the spec's `RegExp(re)` identity shortcut) instead of returning `undefined` from the noop thunk — lodash's `runInContext` module init relied on exactly this. From 781282f1036afaabe00f5a147a98a1c4fc7c9c6e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Wed, 29 Jul 2026 22:52:36 +0200 Subject: [PATCH 28/30] fix: address stacked stdlib review --- .../perry-codegen/src/expr/this_super_call.rs | 32 +++++++++ crates/perry-ext-zlib/src/stream.rs | 14 +++- crates/perry-runtime/src/event_target.rs | 37 +++++++---- scripts/soak/soak.mts | 17 +++-- scripts/soak/soak.test.mts | 45 ++++++++----- scripts/soak/update-deps.mts | 66 +++++++++++++++---- scripts/soak/update-deps.test.mts | 3 + 7 files changed, 163 insertions(+), 51 deletions(-) diff --git a/crates/perry-codegen/src/expr/this_super_call.rs b/crates/perry-codegen/src/expr/this_super_call.rs index 742f331306..e60e61bf43 100644 --- a/crates/perry-codegen/src/expr/this_super_call.rs +++ b/crates/perry-codegen/src/expr/this_super_call.rs @@ -214,6 +214,38 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result { )?; return Ok(double_literal(f64::from_bits(crate::nanbox::TAG_UNDEFINED))); } + // `class X extends DOMException` with a synthesized/pass-through + // constructor (`super(...args)`) must initialize the same surface + // as the fixed-arity super-call path above. Array reads past the + // spread argument count produce `undefined`, matching the optional + // message/name parameters. + let is_dom_exception = ctx + .classes + .get(¤t_class_name) + .and_then(|c| c.extends_name.as_deref()) + .map(|p| p == "DOMException") + .unwrap_or(false); + if is_dom_exception { + let zero_idx = "0".to_string(); + let one_idx = "1".to_string(); + let message = + ctx.block() + .call(DOUBLE, "js_array_get_f64", &[(I64, &arr), (I32, &zero_idx)]); + let name = + ctx.block() + .call(DOUBLE, "js_array_get_f64", &[(I64, &arr), (I32, &one_idx)]); + ctx.block().call( + DOUBLE, + "js_dom_exception_subclass_init", + &[(DOUBLE, &this_box), (DOUBLE, &message), (DOUBLE, &name)], + ); + crate::lower_call::apply_field_initializers_recursive( + ctx, + ¤t_class_name, + crate::lower_call::FieldInitMode::SelfOnly, + )?; + return Ok(double_literal(f64::from_bits(crate::nanbox::TAG_UNDEFINED))); + } if let Some(&child_cid) = ctx.class_ids.get(¤t_class_name) { let cid_str = child_cid.to_string(); let blk = ctx.block(); diff --git a/crates/perry-ext-zlib/src/stream.rs b/crates/perry-ext-zlib/src/stream.rs index 25ceac1410..f4eda4fd8e 100644 --- a/crates/perry-ext-zlib/src/stream.rs +++ b/crates/perry-ext-zlib/src/stream.rs @@ -418,7 +418,10 @@ impl CodecState { // the inner Vec; Decoder::into_inner is tolerant of an // unterminated frame (same stance as BrotliDec above). CodecState::ZstdEnc(w) => w.finish(), - CodecState::ZstdDec(w) => Ok(w.into_inner()), + CodecState::ZstdDec(mut w) => { + w.flush()?; + Ok(w.into_inner()) + } } } } @@ -1445,6 +1448,15 @@ mod stream_tests { ); } + #[test] + fn zstd_decoder_finish_flushes_pending_output() { + let expected = b"zstd decoder output buffered until the stream finishes"; + let compressed = zstd::stream::encode_all(expected.as_slice(), ZSTD_DEFAULT_LEVEL).unwrap(); + let mut decoder = make_codec_state(Codec::ZstdDecompress).expect("zstd decoder"); + decoder.write_chunk(&compressed).unwrap(); + assert_eq!(decoder.finish().unwrap(), expected); + } + #[test] fn gunzip_run_codec_reads_all_members() { let a = stream_compress(Codec::Gzip, &[b"first "]); diff --git a/crates/perry-runtime/src/event_target.rs b/crates/perry-runtime/src/event_target.rs index 9710272fe9..3244711b37 100644 --- a/crates/perry-runtime/src/event_target.rs +++ b/crates/perry-runtime/src/event_target.rs @@ -147,9 +147,13 @@ unsafe fn listener_signal(options: f64) -> Option<*mut ObjectHeader> { } fn set_event_field(event: *mut ObjectHeader, name: &[u8], value: f64) { - js_object_set_field_by_name(event, key(name), value); + let scope = crate::gc::RuntimeHandleScope::new(); + let event = scope.root_raw_mut_ptr(event); + let value = scope.root_nanbox_f64(value); + let field_key = key(name); + js_object_set_field_by_name(event.get_raw_mut_ptr(), field_key, value.get_nanbox_f64()); crate::object::set_builtin_property_attrs( - event as usize, + event.get_raw_mut_ptr::() as usize, String::from_utf8_lossy(name).into_owned(), crate::object::PropertyAttrs::new(true, false, true), ); @@ -335,27 +339,38 @@ static KEEP_JS_EVENT_SUBCLASS_INIT: extern "C" fn(f64, f64, f64, u32, u32) -> f6 /// name. #[no_mangle] pub extern "C" fn js_dom_exception_subclass_init(this_value: f64, message: f64, name: f64) -> f64 { - let Some(exception) = value_as_ptr::(this_value) else { + let scope = crate::gc::RuntimeHandleScope::new(); + let exception = scope.root_nanbox_f64(this_value); + let message = scope.root_nanbox_f64(message); + let name = scope.root_nanbox_f64(name); + if value_as_ptr::(exception.get_nanbox_f64()).is_none() { return undefined_value(); - }; - let message_ptr = optional_string_from_value(message, b""); - let name_ptr = optional_string_from_value(name, b"Error"); + } + let message_ptr = optional_string_from_value(message.get_nanbox_f64(), b""); + let message_ptr = scope.root_string_ptr(message_ptr); + let name_ptr = optional_string_from_value(name.get_nanbox_f64(), b"Error"); + let name_ptr = scope.root_string_ptr(name_ptr); let name_string = unsafe { + let name_ptr = name_ptr.get_raw_const_ptr::(); let len = (*name_ptr).byte_len as usize; let data = (name_ptr as *const u8).add(std::mem::size_of::()); String::from_utf8_lossy(std::slice::from_raw_parts(data, len)).into_owned() }; set_event_field( - exception, + value_as_ptr::(exception.get_nanbox_f64()).unwrap(), b"message", - crate::value::js_nanbox_string(message_ptr as i64), + crate::value::js_nanbox_string(message_ptr.get_raw_const_ptr::() as i64), ); set_event_field( - exception, + value_as_ptr::(exception.get_nanbox_f64()).unwrap(), b"name", - crate::value::js_nanbox_string(name_ptr as i64), + crate::value::js_nanbox_string(name_ptr.get_raw_const_ptr::() as i64), + ); + set_event_field( + value_as_ptr::(exception.get_nanbox_f64()).unwrap(), + b"code", + dom_exception_code(&name_string), ); - set_event_field(exception, b"code", dom_exception_code(&name_string)); undefined_value() } diff --git a/scripts/soak/soak.mts b/scripts/soak/soak.mts index eab961546a..b05a13d2d6 100644 --- a/scripts/soak/soak.mts +++ b/scripts/soak/soak.mts @@ -400,16 +400,21 @@ export function checkToolchainSoak(body: string, file: string): Finding[] { export function checkTazeConfig(body: string, file: string): Finding[] { const out: Finding[] = [] - if (!body.includes('maturityPeriod')) { + const importsSoakDays = + /import\s*\{[^}]*\bSOAK_DAYS\b[^}]*\}\s*from\s*['"][^'"]*scripts\/soak\/constants\.mts['"]/.test( + body, + ) + const usesSoakDays = /\bmaturityPeriod\s*:\s*SOAK_DAYS\b/.test(body) + if (!usesSoakDays) { out.push({ file, what: 'taze maturityPeriod', - saw: '(not set)', + saw: body.includes('maturityPeriod') ? 'not set to SOAK_DAYS' : '(not set)', wanted: 'maturityPeriod: SOAK_DAYS', fix: 'set maturityPeriod: SOAK_DAYS in the taze config', }) } - if (!body.includes('constants.mts')) { + if (!importsSoakDays) { out.push({ file, what: 'taze config soak import', @@ -432,9 +437,6 @@ export function checkTazeConfig(body: string, file: string): Finding[] { * `- package-ecosystem:` line to the next one. */ export function checkDependabotCooldown(body: string, file: string): Finding[] { - if (SOAK_DAYS === 0) { - return [] - } const out: Finding[] = [] for (const block of parseDependabotBlocks(body)) { const days = /^\s+default-days:\s*(\d+)\s*$/m.exec(block.body)?.[1] @@ -485,9 +487,6 @@ export function parseDependabotBlocks(body: string): DependabotBlock[] { // window. A MISSING cooldown block stays a --check finding (line-based // YAML insertion is riskier than telling a human where the two lines go). export function fixDependabotCooldown(body: string): string { - if (SOAK_DAYS === 0) { - return body - } return body.replace(/^(\s+default-days:\s*)\d+\s*$/gm, `$1${SOAK_DAYS}`) } diff --git a/scripts/soak/soak.test.mts b/scripts/soak/soak.test.mts index 380fbc4df9..9f8396b346 100644 --- a/scripts/soak/soak.test.mts +++ b/scripts/soak/soak.test.mts @@ -3,7 +3,7 @@ import { spawnSync } from 'node:child_process' import { test } from 'node:test' import { fileURLToPath } from 'node:url' -import { SOAK_DAYS, addDaysIso, todayIso } from './constants.mts' +import { SOAK_DAYS, SOAK_MINUTES, addDaysIso, todayIso } from './constants.mts' import { checkCargoConfig, checkCatalogParity, @@ -31,7 +31,7 @@ const FRESH_REM = addDaysIso(FRESH_PUB, SOAK_DAYS) const CLEAN_YAML = `catalog: taze: 19.14.1 -minimumReleaseAge: 10080 +minimumReleaseAge: ${SOAK_MINUTES} minimumReleaseAgeExclude: # published: ${FRESH_PUB} | removable: ${FRESH_REM} - 'left-pad@1.3.0' @@ -40,25 +40,28 @@ minimumReleaseAgeExclude: ` test('cargo config: wrong window and missing unstable gate are findings', () => { - const good = '[unstable]\nmin-publish-age = true\n\n[registry]\nglobal-min-publish-age = "7 days"\n' + const good = `[unstable]\nmin-publish-age = true\n\n[registry]\nglobal-min-publish-age = "${SOAK_DAYS} days"\n` assert.equal(checkCargoConfig(good, 'c').length, 0) - assert.equal(checkCargoConfig(good.replace('7 days', '3 days'), 'c').length, 1) - assert.equal(checkCargoConfig('[registry]\nglobal-min-publish-age = "7 days"\n', 'c').length, 1) + assert.equal(checkCargoConfig(good.replace(`${SOAK_DAYS} days`, `${SOAK_DAYS + 1} days`), 'c').length, 1) + assert.equal( + checkCargoConfig(`[registry]\nglobal-min-publish-age = "${SOAK_DAYS} days"\n`, 'c').length, + 1, + ) }) test('npmrc: window must match SOAK_DAYS and fix writes it', () => { - assert.equal(checkNpmrc('min-release-age=7\n', 'n').length, 0) + assert.equal(checkNpmrc(`min-release-age=${SOAK_DAYS}\n`, 'n').length, 0) assert.equal(checkNpmrc('min-release-age=3\n', 'n').length, 1) assert.equal(checkNpmrc('# nothing\n', 'n').length, 1) - assert.match(fixNpmrc('# nothing\n'), /min-release-age=7/) - assert.match(fixNpmrc('min-release-age=3\n'), /min-release-age=7/) + assert.ok(fixNpmrc('# nothing\n').includes(`min-release-age=${SOAK_DAYS}`)) + assert.ok(fixNpmrc('min-release-age=3\n').includes(`min-release-age=${SOAK_DAYS}`)) }) test('npmrc excludes: version pins need dated annotations, globs do not', () => { // The shape a fleet repo actually uses: trusted scopes and bare names // are standing trust and need no annotation. const trusted = [ - 'min-release-age=7', + `min-release-age=${SOAK_DAYS}`, 'min-release-age-exclude[]=@socketsecurity/*', 'min-release-age-exclude[]=sfw', ].join('\n') @@ -83,12 +86,12 @@ test('workspace yaml: clean fixture passes', () => { }) test('workspace yaml: wrong minutes value is a finding', () => { - const bad = CLEAN_YAML.replace('10080', '1440') + const bad = CLEAN_YAML.replace(String(SOAK_MINUTES), String(SOAK_MINUTES + 1)) assert.equal(checkWorkspaceYaml(bad, 'y').filter(f => f.what.includes('minimumReleaseAge')).length, 1) }) test('excludes: flow-style list is rejected outright', () => { - const flow = "minimumReleaseAge: 10080\nminimumReleaseAgeExclude: ['left-pad@1.3.0']\n" + const flow = `minimumReleaseAge: ${SOAK_MINUTES}\nminimumReleaseAgeExclude: ['left-pad@1.3.0']\n` const findings = checkExcludeAnnotations(flow, 'y') assert.equal(findings.length, 1) assert.match(findings[0]!.what, /flow style/) @@ -106,7 +109,7 @@ test('excludes: wrong removable date is a finding; expiry is a warning, not a fi assert.match(checkExcludeAnnotations(wrong, 'y')[0]!.what, /removable date/) // Expired-but-valid is STALE, not unsafe: check exits clean, the stale // list reports it, and --fix / the soak-autofix workflow prunes it. - const expired = `minimumReleaseAgeExclude:\n # published: 2020-01-01 | removable: 2020-01-08\n - 'b@1.0.0'\n` + const expired = `minimumReleaseAgeExclude:\n # published: 2020-01-01 | removable: ${addDaysIso('2020-01-01', SOAK_DAYS)}\n - 'b@1.0.0'\n` assert.deepEqual(checkExcludeAnnotations(expired, 'y'), []) assert.deepEqual(staleExcludes(expired), ['b@1.0.0']) // Fresh and malformed entries are never "stale". @@ -132,14 +135,14 @@ test('fix and stale-list skip a wrong-arithmetic expired annotation', () => { // published + SOAK_DAYS != removable and removable is already past: // this must stay a check failure for a human, not silently prune — // the real window may still be open. - const yaml = `minimumReleaseAge: 10080\nminimumReleaseAgeExclude:\n # published: ${todayIso()} | removable: 2020-01-02\n - 'wrongmath@1.0.0'\n` + const yaml = `minimumReleaseAge: ${SOAK_MINUTES}\nminimumReleaseAgeExclude:\n # published: ${todayIso()} | removable: 2020-01-02\n - 'wrongmath@1.0.0'\n` assert.deepEqual(staleExcludes(yaml), []) assert.ok(fixWorkspaceYaml(yaml).includes('wrongmath@1.0.0')) assert.ok(checkExcludeAnnotations(yaml, 'y').length >= 1) }) test('fix prunes expired pins together with their annotations', () => { - const yaml = `minimumReleaseAge: 10080\nminimumReleaseAgeExclude:\n # published: 2020-01-01 | removable: 2020-01-08\n - 'old@1.0.0'\n # published: ${FRESH_PUB} | removable: ${FRESH_REM}\n - 'fresh@1.0.0'\n` + const yaml = `minimumReleaseAge: ${SOAK_MINUTES}\nminimumReleaseAgeExclude:\n # published: 2020-01-01 | removable: ${addDaysIso('2020-01-01', SOAK_DAYS)}\n - 'old@1.0.0'\n # published: ${FRESH_PUB} | removable: ${FRESH_REM}\n - 'fresh@1.0.0'\n` const fixed = fixWorkspaceYaml(yaml) assert.ok(!fixed.includes('old@1.0.0')) assert.ok(!fixed.includes('2020-01-01')) @@ -163,16 +166,22 @@ test('catalog parity: entries after a blank line are still checked', () => { test('taze config: window must be imported, not hand-copied', () => { const good = "import { SOAK_DAYS } from './scripts/soak/constants.mts'\nexport default { maturityPeriod: SOAK_DAYS }\n" assert.equal(checkTazeConfig(good, 't').length, 0) - assert.equal(checkTazeConfig('export default { maturityPeriod: 7 }\n', 't').length, 1) + assert.equal(checkTazeConfig('export default { maturityPeriod: 7 }\n', 't').length, 2) assert.equal(checkTazeConfig('export default {}\n', 't').length, 2) + const unrelated = + "// constants.mts and maturityPeriod are mentioned, but not wired together\nexport default { maturityPeriod: 1 }\n" + assert.equal(checkTazeConfig(unrelated, 't').length, 2) }) test('toolchain soak: nightly must be SOAK_DAYS old at adoption; stable passes', () => { - const good = '# adopted: 2026-07-11\n[toolchain]\nchannel = "nightly-2026-07-04"\n' + const channel = '2026-07-04' + const adopted = addDaysIso(channel, SOAK_DAYS) + const good = `# adopted: ${adopted}\n[toolchain]\nchannel = "nightly-${channel}"\n` assert.equal(checkToolchainSoak(good, 't').length, 0) - const tooFresh = '# adopted: 2026-07-11\n[toolchain]\nchannel = "nightly-2026-07-08"\n' + const tooFreshChannel = addDaysIso(adopted, -(SOAK_DAYS - 1)) + const tooFresh = `# adopted: ${adopted}\n[toolchain]\nchannel = "nightly-${tooFreshChannel}"\n` assert.match(checkToolchainSoak(tooFresh, 't')[0]!.what, /nightly soak/) - const noDate = '[toolchain]\nchannel = "nightly-2026-07-04"\n' + const noDate = `[toolchain]\nchannel = "nightly-${channel}"\n` assert.match(checkToolchainSoak(noDate, 't')[0]!.what, /adoption date/) const stable = '[toolchain]\nchannel = "1.95.0"\n' assert.equal(checkToolchainSoak(stable, 't').length, 0) diff --git a/scripts/soak/update-deps.mts b/scripts/soak/update-deps.mts index f557338e98..41f470a7a0 100644 --- a/scripts/soak/update-deps.mts +++ b/scripts/soak/update-deps.mts @@ -9,8 +9,8 @@ * - cargo: `cargo update` via rustup's shim. `.cargo/config.toml` carries * min-publish-age for the same window — an [unstable] cargo feature, so * it bites only under a nightly toolchain; on perry's stable toolchain - * the automated window rides dependabot's cooldown and this path is a - * plain update. + * the automated window rides dependabot's cooldown and this manual path + * refuses a mutating update. * * Usage: node scripts/soak/update-deps.mts [--npm|--cargo] [--dry-run] * (no ecosystem flag = both) @@ -67,6 +67,40 @@ function updateCargo(dryRun: boolean): number { console.error('[update-deps] rustup cargo shim not found — refusing a cargo that cannot follow the repo toolchain') return 1 } + // Before a mutating update, run the exact resolver in dry-run mode. Cargo + // stable accepts the config while warning that min-publish-age is unused; + // detecting that only after `cargo update` would be too late because the + // lockfile may already contain fresh releases. + if (!dryRun) { + const preflightArgs = ['update', '--dry-run'] + console.log(`[update-deps] preflight: ${RUSTUP_CARGO} ${preflightArgs.join(' ')} (in .)`) + const preflight = spawnSync(RUSTUP_CARGO, preflightArgs, { + cwd: REPO_ROOT, + encoding: 'utf8', + stdio: ['inherit', 'inherit', 'pipe'], + }) + const preflightStderr = preflight.stderr ?? '' + process.stderr.write(preflightStderr) + if (preflight.error) { + console.error(`[update-deps] ${RUSTUP_CARGO}: ${preflight.error.message}`) + return 1 + } + if (isBlockedByPublishAge(preflightStderr)) { + reportPublishAgeBlock() + return preflight.status ?? 1 + } + if (shouldRefuseUnsupportedCargoUpdate(dryRun, preflightStderr)) { + console.error( + '[update-deps] refusing mutating cargo update: this toolchain ignored\n' + + ' [unstable] min-publish-age, so it cannot enforce the dependency soak.\n' + + ' Use --dry-run for diagnostics or a pinned nightly that supports the key.', + ) + return 1 + } + if ((preflight.status ?? 1) !== 0) { + return preflight.status ?? 1 + } + } const args = dryRun ? ['update', '--dry-run'] : ['update'] // Report honestly when the cargo-side window did not apply. perry rides // stable, where `[unstable] min-publish-age` is a warning-only unused @@ -94,16 +128,7 @@ function updateCargo(dryRun: boolean): number { // bypass this design deliberately does not have. Say so before someone // copy-pastes it out of a red terminal. if (isBlockedByPublishAge(stderr)) { - console.error( - '[update-deps] the cargo soak BLOCKED this re-resolution: a requirement can\n' + - ' only be satisfied by a release younger than the window (see the error above).\n' + - ' This is the window working, not a bug. Options, in order of preference:\n' + - ' 1. wait out the remaining days and re-run;\n' + - ' 2. relax/repin the requirement so an already-soaked version satisfies it;\n' + - ' 3. if the fresh release is genuinely required, adopt it as a deliberate,\n' + - ' reviewable commit — NOT via CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE,\n' + - ' which silently disables the window for every crate in the graph.', - ) + reportPublishAgeBlock() return res.status ?? 1 } if (isMinPublishAgeUnsupported(stderr)) { @@ -116,6 +141,19 @@ function updateCargo(dryRun: boolean): number { return res.status ?? 1 } +function reportPublishAgeBlock(): void { + console.error( + '[update-deps] the cargo soak BLOCKED this re-resolution: a requirement can\n' + + ' only be satisfied by a release younger than the window (see the error above).\n' + + ' This is the window working, not a bug. Options, in order of preference:\n' + + ' 1. wait out the remaining days and re-run;\n' + + ' 2. relax/repin the requirement so an already-soaked version satisfies it;\n' + + ' 3. if the fresh release is genuinely required, adopt it as a deliberate,\n' + + ' reviewable commit — NOT via CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE,\n' + + ' which silently disables the window for every crate in the graph.', + ) +} + /** * cargo emits `unused config key ...` (a warning, exit 0) for an * `[unstable]` key it does not implement, so the ONLY signal that the soak @@ -125,6 +163,10 @@ export function isMinPublishAgeUnsupported(stderr: string): boolean { return /unused config key `unstable\.min-publish-age`/.test(stderr) } +export function shouldRefuseUnsupportedCargoUpdate(dryRun: boolean, stderr: string): boolean { + return !dryRun && isMinPublishAgeUnsupported(stderr) +} + /** * cargo's resolver failure when a requirement's only candidate is inside * the window: `version X is too new (published N days ago, minimum age M diff --git a/scripts/soak/update-deps.test.mts b/scripts/soak/update-deps.test.mts index 7a05973eda..ea0b8fd4da 100644 --- a/scripts/soak/update-deps.test.mts +++ b/scripts/soak/update-deps.test.mts @@ -5,6 +5,7 @@ import { isBlockedByPublishAge, isMinPublishAgeUnsupported, selectEcosystems, + shouldRefuseUnsupportedCargoUpdate, } from './update-deps.mts' test('no ecosystem flag updates both', () => { @@ -30,6 +31,8 @@ test('detects the unused-config-key warning that means the cargo soak did not ap const real = 'warning: unused config key `unstable.min-publish-age` in `/repo/.cargo/config.toml`\n' assert.equal(isMinPublishAgeUnsupported(real), true) + assert.equal(shouldRefuseUnsupportedCargoUpdate(false, real), true) + assert.equal(shouldRefuseUnsupportedCargoUpdate(true, real), false) assert.equal(isMinPublishAgeUnsupported('warning: unused config key `unstable.other`\n'), false) assert.equal(isMinPublishAgeUnsupported(''), false) assert.equal(isMinPublishAgeUnsupported(' Updating crates.io index\n'), false) From d1fa9fa49de668b456db925750ac9d5d807710d2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Wed, 29 Jul 2026 23:09:45 +0200 Subject: [PATCH 29/30] fix: address compile stack review --- .github/workflows/security-audit.yml | 2 +- .npmrc | 1 + .../src/object/global_this/ctor_thunks.rs | 29 ++++++++++++------- external-tools.json | 2 +- package-lock.json | 4 +++ package.json | 8 ++++- 6 files changed, 33 insertions(+), 13 deletions(-) diff --git a/.github/workflows/security-audit.yml b/.github/workflows/security-audit.yml index 15b4eae15e..a36ce88864 100644 --- a/.github/workflows/security-audit.yml +++ b/.github/workflows/security-audit.yml @@ -131,7 +131,7 @@ jobs: for skill in .claude/skills/*/; do echo "::group::skillspector ${skill}" uv tool run --python 3.12 \ - --from git+https://github.com/NVIDIA/skillspector@2eb84478 \ + --from git+https://github.com/NVIDIA/skillspector@2eb844780ab163f01468ecf142c40a2ec0fcaec0 \ skillspector scan "${skill}" --no-llm echo "::endgroup::" done diff --git a/.npmrc b/.npmrc index cc2470c4d1..64bb5f3ec1 100644 --- a/.npmrc +++ b/.npmrc @@ -6,4 +6,5 @@ # tools/pnpm-workspace.yaml, NOT here: pnpm only honors minimumReleaseAge # from a workspace yaml, and a workspace yaml at the repo root would mark # the npm-managed root as a pnpm workspace. +engine-strict=true min-release-age=7 diff --git a/crates/perry-runtime/src/object/global_this/ctor_thunks.rs b/crates/perry-runtime/src/object/global_this/ctor_thunks.rs index 2f7b5f55a8..dda8d7d90a 100644 --- a/crates/perry-runtime/src/object/global_this/ctor_thunks.rs +++ b/crates/perry-runtime/src/object/global_this/ctor_thunks.rs @@ -42,24 +42,33 @@ pub(crate) extern "C" fn regexp_constructor_call_thunk( pattern: f64, flags: f64, ) -> f64 { - let flags_undefined = flags.to_bits() == crate::value::TAG_UNDEFINED; - let pattern_value = crate::value::JSValue::from_bits(pattern.to_bits()); + let scope = crate::gc::RuntimeHandleScope::new(); + let pattern = scope.root_nanbox_f64(pattern); + let flags = scope.root_nanbox_f64(flags); + let flags_undefined = flags.get_nanbox_f64().to_bits() == crate::value::TAG_UNDEFINED; + let pattern_value = crate::value::JSValue::from_bits(pattern.get_nanbox_f64().to_bits()); if flags_undefined && pattern_value.is_pointer() { - let addr = (pattern.to_bits() & 0x0000_FFFF_FFFF_FFFF) as usize; + let addr = (pattern.get_nanbox_f64().to_bits() & 0x0000_FFFF_FFFF_FFFF) as usize; if crate::regex::is_regex_pointer(addr as *const u8) { - return pattern; + return pattern.get_nanbox_f64(); } } - let pattern_ptr = if pattern_value.is_undefined() { - std::ptr::null_mut() + let pattern_string = if pattern_value.is_undefined() { + None } else { - crate::builtins::js_string_coerce(pattern) + Some(scope.root_string_ptr(crate::builtins::js_string_coerce(pattern.get_nanbox_f64()))) }; - let flags_ptr = if flags_undefined { - std::ptr::null_mut() + let flags_string = if flags_undefined { + None } else { - crate::builtins::js_string_coerce(flags) + Some(scope.root_string_ptr(crate::builtins::js_string_coerce(flags.get_nanbox_f64()))) }; + let pattern_ptr = pattern_string + .as_ref() + .map_or(std::ptr::null(), |value| value.get_raw_const_ptr()); + let flags_ptr = flags_string + .as_ref() + .map_or(std::ptr::null(), |value| value.get_raw_const_ptr()); let re = crate::regex::js_regexp_new(pattern_ptr, flags_ptr); crate::value::js_nanbox_pointer(re as i64) } diff --git a/external-tools.json b/external-tools.json index 440998fc34..7f27a9f5b3 100644 --- a/external-tools.json +++ b/external-tools.json @@ -174,7 +174,7 @@ "description": "NVIDIA's third-party-skill security scanner (LangGraph-based; YARA + AST + OSV.dev CVE lookups + optional LLM analysis). No PyPI release / no GH tags upstream — pinned to a git SHA on main + installed via a locked uv project (pyproject.toml + uv.lock, `uv sync --locked`; the fleet uv pin + exclude-newer make it reproducible). Sibling to AgentShield: AgentShield audits the operator's .claude/ config; SkillSpector audits untrusted upstream skills before install.", "release": "uv-project", "repository": "github:NVIDIA/skillspector", - "version": "2eb84478", + "version": "2eb844780ab163f01468ecf142c40a2ec0fcaec0", "versionDate": "2026-05-18" } } diff --git a/package-lock.json b/package-lock.json index be24996754..088c7498f3 100644 --- a/package-lock.json +++ b/package-lock.json @@ -4,6 +4,10 @@ "requires": true, "packages": { "": { + "engines": { + "node": "^22.18.0 || >=23.6.0", + "npm": ">=11.17.0" + }, "dependencies": { "ethers": "^6.17.0", "node-cron": "^4.2.1" diff --git a/package.json b/package.json index 5fe3d3768f..7de215bad7 100644 --- a/package.json +++ b/package.json @@ -1,5 +1,11 @@ { + "private": true, "type": "module", + "packageManager": "npm@12.0.1", + "engines": { + "node": "^22.18.0 || >=23.6.0", + "npm": ">=11.17.0" + }, "scripts": { "soak": "node scripts/soak/soak.mts --check", "soak:fix": "node scripts/soak/soak.mts --fix", @@ -7,7 +13,7 @@ "tools:check": "node scripts/soak/external-tools.mts --check", "tools:fix": "node scripts/soak/external-tools.mts --fix", "tools:install": "node scripts/soak/external-tools.mts --install-all --shims", - "test:scripts": "node --test scripts/soak/*.test.mts" + "test:scripts": "node --test \"scripts/soak/*.test.mts\"" }, "devDependencies": { "mongodb": "^7.0.0", From 0cc026e1b74a3de9a8475615f1007512a790a622 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Thu, 30 Jul 2026 08:29:15 +0200 Subject: [PATCH 30/30] docs: correct compile-gap blocker count --- changelog.d/7021-real-npm-cli-compile-gaps.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/7021-real-npm-cli-compile-gaps.md b/changelog.d/7021-real-npm-cli-compile-gaps.md index 081c235808..18dbfa3552 100644 --- a/changelog.d/7021-real-npm-cli-compile-gaps.md +++ b/changelog.d/7021-real-npm-cli-compile-gaps.md @@ -1,4 +1,4 @@ -**Compile fixes from taking a real npm CLI (Socket Firewall) to a native binary** — four independent blockers, each fixed at its own layer: +**Compile fixes from taking a real npm CLI (Socket Firewall) to a native binary** — five independent blockers, each fixed at its own layer: - **Auto-optimize feature skew**: cross-features the on-disk checkout's `perry-runtime`/`perry-stdlib` don't declare are now dropped (with a warning) instead of failing the whole cargo resolve and silently falling back to a link that's missing the routed ext entrypoints. The cargo-failure fallback message now explains the consequence and remedy. - **`perry-ext-zlib` zstd surface**: `zlib.createZstdCompress`/`createZstdDecompress`, the zstd one-shots, and the streaming write-codec are now implemented in the ext wrapper, so routing `node:zlib` no longer strips the only zstd implementation out of the link (undici's web-fetch content decoding references it unconditionally).