ci: track root causes and restore CI on latest, release/v11, and release/v10 #10027

Purpose

Track restoration of CI and post-release integration on latest, release/v11, and release/v10. Group failures by verified cause, not by runtime or job name. Distinguish a merged fix, a published dependency, a passing job, and verified execution of the affected tests.

Current verified baseline — November 5, 2026

These are the latest completed Release runs on each branch, all at attempt 1. The inventory below accounts for all 13 failed jobs: 7 on latest, 3 on v11, and 3 on v10. This replaces the September/October runs as the current failure baseline.

These Release failures are downstream integration failures, not evidence that npm package publication failed. A green ordinary CI run does not exercise the complete Release Integration matrix. Dependent jobs skipped after a failed build are not counted as failures or passing coverage.

Every current failed job and its coverage

All job links below belong to Release / Release Integration / publish (...).

Uncovered work

• Gap A — Node test header-install stdout failures (four jobs across all three branches). Preserve the Node/addon test coverage and investigate/correct the stdout transport used by the Python header installer. The shared observed failure is nonblocking stdout/flush, not a CITGM fixture failure. Do not attribute the initiating runner behavior to a particular Python or runner release without evidence. Subsequent ccache deletion errors (Resource not accessible by integration) are cleanup errors, not the first failure.
• Gap B — Python version compatibility for older Node source builds (v11/23 and v10/18). Select a supported Python for the affected configure/build and test steps, keeping source templates and generated workflows aligned. Source-download retries cannot fix configure rejecting Python 3.14.4.
• Gap C — v10 nightly startup and result aggregation. release/v10 still calls semver.satisfies without includePrerelease: true; fix: accept supported Node.js prereleases at startup #10175 and its v11 backport fix: accept supported Node.js prereleases at startup #10195 do not cover v10. Backport the supported-prerelease startup behavior and numeric exit/discovery handling; preserve strict smoke stderr assertions.
• Prepare a branch-appropriate ci-integration-v10 patch and assess additional corrections for chore: restore release integration #10234/chore: restore release integration (v11) #10235. No v10 integration PR currently exists for these gaps, and neither existing integration PR claims to solve all 13 jobs.

Gaps A and B are explicitly owned by this tracker pending dedicated child issues/patches. Gap C reuses #10064 rather than inventing a separate issue for the same assertion on another branch. No speculative workaround, fixture removal, retry suppression, or workflow rerun has been applied by this audit.

Open npm/cli PRs and what they actually cover

#10234 | chore: restore release integration

#10235 | chore: restore release integration (v11 backport)

Both are open. Their current ordinary PR checks pass, including lint, smoke tests, Linux/macOS/Windows tests, and Windows Shims Tests. The new Cygwin executable probe runs, and the Cygwin subtest executes; its four existing cwd-related skips remain unchanged.

• latest Windows execution
• v11 Windows execution

The PRs provide install-time fixture environment propagation, native-only legacy npm_config_nodedir, numeric npm result aggregation and checked workspace discovery, CITGM version logging, archive HTTP handling, and Cygwin provisioning improvements. The v11 PR additionally backports existing Node source-download protection.

For the current inventory, their direct native-header changes target the two microtime failures. The latest PR is necessary but not sufficient for the four JSONStream failures: upstream fixture metadata must also be merged, published, and consumed. The remaining seven failures (four Node test jobs, two Node build jobs, and v10 nightly npm) are not covered by these PRs. Ordinary PR checks have not verified repaired Release Integration.

Child issue status and historical dispositions

Checkboxes below reflect GitHub issue closure, not an assumption that a proposed patch has shipped.

• ci: Node.js nightly integration fails while extracting source archive #10016 — Node source archive handling; closed. Existing latest protection is also backported in chore: restore release integration (v11) #10235. Current stable build failures are Python configure failures, not this archive failure.
• ci: binary-split CITGM source download uses stale repository owner #10020 — GitHub issue remains open. The stale binary-split repository metadata was fixed by merged fix: update binary-split repository nodejs/citgm#1142 and released in CITGM 10.0.4. Current latest/v11 binary-split jobs are successful; chore: restore release integration #10234/chore: restore release integration (v11) #10235 add archive diagnostics and version logging rather than repairing the already-corrected owner.
• ci: release integration selects Node 23 outside npm 12 supported engines #10024 — Node 23 outside npm 12's supported engine matrix; closed. The present v11 Node 23 failure is unrelated Python compatibility, not npm 12 engines.
• ci: CITGM Git-dependency fixtures lack npm 12 policy configuration #10026 — Open. JSONStream still fails in all four current latest runtimes; requires upstream metadata and CLI install-time environment propagation.
• ci: CITGM relies on native-build option forwarding rejected by npm 12 #10023 — GitHub issue remains open, but lookup: update thread-sleep metadata nodejs/citgm#1159 is merged, and its thread-sleep cleanup is released in CITGM 10.0.4. Current latest/v11 thread-sleep jobs are successful; the fixture remains marked flaky, so a green job alone is not proof of every command passing.
• ci: live-registry smoke test fails on a socket reset with fetch retries disabled #10022 — Open. No reset reproduced in local large-install validation, and current ordinary CI smoke jobs pass. Initiating cause remains unresolved; no resilience change was made.
• ci: Cygwin provisioning fails before Windows shim tests run #10025 — Open. chore: restore release integration #10234/chore: restore release integration (v11) #10235 update and diagnose Cygwin provisioning; both PRs now have passing Windows execution evidence. Historical installer causality remains unproven.
• ci: nightly npm smoke test rejects expected engine warning #10064 — Open. Startup prerelease handling is merged on latest (fix: accept supported Node.js prereleases at startup #10175) and v11 (fix: accept supported Node.js prereleases at startup #10195); no corresponding nightly npm failure exists in their current runs. v10 still reproduces the warning assertion, and needs the startup backport as well as tracker-owned aggregation correction.
• ci: nightly native CITGM fixtures ignore configured Node headers #10065 — Open. Leveldown was retired by merged lookup: remove leveldown nodejs/citgm#1122 and released in CITGM 10.0.4; do not call retirement a passing native build. Microtime remains failing on latest/v11 nightly and has the workflow fix in chore: restore release integration #10234/chore: restore release integration (v11) #10235.
• release: GitHub 502 leaves npm 11.21.0 release records incomplete #10066 — GitHub issue remains open, but the previously missing release records are already recovered, not currently missing. v11.21.0 and config-v10.14.0 are published. Their tag commit and registry gitHead have identical source trees. Recovery run 38615337436 executed Release Integration; its downstream failures must not be confused with missing records. Do not recreate records or recommend an unconditional release rerun.

Upstream CITGM state

The earlier CITGM 10.0.4 release PR nodejs/citgm#1147 is merged and 10.0.4 is published. The old description of that release as pending is obsolete.

Historical baselines retained for context

These historical runs remain evidence for their child issues; they are not the current open failure list.

Completion criteria

• Resolve or explicitly disposition every owner, including new gaps A/B and v10 scope.
• Merge and consume required npm/cli and CITGM fixes on the appropriate release lines.
• Capture post-fix Release Integration evidence for all three branches: affected installs, compiler/header usage, package tests, Node tests, and strict smoke assertions must execute.
• Account for flaky tolerances, existing skips, fixture retirements, and skipped dependent jobs; do not equate a green summary with restored coverage.
• Reconcile still-open historical child issues using their actual fix/execution evidence without presenting an unresolved infrastructure hypothesis as fact.

Keep this tracker open: the two current PRs do not cover the entire failure inventory, JSONStream metadata is still pending upstream, and complete post-fix Release Integration has not been verified.