Skip to content

Keep the CI package cache out of the packaged app - #267

Merged
juanmaguitar merged 1 commit into
trunkfrom
juanmaguitar/unbreak-the-artifact-builds
Aug 11, 2026
Merged

Keep the CI package cache out of the packaged app#267
juanmaguitar merged 1 commit into
trunkfrom
juanmaguitar/unbreak-the-artifact-builds

Conversation

@juanmaguitar

Copy link
Copy Markdown
Collaborator

Why

No branch has produced a signed macOS or Windows artifact since the last version bump. The build dies during packaging:

⨯ ENOENT: no such file or directory, open '…/vendor/npm/_logs/2026-08-10T14_15_09_297Z-debug-0.log'

Trunk fails the same way, re-running a job does not help, and reviewers have had no way to test a pull request on a real machine — the only reason that build layer exists. Linux is unaffected, which is the first clue: it is the one job that does not use this cache.

Details in #265.

What changes

Root cause. The artifact build keeps npm's package cache at vendor/npm, inside the project directory, and npm writes its own debug logs into it. package.json had no files filter, so electron-builder's default of "everything in the project directory is application content" swept that cache into the app. A cache directory is not static while a build runs: a log file present when the walk begins and gone by the time it is read takes the whole build down.

Two changes, addressing two separate mistakes:

  • package.json — exclude vendor from packaging. This is the fix. A directory that mutates during the build must not be a packaging input. The files array holds only a negation, which electron-builder handles by prepending its own **/* default (fileMatcher.jscontainsOnlyIgnore()), so nothing else about what gets packaged changes. The !x{,/**/*} form is the one electron-builder uses internally for the same job.
  • The CI script — clear npm's logs on both sides of the cache. After the restore, so no build inherits logs written by another build on another machine; before the save, so none are archived onward. This is cache hygiene, not the cure: npm ci writes a fresh log during the build regardless, so only the exclusion above closes the failure.

.gitignore gains vendor/npm alongside the existing vendor/bundle, so the cache cannot be committed by anyone who runs the CI script locally.

Why the version bump was the trigger. The cache key hashes package-lock.json. The bump minted a fresh cache entry, and the build that created it archived its own debug log into it — the timestamp in the error is that build's npm ci. Every build since has restored the poisoned entry.

How to test this

Platforms: macOS and Windows. These are the two jobs that use this cache; Linux never had the problem.

The end-to-end test is this PR's own artifact build. Locally, packaging can be driven directly:

Starting state: a clean checkout with dependencies installed.

  1. Stage a stand-in for the CI cache:

    mkdir -p vendor/npm/_logs && echo x > vendor/npm/_logs/stale-debug-0.log
    
  2. CSC_IDENTITY_AUTO_DISCOVERY=false npx electron-builder --dir --publish never

  3. npx asar list "dist/mac-arm64/WordPress Contributor Toolkit.app/Contents/Resources/app.asar" | grep '^/vendor'

    Expected: no output. On trunk the same command lists /vendor, /vendor/npm, /vendor/npm/_logs and the log file itself — that is the bug, visible without needing to reproduce the race.

  4. Same command without the grep, confirming the app is still whole: /src/main.js, /src/preload.js, /src/renderer/index.js, /package.json and the node_modules tree are all present.

What must not have happened: the app must not have got smaller in any way other than losing vendor. A wrong files pattern would silently drop src or node_modules and the packaging step would still report success — the app would only fail when a contributor launches it. Step 4 is what catches that. Also check the CI log still shows the npm cache being restored and saved: the cleanup must not have removed _cacache, or every build re-downloads every package.

Risks and limitations

  • The crash itself was not reproduced locally. It needs the log file to vanish mid-walk, which is a race between processes on the CI machine and could not be staged. What is verified is the mechanism underneath it: the cache directory is packaged, and it does contain those logs. The green build on this PR is the confirmation.
  • vendor is excluded; other non-app directories are not. docs, test, scripts and fastlane still get packaged, but they are ~217 entries against 28085 for node_modules, so there is nothing to win there. The real artifact bulk is the dependency tree — see Stop shipping build-only deps in the app #23, which measured it and is a separate concern.
  • No unit test. Neither an electron-builder config key nor a CI shell script is reachable from node --test; a test asserting the string "!vendor{,/**/*}" is present would restate the diff rather than test it. Verified by actually packaging instead, twice, and the assertion is written into the testing steps above so it can be re-run.
  • The published artifacts up to now have been carrying the build machine's package cache. Whether that is a meaningful share of their size will show when this build's artifact size is compared against the last one.

Related

Fixes #265. Adjacent to #23 (packaged app carries more than the app), which attacks artifact size through the dependency tree and is stale and conflicting — it needs its own decision.


Design decisions and alternatives considered

Moving npm's cache out of the project directory would have fixed the crash at its root, and was the first instinct. Rejected because the cache path is load-bearing for the CI cache save/restore, which archives that directory by relative path — relocating it means reworking the caching, on a pipeline that is currently red. Excluding it from packaging is the smaller change and the correct invariant regardless of where the cache lives.

Setting npm's logs-dir outside the project is the tidier version of the log cleanup and would make the deletions unnecessary. Not done here because it needs a path that is valid on both the macOS and Windows agents, which is a second decision to get wrong while the pipeline is broken. Worth revisiting once builds are green.

An inclusive files list naming what to package would be more explicit than a lone negation, and electron-builder's own docs argue against it: an inclusive list silently drops anything added later. The negation keeps the default and states only the exception.

Review outcome (required — see AGENTS.md)

Reviewed against the five dimensions in .github/instructions/code-review.instructions.md.

npm run lint clean. npm test — 780 passing, 0 failing.

1 [fix here] · 1 [follow-up] — the [fix here] was fixed before this PR was opened.

  • 🟡 architecture · [fix here] — the first draft cleared _logs only before save_cache. That leaves the already-poisoned cache entry in place, since the save does not overwrite an existing key, so builds would have stayed red until the lockfile changed again. Moved the primary cleanup to just after restore_cache.
  • 🔵 tests · [follow-up] — no automated coverage that vendor stays out of the asar. A packaging assertion would need a build in CI rather than a unit test; noted in Risks above rather than filed, as it is one assertion on a directory that no longer exists in the app.

Considered and cleared: no new dependency, no host binary, nothing crossing IPC, no spawn, no path composition, and rm -rf targets a literal quoted path set two lines above under set -u. The bash script runs on both the macOS and Windows agents and uses nothing platform-specific.

Implementation notes

The negation-only files behaviour is not folklore — app-builder-lib/out/fileMatcher.js:

if (!matcher.isSpecifiedAsEmptyArray && (matcher.isEmpty() || matcher.containsOnlyIgnore())) {
  customFirstPatterns.push("**/*")
}

containsOnlyIgnore() is true when every pattern starts with !, so the default include is prepended and the exclusion is layered on top. The same function adds electron-builder's own exclusions in exactly the !${dir}{,/**/*} form used here.

Local verification, on this branch versus without the files key, with a stand-in cache staged:

/vendor entries in app.asar total entries
without files 6 28308
with files 0 28302

The 6-entry difference is exactly the staged cache. The byte difference is trivial here only because the stand-in cache is a few hundred bytes; on a CI agent it is the full package cache.

The artifact build kept the npm package cache inside the project directory, and
electron-builder had no `files` filter, so it treated that cache as application
content. A cache directory changes while the build runs: a log file that was
present when the walk started and gone by the time it was read aborted the whole
build, which is why every branch and trunk alike stopped producing signed macOS
and Windows artifacts.

Exclude `vendor` from packaging, and clear npm's logs on both sides of the CI
cache so no build inherits another build's diagnostics.

Verified by packaging locally with a stand-in cache in place: without the filter
`vendor/npm/**` lands inside `app.asar`, logs and all; with it the directory is
absent and `src/main.js`, `src/preload.js`, the bundled renderer and all 28085
`node_modules` entries are still there.

Fixes #265

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juanmaguitar
juanmaguitar merged commit 05699bc into trunk Aug 11, 2026
3 checks passed
@juanmaguitar
juanmaguitar deleted the juanmaguitar/unbreak-the-artifact-builds branch August 11, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No branch can produce a signed artifact any more

1 participant