Skip to content

chore: Local Docker build perf - #2406

Open
sergerad wants to merge 1 commit into
nextfrom
sergerad-fix-local-docker
Open

chore: Local Docker build perf#2406
sergerad wants to merge 1 commit into
nextfrom
sergerad-fix-local-docker

Conversation

@sergerad

Copy link
Copy Markdown
Collaborator

Speed up make local-network-build

make local-network-build compiled the full dependency tree (including RocksDB from source) once per image: the Dockerfile's cache mounts are keyed per BIN to keep concurrent CI matrix jobs from corrupting shared mounts, but local builds are sequential, so the keying just multiplied work five-fold. The unconditional source touch also forced a full workspace rebuild on every build.

Changes

  • New builder-local stage: compiles all five binaries in a single cargo invocation with cache mounts shared across binaries. The stage never references BIN, so its layers are identical across all five image builds — the first build compiles, the remaining four hit BuildKit's layer cache.
  • No source touch in the local stage: local build contexts preserve real mtimes (and git bumps the mtime of anything it changes), so Cargo's fingerprinting is sound and rebuilds are genuinely incremental. The .cargo-ok registry heal is kept since Ctrl-C'd local builds corrupt the registry the same way cancelled CI jobs do.
  • Memory-aware job cap (~2GiB per job, override via CARGO_BUILD_JOBS): an uncapped build OOMs the default 8GiB Docker Desktop VM because the release profile carries full debug info.
  • Runtime stage selects its builder via a BUILDER build arg (builder-ci default, renamed from builder). CI is unaffected: it never passes the arg, the builder-ci instructions are unchanged (so no cache invalidation), and builder-local is unreachable in CI builds.
  • Makefile: local-network-build passes BUILDER=builder-local and drops --pull; deduped the doubled docker-build-monitor prerequisite; tightened Dockerfile comments (including correcting the "Docker normalises COPY timestamps" claim — the stale-rlib hazard comes from the target cache mount being shared across builds of divergent sources, not from COPY).

Result: a cold make local-network-build compiles the workspace once instead of five times; a no-change rebuild goes from five full workspace rebuilds to a cache hit; a small change recompiles only affected crates, once.

Changelog

changelog = "none"
reason    = "Dockerfile internals change only."

@sergerad
sergerad enabled auto-merge (squash) July 29, 2026 22:07
@sergerad sergerad changed the title Add builder-ci and builder-local chore: Local Docker build perf Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant