Repository navigation
ci: split the workspace test job and cache the PR Docker build - #495
Merged
Merged
Conversation
…Docker build mounts
samgbafa
force-pushed
the
ci/split-test-and-docker-cache
branch
from
November 1, 2026 03:45
6266b4c to
3378e36
Compare
samgbafa
marked this pull request as ready for review
November 1, 2026 04:32
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
CI-only (
.github/workflows/rust.yml,.github/workflows/docker.yml; no Dockerfile, release.yml, release-plz.yml, tag/deploy/publish, logship, write-probe or personal-node change). Approved by Sam as "the free split" (audit:/var/tmp/ci-audit-2026-11-01/REPORT.md, F5).testis a 4-leg matrix with the samepostgres:16-alpineservice, env, iptables check, failure dump and step timeouts in every leg.cargo test --workspaceis partitioned by cargo target kind, keeping--workspaceso feature unification and build artifacts are identical:unitcargo test --workspace --libintegration-a-ocargo test --workspace --bins --test '[a-o]*';cargo test --workspace --docintegration-restcargo test --workspace --test '[!a-o]*'(exact glob complement: a new test file lands in exactly one leg)duckdb--features duckdbcommands, unchangedA
test-gatejob keeps the old check name and fails unless the whole matrix succeeded.PR-only Docker mount cache. The Dockerfile's
RUN --mount=type=cachedirs (cargo registry,/app/target) start empty on a hosted runner and are not part of the exported layer cache, so the cargo-chef cook layer cached nothing and every PR push recompiled all deps plus the app in release. Two new steps, bothif: github.event_name == 'pull_request':actions/cache+reproducible-containers/buildkit-cache-dance(pinned to the v3.3.2 commit SHA) restore/inject/extract those mounts, keyed per commit. Image, target, Dockerfile,cache-from/cache-to, and every push/tag/release/dispatch path are untouched.Job graph, old vs new (
ciworkflow; docker.yml and others unchanged)Renames: none lost.
test (workspace, PostgreSQL-backed)now is the gate job; new checkstest (workspace, PostgreSQL-backed, unit | integration-a-o | integration-rest | duckdb). Job idtestis now the matrix;test-gateis new.Identical test coverage
Old single job (main push run 41059168544, same source tree as this PR's base
d7f88e23) vs the four new legs (PR run 41267197637),compare2.py, multiset of executedtest <name> ... <status>lines plus per-binaryRunning/Doc-tests/test resultlines:(The log merges stderr
Runninglines with stdout test lines out of order, so a per-binary attribution script shows false mismatches in the old single-job log too; the compare above does not depend on attribution.) Compare scripts/output:/var/tmp/tcnode-ci-sIYe/data/compare2.py,compare-output.txt.Numbers (minutes)
BEFORE, 20 most recent all-green
pull_requesthead SHAs (gh API, base main; PR-to-green = first run created to last workflow update across ci + Docker + Release):test (workspace, PostgreSQL-backed)jobbuildjobAFTER, real runs on this PR. Measuring runs add a one-line comment to
tinycloud-core/src/lib.rs(reverted; the final head diff is the two YAML files only) so Docker really recompiles; a source-identical head hits the existing layer cache from main and builds in about 4 min.buildjobHonest reading: with warm caches, the test side drops from 26.6 to about 9.4 min and Docker from 26.5 to 16.7 min on a warm same-PR push, so PR-to-green goes about 28.6 -> 17.5. The audit's 9.2 ceiling is not reached: the release compile of the app is about 12 min even with deps cached, plus about 3 min of inject/extract. The first push of any PR is still a cold Docker build (about 28 min), because a PR's mount cache is private to that PR; seeding it from main would need a non-publishing job and is not done here. Runs 1, 3 and the final attempts are not a clean steady state: see risks.
Risks / not measured
rust-cachekey (rust-1.97.1-test-duckdb, cold until the first main push after merge saves it) and each PR push saves one mount-cache entry. If it is evicted, that leg is cold (about 22 min, same as the old job). The default legs keep the existingrust-1.97.1-test-default-duckdbkey, warm on day one. I deleted this PR's own caches after measuring. A cap increase or cleanup is Sam's call.save-if.Actions cost
Public repo: standard runners are $0 net. Runner-minutes per ci run go up: legs sum to about 34.4 min warm (565+498+442+561 s) vs 26.6 min median before, +7.8 min, plus the 3 s gate; one more PG service start and cache restore per leg. Docker adds about 3.3 min inject/extract/restore per PR push but saves about 11 min compile on warm pushes. At the $0.006/min list price the extra is about $0.05 per ci run gross, $0 billed.
Independent review
Spawned one reviewer agent (not me): VERDICT: APPROVE, no required changes. Full verdict:
/var/tmp/tcnode-ci-sIYe/review-verdict.md(host path, agent host). Covers coverage/exactly-once, services/features, gate, no release/deploy path or Dockerfile change (docker steps cannot run on push/tag/release/dispatch), supply chain and cache risks. The three scripts that parse docker.yml (test-docker-tags.py,test-deploy-recovery.py,check-deployment-policy-probes.mjs) pass.Merge order
Frozen order stands: 1.25.14 release, then #491 -> #489 -> #490 -> #492 -> #493 -> #494 (TC-2092); this merges after them and will be rebased when asked. Change is two workflow files only. Not merged.