Repository navigation
Bound Go cache writes while preserving main-backed restores #565
Description
🤖 Generated by the Agentic Engineer
Problem and impact
KSail's Go caches alone exceed its stated 10 GB repository cache budget after the mirror-artifact repair. Redundant multi-gigabyte archives leave less room for useful CI caches and prevent the sustained budget outcome in devantler-tech/ksail#7348. The shared Go validation workflow owns part of the remaining write policy.
Reproduction and reviewed source
A complete 2026-10-11 inventory contains 18 distinct entries totaling 11,660,983,540 bytes. The aggregate endpoint agrees on bytes but reports 15 entries; retain that count discrepancy. Six Go archives total 10,537,301,062 bytes, including nearly identical setup-go entries on main (2,638,172,994 bytes) and PR #7680 (2,638,121,633 bytes). No cache was manually deleted.
The successful main Test job, at KSail main c758dc449f9422d98e257bf93420adffcc7088d6, restored the 2.64 GB setup-go archive, then saved a separate 1,355,237,861-byte test compilation cache. The Coverage job saved a separate 1,513,044,674-byte coverage compilation cache. This is successful CI with excessive storage, not a failing-test diagnosis.
That run records validate-go-project.yaml at 8d3327fe87f216d6e107941de4ce1ae019294cea as its invoked source. Its test and coverage jobs enable setup-go caching and also save separate GOCACHE archives. The workflow blob is unchanged on current main c1aaea1b8470627e75a3c3f7caee74e0a93ca4de (6bd4da1d426f21607d2c8a1fa9d43794b95f5aea). Other Go jobs also enable setup-go caching.
The pinned setup-go package paths include both GOMODCACHE and GOCACHE. Its post-job saver saves those paths on a non-exact match. GitHub's cache-scope rules allow PRs to restore main caches, while PR-created entries remain scoped to their merge ref. These facts identify redundant storage; the inventory alone does not identify which racing job first saved each shared key.
Expected outcome and bounded delivery
Provide a reviewed cache policy in shared Go validation that retains main-backed dependency restores while reducing duplicate large PR archives and overlapping build-cache storage. Preserve separate ordinary-test and race/coverage compilation modes. The policy must work for both workflow_call consumers and the organization-required direct invocation; an input that is absent on the direct path is not proof of adoption.
This child owns the shared workflow, its documentation, regression checks, release and native behavior proof. KSail-specific setup-go calls and adoption of the released shared policy remain delivery work under the parent and require their own bounded change before implementation. Keep other consumers compatible, and describe any intentional behavior change explicitly.
Acceptance criteria
- Bind every changed writer to exact source and distinguish module, ordinary-test and race/coverage caches.
- Required tests, scanners, extraction coverage, signed-fix boundaries and mirror artifact integrity checks remain enabled.
- Main cache population and PR restores remain useful; native CI demonstrates the selected PR write policy and main save/restore behavior on both invocation paths. A YAML-only assertion is insufficient.
- Regression controls reject an accidental return of redundant PR writes or removal of the intended restore path, while proving the allowed main write path.
- Merge, release and record immutable shared-policy adoption evidence for the parent. Do not claim the whole-repository budget from this shared change alone.
Priority rationale: High: the remaining cache-budget criterion has a confirmed failed checkpoint and affects repeated KSail CI runs. Effort: Medium. No new implementation starts while the review stage is at its ceiling. Preserve the parent's two failed checkpoints; register a new full observation period only after all remaining repair work actually delivers.
Delivered on: 2026-10-12 — #569, shared policy release v7.9.4. Original failure checkpoints remain unchanged.
Measure on: Event — KSail adopts the released policy on its called and organization-required paths, then completes a main run and dependency PR with Go jobs actually executed. Read the exact source pins, step outcomes and complete cache inventory before deciding the parent outcome.
Current obstacle: outcome — the language-selected required Go source is now migrated by #645/#646, released as v7.16.2 and verified natively in the applied policy and inherited KSail rule. Complete the qualifying main/dependency-PR behavior proof, not merely a source readback. KSail #7864 is a CI-only ordinary consumer, not a dependency update; its direct required Go Test and race/coverage jobs have restored separate main-backed caches and skipped Go saves, and all eight selected Go jobs have now completed successfully. Maintained-caller adoption #7709 and the earlier catalogue native receipts below remain valid. Preserve the original clocks and failed checkpoints; this queued verification remains unfinished WIP.
The dependency-bearing catalogue caller now supplies the previously missing real module archive: main run 38300507280 at dacbc94a362a456a4db44a61b15e0a05b76a8cf6 published one exact go-mod-v1 archive of 14,519,689 bytes. PR run 38297676055 at ba27ace57f952332f2a19e58076afc29f1118007 restored modules in all eight Go setups and ran no Go cache save. The read-only observer's current source was exercised against both runs. The required automated observer is being delivered separately in #576 / #570.
These catalogue results do not satisfy the consumer adoption criterion or prove KSail's full cache-budget recovery. The inspected documentation-only direct consumer runs skipped Go jobs, so they are excluded from behavior proof.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Effort
Projects
- Status
Activity
devantler commentedon Oct 11, 2026
Remaining main writers identified, 2026-10-11
The native logs identify the first successful writers for both large main setup-go entries at KSail
c758dc449f9422d98e257bf93420adffcc7088d6:Both logs show the archived paths as GOMODCACHE and GOCACHE. Tidy saved key suffix
0b0b556bf295be158c20f3de95c755f880e3d2c3c85abee87e31d70adae50f4a; Vulnerability Scan savedce4e07a29b8166e20bc12395117f41b0b76b8729aa7107d5917521ff41d11c97. Their byte counts and timestamps match the complete earlier inventory.The different keys have a source explanation: the Tidy setup leaves setup-go's dependency path at its pinned default, go.mod. The scanner action explicitly keys on go.sum. Their setup-go post steps therefore publish separate multi-gigabyte archives. This identifies the writer and key divergence; it does not measure the overlap inside their compressed archives.
The separate ordinary-test and race/coverage compilation caches remain necessary acceptance scope. A repair must also cover the scanner action's nested setup-go saver, preserve its scan and reviewed scanner version, and exercise both the direct required workflow and workflow_call consumers. Changing only the Test/Coverage steps would leave these two main writers intact.
A fresh paired read at approximately 18:13 UTC returned ten distinct inventory entries totaling 5,540,985,664 bytes, while the aggregate endpoint reported 21 entries and 15,281,780,399 bytes. Preserve this disagreement as unknown; the lower list is not evidence of sustained recovery and cannot erase the earlier confirmed 11.66 GB failure. No cache was manually removed.
chore: agent-claim #565 nonce=a31306306a4aa5dfeca5328a6df08e57
devantler commentedon Oct 12, 2026
Delivered, verification open — 2026-10-12
#569 merged as
cd706d23. The merge closed this issue through aFixesline left in the commit message; I reopened it because three acceptance criteria are not yet evidenced. The card is in Verifying.What changed. Every Go job in shared validation, and the internal scan action, now set up Go without its built-in cache and restore one module archive keyed by the module's
go.sum. Only the default branch's Test job writes that archive, and it starts from an exact match so the archive does not accumulate retired module versions. A pull request falls back to the default branch's newest archive and writes nothing. The separate test and race-coverage build archives are kept and are saved on the default branch only.Evidence so far. Pull-request behaviour on real runners is recorded on #569 (restore attempted, both save steps skipped, the scan action shares the key). The policy check rejects 21 unsafe edits.
Still to record here:
mainrun of this repository after the merge (run 38228902528): the save steps' outcomes.main, and a dependency pull request that writes no Go archive, on both the called and the organisation-required direct path. Then a fresh cache inventory for Mirror caches are evicted before system tests restore them ksail#7348. No whole-repository budget claim before that.Not covered by this change: the whole-repository lint workflow still uses the built-in cache when a caller gives it a Go version file; an automated native observer is #570.
devantler commentedon Oct 12, 2026
Main-branch readback and release — 2026-10-12
cd706d2309d8c4f192f02c902214c6bd1b5a8e69.mainrun after the merge: 38228902528, success. In the Test job:Save Go module cacheran, which it never does on a pull request. It stored nothing and printed a warning, because this repository's fixture has no dependencies and so no module directory exists. The cache list confirms nogo-mod-v1entry;Save Go compilation cachewas skipped because the build archive was an exact hit;So the routing is proven on real runners in both directions: a pull request reads and does not write, and
mainis the only place a save runs. Not yet proven: a real, non-empty module archive being written and then reused. That needs a module with dependencies, which means KSail's adoption of v7.9.4 (or the fixture in #570).mainrun recorded.main, no Go archive from a dependency pull request, on the called and the direct path; then a fresh cache inventory for Mirror caches are evicted before system tests restore them ksail#7348.The warning on every
mainrun for the dependency-free fixture is noise, not a failure; #570 removes it by giving the observer a fixture with a dependency.chore: agent-claim #565 nonce=a3bf6d985bdb460827e5a0d10267edbe
chore: agent-claim #565 nonce=73909cda7f75da8cee08c9bf2cbb159b
devantler commentedon Oct 12, 2026
The catalogue observer child #570 is completed. #576 shipped the required real-run guard, and the supporting database lifecycle repair #581 is merged and released as v7.10.3 at
9ff2ad68306cc73df6814861c7804a11a25e0788. Complete current-main CI passed with 204 successful checks, 54 intentional skips and no pending/failed checks. Its required observer proved all eight restores, exact module reuse, skipped Test module save and one unchanged 14,519,689-byte main archive. Native PR evidence and refusal controls are linked on #570.This does not complete the consumer checkpoint. A fresh public read of KSail's main caller still resolves
validate-go-project.yamlto8d3327fe87f216d6e107941de4ce1ae019294cea(v7.9.1). Retain this issue's existing Measure-on event and original clocks: adopt the released policy on both called and organisation-required consumer paths, then measure real Go main/dependency-PR runs and the complete consumer cache inventory. No KSail budget conclusion follows from this catalogue result.devantler commentedon Oct 12, 2026
KSail's maintained caller adoption is now delivered in merged #7709, main
ab05a334b4901eaefd9875a6a66962c004fdb83d. Native receipts show all eight maintained Go jobs passed, exactly one 2,185,830,388-byte main module archive was saved, separate test/coverage compilation restores worked, and the following reviewed setup consumer restored that exact module archive without another Go save.The required-path criterion remains unmet: the live Go rule still resolves from legacy Actions, with native historical source
6f3d2b43ca2f00c772d4eb62c0c0c0cfa7a23a53confirmed on the consumer PR. #237 already owns that migration. These receipts supersede the maintained-caller old-pin obstacle; they do not justify closing this both-path acceptance or KSail #7348's sustained cache-budget outcome.devantler commentedon Oct 15, 2026
Corrected the stale Current obstacle paragraph to match the already-recorded #7709 adoption, and added the missing native blocked-by relationship to #237. Fresh public reads bind the maintained caller to KSail main
1acebf37148faf47730a9415646596d40c4f38a9and catalogue8ed2071ebe18864b8b86efe0f6b1df8d73d2a6ab; the intervening maintained workflow diff changes hardening/signed-fix pins and preserves cache policy. The live inherited Go workflow rule still selects legacy Actions (repository_id:948529001) atrefs/heads/main. No settings or consumer code changed. This preserves the both-path acceptance criteria and earlier cache-budget checkpoints; #565 stays open.devantler commentedon Oct 18, 2026
The legacy language-selected Go source obstacle is now removed by #646 / v7.16.2. Native deployment and inherited KSail rule readback both name the maintained catalogue, with the complete original protections preserved. Ordinary KSail PR #7864 is executing that direct required workflow at head
8363ebd655cc586e4b6c341e1c0bfdc99a0110ce.Its completed ordinary Test job
117672608677explicitly records setup-go cache disabled, exact main-backed module and ordinary compilation-cache hits, successful real application tests, and skipped Go module/compilation cache saves. The separate schema cache still saves as designed; it is not a redundant Go archive. Remaining selected Go jobs are not yet claimed complete.Keep this issue open: the requested qualifying dependency-PR observation and both-path/main proof still require their own complete evidence. This CI-only consumer is not relabeled a dependency update, and its under-budget point does not clear KSail #7348 or the managed CodeQL writer.