Skip to content

Clone a task's submodules from the repo's own checkout - #418

Draft
tildesrc wants to merge 2 commits into
mainfrom
panopticon/local-submodule-hydration
Draft

tildesrc wants to merge 2 commits into
mainfrom
panopticon/local-submodule-hydration

Conversation

@tildesrc

Copy link
Copy Markdown
Contributor

Spawn-prep filled a per-task clone's submodules by fetching each one from its forge-resolved URL — paid by every task on the repo. For a repo with a large submodule that dominates spawn-prep, while the superproject itself is nearly free (clone --local hardlinks the cache clone's objects).

When the repo's git_url names a checkout on this host, that checkout already holds every submodule's objects, so spawn-prep now clones them from it: a local clone with a hardlinked object store and no network.

The donor is matched by path, never by URL — the donor resolved its relative .gitmodules URLs against its own origin while the per-task clone resolves them against git_url, so the same submodule can legitimately carry two different URLs. Per superproject level:

  1. submodule init — git resolves the declared URLs into config;
  2. for each submodule the donor has checked out, overwrite that resolved URL with the donor's path (one the donor lacks keeps its own URL and is simply fetched);
  3. submodule update for this level only — a nested submodule's URL can't be resolved, let alone redirected, before its parent exists;
  4. recurse into each submodule against the matching donor level.

Then, once at the top, submodule sync --recursive restores the canonical URLs — in the config and in each submodule's own origin — so no donor path reaches the container.

It is only an optimisation, never a precondition: no local checkout, a raise, or any submodule still uninitialized afterwards all fall back to the plain submodule update --init --recursive, which is exactly what a repo without a donor gets. Repos whose git_url is a hosted forge are unchanged; giving them a donor by making the cache clone submodule-aware stays backlogged.

ADR 0011 gains §1c for the mechanism and its fallback rule.

Plan: the task's plan.md artifact.

tildesrc and others added 2 commits September 17, 2026 21:28
Spawn-prep filled a per-task clone's submodules by fetching each one from
its forge-resolved URL — paid by every task on the repo, and for a large
submodule it dominates spawn-prep, while the superproject itself is nearly
free (`clone --local` hardlinks the cache's objects).

When the repo's `git_url` names a checkout on this host, that checkout
already holds every submodule's objects, so clone them from it instead: a
local clone with a hardlinked object store and no network. The donor is
matched by path, never by URL (the donor and the per-task clone resolve
relative `.gitmodules` URLs against different origins), so hydration walks
the tree a level at a time — init, override the resolved URL with the
donor's checkout, update this level, recurse — and finishes with one
`submodule sync --recursive` that puts the canonical URLs back in the
config and in each submodule's own origin.

It is only an optimisation: no local checkout, a raise, or a submodule
still uninitialized afterwards all fall back to the plain recursive
update, which is exactly what a repo without a donor gets.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant