feat(entity,storage): rework speculation tree model and store#231
Open
behinddwalls wants to merge 1 commit into
Open
feat(entity,storage): rework speculation tree model and store#231behinddwalls wants to merge 1 commit into
behinddwalls wants to merge 1 commit into
Conversation
This was referenced Jun 9, 2026
behinddwalls
force-pushed
the
preetam/ext/speculation-store
branch
from
927e240 to
6bc8426
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-rfc
branch
from
380f0b1 to
3eddc3d
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-store
branch
from
6bc8426 to
7b49586
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-rfc
branch
from
67da1e9 to
0ee54c9
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-rfc
branch
5 times, most recently
from
3053dd4 to
58cabeb
Compare
behinddwalls
added a commit
that referenced
this pull request
Jul 7, 2026
## Summary Capture how SubmitQueue speculates before the interfaces land. Speculation is modeled as two layers: **decision seams** — enumeration and **scoring** *describe* the tree of possible bets, selection and prioritization *act* on it — and **signal-driven limit policies** that decide *how much* to allow at each step (dependency limit, selection limit, prioritization limit), scaling with the build system's resources. A path's score is dynamic — recomputed as dependencies land, fail, or have their builds pass — not frozen at enumeration. Covers: the problem (dependent batches serialize behind multi-minute builds), the Base/Head path-and-tree model, where speculation sits in the orchestrator pipeline, the status-vs-action ownership split, the dependency limit as an eligibility gate (batches wait for predecessors to land rather than trimming their base), dynamic rescoring, and how prioritization rations the queue's builds against its budget. Design only; entities, store, and extension interfaces follow in later PRs in this stack. Adds doc/rfc/submitqueue/speculation.md and links it from the RFC index. ### Changes since review Reframed from two seams (enumeration + selection) to describe/act decisions + limits, addressing @sbalabanov's review: - selector resource-awareness → the new **prioritization** layer owns the queue-wide build budget; the selector stays a pure per-batch policy - "variable depth, not a single knob" → the **dependency limit** is a signal-driven policy, not a constant - clarified the dependency limit is an **eligibility gate** on active in-flight dependencies (a batch waits until predecessors land), not a base trim - **dynamic scoring**: split scoring out of enumeration into a **scorer** that recomputes each path's score as dependencies land/fail/pass — enumeration is now structure-only - entire tree persisted (the selector reads all path statuses + scores, so it can cancel a path it earlier asked to build); the redundant fallback path is cancelled on a winning bet; slash-free example names; trimmed impl-detail bullets - recorded as open: extending rescoring to weigh how long a batch has waited ## Test Plan Doc-only. Rendered locally; ASCII diagrams and tables verified; no hard-wrapped prose. ## Issues ## Stack 1. @ #230 1. #231 1. #232
behinddwalls
force-pushed
the
preetam/ext/speculation-store
branch
from
7b49586 to
4606cc7
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-store
branch
from
5c9c4d2 to
4606cc7
Compare
This was referenced Jul 8, 2026
Reshape the speculation tree data model around the Base/Head path that the build stage actually consumes, and align its store with the speculation design.
entity: SpeculationPath{Base, Head} becomes the unit; SpeculationPathInfo carries the path plus its Score, controller-owned Status (candidate/selected/building/passed/failed/cancelled), and BuildID. Score is scorer-computed and controller-persisted dynamic state — recomputed on every respeculate, not set at enumeration. Adds SpeculationPathAction and SpeculationPathDecision for the selector seam. SpeculationTree.Speculations becomes Paths. The Build entity uses the shared SpeculationPath (Head = the batch under verification) and drops its own Score, which was never populated or read; the prioritizer seam re-adds a build priority when it lands.
storage: SpeculationTreeStore.UpdateSpeculations(batchID, []SpeculationInfo) becomes Update(ctx, SpeculationTree) — symmetric with Create, keyed by tree.BatchID. The MySQL column speculations is renamed paths to match the entity field. MySQL impl, mock, and storage integration coverage (create/get round-trip, duplicate, whole-tree overwrite, not-found) added/updated.
behinddwalls
force-pushed
the
preetam/ext/speculation-store
branch
from
4606cc7 to
e0d72ab
Compare
sbalabanov
requested changes
Jul 9, 2026
sbalabanov
left a comment
sbalabanov
left a comment
Contributor
There was a problem hiding this comment.
concurrency comment is critical, others aren't
| const ( | ||
| // SpeculationPathActionUnknown is the default zero value for SpeculationPathAction. | ||
| // SpeculationPathActionUnknown is the unreachable zero value. A real decision | ||
| // always carries Promote or Cancel; a seam expresses "leave this path as-is" |
Contributor
There was a problem hiding this comment.
no need to mention Promote and Cancel in the description of Unknown
| // Score is score for this speculation path. | ||
| // SpeculationPathInfo is the per-path entry in a speculation tree: a path, its | ||
| // latest predicted-success score, its controller-owned status, and a link to | ||
| // the build dispatched for it (if any). |
Contributor
There was a problem hiding this comment.
probably good to indicate what is immutable and what isn't
| return fmt.Errorf("failed to get rows affected from update for batchID=%q: %w", speculationTree.BatchID, err) | ||
| } | ||
|
|
||
| if rowsAffected != 1 { |
Contributor
There was a problem hiding this comment.
we do it all over places, but better design is to rowsAffected == 1 -> success, rowsAffected ==0 -> errNotFound, otherwise generic infra error.
| // Returns ErrNotFound if the speculation tree is not found. | ||
| UpdateSpeculations(ctx context.Context, batchID string, speculations []entity.SpeculationInfo) error | ||
| // Update overwrites the paths of an existing speculation tree, identified by | ||
| // speculationTree.BatchID. Returns ErrNotFound if the speculation tree is not found. |
Contributor
There was a problem hiding this comment.
I think it needs concurrency control / optimistic locking as well.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reshape the speculation tree data model around the Base/Head path that the build stage actually consumes, and align its store with the speculation design.
entity: SpeculationPath{Base, Head} becomes the unit; SpeculationPathInfo carries the path plus its Score, controller-owned Status (candidate/selected/building/passed/failed/cancelled), and BuildID. Score is scorer-computed and controller-persisted dynamic state — recomputed on every respeculate, not set at enumeration. Adds SpeculationPathAction and SpeculationPathDecision for the selector seam. SpeculationTree.Speculations becomes Paths. The Build entity uses the shared SpeculationPath (Head = the batch under verification) and drops its own Score, which was never populated or read; the prioritizer seam re-adds a build priority when it lands.
storage: SpeculationTreeStore.UpdateSpeculations(batchID, []SpeculationInfo) becomes Update(ctx, SpeculationTree) — symmetric with Create, keyed by tree.BatchID. The MySQL column speculations is renamed paths to match the entity field. MySQL impl, mock, and storage integration coverage (create/get round-trip, duplicate, whole-tree overwrite, not-found) added/updated.
Stack