feat(prioritization): add prioritizer + prioritization-limit extensions#318
Closed
behinddwalls wants to merge 5 commits into
Closed
feat(prioritization): add prioritizer + prioritization-limit extensions#318behinddwalls wants to merge 5 commits into
behinddwalls wants to merge 5 commits into
Conversation
Reshape the speculation tree data model around the Base/Head path that the build stage actually consumes, and align its store with the speculation design.
entity: SpeculationPath{Base, Head} becomes the unit; SpeculationPathInfo carries the path plus its Score, controller-owned Status (candidate/selected/building/passed/failed/cancelled), and BuildID. Score is scorer-computed and controller-persisted dynamic state — recomputed on every respeculate, not set at enumeration. Adds SpeculationPathAction and SpeculationPathDecision for the selector seam. SpeculationTree.Speculations becomes Paths. The Build entity uses the shared SpeculationPath (Head = the batch under verification) and drops its own Score, which was never populated or read; the prioritizer seam re-adds a build priority when it lands.
storage: SpeculationTreeStore.UpdateSpeculations(batchID, []SpeculationInfo) becomes Update(ctx, SpeculationTree) — symmetric with Create, keyed by tree.BatchID. The MySQL column speculations is renamed paths to match the entity field. MySQL impl, mock, and storage integration coverage (create/get round-trip, duplicate, whole-tree overwrite, not-found) added/updated.
Add the first speculation seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/. enumerator: given a batch and its ordered active dependencies, mechanically lists the candidate Base/Head paths — pure, deterministic, and purely structural. It sets no score and no status; the controller stamps status on persist and the scorer fills score. (Its earlier scoring responsibility is dropped here, per the RFC, and moves to the scorer seam.) dependencylimit: the "how much" policy bounding how many active (in-flight) dependencies a batch may speculate over. It is the eligibility gate before enumeration; unlike the other speculation limits it is controller-held rather than injected into a seam, keeping the enumerator pure. The value is signal-driven, not a fixed constant. Each follows the repo extension contract (conflict.Analyzer reference shape): Factory.For(Config) (T, error) with Config carrying only QueueName; behavioral knobs and limit signals are integrator-injected at construction. Includes READMEs, gomock packages, and Makefile mock-gen wiring. Interfaces only; concrete impls and controller wiring are deferred.
This was referenced Jul 8, 2026
behinddwalls
force-pushed
the
preetam/ext/speculation-selector
branch
from
a091730 to
5e6a56f
Compare
behinddwalls
force-pushed
the
preetam/ext/prioritization
branch
from
db9cd8a to
e36260b
Compare
Add the scorer seam from the speculation RFC, as a vendor-agnostic extension interface under submitqueue/extension/speculation/scorer/. The scorer computes each speculation path's predicted-success score from the current state: the per-batch scores of the path's base batches (entity.Batch.Score) and which of those dependencies have resolved (landed or build-passed), plus optionally other signals. It is a prediction over live state, so the controller re-runs it on every respeculate right after reconciling status, and persists the result; the scorer owns only the formula. The store is the source of truth: the scorer takes only the batch identity and loads what it needs — the batch's speculation tree (with controller-reconciled statuses) and its dependency batches — through read access injected at its Factory. It returns the scored tree; the controller persists the scores, staying the single writer of tree state. This is the per-path scorer, distinct from the existing per-batch score stage (extension/scorer) that sets entity.Batch.Score — the path scorer consumes those to score whole paths. Follows the repo extension contract: Factory.For(Config) (Scorer, error) with Config carrying only QueueName. Includes README, gomock package, and a programmable fake. Interface only; concrete impls and controller wiring are deferred.
Add the selector seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/. selector: given a batch, loads its speculation tree (with controller-stamped status and score) from storage and returns a per-path action (Build/Cancel) for the paths it chooses to act on. It is the policy seam — it reads stored state and emits actions only, never writes status, and does not decide merging. The store is the source of truth: it takes only the batch identity and reads the tree through read access injected at its Factory, so the controller stays the single writer. selectionlimit: the "how much" policy bounding how many paths a batch may build in parallel. It is the selector's companion — the selector decides which paths, the limit decides how many. Injected into the selector at construction and called by it, keeping the selector interface limit-free; the value is dynamic, not a fixed constant. Each follows the repo extension contract: Factory.For(Config) (T, error) with Config carrying only QueueName. Includes READMEs, gomock packages, and programmable fakes. Interfaces only; concrete impls and controller wiring are deferred.
Add the prioritizer seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/prioritization/. Prioritization is queue-wide and lives at the build stage, not per batch inside speculate, so it sits in its own prioritization/ family rather than under speculation/. prioritizer: the queue-wide policy that rations a shared build budget across every in-flight batch. Bound to its queue at construction, it takes no arguments — the store is the source of truth, so it reads the queue's pending builds and each build's path score from storage (read access injected at its Factory), ranks them by score plus any fairness policy, and returns the admitted subset that fits the queue's concurrent-build budget. It never writes; dispatching the admitted builds is the controller's job. limit: the "how much" policy bounding how many builds the queue runs at once — the queue's concurrent-build budget and the ultimate cap on speculation's demand on CI. Injected into the prioritizer at construction and called by it; the value is dynamic, not a fixed constant. Each follows the repo extension contract: Factory.For(Config) (T, error) with Config carrying only QueueName. Includes READMEs, gomock packages, and programmable fakes. Interfaces only; concrete impls and controller wiring are deferred.
behinddwalls
force-pushed
the
preetam/ext/speculation-selector
branch
from
5e6a56f to
e89bb7c
Compare
behinddwalls
force-pushed
the
preetam/ext/prioritization
branch
from
e36260b to
65c01c4
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-selector
branch
from
e89bb7c to
0ab1871
Compare
Collaborator
Author
|
Superseded by #320 — the prioritizer moved under submitqueue/extension/speculation/ (branch renamed to preetam/ext/speculation-prioritizer). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add the prioritizer seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/prioritization/.
Prioritization is queue-wide and lives at the build stage, not per batch inside speculate, so it sits in its own prioritization/ family rather than under speculation/.
prioritizer: the queue-wide policy that rations a shared build budget across every in-flight batch. Bound to its queue at construction, it takes no arguments — the store is the source of truth, so it reads the queue's pending builds and each build's path score from storage (read access injected at its Factory), ranks them by score plus any fairness policy, and returns the admitted subset that fits the queue's concurrent-build budget. It never writes; dispatching the admitted builds is the controller's job.
limit: the "how much" policy bounding how many builds the queue runs at once — the queue's concurrent-build budget and the ultimate cap on speculation's demand on CI. Injected into the prioritizer at construction and called by it; the value is dynamic, not a fixed constant.
Each follows the repo extension contract: Factory.For(Config) (T, error) with Config carrying only QueueName. Includes READMEs, gomock packages, and programmable fakes. Interfaces only; concrete impls and controller wiring are deferred.
Stack