feat(speculation): add path scorer extension

Makefile

@@ -364,7 +364,7 @@ local-stovepipe-stop: ## Stop the Stovepipe service
 
 mocks: ## Generate mock files using mockgen
 	@echo "Generating mocks..."
-	@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./submitqueue/extension/validator/... ./platform/consumer/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
+	@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./submitqueue/extension/speculation/pathscorer/... ./submitqueue/extension/validator/... ./platform/consumer/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
 	@echo "Mocks generated successfully!"
 
 proto: ## Generate protobuf files from .proto definitions

doc/rfc/submitqueue/speculation.md

@@ -192,7 +192,7 @@ The seams are vendor-agnostic extensions, each in its own package; the exact Go
 **Decision seams:**
 
 - **Enumerator** (`extension/speculation/enumerator`) — given a batch ID and its ordered active dependencies, returns the batch's speculation tree *structure*: the candidate paths, each a Base/Head split. Pure and deterministic; sets no score and no status.
-- **Scorer** (`extension/speculation/scorer`) — given the speculation tree and the current dependency batches, returns each path's predicted-success score. Called by the controller on every respeculate (during reconciliation) so scores track the live state — dependencies landing, dependency builds passing, siblings failing. Owns the score formula; combines the base batches' `Batch.Score` and their resolved/unresolved state (and optionally other signals).
+- **Scorer** (`extension/speculation/pathscorer`) — given the speculation tree and the current dependency batches, returns each path's predicted-success score. Called by the controller on every respeculate (during reconciliation) so scores track the live state — dependencies landing, dependency builds passing, siblings failing. Owns the score formula; combines the base batches' `Batch.Score` and their resolved/unresolved state (and optionally other signals).
 - **Selector** (`extension/speculation/selector`) — given a speculation tree (with each path's controller-stamped status and freshly recomputed score), returns a per-path action (`Build` or `Cancel`) for the paths it chooses to act on. It reads status and score and emits actions only; paths it leaves alone are omitted. It is constructed with its **selection limit** and calls it to cap how many paths it builds in parallel.
 - **Prioritizer** (`extension/speculation/prioritizer`) — given the queue's pending build candidates, returns the subset admitted to run, ranked by score plus any fairness policy. It is constructed with its **prioritization limit** and applies it itself. Operates queue-wide, across all of the queue's in-flight batches.
 

submitqueue/extension/speculation/enumerator/enumerator.go

@@ -29,7 +29,7 @@ import (
 // deliberately dumb and purely structural: it mechanically lists candidate
 // Base/Head paths from the dependency batches it is handed and nothing else. It
 // does not score paths — that is the scorer's job (see
-// extension/speculation/scorer), which the controller re-runs on every
+// extension/speculation/pathscorer), which the controller re-runs on every
 // respeculate — it does not decide which paths to build — that is the selector's
 // job (see extension/speculation/selector) — it does not set path status, and it
 // does not decide how far back to speculate: the controller gates on the

submitqueue/extension/speculation/pathscorer/BUILD.bazel

@@ -0,0 +1,9 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["pathscorer.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/pathscorer",
+    visibility = ["//visibility:public"],
+    deps = ["//submitqueue/entity:go_default_library"],
+)

submitqueue/extension/speculation/pathscorer/README.md

@@ -0,0 +1,17 @@
+# Speculation Path Scorer
+
+Vendor-agnostic interface for scoring the paths in a batch's **speculation tree** — the predicted-success probability of each candidate bet, recomputed as the batch's world changes.
+
+See the [Speculation RFC](/doc/rfc/submitqueue/speculation.md) for the end-to-end design and how scoring fits into the orchestrator pipeline.
+
+## Scorer
+
+A path's score is a **prediction**: *how likely is this bet to pay off, right now?* The scorer answers it from the current state — the per-batch success probabilities of a path's base batches (`entity.Batch.Score`, set by the score stage), which of those dependencies have already landed or had their build pass (resolved assumptions raise confidence), and optionally other signals such as how long the batch has waited or historical pass rates. The score is the common currency the [selector](../selector) and prioritizer both rank on, so keeping it current is what makes both act on the latest reality.
+
+Because it is a prediction over live state, the scorer is **re-run on every respeculate**, right after the controller reconciles path status — so when a dependency lands, its build passes, or a sibling path fails, the surviving paths' scores are recomputed before anything is selected or prioritized. The controller drives *when* to rescore (it is part of reconciliation) and persists the result; the scorer owns the *formula*.
+
+This is the per-**path** scorer, distinct from the per-**batch** [score stage](../../scorer), which sets `entity.Batch.Score`. The path scorer consumes those batch scores to score whole paths. The controller hands it the batch's **speculation tree** directly — the subject it scores — and any richer signal an implementation needs (the dependency batches' scores, historical pass rates) is injected at its factory, not passed in. It never writes: it returns the scored tree and the controller persists the scores, so the controller stays the single writer of tree state. Path structure and status pass through unchanged; only `Score` is recomputed.
+
+## Factory
+
+A per-queue factory returns the scorer for a queue, following the repo's extension contract. It is handed only the queue identity; scoring knobs and read access to any extra signals are injected at construction by the integrator in the wiring layer, which resolves per-queue settings through `queueconfig`. Scoring itself stays config-free.

submitqueue/extension/speculation/pathscorer/fake/BUILD.bazel

@@ -0,0 +1,23 @@
+load("@rules_go//go:def.bzl", "go_library", "go_test")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["fake.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/pathscorer/fake",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/speculation/pathscorer:go_default_library",
+    ],
+)
+
+go_test(
+    name = "go_default_test",
+    srcs = ["fake_test.go"],
+    embed = [":go_default_library"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "@com_github_stretchr_testify//assert:go_default_library",
+        "@com_github_stretchr_testify//require:go_default_library",
+    ],
+)

submitqueue/extension/speculation/pathscorer/fake/fake.go

@@ -0,0 +1,67 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package fake provides a programmable pathscorer.Scorer for tests and examples. By
+// default Score echoes the tree it is given; Returns overrides that with a
+// canned scored tree, and FailWith injects an error on every call. It is
+// intended for examples and tests only, never production.
+package fake
+
+import (
+	"context"
+
+	"github.com/uber/submitqueue/submitqueue/entity"
+	"github.com/uber/submitqueue/submitqueue/extension/speculation/pathscorer"
+)
+
+// Scorer is a programmable pathscorer.Scorer.
+type Scorer struct {
+	tree    entity.SpeculationTree
+	hasTree bool
+	err     error
+}
+
+// New returns a fake Scorer that echoes the tree it is given. Override the
+// returned tree with Returns.
+func New() *Scorer {
+	return &Scorer{}
+}
+
+// Returns makes every Score call return tree instead of echoing its input.
+func (s *Scorer) Returns(tree entity.SpeculationTree) *Scorer {
+	s.tree = tree
+	s.hasTree = true
+	return s
+}
+
+// FailWith makes every Score call return err.
+func (s *Scorer) FailWith(err error) *Scorer {
+	s.err = err
+	return s
+}
+
+// Score returns the canned tree if one was set with Returns, otherwise the tree
+// it was given, unchanged.
+func (s *Scorer) Score(_ context.Context, tree entity.SpeculationTree) (entity.SpeculationTree, error) {
+	if s.err != nil {
+		return entity.SpeculationTree{}, s.err
+	}
+	if s.hasTree {
+		return s.tree, nil
+	}
+	return tree, nil
+}
+
+// ensure the fake satisfies the interface.
+var _ pathscorer.Scorer = (*Scorer)(nil)

submitqueue/extension/speculation/pathscorer/fake/fake_test.go

@@ -0,0 +1,56 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fake
+
+import (
+	"context"
+	"errors"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"github.com/uber/submitqueue/submitqueue/entity"
+)
+
+func TestScore_EchoesInputByDefault(t *testing.T) {
+	tree := entity.SpeculationTree{
+		BatchID: "q/batch/2",
+		Paths: []entity.SpeculationPathInfo{
+			{Path: entity.SpeculationPath{Head: "q/batch/2"}, Score: 0.9},
+		},
+	}
+	got, err := New().Score(context.Background(), tree)
+	require.NoError(t, err)
+	assert.Equal(t, tree, got)
+}
+
+func TestScore_ReturnsCanned(t *testing.T) {
+	in := entity.SpeculationTree{BatchID: "q/batch/2"}
+	scored := entity.SpeculationTree{
+		BatchID: "q/batch/2",
+		Paths: []entity.SpeculationPathInfo{
+			{Path: entity.SpeculationPath{Head: "q/batch/2"}, Score: 0.75},
+		},
+	}
+	got, err := New().Returns(scored).Score(context.Background(), in)
+	require.NoError(t, err)
+	assert.Equal(t, scored, got)
+}
+
+func TestScore_FailWith(t *testing.T) {
+	sentinel := errors.New("boom")
+	_, err := New().FailWith(sentinel).Score(context.Background(), entity.SpeculationTree{BatchID: "q/batch/1"})
+	require.ErrorIs(t, err, sentinel)
+}

submitqueue/extension/speculation/pathscorer/mock/BUILD.bazel

@@ -0,0 +1,13 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["pathscorer_mock.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/pathscorer/mock",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/speculation/pathscorer:go_default_library",
+        "@org_uber_go_mock//gomock:go_default_library",
+    ],
+)

submitqueue/extension/speculation/pathscorer/pathscorer.go

@@ -0,0 +1,73 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package pathscorer
+
+//go:generate mockgen -source=pathscorer.go -destination=mock/pathscorer_mock.go -package=mock
+
+import (
+	"context"
+
+	"github.com/uber/submitqueue/submitqueue/entity"
+)
+
+// Scorer computes the predicted-success score of every path in a batch's
+// speculation tree.
+//
+// A path's score is a prediction — "how likely is this bet to pay off?" — and
+// predictions must move as evidence arrives. The scorer answers "how good is
+// each path right now" from the current state: the per-batch success
+// probabilities of a path's base batches (entity.Batch.Score, set by the score
+// stage), which of those dependencies have already landed or had their build
+// pass (resolved assumptions raise confidence), and optionally other signals
+// (how long the batch has waited, historical pass rates).
+//
+// The controller re-runs the scorer on every respeculate, right after it
+// reconciles path status — so when a dependency lands, its build passes, or a
+// sibling path fails, the surviving paths' scores are recomputed against the new
+// reality before anything is selected or prioritized. The controller drives
+// *when* to rescore and persists the result; the scorer owns the *formula*.
+//
+// This is the per-*path* scorer, distinct from the per-*batch* score stage
+// (extension/scorer), which sets entity.Batch.Score. The path scorer consumes
+// those batch scores to score whole paths.
+//
+// The controller hands the scorer the batch's speculation tree directly — the
+// subject it scores. Any richer signal an implementation needs (the dependency
+// batches' scores, historical pass rates) is injected at its Factory, not passed
+// here. It never writes: it returns the scored tree and the controller persists
+// the scores, keeping the controller the single writer of tree state.
+type Scorer interface {
+	// Score returns the given tree with each path's Score set to its freshly
+	// computed predicted-success value. Path structure and controller-stamped
+	// Status are carried through unchanged; only Score is (re)computed. The
+	// combination formula is the implementation's concern.
+	Score(ctx context.Context, tree entity.SpeculationTree) (entity.SpeculationTree, error)
+}
+
+// Config carries the per-queue identity handed to a Factory. The system knows
+// only the queue name; everything an implementation needs (scoring knobs, and
+// read access to any extra signals such as the dependency batches' scores) is
+// injected at construction by the integrator.
+type Config struct {
+	// QueueName identifies the queue this Scorer serves.
+	QueueName string
+}
+
+// Factory builds the Scorer for a queue. Implementations are provided by
+// integrators (and tests) and inject whatever they need at construction.
+type Factory interface {
+	// For returns the Scorer for the given queue.
+	For(cfg Config) (Scorer, error)
+}