Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions api/stovepipe/proto/stovepipe.proto
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,40 @@ message ListResponse {
string next_page_token = 2;
}

message GetQueueStatusRequest {
// Required stable repo-and-ref queue name.
string queue = 1;
// Optional immutable commit URI. Empty selects the latest materialized state.
string change_uri = 2;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be any commit and not just the ones we processed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's correct.

}

// QueueEnablement is independent of validation results and execution controls.
enum QueueEnablement {
// Unknown is never an explicit disablement.
QUEUE_ENABLEMENT_UNKNOWN = 0;
QUEUE_ENABLEMENT_ENABLED = 1;
QUEUE_ENABLEMENT_DISABLED = 2;
}

// QueueEnablementStatus describes applied enablement.
message QueueEnablementStatus {
// Positive enablement sequence, increasing with each state change.
int64 version = 1;
// Applied enablement. Successful lookups never return UNKNOWN.
QueueEnablement state = 2;
// Inclusive first covered commit. Empty only for initial DISABLED version 1.
string effective_from_commit_uri = 3;
// Application timestamp in Unix milliseconds, not source-control commit time.
int64 changed_at_ms = 4;
}

message GetQueueStatusResponse {
// Queue selected by the request.
string queue = 1;
// Current enablement, or enablement covering change_uri. Always present on success.
QueueEnablementStatus enablement = 2;
}

// Stovepipe provides the Stovepipe API.
service Stovepipe {
// Ping returns a response indicating the service is alive
Expand All @@ -222,4 +256,6 @@ service Stovepipe {
rpc GetProjectStatusByURI(GetProjectStatusByURIRequest) returns (GetProjectStatusByURIResponse) {}
// List returns a queue-scoped page of current request summaries.
rpc List(ListRequest) returns (ListResponse) {}
// GetQueueStatus returns current enablement or enablement for a supplied commit.
rpc GetQueueStatus(GetQueueStatusRequest) returns (GetQueueStatusResponse) {}
}
351 changes: 306 additions & 45 deletions api/stovepipe/protopb/stovepipe.pb.go

Large diffs are not rendered by default.

200 changes: 131 additions & 69 deletions api/stovepipe/protopb/stovepipe.pb.yarpc.go

Large diffs are not rendered by default.

40 changes: 40 additions & 0 deletions api/stovepipe/protopb/stovepipe_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions doc/rfc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Design documents and technical proposals, grouped by scope. Shared/cross-cutting
- [Request History API](stovepipe/request-history-api.md) - Queue-scoped request-ID and URI lookup, public projection, materialization decision, ordering, and retention
- [List API](stovepipe/list-api.md) - Queue-scoped request listing by acceptance time, immutable acceptance time in summaries, cursor pagination, and a portable time-to-request mapping
- [GetProjectStatusByURI API](stovepipe/get-project-status-by-uri-api.md) - Queue-scoped current validation lookup for a commit, with repository and future project-level results
- [Queue State](stovepipe/queue-state.md) - Internal queue enablement, materialized summary and history, and current or commit-specific status lookup

## Runway

Expand Down
88 changes: 88 additions & 0 deletions doc/rfc/stovepipe/queue-state.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Queue enablement and status

Stovepipe needs to answer two questions: is a queue enabled now, and was it enabled for a particular commit? Each answer includes the commit at which that state took effect.

## Why we need this

A client needs to know whether Stovepipe validation is expected for a commit. Today's setting is not enough: a queue can be disabled now even though an earlier commit was covered, or enabled before its first validation finishes.

The existing information cannot answer this reliably:

- A configuration flag describes the desired setting, and the current queue row holds the latest applied state. Neither alone retains earlier commit intervals.
- Request history describes validation attempts. Batching leaves intermediate commits without their own requests, so a missing request does not imply disabled.
- Last-green and validation results describe outcomes, rather than whether validation was expected.

A dedicated lookup needs durable enablement history with commit boundaries, independent of validation work.

## Enablement and pause

Enablement defines which commits are covered by Stovepipe. Pause is a temporary operational control over whether a queue accepts or advances work. An enabled queue can be paused and still owe validation results for its covered commits; enabled does not promise that work is progressing or that results are green.

For example, if a queue is enabled at A and later paused while validating C, the status for C remains enabled from A. Unpausing resumes work without creating a new enablement boundary. Disabling at D instead changes coverage from D onward, while preserving the answers for earlier commits.

Future pause management should follow the [queue pause RFC](../submitqueue/queue-pause-and-unpause.md), which distinguishes refusing new requests from freezing accepted work. That RFC covers SubmitQueue; Stovepipe's ingestion and processing behavior still needs its own design.

Pause can use the same internal-state, summary and history pattern described below, with separate operational fields and history events. Pause and unpause must not advance the enablement version or change commit boundaries. Keep pause out of the enablement enum, and expose it as current operational status rather than mixing it into a commit-specific enablement answer. A future administration API can manage both controls independently; this lookup remains read-only.

## What the endpoint returns

`GetQueueStatus` takes a queue name and an optional `change_uri`. Omitting the commit selects the latest queue summary; supplying it selects the enablement covering that commit. The response contains an `enablement` object of type `QueueEnablementStatus`:

| Field | Meaning |
|---|---|
| `state` | `ENABLED` or `DISABLED`. |
| `effective_from_commit_uri` | First covered commit, inclusive. |
| `version` | Sequence number that increases for each enablement change. |
| `changed_at_ms` | Recorded change timestamp in Unix milliseconds, distinct from commit time. |

The boundary, version and timestamp describe enablement only. Future operational fields, such as pause status, belong beside this object and can change independently.

See the [proto](../../../api/stovepipe/proto/stovepipe.proto) for the wire contract.

Suppose commits arrive in order A through G, with enablement changes at A, D and G:

| Lookup | State | Effective from |
|---|---|---|
| No commit supplied | Enabled | G |
| C | Enabled | A |
| D or F | Disabled | D |
| G | Enabled | G |

C and F need no validation requests. Their ancestry relative to the boundaries determines the answer.

Registration starts a queue as disabled version 1, with no boundary; this covers commits before its first enablement. Missing state is an error, not disabled. If changes share a boundary, the highest version wins. Versions order changes; timestamps do not.

## Internal state, summary and history

Follow the model used for request summaries and history, while keeping the internal queue authoritative:

| Record | Purpose |
|---|---|
| Existing `queue` row | Current enablement alongside pipeline coordination fields. |
| `queue_log` | Immutable applied changes, keyed by queue and enablement version. |
| `queue_summary` | Current read view derived from those changes. |

The queue row and summary each have a storage version for optimistic locking. The API exposes a separate enablement version, which advances only for enabled/disabled changes. Routine pipeline updates create no enablement history entries or gaps.

The transition controller captures the queue's tip and a timestamp once when preparing a change. Retries keep those values; requesting the existing state is a no-op. Application first finishes recording the previous state, then conditionally updates the queue and records the new state through `queuelog.Materializer.PersistLog`.

History is written before its summary. If either write fails after the queue update, the queue retains the missing change. Initialization and transition retries repair the read models before another enablement change can replace it. Repeating an identical change is safe, and older changes cannot move the summary backward. Concurrent pipeline updates preserve their coordination fields; competing enablement decisions are rejected as stale.

## Commit lookup and consistency

Lookup reads one summary and its matching history entry, and uses that version throughout the call. It never writes or repairs state. “Current” therefore means the latest state in the read models, which may lag the internal queue until a writer retries.

For a supplied commit, the existing source-control integration verifies that the commit and latest boundary belong to the queue's current lineage. The controller walks changes newest-first and selects the first boundary that is an ancestor of, or equal to, the commit. Before walking backward, it verifies that the commit precedes the boundary; incomparable commits and unsupported history rewrites fail instead of becoming disabled. This resolves intermediate commits without a commit index or local checkout. Current-only lookup needs no source-control call.

| Failure | RPC error |
|---|---|
| Malformed selector | `INVALID_ARGUMENT` |
| No queue summary | `NOT_FOUND` |
| Unresolved commit or unsupported lineage | `FAILED_PRECONDITION` |
| Missing or inconsistent history | `INTERNAL` |

Dependency and cancellation errors retain their transport classification. An unresolved commit fails the whole call.

## Scope

This stack records enablement and exposes status lookup. It does not enforce ingestion or execution controls. Configuration freshness, administration, pause enforcement and public history access remain separate work. Validation results and last-green stay in their existing APIs.
3 changes: 3 additions & 0 deletions service/stovepipe/server/handler/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ go_test(
name = "go_default_test",
srcs = [
"list_test.go",
"queue_status_test.go",
"server_test.go",
],
embed = [":go_default_library"],
Expand All @@ -27,6 +28,8 @@ go_test(
"//stovepipe/extension/storage:go_default_library",
"@com_github_stretchr_testify//assert:go_default_library",
"@com_github_stretchr_testify//require:go_default_library",
"@org_golang_google_grpc//codes:go_default_library",
"@org_golang_google_grpc//status:go_default_library",
"@org_uber_go_mock//gomock:go_default_library",
],
)
32 changes: 32 additions & 0 deletions service/stovepipe/server/handler/queue_status_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package handler

import (
"context"
"testing"

"github.com/stretchr/testify/require"
pb "github.com/uber/submitqueue/api/stovepipe/protopb"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)

func TestQueueStatusContractIsUnimplemented(t *testing.T) {
server := &StovepipeServer{}
response, err := server.GetQueueStatus(context.Background(), &pb.GetQueueStatusRequest{Queue: "repo/main"})
require.Nil(t, response)
require.Equal(t, codes.Unimplemented, status.Code(err))
}
Loading
Loading