Repository navigation
Add explicit deployment state modes and optional DMS history #7151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
shreyas-goenka
merged 52 commits into
main
from
shreyas-goenka_data/shreyas/statedb-modes
Oct 26, 2026
+447
−184
Merged
Changes from all commits
Commits
Show all changes
52 commits
Select commit
Hold shift + click to select a range
4288b81
Add explicit deployment state modes and optional DMS history
shreyas-goenka bbbf210
Document deployment state modes and history error handling
shreyas-goenka cfa1615
Use a fixed history drain timeout and clarify mode documentation
shreyas-goenka 50ed19e
Scale DMS history drain budget and cap requests at one minute
shreyas-goenka aa5e06a
Set history drain budget to five seconds per operation
shreyas-goenka f734f16
Rename best-effort history feature and store computed drain timeout
shreyas-goenka b9f937e
Use a fixed DMS buffer and wait for queue space
shreyas-goenka 4cd4e08
Clarify the file-mode guard in version completion
shreyas-goenka 34a4070
Use direct enum comparisons for deployment state modes
shreyas-goenka 399b173
Remove redundant version-completion commentary
shreyas-goenka 72fe431
Make state save and delete modes explicit
shreyas-goenka a215726
Guard version completion with RecordsHistory
shreyas-goenka 3042a15
Restore WithRecovery documentation
shreyas-goenka c6a030a
Use an unbounded DMS queue and enforce timeouts only at drain
shreyas-goenka 85cb141
Remove redundant mode validation and clarify history errors
shreyas-goenka 7818b7a
Document Open behavior for each deployment state mode
shreyas-goenka 0f7af03
Separate write initialization from mode transitions
shreyas-goenka 58b0549
Restore DMS helper call for serial lookup
shreyas-goenka 09d854c
Use both version counters for file-backed deployment history
shreyas-goenka 09161b1
Clear deployment state during reset and version completion
shreyas-goenka 5beb2cd
Simplify WAL feature recovery to use header features directly
shreyas-goenka 3dc54f1
Explain why WAL feature changes need persistence
shreyas-goenka 535cda7
Explain state source validation when upgrading to write
shreyas-goenka c8c6be1
Use one writable DMS mode and skip DMS write upgrades
shreyas-goenka 1f230f8
Name WAL setup explicitly and spell out mode branches
shreyas-goenka cc16973
Preserve the ordinary WAL path and separate history branches
shreyas-goenka efae162
Remove redundant operation buffer write assertion
shreyas-goenka 05e55a9
Reset state unconditionally when completing a version
shreyas-goenka 76c0343
Inline normal file persistence in save and delete mode cases
shreyas-goenka 198e51d
Clarify which recording fields survive finalization and why
shreyas-goenka d8206d6
Remove optional best-effort marker check when opening state
shreyas-goenka 33449b7
Use stored modes for write setup and explicit drain policies
shreyas-goenka e695feb
Preserve DMS header timing for plans and downgrade warnings
shreyas-goenka 8c94c94
Inline DMS mode checks when opening deployment state
shreyas-goenka a735d8e
Remove optional WAL mode checks and requested failure test
shreyas-goenka d61e89b
Remove defensive panic when recording after drain
shreyas-goenka ad5ce69
Simplify unbounded queue coordination with a condition variable
shreyas-goenka 22b54a5
Simplify operation buffering with 100ms idle polling
shreyas-goenka 2333f44
Preserve existing DMS behavior and restore acceptance coverage
shreyas-goenka 9978aab
Explain releasing the empty history queue backing array
shreyas-goenka 90f22a1
Clarify history draining polling and cancellation with short comments
shreyas-goenka 203338b
Clarify that file history does not backpressure the deployment engine
shreyas-goenka 2a4e9cb
Explain the expected cancellation error in the history worker
shreyas-goenka cc302bc
Document polling as the initial history queue design
shreyas-goenka 3066b3d
Continue recording after upload failures and reduce mode test scope
shreyas-goenka 7afb46e
Remove broad mode tests and retain two small state edge cases
shreyas-goenka dae6556
Remove the header-only history feature recovery test
shreyas-goenka c60e9d9
Size history channels and drain budgets from staged operations
shreyas-goenka ab75f6e
Remove redundant locking detail from record comment
shreyas-goenka 3d75aa8
Name deployment state modes explicitly in Drain documentation
shreyas-goenka 529f38c
Explain the stalled writer drain test setup and assertions
shreyas-goenka 1723931
Explain fake clock behavior in the history drain test
shreyas-goenka File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| package dstate | ||
|
|
||
| // Mode selects the source of resource state and the persistence required for writes. | ||
| type Mode int | ||
|
|
||
| const ( | ||
| // ReadingFile reads resources.json and optionally recovers a leftover WAL. | ||
| // State writes are disabled. | ||
| ReadingFile Mode = iota | ||
|
|
||
| // WritingFile records state changes in WAL. | ||
| // Finalize merges them into resources.json. DMS is unused. | ||
| WritingFile | ||
|
|
||
| // WritingDMS reads and writes resource state through DMS without creating a WAL. | ||
| // DMS errors fail the deployment. | ||
| WritingDMS | ||
|
|
||
| // WritingFileWithHistory writes state through WAL to resources.json and records DMS history. | ||
| // DMS errors and drain timeouts let the deployment continue. | ||
| WritingFileWithHistory | ||
|
|
||
| // WritingFileWithHistoryEnforced writes state through WAL to resources.json and records DMS history. | ||
| // DMS errors and drain timeouts let the deployment continue. | ||
| // Persists deployment_history_best_effort to reject clients that cannot honor history recording. | ||
| WritingFileWithHistoryEnforced | ||
| ) | ||
|
|
||
| func (m Mode) isWritable() bool { | ||
| switch m { | ||
| case WritingFile, WritingDMS, WritingFileWithHistory, WritingFileWithHistoryEnforced: | ||
| return true | ||
| default: | ||
| return false | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a bit inconsistent, WritingFile and WritingDMS combined results in WritingFileWithHistoryEnforced, "DMS" & "History" both used interchangeably.
Perhaps
I also wonder should we drop the last one for now and implement WritingFileShadowDMSOptional end to end?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
q: do both WritingFileWithHistory and WritingFileWithHistoryEnforced trigger migration?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only downside is we make the feature weaker since we can no longer guarentee history. Since I've already implemented this I'd be inclined to keep it, given it's the end state we are moving towards anyways.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a qualitative difference.
WritingDMSreads from DMS and treats it as SOT.WritingFileWithHistoryEnforceddoes not.