Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
b93a578
feat(controller): add migration dashboard
thesyncim Aug 21, 2026
d97eabd
fix(controller): harden controls and progress
thesyncim Aug 21, 2026
6d4c00b
feat(controller): add mutable configuration API [blueprint:task-1]
thesyncim Aug 21, 2026
1c4b857
feat(controller): add complete UI configuration forms [blueprint:task-2]
thesyncim Aug 21, 2026
9e38b90
test(controller): prove UI configuration end to end [blueprint:task-3]
thesyncim Aug 21, 2026
58ec55f
fix: address reviewer findings (round 1) [blueprint:fix]
thesyncim Aug 21, 2026
50a1654
fix: address reviewer findings (round 2) [blueprint:fix]
thesyncim Aug 21, 2026
4ab6d8c
fix(controller): make authentication state explicit
thesyncim Aug 21, 2026
22ea7cc
fix(controller): show live copy throughput
thesyncim Aug 21, 2026
568e6ee
fix(controller): clarify lifecycle and findings
thesyncim Aug 21, 2026
54aa53c
fix(controller): report crash-safe replay progress
thesyncim Aug 21, 2026
53e2d29
fix(controller): isolate restartable workers
thesyncim Aug 21, 2026
7ff8926
fix(setup): keep exported snapshot alive
thesyncim Aug 21, 2026
7ecaab9
fix(controller): confirm fresh base copy restart
thesyncim Aug 21, 2026
8f735ea
fix(setup): disable snapshot holder session timeouts
thesyncim Aug 21, 2026
a01dfbe
fix(cdc): use full replica identity for batch apply
thesyncim Aug 23, 2026
56e20d5
perf(cdc): batch non-unique indexed tables
thesyncim Aug 23, 2026
b5eb464
Revert "perf(cdc): batch non-unique indexed tables"
thesyncim Aug 23, 2026
0be0ac4
perf(cdc): replay only changed columns
thesyncim Aug 23, 2026
05eba26
fix(cdc): fall back from array inspection
thesyncim Aug 23, 2026
3783ba0
perf(cdc): order selective replay probes
thesyncim Aug 23, 2026
3f1060f
perf(cdc): bitmap large replay probes
thesyncim Aug 23, 2026
0d5dc8c
fix(cdc): use exact bitmap identities
thesyncim Aug 23, 2026
90c70cd
perf(cdc): bitmap selective updates
thesyncim Aug 23, 2026
0b7b050
perf(cdc): bitmap batched deletes
thesyncim Aug 24, 2026
fb9fb80
perf(cdc): adapt probes to target cache
thesyncim Aug 24, 2026
f15933e
perf(cdc): keep exact keys for cold updates
thesyncim Aug 24, 2026
ba127ad
fix(cdc): bound composite identity replay
thesyncim Aug 24, 2026
1c6cb55
fix(cdc): keep generated columns out of replay gates
thesyncim Aug 24, 2026
7d12df8
fix(cdc): select custom-type updates precisely
thesyncim Aug 24, 2026
0ff0898
perf(cdc): keep hot composite probes direct
thesyncim Aug 24, 2026
1bc85b9
perf(cdc): force composite primary-key probes
thesyncim Aug 24, 2026
22b0007
perf(cdc): force composite update primary keys
thesyncim Aug 24, 2026
541d282
perf(cdc): replay updates as primary-key upserts
thesyncim Aug 24, 2026
d201982
feat(cdc): expose crash-safe replay batch limits
thesyncim Aug 24, 2026
ffd8ca8
fix(cdc): order batched deletes by target primary key
thesyncim Aug 24, 2026
4072851
fix(cdc): force batched deletes through target primary key
thesyncim Aug 24, 2026
85a9811
feat(cdc): add durable concurrent replay claims
thesyncim Aug 24, 2026
0bbdd6e
perf(cdc): remove replay commit bottlenecks
thesyncim Aug 24, 2026
cb8df4b
perf(cdc): keep relation-local replay concurrent
thesyncim Aug 24, 2026
c28856c
perf(cdc): parallelize durable segment recovery
thesyncim Aug 24, 2026
435eaf5
perf(cdc): shard trusted unaccent replay by key
thesyncim Aug 24, 2026
215501a
fix(replay): harden durable resume paths
thesyncim Aug 24, 2026
0cadfde
fix(replay): recover safely from a lost source stream
thesyncim Aug 24, 2026
baeeca6
fix(controller): retire superseded recovery blockers
thesyncim Aug 24, 2026
e11b942
test(controller): verify recovery retires stale blockers
thesyncim Aug 24, 2026
80d445f
fix(controller): retire orphaned replay finding
thesyncim Aug 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ coverage.*
*.prof
*.pprof

# Local blueprint execution logs
/docs/blueprints/*.log

# Go workspace files, which are always local to one checkout
go.work
go.work.sum
Expand Down
10 changes: 9 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
GO ?= go
GOFLAGS ?=

.PHONY: fmt vet test race integration bench cdc-bench e2e crash-e2e
.PHONY: fmt vet test race integration bench cdc-bench e2e controller-e2e restart-e2e crash-e2e

fmt:
$(GO) $(GOFLAGS) fmt ./...
Expand Down Expand Up @@ -29,6 +29,14 @@ e2e:
$(GO) $(GOFLAGS) build -o ./pgmigrate ./cmd/pgmigrate
test/e2e/scripts/run-migration.sh

controller-e2e:
$(GO) $(GOFLAGS) build -o ./pgmigrate ./cmd/pgmigrate
PGMIGRATE_DRIVER=controller test/e2e/scripts/run-migration.sh

restart-e2e:
$(GO) $(GOFLAGS) build -o ./pgmigrate ./cmd/pgmigrate
PGMIGRATE_DRIVER=controller PGMIGRATE_TEST_DROP_SLOT_RESTART=1 test/e2e/scripts/run-migration.sh

crash-e2e:
$(GO) $(GOFLAGS) build -o ./pgmigrate ./cmd/pgmigrate
test/e2e/scripts/run-crash-loop.sh
258 changes: 219 additions & 39 deletions README.md

Large diffs are not rendered by default.

92 changes: 92 additions & 0 deletions docs/blueprints/controller-ui-configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Controller UI Configuration Implementation Blueprint

## Meta

- **Design Doc:** N/A

## Overview

Allow an authenticated operator to configure every setting used by controller-managed preflight, run, and verification from the embedded dashboard. Controller bootstrap settings remain process-owned, credentials remain write-only and memory-only, and cutover and sequence advancement remain CLI-only.

### Cross-Cutting Requirements

- The controller must continue to start idle and must not create migration state or connect to either database on startup.
- Controller token, listen address, and migration directory remain startup-only.
- Source and target DSNs are never returned by an API, logged, written to state, or stored in browser storage.
- Configuration updates are rejected while a migration or verification operation is active.
- Existing CLI behavior and lifecycle guards remain unchanged.
- All new code passes `go vet ./...`, `go test ./...`, and `go test -race ./...`.

---

## Tasks

### Task 1: Add an authenticated mutable configuration API
**Type:** code

**Subtasks:**
- Add a concurrency-safe controller configuration store initialized from CLI/environment defaults.
- Add authenticated `GET /api/config` and `PUT /api/config` routes covering every configuration field used by preflight, run, and verify, excluding status-only, cutover/sequence, directory, listener, and token settings.
- Make source and target DSNs write-only: GET reports only whether each is configured, and an omitted/blank PUT value retains the current DSN.
- Parse human-readable durations and numeric settings, validate the complete candidate configuration, and atomically replace it only when no operation is active.
- Snapshot the current configuration when an action starts so an in-flight action cannot observe later mutations.
- Add handler and concurrency tests for authentication, redaction, validation, update locking, default preservation, and action snapshots.

**Acceptance Criteria:**
- AC1.1: An authenticated client can configure source, target, and every preflight/run/verify option after controller startup.
- AC1.2: Neither config GET nor status responses contain either DSN.
- AC1.3: Invalid configuration returns HTTP 400 without changing the active configuration.
- AC1.4: Configuration updates during active migration or verification return HTTP 409.
- AC1.5: Controller and CLI unit tests pass under the race detector.

---

### Task 2: Add complete configuration forms to the embedded UI
**Type:** code

**Subtasks:**
- Add database connection, migration, copy, tuning, and verification form sections with basic settings visible and advanced settings collapsible.
- Load non-secret defaults from the config API after authentication without placing DSNs in the DOM or browser storage.
- Save configuration through the authenticated API, clearly report validation errors, and show configured/not-configured connection state.
- Keep controls disabled until a valid configuration is saved and preserve all existing lifecycle, confirmation, progress, and stop behavior.
- Add static UI regression assertions for the configuration form, write-only DSNs, and absence of DSN browser persistence.
- Update README controller documentation with configuration security and lifecycle behavior.

**Acceptance Criteria:**
- AC2.1: Every preflight/run/verify configuration field can be edited from the dashboard.
- AC2.2: Source/target inputs are password fields, remain empty after reload, and are never stored in localStorage or sessionStorage.
- AC2.3: Bootstrap settings and CLI-only cutover/sequences are not editable from the dashboard.
- AC2.4: Existing progress and action controls remain functional and accessible.
- AC2.5: Controller tests and `git diff --check` pass.

---

### Task 3: Prove UI-supplied configuration end to end
**Type:** go-tests

**Subtasks:**
- Change the controller E2E driver to start without source/target DSNs and populate the complete action configuration through the authenticated config API.
- Exercise authenticated preflight, run, live verification, final verification, CLI-only cutover, cleanup checks, and independent source/target comparison.
- Add focused coverage that controller startup alone leaves the migration directory untouched.
- Validate the dashboard in a browser from unauthenticated state through configuration save, action enablement, progress rendering, and completed-state locking.

**Acceptance Criteria:**
- AC3.1: `make controller-e2e` passes while supplying both DSNs through the controller config API.
- AC3.2: Independent table inventory, row counts, and canonical source/target digests match after cutover.
- AC3.3: Starting the controller without taking an action creates no migration state and opens no database connection.
- AC3.4: `go vet ./...`, `go test ./...`, and `go test -race ./...` pass.

## Files to Modify

- `internal/controller/controller.go` - mutable config API and action snapshots.
- `internal/controller/controller_test.go` - API, redaction, locking, and startup regression tests.
- `internal/controller/ui.html` - complete configuration dashboard.
- `test/e2e/scripts/run-migration.sh` - configure controller through API.
- `README.md` and `test/README.md` - operator and E2E documentation.

## References

- Current controller server: `internal/controller/controller.go`
- Current embedded dashboard: `internal/controller/ui.html`
- CLI configuration flags: `internal/cli/cli.go`
- Shared configuration model: `internal/config/config.go`
Loading