Skip to content

Avoid switchover after migrating a single-pod cluster - #3187

Open
Shubham-Padkonde wants to merge 1 commit into
zalando:masterfrom
Shubham-Padkonde:fix/single-pod-node-migration
Open

Shubham-Padkonde wants to merge 1 commit into
zalando:masterfrom
Shubham-Padkonde:fix/single-pod-node-migration

Conversation

@Shubham-Padkonde

Copy link
Copy Markdown

Problem description

Migrating a single-pod cluster off an unschedulable node recreates the master, then attempts a Patroni switchover to itself using the deleted pod's IP. masterCandidatePod starts as oldMaster, making the existing recreate-only branch unreachable.

Leave the candidate unset until a replica is selected. Single-pod clusters now take the existing recreate-and-return path; clusters with replicas still select a candidate and switch over to it.

Added tests exercise successful single-pod relocation without any Patroni request, deletion failure, relocation to the same node, and successful switchover to a healthy replica. The migration parameter documentation now explains the single-pod behavior and downtime.

Validation on Linux with Go 1.26.4:

  • Before the fix, the new relocation regression fails on the unexpected Patroni request; after the fix, all migration and candidate-selection cases pass.
  • make test passes across the repository.
  • go test -race ./pkg/cluster -run 'TestMigrate|TestGetSwitchoverCandidate' -count=1 passes.
  • make local builds the operator.
  • gofmt and git diff --check pass.

The tests use the Kubernetes fake client, pod events, and a mocked Patroni HTTP client. Live Kubernetes node rotation and database end-to-end tests were not run.

AI assistance: Codex (GPT-6) assisted with investigation, code, tests, documentation and this description.

Linked issues

Fixes #3059. Proposed approach posted in the issue before implementation.

Checklist

  • Go code is formatted.
  • Generated API code checked: no API fields changed, so no regeneration is needed.
  • Configuration/CRD/chart impact checked: no configuration options added.
  • Behavior is covered by unit tests.
  • Existing open PRs checked for overlap; none found for Operator cache obsolete IP address during cluster migration #3059.

Leave the replica candidate unset for single-pod clusters so migration returns after recreating the master instead of contacting its stale IP. Add migration regressions and document the recreate-only path.

Signed-off-by: Shubham Padkonde <shubhampadkonde12@gmail.com>
@Shubham-Padkonde

Copy link
Copy Markdown
Author

The COMPR/pr-labels check requires one of the project classification labels. Could a maintainer add bugfix? This account cannot apply labels. The fix addresses #3059; full Go tests, focused race checks and the operator build passed locally, as detailed above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Operator cache obsolete IP address during cluster migration

1 participant