Skip to content

fix(qdrant-backup-restore): SPHER-7668 capacity gate warns instead of… - #118

Open
ddakv wants to merge 2 commits into
mainfrom
fix/SPHER-7668-qdrant-capacity-gate-wcf-warning
Open

ddakv wants to merge 2 commits into
mainfrom
fix/SPHER-7668-qdrant-capacity-gate-wcf-warning

Conversation

@ddakv

@ddakv ddakv commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

What

recover_snap_shards refused every collection whose write_consistency_factor exceeded the target's peer count (capacity gate failed: write_consistency_factor=2 > target peers=1). On CST-397 this skipped all 69 collections when restoring into a single-node target: our document-index chart sets write_consistency_factor: 2 server-wide, so every manifest carries 2.

The premise was wrong. Qdrant places min(RF, peers) replicas per shard and requires min(wcf, placed replicas) acks per write (shard_distribution.rs, replica_set/update.rs, identical in 1.15.1, 1.16.0 and dev), so the collection is created writable. The gate now warns and proceeds, mirroring the existing peers < replication_factor branch.

How it works

One file, +3/−4: the wcf > peers branch of check_capacity_gate prints a WARNING instead of return 8; the function header comment states the real semantics. rc 8 remains for non-numeric input. Restored collections keep the manifest's replication_factor/write_consistency_factor, so the intended durability applies once peers are added.

Guarantees & caveats

  • Gate order unchanged: still before the state gate, so nothing is created or deleted before it runs.
  • On a peer-short target the restored data sits on min(RF, peers) replicas until peers join; both shortfalls are logged as WARNINGs.
  • Not in scope: the ConfigMap copy (regenerated by the CI injector on push) and the unit assertion on the test branch (tests/test_gates.sh expects rc 8 for check_capacity_gate 4 3 3; one-line flip prepared).

Testing

  • Unfixed script from main vs a one-peer qdrant/qdrant:v1.15.1 (cluster mode on) with a wcf=2, rf=2 backup: reproduces the customer's exact error, exit 1, target untouched.
  • Fixed script, same set, same target: both WARNINGs, VERIFIED (points=300, manifest=300), exit 0; collection green, rf/wcf preserved, payload index restored, both shards Active, upsert accepted.
  • 18/18 compose drills PASS on the 3-node lab; unit suite unchanged (only the 3 pre-existing failures from the test branch predating the round-2 refactor); shellcheck clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant