From 807afbc30878ec39f5f0e91b2ed2c56268ff0c70 Mon Sep 17 00:00:00 2001 From: Lari Hotari Date: Mon, 27 Jul 2026 00:50:41 +0300 Subject: [PATCH] [fix][meta][branch-4.0] Tolerate concurrent creation of the underreplication LAYOUT node ### Motivation `PulsarLedgerUnderreplicationManager.checkLayout()` creates `/underreplication/LAYOUT` with a check-then-act sequence: it tests `store.exists(layoutPath)` and, when the node is absent, writes it with expected version `-1` (create-if-absent). When several managers are constructed concurrently against the same metadata store, they all observe the node as absent and all attempt the create. Every loser receives a `BadVersionException`, which the bare `join()` propagated out of the constructor instead of retrying. Reported in #16455 as a bookie failing to start when several bookies are started at once: java.util.concurrent.CompletionException: MetadataStoreException$BadVersionException: KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /ledgers/underreplication/LAYOUT at ZKMetadataStore.handlePutResult(ZKMetadataStore.java:210) The same race makes `AuditorPeriodicBookieCheckTest` flaky on branch-4.0, where an `AuditorElector` and the test's own `LedgerUnderreplicationManager` race to create the node. ### Modifications Ignore `BadVersionException` from the create and let the enclosing `while (true)` loop run again: the next iteration observes the node, validates the layout type and version, and breaks. Any other failure is still rethrown. This ports the `checkLayout()` change already present on master and branch-4.2, where it landed as part of #25219 (PIP-454) rather than as a standalone fix, which is why it was never backported. branch-4.1 and branch-3.0 need the same change. --- .../bookkeeper/PulsarLedgerUnderreplicationManager.java | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java b/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java index 1ed465c5c7da0..920b022905ea2 100644 --- a/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java +++ b/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java @@ -186,7 +186,13 @@ private void checkLayout() throws ReplicationException.CompatibilityException { if (!store.exists(layoutPath).join()) { LedgerRereplicationLayoutFormat.Builder builder = LedgerRereplicationLayoutFormat.newBuilder(); builder.setType(LAYOUT).setVersion(LAYOUT_VERSION); - store.put(layoutPath, builder.build().toString().getBytes(UTF_8), Optional.of(-1L)).join(); + try { + store.put(layoutPath, builder.build().toString().getBytes(UTF_8), Optional.of(-1L)).get(); + } catch (ExecutionException | InterruptedException e) { + if (!(e.getCause() instanceof MetadataStoreException.BadVersionException)) { + throw new RuntimeException(e); + } + } } else { byte[] layoutData = store.get(layoutPath).join().get().getValue();