Skip to content

Keep distributed_plan_workers_provisioning_timeout_ms at 10s under compatibility - #118996

Merged
serxa merged 2 commits into
masterfrom
distributed-plan-provisioning-timeout-compat
Sep 9, 2026
Merged

Keep distributed_plan_workers_provisioning_timeout_ms at 10s under compatibility#118996
serxa merged 2 commits into
masterfrom
distributed-plan-provisioning-timeout-compat

Conversation

@serxa

@serxa serxa commented Sep 9, 2026

Copy link
Copy Markdown
Member

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Description

distributed_plan_workers_provisioning_timeout_ms (default 10000) was recorded in the settings history as a change from 0 to 10000, so compatibility below 26.9 rolled it back to 0. Per the setting's own description, 0 makes a query wait only for the initial lease-and-verify pass, with no retries — the pre-feature no-wait behavior.

The wait-and-retry provisioning is brand-new functionality with no previously released default to preserve, so restoring 0 under compatibility has no backward-compatibility benefit and only makes worker provisioning less robust (a query can give up on workers that would have become reachable within the 10s budget).

This sets the recorded previous value equal to the new default (10000) so compatibility keeps the 10s budget in all modes, matching the established pattern for new settings that should not be rolled back by compatibility (for example statistics_max_set_size_for_exact_selectivity_estimation, whose entry is {10000, 10000} for the same reason).

Only src/Core/SettingsChangesHistory.cpp changes; the compiled default is unchanged.


Workflow [PR]
Sync PR [sync-upstream/pr/118996]

…compatibility

The settings history recorded this new setting with previous_value 0, so
`compatibility` below 26.9 restored the pre-feature no-wait behavior (only the
initial lease-and-verify pass, no retries). The wait-and-retry provisioning is
brand-new functionality with no released default to preserve, and restoring the
no-wait behavior only makes worker provisioning less robust.

Set the recorded previous value equal to the new default (10000) so that
`compatibility` keeps the 10s budget in all modes, matching the established
pattern for new settings that should not be rolled back by `compatibility`.
@clickhouse-gh

clickhouse-gh Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Workflow [PR], commit [e2eb1f7]

Summary:


AI Review

Summary

This PR changes the 26.9 SettingsChangesHistory.cpp row for distributed_plan_workers_provisioning_timeout_ms from {0, 10000} to {10000, 10000} so compatibility no longer rolls the timeout back to 0. The change is not correct as written: it erases the released 26.7/26.8 boundary instead of modeling it, so older compatibility modes would keep the new 10-second provisioning budget rather than restoring the pre-feature no-wait behavior.

Findings

❌ Blockers

  • [src/Core/SettingsChangesHistory.cpp:76] compatibility is supposed to emulate the first released version that shipped a behavior, not the current master default. Released v26.8.2.7-lts already shipped distributed_plan_workers_provisioning_timeout_ms = 10000, and its history row says compatibility below 26.8 restores the legacy 0 no-wait behavior. With this PR, there is no 26.8 row on master, and applyCompatibilitySetting in [src/Core/Settings.cpp:9845] therefore leaves the default 10000 in place for both compatibility = '26.8' and compatibility = '26.7'. That changes the released contract for 26.7 users and also makes the history/documentation path report the setting as if it were only introduced in 26.9. Suggested fix: keep the no-op 26.9 row only if you also restore the real 26.8 boundary row with {0, 10000}.
Tests
  • ⚠️ Add a focused stateless test that checks getSetting('distributed_plan_workers_provisioning_timeout_ms') under compatibility = '26.8' and compatibility = '26.7'. That is the smallest proof that the version boundary is modeled correctly and would catch the missing 26.8 history row on master.
Final Verdict

❌ Not ready to merge. The fix needs to preserve the released 26.8 boundary while keeping 26.8 itself at 10000; changing the 26.9 row to a global no-op does not do that.

LLVM Coverage Report

Measured on commit e2eb1f7.

Metric Baseline Current Δ
Lines 88.90% 88.90% +0.00%
Functions 91.80% 91.80% +0.00%
Branches 81.30% 81.30% +0.00%

Changed lines: Changed C/C++ lines covered: 7/7 (100.00%) · Uncovered code

Full report · Diff report

@clickhouse-gh clickhouse-gh Bot added the pr-not-for-changelog This PR should not be mentioned in the changelog label Sep 9, 2026
Comment thread src/Core/SettingsChangesHistory.cpp Outdated
{"force_write_through_distributed_cache", "auto", "auto", "New setting overriding the server setting `enable_write_through_distributed_cache` for a single query."},
{"distributed_cache_min_inflight_bytes_to_discard_connection_on_seek", 0, 4 * 1024 * 1024, "New setting to drop and reopen a distributed cache connection on a seek when too many in-flight bytes would otherwise be discarded. Defaults to 4 MiB; 0 restores the previous behavior (always reuse the connection via the read range id)."},
{"distributed_plan_workers_provisioning_timeout_ms", 0, 10000, "New setting bounding how long a query waits for leased stateless workers to become reachable before execution; `compatibility` below 26.9 restores the previous no-wait behavior."},
{"distributed_plan_workers_provisioning_timeout_ms", 10000, 10000, "New setting bounding how long a query waits for leased stateless workers to become reachable before execution. The previous value is deliberately equal to the new default so `compatibility` does not restore the pre-feature no-wait behavior: the wait is brand-new functionality with no released default to preserve, and restoring the no-wait behavior would only make worker provisioning less robust."},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compatibility is supposed to emulate the first released version that shipped a behavior, not the current master shape. Released v26.8.2.7-lts already ships distributed_plan_workers_provisioning_timeout_ms = 10000, and its SettingsChangesHistory.cpp records that rollback boundary in the 26.8 block with previous_value = 0. Changing the 26.9 copy to {10000, 10000} fixes compatibility = '26.8', but it also stops compatibility = '26.7' and older from restoring the legacy no-wait behavior.

So this needs a version-boundary fix, not a permanent no-op: move the row to the 26.8 block with previous_value = 0, and pin it with a focused stateless test for compatibility = '26.8' versus compatibility = '26.7'.

@serxa
serxa marked this pull request as ready for review September 9, 2026 11:16
@clickhouse-gh clickhouse-gh Bot added the comp-settings Settings & configuration infrastructure (server/user/query settings, profiles). label Sep 9, 2026
@serxa
serxa enabled auto-merge September 9, 2026 12:52
@clickhouse-gh

clickhouse-gh Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Build profile diff (arm_release)

Comparing e2eb1f7e1 with master 8d16f3272 (stripped binary size, per-symbol sizes and ThinLTO time; object sizes against the warmup build of e76c8434d; compile times per translation unit against the most recent warmup build that recompiled it).

✅ No significant changes.

Binary sizes

programs/clickhouse-stripped: smaller than the master baseline by the known offset between the two builds, so the difference is not shown. A delta that differs from the offset by more than 50% of it is shown, in either direction.

The official master build is compiled with -g and a pull request build is not, and XRay counts debug instructions towards its instrumentation threshold, so master instruments thousands of functions more and its binary is ~0.4% larger no matter what the pull request does.

Compile time of recompiled translation units

8 translation units recompiled, 11 s compile time in total, 8 of them have a recent master baseline.

Job report

@serxa
serxa added this pull request to the merge queue Sep 9, 2026
Merged via the queue into master with commit d40b62f Sep 9, 2026
174 checks passed
@serxa
serxa deleted the distributed-plan-provisioning-timeout-compat branch September 9, 2026 20:11
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp-settings Settings & configuration infrastructure (server/user/query settings, profiles). pr-not-for-changelog This PR should not be mentioned in the changelog pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants