Fix join filter pushdown - #2249
Conversation
…in header Unused-column removal and `JoinStepLogical` aliases can hide a one-sided `WHERE` from `get_available_columns_for_filter`. Include those names so existing split and remap can push the predicate under the JOIN. Co-authored-by: Cursor <cursoragent@cursor.com>
…r can prune files Initiator listing runs on the wrap subquery (`SELECT cols FROM icebergCluster`), which previously had no WHERE. A left-only predicate on `count()` of `SELECT * … JOIN` never reached min/max file listing. Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the duplicated WHERE walker and the PK-walk-through-JOIN remapping. Wrap listing still uses collectFiltersForAnalysis and tryAddClusterWrapFilter. Co-authored-by: Cursor <cursoragent@cursor.com>
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e1b754451
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (parent_query->hasWhere()) | ||
| { | ||
| if (auto pred = copy_left_only(parent_query->getWhere())) | ||
| wrap_query.getWhere() = std::move(pred); |
There was a problem hiding this comment.
When a wrapped remote table is on the null-producing side of an outer join, copying every table-local predicate into its subquery changes join semantics. For example, with a remote right side of a LEFT JOIN, WHERE isNull(r.value) is copied below the join; rows matching a non-null r.value are then removed before the join, become null-extended unmatched rows, and incorrectly pass the original outer predicate. Check parent_join_tree and the join kind/side before copying a predicate, rather than treating every wrapped table expression as safe.
Useful? React with 👍 / 👎.
| if (parent_query->hasWhere()) | ||
| { | ||
| if (auto pred = copy_left_only(parent_query->getWhere())) | ||
| wrap_query.getWhere() = std::move(pred); |
There was a problem hiding this comment.
The copied predicate is added to the wrapper while the original remains above the join, and removeExpressionsThatDoNotDependOnTableIdentifiers does not reject nondeterministic expressions. Thus an IStorageCluster join with a left-only condition such as WHERE rand() % 2 = 0 evaluates independent rand calls in the wrapper and again after the join, changing the expected cardinality from roughly one half to one quarter. Listing predicates must not become an additional execution filter unless they are proven safe to duplicate.
Useful? React with 👍 / 👎.
| @@ -0,0 +1,86 @@ | |||
| -- Tags: no-parallel-replicas | |||
There was a problem hiding this comment.
The test explicitly executes SET enable_parallel_replicas = 0, so the no-parallel-replicas tag is not needed to establish its required execution mode. Remove the tag rather than excluding the test from that runner, as repository guidance prohibits adding no-* tags unless they are strictly necessary.
AGENTS.md reference: AGENTS.md:L143-L143
Useful? React with 👍 / 👎.
Does it also work for non count cases? |
…JOIN Copying a table-local WHERE such as `isNull(r.x)` under a LEFT JOIN remote right table changes join semantics. Use the same `isLeftOrFull` / `isRightOrFull` sides as JOIN filter pushdown. Related: #2249 Co-authored-by: Cursor <cursoragent@cursor.com>
Share the existing `and`-conjunct filter in Analyzer/Utils so wrap copy can drop `rand` and similar after the table-local strip. Those conjuncts stay only in the original WHERE above the JOIN. Related: #2249 Co-authored-by: Cursor <cursoragent@cursor.com>
…wn test `SET enable_parallel_replicas = 0` already pins the EXPLAIN plan, so skipping the ParallelReplicas suite is not needed. Co-authored-by: Cursor <cursoragent@cursor.com>
Issue (and fix) also cover cases with any aggregation functions that do not use columns from 'where' condition. I have column Combination of |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 78a27b3767
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (const auto * parent_query = select_query_info.query_tree->as<QueryNode>(); | ||
| parent_query && joinTreePreservesRowsForTable(parent_query->getJoinTree(), original_table_expression)) |
There was a problem hiding this comment.
The new guard excludes null-producing outer-join sides, but it still admits join shapes where prefiltering changes which rows are joined. For example, when the remote right side of an INNER ASOF JOIN has a right-only WHERE, copying it into the wrapper can discard the nearest match and select an older matching row instead; similarly, prefiltering either side of a PASTE JOIN shifts positional alignment. The normal pushdown path in filterPushDown.cpp explicitly disables right-side ASOF pushdown and all PASTE pushdown, so this wrapper path needs equivalent restrictions before copying the predicate.
Useful? React with 👍 / 👎.
| removeExpressionsThatDoNotDependOnTableIdentifiers(cloned, original_table_expression, query_context); | ||
| removeExpressionsThatAreNotDeterministicInScopeOfQuery(cloned, query_context); |
There was a problem hiding this comment.
This removes functions that are nondeterministic within a query, but stateful functions can still report themselves as deterministic; aiEmbed, for example, returns true from isDeterministicInScopeOfQuery while documenting that each call consumes quota and performs a potentially expensive external request. In an IStorageCluster join, such a WHERE is consequently evaluated in the wrapper and again above the join, doubling external calls and potentially exceeding ai_function_max_api_calls_per_query. The ordinary filter-pushdown path avoids this through ActionsDAG::hasStatefulFunctions, so copied predicates should apply the same exclusion.
Useful? React with 👍 / 👎.
Prefiltering those sides changes nearest-match and positional pairing. Skip the same sides for wrap listing so icebergCluster cannot drop the matching file either. Co-authored-by: Cursor <cursoragent@cursor.com>
Functions such as `aiEmbed` and `timeSeriesStoreTags` can be deterministic in a query while still having side effects. Skip them in the cluster wrap the same way JOIN filter pushdown uses `hasStatefulFunctions`. Co-authored-by: Cursor <cursoragent@cursor.com>
Wrap copy restated the same outer/`ASOF`/`PASTE`/`FULL` checks as filter pushdown, and stripped wrap-unsafe conjuncts in two walks. Co-authored-by: Cursor <cursoragent@cursor.com>
…andom settings `EXPLAIN` `Prewhere` is absent when parallel replicas rewrite the plan or when `optimize_move_to_prewhere` / `query_plan_optimize_prewhere` are randomized off. Co-authored-by: Cursor <cursoragent@cursor.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 20181690d7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| auto copy_left_only = [&](const QueryTreeNodePtr & predicate) -> QueryTreeNodePtr | ||
| { | ||
| auto cloned = predicate->clone(); | ||
| removeExpressionsThatDoNotDependOnTableIdentifiers(cloned, original_table_expression, query_context); |
There was a problem hiding this comment.
When an eligible IStorageCluster table participates in an inner join and the outer predicate is a bare column from the other side, such as WHERE r.flag, this call leaves the predicate unchanged: filterConjunctions returns immediately when its root is not a FunctionNode, so removeExpressionsThatDoNotDependOnTableIdentifiers never invokes its dependency check. The predicate is then attached to the cluster-only wrapper despite referencing r, which is absent from that wrapper's FROM clause, causing a valid join query to fail during planning; apply the keep predicate to non-function roots as well.
Useful? React with 👍 / 👎.
`filterConjunctions` skipped a ColumnNode root, so `WHERE r.flag` was copied onto the `IStorageCluster` wrap and planning failed. Related: #2249 Co-authored-by: Cursor <cursoragent@cursor.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a1d4502159
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (extension && !(predicate && !extension_has_predicate)) | ||
| return; |
There was a problem hiding this comment.
In the new wrapped-IStorageCluster path, tryAddClusterWrapFilter calls applyFilters while constructing the branch, before the overall query-plan optimizer can push additional predicates from enclosing subqueries down to the same source. If that early call creates an extension with predicate a, a later applyFilters carrying a AND b reaches this guard with both extension and extension_has_predicate set and is ignored, so object-storage listing never benefits from b (for example, an outer WHERE around the wrapped JOIN). Track predicate/filter changes or recreate the extension on subsequent filter applications rather than only for the null-to-non-null transition.
Useful? React with 👍 / 👎.
`canPrefilterJoinSide` only marks ordinary columns unavailable on the null-producing side. Attaching a rewritten equivalent-key filter to that child is still `allow_push_down_to_right` (dictionary / lookup / `ASOF`). Related: #2249 Co-authored-by: Cursor <cursoragent@cursor.com>
…te changes The cluster JOIN wrap applies the copied `WHERE` before the optimizer can push an outer filter, so listing used to keep `a` and miss `a AND b`. https: //github.com//pull/2249 Co-authored-by: Cursor <cursoragent@cursor.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d9ebb0b72e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| node, | ||
| [](const FunctionBasePtr & function_base) | ||
| { | ||
| return function_base->isDeterministicInScopeOfQuery() && !function_base->isStateful(); |
There was a problem hiding this comment.
When an IStorageCluster join predicate contains a server-local constant such as hostName, this check treats it as safe because the function is deterministic within a query and non-stateful. The original predicate is evaluated on the initiator after the join, but its copied wrapper predicate is serialized and evaluated on each remote server, where hostName can return a different value; for example, WHERE hostName() = '<initiator>' can pass locally while the wrapper removes every remote row. Also reject functions whose isServerConstant flag is set before copying the conjunct.
AGENTS.md reference: AGENTS.md:L7-L7
Useful? React with 👍 / 👎.
The wrap `WHERE` is sent to remote cluster nodes, where `hostName` can differ from the initiator and drop every row. https: //github.com//pull/2249 Co-authored-by: Cursor <cursoragent@cursor.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cf210cbe24
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return function_base->isDeterministicInScopeOfQuery() | ||
| && !function_base->isStateful() | ||
| && !function_base->isServerConstant(); |
There was a problem hiding this comment.
When the predicate calls dictGet or another external-dictionary function, this check accepts it: those functions inherit the IFunction defaults of deterministic-in-query, non-stateful, and non-server-constant. The copied predicate is then evaluated by the remote IStorageCluster wrapper even though the original predicate runs on the initiator; if the dictionary exists only on the initiator the formerly valid query now fails remotely, and if replicas have different dictionary contents rows can be filtered incorrectly before the join. Treat functions that depend on node-local external state as unsafe to duplicate across servers.
Useful? React with 👍 / 👎.
The wrap `WHERE` runs on remote cluster nodes. `dictGet`, `joinGet`, and other non-deterministic functions must stay on the initiator so a missing dictionary or different node-local state cannot drop rows or fail the query. https: //github.com//pull/2249 Co-authored-by: Cursor <cursoragent@cursor.com>
|
@codex review |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Build failed, cause some CI issues: |
CI triage for #2249 —
|
| Check | Class | PR-caused? |
|---|---|---|
Stateless (amd_asan_ubsan, distributed plan, parallel, 1/2) + all DROPPED jobs + PR gate |
infra / flaky | No |
Regression release iceberg_2 |
pre-existing (base branch) | No |
Regression release s3_export_part |
pre-existing (base branch) | No |
GrypeScanServer / -alpine |
infra (base-image CVE) | No |
1. Stateless tests (amd_asan_ubsan, distributed plan, parallel, 1/2) — infra/flaky
The only real failure in the PR workflow; everything marked DROPPED (integration, stress, other stateless shards, install/compat, AST fuzzer, SQLLogic/SQLStorm…) and the top-level PR gate are just downstream of it.
The failing step is "Start ClickHouse Server", not a test. The server's clickhouse-server.err.log shows two runner-environment problems during setup:
- Port conflicts — a leftover server was still holding the ports:
Listen [0.0.0.0]:9000 failed: ... Address already in use: 0.0.0.0:9000 (same for 8123, 8443, 9181, 9009, 9010, 9440, 9022, 9004, 9005, 9988) - DNS failure to the dataset bucket while loading the TPCH fixtures in setup:
DNSResolver: Cannot resolve host (clickhouse-datasets.s3.amazonaws.com), error 0: DNS error ... in query: INSERT INTO tpch.nation SELECT * FROM s3('https://clickhouse-datasets.s3.amazonaws.com/h/1/nation.tbl', NOSIGN, CSV)
Neither has anything to do with the Analyzer/join changes. The two sibling shards on the same build were green — amd_debug, parallel (10998 passed) and amd_asan_ubsan …, 2/2 (5509 passed) — as were both AST fuzzers and unit tests (12115 passed).
Fix: just re-run the PR workflow (or that one shard). No code change needed.
2. Regression iceberg_2 and s3_export_part — pre-existing, base branch
Both are EXPORT PARTITION / EXPORT PART partition-compatibility tests. This PR changes no export code — its diff is entirely join-filter-pushdown (src/Analyzer/Utils.*, src/Planner/*, filterPushDown.cpp, IStorageCluster.*, FunctionJoinGet.cpp) plus new stateless tests 04673–04678.
s3_export_part→.../export part/error handling/different partition key: the query is still rejected with exit code 36, but the message changed, so the string assertion fails:assert "Tables have different partition key" in results[0].output actual: Code: 36. ... the destination partition expression uses column 'p', which is not part of the source MergeTree partition key. (BAD_ARGUMENTS)iceberg_2→ 6 scenarios under.../partition compatibility/rejected/(reversed order, transform-vs-identity, truncate width mismatch, field-count mismatch, unsupportedintDivexpression, unpartitioned destination): each expects exit code 36 (rejected) but now gets exit code 0 (accepted).
That loosening of partition-compatibility comes from base-branch work already merged into antalya-26.6 — #2253 (export-partition-monotonic-compat) and #2229 (allow_non_matching_schema_export_partition_by_position) — which the regression suite exercises via the merge commit. The EXPORT PARTITION "rejected" expectations and the error-message assertion are simply stale against the new behavior.
Fix (not this PR): the export-partition regression expectations need updating to match #2253/#2229 — belongs to the export-partition feature owners. Worth confirming these two suites are already red on antalya-26.6 itself.
3. GrypeScanServer (server / -alpine) — infra
Container-image CVE scans of the built server image (-alpine reports 1 high/critical). Independent of the C++ source — this PR changes no image or dependencies. Track/waive with the image maintainers; not a blocker for this PR.
Health check
The join-filter-pushdown change itself is in good shape: Fast test (9305 passed), all builds, unit tests (12115 passed), both AST fuzzers, and the two stateless shards that actually ran are green — including amd_debug, parallel, which exercises the PR's new 04673–04678 cluster-wrap/pushdown tests. The only obstacles to a green board are a runner flake (re-run) and two categories of red that pre-date this PR (export-partition regression expectations on the base branch, and base-image CVE scans). I'd re-run the PR workflow and, separately, get the export-partition regression expectations refreshed against antalya-26.6.
— @blau-ai · analysis only; no code changes made
| 'TSV', | ||
| 'n UInt64') AS l | ||
| LEFT JOIN t_04677_right AS r ON l.n = r.n | ||
| WHERE l.n < 2 AND hostName() = hostName(); |
There was a problem hiding this comment.
What is the goal of hostName() = hostName() ? In what cases it would be false?
There was a problem hiding this comment.
It's a check that 'always true' part is properly removed from 'where' condition. See #2249 (comment)
If not removed, pruning can't be executed on initiator.
Add a test that hostName really removed from condition.
| 'TSV', | ||
| 'n UInt64') AS l | ||
| LEFT JOIN t_wrap_right AS r ON l.n = r.n | ||
| WHERE l.n < 2 AND timeSeriesStoreTags(l.n, []) = l.n; |
There was a problem hiding this comment.
I need to investigate this message from ai, but seems it makes seanse.
this test does not distinguish whether the fix is present. count() is 2 either way (timeSeriesStoreTags(l.n, []) = l.n doesn't change the row set, and the cluster is single-node, so the "tags stored twice" effect is invisible). With removeExpressionsThatAreUnsafeToDuplicate turned into a no-op, the test still passes.
Suggestion: assert the side effect itself (number of tag sets in the target TimeSeries table — one row per key, not two), or use EXPLAIN on the wrap subquery to check that timeSeriesStoreTags is absent from its WHERE while n < 2 is present.
| { | ||
| if (function->isWindowFunction() || function->isAggregateFunction()) | ||
| return false; | ||
| if (function->isOrdinaryFunction()) |
There was a problem hiding this comment.
What we should to do if function not isOrdinaryFunction ? Just ignoring?
There was a problem hiding this comment.
Window and aggregate functions are checked above, so here can be only ordinary or unresolved functions.
But all functions must be resolved when this method is called.
I rewrite test to be more clean, without logic change.
| return; | ||
|
|
||
| auto filter_dag = filter_actions->clone(); | ||
| const auto filter_column_name = filter_dag.getOutputs().at(0)->result_name; |
There was a problem hiding this comment.
nit: I prefer to check size before use at if we receive vector outside.
| return function_base->isDeterministic() | ||
| && function_base->isDeterministicInScopeOfQuery() | ||
| && !function_base->isStateful() | ||
| && !function_base->isServerConstant() | ||
| && !functionIsDictGet(function_base->getName()) | ||
| && !functionIsJoinGet(function_base->getName()); |
There was a problem hiding this comment.
dictGet implementations already return false from isDeterministic(), and this PR does the same for the resolved FunctionJoinGet. Can we remove the explicit functionIsDictGet and functionIsJoinGet checks here?
There was a problem hiding this comment.
I'm not sure about all dictGet family.
For dictGetDescendants and dictGetChildren FunctionDictGetDescendantsOverloadResolverImpl::buildIml returns FunctionDictGetDescendantsBase, which does not override isDeterministic.
I don't know is it bug or not.
This additional checks do not make something worse.
|
|
||
| if (extension) | ||
| return; | ||
| { |
There was a problem hiding this comment.
createExtension is no longer one-shot: if (extension) return; was replaced by a rebuild whenever filter_hash changes.
On the icebergCluster JOIN wrap path a single ReadFromCluster builds the task iterator twice: first in tryAddClusterWrapFilter for the copied predicate a, then in optimizePrimaryKeyConditionAndLimit with the final a AND b (different hash) a second time. Even without an outer WHERE this can repeat if the hash of the renamed/merged DAG differs from the hash of the DAG that optimizePrimaryKeyConditionAndLimit composes from the same a.
For Iceberg this is not free: the IcebergIterator constructor is not lazy — it walks the delete manifests synchronously and starts a producer thread over the data manifests. The first iterator does metadata I/O before it gets replaced.
Could we build the iterator once in initializePipeline, after filter pushdown is complete? The pruning test only checks the final predicate and does not catch the repeated initialization.
Later empty `applyFilters` must not drop a wrap `WHERE`, and recreating the iterator on each hash change listed twice. Keep the first non-empty DAG in `applyFilters` and create the extension only in `initializePipeline`. Co-authored-by: Cursor <cursoragent@cursor.com>
CI triage — 7 failing checks: 0 PR-caused, 5 infra, 2 pre-existing (+ regression flake)Short version: nothing here points at this PR's code. The build is green everywhere, the PR's own new tests pass, and every red check is explained by infra (OOM / lost disk lease / failed Pre-existing (not this PR)Integration tests (amd_asan_ubsan, targeted) — 12/13 and Integration tests (amd_asan_ubsan, db disk, old analyzer, 4/8) — 12/1090 i.e. the ClickHouse node dies on the data read and never comes back. This is not a join/filter-pushdown code path — it's a bare Infra / environment (safe to re-run)Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 1/2) — 31 failed
The same tests pass on Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 2/2) — 3 failed
SQLLogic test — step "Clone sqllogictest repo" failed → external Regression release swarms — 1 scenario of 1521 failed (plus Health checkEverything that reflects this PR's code is green: all builds ( (Automated triage from the praktika S3 reports; I can't build/run ClickHouse in this job, so classifications are evidence-based from CI logs.) |
|
I have a pretty list of issues for this from couple of audit review rounds @ianton-ru Audit review — PR #2249 "Fix join filter pushdown"Repository: Scope of the change: JOIN filter pushdown when side column names do not match the JOIN header Confirmed issues1.
|
`formatWithSecretsOneLine` printed injected access keys to any user running `EXPLAIN actions = 1`. Use the same privilege-aware formatter as `SHOW CREATE`. Co-authored-by: Cursor <cursoragent@cursor.com>
`addFilterOnTop` applies the extracted DAG to the child header, so identifier-renamed inputs such as `__table1.a` must be rewritten to physical names first. Co-authored-by: Cursor <cursoragent@cursor.com>
Empty later `applyFilters` still must not drop wrap `WHERE`, but a later non-empty DAG (outer `datetime` after JOIN wrap) has to tighten initiator file listing without recreating the iterator. Co-authored-by: Cursor <cursoragent@cursor.com>
Remote `Distributed` wraps and cluster fallback sources must keep the optimizer's later `applyFilters` so MergeTree index analysis is not pinned to the dummy-analysis DAG. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Add fixes for high and medium issues from #2249 (comment) |
5 participants
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix join filter pushdown
Documentation entry for user-facing changes
Solved #2245
Push left-only JOIN filters when column names do not match the join header
A left-only
WHEREoncount()ofSELECT * … JOINwas not pushed under the JOIN. The filtered column often disappeared from the JOIN output (unused-column removal aftercount()ofSELECT *), while the Filter DAG still referenced it — sometimes under an identifier name such as__table1.arather thana.get_available_columns_for_filterrequired the name to appear in the JOIN header, sosplitActionsForJOINFilterPushDownnever saw it.That is not Iceberg-specific. On MergeTree the same shape skipped
Prewhere/ index analysis on the left read. The filter still ran after the JOIN, so the result was correct but the left table was scanned without the predicate.JoinStepLogicalcan also alias a side input (a) to a JOIN-output / filter name (__table1.a). Pushdown matches filter inputs against the available-column list, thenfix_predicate_for_join_logical_stepremaps aliases back to input names. The output alias has to be listed or the split never runs.JOIN filter pushdown
In
tryPushDownOverJoinStep:JoinStepLogical, output actions thatfromLeft()/fromRight()are added to that list (including aliases). The existing split and remap then push the predicate under the JOIN.Covered by
04673_join_filter_pushdown_count_subquery.sql: MergeTree left + Memory right,count()ofSELECT * … LEFT JOIN … WHERE foo.a < 40(and the same with an extra(SELECT * FROM t_left) AS foowrap).EXPLAIN actions = 1must containPrewhere.icebergClusterfile listingicebergCluster(IStorageCluster) lists files on the initiator. The planner wraps the left cluster table so remotes do not get the JOIN (SELECT cols FROM icebergCluster). That wrap had noWHERE, so initiator listing stayed unfiltered even after JOIN pushdown.Left-only
WHERE/PREWHEREis copied onto the wrap withremoveExpressionsThatDoNotDependOnTableIdentifiers(same helper asIStorageCluster::updateQueryWithJoinToSendIfNeeded). Wrap planning runscollectFiltersForAnalysis. Listing-only filters are attached on the wrap source without adding aFilterStepthat would drop unused wrap columns.Covered by
test_cluster_join_filter_minmax_pruning.py:IcebergMinMaxIndexPrunedFilesforicebergS3Clusterwith a plainWHERE, withJOIN … WHERE, and with outercount()ofSELECT * … JOIN … WHERE.CI/CD Options
Exclude tests:
Regression jobs to run: