Repository navigation
fix(databricks): keep parenthesized queries, clauses and spellings apart in trees - #249
Merged
Merged
Conversation
…ial value types in trees Each of these spellings parsed to the same tree as another one, so a case could not tell them apart: - a backquoted resource kind (USING `JAR`), STORED AS format and EXTRACT field now set kind_quoted, stored_as_quoted and field_quoted; - SET TAGS and UNSET TAGS keys record which were backquoted and which were strings, as COPY credential keys already did, through one shared key_spellings helper; - a COPY credential value that is not a string (k = 1, k = true) is listed in bare_values; - TIMESTAMP WITH LOCAL TIME ZONE keeps its spelling. The new shared fields serialise only when set, and the Snowflake parser fills their defaults (through set_tag and unset_tag for the tag actions), so other dialects' trees are unchanged.
A subquery node holding an ORDER BY or LIMIT written after a parenthesized query was read through its inner query only, so its ORDER BY was never checked: ( SELECT id FROM orders ) ORDER BY nope gave no finding in Snowflake, DuckDB and PostgreSQL, though the bare query does. An unaliased subquery node's ORDER BY now resolves in the scope of the query it holds, as that query's own ORDER BY does, with the same node checks, and a set operation's outputs are checked as for its own ORDER BY. Its ORDER BY keys decide rows when a LIMIT or OFFSET follows them, and a nested query in an expression stops at such a node, so its clauses are not read at the outer level. Snowflake cases cover the query with and without a LIMIT inside the parentheses.
…a query Extra parentheses parsed to the same tree as fewer ones: - FROM: a table's first pair still sets its parenthesized flag; every further pair, and a pair around any other relation (a table function, a join, an inline table), is now a paren node, so ( ( t1 ) ) and ( range ( 1 ) ) differ from ( t1 ) and range ( 1 ); - queryPrimary keeps ( query ) as a paren node, so TABLE ( ( SELECT 1 ) ), USING INVENTORY ( ( SELECT 1 ) ), set operands and IN or EXISTS subqueries in a second pair differ from one pair; - CREATE TABLE AS and CREATE VIEW move one outer pair back into the flag they already set, so a single pair keeps its existing tree and a second pair shows as a paren node. Clauses written after a parenthesized query, and WITH before it, keep their place without a synthesized SELECT * FROM ( ... ): - ORDER BY, SORT BY, CLUSTER BY, DISTRIBUTE BY, LIMIT and OFFSET after the parentheses go on an unaliased subquery node holding the query, the tree Snowflake gives, so they stay apart from the same clauses inside the parentheses; WINDOW moves onto a query that can take it; - a pipe step after the parentheses records its mark on the select inside instead of wrapping it; - WITH before the parentheses sets a new ctes_outside flag on that node, skipped when false, so other dialects' trees are unchanged. Lineage and findings read through the paren and subquery nodes: semantics tests check that each extra pair and each clause after the parentheses read as the bare query, including an unknown ORDER BY column after a LIMIT inside, and that a pipe step keeps its outputs' ranges. Written-forms cases cover the parentheses, the clauses at one to three pairs, in CTAS, IN and scalar subqueries, and both WITH positions.
The unused-CTE-output and key rules matched only a SELECT root or CTE body, so a whole query in parentheses, WITH before a parenthesized query, a clause after one, or a CTE body in a second pair of parentheses lost their diagnostics. Both rules now read the query within any parentheses or unaliased subquery node, and count that node's clause expressions as reads, so a CTE column read only by an ORDER BY after the parentheses is kept. A CTE body's SELECT is found after any number of opening parentheses, so its fix still applies. A Databricks test covers each form beside its bare control, and the kept column.
…urrent parser tree The spelling and parentheses fixes changed the parser tree and two of alter-table's trees, so two records went stale: - object-type's exclusion is repeated unchanged: the catalogue and obligations are the same, and the parser still refuses OBJECT as input in all three cases; - alter-table is re-read: its two changed trees only add that a tag key was a string, two of its earlier findings now have trees of their own, and SET TAGS without = still waits on a receipt, so it stays held.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Databricks tree screen found spellings that parsed to the same tree as another one: backquoted names, tag key and credential value spellings, and extra pairs of parentheses around a relation or a query. A review of the parentheses fix also found that clauses after a parenthesized query were read in the wrong scope, which hid unknown-column findings in Databricks, Snowflake, DuckDB and PostgreSQL.
Changes
ctes_outsideflag, serialized only when set.(SELECT id FROM orders) ORDER BY nopeis reported in every dialect.Verification
make check-cion the final head.