Repository navigation
fix(databricks)!: keep raw strings, insert parentheses and WINDOW placement apart in trees - #254
Merged
Merged
Conversation
…re one INSERT INTO t1 ( SELECT 1 ) dropped its parentheses when no alias followed, so it parsed to the same tree as INSERT INTO t1 SELECT 1, and INSERT INTO t1 ( VALUES ( 1 ) ) read as bare VALUES rows. The pair is now a paren node, as queryPrimary keeps elsewhere; an aliased query and REPLACE ON's query keep their subquery node. A written-forms case covers no, one and two pairs, VALUES and INSERT OVERWRITE.
Name and scope checks walked only a SELECT's top-level joins, so a join inside parentheses, as in FROM ( t1 JOIN t2 ON ... ), skipped its ON and USING checks and its nested derived tables, in every dialect. Lineage already flattened such joins. One walk in the source tracing now lists every FROM part and join, through any number of parentheses, and the ON, USING, scalar-subquery, duplicate-alias, derived-table and LATERAL checks read it. Each part records where its sources start, so a LATERAL item inside parentheses reads the tables before it there. Databricks and PostgreSQL tests cover ON, USING, a nested derived table, a join of a parenthesised join, a scalar subquery and a repeated alias, with LATERAL and non-LATERAL controls. A Databricks case also checks that an insert's parenthesised query reads as the bare one.
…TIME ZONE TIMESTAMP WITHOUT TIME ZONE WITH LOCAL TIME ZONE parsed as a WITHOUT TIME ZONE type with a time zone. A TIMESTAMP now takes at most one of the two clauses, so the text is refused, as Spark's grammar does.
…pipe step's input A pipe operator wraps a query it cannot extend as SELECT * FROM ( query ), and extended a written SELECT * FROM ( query ) in place, so ( q ) |> WHERE c read as SELECT * FROM ( ( q ) ) |> WHERE c, and q |> WHERE c as SELECT * FROM ( q ) |> WHERE c. A written select of that shape, one unaliased subquery and no pipe step before it, is now wrapped like any other input; chained pipe steps and SELECT * FROM t still extend in place. A written-forms case covers the pipe forms with no, one and two pairs of parentheses, written star selects, an alias and chained steps; the insert case adds a column list before a parenthesized query.
A string literal and a property key lost how they were written: r'k' parsed to the same tree as 'k' wherever a string literal or a tag or credential key was read, and adjacent strings such as 'a' 'b' to the same tree as 'ab'. - A single raw string stays a raw string literal at every Databricks string-literal site, through one shared one_string helper. - Adjacent strings at those sites are a new Literal::JoinedString holding the joined value and each piece as written. value_str gives the joined value and is_string holds, and the shared semantics read it as the string it joins to. - Tag and COPY credential keys record those written as a raw string or adjacent strings in a new written_keys field, serialized only when set. Adjacent strings stored in a plain text field (comments, LOCATION, formats, function bodies and the like) are still joined; BigQuery also still joins them. A written-forms case covers raw and adjacent strings and tag keys, and a semantics case checks that adjacent strings read as the joined one. BREAKING CHANGE: sqlquack-syntax adds the Literal::JoinedString variant; code that matches Literal exhaustively must handle it.
…INDOW over a star of it A WINDOW clause after a parenthesized query whose query could not take it, such as one with a LIMIT inside, wrapped the query in a synthesized SELECT * FROM ( ... ), so ( SELECT 1 LIMIT 1 ) WINDOW w AS ( ) parsed like SELECT * FROM ( ( SELECT 1 LIMIT 1 ) ) WINDOW w AS ( ). The window now goes on the unaliased subquery node with the other clauses written after the parentheses, in a new windows field serialized only when set. A query that can take the window still takes it. A written-forms case covers WINDOW after the parentheses with and without a LIMIT inside, against WINDOW over a star of the same query.
… at the current parser tree Keeping adjacent and raw string spellings changed the parser tree and one tree of string-type.subscript-dot, so two records went stale: - string-type.subscript-dot is re-read and approved: 'c' 'c' now keeps both pieces and no longer shares the tree of 'cc', which was the only reason it was held. All four cases are accepted on both sides, and the tree screen finds no same-tree variant in 128; - object-type's exclusion is repeated unchanged: the catalogue and obligations are the same, and the parser still refuses OBJECT as input in all three cases.
…e screen The screen makes no case variants, because the lexer reads a keyword and the r or R prefix of a raw string case-insensitively. The allowlist now says so, so r'x' and R'x' giving one tree reads as a deliberate allowance, and a receipt that shows Databricks reading them differently would make it a finding.
…INDOW inside it A WINDOW clause after a parenthesized query still moved into the query when that query could take it, so ( SELECT 1 ) ORDER BY 1 WINDOW w AS ( ) read as ( SELECT 1 ORDER BY 1 ) WINDOW w AS ( ), and a WITH before the parentheses read as if the WINDOW were inside them. The clauses after the parentheses now always stay on the subquery node that holds them, WINDOW included. Name checks now read those windows: a window function inside the parentheses resolves a window named after them, and the window's partition and order keys are checked in the inner query's scope, so ( SELECT sum ( c1 ) OVER w FROM t1 LIMIT 1 ) WINDOW w AS ( ORDER BY c1 ) no longer reports an unknown named window. A Databricks case shows the six spellings give six trees, and lineage cases compare the windowed query with its unparenthesized form.
…wn sides A USING check took every earlier FROM part as its left side and only the next part as its right, so a join written in parentheses was checked against the wrong tables, in every dialect. A key missing from the table before USING inside the parentheses went unreported when a table outside had it, and USING after the parentheses reported a key that a later table inside them had. Each FROM part now records the parts on either side of the join that adds it: inside parentheses the left side is the earlier parts there, and the join that adds a parenthesised join has all of it on its right. Databricks and PostgreSQL cases cover USING inside and after the parentheses.
A raw string such as r'a' had no type in Databricks lineage, though a plain string had STRING. It is now STRING too, as typeof reports.
Keeping a WINDOW written after a parenthesized query on the node after the parentheses changed the parser tree, so object-type's exclusion went stale. It is repeated unchanged: the catalogue and obligations are the same, and the parser still refuses OBJECT as input in all three cases.
…its findings The case for an unknown column in such a window compared two empty finding lists, because named window specs are not checked for unknown columns in any form. The test now shows what the change does: the window function inside the parentheses finds the window named after them, and a name no WINDOW clause defines is still unknown.
kvlonge
enabled auto-merge
November 18, 2026 14:58
kvlonge
disabled auto-merge
November 18, 2026 15:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Databricks tree screen still found spellings that parsed to the same tree as another: raw and adjacent strings, an insert's parenthesised query, a written
SELECT * FROM ( query )against a pipe step's input, and a WINDOW after a parenthesized query. Name checks also skipped joins written in parentheses, in every dialect.Changes
r'c'stays a raw string, and adjacent strings become a newLiteral::JoinedStringthat keeps each piece. This is breaking: a new public variant, and the tree JSON of these spellings changes. An insert's parenthesised query, a writtenSELECT * FROM ( query )and a WINDOW after the parentheses each keep their own tree. A TIMESTAMP with both time zone spellings is refused.t1 a, t2 b JOIN t3 USING (c1) JOIN t1 USING (c2)now reports E201, a true positive under DuckDB's rules.Verification
make check-cion the final head.JoinedStringand never referencesLiteralorSubquery.