Repository navigation
feat(databricks): verify Databricks SQL syntax coverage against 2026.45 receipts - #248
Merged
Merged
Conversation
Function and AggregateFunction replace their distinct flag with set_quantifier: none, all or distinct. The Databricks parser and Snowflake's generic aggregates record ALL, which was dropped, so a call written with ALL no longer has the same tree as one without it. Translation, lint and semantics read only whether the quantifier is DISTINCT, so ALL behaves exactly as no quantifier; a conversion test holds that. BREAKING CHANGE: serialised call trees carry set_quantifier in place of distinct.
The set quantifier rename serialises a call's quantifier as "none" or "all" rather than a false distinct flag, and the check for COUNT(*) alone read any non-empty field as something it does not understand. COUNT(*) therefore looked like it read the source's columns, and converting it over a table with ARRAY or VARIANT columns checked their JSON for nothing. The check now reads the quantifier the way the rest of conversion does: only DISTINCT counts. The explicit-ALL conversion test moves to its own table-driven file, since test_convert.rs passed 2,000 lines.
…quantifier Snowflake and SQLite reviews whose forms' trees differ from the reviewed ones only by the call flag now read as set_quantifier are re-bound, each noted as diff-checked. A re-bound review check refuses every re-bind whose form changed by more than its class allows: a set_quantifier re-bind only by that flag in the trees, and a stage re-bind only by the stage of cases expected to be accepted, each the stage of its receipt phase.
…p what was written The Databricks parser reads each documented statement into a tree instead of opaque text, and keeps in that tree every spelling the grammar tells apart, so a form's evidence can be told from its neighbours': - documented statements checked against their syntax pattern become trees of clauses, with a paren node only on a ( whose content a sibling choice spells without it; compound statements become the shared scripting tree; - DDL, DML, COPY INTO, directory inserts, pipe operators, hints, STREAM, TABLESAMPLE, NEAREST BY and WATERMARK keep their clauses; - optional keywords (VARIABLE, AS, DATABASE or NAMESPACE, TEMP), omitted = and commas, parenthesised lists and relations, quoting of names and keys, and written clause order stay in the tree. The shared tree gains optional fields that serialise only when set, so the other dialects' trees and formatting are unchanged; their parsers and the semantics and diagnostics readers fill or ignore the defaults. Scripting types move to their own expressions/scripting domain, whose entry is the scripting_sql wrapper SQL Server and Databricks shared.
…es are not reviewed The Databricks parser now reads as trees many forms that used to be gaps. The formatter and lint read Databricks SQL through parse_strict_reviewed, which keeps forms whose layout is not reviewed yet as gaps, so they still refuse such a statement with the same opaque-statement message as before, while the coverage check sees the trees. A form leaves that list only with reviewed layout and formatter checks.
…e on it The coverage check gains a tree screen: for each accepted case, variants that change one lexer token (drop, synonym swap, backquote, string, double quote, requote, unquote, insert AS, insert =, insert , and a matching paren pair) are parsed, and any that gives the same tree marks a spelling the tree does not keep. validate fails closed when an approved form has a same-tree variant that screen.toml does not allow; an entry is pinned to a token, the last token, or text no lexer token covers, and an entry for an unknown or unscreened form fails. The checker also: - lets an approval resolve the changes_requested reviews it answers, and lets a changes_requested review block a current approval; - shows retired forms in the summary, apart from excluded ones, and reports suppressed obligations; - counts a receipt from any engine release of the pinned year; - treats a native parser's resource guard as a guard failure, in the depth probe's analysis too; - refuses a bad resolves entry and a family review that excludes unlisted forms.
…ling Scripts that turn the pinned Databricks pages into catalogue forms, derive each form's obligations and cases, and record and compare the warehouse's receipts: - the catalogue builder reads Databricks SQL 2026.45's documentation; - obligations pair each clause-order interaction with the two clauses it reorders, and cases settle by obligation kind, never against a kind that refutes the receipt's phase; - the referee maps warehouse error classes to the phase that raised them, records runs sequentially and compares two runs batch by batch.
…nd cases The Databricks coverage root gains its forms from the pinned documentation pages, each form's obligations and hand-written limits, retirements for forms the pages no longer document, aliases, and the screen allowlist, whose entries cover only respellings of one token and comment text.
The pinned sqlquack-evidence commit now holds the Databricks 2026.45 cases and receipts the coverage root reads.
… screen Adds a review record for every Databricks form read so far, each with its own note: approvals bound to the form's fingerprint and definition, and changes_requested records that withdraw a form where the screen or a reader found a collapsed spelling, each naming the pair that showed it. One exclusion keeps object-type, an output-only type, out of the count. check-ci runs dialect-validate, so a change that makes an approved form's tree lose a spelling fails the gate. 1062 of 1219 Databricks forms are verified, 1 is excluded and 13 are retired.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
SQLQuack claims Databricks SQL syntax coverage only where a form's cases parse as the Databricks 2026.45 warehouse does and each accepted case keeps what was written in its tree. This brings Databricks to that standard: documented forms, obligations, cases and receipts, parser fixes for what the receipts and the tree screen found, and per-form reviews.
Changes
=, commas, parentheses, quoting) kept in the tree.sqlquack-syntax: a call's set quantifier replaces itsdistinctflag (breaking for tree JSON readers); optional fields that serialize only when set; scripting types in their own domain.Verification
make check-cion the final head.