Error highlighting in query strings

[renzobautista]

Hi all, I'm currently using DataFusion to generate logical plans from a user-inputted SQL query. I was wondering if it's possible to analyze a SQL query string to identify errors and their corresponding locations (line number/column). Using SessionContext::sql_with_options, I am able to get a single error, but it doesn't include location within the query string.

Ideally, I'd like to be able to get all errors and locations, for the purposes of highlighting in a text editor -- both syntactic and plan/schema errors.

[Jefffrey]

Probably more appropriate for the sqlparser crate, see repo: https://github.com/apache/datafusion-sqlparser-rs/

[ryux1]

This is now partly supported in current DataFusion.

Enable datafusion.sql_parser.collect_spans = true on the session configuration (it defaults to false). During SQL-to-logical-plan conversion, identifiers and other supported AST nodes retain Span { start: Location { line, column }, end: ... }. Planning failures can carry a Diagnostic with a primary span plus optional note/help spans; call DataFusionError::diagnostic() for the outer diagnostic.

For multiple planning errors, DataFusion now has DataFusionError::Collection, and DataFusionError::iter() flattens the collected errors. The planner uses this in recoverable areas such as separate SELECT expressions and UNION branches, so an editor can iterate the errors and then inspect error.diagnostic().and_then(|d| d.span) for each one. Non-fatal planner warnings are also represented as Diagnostic values; the lower-level SqlToRel::take_warnings() API drains them.

The tests in datafusion/sql/tests/cases/diagnostic.rs are useful executable examples: they build SqlToRel with ParserOptions { collect_spans: true, .. } and assert spans for missing tables/columns and other plan errors.

The important boundary is syntax recovery. The SQL parser reports token locations for a parse failure, but it does not currently recover through an invalid statement to return every independent syntax error in one pass. Also, not every planner error has been annotated with a diagnostic span yet, so diagnostic() remains optional. For editor integration, the practical split is therefore:

1. parse and surface the parser location if parsing fails;
2. if parsing succeeds, plan with span collection enabled;
3. iterate a DataFusionError::Collection and render each available Diagnostic span, while retaining a fallback for unspanned errors.

That provides multi-error highlighting for the planning/schema phase today, but full multi-error syntax diagnostics would still require recovery support in datafusion-sqlparser-rs.