fix(core,adapters): improve driver lifecycle, query compilation, and adapter consistency

[cofin]

Summary

Improves statement caching, driver execution lifecycles, dialect transpilation, schema introspection, storage backends, and adapter consistency across core modules, framework extensions, and database adapters.

Key Changes

Core Modules

sqlspec (Root)

• Refactoring & Typing:
  • Use PEP 562 lazy module exports (__getattr__ and __dir__) across top-level package barrels to keep import times minimal.
  • Standardize module __all__ exports and class member ordering.

sqlspec.core

• Fixes:
  • Include pre-parsed expression identity in SQLProcessor compilation cache keys.
  • Copy cached AST nodes on parse-cache lookup so statement transformers do not mutate shared cache entries.
  • Preserve positional and named parameter ordering in ParameterConverter.
• Enhancements:
  • Use OrderedDict LRU eviction in statement and compilation caches.

sqlspec.driver

• Fixes:
  • Clone SlotPooledStatement instances when attaching to SQLResult.statement before returning pooled statements to _STATEMENT_POOL.
  • Restore transaction and savepoint state cleanly when rollback operations fail.
  • Fix select_with_total window-count extraction and cursor row-mapping dispatch.

sqlspec.builder

• Fixes:
  • Route QueryBuilder.to_statement() through self.build() so optimizer settings apply consistently.
  • Synthesize single-table MappingSchema instances when schema=None for compatibility with sqlglot >= 30.22.0 column qualification.
  • Restrict MERGE string column reference detection to qualified table/source aliases.
  • Render BigQuery PARTITION BY clauses accurately in CreateTable.partition_by().

sqlspec.dialects

• Fixes:
  • Scope Spanner, Spangres, DB2, PGVector, and ParadeDB parser and generator transforms to their respective dialects and refresh sqlglot[c] generator dispatch tables.

sqlspec.data_dictionary

• Fixes:
  • Include self-referential root tables in tables.sql topological dependency queries across all dialects.
  • Join composite foreign key columns on ordinal_position to prevent duplicate column pairs.

sqlspec.migrations

• Fixes:
  • Write squashed migration files before removing source migration files in MigrationSquasher.
  • Validate target revision existence in stamp and avoid redundant commits on autocommit drivers in AsyncMigrationTracker.
• Enhancements:
  • Support reversible downgrade generation in squashed .py and .sql migrations.

sqlspec.storage & sqlspec.utils

• Fixes:
  • Normalize Windows file:///D:/... URIs across LocalStore, ObStoreBackend, and FSSpecBackend.
  • Use a sentinel value instead of raising StopIteration across worker thread futures in AsyncThreadedBytesIterator.
  • Add thread-reentrancy checks and timeout cancellation cleanup in PortalProvider.
• Enhancements:
  • Enforce OrderedDict LRU eviction in StorageRegistry.

sqlspec.service & sqlspec.observability

• Fixes:
  • Support re-entrant begin_transaction() calls with nested savepoints in SQLSpecSyncService and SQLSpecAsyncService.
  • Reject conflicting LimitOffsetFilter and CursorFilter arguments in paginate() and skip redundant count queries on short first pages.
  • Extend RedactionConfig parameter masking to scalar and sequence parameters and skip SQL comments during literal masking.
  • Isolate telemetry provider exceptions in SpanManager and use bounded deque buffers for duration percentile calculations.

sqlspec.extensions

• Fixes:
  • Fix async execution in Litestar CLI show-current-revision and session store key cleanup.
  • Fix ADK session and memory store pagination and parameter binding.
  • Improve Events queue lease/retry handling and listener hub shutdown cleanup.
  • Ensure request-scoped session teardown across FastAPI, Flask, Starlette, and Sanic plugins.

---

Database Adapters

PostgreSQL & CockroachDB (asyncpg, psycopg, psqlpy, cockroach_asyncpg, cockroach_psycopg)

• Fixes:
  • Enter self.connection.transaction(readonly=True) before SET TRANSACTION AS OF SYSTEM TIME for follower reads in cockroach_asyncpg.
  • Coerce binary bytes payloads for psqlpy Litestar session stores and release pooled connections if initial setup fails.
  • Preserve psycopg null-pool sizing and asyncpg per-query timeouts and pgbouncer_mode settings.

SQLite (sqlite, aiosqlite)

• Fixes:
  • Use identity checks (autocommit is True) for Python 3.12+ PEP 249 autocommit detection.
  • Inspect connection transaction state on the worker thread in aiosqlite for Python 3.14 thread-check compatibility.
  • Roll back adapter-owned transactions on batch failure or cancellation in load_from_arrow.

MySQL (asyncmy, aiomysql, pymysql, mysqlconnector)

• Fixes:
  • Route SHOW and DESCRIBE statements through row-returning execution across all MySQL adapters.
  • Terminate hung aiomysql and asyncmy pool connections on close timeout and roll back dirty transactions on PyMySQLConnectionPool release.

SQL Server (mssql_python, pymssql)

• Fixes:
  • Stream Arrow batches in mssql_python load_from_arrow without materializing full tables in memory.
  • Preserve default-schema restoration state if cleanup fails during SQL Server migrations.

Cloud, Enterprise & Analytics (spanner, bigquery, oracledb, db2, duckdb, adbc, arrow_odbc)

• Fixes:
  • Unwrap Aborted exceptions inside run_in_transaction retries, clear mutation buffers on rollback, and await two-stage async update_ddl operations in spanner.
  • Finalize BigQuery Storage Write streams in try / finally blocks and route interactive queries via query_and_wait.
  • Keep Oracle input/output type handler registration idempotent on pooled connections.
  • Retain cursors on Db2 savepoint rollback and manage thread-local pooled connections in db2.
  • Restore UUID column types across cache hits and streams in duckdb, pass Arrow streams directly to adbc_ingest in adbc, and tokenize T-SQL pagination parameters lexically in arrow_odbc.
• Enhancements:
  • Complete driver_features type-system coverage across all 20 adapters and expand MyPyC compilation to adapter core, type_converter, pool, and _typing modules.

Verification

• uv run pytest tests/unit -q
• uv run ruff check sqlspec tests & uv run ruff format --check sqlspec tests
• uv run mypy sqlspec & uv run pyright sqlspec
• uv run slotscheck sqlspec
• uv run python tools/scripts/mypyc_inventory.py --check & uv run python tools/scripts/mypyc_smoke.py
• uv run sphinx-build -M html docs docs/_build/ -j auto -W --keep-going

[codecov-commenter]

Codecov Report

❌ Patch coverage is 75.46012% with 1040 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.40%. Comparing base (4a57235) to head (7e34399).

Files with missing lines	Patch %	Lines
sqlspec/adapters/aiosqlite/pool.py	60.67%	93 Missing and 12 partials ⚠️
sqlspec/adapters/bigquery/core.py	54.49%	47 Missing and 34 partials ⚠️
sqlspec/adapters/adbc/core.py	58.27%	36 Missing and 22 partials ⚠️
sqlspec/adapters/bigquery/driver.py	66.66%	36 Missing and 20 partials ⚠️
sqlspec/adapters/adbc/adk/store.py	62.90%	33 Missing and 13 partials ⚠️
sqlspec/adapters/aiosqlite/adk/store.py	69.93%	39 Missing and 4 partials ⚠️
sqlspec/adapters/aiomysql/adk/store.py	57.64%	30 Missing and 6 partials ⚠️
sqlspec/adapters/aiosqlite/core.py	66.97%	19 Missing and 17 partials ⚠️
sqlspec/adapters/bigquery/config.py	59.03%	28 Missing and 6 partials ⚠️
sqlspec/adapters/asyncmy/adk/store.py	69.00%	27 Missing and 4 partials ⚠️
... and 60 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #837      +/-   ##
==========================================
- Coverage   81.71%   81.40%   -0.32%     
==========================================
  Files         522      527       +5     
  Lines       80378    85274    +4896     
  Branches    11626    15789    +4163     
==========================================
+ Hits        65684    69420    +3736     
- Misses      10929    11731     +802     
- Partials     3765     4123     +358     

Flag	Coverage Δ
integration	61.09% <50.49%> (-1.28%)	⬇️
py3.11	79.78% <73.99%> (-0.31%)	⬇️
py3.12	79.78% <74.04%> (+7.63%)	⬆️
py3.13	79.79% <74.04%> (-0.30%)	⬇️
py3.14	80.79% <75.11%> (-0.30%)	⬇️
unit	73.03% <63.23%> (+0.35%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sqlspec/__init__.py	82.75% <ø> (ø)
sqlspec/adapters/__init__.py	100.00% <100.00%> (ø)
sqlspec/adapters/adbc/__init__.py	100.00% <ø> (ø)
sqlspec/adapters/adbc/_typing.py	85.45% <100.00%> (-5.62%)	⬇️
sqlspec/adapters/adbc/data_dictionary.py	70.64% <100.00%> (+0.09%)	⬆️
sqlspec/adapters/adbc/type_converter.py	100.00% <100.00%> (+22.22%)	⬆️
sqlspec/adapters/aiomysql/__init__.py	100.00% <ø> (ø)
sqlspec/adapters/aiomysql/_typing.py	95.31% <100.00%> (+0.48%)	⬆️
sqlspec/adapters/aiomysql/core.py	95.77% <100.00%> (+0.12%)	⬆️
sqlspec/adapters/aiosqlite/__init__.py	100.00% <ø> (ø)
... and 257 more

... and 73 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.