Repository navigation
feat(django-google-spanner): support DDL batching via ddl_batch(), START BATCH DDL, and opt-in AUTO_BATCH_DDL - #19037
sakthivelmanii wants to merge 1 commit into
Conversation
…ART BATCH DDL, and opt-in AUTO_BATCH_DDL
There was a problem hiding this comment.
Code Review
This pull request introduces automatic and programmatic DDL batching for the Django Cloud Spanner backend, allowing multiple schema changes to be executed in a single RPC. It adds a DDL-batch-aware cursor proxy, wrapper-level batch lifecycle management, and safe literal quoting utilities. The review feedback recommends executing buffered DDL statements individually instead of joining them with semicolons to avoid parsing fragility, and stripping trailing semicolons in prepare_sql_script to ensure compatibility with Cloud Spanner.
| self.connection.begin() | ||
| self._ddl_tx_begun = True | ||
| try: | ||
| raw_cursor.execute("; ".join(statements)) |
There was a problem hiding this comment.
Instead of joining the statements with "; " and passing them to raw_cursor.execute(), it is safer and more robust to execute each statement individually in a loop. Joining them with semicolons forces the underlying spanner_dbapi to parse and split them again, which can be fragile and fail if any statement contains a semicolon inside a string literal or comment.
| raw_cursor.execute("; ".join(statements)) | |
| for stmt in statements: | |
| raw_cursor.execute(stmt) |
| if formatted and formatted.rstrip(";").strip(): | ||
| statements.append(formatted) |
There was a problem hiding this comment.
To ensure compatibility with Cloud Spanner (which does not allow trailing semicolons in DDL/DML statements executed via the API), it is safer to strip the trailing semicolons from the returned statements in prepare_sql_script.
| if formatted and formatted.rstrip(";").strip(): | |
| statements.append(formatted) | |
| stmt = formatted.rstrip(";").strip() | |
| if stmt: | |
| statements.append(stmt) |
Summary
django-google-spannerreusingspanner_dbapiclient-transaction DDL queue (Connection.begin()/commit()/rollback()+_ddl_statements+run_prior_DDL_statements()) scoped strictly around DDL bursts while keeping all interleaved DQL, DML backfills, snapshot reads, andRunPythontransactions in autocommit mode.with connection.ddl_batch(auto_flush=True):(start_batch_ddl(),flush_ddl_batch(),run_batch(),abort_batch()) and SQLSTART BATCH DDL/RUN BATCH/ABORT BATCHcursor interception out of the box.AUTO_BATCH_DDL(DATABASES[alias]/OPTIONS, defaulting toFalse) soDatabaseSchemaEditorbatches migration DDL statements automatically when enabled and flushes beforeMigrationRecorderinserts intodjango_migrations.quote_ddl_literal(django_spanner/utils.py) for GoogleSQL string/control-character escaping,UUIDhex literals,JsonObject,DateStr/TimestampStr,USE_TZ-aware UTCdatetimenormalization, and finite numeric validation acrossquote_value,column_sql, and parameterized DDL.DatabaseOperations.prepare_sql_script(django_spanner/operations.py) to strip comment-only and bare-semicolon fragments fromRunSQLscripts.README.rstanddocs/limitations-spanner.rstwith configuration, usage examples, and operational caveats.Behavior Changes
DatabaseSchemaEditor.quote_valueandcolumn_sqlnow format string literals using GoogleSQL backslash escapes and quoteUUIDvalues as hex strings instead of SQL doubled single quotes.DatabaseOperations.prepare_sql_scriptstrips SQL comments and drops empty or bare-;fragments before returning statements.DatabaseWrapper.init_connection_statepreserves theread_onlyconnection option when re-initializing the underlyingspanner_dbapi.Connection.Verification
176 passed).nox -s lint(ruff==0.14.14,flake8, and strictdocutilsRST validation).