Skip to content

feat(django-google-spanner): support DDL batching via ddl_batch(), START BATCH DDL, and opt-in AUTO_BATCH_DDL - #19037

Closed
sakthivelmanii wants to merge 1 commit into
mainfrom
feat/django-spanner-ddl-batching
Closed

sakthivelmanii wants to merge 1 commit into
mainfrom
feat/django-spanner-ddl-batching

Conversation

@sakthivelmanii

Copy link
Copy Markdown
Contributor

Summary

  • Add connection-scoped DDL batching in django-google-spanner reusing spanner_dbapi client-transaction DDL queue (Connection.begin() / commit() / rollback() + _ddl_statements + run_prior_DDL_statements()) scoped strictly around DDL bursts while keeping all interleaved DQL, DML backfills, snapshot reads, and RunPython transactions in autocommit mode.
  • Expose programmatic with connection.ddl_batch(auto_flush=True): (start_batch_ddl(), flush_ddl_batch(), run_batch(), abort_batch()) and SQL START BATCH DDL / RUN BATCH / ABORT BATCH cursor interception out of the box.
  • Add opt-in AUTO_BATCH_DDL (DATABASES[alias] / OPTIONS, defaulting to False) so DatabaseSchemaEditor batches migration DDL statements automatically when enabled and flushes before MigrationRecorder inserts into django_migrations.
  • Add quote_ddl_literal (django_spanner/utils.py) for GoogleSQL string/control-character escaping, UUID hex literals, JsonObject, DateStr/TimestampStr, USE_TZ-aware UTC datetime normalization, and finite numeric validation across quote_value, column_sql, and parameterized DDL.
  • Override DatabaseOperations.prepare_sql_script (django_spanner/operations.py) to strip comment-only and bare-semicolon fragments from RunSQL scripts.
  • Update README.rst and docs/limitations-spanner.rst with configuration, usage examples, and operational caveats.

Behavior Changes

  • DatabaseSchemaEditor.quote_value and column_sql now format string literals using GoogleSQL backslash escapes and quote UUID values as hex strings instead of SQL doubled single quotes.
  • DatabaseOperations.prepare_sql_script strips SQL comments and drops empty or bare-; fragments before returning statements.
  • DatabaseWrapper.init_connection_state preserves the read_only connection option when re-initializing the underlying spanner_dbapi.Connection.

Verification

  • Unit & MockServer tests on Django 5.2 and Django 6.0 (176 passed).
  • nox -s lint (ruff==0.14.14, flake8, and strict docutils RST validation).

@sakthivelmanii
sakthivelmanii requested a review from a team as a code owner November 10, 2026 14:39

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces automatic and programmatic DDL batching for the Django Cloud Spanner backend, allowing multiple schema changes to be executed in a single RPC. It adds a DDL-batch-aware cursor proxy, wrapper-level batch lifecycle management, and safe literal quoting utilities. The review feedback recommends executing buffered DDL statements individually instead of joining them with semicolons to avoid parsing fragility, and stripping trailing semicolons in prepare_sql_script to ensure compatibility with Cloud Spanner.

self.connection.begin()
self._ddl_tx_begun = True
try:
raw_cursor.execute("; ".join(statements))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Instead of joining the statements with "; " and passing them to raw_cursor.execute(), it is safer and more robust to execute each statement individually in a loop. Joining them with semicolons forces the underlying spanner_dbapi to parse and split them again, which can be fragile and fail if any statement contains a semicolon inside a string literal or comment.

Suggested change
raw_cursor.execute("; ".join(statements))
for stmt in statements:
raw_cursor.execute(stmt)

Comment on lines +124 to +125
if formatted and formatted.rstrip(";").strip():
statements.append(formatted)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure compatibility with Cloud Spanner (which does not allow trailing semicolons in DDL/DML statements executed via the API), it is safer to strip the trailing semicolons from the returned statements in prepare_sql_script.

Suggested change
if formatted and formatted.rstrip(";").strip():
statements.append(formatted)
stmt = formatted.rstrip(";").strip()
if stmt:
statements.append(stmt)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant