Skip to content

feat(bigquery): add bigquery_query_manifest_external_table sample to … - #14715

Merged
iennae merged 1 commit into
mainfrom
feat/bq-query-manifest-external-table
Nov 3, 2026
Merged

iennae merged 1 commit into
mainfrom
feat/bq-query-manifest-external-table

Conversation

@iennae

@iennae iennae commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

…query an external table defined by a manifest file

Problem:
The BigQuery documentation (https://cloud.google.com/bigquery/docs/query-open-table-format-using-manifest-files) needs a standalone, tested Python code sample demonstrating how to query an external table defined by a manifest file (region tag bigquery_query_manifest_external_table).

Solution:

  • Add query_manifest_external_table(project_id: str, dataset_id: str, table_id: str, manifest_uri: str) -> None in bigquery/biglake/bigquery_query_manifest_external_table.py (Run a GoogleSQL query against an open-table-format external table backed by a manifest file on Cloud Storage.)
  • Add unit tests (bigquery/biglake/bigquery_query_manifest_external_table_test.py) covering both the success path and API error handling (GoogleAPICallError)

Checklist

Testing

  • I have tested this change on a live environment and verified it works as intended.

Compliance & Style


Post-Approval Actions

  • Please merge this PR for me once it is approved

@iennae
iennae requested review from a team and chalmerlowe as code owners November 1, 2026 20:21
@snippet-bot

snippet-bot Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Here is the summary of changes.

You are about to add 1 region tag.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@product-auto-label product-auto-label Bot added api: bigquery Issues related to the BigQuery API. samples Issues that are directly related to samples. labels Nov 1, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new Python sample, query_manifest_external_table, along with its corresponding unit tests and configuration files. The sample demonstrates how to configure a newline-delimited manifest on a BigQuery external table and query it. The review feedback points out that the test configuration in noxfile_config.py should be updated to test only the oldest (3.10) and newest (3.13) supported Python versions, meaning intermediate versions (3.11 and 3.12) should be ignored instead of 3.10.

Comment thread bigquery/biglake/noxfile_config.py Resolved

@XrossFox XrossFox left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, some dependencies are outdated.

Comment thread bigquery/biglake/requirements.txt Outdated
Comment on lines +1 to +2
google-cloud-bigquery==3.35.1
google-cloud-bigquery-storage==2.32.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bigquery and bigquery storage should be upgraded, to 3.47.0 and 3.17.1 versions respectively.

@iennae
iennae force-pushed the feat/bq-query-manifest-external-table branch 2 times, most recently from 79568e2 to b585eba Compare November 2, 2026 22:35
…query an external table defined by a manifest file

Problem:
The BigQuery documentation (https://cloud.google.com/bigquery/docs/query-open-table-format-using-manifest-files)
needs a standalone, tested Python code sample demonstrating how to
query an external table defined by a manifest file (region tag `bigquery_query_manifest_external_table`).

Solution:
- Add `query_manifest_external_table(project_id: str, dataset_id: str, table_id: str, manifest_uri: str) -> None` in `bigquery/biglake/bigquery_query_manifest_external_table.py`
  (Run a GoogleSQL query against an open-table-format external table backed by a manifest file on Cloud Storage.)
- Add unit tests (`bigquery/biglake/bigquery_query_manifest_external_table_test.py`) covering both the success
  path and API error handling (`GoogleAPICallError`)
@iennae
iennae force-pushed the feat/bq-query-manifest-external-table branch from b585eba to 4031908 Compare November 2, 2026 23:33
@iennae
iennae merged commit 49d301c into main Nov 3, 2026
13 of 21 checks passed
@iennae
iennae deleted the feat/bq-query-manifest-external-table branch November 3, 2026 02:23
iennae added a commit that referenced this pull request Nov 3, 2026
… up test mocks

Problem:
Following review feedback across the BigQuery sample PRs (#14714, #14715,
#14717):
- `bigquery_create_search_index.py` should use `CREATE SEARCH INDEX IF NOT EXISTS`
  and handle `exceptions.NotFound` when the parent table or dataset is missing.
- `bigquery_drop_vector_index.py` `NotFound` error message should reflect that
  `DROP VECTOR INDEX IF EXISTS` raises `NotFound` when the parent table or
  dataset is missing.
- Unit tests in `bigquery_create_search_index_test.py`,
  `bigquery_drop_vector_index_test.py`, and
  `bigquery_query_manifest_external_table_test.py` defined unused mock
  attributes (`mock_row` on DDL statements and `mock_job.statement_type` on
  SELECT queries).

Solution:
- Restore `CREATE SEARCH INDEX IF NOT EXISTS` and catch `exceptions.NotFound`
  in `bigquery/search/bigquery_create_search_index.py` and its unit test.
- Update the `NotFound` error message in
  `bigquery/vector_search/bigquery_drop_vector_index.py`.
- Remove unused `mock_row` and `mock_job.statement_type` setup in the three
  corresponding unit test files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery Issues related to the BigQuery API. samples Issues that are directly related to samples.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants