Skip to content

docs(firestore-bigquery-export): document failure handling and recovery - #3131

Merged
cabljac merged 1 commit into
feat/kits-fbe-cloud-tasks-write-bufferfrom
docs/kits-fbe-failure-handling
Sep 8, 2026
Merged

docs(firestore-bigquery-export): document failure handling and recovery#3131
cabljac merged 1 commit into
feat/kits-fbe-cloud-tasks-write-bufferfrom
docs/kits-fbe-failure-handling

Conversation

@cabljac

@cabljac cabljac commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Last layer of the Cloud Tasks write buffer stack. README only.

Adds a "Failure handling" section describing the sync queue, how to recover rows parked in BACKUP_COLLECTION back into BigQuery, and the known limits that have not been verified live. Adds the two new params to the settings table, syncBigQuery to the list of functions to re-export, roles/cloudtasks.enqueuer to the roles list, and updates the differences section so failed writes are described as the same buffer the extension has rather than a trigger retry.

Part of #3031.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the documentation in README.md to reflect the introduction of the syncBigQuery Cloud Tasks queue for buffering and retrying failed BigQuery writes. It details the new configuration parameters, IAM roles, failure handling behavior, and API surface changes. The feedback suggests a minor wording improvement in the failure handling section to clarify that a row is dropped along with the task if no backup collection is configured.

Comment thread kits/firestore-bigquery-export/README.md Outdated
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch from 4062cf2 to 3912c71 Compare September 7, 2026 16:45
@cabljac
cabljac marked this pull request as ready for review September 7, 2026 16:50
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch from 3912c71 to fa2b405 Compare September 7, 2026 16:52
@cabljac cabljac mentioned this pull request Sep 7, 2026
66 tasks
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch from fa2b405 to 78d2e27 Compare September 7, 2026 17:26
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch from 78d2e27 to b435f88 Compare September 8, 2026 09:06
@cabljac
cabljac requested a review from a team as a code owner September 8, 2026 10:03
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch 2 times, most recently from 78d2e27 to e5ebae4 Compare September 8, 2026 10:25
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch 2 times, most recently from 7488874 to c489ac1 Compare September 8, 2026 10:49
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch from c489ac1 to eed63b8 Compare September 8, 2026 10:52
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch from eed63b8 to bc6874a Compare September 8, 2026 10:55

@IzaakGough IzaakGough left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs only, and I checked the claims against the code on the base branch plus the published @firebaseextensions/firestore-bigquery-change-tracker@2.2.1, firebase-admin@14.2.0 and firebase-tools. The role, both new params, the queue shape (5 attempts, 60s backoff, dispatch throttling), the trigger having no retry policy, enqueue dedupe by task id, onSuccess coming only from the queue handler in both kit and extension, the queue handler not provisioning, and the older-CLI enqueue failure all check out.

Five comments inline. All of it is from reading source, nothing deployed, and I have said where a claim rests on documented behaviour rather than something I exercised.

Comment thread kits/firestore-bigquery-export/README.md
Comment thread kits/firestore-bigquery-export/README.md Outdated
Comment thread kits/firestore-bigquery-export/README.md Outdated
Comment thread kits/firestore-bigquery-export/README.md Outdated
Comment thread kits/firestore-bigquery-export/README.md Outdated
Adds the failure handling section: the sync queue, recovering parked rows
from BACKUP_COLLECTION, known limits, and the new params.
@cabljac
cabljac force-pushed the docs/kits-fbe-failure-handling branch from bc6874a to 08b6524 Compare September 8, 2026 11:06
@cabljac
cabljac merged commit 9307079 into kits Sep 8, 2026
16 checks passed
cabljac added a commit that referenced this pull request Sep 8, 2026
…er (#3130)

Fourth layer of the Cloud Tasks write buffer stack and the behaviour
change, per the option D decision on #3031: migrating users keep the
failure behaviour and cost profile they have today. Supersedes #3103.

A failed inline write now enqueues onto a new `syncBigQuery` task queue
(5 attempts, 60s minimum backoff, throttled by
`MAX_DISPATCHES_PER_SECOND`) after up to `MAX_ENQUEUE_ATTEMPTS`
in-process attempts, instead of self-healing once and rethrowing to the
trigger retry policy. The task handler re-attempts the write and
rethrows so Cloud Tasks retries; the tracker parks rows whose insert is
rejected in `BACKUP_COLLECTION`, on the inline attempt and on each queue
attempt. A change that cannot be enqueued at all is logged first, then
published as `onError`, and dropped, as the extension does; the
log-before-publish order is deliberate so the only trace of a dropped
row survives a failing events channel. The trigger drops its retry
policy. The function needs `roles/cloudtasks.enqueuer`. The queue
function sets `maxInstances` to 500 to match the gen1 handler, since the
gen2 default of 100 would 429 dispatches and burn attempts. The queue's
`rateLimits.maxDispatchesPerSecond` is the CEL ternary
`params.MAX_DISPATCHES_PER_SECOND < 1 ? 100 :
params.MAX_DISPATCHES_PER_SECOND`: a blank `.env` value reaches
deploy-time CEL as 0, and the ternary restores the same default the
runtime falls back to. `REQUIRED_APIS` keeps `firestore.googleapis.com`
from #3119, now pinned by `tests/required-apis.test.ts`.

One deliberate change from the extension: the success event is published
after the insert and swallowed on failure, closing a duplicate-row
route. The README rewrite for the new export and failure path is the
next layer, #3131.

96 tests pass on this layer. Verified live on a test project with
Firebase CLI 15.29.0 from the top of the stack: the deployed env carries
`FIREBASE_KIT_INSTANCE_ID` and `FUNCTION_REGION`, the gen2 task function
accepted the SDK's ID token, the queue was created with the resolved
throttle, and with the changelog table removed the trigger enqueued, the
tracker wrote backup documents keyed by event id, and the queue drained
without duplicate rows once the table was re-provisioned. Not exercised
live: the Cloud Tasks task size limit. A change near Firestore's 1 MB
document cap, doubled by `oldData` on an update, exceeds the task body
limit, so the enqueue fails and the row is logged and dropped. This is
parity: the extension enqueues the same payload through the same path,
and `EXCLUDE_OLD_DATA` is its mitigation. Listed under known limits in
the README in #3131.

Fixes #3031.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants