docs(firestore-bigquery-export): document failure handling and recovery - #3131
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the documentation in README.md to reflect the introduction of the syncBigQuery Cloud Tasks queue for buffering and retrying failed BigQuery writes. It details the new configuration parameters, IAM roles, failure handling behavior, and API surface changes. The feedback suggests a minor wording improvement in the failure handling section to clarify that a row is dropped along with the task if no backup collection is configured.
4062cf2 to
3912c71
Compare
3912c71 to
fa2b405
Compare
fa2b405 to
78d2e27
Compare
78d2e27 to
b435f88
Compare
78d2e27 to
e5ebae4
Compare
7488874 to
c489ac1
Compare
c489ac1 to
eed63b8
Compare
eed63b8 to
bc6874a
Compare
IzaakGough
left a comment
There was a problem hiding this comment.
Docs only, and I checked the claims against the code on the base branch plus the published @firebaseextensions/firestore-bigquery-change-tracker@2.2.1, firebase-admin@14.2.0 and firebase-tools. The role, both new params, the queue shape (5 attempts, 60s backoff, dispatch throttling), the trigger having no retry policy, enqueue dedupe by task id, onSuccess coming only from the queue handler in both kit and extension, the queue handler not provisioning, and the older-CLI enqueue failure all check out.
Five comments inline. All of it is from reading source, nothing deployed, and I have said where a claim rests on documented behaviour rather than something I exercised.
Adds the failure handling section: the sync queue, recovering parked rows from BACKUP_COLLECTION, known limits, and the new params.
bc6874a to
08b6524
Compare
…er (#3130) Fourth layer of the Cloud Tasks write buffer stack and the behaviour change, per the option D decision on #3031: migrating users keep the failure behaviour and cost profile they have today. Supersedes #3103. A failed inline write now enqueues onto a new `syncBigQuery` task queue (5 attempts, 60s minimum backoff, throttled by `MAX_DISPATCHES_PER_SECOND`) after up to `MAX_ENQUEUE_ATTEMPTS` in-process attempts, instead of self-healing once and rethrowing to the trigger retry policy. The task handler re-attempts the write and rethrows so Cloud Tasks retries; the tracker parks rows whose insert is rejected in `BACKUP_COLLECTION`, on the inline attempt and on each queue attempt. A change that cannot be enqueued at all is logged first, then published as `onError`, and dropped, as the extension does; the log-before-publish order is deliberate so the only trace of a dropped row survives a failing events channel. The trigger drops its retry policy. The function needs `roles/cloudtasks.enqueuer`. The queue function sets `maxInstances` to 500 to match the gen1 handler, since the gen2 default of 100 would 429 dispatches and burn attempts. The queue's `rateLimits.maxDispatchesPerSecond` is the CEL ternary `params.MAX_DISPATCHES_PER_SECOND < 1 ? 100 : params.MAX_DISPATCHES_PER_SECOND`: a blank `.env` value reaches deploy-time CEL as 0, and the ternary restores the same default the runtime falls back to. `REQUIRED_APIS` keeps `firestore.googleapis.com` from #3119, now pinned by `tests/required-apis.test.ts`. One deliberate change from the extension: the success event is published after the insert and swallowed on failure, closing a duplicate-row route. The README rewrite for the new export and failure path is the next layer, #3131. 96 tests pass on this layer. Verified live on a test project with Firebase CLI 15.29.0 from the top of the stack: the deployed env carries `FIREBASE_KIT_INSTANCE_ID` and `FUNCTION_REGION`, the gen2 task function accepted the SDK's ID token, the queue was created with the resolved throttle, and with the changelog table removed the trigger enqueued, the tracker wrote backup documents keyed by event id, and the queue drained without duplicate rows once the table was re-provisioned. Not exercised live: the Cloud Tasks task size limit. A change near Firestore's 1 MB document cap, doubled by `oldData` on an update, exceeds the task body limit, so the enqueue fails and the row is logged and dropped. This is parity: the extension enqueues the same payload through the same path, and `EXCLUDE_OLD_DATA` is its mitigation. Listed under known limits in the README in #3131. Fixes #3031.
Last layer of the Cloud Tasks write buffer stack. README only.
Adds a "Failure handling" section describing the sync queue, how to recover rows parked in
BACKUP_COLLECTIONback into BigQuery, and the known limits that have not been verified live. Adds the two new params to the settings table,syncBigQueryto the list of functions to re-export,roles/cloudtasks.enqueuerto the roles list, and updates the differences section so failed writes are described as the same buffer the extension has rather than a trigger retry.Part of #3031.