Repository navigation
Add postgres_data_apis bundle resource - #7160
Open
ruslantalpa wants to merge 12 commits into
Open
ruslantalpa wants to merge 12 commits into
ruslantalpa wants to merge 12 commits into
Conversation
ruslantalpa
added a commit
that referenced
this pull request
Oct 22, 2026
Co-authored-by: Isaac <no-reply@databricks.com>
Add the `postgres_data_apis` resource, which enables and configures the
Data API (PostgREST) of a Lakebase Postgres database.
The Data API is a per-database singleton with no ID of its own: its
resource name is "{parent}/data-api", and `parent` is a provided id
field, so changing it recreates. Create, update, and delete are
long-running operations. Updates send the full spec with
update_mask=spec, so a field removed from the config resets to its
server default.
spec is INPUT_ONLY in the API, so GET does not return it and remote
spec changes are not detected (ignore_remote_changes spec:input_only,
generated). Deleting the resource turns the Data API off; the database
and its data are unaffected.
The resource is labelled Private Preview, matching the API's launch
stage, rather than the Beta its postgres_* siblings are pinned to.
Co-authored-by: Isaac <no-reply@databricks.com>
Observed against a real workspace: - status reports the configuration in effect, server defaults included: db_aggregates_enabled=true, db_schemas=["public"], jwt_role_claim_key=".sub", openapi_mode=OPEN_API_MODE_DISABLED and server_cors_allowed_origins=["none"]. A bool that was never set and is false is omitted; once set it is reported, false included. - update_mask=spec keeps the current value of a field the request omits; it does not reset it to the default. - Errors: 400 INVALID_PARAMETER_VALUE "Field 'data_api' is required and must contain at least one subfield with a non-default value!" for an empty spec, 409 ALREADY_EXISTS "Data API is already enabled for this database", and 404 NOT_FOUND "Data API is not enabled for this database" for get, update and delete when it is not enabled. The acceptance tests now also print server_cors_allowed_origins, and the update test shows that a field removed from the config keeps its value. Co-authored-by: Isaac <no-reply@databricks.com>
GET does not return spec (it is INPUT_ONLY), so the plan could not see a remote spec change, and a bound Data API kept its old configuration until the bundle config changed. status reports every spec field in effect, so DoRead now fills the remote spec from it. The generated spec:input_only rules no longer apply, so apitypes.yml drops the type for postgres_data_apis and the field rules are hand-written: every spec field is a backend_default with no allowed values. A field the config omits is left to the server: create applies the server default, and update_mask=spec keeps the current value of an omitted field, so a field removed from the config keeps its last value. Leaf update masks would reset it, but resetting db_max_rows or jwt_role_claim_key that way leaves the Data API unable to serve requests, so the mask stays at spec. Status omits a false bool that was never set, so the remote spec forces db_aggregates_enabled and server_timing_enabled: an explicit false in the config compares against false, not against an unknown value. With this, binding an existing Data API plans an update that applies the configured spec, and an out-of-band spec change shows up in the plan. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
The real API reports the serving URL in status.url:
https://{read-write endpoint host}/api/2.0/workspace/{workspace_id}/rest/{postgres_database}.
The fake now fills it in GET and in the create and update operation
responses, from the branch's read-write endpoint host, the fake
workspace ID and the database's Postgres name.
The new url_reference acceptance test has a job parameter that references
${resources.postgres_data_apis.<key>.status.url} and shows it resolves on
the first deploy. It also records that an in-place update of the Data API
currently plans an update of the job too.
Co-authored-by: Isaac <no-reply@databricks.com>
status.url is built from the read-write endpoint host, the workspace and
the database's Postgres name. An update of the Data API changes none of
them, but without this the reference resolver delays any
${resources.postgres_data_apis.<key>.status.url} reference during an
in-place update, so every dependent got a no-op update planned too.
The url_reference acceptance test now shows that changing db_max_rows
plans an update of the Data API only, and the next plan is clean.
Co-authored-by: Isaac <no-reply@databricks.com>
The invariant config now also sets db_aggregates_enabled: false, server_timing_enabled: true, jwt_cache_max_lifetime: 300s and openapi_mode: OPEN_API_MODE_IGNORE_PRIVILEGES, so no_drift checks the status readback for a forced false bool, a true bool, a duration and an enum. Every optional field is tagged CAN_REMOVE, the bools included. Removing either bool on its own plans an update and converges: the server keeps the last value, status reports it, and the field is a backend default. Co-authored-by: Isaac <no-reply@databricks.com>
The resource's schema description was inherited from the embedded SDK
spec type ("Desired PostgREST configuration (input)."). Override it to
say what the resource is, that a database has at most one Data API,
where the serving URL is (status.url) and how durations are written
(300s).
Also override the postgres.OpenApiMode description, which mentioned a
value that may be added later, with what the two supported values do.
Co-authored-by: Isaac <no-reply@databricks.com>
The comment above the PostgresDataApi entry split gofmt's alignment block, which re-indented the unrelated PostgresBranch and PostgresCatalog lines. Make it a trailing comment, so the diff against main is only the added entry. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Commit: Regression test reportSelected 13 changed tests (limit=20, 0 not selected) 649 tests: regression=9 unreleased=0 coverage=640 failing=0
9 more failure details omitted, see the full report in the run. |
ruslantalpa
force-pushed
the
ruslantalpa/postgres-data-apis
branch
from
October 22, 2026 11:14
3be15c5 to
1a763c3
Compare
Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
Integration test reportCommit: 300f4d8
Top 3 slowest tests (at least 2 minutes):
|
The Integration Tests failed on AWS for every postgres_data_apis test in the DMS=true variant. Deployment history rejects the new resource-key prefix: failed to create deployment version: operations.resource_key must have a known resource type prefix (e.g. 'jobs.', 'pipelines.'): postgres_data_apis.my_data_api (400 INVALID_PARAMETER_VALUE) The testserver's dmsResourceTypes is meant to mirror the real service, so listing postgres_data_apis there hid the failure locally. Remove it, so the fake rejects the prefix the same way the real service does, and exclude DMS=true for the postgres_data_apis resource tests and the invariant config. This matches how the features resource handles the same gap. Co-authored-by: Isaac <no-reply@databricks.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add the
postgres_data_apisresource (Private Preview). It enables and configures the Data API (PostgREST) of a Lakebase Postgres database:{parent}/data-api; changingparentrecreates it. Removing the resource turns the Data API off; the database and its data are unaffected.status.url; reference it as${resources.postgres_data_apis.<key>.status.url}. It is a stable output field: it is built from the read-write endpoint host, the workspace and the database name, so an in-place update of the Data API cannot change it, and the reference resolves without planning a no-op update of the resources that use it.jwt_cache_max_lifetimeuse the protobuf seconds format, for example300s.update_mask=spec.specis INPUT_ONLY, so GET does not return it.DoReadfills the remote spec fromstatus, which reports every setting in effect, sobundle plandetects out-of-band changes andbundle deployment bindapplies the configured spec on the next deploy. The generatedspec:input_onlyrules are dropped (apitypes.yml: postgres_data_apis: null) and every spec field is a hand-writtenbackend_default, so a field the config omits is left to the server.falseand was never set, so the bool fields are forced on the remote side.update_mask=specmerges, and per-field clearing is not used because clearing some fields currently leaves the Data API unable to serve requests. The removal plans one no-op update; the next plan is clean.postgres_*resources are Beta), so it is not suggested in the JSON schema and not exposed in PyDABs yet.Why
Lakebase users already manage projects, branches and databases with bundles and want to manage the Data API alongside them. The API is at Private Preview.
Notes for reviewers:
postgres_*resource, this one waits on its operation inline in create, update and delete, because the operation ID isn't persisted.experimental.deployment_historyenabled reject this resource type, so the cloud acceptance cells that run with deployment history on (DMS=true) will fail. Deploys without it are unaffected.jwt_role_claim_keyandjwt_cache_max_lifetimeare exposed because the API accepts them. Their runtime behaviour is still evolving at Private Preview; happy to hide them if you prefer.Tests
status.urlresolves on the first deploy, and an in-place update of the Data API plans no update of the job). The test server's Data API fake mirrors the real API's defaults,status.url, merge-on-update semantics, and error codes and messages.trueand an explicitfalse), a duration and an enum: no drift, removal of every optional field converges, delete/destroy idempotent.status.urlresolved on the first deploy, update (row cap applied, job not updated, next plan clean), removed field (value kept), out-of-band change detected and reverted, bind of an existing Data API applied on deploy, destroy (Data API off, data intact).This pull request and its description were written by Isaac.