From a10ff449863019e6503514a9cf054e8f9a8b3cec Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:13:36 +0000 Subject: [PATCH 01/10] Add knowledge_base example for presets and multi-workspace targets. Closes the KB-3 gap: three hosts/profiles, deployment presets, and target list-merge overrides for tasks and environments. Co-authored-by: omnigent --- .../.gitignore | 1 + .../README.md | 106 ++++++++++++++++++ .../databricks.yml | 80 +++++++++++++ .../resources/presets_demo.job.yml | 40 +++++++ .../src/prepare.py | 12 ++ .../src/process.py | 12 ++ 6 files changed, 251 insertions(+) create mode 100644 knowledge_base/presets_and_multi_workspace_targets/.gitignore create mode 100644 knowledge_base/presets_and_multi_workspace_targets/README.md create mode 100644 knowledge_base/presets_and_multi_workspace_targets/databricks.yml create mode 100644 knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml create mode 100644 knowledge_base/presets_and_multi_workspace_targets/src/prepare.py create mode 100644 knowledge_base/presets_and_multi_workspace_targets/src/process.py diff --git a/knowledge_base/presets_and_multi_workspace_targets/.gitignore b/knowledge_base/presets_and_multi_workspace_targets/.gitignore new file mode 100644 index 00000000..15bcc6dd --- /dev/null +++ b/knowledge_base/presets_and_multi_workspace_targets/.gitignore @@ -0,0 +1 @@ +.databricks diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md new file mode 100644 index 00000000..3ef05e63 --- /dev/null +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -0,0 +1,106 @@ +# Presets and multi-workspace targets + +## What this example shows + +This example shows `presets` on three targets: `dev`, `staging`, and `prod`. + +Each target uses a different `workspace.host` placeholder. You can set an optional `profile`. + +The bundle defines one job with two tasks. The job uses serverless environments with `environment_version`. The job has a schedule. + +Target overrides show: + +1. A task override by `task_key` (list merge). +2. An environment selection override on a task (`environment_key`). Job clusters merge by `job_cluster_key` the same way tasks merge by `task_key`. +3. A staging-only job parameter change. + +`mode: development` sets defaults such as a name prefix and paused schedules. Most `presets` values override those defaults. In development mode, schedules stay paused even if you set `trigger_pause_status: UNPAUSED`. + +## Prerequisites + +* Databricks CLI v0.236.0 or above (`source_linked_deployment` needs this version or later). +* Three Databricks workspaces, or one workspace for a dry run. +* Matching profiles in `~/.databrickscfg` when you use `profile`. + +## Configure + +1. Open `databricks.yml`. +2. Set `workspace.host` under `dev` to your development workspace URL. +3. Set `workspace.host` under `staging` to your staging workspace URL. +4. Set `workspace.host` under `prod` to your production workspace URL. +5. Set `workspace.root_path` under `prod` to a user or principal path you own. +6. Uncomment and set `profile` under a target when that target must use a named profile. +7. Confirm each `profile` name exists in `~/.databrickscfg`. + +## Deploy to each target + +1. Validate the default target: `databricks bundle validate -t dev` +2. Deploy development: `databricks bundle deploy -t dev` +3. Validate staging: `databricks bundle validate -t staging` +4. Deploy staging: `databricks bundle deploy -t staging` +5. Validate production: `databricks bundle validate -t prod` +6. Deploy production: `databricks bundle deploy -t prod` + +## What changes per target + +| Setting | `dev` | `staging` | `prod` | +| --- | --- | --- | --- | +| Job name | `[dev presets ] presets_demo` | `[staging] presets_demo` | `[prod] presets_demo` | +| Tags | `env=dev`, `team=examples` (plus development `dev` tag) | `env=staging`, `team=examples` | `env=prod`, `team=examples` | +| Schedule pause status | `PAUSED` (development mode) | `PAUSED` (preset) | `UNPAUSED` (preset) | +| `max_concurrent_runs` | `4` | `2` | `1` | +| `pipelines_development` | `true` | `false` | `false` | +| `source_linked_deployment` | `true` (dev only) | unset | unset | +| `batch_size` parameter | `100` | `50` | `100` | +| `process` `timeout_seconds` | unset | `1800` | `3600` | +| `process` `environment_key` | `default` (`environment_version` `4`) | `default` (`4`) | `heavy` (`5`) | + +## How target overrides merge + +* Lists of tasks merge by `task_key`. Matching keys join fields. New keys append. +* Lists of job clusters merge by `job_cluster_key`. +* Lists of environments do not key-merge. Select another environment from the base list with `environment_key` on a task. +* Maps join by key. The target value replaces the top-level value for the same key. +* Scalars in the target replace top-level scalars. +* You cannot unset a field with an empty target value. Omit the override, or set an explicit new value. + +## See the result + +After you set real hosts (and profiles if needed), run: + +```bash +databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo.name' +databricks bundle validate -t staging -o json | jq -r '.resources.jobs.presets_demo.name' +databricks bundle validate -t prod -o json | jq -r '.resources.jobs.presets_demo.name' +``` + +Expected names: + +* `dev`: `[dev presets ] presets_demo` +* `staging`: `[staging] presets_demo` +* `prod`: `[prod] presets_demo` + +Inspect pause status and concurrency: + +```bash +databricks bundle validate -t dev -o json | jq '.resources.jobs.presets_demo | {name, max_concurrent_runs, schedule, tags, parameters}' +databricks bundle validate -t staging -o json | jq '.resources.jobs.presets_demo | {name, max_concurrent_runs, schedule, tags, parameters, tasks}' +databricks bundle validate -t prod -o json | jq '.resources.jobs.presets_demo | {name, max_concurrent_runs, schedule, tags, parameters, tasks, environments}' +``` + +## Edge cases + +* **Development mode vs presets:** Presets override most development defaults. Development mode still forces paused schedules and triggers. A custom `name_prefix` in development mode must include your username or `${workspace.current_user.short_name}`. +* **Resource pause vs presets:** If a job sets `schedule.pause_status: UNPAUSED`, that value overrides the `trigger_pause_status` preset. +* **`source_linked_deployment`:** Use this preset only on `dev`. It applies to workspace UI deploys. The CLI `databricks bundle deploy` command ignores it. +* **Wrong profile or host:** A profile that points to a different host than `workspace.host` causes auth or deploy errors. Align `profile` and `host` for each target. +* **Production `root_path`:** `mode: production` requires `workspace.root_path` so only one copy of the bundle is deployed. +* **Serverless:** This job uses serverless environments. Classic `job_clusters` merge by `job_cluster_key`. + +## Clean up + +1. Destroy development: `databricks bundle destroy -t dev` +2. Destroy staging: `databricks bundle destroy -t staging` +3. Destroy production: `databricks bundle destroy -t prod` + +Confirm each prompt. Destroy removes only resources that this bundle owns for that target. diff --git a/knowledge_base/presets_and_multi_workspace_targets/databricks.yml b/knowledge_base/presets_and_multi_workspace_targets/databricks.yml new file mode 100644 index 00000000..7b84259c --- /dev/null +++ b/knowledge_base/presets_and_multi_workspace_targets/databricks.yml @@ -0,0 +1,80 @@ +# Presets and multi-workspace targets example. +# Replace each workspace.host (and optional profile) before you deploy. +# Placeholder hosts match other knowledge_base examples (for example development_cluster). +bundle: + name: presets_and_multi_workspace_targets + +include: + - resources/*.yml + +targets: + # Development target. mode: development applies defaults; presets override most of them. + # Exception: in development mode, schedules and triggers stay PAUSED. + dev: + default: true + mode: development + workspace: + host: https://dev-workspace.cloud.databricks.com + # Optional. Must match a section name in ~/.databrickscfg. + # profile: my-dev + presets: + # Development mode requires the username (or ${workspace.current_user.short_name}) in the prefix. + name_prefix: "[dev presets ${workspace.current_user.short_name}] " + tags: + env: dev + team: examples + trigger_pause_status: PAUSED + jobs_max_concurrent_runs: 4 + pipelines_development: true + # Dev only. Applies to workspace UI deploys; CLI deploy ignores this preset. + source_linked_deployment: true + + staging: + workspace: + host: https://staging-workspace.cloud.databricks.com + # profile: my-staging + presets: + name_prefix: "[staging] " + tags: + env: staging + team: examples + trigger_pause_status: PAUSED + jobs_max_concurrent_runs: 2 + pipelines_development: false + resources: + jobs: + presets_demo: + # (c) Staging-only job parameter change. + parameters: + - name: batch_size + default: "50" + # (a) Override a task by task_key (list merge). + tasks: + - task_key: process + timeout_seconds: 1800 + + prod: + mode: production + workspace: + host: https://prod-workspace.cloud.databricks.com + # profile: my-prod + # Required for mode: production. Replace the user path before you deploy. + root_path: /Workspace/Users/user@company.com/.bundle/${bundle.name}/${bundle.target} + presets: + name_prefix: "[prod] " + tags: + env: prod + team: examples + trigger_pause_status: UNPAUSED + jobs_max_concurrent_runs: 1 + pipelines_development: false + resources: + jobs: + presets_demo: + # (a) Override a task by task_key. + # (b) Point the task at a different environment (environments list does not + # key-merge; job_clusters merge by job_cluster_key instead). + tasks: + - task_key: process + timeout_seconds: 3600 + environment_key: heavy diff --git a/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml b/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml new file mode 100644 index 00000000..74cec96a --- /dev/null +++ b/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml @@ -0,0 +1,40 @@ +resources: + jobs: + presets_demo: + name: presets_demo + + # Quartz schedule. Presets (and mode: development) control pause_status. + schedule: + quartz_cron_expression: "0 0 9 * * ?" + timezone_id: UTC + + parameters: + - name: batch_size + default: "100" + + tasks: + - task_key: prepare + environment_key: default + spark_python_task: + python_file: ../src/prepare.py + parameters: + - "{{job.parameters.batch_size}}" + + - task_key: process + depends_on: + - task_key: prepare + environment_key: default + spark_python_task: + python_file: ../src/process.py + parameters: + - "{{job.parameters.batch_size}}" + + # Serverless environments. Prefer environment_version over deprecated client. + # Job clusters merge by job_cluster_key the same way tasks merge by task_key. + environments: + - environment_key: default + spec: + environment_version: "4" + - environment_key: heavy + spec: + environment_version: "5" diff --git a/knowledge_base/presets_and_multi_workspace_targets/src/prepare.py b/knowledge_base/presets_and_multi_workspace_targets/src/prepare.py new file mode 100644 index 00000000..c3dca5ec --- /dev/null +++ b/knowledge_base/presets_and_multi_workspace_targets/src/prepare.py @@ -0,0 +1,12 @@ +"""Prepare step for the presets demo job.""" + +import sys + + +def main() -> None: + batch_size = sys.argv[1] if len(sys.argv) > 1 else "unset" + print(f"prepare: batch_size={batch_size}") + + +if __name__ == "__main__": + main() diff --git a/knowledge_base/presets_and_multi_workspace_targets/src/process.py b/knowledge_base/presets_and_multi_workspace_targets/src/process.py new file mode 100644 index 00000000..07cffc9b --- /dev/null +++ b/knowledge_base/presets_and_multi_workspace_targets/src/process.py @@ -0,0 +1,12 @@ +"""Process step for the presets demo job.""" + +import sys + + +def main() -> None: + batch_size = sys.argv[1] if len(sys.argv) > 1 else "unset" + print(f"process: batch_size={batch_size}") + + +if __name__ == "__main__": + main() From 80ad858e5727c996b47c95dd7eab1072e10305e0 Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:17:57 +0000 Subject: [PATCH 02/10] Shrink presets example README to match sibling knowledge_base docs. Co-authored-by: omnigent --- .../README.md | 114 ++++-------------- 1 file changed, 23 insertions(+), 91 deletions(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md index 3ef05e63..ddf194bb 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/README.md +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -1,106 +1,38 @@ # Presets and multi-workspace targets -## What this example shows +This Declarative Automation Bundles example shows `presets` on `dev`, `staging`, and `prod` targets with different `workspace.host` placeholders and optional `profile` values. It also shows target overrides for tasks, environments, and a staging-only job parameter. -This example shows `presets` on three targets: `dev`, `staging`, and `prod`. - -Each target uses a different `workspace.host` placeholder. You can set an optional `profile`. - -The bundle defines one job with two tasks. The job uses serverless environments with `environment_version`. The job has a schedule. - -Target overrides show: - -1. A task override by `task_key` (list merge). -2. An environment selection override on a task (`environment_key`). Job clusters merge by `job_cluster_key` the same way tasks merge by `task_key`. -3. A staging-only job parameter change. - -`mode: development` sets defaults such as a name prefix and paused schedules. Most `presets` values override those defaults. In development mode, schedules stay paused even if you set `trigger_pause_status: UNPAUSED`. +For more information, see the [deployment modes documentation](https://docs.databricks.com/aws/en/dev-tools/bundles/deployment-modes). ## Prerequisites -* Databricks CLI v0.236.0 or above (`source_linked_deployment` needs this version or later). -* Three Databricks workspaces, or one workspace for a dry run. -* Matching profiles in `~/.databrickscfg` when you use `profile`. - -## Configure - -1. Open `databricks.yml`. -2. Set `workspace.host` under `dev` to your development workspace URL. -3. Set `workspace.host` under `staging` to your staging workspace URL. -4. Set `workspace.host` under `prod` to your production workspace URL. -5. Set `workspace.root_path` under `prod` to a user or principal path you own. -6. Uncomment and set `profile` under a target when that target must use a named profile. -7. Confirm each `profile` name exists in `~/.databrickscfg`. - -## Deploy to each target - -1. Validate the default target: `databricks bundle validate -t dev` -2. Deploy development: `databricks bundle deploy -t dev` -3. Validate staging: `databricks bundle validate -t staging` -4. Deploy staging: `databricks bundle deploy -t staging` -5. Validate production: `databricks bundle validate -t prod` -6. Deploy production: `databricks bundle deploy -t prod` - -## What changes per target - -| Setting | `dev` | `staging` | `prod` | -| --- | --- | --- | --- | -| Job name | `[dev presets ] presets_demo` | `[staging] presets_demo` | `[prod] presets_demo` | -| Tags | `env=dev`, `team=examples` (plus development `dev` tag) | `env=staging`, `team=examples` | `env=prod`, `team=examples` | -| Schedule pause status | `PAUSED` (development mode) | `PAUSED` (preset) | `UNPAUSED` (preset) | -| `max_concurrent_runs` | `4` | `2` | `1` | -| `pipelines_development` | `true` | `false` | `false` | -| `source_linked_deployment` | `true` (dev only) | unset | unset | -| `batch_size` parameter | `100` | `50` | `100` | -| `process` `timeout_seconds` | unset | `1800` | `3600` | -| `process` `environment_key` | `default` (`environment_version` `4`) | `default` (`4`) | `heavy` (`5`) | +* Databricks CLI v0.236.0 or above +* Three workspaces, or one workspace for a dry run +* Profiles in `~/.databrickscfg` when you set `profile` -## How target overrides merge +## Usage -* Lists of tasks merge by `task_key`. Matching keys join fields. New keys append. -* Lists of job clusters merge by `job_cluster_key`. -* Lists of environments do not key-merge. Select another environment from the base list with `environment_key` on a task. -* Maps join by key. The target value replaces the top-level value for the same key. -* Scalars in the target replace top-level scalars. -* You cannot unset a field with an empty target value. Omit the override, or set an explicit new value. - -## See the result - -After you set real hosts (and profiles if needed), run: +1. Set each target `workspace.host` in `databricks.yml`. Uncomment `profile` if needed. +2. Set `workspace.root_path` under `prod` to a path you own. +3. Validate: `databricks bundle validate -t dev` +4. Deploy: `databricks bundle deploy -t dev` +5. Repeat steps 3–4 with `-t staging` and `-t prod`. +6. Check prefixed names: ```bash databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo.name' -databricks bundle validate -t staging -o json | jq -r '.resources.jobs.presets_demo.name' -databricks bundle validate -t prod -o json | jq -r '.resources.jobs.presets_demo.name' ``` -Expected names: - -* `dev`: `[dev presets ] presets_demo` -* `staging`: `[staging] presets_demo` -* `prod`: `[prod] presets_demo` - -Inspect pause status and concurrency: - -```bash -databricks bundle validate -t dev -o json | jq '.resources.jobs.presets_demo | {name, max_concurrent_runs, schedule, tags, parameters}' -databricks bundle validate -t staging -o json | jq '.resources.jobs.presets_demo | {name, max_concurrent_runs, schedule, tags, parameters, tasks}' -databricks bundle validate -t prod -o json | jq '.resources.jobs.presets_demo | {name, max_concurrent_runs, schedule, tags, parameters, tasks, environments}' -``` - -## Edge cases - -* **Development mode vs presets:** Presets override most development defaults. Development mode still forces paused schedules and triggers. A custom `name_prefix` in development mode must include your username or `${workspace.current_user.short_name}`. -* **Resource pause vs presets:** If a job sets `schedule.pause_status: UNPAUSED`, that value overrides the `trigger_pause_status` preset. -* **`source_linked_deployment`:** Use this preset only on `dev`. It applies to workspace UI deploys. The CLI `databricks bundle deploy` command ignores it. -* **Wrong profile or host:** A profile that points to a different host than `workspace.host` causes auth or deploy errors. Align `profile` and `host` for each target. -* **Production `root_path`:** `mode: production` requires `workspace.root_path` so only one copy of the bundle is deployed. -* **Serverless:** This job uses serverless environments. Classic `job_clusters` merge by `job_cluster_key`. - -## Clean up +| Target | Name prefix | Pause status | Concurrency | +| --- | --- | --- | --- | +| `dev` | `[dev presets ]` | `PAUSED` | `4` | +| `staging` | `[staging]` | `PAUSED` | `2` | +| `prod` | `[prod]` | `UNPAUSED` | `1` | -1. Destroy development: `databricks bundle destroy -t dev` -2. Destroy staging: `databricks bundle destroy -t staging` -3. Destroy production: `databricks bundle destroy -t prod` +## Notes -Confirm each prompt. Destroy removes only resources that this bundle owns for that target. +* `mode: development` adds its own defaults. Presets override most of them. Schedules stay paused in development mode. +* Tasks merge by `task_key`. Job clusters merge by `job_cluster_key`. +* Environment lists append on override. Select another base environment with `environment_key` on a task. +* You cannot unset a field with an empty override. Omit the key, or set a new value. +* `source_linked_deployment` applies only to workspace UI deploys. The CLI ignores it. From d574d5b46144295a7373c3915dd0917026942325 Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:18:40 +0000 Subject: [PATCH 03/10] Require Databricks CLI v1.20.0 in presets example README. Co-authored-by: omnigent --- knowledge_base/presets_and_multi_workspace_targets/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md index ddf194bb..94907ca0 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/README.md +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -6,7 +6,7 @@ For more information, see the [deployment modes documentation](https://docs.data ## Prerequisites -* Databricks CLI v0.236.0 or above +* Databricks CLI v1.20.0 or above * Three workspaces, or one workspace for a dry run * Profiles in `~/.databrickscfg` when you set `profile` From 66fa9b67429b978f7d691d0f09638ca8d942e75e Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:20:30 +0000 Subject: [PATCH 04/10] Correct development-mode pause and preset docs in the KB example. Co-authored-by: omnigent --- .../presets_and_multi_workspace_targets/README.md | 10 +++++----- .../databricks.yml | 11 ++++++----- .../resources/presets_demo.job.yml | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md index 94907ca0..0c6d3ecf 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/README.md +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -31,8 +31,8 @@ databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo. ## Notes -* `mode: development` adds its own defaults. Presets override most of them. Schedules stay paused in development mode. -* Tasks merge by `task_key`. Job clusters merge by `job_cluster_key`. -* Environment lists append on override. Select another base environment with `environment_key` on a task. -* You cannot unset a field with an empty override. Omit the key, or set a new value. -* `source_linked_deployment` applies only to workspace UI deploys. The CLI ignores it. +* `mode: development` rejects `presets.trigger_pause_status: UNPAUSED`. A job may still set `schedule.pause_status: UNPAUSED`. Development mode also adds a `dev=` tag beside `env` and `team`. +* Tasks merge by `task_key`. Job clusters merge by `job_cluster_key`. Job parameters merge by `name`. +* Environment lists append on override. Select another base environment with `environment_key` on a task. You cannot unset a field with an empty override. +* `source_linked_deployment` works in the workspace UI. The CLI warns that it is available only there and deploys file copies. +* `pipelines_development` only affects pipelines. This bundle has none. diff --git a/knowledge_base/presets_and_multi_workspace_targets/databricks.yml b/knowledge_base/presets_and_multi_workspace_targets/databricks.yml index 7b84259c..f85dd4f8 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/databricks.yml +++ b/knowledge_base/presets_and_multi_workspace_targets/databricks.yml @@ -9,7 +9,7 @@ include: targets: # Development target. mode: development applies defaults; presets override most of them. - # Exception: in development mode, schedules and triggers stay PAUSED. + # Exception: mode: development rejects presets.trigger_pause_status: UNPAUSED. dev: default: true mode: development @@ -25,8 +25,9 @@ targets: team: examples trigger_pause_status: PAUSED jobs_max_concurrent_runs: 4 + # Only affects pipelines. This bundle has none; kept to show the preset key. pipelines_development: true - # Dev only. Applies to workspace UI deploys; CLI deploy ignores this preset. + # Dev only. Workspace UI can link to source. The CLI warns and deploys file copies. source_linked_deployment: true staging: @@ -44,7 +45,7 @@ targets: resources: jobs: presets_demo: - # (c) Staging-only job parameter change. + # (c) Staging-only job parameter change (parameters merge by name). parameters: - name: batch_size default: "50" @@ -72,8 +73,8 @@ targets: jobs: presets_demo: # (a) Override a task by task_key. - # (b) Point the task at a different environment (environments list does not - # key-merge; job_clusters merge by job_cluster_key instead). + # (b) Point the task at another environment. Environment lists append on + # override; job clusters merge by job_cluster_key instead. tasks: - task_key: process timeout_seconds: 3600 diff --git a/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml b/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml index 74cec96a..d15ff8a0 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml +++ b/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml @@ -3,7 +3,7 @@ resources: presets_demo: name: presets_demo - # Quartz schedule. Presets (and mode: development) control pause_status. + # Quartz schedule. Presets set pause_status unless the job sets its own value. schedule: quartz_cron_expression: "0 0 9 * * ?" timezone_id: UTC From 1bffe201fef54ca5c6f62de8953963d107cbd6e6 Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:23:15 +0000 Subject: [PATCH 05/10] Polish presets example README intro, notes, and job comments. Co-authored-by: omnigent --- .../presets_and_multi_workspace_targets/README.md | 6 +++++- .../resources/presets_demo.job.yml | 1 - 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md index 0c6d3ecf..e45ddb75 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/README.md +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -1,6 +1,8 @@ # Presets and multi-workspace targets -This Declarative Automation Bundles example shows `presets` on `dev`, `staging`, and `prod` targets with different `workspace.host` placeholders and optional `profile` values. It also shows target overrides for tasks, environments, and a staging-only job parameter. +This Declarative Automation Bundles example shows `presets` on `dev`, `staging`, and `prod` targets. +Each target uses a different `workspace.host` placeholder and an optional `profile`. +It also shows target overrides for tasks, environments, and a staging-only job parameter. For more information, see the [deployment modes documentation](https://docs.databricks.com/aws/en/dev-tools/bundles/deployment-modes). @@ -36,3 +38,5 @@ databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo. * Environment lists append on override. Select another base environment with `environment_key` on a task. You cannot unset a field with an empty override. * `source_linked_deployment` works in the workspace UI. The CLI warns that it is available only there and deploys file copies. * `pipelines_development` only affects pipelines. This bundle has none. +* Run `databricks bundle destroy -t ` to remove a target. +* The CLI profile must point to the same host as that target's `workspace.host`. diff --git a/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml b/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml index d15ff8a0..7b34bdb5 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml +++ b/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml @@ -30,7 +30,6 @@ resources: - "{{job.parameters.batch_size}}" # Serverless environments. Prefer environment_version over deprecated client. - # Job clusters merge by job_cluster_key the same way tasks merge by task_key. environments: - environment_key: default spec: From 7a8e9646df3bfd646ac15014c19fe2af4e85237f Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:46:04 +0000 Subject: [PATCH 06/10] Deploy presets example under /Workspace/bundle-examples root paths. Co-authored-by: omnigent --- .../presets_and_multi_workspace_targets/README.md | 10 +++++----- .../presets_and_multi_workspace_targets/databricks.yml | 5 +++-- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md index e45ddb75..884e389b 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/README.md +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -15,11 +15,10 @@ For more information, see the [deployment modes documentation](https://docs.data ## Usage 1. Set each target `workspace.host` in `databricks.yml`. Uncomment `profile` if needed. -2. Set `workspace.root_path` under `prod` to a path you own. -3. Validate: `databricks bundle validate -t dev` -4. Deploy: `databricks bundle deploy -t dev` -5. Repeat steps 3–4 with `-t staging` and `-t prod`. -6. Check prefixed names: +2. Validate: `databricks bundle validate -t dev` +3. Deploy: `databricks bundle deploy -t dev` +4. Repeat steps 2–3 with `-t staging` and `-t prod`. +5. Check prefixed names: ```bash databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo.name' @@ -33,6 +32,7 @@ databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo. ## Notes +* The bundle deploys under `/Workspace/bundle-examples`. Create that folder and limit who can manage it, or change `root_path`. * `mode: development` rejects `presets.trigger_pause_status: UNPAUSED`. A job may still set `schedule.pause_status: UNPAUSED`. Development mode also adds a `dev=` tag beside `env` and `team`. * Tasks merge by `task_key`. Job clusters merge by `job_cluster_key`. Job parameters merge by `name`. * Environment lists append on override. Select another base environment with `environment_key` on a task. You cannot unset a field with an empty override. diff --git a/knowledge_base/presets_and_multi_workspace_targets/databricks.yml b/knowledge_base/presets_and_multi_workspace_targets/databricks.yml index f85dd4f8..9a7024bb 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/databricks.yml +++ b/knowledge_base/presets_and_multi_workspace_targets/databricks.yml @@ -17,6 +17,7 @@ targets: host: https://dev-workspace.cloud.databricks.com # Optional. Must match a section name in ~/.databrickscfg. # profile: my-dev + root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target}/${workspace.current_user.short_name} presets: # Development mode requires the username (or ${workspace.current_user.short_name}) in the prefix. name_prefix: "[dev presets ${workspace.current_user.short_name}] " @@ -34,6 +35,7 @@ targets: workspace: host: https://staging-workspace.cloud.databricks.com # profile: my-staging + root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target} presets: name_prefix: "[staging] " tags: @@ -59,8 +61,7 @@ targets: workspace: host: https://prod-workspace.cloud.databricks.com # profile: my-prod - # Required for mode: production. Replace the user path before you deploy. - root_path: /Workspace/Users/user@company.com/.bundle/${bundle.name}/${bundle.target} + root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target} presets: name_prefix: "[prod] " tags: From 6c8affeecb1a156570d565a552a8b1d0d2c424c3 Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:47:20 +0000 Subject: [PATCH 07/10] Drop standard DAB behavior notes from the presets example docs. Co-authored-by: omnigent --- .../presets_and_multi_workspace_targets/README.md | 7 +------ .../presets_and_multi_workspace_targets/databricks.yml | 4 +--- 2 files changed, 2 insertions(+), 9 deletions(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md index 884e389b..bb34fe78 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/README.md +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -33,10 +33,5 @@ databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo. ## Notes * The bundle deploys under `/Workspace/bundle-examples`. Create that folder and limit who can manage it, or change `root_path`. -* `mode: development` rejects `presets.trigger_pause_status: UNPAUSED`. A job may still set `schedule.pause_status: UNPAUSED`. Development mode also adds a `dev=` tag beside `env` and `team`. -* Tasks merge by `task_key`. Job clusters merge by `job_cluster_key`. Job parameters merge by `name`. -* Environment lists append on override. Select another base environment with `environment_key` on a task. You cannot unset a field with an empty override. -* `source_linked_deployment` works in the workspace UI. The CLI warns that it is available only there and deploys file copies. -* `pipelines_development` only affects pipelines. This bundle has none. -* Run `databricks bundle destroy -t ` to remove a target. +* Environment lists append on override. Select another base environment with `environment_key` on a task. * The CLI profile must point to the same host as that target's `workspace.host`. diff --git a/knowledge_base/presets_and_multi_workspace_targets/databricks.yml b/knowledge_base/presets_and_multi_workspace_targets/databricks.yml index 9a7024bb..69045f6f 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/databricks.yml +++ b/knowledge_base/presets_and_multi_workspace_targets/databricks.yml @@ -8,8 +8,7 @@ include: - resources/*.yml targets: - # Development target. mode: development applies defaults; presets override most of them. - # Exception: mode: development rejects presets.trigger_pause_status: UNPAUSED. + # Development target. dev: default: true mode: development @@ -19,7 +18,6 @@ targets: # profile: my-dev root_path: /Workspace/bundle-examples/${bundle.name}/${bundle.target}/${workspace.current_user.short_name} presets: - # Development mode requires the username (or ${workspace.current_user.short_name}) in the prefix. name_prefix: "[dev presets ${workspace.current_user.short_name}] " tags: env: dev From d3711ba1bf3171ad024b463797d34fc911206824 Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:50:24 +0000 Subject: [PATCH 08/10] Remove root path note from presets README Co-authored-by: omnigent --- knowledge_base/presets_and_multi_workspace_targets/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md index bb34fe78..08e04097 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/README.md +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -32,6 +32,5 @@ databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo. ## Notes -* The bundle deploys under `/Workspace/bundle-examples`. Create that folder and limit who can manage it, or change `root_path`. * Environment lists append on override. Select another base environment with `environment_key` on a task. * The CLI profile must point to the same host as that target's `workspace.host`. From 22c8969de42ef0897e7a20628f66bfca7c196ee5 Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Wed, 21 Oct 2026 22:54:19 +0000 Subject: [PATCH 09/10] Align README with sibling examples Co-authored-by: omnigent --- knowledge_base/presets_and_multi_workspace_targets/README.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/README.md b/knowledge_base/presets_and_multi_workspace_targets/README.md index 08e04097..cf1aa676 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/README.md +++ b/knowledge_base/presets_and_multi_workspace_targets/README.md @@ -29,8 +29,3 @@ databricks bundle validate -t dev -o json | jq -r '.resources.jobs.presets_demo. | `dev` | `[dev presets ]` | `PAUSED` | `4` | | `staging` | `[staging]` | `PAUSED` | `2` | | `prod` | `[prod]` | `UNPAUSED` | `1` | - -## Notes - -* Environment lists append on override. Select another base environment with `environment_key` on a task. -* The CLI profile must point to the same host as that target's `workspace.host`. From a810eaf0843c1d82805bd445f3bd7e26039affc5 Mon Sep 17 00:00:00 2001 From: Andres Garcia Date: Thu, 22 Oct 2026 02:29:53 +0000 Subject: [PATCH 10/10] Bump presets demo serverless environments to 5 and 6. Co-authored-by: omnigent --- .../resources/presets_demo.job.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml b/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml index 7b34bdb5..d0f71892 100644 --- a/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml +++ b/knowledge_base/presets_and_multi_workspace_targets/resources/presets_demo.job.yml @@ -33,7 +33,7 @@ resources: environments: - environment_key: default spec: - environment_version: "4" + environment_version: "5" - environment_key: heavy spec: - environment_version: "5" + environment_version: "6"