Skip to content

Support best effort deployment history mode - #7169

Open
shreyas-goenka wants to merge 17 commits into
mainfrom
shreyas-goenka_data/shreyas/history-best-effort-default
Open

Support best effort deployment history mode#7169
shreyas-goenka wants to merge 17 commits into
mainfrom
shreyas-goenka_data/shreyas/history-best-effort-default

Conversation

@shreyas-goenka

@shreyas-goenka shreyas-goenka commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

When experimental.deployment_history: true or DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=true are set use the best effort deployment history mode instead of treating DMS as SOT.

@github-actions

github-actions Bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Commit: e6e83d731 | Workflow run

Regression test report

Selected 20 changed tests (limit=20, 3 not selected)

21 tests: regression=19 unreleased=0 coverage=2 failing=0

test branch main (e773191) latest (v1.22.1)
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS= ✅ ✅ ✅
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/bind-unbind-unsupported/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/declined-deploy/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/depends-on/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/deployment-metadata-change/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/empty-grants/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/existing-state/DMS= ✅ ❌ ❌
TestAccept/bundle/dms/failed-delete/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/failed-recreate/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/failed-update-permissions/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/mismatch-before-read/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/multiple-resources/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/no-drift/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/no-resources/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/operation-upload-fails-once/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/plan-without-dms/DMS= ✅ ✅ ✅
TestAccept/bundle/dms/record-failure/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/stale-local-state/DMS= ✅ ❌ ❌
TestAccept/bundle/dms/state-features/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/version-never-created/DMS=true ✅ ❌ ❌
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1311
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1081
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:659
        	Error:      	Not equal: 
        	            	expected: "\n>>> musterr [CLI] bundle deploy --plan plan.json\nWarn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]\nError: plan lineage \"different-lineage\" does not match state lineage \"\"; the state may have been modified by another process\n\n"
        	            	actual  : "\n>>> musterr [CLI] bundle deploy --plan plan.json\nWarn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]\nError: this plan was created for a different set of state features than the target now has; run 'bundle plan' again\n\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,3 +3,3 @@
        	            	 Warn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]
        	            	-Error: plan lineage "different-lineage" does not match state lineage ""; the state may have been modified by another process
        	            	+Error: this plan was created for a different set of state features than the target now has; run 'bundle plan' again
        	            	 
        	Test:       	TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true
        	Messages:   	bundle/deploy/readplan/lineage-mismatch/output.txt vs /tmp/TestAcceptbundledeployreadplanlineage-mismatchDMS=true436225051/002/output.txt

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/bind-unbind-unsupported/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1311
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1081
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:659
        	Error:      	Not equal: 
        	            	expected: "\n=== bind is refused while recording deployment history\n>>> musterr [CLI] bundle deployment bind foo [JOB_ID]\nError: bind is not supported for a bundle target that records deployment history\n\n\n=== Unbind is allowed for file-backed history state\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-bind-unbind-[UNIQUE_NAME]/default/files...\nCreated jobs.foo\nFiles: 2 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> [CLI] bundle deployment unbind foo\n\n>>> jq {features, state} .databricks/bundle/default/resources.json\n{\n  \"features\": null,\n  \"state\": {}\n}\n"
        	            	actual  : "\n=== bind is refused while recording deployment history\n>>> musterr [CLI] bundle deployment bind foo [JOB_ID]\nError: bind is not supported for a bundle target that records deployment history\n\n\n=== Unbind is allowed for file-backed history state\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-bind-unbind-[UNIQUE_NAME]/default/files...\nCreated jobs.foo\nFiles: 2 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> [CLI] bundle deployment unbind foo\nError: unbind is not supported for a bundle target that records deployment history\n\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -14,8 +14,6 @@
        	            	 >>> [CLI] bundle deployment unbind foo
        	            	+Error: unbind is not supported for a bundle target that records deployment history
        	            	 
        	            	->>> jq {features, state} .databricks/bundle/default/resources.json
        	            	-{
        	            	-  "features": null,
        	            	-  "state": {}
        	            	-}
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/bundle/dms/bind-unbind-unsupported/DMS=true
        	Messages:   	bundle/dms/bind-unbind-unsupported/output.txt vs /tmp/TestAcceptbundledmsbind-unbind-unsupportedDMS=true2263410637/002/output.txt

latest (v1.22.1):

acceptance_test.go:1311: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1311
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1081
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:659
        	Error:      	Not equal: 
        	            	expected: "\n=== bind is refused while recording deployment history\n>>> musterr [CLI] bundle deployment bind foo [JOB_ID]\nError: bind is not supported for a bundle target that records deployment history\n\n\n=== Unbind is allowed for file-backed history state\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-bind-unbind-[UNIQUE_NAME]/default/files...\nCreated jobs.foo\nFiles: 2 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> [CLI] bundle deployment unbind foo\n\n>>> jq {features, state} .databricks/bundle/default/resources.json\n{\n  \"features\": null,\n  \"state\": {}\n}\n"
        	            	actual  : "\n=== bind is refused while recording deployment history\n>>> musterr [CLI] bundle deployment bind foo [JOB_ID]\nError: bind is not supported for a bundle target that records deployment history\n\n\n=== Unbind is allowed for file-backed history state\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-bind-unbind-[UNIQUE_NAME]/default/files...\nCreated jobs.foo\nFiles: 2 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> [CLI] bundle deployment unbind foo\nError: unbind is not supported for a bundle target that records deployment history\n\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -14,8 +14,6 @@
        	            	 >>> [CLI] bundle deployment unbind foo
        	            	+Error: unbind is not supported for a bundle target that records deployment history
        	            	 
        	            	->>> jq {features, state} .databricks/bundle/default/resources.json
        	            	-{
        	            	-  "features": null,
        	            	-  "state": {}
        	            	-}
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/bundle/dms/bind-unbind-unsupported/DMS=true
        	Messages:   	bundle/dms/bind-unbind-unsupported/output.txt vs /tmp/TestAcceptbundledmsbind-unbind-unsupportedDMS=true1062068829/002/output.txt
FAIL
TestAccept/bundle/dms/declined-deploy/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/declined-deploy/output.txt
        +++ /tmp/TestAcceptbundledmsdeclined-deployDMS=true625865778/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -81,8 +81,16 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         >>> print_state.py --no-dms
         {
        @@ -91,17 +97,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_declined_deploy_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "name": "dms_declined_deploy_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A destructive change without --auto-approve is declined: this console cannot prompt
        @@ -124,6 +121,10 @@
           "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         === The recorded resources are unchanged by the decline: the schema is still at version 1
         >>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json
        @@ -160,6 +161,10 @@
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
         {
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
           "method": "POST",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions",
           "q": {
        
    acceptance_test.go:1118: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793111199533,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_pohcsyp7lravbmvxxbdoes3yue",
        LOG.catalog:   "metastore_id": "7a07680c-4eba-d49b-7a07-680c4ebad581",
        LOG.catalog:   "name": "dms_other_pohcsyp7lravbmvxxbdoes3yue",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793111199533,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
FAIL

latest (v1.22.1):

acceptance_test.go:1311: Diff:
        --- bundle/dms/declined-deploy/output.txt
        +++ /tmp/TestAcceptbundledmsdeclined-deployDMS=true924479036/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -81,8 +81,16 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         >>> print_state.py --no-dms
         {
        @@ -91,17 +97,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_declined_deploy_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "name": "dms_declined_deploy_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A destructive change without --auto-approve is declined: this console cannot prompt
        @@ -124,6 +121,10 @@
           "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         === The recorded resources are unchanged by the decline: the schema is still at version 1
         >>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json
        @@ -160,6 +161,10 @@
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
         {
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
           "method": "POST",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions",
           "q": {
        
    acceptance_test.go:1118: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793111211892,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_f4njtph3z5c7ti6fqxl6jmz5di",
        LOG.catalog:   "metastore_id": "7a07680f-2f5b-e26c-7a07-680f2f5be366",
        LOG.catalog:   "name": "dms_other_f4njtph3z5c7ti6fqxl6jmz5di",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793111211892,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
TestAccept/bundle/dms/depends-on/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/depends-on/output.txt
        +++ /tmp/TestAcceptbundledmsdepends-onDMS=true2456020464/002/output.txt
        @@ -5,7 +5,7 @@
           "plan_version": [PLAN_VERSION],
           "cli_version": "[CLI_VERSION]",
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -61,7 +61,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -91,53 +91,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.child": {
        -   "__id__": "[CHILD_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-depends-on-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "description": "depends on [PARENT_ID]",
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "child",
        -    "queue": {
        -     "enabled": true
        -    }
        -   },
        -   "depends_on": [
        -    {
        -     "node": "resources.jobs.parent",
        -     "label": "${resources.jobs.parent.id}"
        -    }
        -   ]
        -  },
        -  "resources.jobs.parent": {
        -   "__id__": "[PARENT_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-depends-on-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "parent",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         >>> [CLI] bundle destroy --auto-approve

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/deployment-metadata-change/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/deployment-metadata-change/output.txt
        +++ /tmp/TestAcceptbundledmsdeployment-metadata-changeDMS=true3701625437/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -128,7 +128,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/empty-grants/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/empty-grants/output.txt
        +++ /tmp/TestAcceptbundledmsempty-grantsDMS=true2705709697/002/output.txt
        @@ -138,7 +138,7 @@
           "lineage": "[UUID]",
           "serial": 2,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -163,17 +163,9 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_empty_grants_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "name": "dms_empty_grants_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         >>> [CLI] bundle destroy --auto-approve

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/existing-state/DMS=: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/existing-state/output.txt
        +++ /tmp/TestAcceptbundledmsexisting-stateDMS=2198908566/002/output.txt
        @@ -73,6 +73,6 @@
         {
           "state_version": 3,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/failed-delete/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/failed-delete/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-deleteDMS=true3189618187/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,28 +92,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.stuck": {
        -   "__id__": "[STUCK_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-delete-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "stuck",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A resource that fails to delete is recorded as a failed operation carrying the error, so the history says why rather than showing the resource gone
        @@ -201,30 +181,11 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
        + "serial": 2,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.stuck": {
        -   "__id__": "[STUCK_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-delete-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "stuck",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === The remote checkpoint stays at serial 1 while DMS advances to version 2

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/failed-recreate/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/failed-recreate/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-recreateDMS=true2143072630/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,18 +92,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_failed_recreate_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "comment": "v1",
        -    "name": "dms_failed_recreate_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A recreate deletes and then fails to create. The delete went through, so the resource really is gone: the failed operation clears its state (self-contained, not assuming the delete's own write reached the service), rather than describing the resource as it was before the deploy
        @@ -188,7 +178,7 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
          "state": {}
         }
        
    acceptance_test.go:1118: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793111195297,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_esymdso6jze3dfdnfquurrqvcy",
        LOG.catalog:   "metastore_id": "7a07680b-5235-2428-7a07-680b52352536",
        LOG.catalog:   "name": "dms_other_esymdso6jze3dfdnfquurrqvcy",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793111195297,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
    acceptance_test.go:1118: 
        LOG.plan_final: create schemas.foo
        LOG.plan_final: 
        LOG.plan_final: Plan: 1 to add, 0 to change, 0 to delete, 0 unchanged

latest (v1.22.1):

acceptance_test.go:1311: Diff:
        --- bundle/dms/failed-recreate/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-recreateDMS=true2569341401/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,18 +92,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_failed_recreate_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "comment": "v1",
        -    "name": "dms_failed_recreate_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A recreate deletes and then fails to create. The delete went through, so the resource really is gone: the failed operation clears its state (self-contained, not assuming the delete's own write reached the service), rather than describing the resource as it was before the deploy
        @@ -188,7 +178,7 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
          "state": {}
         }
        
    acceptance_test.go:1118: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793111208378,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_h4xslegjc5ev7gy4r2rsujstnm",
        LOG.catalog:   "metastore_id": "7a07680e-5dec-6d76-7a07-680e5dec6e5c",
        LOG.catalog:   "name": "dms_other_h4xslegjc5ev7gy4r2rsujstnm",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793111208378,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
    acceptance_test.go:1118: 
        LOG.plan_final: create schemas.foo
        LOG.plan_final: 
        LOG.plan_final: Plan: 1 to add, 0 to change, 0 to delete, 0 unchanged
TestAccept/bundle/dms/failed-update-permissions/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/failed-update-permissions/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-update-permissionsDMS=true3331617065/002/output.txt
        @@ -15,7 +15,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -45,50 +45,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-update-permissions-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.foo.permissions": {
        -   "__id__": "/jobs/[FOO_ID]",
        -   "state": {
        -    "object_id": "/jobs/[FOO_ID]",
        -    "__embed__": [
        -     {
        -      "level": "CAN_VIEW",
        -      "user_name": "viewer@example.com"
        -     },
        -     {
        -      "level": "IS_OWNER",
        -      "user_name": "[USERNAME]"
        -     }
        -    ]
        -   },
        -   "depends_on": [
        -    {
        -     "node": "resources.jobs.foo",
        -     "label": "${resources.jobs.foo.id}"
        -    }
        -   ]
        -  }
        - }
        + "state": {}
         }
         
         === A permissions update that fails records the failure with the permissions it still has. The resource ID here is a path rather than a number, and the state is the embedded permission list, so this covers a shape the resource tests do not
        @@ -139,52 +97,11 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
        + "serial": 2,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-update-permissions-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.foo.permissions": {
        -   "__id__": "/jobs/[FOO_ID]",
        -   "state": {
        -    "object_id": "/jobs/[FOO_ID]",
        -    "__embed__": [
        -     {
        -      "level": "CAN_VIEW",
        -      "user_name": "viewer@example.com"
        -     },
        -     {
        -      "level": "IS_OWNER",
        -      "user_name": "[USERNAME]"
        -     }
        -    ]
        -   },
        -   "depends_on": [
        -    {
        -     "node": "resources.jobs.foo",
        -     "label": "${resources.jobs.foo.id}"
        -    }
        -   ]
        -  }
        - }
        + "state": {}
         }
         
         >>> print_requests.py --dms //api/2.0/bundle --oneline

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/mismatch-before-read/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1311
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1081
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:659
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-mismatch-before-read-[UNIQUE_NAME]/default/files...\nCreated jobs.foo\nFiles: 2 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> fault.py GET /api/2.0/bundle/deployments/* 500 0 1\n\n>>> env DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=false [CLI] bundle plan\nupdate jobs.foo\n\nPlan: 0 to add, 1 to change, 0 to delete, 0 unchanged\n\n>>> env DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=false [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-mismatch-before-read-[UNIQUE_NAME]/default/files...\nUpdated jobs.foo\nFiles: 1 uploaded, 0 deleted\nResources: 0 created, 1 changed, 0 deleted, 0 unchanged\n\n>>> jq .features // {} .databricks/bundle/default/resources.json\n{}\n"
        	            	actual  : "\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-mismatch-before-read-[UNIQUE_NAME]/default/files...\nCreated jobs.foo\nFiles: 2 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> fault.py GET /api/2.0/bundle/deployments/* 500 0 1\n\n>>> env DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=false [CLI] bundle plan\nError: deployment history setting (false) does not match the existing state (true)\n\nUpdate experimental.deployment_history to match the existing deployment, or run \"databricks bundle destroy\" to start over\n\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -10,14 +10,8 @@
        	            	 >>> env DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=false [CLI] bundle plan
        	            	-update jobs.foo
        	            	+Error: deployment history setting (false) does not match the existing state (true)
        	            	 
        	            	-Plan: 0 to add, 1 to change, 0 to delete, 0 unchanged
        	            	+Update experimental.deployment_history to match the existing deployment, or run "databricks bundle destroy" to start over
        	            	 
        	            	->>> env DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=false [CLI] bundle deploy
        	            	-Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-mismatch-before-read-[UNIQUE_NAME]/default/files...
        	            	-Updated jobs.foo
        	            	-Files: 1 uploaded, 0 deleted
        	            	-Resources: 0 created, 1 changed, 0 deleted, 0 unchanged
        	            	 
        	            	->>> jq .features // {} .databricks/bundle/default/resources.json
        	            	-{}
        	            	+Exit code: 1
        	            	 
        	Test:       	TestAccept/bundle/dms/mismatch-before-read/DMS=true
        	Messages:   	bundle/dms/mismatch-before-read/output.txt vs /tmp/TestAcceptbundledmsmismatch-before-readDMS=true3855835970/002/output.txt

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/multiple-resources/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/multiple-resources/output.txt
        +++ /tmp/TestAcceptbundledmsmultiple-resourcesDMS=true3956012405/002/output.txt
        @@ -25,7 +25,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -70,100 +70,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.five": {
        -   "__id__": "[FIVE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "five",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.four": {
        -   "__id__": "[FOUR_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "four",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.one": {
        -   "__id__": "[ONE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "one",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.three": {
        -   "__id__": "[THREE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "three",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.two": {
        -   "__id__": "[TWO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "two",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === Redeploy with no changes: nothing is applied, so no version is created at all - the deployment's version tracks state changes, like serial
        @@ -182,7 +90,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -227,98 +135,7 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.five": {
        -   "__id__": "[FIVE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "five",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.four": {
        -   "__id__": "[FOUR_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "four",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.one": {
        -   "__id__": "[ONE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "one",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.three": {
        -   "__id__": "[THREE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "three",
        -    "queue": {
... (25 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/no-drift/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/no-drift/output.txt
        +++ /tmp/TestAcceptbundledmsno-driftDMS=true1012390018/002/output.txt
        @@ -35,7 +35,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -50,52 +50,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.pipelines.bar": {
        -   "__id__": "[BAR_ID]",
        -   "state": {
        -    "catalog": "main",
        -    "channel": "CURRENT",
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edition": "ADVANCED",
        -    "libraries": [
        -     {
        -      "file": {
        -       "path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/files/transform.py"
        -      }
        -     }
        -    ],
        -    "name": "bar-[UNIQUE_NAME]",
        -    "schema": "default",
        -    "serverless": true
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === Plan without touching anything: the deployment stamp must not show as drift
        @@ -224,7 +180,7 @@
           "lineage": "[UUID]",
           "serial": 2,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -243,53 +199,9 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.pipelines.bar": {
        -   "__id__": "[BAR_ID]",
        -   "state": {
        -    "catalog": "main",
        -    "channel": "CURRENT",
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "2"
        -    },
        -    "development": true,
        -    "edition": "ADVANCED",
        -    "libraries": [
        -     {
        -      "file": {
        -       "path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/files/transform.py"
        -      }
        -     }
        -    ],
        -    "name": "bar-[UNIQUE_NAME]",
        -    "schema": "default",
        -    "serverless": true
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === Plan again: no residual drift, and the version stamps are ignored

latest (v1.22.1): same output as main (e773191)

TestAccept/bundle/dms/no-resources/DMS=true: branch ✅ | main (e773191) ❌ | latest (v1.22.1) ❌

main (e773191):

acceptance_test.go:1311: Diff:
        --- bundle/dms/no-resources/output.txt
        +++ /tmp/TestAcceptbundledmsno-resourcesDMS=true1677862848/002/output.txt
        @@ -35,9 +35,8 @@
           "plan_version": [PLAN_VERSION],
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
        -  "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -54,9 +52,8 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
          "state": {}
         }
        @@ -74,6 +70,10 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
         {
        @@ -82,8 +82,16 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         === The plan records the deployment_history feature
         >>> [CLI] bundle plan -o json
        @@ -91,9 +97,8 @@
           "plan_version": [PLAN_VERSION],
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
        -  "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -110,9 +114,8 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
          "state": {}
         }

latest (v1.22.1): same output as main (e773191)

5 more failure details omitted, see the full report in the run.

@shreyas-goenka shreyas-goenka mentioned this pull request Oct 23, 2026
Draft
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: e6e83d7

Run: 40520304031

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 308 25 6:33
✅​ aws-windows-latest-4core-16gb 310 23 5:39
✅​ azure linux-2core-8gb 307 25 6:44
✅​ azure-windows-latest-4core-16gb 309 23 5:21
✅​ gcp linux-2core-8gb 308 25 6:27
✅​ gcp-windows-latest-4core-16gb 310 23 6:58
Top 6 slowest tests (at least 2 minutes):
duration env testname
4:44 gcp-windows-latest-4core-16gb TestAccept
4:01 aws linux-2core-8gb TestAccept
3:59 azure linux-2core-8gb TestAccept
3:54 gcp linux-2core-8gb TestAccept
3:23 aws-windows-latest-4core-16gb TestAccept
3:12 azure-windows-latest-4core-16gb TestAccept

@shreyas-goenka shreyas-goenka changed the title Use enforced best-effort history for the YAML opt-in Use enforced best-effort deployment history for all opt-ins Oct 26, 2026
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 4017609 to 0b70621 Compare October 26, 2026 05:25
@shreyas-goenka shreyas-goenka changed the title Use enforced best-effort deployment history for all opt-ins Default deployment history to enforced file-backed recording Oct 26, 2026
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch 8 times, most recently from 0471b06 to 8542de3 Compare October 26, 2026 10:47
shreyas-goenka
shreyas-goenka commented Oct 26, 2026
View reviewed changes
Comment thread acceptance/bundle/dms/no-drift/script

title "Deploy job foo and pipeline bar; both are recorded at version 1"
trace $CLI bundle deploy
replace_ids.py

@shreyas-goenka shreyas-goenka Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added because resources.json now contains the actual state, so we need to repl the IDs.

shreyas-goenka
shreyas-goenka commented Oct 26, 2026
View reviewed changes
Comment thread bundle/direct/dstate/state.go Outdated
case WritingDMS:
return db.VersionID
case WritingFileWithHistory, WritingFileWithHistoryEnforced:
case ReadingFile, WritingFileWithHistory, WritingFileWithHistoryEnforced:

@shreyas-goenka shreyas-goenka Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

planning happens before upgrade to write, so we need to use max here as well. Safe because db.VersionID is 0 by default.

Base automatically changed from shreyas-goenka_data/shreyas/statedb-modes to main October 26, 2026 10:54
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 8542de3 to f1e49cf Compare October 26, 2026 10:58
@github-actions github-actions Bot added the DABs DABs related issues label Oct 26, 2026
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from f1e49cf to ed3ba2e Compare October 26, 2026 11:09
@shreyas-goenka shreyas-goenka changed the title Default deployment history to enforced file-backed recording Support best effort deployment history mode Oct 26, 2026
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch 2 times, most recently from 8692c4f to 8ca44ba Compare October 26, 2026 11:37
shreyas-goenka
shreyas-goenka commented Oct 26, 2026
View reviewed changes
Comment thread cmd/bundle/utils/process.go
}
return enforceDeploymentHistorySetting(ctx, b, stateDesc, false)
stateDesc.Features = b.DeploymentBundle.StateDB.StateFeatures()
if err := enforceDeploymentHistorySetting(ctx, b, stateDesc, false); err != nil {

@shreyas-goenka shreyas-goenka Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll remove this in a followup PR. Kept here to keep the diff minimal for this one.

@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 8ca44ba to 49f895d Compare October 26, 2026 12:34
denik
denik reviewed Oct 26, 2026
View reviewed changes
Comment thread acceptance/bundle/dms/bind-unbind-unsupported/script
# The gate that refuses bind while recording deployment history is on the config, so it fires
# before any deploy. Create a real workspace job so bind gets past fetching the resource and
# reaches the gate, instead of failing because the resource does not exist.
title "bind is refused while recording deployment history"

@denik denik Oct 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With direct being source of truth, we can do bind now, right?

@shreyas-goenka shreyas-goenka Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes absolutely. I just plan on addressing it in a followup PR to keep the scope narrow here.

@denik denik Oct 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense; in this case I'd leave this test unchanged if possible.

@shreyas-goenka shreyas-goenka Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes are additive. They assert that unbind always uses WriteFile mode. Both are something we'll eventually need to fix in the followup so it's nice to record them here.

Comment thread acceptance/bundle/dms/bind-unbind-unsupported/databricks.yml.tmpl
bundle:
name: dms-bind-unbind-$UNIQUE_NAME
experimental:
deployment_history: true

@denik denik Oct 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is deployment_history set in this test? It is set somehow since bind outputs "Error: bind is not supported for a bundle target that records deployment history" but I don't see where. Env var somewhere?

@shreyas-goenka shreyas-goenka Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test only runs for EnvMatrix.DMS = ["true"] so this bit was redundant.

Comment thread acceptance/bundle/dms/declined-deploy/output.txt Outdated
"serial": 1,
"features": {
"deployment_history": {}
"deployment_history_best_effort": {}

@denik denik Oct 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need the feature anymore with direct being source of truth, right? Sure older clients will not write to DMS but that's defensible for beta. Advantage is that you can drop DMS without any back migration.

@shreyas-goenka shreyas-goenka Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, we'll keep the state, but rename it to deployment_history_shadow in a followup PR.

Comment thread acceptance/bundle/dms/mismatch-before-read/script
update_file.py databricks.yml 'deployment_history: true' 'deployment_history: false'

# The configuration mismatch must be rejected before trying to read the deployment from DMS.
# Disabling file-backed history must not depend on DMS being available.

@denik denik Oct 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is file-backed history?

@shreyas-goenka shreyas-goenka Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just our best effort history feature. We should be abel to disable this without talking to DMS or making any APIs calls.

@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 49f895d to 91e0249 Compare October 26, 2026 13:01
shreyas-goenka
shreyas-goenka commented Oct 26, 2026
View reviewed changes
Comment thread bundle/phases/bind.go
if b.Config.Experimental != nil {
configured = b.Config.Experimental.DeploymentHistory
}
if stateDesc.IsDMS() || bundleenv.RecordsDeploymentHistory(ctx, configured) {

@shreyas-goenka shreyas-goenka Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll remove this check all together in a followup, given DMS is no longer the source of truth.

@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch 2 times, most recently from 8f9a4fa to bbd5aa4 Compare October 26, 2026 13:21
@shreyas-goenka
shreyas-goenka requested review from a team as code owners October 26, 2026 14:38
denik
denik reviewed Oct 27, 2026
View reviewed changes
Comment thread acceptance/bundle/deploy/readplan/lineage-mismatch/plan.dms.json Outdated
Comment thread acceptance/bundle/dms/bind-unbind-unsupported/script
# The gate that refuses bind while recording deployment history is on the config, so it fires
# before any deploy. Create a real workspace job so bind gets past fetching the resource and
# reaches the gate, instead of failing because the resource does not exist.
title "bind is refused while recording deployment history"

@denik denik Oct 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense; in this case I'd leave this test unchanged if possible.

Comment thread cmd/bundle/utils/process.go Outdated
denik
denik reviewed Oct 27, 2026
View reviewed changes
Comment thread cmd/bundle/utils/process.go Outdated
@shreyas-goenka
shreyas-goenka requested a review from denik October 27, 2026 10:29
denik
denik reviewed Oct 27, 2026
View reviewed changes
Comment thread cmd/bundle/utils/process.go
if stateDesc.Engine.IsDirect() && !opts.SkipEnforcingDeploymentHistorySetting {
if err := enforceDeploymentHistorySetting(ctx, b, stateDesc, opts.Deploy || opts.PreDeployChecks); err != nil {
if stateDesc.Engine.IsDirect() && (opts.Deploy || opts.PreDeployChecks) {
if err := enforceDeploymentHistorySetting(ctx, b, stateDesc, true); err != nil {

@denik denik Oct 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We no longer want to enforce it, right? can we delete this enforceDeploymentHistorySetting and this block?

@shreyas-goenka shreyas-goenka Oct 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No we don't. I'm planning to remove in a small followup since it's fairly independent from current changes.

denik
denik reviewed Oct 27, 2026
View reviewed changes
Comment thread cmd/bundle/utils/process.go Outdated
Comment thread bundle/direct/dstate/state.go Outdated
Comment thread bundle/direct/apply.go
Comment thread bundle/phases/deploy.go Outdated
Comment thread bundle/phases/destroy.go Outdated
Comment thread bundle/phases/deploy.go Outdated
Comment thread bundle/phases/deploy.go Outdated
Comment thread bundle/direct/dstate/state.go
//
// DMS reads require a workspace client on ctx. DMS writes and history recording
// start after a version exists and InitializeOperationBuffer is called.
func (db *DeploymentState) Open(ctx context.Context, path string, mode Mode, withRecovery WithRecovery, dmsDeployment OpenDmsArgs) error {

@denik denik Oct 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove mode now, right? We always open in ReadingFile, then we upgrade to either WritingFile, WritingDMS or WritingDMSShadow. This can handled by UpgradeToWrite which can either uses features or maybe additional parameter.

@shreyas-goenka shreyas-goenka Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, done.

@shreyas-goenka shreyas-goenka Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted. Let's do this in a followup? The diff is a bit messy and unrelated to the rest of the PR.

Previously we had a withWrite parameter also supported in Open which we removed. I'll properly clean this up in a fast followup.

Comment thread bundle/direct/dstate/state.go
@shreyas-goenka
Remove the unused unenforced shadow mode
b1fe936 
Addresses denik review comment 4387690767.
@shreyas-goenka
Name the DMS authoritative mode DeploymentHistoryOnly
515557e 
Addresses denik review comment 4387791458.
@shreyas-goenka
Add the shadow feature without replacing other features
ac1f34f 
Addresses denik review comment 4387644243.
@shreyas-goenka
Remove deployment-history configuration enforcement
c60abb0 
Addresses denik review comment 4387609280. Existing file state can enable or disable shadow history without being rejected.
@shreyas-goenka
Revert "Remove deployment-history configuration enforcement"
f31c3f1 
This reverts commit c60abb0.
@shreyas-goenka
Warn for shadow history completion errors and fail history-only mode
ad28701 
Addresses denik review comment 4387717117. CompleteVersion returns service errors; callers warn for shadow mode and report fatal diagnostics for authoritative history-only mode.
@shreyas-goenka
Describe missing history directly in shadow warnings
a4ad7df 
Addresses denik review comment 4387758441. Shadow state always uses the file, so warnings describe the failed history action without implying a state-source fallback.
@shreyas-goenka
Select history write mode inside UpgradeToWrite
8a192c5 
Addresses denik review comments 4387734400 and 4387743185. Deploy and destroy use the same upgrade method; the state features determine its write policy.
shreyas-goenka
shreyas-goenka commented Oct 27, 2026
View reviewed changes
Comment thread bundle/direct/dstate/state.go Outdated
return fmt.Errorf("migrating state %s: %w", path, err)
}

if db.isDeploymentMetadataService() && dmsDeployment == nil {

@shreyas-goenka shreyas-goenka Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will be removed in followup soon.

@shreyas-goenka
Open deployment state for reading and defer write setup
d56e00d 
Addresses denik review comment 4387774406. Remove Open mode selection; use loaded feature markers for the resource source and UpgradeToWrite for write policy. Nil DMS metadata retains file-only compatibility checks for bind and unbind.
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from b120dfc to d56e00d Compare October 27, 2026 14:05
@shreyas-goenka
shreyas-goenka requested a review from denik October 27, 2026 14:14
denik
denik approved these changes Oct 27, 2026
View reviewed changes
Comment thread bundle/direct/dstate/state.go

switch db.mode {
case WritingDMS, WritingFileWithHistory, WritingFileWithHistoryEnforced:
case WritingDeploymentHistoryOnly, WritingDeploymentHistoryShadow:

@denik denik Oct 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we fall to errors.New below? It's still an error to call UpgradeToWrite twice, right?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@denik denik denik approved these changes
@renaudhartert-db renaudhartert-db Awaiting requested review from renaudhartert-db renaudhartert-db is a code owner automatically assigned from databricks/eng-deco-cli
@pietern pietern Awaiting requested review from pietern pietern is a code owner automatically assigned from databricks/eng-deco-dabs

Assignees

No one assigned

Labels

DABs DABs related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shreyas-goenka @eng-dev-ecosystem-bot @denik