Skip to content

[draft] Apply configured MLflow experiment tags on bundle update - #7302

Open
lennartkats-db wants to merge 2 commits into
mainfrom
factory/dabs/3013c1dca491
Open

lennartkats-db wants to merge 2 commits into
mainfrom
factory/dabs/3013c1dca491

Conversation

@lennartkats-db

@lennartkats-db lennartkats-db commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Changes

  • Apply added or changed configured MLflow experiment tags during bundle updates through the experiment tag API.
  • Detect local tag changes while ignoring tags generated by the backend.
  • Extend the experiment acceptance fixture to verify an added tag and a stable experiment ID. Sort its tag output so API ordering does not affect the snapshot.

Why

Issue #6521 reports that configured tags, including mlflow.workload_creation_policy_id, are ignored when an experiment already exists. Recreating the experiment to apply a tag would lose its existing runs or traces.

Tests

  • go test ./bundle/direct/dresources -run TestExperimentDoUpdateSetsConfiguredTags -count=1 passed at 39dfd4f. The regression read old-policy after requesting new-policy at base 3aa5c96.
  • go test ./acceptance -run 'TestAccept/bundle/resources/experiments/basic' -tail -test.v passed locally in both DMS modes.
  • ./task check-changelog passed.
  • The previous cloud failure was a tag-order snapshot mismatch. At head 39dfd4f, the Integration Tests check passed with all six cloud test jobs successful. Linux, macOS, and Windows direct/directdms test jobs, lint, validate-generated, and regression-report checks also passed.

This PR was written by Codex.

@lennartkats-db lennartkats-db added DABs DABs related issues software-factory PR produced by the software factories task labels Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 39dfd4f7a | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

2 tests: regression=2 unreleased=0 coverage=0 failing=0
test branch main (7adc80d) latest (v1.24.0)
TestAccept/bundle/resources/experiments/basic/DMS= ✅ ❌ ❌
TestAccept/bundle/resources/experiments/basic/DMS=true ✅ ❌ ❌
TestAccept/bundle/resources/experiments/basic/DMS=: branch ✅ | main (7adc80d) ❌ | latest (v1.24.0) ❌

main (7adc80d):

acceptance_test.go:1345: Diff:
        --- bundle/resources/experiments/basic/output.txt
        +++ /tmp/TestAcceptbundleresourcesexperimentsbasicDMS=2966919059/002/output.txt
        @@ -150,10 +150,6 @@
               "value": "v1"
             },
             {
        -      "key": "k2",
        -      "value": "v2"
        -    },
        -    {
               "key": "mlflow.experiment.sourceName",
               "value": "/Users/[USERNAME]/new-name-[UNIQUE_NAME]"
             },
        
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1098
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:660
        	Error:      	Not equal: 
        	            	expected: "update experiments.my_experiment\n\nPlan: 0 to add, 1 to change, 0 to delete, 0 unchanged\n"
        	            	actual  : "Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1,4 +1,2 @@
        	            	-update experiments.my_experiment
        	            	+Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        	            	 
        	            	-Plan: 0 to add, 1 to change, 0 to delete, 0 unchanged
        	            	-
        	Test:       	TestAccept/bundle/resources/experiments/basic/DMS=
        	Messages:   	bundle/resources/experiments/basic/out.plan.txt vs /tmp/TestAcceptbundleresourcesexperimentsbasicDMS=2966919059/001/out.plan.txt

latest (v1.24.0): same output as main (7adc80d)

TestAccept/bundle/resources/experiments/basic/DMS=true: branch ✅ | main (7adc80d) ❌ | latest (v1.24.0) ❌

main (7adc80d):

acceptance_test.go:1345: Diff:
        --- bundle/resources/experiments/basic/output.txt
        +++ /tmp/TestAcceptbundleresourcesexperimentsbasicDMS=true1808210503/002/output.txt
        @@ -150,10 +150,6 @@
               "value": "v1"
             },
             {
        -      "key": "k2",
        -      "value": "v2"
        -    },
        -    {
               "key": "mlflow.experiment.sourceName",
               "value": "/Users/[USERNAME]/new-name-[UNIQUE_NAME]"
             },
        
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1098
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:660
        	Error:      	Not equal: 
        	            	expected: "update experiments.my_experiment\n\nPlan: 0 to add, 1 to change, 0 to delete, 0 unchanged\n"
        	            	actual  : "Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1,4 +1,2 @@
        	            	-update experiments.my_experiment
        	            	+Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        	            	 
        	            	-Plan: 0 to add, 1 to change, 0 to delete, 0 unchanged
        	            	-
        	Test:       	TestAccept/bundle/resources/experiments/basic/DMS=true
        	Messages:   	bundle/resources/experiments/basic/out.plan.txt vs /tmp/TestAcceptbundleresourcesexperimentsbasicDMS=true1808210503/001/out.plan.txt
FAIL

latest (v1.24.0): same output as main (7adc80d)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 39dfd4f

Run: 41974864513

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 274 15 4:17
🔄​ aws-windows-latest-4core-16gb 11 265 13 14:51
✅​ azure linux-2core-8gb 273 15 4:17
✅​ azure-windows-latest-4core-16gb 275 13 4:15
✅​ gcp linux-2core-8gb 274 15 4:14
✅​ gcp-windows-latest-4core-16gb 276 13 4:29
11 interesting tests: 11 flaky
Test Name aws-windows-latest-4core-16gb
🔄​ TestFsCatOnADir 🔄​f
🔄​ TestFsCatOnADir/uc-volumes 🔄​f
🔄​ TestFsCatOnNonExistentFile 🔄​f
🔄​ TestFsCatOnNonExistentFile/uc-volumes 🔄​f
🔄​ TestFsCpDir/dbfs_to_uc-volumes 🔄​f
🔄​ TestFsCpDir/local_to_uc-volumes 🔄​f
🔄​ TestFsCpDir/uc-volumes_to_dbfs 🔄​f
🔄​ TestFsCpDir/uc-volumes_to_local 🔄​f
🔄​ TestFsCpDir/uc-volumes_to_uc-volumes 🔄​f
🔄​ TestFsCpFileToFile 🔄​f
🔄​ TestFsCpFileToFile/local_to_uc-volumes 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:52 aws-windows-latest-4core-16gb TestAccept
3:34 gcp-windows-latest-4core-16gb TestAccept
3:22 azure-windows-latest-4core-16gb TestAccept

@lennartkats-db lennartkats-db added software-factory PR produced by the software factories task and removed software-factory PR produced by the software factories task labels Nov 3, 2026
@lennartkats-db
lennartkats-db marked this pull request as ready for review November 4, 2026 19:21
@lennartkats-db
lennartkats-db requested review from a team as code owners November 4, 2026 19:21
@lennartkats-db
lennartkats-db marked this pull request as draft November 4, 2026 19:21
@lennartkats-db
lennartkats-db marked this pull request as ready for review November 4, 2026 22:00
@lennartkats-db lennartkats-db changed the title [draft] Fix bundle/direct/dresources/experiment.go [draft] Apply configured MLflow experiment tags on bundle update Nov 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues software-factory PR produced by the software factories task

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants