[nivle1661]

Summary

Run graph presentation and the shared four-phase MLOps lifecycle in dedicated Databricks Jobs notebooks. Each manual dispatch builds its own wheel and passes the same verified runtime candidate to both scenarios.

• Read the dedicated PAT, workspace host and expected principal from the protected environment; keep raw workspace/job/notebook evidence private and upload aggregates only.
• Validate existing single-node x86 test compute and let Jobs start terminated compute. Check actual Jobs HTML graph controls, saved outputs and exports; lifecycle phases use runtime credentials in fresh processes.
• Publish the actual interactive/saved-output result dialect as safe statuses/counts; malformed or incomplete graph reports cannot pass.
• Remove owned notebooks, model versions/models, experiment records and output prefixes, including partial failures. Product Volume/dependency fixes are isolated in Read scoped Unity Catalog Volume Parquet sources #293.

Change map

Maintained change map

Testing Done

• Final merged regressions: 151 passed for the final Databricks/summary suites, including 131 Databricks tests and 20 summary tests; the earlier broader shared run passed 328 with one optional skip. Independent review and lint/import/diff checks passed.
• Manual workflow 42938850179 passed graph UI and all four engine phases, including cleanup. Both interactive and saved-output browser checks ran.
• Six historical raw Databricks artifacts were deleted; metadata and download endpoints return 404. Aggregate artifacts and wheels were preserved.
• Automated platform CI coverage remains a separate follow-up.

Limits

Browser checks inspect the actual Jobs HTML export and reopening in a fresh context. Native notebook-editor/kernel reopening and experiment/registry pages remain outside this browser coverage; engine summaries preserve the existing qualification gaps.

Final-head existing CI passed or was skipped, with successful combined commit status (test workflow). Automated platform-harness CI integration remains a separate follow-up.

[nivle1661]

Change map

Requirement: Add protected manual Databricks notebook validation using the shared small MLOps lifecycle and graph fixtures. Both scenarios require the same dedicated single-node x86 test cluster. Jobs starts assigned idle compute; the runner validates its identity and actual driver type without creating, resizing or directly starting clusters. Verify executed cells and receipts, graph controls, saved/reopened Jobs HTML and exports. Clean only attempt-owned notebooks, Volume outputs, model versions, models and tagged experiments; upload aggregate results only. Existing local-only results do not prove managed platform execution; selected live evidence and focused regressions are required.

File	Function / scope	What changed	Why
.github/workflows/kumo-engine-databricks-notebook.yml	Protected dispatch and current-source build	Build github.sha once and give both cloud scenarios the same wheel and runtime candidate.	Avoid expired historical artifacts and qualify the dispatched product sources.
docs/development/databricks-notebook-tests.md	Local and runner operation	Document per-dispatch builds, local runtime manifests, credentials, compute startup and exact UI limits.	Make both execution paths reproducible without a stale wheel pin.
packages/kumo-engine/test/databricks_notebook/check_environment.py	Environment approval guard	Check exact deployment refs, maintainer reviewer and no administrator bypass.	Prevent credentials from reaching unapproved dispatches.
packages/kumo-engine/test/databricks_notebook/cleanup.py	Attempt-owned resource cleanup	Normalize directory-only trailing separators and remove exhaustively admitted model versions before the empty model with identity rechecks	Match actual provider behavior while refusing foreign, changed or ambiguous resources and preserving shared caches
packages/kumo-engine/test/databricks_notebook/client.py	Dedicated compute admission	Validate cluster ID/name, effective driver and node types, and single-node size through GET-only admission	Allow assigned terminated test compute to auto-start through Jobs while rejecting foreign resources and error/deleting states
packages/kumo-engine/test/databricks_notebook/kernel.py	Databricks notebook platform harness	Prepare/submit test notebook jobs, collect receipts and verify outputs.	Bind platform results to the selected attempt and configured identity.
packages/kumo-engine/test/databricks_notebook/lifecycle_kernel.py	Databricks notebook platform harness	Prepare/submit test notebook jobs, collect receipts and verify outputs.	Bind platform results to the selected attempt and configured identity.
packages/kumo-engine/test/databricks_notebook/lifecycle_notebook.py	Four-phase notebook lifecycle	Admit the runtime candidate after exact source checks and execute isolated workers with owned cleanup.	Exercise current-product recovery, tracking and permission behavior without exposing the runner PAT.
packages/kumo-engine/test/databricks_notebook/run.py	Graph submission and candidate admission	Verify the runtime descriptor before cloud access, execute on validated x86 test compute and collect actual Jobs HTML.	Bind graph/UI evidence to the selected wheel and dedicated resources.
packages/kumo-engine/test/databricks_notebook/verify_export.cjs	Databricks notebook platform harness	Prepare/submit test notebook jobs, collect receipts and verify outputs.	Bind platform results to the selected attempt and configured identity.
packages/kumo-engine/test/mlops_platforms/databricks.json	Shared validation support	Provide the lifecycle/packaging/configuration support used by the notebook tests.	Keep platform execution reproducible and evidence checks explicit.
packages/kumo-engine/test/snowflake_notebook/graph_ui.cjs	Shared Snowflake notebook/browser support	Prepare/submit notebook scenarios or validate authenticated rendered graph/UI evidence.	Reuse platform assertions and bind outputs to dedicated test resources.
packages/kumo-engine/test/wheel_mlops_lifecycle.py	Shared validation support	Provide the lifecycle/packaging/configuration support used by the notebook tests.	Keep platform execution reproducible and evidence checks explicit.
packages/kumo-engine/test/wheel_mlops_platform.py	Shared validation support	Provide the lifecycle/packaging/configuration support used by the notebook tests.	Keep platform execution reproducible and evidence checks explicit.
test/scripts/data/mlops_platforms/databricks.json	Shared validation support	Provide the lifecycle/packaging/configuration support used by the notebook tests.	Keep platform execution reproducible and evidence checks explicit.
test/scripts/test_engine_databricks_cleanup.py	Provider-shape and ownership regressions	Reproduce six-file trailing-directory metadata and nonempty-model rejection; cover bounded pagination, partial versions, identity changes and traversal	Verify cleanup before publishing and against the actual completed producer
test/scripts/test_engine_databricks_lifecycle.py	Focused regression coverage	Validate the behavior, evidence admission and negative/safety boundaries changed by this PR.	Reject invalid evidence and prevent false qualification.
test/scripts/test_engine_databricks_notebook.py	Dispatch and admission regressions	Test same-run artifact selection, both jobs using the manifest and rejection before cloud access.	Prevent stale candidates or missing provenance from reaching test resources.
test/scripts/test_engine_mlops_platform.py	Focused regression coverage	Validate the behavior, evidence admission and negative/safety boundaries changed by this PR.	Reject invalid evidence and prevent false qualification.
test/scripts/test_engine_snowflake_ui.py	Shared graph diagnostic regression	Model iframe ancestry in the existing trusted-error test.	Exercise the Databricks-scoped frame path with an accurate browser double.
packages/kumo-engine/test/platform_summary.py	Databricks graph result dialect	Publish interactive/saved-output statuses and browser-error counts; reject incomplete reports.	Preserve actual UI outcomes without exporting private browser errors or samples.
test/scripts/test_engine_platform_summary.py	Actual graph result regressions	Cover real producer format, malformed/missing checks, browser failures and private values.	Prevent false passes and misleading not-run results in public aggregates.

Verification: 151 final Databricks/summary regressions passed, including 131 Databricks checks and 20 aggregate-format cases; independent review passed. Manual workflow 42938850179 passed graph UI and all four lifecycle phases, including cleanup. Actual logs confirm interactive and saved-output checks. Six historical raw artifacts now return 404 for metadata and downloads. Automated platform CI coverage is user-deferred.

Stopping condition: selected checks have bound execution evidence and owned-resource cleanup. Native experiment/registry browser UI and full qualification remain outside the selected coverage. Shared caches, supplied compute and Job history remain.
Final-head existing CI passed or was skipped with successful combined commit status, including both engine package suites, all S3 integrations and the GPU smoke job.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Pre-merge checks: 1 · 1 · 1

Failed checks: 1 warning · 1 inconclusive

Check name	Status	Explanation	Resolution
Pr Size		The PR changes 110 files against the origin/main merge base. This exceeds the limit of 40 files.	Split the PR into reviewable units of 40 files or fewer.
Change Map Design Review		The linked change-map comment and problem context were not supplied, and live retrieval is unavailable. The PR description links to the comment but does not contain its design rationale. The repositor…	Provide the marked change-map comment and the linked problem or release requirement so the design can be reviewed against the diff.

Passed checks: 1 passed

Check name	Status	Explanation
Public Resource Access		No added public resource reference was confirmed inaccessible. Anonymous requests returned HTTP 200 for 207 literal URLs and all nine pinned workflow action references. The two other extracted strings…

Full details: Public Resource Access

Explanation

No added public resource reference was confirmed inaccessible. Anonymous requests returned HTTP 200 for 207 literal URLs and all nine pinned workflow action references. The two other extracted strings are a runtime API template and a URL-matching expression, not public resource links.

Full details: Change Map Design Review

Explanation

The linked change-map comment and problem context were not supplied, and live retrieval is unavailable. The PR description links to the comment but does not contain its design rationale. The repository diff alone cannot establish whether the comment explains necessity, preserved behavior, verification, and the stopping condition.

Resolution

Provide the marked change-map comment and the linked problem or release requirement so the design can be reviewed against the diff.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

[Vedaant-J]

Some of the same points as #239 apply here, plus a few specific to Databricks:

1. The test account's email is hard-coded. It appears in lifecycle_kernel.py:58, lifecycle_notebook.py:87, docs/development/databricks-notebook-tests.md:161-173 and the two test/scripts tests. This repo is public, and it also ties the harness to that one principal. Could it come from the environment's configuration instead?

2. Artifacts can expose workspace details. The upload steps (workflow lines 96-103, 135-142) publish the whole output directory, including the raw jobs/runs/get response in run.json (run_page_url with the workspace host and org id, creator_user_name), the cluster id in submission.json, and the full notebook output. The host is a secret, but GitHub masks secrets only in logs, not in artifacts, and any signed-in user can download artifacts here. Could the uploads keep only the summarised results, or redact these fields?

3. The workflow can't run once merged. Every job requires refs/heads/egu/databricks-ui-validation (lines 33, 54, 109), and check_environment.py:13 pins that branch, so dispatches from main or Add Snowflake notebook lifecycle and graph UI validation #239's branch skip.

4. Nothing removes what each run creates. Each run registers a new Unity Catalog model with two versions (wheel_mlops_platform.py:242), leaves a workspace notebook (run.py:268, lifecycle_notebook.py:194) and keeps runs/<attempt>/outputs on the Volume. On failure only the job run is cancelled (run.py:224-229). Could each run delete its model, notebook and outputs, including on failure?

5. The databricks extra change is a product change. Adding mlflow-skinny[databricks]>=3.5,<4 pulls databricks-agents, tiktoken and more (+263 lines in uv.lock) into every kumo-engine[databricks] install. Together with the Volume-path support in kumo_databricks/backend.py (which looks good and is tested), could it go in its own PR?

6. The cited trial runs predate the head. They ran at 484e5185 and a89297c0; since then the shared graph_ui.cjs, client.py and lifecycle_notebook.py changed. A run at the current head would show the PR as it will merge.

[nivle1661]

1. Removed the hard-coded email; lifecycle workers verify the expected principal from protected environment configuration.
2. Public artifacts now contain aggregates only. Raw Jobs responses, workspace details and notebook outputs stay private.
3. Enabled manual dispatch from main while retaining the explicitly protected trial branch.
4. Added owned-resource cleanup for notebooks, model versions/models, experiments and output prefixes, including partial failures. The runner validates dedicated x86 test compute; Jobs handles starting terminated compute.
5. Moved the Volume/dependency product changes into Read scoped Unity Catalog Volume Parquet sources #293. The workflow now builds the dispatched commit's wheel and passes the same runtime candidate to both scenarios.