Skip to content

Support uv workspace members in environments setup-local - #7018

Open
rugpanov wants to merge 21 commits into
mainfrom
uv-workspace-setup-local-2
Open

rugpanov wants to merge 21 commits into
mainfrom
uv-workspace-setup-local-2

Conversation

@rugpanov

@rugpanov rugpanov commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Stack (merge in this order):

  1. localenv: add venv-folder and per-file pyproject seams to setup-local #7017 localenv: add venv-folder and per-file pyproject seams to setup-local (see diff)
  2. Support uv workspace members in environments setup-local #7018 Support uv workspace members in environments setup-local (see diff) <- this PR
  3. Honor UV_PROJECT_ENVIRONMENT in environments setup-local #7026 Honor UV_PROJECT_ENVIRONMENT in environments setup-local (see diff)

Changes

databricks environments setup-local now works in a member of a uv workspace (one shared venv for several bundles):

  • The command asks uv for the workspace root (uv workspace dir, uv 0.9.9 or later). It compares folders by identity, so symlinks and letter case do not change the result.
  • For a member, the venv is <root>/.venv. venvPath stays relative to the project folder, for example ../../.venv.
  • The [tool.uv] constraint block goes to the root pyproject.toml, because uv reads constraints only from there. requires-python and databricks-connect stay in the member. A stale constraint block in the member is removed.
  • The command does not write a root file that uv cannot read. It rejects a root that defines [tool.uv] without a [tool.uv] header, and the merged root must parse as TOML.
  • New warning W_UV_WORKSPACE_CONSTRAINTS_REPLACED: the run replaced root pins that an earlier run or the user wrote. All members share these pins.
  • New omitempty JSON fields: uvWorkspaceRoot, uvWorkspaceBackupPath, plan.wouldWriteUvWorkspace, plan.wouldBackupUvWorkspace (the uv prefix keeps them apart from a Databricks workspace). The output for other projects does not change.
  • The text summary shows the workspace root. The help text explains the shared venv.
  • An interrupt during discovery stops the run at preflight, before any file is written.

Why

In a member, uv sync creates the venv at the workspace root, but the command validated <member>/.venv and failed. It also wrote the pins to the member file, where uv ignores them (databricks/databricks-vscode#2157).

Tests

  • Unit tests: discovery (member, standalone, greenfield, uv failure, symlinks, letter case), the root merge and its guards, an idempotent re-run, --no-constraints, and a read-only root.
  • Acceptance tests with a fake uv: a dry run in a member, a real run and a text-mode re-run in a member, and uv without workspace dir.
  • Integration test with real uv and a two-member workspace (gated by DATABRICKS_LOCALENV_TEST_PROVISION). It fails before this change and passes after it. It also checks that uv.lock applies the root pin.

Reviewer notes

  • The VS Code extension needs a follow-up: resolve venvPath against the project folder, and restore uvWorkspaceBackupPath after a failure.

Known limitations

  • With uv older than 0.9.9, discovery fails and the command keeps the standalone behavior. A member then fails as before.
  • A new folder with no pyproject.toml is not detected as a member. If a members glob matches it, the first run fails and a second run works. A follow-up fix is planned.
  • The pre-sync conflict check reads only the current member. uv sync still reports conflicts with other members or the root as E_PROVISION_CONFLICT, after the root file changed.
  • The command replaces constraint-dependencies that a user wrote in the root pyproject.toml. It emits W_UV_WORKSPACE_CONSTRAINTS_REPLACED and keeps a backup.
  • uv sync is exact, so a run for one member removes packages that only other members need. All members must target compatible compute.
  • venvPath is counted from the symlink-resolved project folder. If a symlink between the root and the member changes the folder depth, a consumer that joins paths by text gets a wrong folder. uvWorkspaceRoot + /.venv is correct in all cases.
  • On Windows, a member reached through a junction can get a wrong relative venvPath, because Windows resolves .. by text.

This pull request and its description were written by Isaac.

rugpanov and others added 4 commits October 12, 2026 13:08
Co-authored-by: Isaac <no-reply@databricks.com>
…elpers

Co-authored-by: Isaac <no-reply@databricks.com>
…ath at preflight

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: d40a322

Run: 38243539486

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 276 20 5:55
✅​ aws-windows-latest-4core-16gb 278 18 3:32
✅​ azure linux-2core-8gb 275 20 5:28
✅​ azure-windows-latest-4core-16gb 277 18 3:42
✅​ gcp linux-2core-8gb 276 20 7:01
✅​ gcp-windows-latest-4core-16gb 278 18 3:29
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:56 aws linux-2core-8gb TestAccept
3:49 gcp linux-2core-8gb TestAccept
3:34 azure-windows-latest-4core-16gb TestAccept
3:30 aws-windows-latest-4core-16gb TestAccept
3:27 gcp-windows-latest-4core-16gb TestAccept
2:49 azure linux-2core-8gb TestAccept

@rugpanov
rugpanov force-pushed the uv-workspace-setup-local-2 branch from d40a322 to 5b12fd3 Compare October 12, 2026 11:59
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: cd21b08

Run: 38275974071

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 276 20 5:33
✅​ aws-windows-latest-4core-16gb 278 18 4:03
✅​ azure linux-2core-8gb 275 20 5:10
✅​ azure-windows-latest-4core-16gb 277 18 4:17
✅​ gcp linux-2core-8gb 276 20 5:25
✅​ gcp-windows-latest-4core-16gb 278 18 4:11
Top 6 slowest tests (at least 2 minutes):
duration env testname
4:14 azure-windows-latest-4core-16gb TestAccept
4:08 gcp-windows-latest-4core-16gb TestAccept
4:00 aws-windows-latest-4core-16gb TestAccept
3:55 gcp linux-2core-8gb TestAccept
3:54 aws linux-2core-8gb TestAccept
3:50 azure linux-2core-8gb TestAccept

@rugpanov
rugpanov changed the base branch from uv-workspace-setup-local to main October 12, 2026 13:57
@rugpanov
rugpanov marked this pull request as ready for review October 12, 2026 15:42
@rugpanov
rugpanov requested review from a team as code owners October 12, 2026 15:42
rugpanov and others added 15 commits October 12, 2026 18:09
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…e root

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…mber

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
uv prints the root in the on-disk letter case, while the shell keeps the case
the user typed, so a name comparison treated a standalone project on a
case-insensitive file system as a member of itself.

Co-authored-by: Isaac <no-reply@databricks.com>
…root

Removing a stale managed block from a member now keeps its [tool.uv] header
when user keys follow the block. A workspace root that defines [tool.uv] with
dotted keys fails the merge instead of getting a second [tool.uv] table.

Co-authored-by: Isaac <no-reply@databricks.com>
- Warn when the root pins are replaced, also when the user wrote them.
- Report diskMutated from both pyproject writes; name the workspace root in
  root write errors.
- Fix the Windows activate hint for a relative venv path.
- Say in the help that uv sync removes packages only other members need.
- Rename SkipToolUv to ConstraintsAtWorkspaceRoot and set it with the other
  merge options; test WorkspaceRoot; tidy tests and acceptance scripts.

Co-authored-by: Isaac <no-reply@databricks.com>
- Count the venv path levels from the symlink-resolved project folder, so a
  symlink that changes the folder depth reports the venv uv uses.
- Reject a root that defines [tool.uv] with quoted or inline keys too.
- Warn about replaced root pins only when the file had constraint pins, not
  when a comment names the key.
- Remove unmarked constraint pins that an older run left in a member.
- Run uv workspace dir through runUvOutput; label the root region in the
  dry-run plan; correct the greenfield comments; extend the help text.

Co-authored-by: Isaac <no-reply@databricks.com>
The line-based merge matches only the plain [tool.uv] spelling, so a root that
spells it differently could get a table or key defined twice. The merged root
must now parse as TOML. An empty managed block no longer counts as pins to
replace, and the member acceptance test re-runs in text mode.

Co-authored-by: Isaac <no-reply@databricks.com>
…nk lines

The dry-run plan now names the workspace root file only when the run would
change it. Removing a header-less managed block from a member keeps the user's
blank line before it, and an empty constraint-dependencies list no longer
counts as pins to replace.

Co-authored-by: Isaac <no-reply@databricks.com>
rugpanov and others added 2 commits October 12, 2026 18:09
In this CLI, "workspace" usually means a Databricks workspace. The new fields
are uvWorkspaceRoot, uvWorkspaceBackupPath, plan.wouldWriteUvWorkspace, and
plan.wouldBackupUvWorkspace, and the text output says "uv workspace root".

Co-authored-by: Isaac <no-reply@databricks.com>
…the names

- An interrupt while uv workspace dir runs no longer falls back to the
  standalone behavior: with cached constraints, that run went on to rewrite the
  member pyproject.toml before it reported E_CANCELED.
- Rename the warning to W_UV_WORKSPACE_CONSTRAINTS_REPLACED and the Go fields
  to match the uv-prefixed JSON names.

Co-authored-by: Isaac <no-reply@databricks.com>
@rugpanov
rugpanov force-pushed the uv-workspace-setup-local-2 branch from 3914917 to cd21b08 Compare October 12, 2026 16:13
@rugpanov
rugpanov requested a review from misha-db October 12, 2026 16:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants