Repository navigation
[SSH] Add versioning to agent dependency directory names - #7188
Conversation
Install uv and Node.js under version-suffixed directories and the Unity Gateway CLI under a directory keyed by its pinned commit, and check for ug at that path instead of on PATH. Bumping a pin then installs the new version instead of reusing an older install left on a long-lived cluster. Co-authored-by: Isaac <no-reply@databricks.com>
Integration test reportCommit: 6290e3f
Top 6 slowest tests (at least 2 minutes):
|
anton-107
left a comment
anton-107
left a comment
There was a problem hiding this comment.
Non-blocking from my side; one follow-up below.
| checksum string | ||
| } | ||
|
|
||
| func (s archiveSpec) installDirName() string { |
There was a problem hiding this comment.
Non-blocking: nothing removes the previous install once a bump lands, so every pin change leaves the old node-* (~190 MB extracted), uv-* and uv_tools-<commit> / uv_tools_bin-<commit> dirs behind in ~/.agent-shim/deps, plus the pre-versioning node / uv / uv_tools* dirs. On the long-lived dedicated clusters this is for, that keeps growing until the cluster restarts.
Could be a follow-up: after a successful install, under the setup lock, prune the sibling <dirName>-* / uv_tools*-* dirs that aren't the current version. One thing to watch: a session from an older CLI version still running on the same cluster may be using its own dir, so pruning would have to skip anything in use, or be limited to the unversioned legacy dirs.
There was a problem hiding this comment.
Yeah I wanted to add cleanup but I skipped it for now since I didn't have a clear idea how to not cleanup directories from older CLI versions that are still in use
Changes
Install agent dependencies (
uv,npm,ug) under versioned directories so old dependencies are not used when dependency pins are updated.Why
This is needed for dedicated cluster support, since dedicated clusters might remaining running for a long time and we want to be able to update dependencies without requiring customers to restart their clusters
Tests
updated/added unit tests