Drop pandas on DBR 16.4 + serverless-v3 (no cp312 wheel) - #23
Merged
Conversation
*Why*
DBR 16.4 and serverless-v3 pin pandas~=1.5.3 under requires-python==3.12.*, faithful
to the image (confirmed against the DBR 16.4 LTS release notes). But pandas 1.5.3 has
no cp312 wheel — pandas ships Python 3.12 wheels only from 2.1.1 — so `uv sync` /
`pip install -c` can't install it locally and falls back to a failing source build.
These are the only two Python-3.12 runtimes still on pandas 1.5.x: 13.3/14.3/15.4
predate 3.12, and 17.3+ / serverless-v4+ already ship pandas 2.x. No in-range (~=1.5)
version has a cp312 wheel, so widening can't salvage it.
*What*
- envgen.py: add DROP_BY_ENV, an environment-scoped drop applied in _filtered
(threaded env_name through build_pyproject/build_constraints). pandas is dropped for
16.4.x-scala2.12, 16.4.x-{cpu,gpu}-ml-scala2.12, and serverless-v3 only; every other
env keeps its pandas pin.
- The drop is version-gated (DROP_BY_ENV value = version prefix "1.5."): it only fires
when the pin is actually the incompatible 1.5.x, so it self-expires — if one of these
runtimes is ever re-pinned to a cp312-wheel 2.x, the pin is kept, not silently
discarded. Documented in the docstring, the DROP_BY_ENV comment, and the README.
- test_envgen.py: EnvScopedDropTest covers the drop on the four targets, the keep
elsewhere, and the keep-if-repinned-to-2.x (version gate).
- Regenerated: pandas removed from the four target envs' pyproject.toml + constraints.txt.
*Verification*
- python -m unittest test_envgen: 14 passing.
- pandas absent from the 4 targets, present in 15.4 (1.5.3), 17.3 (2.2.3),
serverless-v4 (2.2.3); all 33 pyproject.toml valid TOML.
Co-authored-by: Isaac <no-reply@databricks.com>
rugpanov
force-pushed
the
envgen/pandas-py312-drop
branch
from
August 21, 2026 17:11
0d7b017 to
3467ab9
Compare
rclarey
approved these changes
Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #18. (Split out of #21, which is now merged; this was previously PR #22, auto-closed when #21's branch was deleted.)
Why
DBR 16.4 and serverless-v3 pin
pandas~=1.5.3underrequires-python==3.12.*— faithful to the image (confirmed against the DBR 16.4 LTS release notes, which list pandas 1.5.3 on Python 3.12.3). But pandas 1.5.3 has no cp312 wheel — pandas ships Python 3.12 wheels only from 2.1.1 — souv sync/pip install -ccan't install it locally and falls back to a failing source build.These are the only two Python-3.12 runtimes still on pandas 1.5.x: 13.3/14.3/15.4 predate 3.12, and 17.3+ / serverless-v4+ already ship pandas 2.x. No in-range (
~=1.5) version has a cp312 wheel, so widening can't salvage it.What
envgen.py— addDROP_BY_ENV, an environment-scoped drop applied in_filtered(threadedenv_namethroughbuild_pyproject/build_constraints).pandasis dropped for16.4.x-scala2.12,16.4.x-{cpu,gpu}-ml-scala2.12, andserverless-v3only; every other env keeps its pandas pin. Documented in the module docstring, theDROP_BY_ENVcomment, and the README "What is intentionally not included" section.test_envgen.py—EnvScopedDropTestcovers the drop on the four targets and the keep elsewhere.pyproject.toml+constraints.txt(pandas removed).Result:
pandasresolves to an installable version locally on those two runtimes instead of a harduv syncfailure. (The durable general form — a PyPI wheel-availability guard insync.py— is noted as future work on #18.)Verification
python -m unittest test_envgen— 13 passing.pandasabsent from the 4 targets, present in 15.4 (1.5.3), 17.3 (2.2.3), serverless-v4 (2.2.3); all 33pyproject.tomlvalid TOML.This pull request and its description were written by Isaac.