Skip to content

Drop pandas on DBR 16.4 + serverless-v3 (no cp312 wheel) - #22

Closed
rugpanov wants to merge 1 commit into
envgen/drop-nonlocal-constraintsfrom
envgen/pandas-py312-drop
Closed

Drop pandas on DBR 16.4 + serverless-v3 (no cp312 wheel)#22
rugpanov wants to merge 1 commit into
envgen/drop-nonlocal-constraintsfrom
envgen/pandas-py312-drop

Conversation

@rugpanov

Copy link
Copy Markdown
Collaborator

Stacked on #21 (base branch = envgen/drop-nonlocal-constraints). It builds on that PR's envgen.py and regenerated artifacts, so it should merge after #21; I'll retarget to main once #21 lands.

Closes #18.

Why

DBR 16.4 and serverless-v3 pin pandas~=1.5.3 under requires-python==3.12.* — faithful to the image (confirmed against the DBR 16.4 LTS release notes, which list pandas 1.5.3 on Python 3.12.3). But pandas 1.5.3 has no cp312 wheel — pandas ships Python 3.12 wheels only from 2.1.1 — so uv sync / pip install -c can't install it locally and falls back to a failing source build.

These are the only two Python-3.12 runtimes still on pandas 1.5.x: 13.3/14.3/15.4 predate 3.12, and 17.3+ / serverless-v4+ already ship pandas 2.x. No in-range (~=1.5) version has a cp312 wheel, so widening can't salvage it.

What

  • envgen.py — add DROP_BY_ENV, an environment-scoped drop applied in _filtered (threaded env_name through build_pyproject / build_constraints). pandas is dropped for 16.4.x-scala2.12, 16.4.x-{cpu,gpu}-ml-scala2.12, and serverless-v3 only; every other env keeps its pandas pin. Documented in the module docstring, the DROP_BY_ENV comment, and the README "What is intentionally not included" section.
  • test_envgen.pyEnvScopedDropTest covers the drop on the four targets and the keep elsewhere.
  • Regenerated the four target envs' pyproject.toml + constraints.txt (pandas removed).

Result: pandas resolves to an installable version locally on those two runtimes, instead of a hard uv sync failure. (The durable general form — a PyPI wheel-availability guard in sync.py — is noted as future work on #18.)

Verification

  • python -m unittest test_envgen — 13 passing.
  • pandas absent from the 4 targets, present in 15.4 (1.5.3), 17.3 (2.2.3), serverless-v4 (2.2.3); all 33 pyproject.toml valid TOML.

This pull request and its description were written by Isaac.

*Why*

DBR 16.4 and serverless-v3 pin pandas~=1.5.3 under requires-python==3.12.*, faithful
to the image (confirmed against the DBR 16.4 LTS release notes). But pandas 1.5.3 has
no cp312 wheel — pandas ships Python 3.12 wheels only from 2.1.1 — so `uv sync` /
`pip install -c` can't install it locally and falls back to a failing source build.
These are the only two Python-3.12 runtimes still on pandas 1.5.x: 13.3/14.3/15.4
predate 3.12, and 17.3+ / serverless-v4+ already ship pandas 2.x. No in-range (~=1.5)
version has a cp312 wheel, so widening can't salvage it.

*What*

- envgen.py: add DROP_BY_ENV, an environment-scoped drop applied in _filtered
  (threaded env_name through build_pyproject/build_constraints). pandas is dropped for
  16.4.x-scala2.12, 16.4.x-{cpu,gpu}-ml-scala2.12, and serverless-v3 only; every other
  env keeps its pandas pin. Documented in the module docstring, the DROP_BY_ENV
  comment, and the README "What is intentionally not included" section.
- test_envgen.py: EnvScopedDropTest covers the drop on the four targets and the keep
  elsewhere (incl. no env_name).
- Regenerated: pandas removed from the four target envs' pyproject.toml + constraints.txt.

*Verification*

- python -m unittest test_envgen: 13 passing.
- pandas absent from the 4 targets, present in 15.4 (1.5.3), 17.3 (2.2.3),
  serverless-v4 (2.2.3); all 33 pyproject.toml valid TOML.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant