Fix PyPI vendored wheels lost to gitignore/LFS (#1572)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #1572

Summary

Vendored PyPI wheels now get the same commit protection npm, bun, NuGet and the JVM trees already have. Before this, a repo that ignored *.whl or .socket/, or tracked *.whl with Git LFS, would commit a lock pointing at a wheel that every fresh clone was missing (or got as an LFS pointer). pipenv sync, pip install -r, uv sync and the rest then failed on that clone, even though vendoring had exited 0.

Root cause

All PyPI flavors (requirements.txt, uv, Poetry, PDM, Pipenv, Hatch, pylock) go through vendor_pypi_with_pipenv_version in vendor/pypi.rs. That function wrote the wheel into .socket/vendor/pypi/<uuid>/ without a git check-ignore probe and without an owned <uuid>/.gitignore / .gitattributes. Because the gap is in that shared path, one fix covers every flavor.

Changes

• Write-time protection (keep_wheel_committable): once the wheel is in place (fresh download, reused committed wheel, or in-sync rebuild), write <uuid>/.gitignore (!*) and <uuid>/.gitattributes (* -text -filter -ident -working-tree-encoding). Then probe the wheel and both files with git check-ignore. If they're still ignored, the run refuses vendor_artifact_gitignored and sweeps a uuid dir it created. If git can't answer, the run warns vendor_artifact_gitignore_unchecked.
• Directory rule refusal: a .socket/ or .socket/vendor/ rule can't be overridden from inside the uuid dir. So when a run is about to write the dir, it refuses vendor_artifact_gitignored before downloading or writing anything, dry run included. The check sits in pypi_prelude, so the download plan (service_preflight) gives the same answer. A hosted→vendored takeover that hits this keeps the hosted pin, because TakeoverUndo already rolls the restore back.
• Healing existing projects: an in-sync re-run (wheel already vendored and wired) now adds the missing dotfiles, so wheels vendored before this fix get protected too. repair writes them again when it restores a deleted uuid dir. An in-sync re-run that writes nothing new is not refused by a .socket/ rule.
• The dotfile writer is shared with npm (npm_dir::write_uuid_metadata). CLI_CONTRACT.md now lists PyPI in the vendor_artifact_gitignored and vendor_artifact_gitignore_unchecked rows.

Test evidence

Covers	Test	Without fix	With fix
#1572 *.whl rule	vendor::pypi::tests::a_whl_ignore_rule_is_overridden_by_the_uuid_gitignore	FAILED	ok
#1572 LFS variant	vendor::pypi::tests::the_uuid_gitattributes_unsets_a_repository_lfs_rule_for_wheels	FAILED	ok
#1572 .socket/ rule (.socket/, .socket/vendor/, pypi/; wet + dry run)	vendor::pypi::tests::a_directory_ignore_rule_refuses_before_any_write	FAILED	ok
existing vendored projects	vendor::pypi::tests::an_in_sync_rerun_restores_the_uuid_dotfiles	FAILED	ok
in-sync rebuild keeps its wired wheel on a commit-check failure (Bugbot)	vendor::pypi::tests::an_in_sync_rebuild_keeps_its_wheel_when_the_dotfiles_fail	FAILED	ok
takeover keeps the hosted pin	in_process_vendor_pypi_takeover::scan_vendored_over_hosted_with_socket_dir_ignored_keeps_the_hosted_pin	FAILED (exit 0, wheel ignored)	ok

Local runs:

• cargo fmt --all -- --check: clean.
• cargo clippy --workspace --all-features -- -D warnings: clean.
• cargo test --workspace --all-features --no-fail-fast: everything passes except 9 tests, and all 9 are artifacts of this sandbox, not of this change:
  • 8 are chmod-based write-failure tests that can't fail under uid 0. The two PyPI ones (pypi_poetry::wire_write_failure…, pypi_requirements::wire_failure_rolls_back…) pass when re-run as nobody.
  • 1 bun test needs patches-api.socket.dev, which the sandbox blocks.
• e2e_vendor_pypi_build --include-ignored (uv 0.11.32): 55/55. mode_migration_pypi: 64/64. e2e_pypi needs the live patch API, which isn't reachable from the sandbox, so it's left to CI.
• The npm/pypi/gem wrappers are untouched: they only dispatch the binary.

Review notes

Bugbot (43f8e3a): an in-sync rebuild was swept when the commit check failed. Fixed in 85a0d5e, which also moves the check after the in-sync pin check.

/code-review high:
Fixed: in-sync re-runs skipped the dotfiles; repair didn't restore them; the root refusal blocked in-sync runs over force-added wheels; the contract row for gitignore_unchecked; a weak assertion; duplicated metadata writer; extra git spawns on in-sync runs (the probe now runs only when the run writes).

Not changed:

• Hoist the check into the shared vendor dispatch for every ecosystem. gem, cargo, composer and golang likely have the same gap, but that is a cross-ecosystem refactor outside this issue's scope (Vendored Maven, Gradle and NuGet artifacts can be git-ignored silently, because only npm checks .gitignore #1061 tracked the same idea). Left as a follow-up.
• A reused wheel in a partly writable uuid dir could be left with one new dotfile. This needs a uuid dir where one atomic write succeeds and the next fails. The run then fails loudly and the leftover file is the one a successful run would write anyway, so no extra handling.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MccSpwBKRT9e2VHyxDyqzH

---

Note

Medium Risk
Changes shared PyPI vendor/repair paths and git commit semantics; failures are explicit refusals rather than silent broken clones.

Overview
PyPI vendoring now matches npm-family ecosystems: vendored wheels under .socket/vendor/pypi/<uuid>/ get a uuid-level .gitignore (!*) and .gitattributes (to defeat *.whl ignore rules and Git LFS), then git check-ignore runs so commits cannot silently omit the wheel.

Refusal and rollback: If the uuid directory itself is ignored (e.g. .socket/), the run fails with vendor_artifact_gitignored before download or write (including dry run). If the wheel still looks ignored after the dotfiles, the new uuid dir is removed (except reused committed wheels). Hosted→vendored takeover keeps the hosted pin when refusal happens (#1572).

Healing: In-sync re-runs add missing dotfiles on wheels vendored before this fix; repair restores them when rebuilding a deleted uuid dir. npm_dir::restore_uuid_metadata is split into write_uuid_metadata so PyPI can supply its own ignore text.

CLI_CONTRACT.md documents PyPI under vendor_artifact_gitignored / vendor_artifact_gitignore_unchecked. Tests cover *.whl, LFS, directory rules, in-sync repair, and takeover behavior.

^{Reviewed by Cursor Bugbot for commit 43f8e3a. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 85a0d5e. Configure here.}

[Mikola Lysenko (mikolalysenko)]

[agent] Ready for review at 85a0d5e.

• CI: 35 checks green (19 skipped), ci-ok green.
• Bugbot: its one finding on 43f8e3a is fixed and resolved; its review of 85a0d5e found nothing new.
• Every behavior change has a regression test that fails without the fix (see the table in the description).

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Labeled Ready for review at head 85a0d5e.

• CI: 35/35 checks green (19 skipped), ci-ok green. No main-wide failures.
• Bugbot: its one finding on 43f8e3a was fixed and resolved; its review of 85a0d5e found no new issues. No open review threads.
• No merge conflicts; no CHANGELOG.md changes.

---

Generated by Claude Code