Vendored pip: a pin in a -r include outside the project root is ignored when a pylock.toml exists, so vendor --check passes and vex attests not_affected while pip install -r requirements.txt installs the unpatched release #1851

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

Consider a project whose root requirements.txt pins a package only through a -r include outside the project root (for example -r ../shared/base.txt holding six==1.16.0), and which also has a pip lock pylock.toml. In that project, scan --mode vendored (and the hosted → vendored eject) wires the package in pylock.toml only. It leaves the out-of-root pin on PyPI and exits 0. Every follow-up check then calls the project patched:

• The scan reports applied with a pypi_multiple_lockfiles skip ("wiring pylock.toml; installs driven by requirements.txt retain their existing sources"), and exits 0.
• vendor --check prints pkg:pypi/six@1.16.0: committed artifact and wiring verified and exits 0.
• vex, run against a venv installed with pip install -r requirements.txt (unpatched six), attests pkg:pypi/six@1.16.0 as not_affected and exits 0. Its only hint is "the installed tree does not match its vendored artifact … re-run your package manager's install to resync it". That advice is wrong: reinstalling from requirements.txt installs the upstream wheel again.

Two related layouts behave correctly:

• No pylock.toml: the same layout is refused with pypi_requirements_outside_root ("six is pinned in a requirements include outside the project root, which vendor cannot edit …"), exit 1.
• In-root include or root file that pylock can't co-wire: an in-root pin the wiring can't express (six[foo]==1.16.0, the Hosted → vendored over an extras pin (six[foo]==X) in requirements.txt beside a pip lock pylock.toml still un-hosts requirements.txt and leaves it unpatched, exit 0 (#1758 residual) #1794 shape) is reported as contested. vendor --check exits 1 with wiring contested: … requirements.txt resolves the same version from elsewhere, and vex omits six.

So the out-of-root include gets past both the planner's refusal and discovery's contest check.

Impact

Any pip install -r requirements.txt consumer (CI, Dockerfiles) of a monorepo-style project that shares a base requirements file from a parent directory stays on the vulnerable release. Meanwhile the CLI, vendor --check and the VEX document all report it as patched. The false VEX attestation is the serious part.

Repro

This uses the repo's usual patch-API mock: batch / by-package / view / package grant pkg:pypi/six@1.16.0 and pkg:pypi/idna@3.7, and each artifact is the real PyPI wheel with SAFE = True appended. idna and certifi come from real PyPI. Real pip 26.2.2 / CPython 3.12.

mkdir -p shared app && cd app
printf 'six==1.16.0\n' > ../shared/base.txt
printf -- '-r ../shared/base.txt\nidna==3.7\ncertifi==2024.8.30\n' > requirements.txt
python -m venv .venv && .venv/bin/pip install -r requirements.txt
.venv/bin/pip lock -r requirements.txt -o pylock.toml
VIRTUAL_ENV=$PWD/.venv socket-patch scan --mode vendored --yes   # exit 0
#   skipped pkg:pypi/six@1.16.0 pypi_multiple_lockfiles: "wiring pylock.toml; installs driven by requirements.txt retain their existing sources"
#   requirements.txt: idna line vendored; ../shared/base.txt untouched (six==1.16.0)
#   pylock.toml: six and idna -> .socket/vendor/pypi/<uuid>/...whl
socket-patch vendor --check                                    # exit 0, "six: committed artifact and wiring verified"
python -m venv fresh && fresh/bin/pip install --no-cache-dir -r requirements.txt
fresh/bin/python -c 'import six; print(hasattr(six, "SAFE"))'    # False (upstream PyPI wheel)
VIRTUAL_ENV=$PWD/fresh socket-patch vex --product pkg:pypi/app@1 -O vex.json   # exit 0
#   statements[0].status = not_affected, subcomponents = [idna@3.7, six@1.16.0]

Control: delete pylock.toml first, and the same scan --mode vendored fails six with pypi_requirements_outside_root (exit 1).

Expected vs actual

• Expected: this layout gets the same refusal as the no-pylock case. The planner's own comment on pypi_requirements_outside_root says: "an include outside the project root cannot be edited by a committable vendor flow; rewriting only the in-root copy would leave pip a duplicate requirement. Fail closed." A pylock beside it doesn't change what pip install -r requirements.txt installs. Failing that, discovery should contest the pylock wiring, as it does for an in-root pin (CLI_CONTRACT.md "Contested locks": a lock that resolves the same version from elsewhere contests the wiring, vendor --check fails and vex omits it). vex must not attest not_affected for an install source that delivers the upstream release.
• Actual: exit 0 and vendor --check exit 0. vex attests not_affected while pip installs unpatched six.

Matrix (Linux; the code paths are OS-independent)

Tested on main c07af61 (v5.0.1 is the latest release). No first-bad bisect: the out-of-root filter in discovery dates from the include walk itself.

Suspect code

• crates/socket-patch-core/src/vendor/pypi_requirements.rs:1231 (requirements_include_names_in): if !is_in_root_rel(rel) { return Ok(false); } drops out-of-root includes from the list that vex/discover/pypi_other.rs:240 (extract_requirements) reads. Discovery therefore never sees the registry pin that contests the pylock wiring. The module doc at pypi_other.rs:61 ("every in-root -r include") makes this explicit, but an out-of-root include is still something pip installs.
• crates/socket-patch-core/src/vendor/pypi.rs:395 / :429 (detect_pypi_flavor): sibling_pin_wirable (pypi_requirements.rs:427) is false for the read-only include, so requirements.txt is pushed as a pypi_multiple_lockfiles loser with a warning. The requirements-only planner instead raises pypi_requirements_outside_root (pypi_requirements.rs:893).
• Related: Hosted → vendored over an extras pin (six[foo]==X) in requirements.txt beside a pip lock pylock.toml still un-hosts requirements.txt and leaves it unpatched, exit 0 (#1758 residual) #1794 / PR Fix extras pin un-hosted beside a pylock (#1794) #1796 add a refusal for an extras pin in the same loser path (fresh_sibling_pin), but they don't cover a non-editable out-of-root include.

No probe runs: this routine's probe branches are still blocked, so this was reproduced on Linux only.