Skip to content

STAC-25540: fetch private-index wheels, then drop the credential - #364

Merged
LouisParkin merged 1 commit into
STAC-25463-integrations-github-cifrom
STAC-25540-credfree-private-index
Aug 5, 2026
Merged

STAC-25540: fetch private-index wheels, then drop the credential#364
LouisParkin merged 1 commit into
STAC-25463-integrations-github-cifrom
STAC-25540-credfree-private-index

Conversation

@LouisParkin

Copy link
Copy Markdown
Contributor

What

Removes the private-package-index GitHub Environment gate and replaces it with a narrower fix for the same finding: the credential is now written, used for one fixed download, and destroyed before any repository code runs.

Closes STAC-25540. Stacked on #362review and merge that first; this targets its branch and will retarget automatically.

Why

vsphere is the only suite needing a credential. It pins vsphere-automation-sdk==1.82.0, an unmodified upstream VMware wheel that VMware withdrew from public PyPI. We self-host it in the GitLab Package Registry only because that org was private. Public PyPI serves a 0.0.1 placeholder squatting the name, which installs cleanly and fails much later.

setup_artifact_registry.sh left the password in a 0600 ~/.netrc for the rest of the job, so the suite's tests, its tox environment and its entire dependency tree could read it — and that code is PR-authored. That was @LouisLotter's P1 in #362.

The environment gate answered it by requiring a human to release each run. It worked, but SHARED_PATHS covers the CI files, so it fired on roughly one commit in six (28 of 176 in the last year) and blocked authors on their own PRs. prevent_self_review has since been disabled as a stopgap; this removes the need for the gate at all.

How

.setup-scripts/fetch_private_wheels.sh writes the netrc, downloads a fixed package set into a wheelhouse under RUNNER_TEMP, deletes the netrc, and rewrites ~/.pip/pip.conf to resolve that wheelhouse through find-links. Everything after it runs with no credential on disk and no authenticated index configured.

Four details are load-bearing:

  • The package set is hardcoded, not read from the tree — while the credential is on disk, a PR must not be able to redirect pip at a package of its choosing.
  • --only-binary=:all: — downloading an sdist executes its setup.py, which would hand upstream code a process with a readable ~/.netrc.
  • The system interpreter, not the toolchain virtualenv, whose pip is built by repository code.
  • An EXIT trap revokes the credential on failure paths too; the script then asserts the netrc is gone and the wheel actually arrived, rather than silently falling through to the PyPI placeholder.

Rejected alternative

A separate credentialed job publishing the wheels via upload-artifact. Workflow artifacts on a public repository are anonymously downloadable, so that would have amounted to publicly redistributing a vendor artifact VMware withdrew, under no SPDX licence. Re-hosting it publicly is a licensing question, not an engineering one.

Residual risk, stated plainly

A PR that edits this workflow can still reach the secret, because pull_request runs the PR's own copy of the workflow. That is true of every secret in every repository and is contained by review and branch protection, not by CI. What changed is that ordinary test code and its dependency tree no longer see it, and that needed no workflow edit at all.

Fork PRs still skip vsphere — they get no Actions secrets on a public repo, so they could not fetch the wheel regardless. The fork guard stays.

Validation

  • shellcheck clean on the new script and select-checks.sh; actionlint clean; Zizmor clean, no findings.
  • Locally verified: missing argument, missing credentials, and a failed download each exit non-zero with no netrc left behind and the password present in no file and no log output.
  • Locally verified that find-links in ~/.pip/pip.conf is honoured with no command-line flags, and that the same install fails without it — this is the mechanism vsphere/tox.ini depends on via passenv HOME.
  • The successful download itself can only be exercised in CI, since it needs the registry credential.

setup_artifact_registry.sh is untouched — .gitlab-ci.yml still references it in three places.

After merge

The private-package-index environment can be deleted from repo settings; nothing will reference it.

The vsphere suite is the only one that needs a credential: it pins
vsphere-automation-sdk==1.82.0, an unmodified upstream VMware wheel that
VMware withdrew from public PyPI, so it resolves only from our GitLab
Package Registry. Public PyPI serves a 0.0.1 placeholder squatting the
name, which installs cleanly and fails much later.

setup_artifact_registry.sh left the registry password in a 0600 ~/.netrc
for the remainder of the job, so the suite's tests, its tox environment
and its whole dependency tree could read it. Test code is PR-authored,
which made the credential effectively readable by whoever opened the pull
request (STAC-25463 review, P1).

That was contained by putting the job behind the private-package-index
GitHub Environment, releasing the credential only after a human approved
each run. It worked, but SHARED_PATHS covers the CI files, so it fired on
roughly one commit in six and blocked authors on their own pull requests.

Replace it with a narrower fix. fetch_private_wheels.sh writes the netrc,
downloads one hardcoded package set into a wheelhouse under RUNNER_TEMP,
deletes the netrc, and rewrites ~/.pip/pip.conf to resolve that wheelhouse
via find-links. Everything afterwards runs with no credential on disk and
no authenticated index configured, and the environment gate is gone.

Details that matter:

  * The package set is hardcoded, not read from the tree: while the
    credential is on disk, a pull request must not be able to redirect pip.
  * --only-binary=:all:, because downloading an sdist executes its
    setup.py and would hand upstream code a readable ~/.netrc.
  * A system interpreter, never the toolchain virtualenv, whose pip is
    built by repository code. The path is resolved rather than hardcoded
    -- bci/python:3.13 ships /usr/bin/python3.13 and no /usr/bin/python3,
    and `python3` on PATH is a /usr/local/bin shim -- and the result is
    then rejected if it resolves inside GITHUB_WORKSPACE, which is what
    makes "system" a guarantee rather than an assumption.
  * An EXIT trap revokes the credential on the failure paths too, and the
    script then asserts the netrc is gone and the wheel actually arrived.

Rejected: a separate credentialed job publishing the wheels with
upload-artifact. Workflow artifacts on a public repository are anonymously
downloadable, so that would have amounted to publicly redistributing a
vendor artifact VMware withdrew, under no SPDX licence.

Residual risk, recorded in the workflow: a pull request that edits the
workflow itself can still reach the secret, since pull_request runs the
PR's own copy. That holds for every secret in every repository and is
contained by review, not CI. What changed is that ordinary test code and
its dependencies no longer see it.

setup_artifact_registry.sh is untouched; .gitlab-ci.yml still uses it.

Verified: shellcheck and actionlint clean, Zizmor clean (no findings).
Verified inside the pinned BCI image that the interpreter resolves to
/usr/bin/python3.13 and that a workspace-resident interpreter is refused.
Verified that a missing argument, missing credentials, a workspace
interpreter and a failed download all exit non-zero with no netrc left
behind and the password in no file or log; and that find-links in
~/.pip/pip.conf is honoured with no command-line flags, which is what tox
relies on via passenv HOME.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@LouisParkin
LouisParkin force-pushed the STAC-25540-credfree-private-index branch from bf43e63 to e6d160f Compare August 5, 2026 13:11
@LouisParkin

Copy link
Copy Markdown
Contributor Author

Verified end-to-end in CI

Run 31009062454 is green, including Check tests, private index (vsphere) — with no environment gate and no human approval. From that job's log:

→ Downloading private-index packages into /__w/_temp/private-wheels
    vsphere-automation-sdk==1.82.0
  using /usr/bin/python3.13
  Downloading https://gitlab.com/api/v4/.../vsphere_automation_sdk-1.82.0-py3-none-any.whl
Saved /__w/_temp/private-wheels/vsphere_automation_sdk-1.82.0-py3-none-any.whl
...
find-links = /__w/_temp/private-wheels
Credential revoked; no /github/home/.netrc remains.

and then, in the suite itself, with no credential present:

Processing /__w/_temp/private-wheels/vsphere_automation_sdk-1.82.0-py3-none-any.whl (from -r requirements.in (line 2))
Successfully installed ... vapi-runtime-2.40.0 vcenter-bindings-4.1.0 ... vsphere-automation-sdk-1.82.0
======================= 58 passed, 150 warnings in 5.06s =======================

One finding worth recording. The wheelhouse came to 23 wheels, and the private closure is larger than the single pinned package — vapi_runtime, vapi_common_client, vcenter_bindings, vmc_client_bindings, vmc_draas_client_bindings and the four nsx_* packages are not on public PyPI either. Downloading the full dependency closure rather than --no-deps is therefore load-bearing, not incidental.

Also note the first attempt failed, which is worth keeping visible: /usr/bin/python3 does not exist in bci/python:3.13 (it ships /usr/bin/python3.13, and python3 on PATH is a /usr/local/bin shim). The interpreter is now resolved from a candidate list and then rejected if it resolves inside GITHUB_WORKSPACE — which is what makes "system interpreter, not the PR-built venv" an actual guarantee rather than an assumption about PATH. Both branches are verified inside the pinned image.

That failure also exercised the error path for real: the EXIT trap revoked the credential even though the download never completed.

@LouisParkin
LouisParkin merged commit cd1e292 into STAC-25463-integrations-github-ci Aug 5, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant