STAC-25463 Add GitHub Actions check-test workflow (integrations CI migration phase 1)

[LouisParkin]

Jira: https://stackstate.atlassian.net/browse/STAC-25463 (parent: STAC-25142)

First phase of moving stackstate-agent-integrations CI off GitLab. Purely additive — .gitlab-ci.yml is untouched and keeps running, so this can land without a cutover.

What moved

GitLab	GitHub Actions
the test_<check> job family (20 near-identical jobs)	one check-tests matrix
per-job changes: rules	.github/scripts/select-checks.sh (plain git diff vs merge base)
checksdev validate * riding inside test_stackstate_checks_base	its own validate job
services: docker:24-dind + DOCKER_HOST: tcp://docker:2375	dropped — both public ARC runners already ship a DinD sidecar

Path filtering is done in git rather than a path-filter action because StackVista enforces a strict third-party action allowlist, and git diff against the merge base is precisely what the GitLab rule meant.

Scoped out (each reasoned in the workflow header)

• The 5 Docker-dependent suites — splunk_{base,health,metric,topology} + stackstate_checks_dev. These are the only suites needing a Docker daemon. Phase 2; the runners already provide DinD, so it is about exercising that path, not provisioning it.
• test_postgres — dead config. It tests a postgres check that does not exist in this repository. Dropped, not pending.
• print_env — not ported at any phase. It is a bare printenv, and this repo is public, so it publishes every CI credential in scope to a world-readable log. Same hazard class as the sign_debian_package.sh finding on the agent.
• No Windows lane exists to port. .gitlab-ci.yml defines a .windows_env anchor that no job has ever referenced, and Windows is not a supported target. The orphaned conda_env.ps1 / windows_*.cmd helpers can retire with the GitLab pipeline (STAC-25464).
• Publishing + the runner-image docker job stay on GitLab until integrations gets its own publishing role (pulumi-infra section 7.4).
• Cerberus notify — unlike the agent, this pipeline has never had a notify job, so adding one is new capability, and CERBERUS_LAMBDA_URL is a private-visibility org secret unreadable from a public repo. Follow-up.

Credentials

Phase 1 needs no new secret provisioning: REGISTRY_HOST / REGISTRY_USER / REGISTRY_PASSWORD are org-level visibility: all (STAC-25350) and already reach this public repo. Note the image ref composes vars.REGISTRY_HOST + /docker rather than using vars.REGISTRY_DOCKER_URL, which is private-visibility and not readable here.

Validation

• actionlint clean — added .github/actionlint.yaml declaring the self-hosted *-public labels

• zizmor clean (2 suppressed = the documented unpinned-images ignores on the container ref)

• shellcheck clean

• Selector exercised against real git history:

  scenario	selected
  push / dispatch	all 15
  PR touching only zabbix/	["zabbix"]
  PR touching kubelet/ + vsphere/	["kubelet","vsphere"]
  PR touching a shared library	all 15
  PR touching nothing relevant	[] (matrix skips)

This PR self-validates — the workflow runs against this very PR, so the image path and runner labels are proven before merge.

[LouisParkin]

[LouisParkin]

[LouisLotter]

Requesting changes for one credential-exposure issue and three CI migration gaps: a stable required gate, release-tag coverage, and BCI compliance.

[LouisParkin]

[LouisParkin]

[LouisParkin]

Follow-up found while this PR was waiting: branch protection here is broken independently of this change

While checking what actually gates merges on this repo, I found the branch protection is in a bad state, and it makes the ci-success job requested in review load-bearing rather than cosmetic. Filed as STAC-25522 (blocked by this PR).

Verified today:

Branch	Required checks	Effect
master	ci/gitlab/gitlab.com	Permanently unmergeable
legacy-agent-support	none	CI gates nothing
stackstate-7.51.1	none	CI gates nothing
stackstate-7.71.2	none	CI gates nothing
stackstate-7.78.2 (this PR's base)	none	CI gates nothing

The GitLab project was archived, so nothing can ever report ci/gitlab/gitlab.com again. master's HEAD commit status is pending with zero contexts, and PR #316 shows an empty check rollup while still sitting at BLOCKED — it can never merge. This is the same failure mode STAC-25420 already fixed for stackstate-process-agent.

The release branches have the opposite problem: they require nothing, so the workflow in this PR would run and be ignored by branch protection.

The fix (STAC-25522) is to adopt this repo into pulumi-infra — it is currently absent from github/repositories/resources.yaml entirely, so today's settings are unmanaged drift — and require the aggregate CI success context, mirroring the stackstate-agent entry.

Worth stating explicitly, because it is the reason the aggregate job exists: branch protection must not require the per-suite Check tests (<suite>) contexts. That job is a matrix whose membership is selected dynamically by .github/scripts/select-checks.sh, so those context names are not stable, and requiring them would hang any PR that legitimately skips a suite. ci-success gives protection exactly one fixed name to gate on.

Sequencing: STAC-25522 must land after this PR, otherwise protection would require a check that master does not yet produce — recreating the very deadlock it is meant to fix.

[LouisLotter]

Three follow-ups remain after the first review: private-PyPI credentials are still exposed to PR-controlled code, the aggregate gate succeeds when all substantive CI is skipped, and the direct BCI pull violates the self-hosted registry-proxy policy.

[LouisParkin]

Thanks @LouisLotter — both P1s were real. I've pushed 863b1cd which restructures the credential handling rather than patching around it. Details per finding:

P1 — credentials reachable by PR-controlled code ✅ fixed

You were right that the fork guard wasn't the boundary I implied. setup_artifact_registry.sh writes ~/.netrc, so scoping env: to that one step was cosmetic — every later step in the job, including the PR-authored test code we're about to execute, could read the file. A same-repo PR (the common case for us) had full access.

Fixed by making the split structural instead of procedural:

• select-checks.sh now emits two outputs — checks (14 suites, no credentials) and private_checks (vsphere, the only suite needing the private index).
• Those feed two separate jobs. check-tests no longer references any secret at all — the credential step is gone from it entirely.
• check-tests-private-index carries environment: private-package-index, a protected Environment with prevent_self_review: true and dev-team as required reviewer.

So the credential is no longer reachable from a job that runs untrusted code without a human first approving that specific run. Verified live on this PR's run — current_user_can_approve=false for me as the author, and the job is sitting in waiting.

I also made the registry setup fail loudly (exit 1) instead of warn-and-continue. Previously a missing secret produced a silent pass.

Residual, and I want to be honest about it rather than overclaim: the secrets are still repo-level (GITLAB_PACKAGE_REGISTRY_USER) and org-level (GITLAB_PACKAGE_REGISTRY_READONLY_PASSWORD). A future workflow edit could reference them from an ungated job. Making them environment-scoped requires an org-level change that also affects stackstate-agent, so I've left it — but any such edit is diff-visible in this file.

P1 — aggregate gate green when everything skips ✅ fixed

Confirmed and worse than described. The loop only failed on failure/cancelled, so skipped counted as success. Combined with the fork guards that were on every job, a fork PR could satisfy branch protection having run no CI whatsoever.

ci-success now distinguishes the two meanings of "skipped":

• select-checks, validate, workflow-security must be success. No skip is legitimate.
• check-tests may skip only when the selector legitimately chose zero suites.
• check-tests-private-index may additionally skip when the PR is from a fork.

Anything else fails the gate.

Related: I found this repo's branch protection currently requires the ci/gitlab/gitlab.com context, which can no longer ever report since the GitLab project was archived — master is presently unmergeable, with an empty check rollup that reads as "no checks" rather than an error. Filed as STAC-25522, to land after this PR so CI success exists as a target first. That makes this gate load-bearing, which is partly why I wanted the skip semantics correct.

P2 — BCI pulled directly rather than through the proxy 🔄 moot

This one I'd have pushed back on, but it's moot now: every job moved to ubuntu-latest. Once vsphere was the only suite needing credentials, there was no reason to keep 14 suites on self-hosted runners.

For the record on the original point — the merged precedent in stackstate-agent (build-binaries.yml:301, build-deb.yml:380) pulls BCI direct from registry.suse.com on self-hosted runners, with the comment "the BCI stages come from registry.suse.com and need no auth"; the proxy is used for the auth-requiring Docker Hub/Ubuntu stages. registry.tooling.stackstate.io appears in no GitHub Actions workflow in the org today. Happy to be corrected if that's drifted from current intent.

Also worth noting on the Dependabot sub-point: it isn't enabled on this repo, so there's nothing consuming that config today.

Runner move — it's faster

Somewhat to my surprise, GitHub-hosted beat the self-hosted lane on this workload, presumably because we weren't queuing behind other jobs:

	self-hosted (prev run)	ubuntu-latest (this run)
slowest suite	172s	99s
fastest suite	99s	68s

All 16 credential-free jobs green, no changes needed to the BCI container or zypper setup.

Validated with actionlint, shellcheck and Zizmor (No findings). select-checks.sh unit-tested across full-run / empty-selection / single-public / vsphere-only under set -u.

---

🙏 One ask: this PR's vsphere job is blocked awaiting a dev-team approval, and by design I can't approve my own run. If someone could approve the pending deployment, that'll confirm the credentialed path still works end-to-end before merge.

[LouisParkin]

Follow-up commit b1715b4 — comments only, no behaviour change, but it corrects a factual claim reviewers would otherwise rely on.

The header said "releases keep running from GitLab, so the GitLab pipeline stays live." That is no longer true. The GitLab project (stackvista/agent/stackstate-agent-integrations, id 13169244) reports archived: true, last activity 2026-07-20. Archived projects are read-only, so publish-checks-dev cannot run there either.

Two consequences worth being explicit about, because they change the deferred items from "scheduled" to "currently broken":

• the five unported suites (splunk_{base,health,metric,topology}, stackstate_checks_dev) have no CI on any platform right now;
• there is no way to publish a new stackstate_checks_dev until publishing has a home.

Neither is caused by this PR — this PR is what restores coverage for the other 15 — but the header shouldn't imply a safety net that isn't there.

I also dropped a second stale claim: phase 2 no longer rides on an ARC DinD sidecar, since every job now runs on ubuntu-latest, which already provides a Docker daemon. Phase 2 just needs a docker client in the job image.

Deferred work now carries ticket references rather than vague pointers:

STAC-25531	the five Docker-daemon suites
STAC-25532	publish-checks-dev
STAC-25533	Cerberus failure notification

actionlint, shellcheck and Zizmor clean; selector smoke-tested (14 + 1 unchanged).

[LouisParkin]

@LouisLotter apologies — you approved this at 08:19 and I knocked it straight back out at 08:37.

This base branch has dismiss_stale_reviews: true (and require_last_push_approval: true), so my follow-up push auto-dismissed your approval. That push was comments only — b1715b4, +23/−16 across the two CI files, no behaviour change. It corrects a claim in the header that I'd discovered was false: the header said "releases keep running from GitLab, so the GitLab pipeline stays live", but the GitLab project is archived (archived: true, last activity 2026-07-20), so the deferred jobs can't run there either. I also swapped the vague "phase 2 / a follow-up" pointers for STAC-25531/25532/25533.

Rebuilding your approval on a comment-only diff is a poor trade for your time, and I should have batched it with the review response instead of pushing separately. Re-requesting now.

Nothing else changed since you approved: still 16/16 credential-free jobs green, and the vsphere job is still gated behind the private-package-index environment.

One thing that still needs a second person regardless of the approval: the run is sitting in waiting on a dev-team approval for the vsphere deployment, and prevent_self_review means I can't release it myself. Whoever re-approves the PR can also approve that pending deployment on run 30989773234 — that's the last unproven path before merge.

[LouisParkin]

private-package-index: disabled prevent_self_review, and a better fix for the P1

@LouisLotter — heads-up on a change to the control I added for your P1 about credentials reaching PR-authored code, plus a proposal that I think closes it properly.

What changed: prevent_self_review on the private-package-index environment is now false. Required reviewers (dev-team) and the job's fork guard are unchanged.

Why. The gate as configured made the credentialed vsphere leg a required check that the PR author structurally could not satisfy. Because SHARED_PATHS includes the CI files, every CI-touching PR selects the full matrix including vsphere — roughly 16% of commits on this repo over the last year — so the author needed a second person on each one. That was my configuration choice, not something your review asked for.

The threat it was carrying is already handled twice for the case that actually matters:

• the job's if: requires head.repo.full_name == github.repository, so fork PRs skip it; and
• GitHub does not expose Actions secrets to fork pull_request runs on public repos at all.

So the environment gate only ever constrained the 27 accounts that already hold push access — and write access to a repo is already trust over its secrets under GitHub's model. Your narrower point stands (an in-repo branch is unreviewed at the time CI runs), but a self-approvable gate answers that with an audit record rather than an access boundary, which is what it was really providing.

The proper fix, which I'll do next. Your first suggested remedy — a credential-free read path — turns out to be the right one, just not by re-hosting. Context I got wrong earlier: vsphere-automation-sdk==1.82.0 is not something we patch. It is an unmodified upstream wheel that VMware withdrew from public PyPI, self-hosted in our GitLab registry only because that org was private. Upstream carries no SPDX license and the version predates their current tags, so publishing it publicly is a licensing question, not an engineering one — I'm not proposing that.

Instead I'll split the credential away from PR code:

1. a fetch-private-deps job that checks out no executable repo content and uses a hardcoded package spec, so nothing PR-authored can influence it, pulls the wheel from the GitLab registry, and uploads it with actions/upload-artifact;
2. the vsphere job downloads that artifact and installs --no-index offline.

The credential then never shares a job with PR-authored code, nothing is redistributed, the environment gate can be deleted, and vsphere rejoins the normal matrix — including on fork PRs, which currently skip it and lose that coverage.

I'll raise that as a separate PR rather than pushing here, since dismiss_stale_reviews + require_last_push_approval mean any further push to this branch re-dismisses your review.

[LouisParkin]

Status: fully green, and the environment gate is gone

@LouisLotter — #364 merged into this branch, so what you'll be reviewing has changed materially since your last pass. Summary of where your P1 now stands.

Your finding: setup_artifact_registry.sh wrote the registry password to ~/.netrc, which persisted for the rest of the job and was readable by the subsequent checksdev test and benchmark steps — i.e. by PR-authored code. You offered two remedies: a credential-free read path, or a protected environment with required approval.

I originally took the second. It worked but was the wrong trade: SHARED_PATHS covers the CI files, so it fired on ~28 of the last 176 commits (~1 in 6) and structurally blocked authors on their own PRs.

This branch now implements your first remedy instead. .setup-scripts/fetch_private_wheels.sh writes the netrc, downloads a hardcoded package set, deletes the netrc, and repoints ~/.pip/pip.conf at a local wheelhouse via find-links. The suite then installs offline with no credential on disk and no authenticated index configured. environment: private-package-index is removed.

Four details are load-bearing, and I'd welcome your eye on them specifically:

• the package set is hardcoded rather than read from the tree, so a PR cannot redirect pip while the credential is on disk;
• --only-binary=:all:, because downloading an sdist executes its setup.py;
• a system interpreter, resolved from candidates and then rejected if it resolves inside GITHUB_WORKSPACE — the toolchain venv is built by repo code, so its pip must never be in the credential's path;
• an EXIT trap revokes the credential on failure paths, then the script asserts the netrc is gone and the wheel actually arrived rather than silently falling through to the PyPI 0.0.1 placeholder.

Latest run 31009548529: all 19 jobs green, vsphere included, zero pending deployment approvals.

Two things I want to be straight about:

• Fork PRs still skip vsphere. I claimed earlier they'd regain coverage; that was wrong. Forks get no Actions secrets on a public repo, so they can't fetch the wheel at all. The fork guard stays.
• Residual risk is unchanged and stated in the workflow: a PR that edits the workflow itself can still reach the secret, because pull_request runs the PR's own copy. That's true of every secret in every repo and is contained by review, not CI. What changed is that ordinary test code and its dependency tree no longer see it — and that needed no workflow edit at all.

Also, apologies again for dismissing your approval this morning; that was my push, not you.

Ready for another look whenever suits.

[LouisParkin]

@LouisLotter — agreed on the call, and #365 is merged into this branch. Thread resolved.

Your one remaining condition, checked

The only reason to continue blocking would be if the GitLab token has permissions beyond read-only access to project 71271774.

Verified against the GitLab API rather than taken on trust. Project 71271774 is stackvista/ci/packages (private). Its deploy tokens:

token	active	scopes
CI Package registry access	revoked	read_package_registry, write_package_registry
gitlab-proxy	revoked	read_package_registry
gitlab-proxy-read-package-registry (13698947)	revoked	read_package_registry
gitlab-proxy-read-package-registry (13702516)	revoked	read_repository, read_package_registry
gitlab-proxy-read-package-registry (13702666)	active	read_package_registry

The only live token carries read_package_registry and nothing else — no write_package_registry, no read_repository. The one token that ever had write scope is revoked. So your condition is not met, and there's no reason to keep blocking. (Caveat for completeness: I can't read the secret's value, so I'm matching on the project's token inventory rather than proving which credential the secret holds. If it were a group/personal token instead of a deploy token, this wouldn't cover it.)

One thing worth recording that supports moving on: GITLAB_PACKAGE_REGISTRY_READONLY_PASSWORD is an org-level secret with visibility: all (orgVariables/resources_github.yaml:47), so every repo in the org already receives it. Gating it in this one repo could never have contained it — which is a further argument that the environment gate was buying less than it cost. This repo receives only the simple URL and the username; the write pair (GITLAB_PACKAGE_REGISTRY_PASSWORD) is not granted here.

Follow-up ticket filed as promised: STAC-25544 — move vsphere to the publicly-published VMware SDK packages (vmware-vapi-runtime, vmware-vapi-common-client, pyvmomi, all 9.1.0.0 on public PyPI, plus VMware's own public PEP 503 index for the NSX/VMC wheels). That deletes the credential, the check-tests-private-index job, and the PR coverage gap in one change. We're only on the private index because of the ancient vsphere-automation-sdk==1.82.0 pin.

Sorry — one more approval needed

Merging #365 pushed to this branch, so dismiss_stale_reviews + require_last_push_approval dismissed your approval for the third time. CI is re-running now. Nothing has changed since you approved except the #365 contents you already reviewed.

Worth fixing as part of STAC-25522 (branch protection): required_status_checks.contexts is currently empty on stackstate-7.78.2, so CI success isn't actually enforced as a required check yet — which is the thing that would let this settle down.