Run the integration suite in CI - #489
Open
pksgit wants to merge 5 commits into
Open
Conversation
Nothing behind the integration build tag has ever run on a pull request. The integration job builds the ingress image and stops there, and the test image copies only pkg/ and version/, so test/ is neither compiled nor linted and a build error there goes green. Copy cmd/ and test/ into the test image, build the ingress binary the suite spawns as its handler, and compile the suite ahead of time so a break under test/ fails the image. The unit test job shares that image and so gains the same coverage. Add a workflow that runs the suite against a Redis and a livekit-server in dev mode, both as containers on the host network. Dev mode supplies the devkey/secret pair the generated config uses, so this needs no repository secrets and runs on a fork. Redis carries nothing else: another service registering an IOInfoServer on the same bus takes a share of the state updates and the ingress under test never reaches a terminal state. The matrix covers rtmp and url through the existing rtmp_only and url_only config flags. WHIP needs the livekit-whip-bot submodule built into the image first. CS-2036 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
milos-lk
reviewed
Sep 8, 2026
milos-lk
reviewed
Sep 8, 2026
Contributor
|
what do you think about have a docker compose file like this: The generated config then points at service names and it would allow us to easily share it between the CI workflow |
The workflow started Redis and a room server with docker run, polled each with its own retry loop, and built the image in a separate step. A compose file carries all of that, and the same file runs on a developer's machine through mage IntegrationDocker, so a local pass and a green check mean the same thing. Integration still runs the suite natively against Homebrew GStreamer, which is a different environment from CI. The services find each other by name on a compose network rather than sharing the host's. That is why the room server needs an explicit bind: dev mode listens on loopback only, so without it every other container is refused, and a healthcheck on localhost reports the service up while nothing can reach it. CS-2036 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
.gst-version arrived on main and the test Dockerfile now takes GSTVERSION as a build arg with no default, so a build without it fails at parse time rather than using a stale base image. The compose file and the workflow pass it, read from the pin the same way test.yaml reads it, and mage takes it from the getGstVersion helper that came with it. A GStreamer bump is now a reason to run this suite, so .gst-version joins the paths filter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
compose run starts whatever image already exists and only builds when there is none, so a second run after a source or Dockerfile change tests the previous build. A local run silently passed against an image two GStreamer versions behind the pin. CS-2036 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
.go-version, go.sum and version/ are copied into the test image but were missing from the paths filter, so a pull request touching only those got no integration run at all rather than one that passed. Two such changes landed on main last week: a Go toolchain bump that touched .go-version alone, and a grpc security bump that touched go.mod and go.sum. CS-2036 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CS-2036. Nothing behind the
integrationbuild tag has ever run on a pullrequest, and
test/is not even compiled, so a build error there goes greentoday. Found while adding an integration test in #480.
Two independent gaps
integrationjob inbuild.yamlhas a single step,docker build -f ./build/ingress/Dockerfile. It never invokesgo test -tags integration.build/test/Dockerfilecopies onlypkg/andversion/, sotest/is neither compiled nor linted..golangci.yamlalready setsbuild-tags: [integration], so the config was right all along; the files were never in the image for it to read.What this does
build/test/Dockerfilealso copiescmd/andtest/, builds theingressbinary the suite spawns as its handler, and compiles the suite ahead of time so a
break under
test/fails the image. The existingTestjob shares that imageand gains the same coverage for about 20 seconds.
build/test/entrypoint.shgains anINTEGRATIONbranch that runs the prebuiltsuite. Without it the unit path runs exactly as before.
test-integration.yamlruns the suite against a Redis and alivekit-serverin dev mode, both containers on the host network. Dev mode supplies the
devkey/secret pair the generated config uses, so this needs no repository
secrets and works on a fork. That is a deliberate departure from the egress
workflow, which takes its config from
secrets.EGRESS_CONFIG_STRINGpointing ata remote room server.
Redis carries this suite and nothing else, on purpose: any other service
registering an
IOInfoServeron the same bus takes a share of the state updatesand the ingress under test never reaches a terminal state. A separate Redis
database does not help, since psrpc rides pub/sub and pub/sub is not scoped to
one. That comment is in the file so nobody folds it into a shared service later.
The matrix covers
rtmpandurlthrough the existingrtmp_onlyandurl_onlyconfig fields, so no new Go code. AnINTEGRATION_TYPEselector ofthe kind egress has belongs with the harness refactor in CS-2037.
Verified locally
docker build -f build/test/Dockerfile, lint stage uncachedtest/in scopetest/// recieveintest/url.go, rebuilt--target lint: failed withmisspell, exit 1go test -c -tags integration ./test/ingresson PATH,test.testpresentflvmux,rtmp2sink,audiotestsrc,faac,videotestsrc,x264encall presentINTEGRATIONunset)go test ./pkg/..., 36 tests, 0 failuresrtmpleg with this workflow's configurllegDraft, because one thing cannot be checked locally
Whether
--network hostbehaves as expected onubuntu-latest, and whether alivekit-servercontainer completes ICE with a handler in a sibling containersharing that network. Both legs above ran natively against a local Redis and
room server, which exercises the config, the credentials and the publishers, but
not the container topology. This PR's own run settles it, since the paths filter
matches the files it changes.
Worth knowing before review
Green here means "did not error", not "media verified". Both tests end on
require.NotEqual(t, final.State.Status, ENDPOINT_ERROR)and each run produced asingle state update. Media did flow in both local runs, but nothing asserts it.
Pre-existing, and the clearest argument for the
checkUpdatehelper in CS-2037.The
urlleg reachesdevimages.apple.com, whichRunURLTesthas pulled sinceSupport for pulling URLs (#138). It is the only external URL in the Go testcode, and lighter than what egress already depends on.
WHIP is left out: its publisher is the
livekit-whip-botsubmodule, which has tobe checked out and built into the image first. Stated in the workflow rather than
left implicit.
redis:7-alpineandlivekit/livekit-server:v1.13.6are version-pinned ratherthan digest-pinned, unlike the action SHAs and
GSTVERSION.🤖 Generated with Claude Code