Repository navigation
fix(ci): repair end-to-end and integration tests - #50
Merged
Merged
Conversation
Cog 0.2x keeps build state in .cog/build inside the project directory. Parallel cog runs in one directory delete each other's state and fail with "open .cog/build/Dockerfile: no such file or directory". The e2e tests run under xdist and share fixtures such as base and image-base.
The test image lived in a GCS bucket that now returns 403. Use the PM5544 test pattern from multimedia-example-files.replicate.dev, the host the fuzzer already depends on. Update the AI prompts and the dimensions to match the new file.
The e2e job installs the latest Cog release. Printing the version makes a Cog-caused failure easy to spot in the log.
Fails with ruff 0.16 and later. Their default rule set flags B008 on `image: Path = Input(...)`.
Fails when Replicate wraps a single URI output in a one-item list.
Ruff 0.16 expanded its default rules from 59 to 413. Models without a ruff config now fail the predict.py lint on idiomatic Cog code such as `image: Path = Input(...)`. Select the pre-0.16 defaults explicitly. User ignores and per-file ignores still apply.
lucataco
force-pushed
the
lcatacora/fix-ci
branch
from
November 5, 2026 15:56
a3173f9 to
b33ca99
Compare
Replicate returns a single URI output as a bare string on some predictions and as a one-item list on others, for the same version. That made CheckOutputsMatch fail with "The types of the outputs don't match" at random. Unwrap the list when the Output schema type is not array.
Cleanup in two e2e runs failed with an empty-body 500 from
DELETE /models/{owner}/{name}/versions/{id} after the test body had
passed. Retry with a delay, and treat 404 as already deleted.
lucataco
force-pushed
the
lcatacora/fix-ci
branch
from
November 5, 2026 16:29
a6b524d to
527ffe7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
CI on
mainhas been red, so a red check tells nobody anything. This PR makesend-to-end-testandintegration-testpass again by fixing each cause at its source. No test is skipped or loosened.What changed
fast-car.jpgnow returns 403. The tests readphilips-pm5544.768x576.retro-tv-test.pngfrommultimedia-example-files.replicate.dev, the host the fuzzer already uses. The AI prompts and sizes match the new file..cog/buildunder the project. Twocog pushruns in one fixture directory delete each other's state. Each e2e test now works in its own copy of the fixture.lint_filethen rejected idiomatic code such asimage: Path = Input(...)(B008) in any model without a ruff config. The repo's ownruff.tomlhid this in CI.lint_filenow selectsE4,E7,E9,F, the old defaults.CheckOutputsMatchthen failed with "The types of the outputs don't match".predictnow unwraps a one-item list when theOutputschema is not an array.Scope
mainfailures have expired, so I could not check that they had the same causes.actions/checkout@v3Node 20 warning and the CogERROR: ... no access allowed to dir "context"build noise alone. Builds succeed with the noise.Tradeoffs
raw.githubusercontent.comkeeps the content but needs a commit-pinned URL. The Replicate example host needs no pinning.Blast Radius
The two product changes affect users of the tool.
lint_fileis more lenient than ruff's new defaults and equal to the old ones.predictchanges only a one-item list under a non-array schema. Everything else is test code and CI. Leavingmainred keeps hiding real regressions.Verification
test_lint_file_accepts_idiomatic_cog_defaultsand the one-item list case oftest_predict_output_follows_output_schema..cog/buildrace locally with Cog 0.23.1. Acog buildthat started while acog predictfinished in the same directory failed 3 of 3 times. In separate copies of the fixture it passed 2 of 2 times.end-to-end-testat 8 passed andintegration-testat 93 passed: https://github.com/replicate/cog-safe-push/actions/runs/42064940453. The previous head also passed twice in a row, in attempt 1 and attempt 2 of https://github.com/replicate/cog-safe-push/actions/runs/42061494182.