Skip to content

test: fix flaky template download and build lookup failures - #1457

Open
l2ysho wants to merge 10 commits into
masterfrom
claude/fix-flaky-tests-0000b8
Open

l2ysho wants to merge 10 commits into
masterfrom
claude/fix-flaky-tests-0000b8

Conversation

@l2ysho

@l2ysho l2ysho commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Note

TL;DR Fixes the 3 open apify-cli flakes from the flake triage log (2026-09-24 sweep).

Changes

  • Retry transient template fetches (src/lib/utils.ts, src/commands/create.ts): the manifest fetch retries on any error. The archive download retries on network errors and HTTP 408/429/5xx. 3 attempts, 1s then 2s delay. Other 4xx errors still fail at once.
    • Fixes python-scrapy-template-works.test.ts (4/48) and python/works-with-spaces-in-path-to-actor.test.ts (2/48). Also fixes a single ENOENT sighting in propagates-non-zero-exit-code.test.ts.
    • Evidence: in every failing job, the whole test file took 265–770ms. A normal run takes 12s or more, because create installs dependencies. The ENOENT paths (my_actor/main.py, src/) show the archive was never unzipped. Vitest runs silent, so the create error never reached the log.
  • Wait for the latest build (test/__setup__/build-utils.ts): call.test.ts and task/run.test.ts now poll taggedBuilds.latest (60s limit) and do not read builds().list() right after push. That list can still be empty, which caused lastBuild to be undefined in call.test.ts (2/48).
  • Unit test for the download retry.

Follow-ups

  • push.ts, templates/ls.ts and wrapScrapyProject.ts also call fetchManifest() without a retry. I left them out to keep this diff small.

🤖 Generated with Claude Code

- Retry transient failures when fetching the template manifest and
  downloading archives. A failed fetch made `apify create` exit early,
  so tests failed later with ENOENT or a missing dataset.
- API tests wait for the Actor's `latest` build instead of reading the
  builds list right after push, which can still be empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@l2ysho l2ysho added adhoc Ad-hoc unplanned task added during the sprint. t-builders Issues owned by the Builders team. labels Sep 24, 2026
@pkg-pr-new

pkg-pr-new Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npx https://pkg.pr.new/apify-cli@1457

commit: d64e773

@l2ysho
l2ysho marked this pull request as ready for review November 6, 2026 12:02
@l2ysho
l2ysho requested a review from DaveHanns as a code owner November 6, 2026 12:02
@l2ysho
l2ysho requested review from apify-factory and removed request for DaveHanns November 6, 2026 12:02
@apify-service-account apify-service-account added the tested Temporary label used only programatically for some analytics. label Nov 6, 2026

@apify-factory apify-factory left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, nothing blocking.

🏭 Factory review — ℹ️ not auto-approved

This change was reviewed but cannot be auto-approved: a gate failed, so the review cannot end in an approval.

check result
Static gates ❌ 15/16 — no-risky-content
Review findings ✅ none
Reviewer escalation ❌ Adds new retry behavior with delays to production network calls (template manifest fetch and every downloadZip caller, including actors pull), which is a runtime retry change beyond test-only fixes.
Juror 1 — claude-sonnet-5-5, adversarial ⏭️ skipped — a gate failed
Why it was not auto-approved

Failed gates:

  • no-risky-content: src/lib/utils.ts: network call (network-call); test/local/lib/utils.test.ts: network call (network-call)

The comments above are the review itself; this section only explains the approval decision. Push a change and request @apify-factory again to re-run it, or ask a human to review.

Reviewed f2fd44cde · workflow run

@l2ysho
l2ysho requested a review from DaveHanns November 6, 2026 12:06

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-builders Issues owned by the Builders team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants