Skip to content

Add download --skip-existing to resume a partial download - #71

Open
reacher-z wants to merge 1 commit into
mainfrom
feat/skip-existing
Open

reacher-z wants to merge 1 commit into
mainfrom
feat/skip-existing

Conversation

@reacher-z

Copy link
Copy Markdown
Collaborator

Closes #10

After an interrupted download, rerunning websitebench download stopped at the first task directory that already existed. --skip-existing checks each existing directory against the release hash:

  • it matches: SKIP <task> (matches the release);
  • it differs: an error naming the task, with pass --force to replace it;
  • the rest are downloaded as usual.

--skip-existing and --force are mutually exclusive. docs/pypi.md and docs/running.md describe the flag.

Tests

Four new tests: matching task skipped, differing task rejected, missing tasks downloaded, and the flag conflict.

  • uv run python -m unittest discover -s tests
  • uv run ruff check scripts src tests
  • uv run ruff format --check scripts src tests

Rerunning websitebench download after an interruption failed on the first
task directory that already existed, so finishing a 70-task download
meant working out the missing names by hand or downloading everything
again with --force. --skip-existing checks each existing task against
the release hash, keeps it when it matches, stops with an error when it
differs, and downloads only the rest.
@reacher-z reacher-z added enhancement New feature or request cli The websitebench command and download path labels Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli The websitebench command and download path enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Resume a partial websitebench download without --force

1 participant