Skip to content

Download: add a 'resume' option - #105

Open
fingolfin wants to merge 1 commit into
mh/download-target-cleanupfrom
mh/download-resume
Open

Download: add a 'resume' option#105
fingolfin wants to merge 1 commit into
mh/download-target-cleanupfrom
mh/download-resume

Conversation

@fingolfin

@fingolfin fingolfin commented Aug 12, 2026

Copy link
Copy Markdown
Member

An interrupted download starts again from zero. For a data set of any size
that is the difference between a retry and an afternoon.

With resume := true and target given, the via curl method continues the
partial file (-C -), and a failure keeps that file so the next attempt can
carry on.

Methods that cannot resume decline rather than proceed, because they run
before via curl and would destroy the file it needs: wget's -c does not
resume when combined with -O — the output file is always created anew — and
SingleHTTPRequest has no range support. Measured before adding the decline:

via SingleHTTPRequest   file=fail     (deleted)
via wget                file=fail     (truncated by -O, then removed)
via curl                file=01234    (kept)

If no resuming method is available the download fails and the caller can
retry without resume.

The test server answers a Range request with the remainder in upper case, so
the test distinguishes a resumed download from a restarted one; removing the
-C - makes it fail with did not resume: abcdefghijklmnopqrst.

Based on #103, which introduced the "no target file after a failure"
guarantee that resume has to except.

Written with Claude Opus 5 via Claude Code; reviewed by me.

CC @ThomasBreuer

A download interrupted halfway currently starts again from zero, which for a
data set of any size is the difference between a retry and an afternoon.

With 'resume' set and a target given, the curl method continues the partial
file via '-C -', and a failure keeps that file so the next attempt can carry
on from it.

Methods that cannot resume decline the request rather than proceeding: wget's
'-c' does not resume when combined with '-O', and SingleHTTPRequest has no
range support, so both would truncate or delete the very file the curl method
needs. They run first, so without this a resumed download would never see its
partial file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.38%. Comparing base (b7cd336) to head (a7c6643).

Additional details and impacted files
@@                      Coverage Diff                       @@
##           mh/download-target-cleanup     #105      +/-   ##
==============================================================
+ Coverage                       86.23%   86.38%   +0.14%     
==============================================================
  Files                              32       32              
  Lines                            1918     1939      +21     
==============================================================
+ Hits                             1654     1675      +21     
  Misses                            264      264              
Files with missing lines Coverage Δ
lib/download.gi 93.16% <100.00%> (+0.67%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cdwensley

Copy link
Copy Markdown
Collaborator

@fingolfin Be sure to tell me when these four (or more?) PRs are ready to be merged and a release made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants