Skip to content

Download: never leave the target file behind after a failure - #103

Open
fingolfin wants to merge 1 commit into
masterfrom
mh/download-target-cleanup
Open

Download: never leave the target file behind after a failure#103
fingolfin wants to merge 1 commit into
masterfrom
mh/download-target-cleanup

Conversation

@fingolfin

Copy link
Copy Markdown
Member

When opt.target is set and the download fails, the three methods disagree:

  • via wget removes the partial target file;
  • via curl leaves it;
  • via SingleHTTPRequest writes the response body -- a 404 error page, say --
    to the target and then returns success := false.

So whether the file exists after a failed download, and what is in it, depends
on which method happened to be available. A caller cannot write correct
cleanup code against that, and one that tests IsExistingFile( target ) to
decide whether it has the data will accept an error page.

tst/download.tst already recorded the problem, in a comment:

the backends do not behave consistently in the case of failure ... which
makes them useless as automatic tests

This fixes via curl and via SingleHTTPRequest to match via wget, and
additionally removes the target in Download itself after any method reports
failure, so the guarantee also covers methods added to Download_Methods from
outside the package. Documented under the target option.

On the test

The test drives each available method directly rather than going through
Download. That is deliberate: through Download the bug hides whenever a
well-behaved method runs after a leaking one. On my machine wget runs after
SingleHTTPRequest and cleans up, so a Download-level test passed even
without the fix. Driven directly, it reports

leftover target file after via SingleHTTPRequest (from the IO package)

gap --packagedirs $PWD tst/testall.g: 0 failures in 22 files.

One question

via wget raises Error( "Download cannot remove unwanted file ", ... ) if
its own RemoveFile fails. I left that alone -- it is loud rather than leaky
-- but it now makes wget the only method that can turn a failed download into
a break loop. Happy to make it consistent with the others if you prefer.

Found while writing https://github.com/gap-packages/ArtifactManager, which
deletes the target before every attempt to work around this.

CC @ThomasBreuer

The wget method removed a partial target file, the curl method did not, and
the SingleHTTPRequest method wrote the response body -- an error page, say --
to the target and then reported failure.  So whether the file existed after a
failed download, and what was in it, depended on which method happened to be
available.

Fixed in curl and SingleHTTPRequest, and enforced in 'Download' as well so it
holds for methods added to 'Download_Methods' from elsewhere.  Documented.

The new test drives each available method directly; going through 'Download'
alone hides the bug whenever a well-behaved method runs last.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.25000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.23%. Comparing base (a4c23a8) to head (b7cd336).

Files with missing lines Patch % Lines
lib/download.gi 81.25% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #103      +/-   ##
==========================================
- Coverage   86.24%   86.23%   -0.02%     
==========================================
  Files          32       32              
  Lines        1905     1918      +13     
==========================================
+ Hits         1643     1654      +11     
- Misses        262      264       +2     
Files with missing lines Coverage Δ
lib/download.gi 92.48% <81.25%> (-0.52%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant