fix(cli): honor --unzip for single file downloads and add it to competitions download

[aadi-joshi]

Summary

--unzip did nothing when downloading a single file with -f, and kaggle competitions download had no --unzip at all. Both halves are what #158 asks for, and the single file case is also what is reported in #9.

Problem

Kaggle serves large dataset files inside a zip archive named after the request, so creditcard.csv arrives as creditcard.csv.zip. That is the case --unzip exists for, and it was the one case the flag did not cover:

kaggle datasets download mlg-ulb/creditcardfraud -f creditcard.csv -p out --unzip

Downloading creditcard.csv.zip to out

out/creditcard.csv.zip was 69 MB and still zipped, and the command exited 0. dataset_download_cli forwarded unzip to dataset_download_files but not to dataset_download_file:

if file_name is None:
    self.dataset_download_files(dataset, path=path, unzip=unzip, force=force, quiet=quiet, licenses=licenses)
else:
    self.dataset_download_file(dataset, file_name, path=path, force=force, quiet=quiet, licenses=licenses)

Separately, --unzip was registered only on the datasets parser, so a competition bundle always had to be extracted by hand.

Solution

unzip is forwarded to dataset_download_file, the flag is added to the competitions download parser and threaded through both competition download paths, and one _extract_and_remove_zip() helper now serves all four call sites, replacing the extraction block that was inlined in dataset_download_files.

Extraction of a single file is limited to a response named exactly <requested>.zip:

def _is_auto_compressed(outfile: str, file_name: str) -> bool:

A download is never renamed, so a file that is genuinely a .zip inside a dataset does not match and is left alone. A competition bundle served in any format other than zip is also left as is.

zipfile.extractall() drops .. segments and absolute prefixes from member names, so extraction stays inside the destination. There is a test for that.

The helper reuses the existing ISSUE_TRACKER_URL constant instead of the two hardcoded variants of the repository URL that were in the old block, one of which pointed at the repository root rather than the issue tracker.

Behavior

kaggle datasets download mlg-ulb/creditcardfraud -f creditcard.csv -p out --unzip
# out/creditcard.csv, 150828752 bytes, no archive left behind

kaggle competitions download titanic -p out --unzip
# out/train.csv, out/test.csv, out/gender_submission.csv

Without --unzip nothing changes. An archive that is already on disk is extracted without being downloaded again, which matches how a full dataset download has behaved since #1086.

Testing

• hatch -e test run pytest tests/unit/test_download_unzip.py -v (26 passed)
• hatch -e test run pytest tests/unit (1336 passed, 3 skipped)
• hatch run lint:all
• Verified against the live API: the single file case above extracts to 150828752 bytes with no archive left; competitions download --unzip on titanic writes the three csv files and no zip; a full dataset download with --unzip still writes 68 files and no zip through the shared helper; a cached archive is extracted without a second download; and a plain uncompressed file, a competition bundle download without the flag, and a competition single file with the flag are all unchanged
• Maintainer: please run /gcbrun when convenient

New tests cover the wrapper rule, a file that is itself a .zip, an uncompressed response, a nested request, a cached archive, the bundle case with and without the flag, a bundle that is not a zip, extraction that would escape the destination, a corrupted archive, a missing archive, and that both CLI wrappers forward the flag. The eight behavior tests fail against the previous code.

Docs updated in docs/datasets.md, docs/competitions.md, skills/references/datasets.md, and skills/references/competitions.md.

Related

Fixes #158
Fixes #9

[sridipbasu]

Could we also handle the case where the ZIP contains the full path that was asked for? For example, if file_name is WICAgencies2014ytd/Food_Costs.csv and the ZIP contains that same path, extracting into dirname(outfile) could result in WICAgencies2014ytd/WICAgencies2014ytd/Food_Costs.csv.

The current nested-path test only uses Food_Costs.csv in the ZIP. Can we add a test for this case, or can we be sure that the API always returns the basename here?

[aadi-joshi]

I had assumed the base name rather than checked it, so I verified it. Reading the local file header of a wrapper over a range request, which avoids pulling the file:

requested 'cord_19_embeddings/cord_19_embeddings_2022-06-02.csv'
  served   'cord_19_embeddings_2022-06-02.csv.zip'
  member   'cord_19_embeddings_2022-06-02.csv'

So the archive holds the base name today and the double nesting does not happen. That is not something worth depending on, so I removed the dependency in 9196756 rather than documenting it.

_extract_and_remove_zip now takes unwrap_to. When the archive is the single file wrapper, its one member is written to that exact path instead of to the name recorded inside the archive, so the result is the same whichever the server stores. An archive with any other number of members is still extracted as a bundle.

Three tests cover it. One builds an archive holding WICAgencies2014ytd/Food_Costs.csv and asserts it lands at Food_Costs.csv, one asserts end to end that no WICAgencies2014ytd/WICAgencies2014ytd appears, and one covers the multi member fallback. The first two fail against the previous extractall.

[stevemessick]

/gcbrun