Skip to content

I209 safer downloads - #210

Open
nhsavage wants to merge 6 commits into
mainfrom
i209_safer_downloads
Open

nhsavage wants to merge 6 commits into
mainfrom
i209_safer_downloads

Conversation

@nhsavage

Copy link
Copy Markdown
Contributor

closes #209 - adds temporary filenames and also checks if files are zero size

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Coverage

Coverage Report
FileStmtsMissCoverMissing
lib/osop
   compare_terciles.py76760%9–335
   compute_products_func.py1341340%8–448
   compute_scores_func.py1541540%17–414
   ens_plotting.py1425462%112–148, 240, 250, 318–337, 357–418
   plot_verify.py17714518%129–166, 198–289, 316–349, 379–474, 498–575
   pycpt_convert.py39428727%76–99, 116–122, 140, 163–169, 189–205, 219–225, 242–270, 290, 311–313, 340–531, 554, 569–571, 663, 736–744, 759–765, 776–791, 796–799, 804–808, 816–827, 843–897, 917–971, 1009–1044
   run_pycpt.py1261260%11–362
   util.py801581%91, 105, 133, 161–174, 181–186, 204, 228–229
lib/osop/test
   test_util.py98298%151, 155
TOTAL165999340% 

Tests Skipped Failures Errors Time
116 0 💤 0 ❌ 0 🔥 15.290s ⏱️

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Existing zero-byte destinations cannot be replaced on Windows because all three download paths use os.rename.

1 open finding
What changed in this PR

Addresses #209 by staging ERA5 and hindcast downloads in temporary files to avoid exposing partial downloads.

Changes:

  • Downloads to .part files and retries existing zero-byte destinations.
  • Adds tests for skipping, downloading, and ERA5 retrieval failures.
  • Enables forecasts in the default configuration.
File Description
scripts/​tests/​test_get_era5.py Tests ERA5 download and failure handling.
scripts/​tests/​test_get_any_hindcast.py Tests both hindcast download modes.
scripts/​get_era5.py Adds temporary downloads and zero-byte retries.
scripts/​get_any_hindcast.py Adds temporary downloads and zero-byte retries in both modes.
osop_config.yml Enables the forecast workflow.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/get_era5.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Shared temporary filenames can publish incomplete data when downloads overlap.

2 open findings

🧠 Review effort: Balanced

Comment thread lib/osop/util.py Resolved
@nhsavage
nhsavage requested a review from Eleanor-Dean October 16, 2026 12:51

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

implement temporary files for downloads

2 participants