Raise a failed download thread's error from iteration and reject a cache_limit that local-only shards would fill

[avgvi]

Problem

When one of the two threads behind __iter__ (download, ready) raises, the error never reaches the caller. on_exception re-raised inside a futures done-callback, where only the concurrent.futures logger sees it, and the thread skipped its it.on_exit(). Depending on timing, the epoch ended early with no error, or _Iterator.exit() waited forever for a thread that was already dead.

#27 made _evict_coldest_shard raise instead of deleting shards of a stream with no remote, which turned a silent data loss in 0.13.0 into this: a cache_limit too tight for a mix of local-only and remote streams hung in 6 of 10 runs and returned an empty epoch in the other 4.

Fix

• _run_thread wraps the download and ready threads: on any exception it records the error, sets the event, calls on_exit() for the dead thread and re-raises.
• _raise_if_background_failed raises a RuntimeError chained to that error in get_item, at the start of __iter__ and after the yield loop, so a failed thread ends the epoch with its exception. on_exception is removed.
• Construction rejects a cache_limit that cannot hold the index files, the local-only shards and four times the largest shard, and the message gives the numbers. The mixed case fails there now.

Tests

• test_background_thread_failure_raises: a download thread that raises ends iteration with a chained RuntimeError, and the next epoch reports it too.
• test_cache_limit_leaves_room_above_local_only_streams: one byte under the bound fails at construction; at the bound the epoch completes and the local-only shards stay.
• test_streaming, test_eviction and test_barrier pass on Linux 3.12 with Fix forked DataLoader workers hanging after the parent iterated the dataset #48 included (228 tests).

Found by the differential test of 0.14.1 against 0.13.0.

[read-the-docs-community]

Documentation build overview

📚 streaming-community | 🛠️ Build #35084337 | 📁 Comparing fe3ea85 against latest (2637c52)

  🔍 Preview build  

3 files changed

± genindex.html
± api_reference/generated/streaming.StreamingDataset.html
± _modules/streaming/base/dataset.html