Chore/merge gsoc

[jacob50231]

New Features

• Enhanced Asynchronous Multiple File Download: Introduces a high-performance async_download_multiple method utilizing hybrid concurrency (multiprocessing + asyncio) with memory-efficient queue-based management
• New CLI Command: Adds gen3 download-multiple-async command with advanced options like resuming downloads (--skip-completed), flexible filename formatting, and progress bars
• Performance Testing Suite: Added comprehensive benchmarking framework comparing the new implementation against CDIS Data Client for download speed, memory usage, and concurrency efficiency

Dependency updates
Updated fastavro from 1.8.4 to 1.11.1
Updated pypfb to include extras: pypfb = {extras = ["gen3"], version = "^0.5.33"}
Updated importlib-metadata from 8.5.0 to 4.13.0

[github-actions]

The style in this PR agrees with black. ✔️

This formatting comment was generated automatically by a script in uc-cdis/wool.

[github-actions]

Integration Tests

Test summary after running integration tests

filepath	passed	failed	skipped	SUBTOTAL
tests/test_dbgap.py	4	0	1	5
tests/test_ras_passport.py	0	0	2	2
tests/test_data_upload.py	8	0	1	9
tests/test_graph_submit_and_query.py	11	2	1	14
tests/test_presigned_url.py	7	0	0	7
tests/test_centralized_auth.py	5	0	0	5
tests/test_google_data_access.py	1	0	0	1
tests/test_audit_service.py	1	0	0	1
tests/test_drs_endpoint.py	2	0	0	2
tests/test_gen3_sdk.py	1	0	0	1
TOTAL	40	2	5	47

Test summary after rerunning failed integration tests

filepath	passed	failed	SUBTOTAL
tests/test_graph_submit_and_query.py	1	1	2
TOTAL	1	1	2

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

[github-actions]

Integration Tests

Test summary after running integration tests

filepath	passed	failed	skipped	SUBTOTAL
tests/test_dbgap.py	4	0	1	5
tests/test_ras_passport.py	0	0	2	2
tests/test_data_upload.py	8	0	1	9
tests/test_graph_submit_and_query.py	12	1	1	14
tests/test_presigned_url.py	7	0	0	7
tests/test_centralized_auth.py	5	0	0	5
tests/test_google_data_access.py	1	0	0	1
tests/test_audit_service.py	1	0	0	1
tests/test_drs_endpoint.py	2	0	0	2
tests/test_gen3_sdk.py	1	0	0	1
TOTAL	41	1	5	47

Test summary after rerunning failed integration tests

filepath	passed	SUBTOTAL
tests/test_graph_submit_and_query.py	1	1
TOTAL	1	1

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

[github-actions]

Integration Tests

Test summary after running integration tests

filepath	passed	failed	skipped	SUBTOTAL
tests/test_dbgap.py	4	0	1	5
tests/test_ras_passport.py	0	0	2	2
tests/test_data_upload.py	8	0	1	9
tests/test_graph_submit_and_query.py	12	1	1	14
tests/test_presigned_url.py	7	0	0	7
tests/test_centralized_auth.py	5	0	0	5
tests/test_google_data_access.py	1	0	0	1
tests/test_gen3_sdk.py	1	0	0	1
tests/test_audit_service.py	1	0	0	1
tests/test_drs_endpoint.py	2	0	0	2
TOTAL	41	1	5	47

Test summary after rerunning failed integration tests

filepath	passed	SUBTOTAL
tests/test_graph_submit_and_query.py	1	1
TOTAL	1	1

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

[SpencerAxelrod]

Is there supposed to be a timeout around here? I see a commit called "removed timeout" in the history. But also, should we not send a better completion signal here (instead of a timeout)?

[SpencerAxelrod]

like have _guid_producer put one None (input_queue.put(None)) per worker after the GUIDs. Have this loop break when it reads one.

[SpencerAxelrod]

if every worker dies before producing all len(guids) results, this blocks forever instead of erroring out and the except Empty right below never gets hit