Skip to content

ci(storage): deflake connection_impl_test - #16517

Closed
scotthart wants to merge 1 commit into
googleapis:mainfrom
scotthart:storage_deflake_connection_impl_test
Closed

scotthart wants to merge 1 commit into
googleapis:mainfrom
scotthart:storage_deflake_connection_impl_test

Conversation

@scotthart

Copy link
Copy Markdown
Member

@scotthart
scotthart requested review from a team as code owners September 30, 2026 21:34
@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label Sep 30, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the 'HedgedReadRecordsMetricsOnGlobalMeterProvider' unit test in 'connection_impl_test.cc'. It replaces a generic '.WillRepeatedly' mock expectation with a sequence of explicit '.WillOnce' expectations to strictly verify the thread execution order (primary vs. hedge). Additionally, it removes the 'primary_closed' promise/future synchronization and instead flushes the single-threaded read pool at the end of the test using a synchronous read to guarantee that any background tasks and captured references are fully cleaned up before teardown. I have no feedback to provide.

@codecov

codecov Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.36%. Comparing base (78b69cf) to head (11d48d3).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16517      +/-   ##
==========================================
- Coverage   92.37%   92.36%   -0.02%     
==========================================
  Files        2262     2262              
  Lines      217100   217115      +15     
==========================================
- Hits       200548   200535      -13     
- Misses      16552    16580      +28     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

});
return StatusOr<std::unique_ptr<ObjectReadSource>>(std::move(source));
})
.WillOnce([primary_thread](auto&, auto const&,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI is failing because on slower runners, a second hedge timer can occasionally fire. If that happens, this extra hedge request consumes your 3rd .WillOnce() (which was meant for the final sync read) and trips the thread ID assertion since it runs on the background thread.

Comment thread google/cloud/storage/internal/connection_impl_test.cc
StatusOr<std::unique_ptr<ObjectReadSource>> sync_source =
client->ReadObject(ReadObjectRangeRequest("test-bucket", "sync"));
ASSERT_THAT(sync_source, IsOk());
ASSERT_THAT((*sync_source)->Read(buffer.data(), buffer.size()), IsOk());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, the "sync" read does not eliminate the teardown race because it still runs with hedging enabled, which means it will enqueue a new task on read_pool_ that captures self = shared_from_this(), and it does not flush hedge_pool_.

Since winning tasks on both pools unblock the caller before the worker thread finishes and destroys its captured client reference, a preempted worker on either pool can still outlive the test scope and self-detach.

@scotthart

Copy link
Copy Markdown
Member Author

I'm just going to skip this test for now in a separate PR.

@scotthart scotthart closed this Oct 1, 2026

This branch was successfully deployed

1 active deployment
false — 11d48d30 Deployed Sep 30, 2026 by scotthart via Save PR ref #12187
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants