CAS-S3 ASan/MSan stateless shards hit GitHub's 6h timeout with `cas_gc_interval_sec` 5

[DimensionWieldr]

Type of problem

CI timeout — ASan/MSan CAS-over-S3 stateless shards are killed by GitHub's 6h job limit when the test disk uses cas_gc_interval_sec 5.

Summary

The CAS-S3 test policy overrides the product default (60s) to 5s so reclamation happens during the run:

<cas_gc_enabled>1</cas_gc_enabled>
<cas_gc_interval_sec>5</cas_gc_interval_sec>

File: tests/config/config.d/cas_s3_storage_policy_for_merge_tree_by_default.xml

On MasterCI tag v26.6.2.20001.altinityantalya (SHA fc67ca28aab) those shards were cancelled at 6h, not hung on one test:

Job	Progress at kill	Median test time
Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 1/2)	4333/5581 (78%)	~15s
Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 2/2)	4380/5604 (78%)	~14s
Stateless tests (amd_msan, cas s3 storage, parallel, 2/3)	2073/3768 (55%)	~30s
Stateless tests (amd_msan, cas s3 storage, parallel, 3/3)	1996/3668 (54%)	~27s

Throughput dropped over the run (ASan ~458 → ~183 tests / 30 min). The same-run non-CAS ASan shards finished in ~1.5h; TSan plain S3 in ~2h.

Control

Throwaway PRs against antalya-26.6 (do not merge) with only that XML changed:

• cas_gc_interval_sec 60 (product default): PR #2296 — ASan 1/2 3h14m, 2/2 2h59m (finished; OOMs tracked separately)
• cas_gc_enabled 0: PR #2297 — ASan 1/2 2h07m, 2/2 2h51m (finished; no OOM storm)

GC at 5s is what makes this lane miss 6h. GC is not runtime-reloadable; the interval is baked into the test disk config.

Likely fix direction

Stop using a 5s GC interval in the CAS-S3 (and possibly local CAS) test policy. Product default is already 60s. Keep a shorter interval only in a dedicated GC-focused test, not the full sanitizer stateless suite.

Links

Related: https://github.com/Altinity/ClickHouse/actions/runs/32969478793
Related: https://github.com/Altinity/ClickHouse/actions/runs/32969478793/job/98559503678
Related: #2296
Related: #2297

Project Antalya Build Version: 26.6.2.20001.altinityantalya
Related: #2299

[DimensionWieldr]

ASan CAS-S3 parallel shards only (amd_asan_ubsan, cas s3 storage, parallel). Same job pair on every run.

GC	Source	Shard 1/2	Shard 2/2	Finished inside 6h?
Off	PR #2297	2h 7m (2 fails)	2h 52m (pass)	yes
5s	MasterCI 32969478793	6h 0m (cancelled at 78%)	6h 0m (cancelled at 78%)	no (~7.5h projected)
60s	PR #2296	3h 14m (14 fails, OOM)	2h 59m (10 fails, OOM)	yes
120s	PR #2306	2h 11m (pass)	3h 5m (pass)	yes

The 5s row is GitHub’s kill, not a completed suite. Off / 60s / 120s all finish in about 2–3h; 5s is the outlier. 60s vs 120s is in the noise of shard-to-shard spread (about an hour), not a clear extra win for 120s. 60s paid for that finish with an OOM storm; 120s and GC-off did not.