[SungJin1212]

What this PR does:

Problem Statement

In the current Parquet mode, the store-gateway only serves blocks that have already been converted to Parquet. Any block that hasn't been converted yet is not queryable at all, since the Parquet store has no way to read TSDB blocks.

In practice, this forces operators to either stay on tsdb mode or wait for full conversion of all historical blocks before flipping the switch. Even then, the gap doesn't fully go away: new blocks are produced continuously (from Compactor) and take some time to be converted.

Solutions

This PR adds a hybrid mode to the Parquet bucket store. The store-gateway runs both a Parquet store and a TSDB sub-store and routes each requested block to one of them, so every block is queryable regardless of its conversion state.

• Sync: the TSDB sub-store syncs only blocks that are not converted yet. IgnoreParquetBlocksFilter excludes converted blocks using the bucket index.
• Routing: the same filter records the Parquet block IDs from the bucket index snapshot it used. A block is routed to the Parquet store iff it is recorded there, so excluding a block from the TSDB sub-store and routing it to Parquet always use the same snapshot. There is no gap during the TSDB -> Parquet handover. Before a tenant's first sync, all blocks are routed to the TSDB sub-store, which matches the tsdb mode behavior.
• Merging: when a request spans both stores, Series queries both concurrently and stream-merges their label-sorted results without buffering either side. LabelNames/LabelValues results and the response hints (QueriedBlocks, QueryStats) are merged as well. If one sub-store fails, its own error is returned so the querier can still classify it (retry, limit, access denied).

Configuration

• Hybrid mode is the default for -blocks-storage.bucket-store.bucket-store-type=parquet and requires the bucket index. The config validation fails if the bucket index is disabled.
• New experimental flag -blocks-storage.bucket-store.parquet-fallback-disabled (default false). Setting it to true keeps the previous Parquet-only behavior: no TSDB sub-store, no local state, and the bucket index is not required.

Behavior change

With the default hybrid mode, the Parquet bucket store changes from stateless to semi-stateful: it keeps index-headers on local disk for blocks that are not converted yet. A block that is later converted has its index-header downloaded while the conversion is pending and removed once the store-gateway syncs the converted state.

New metrics

• cortex_hybrid_bucket_stores_blocks_routed_total{store}: requested blocks routed to each sub-store.
• cortex_hybrid_bucket_stores_operations_total{store, method}: operations served by parquet, tsdb or mixed.
• cortex_blocks_meta_synced{state="parquet-converted"}: blocks excluded from the TSDB sub-store because they are converted.

Which issue(s) this PR fixes:
Fixes #7140

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
• docs/configuration/v1-guarantees.md updated if this PR introduces experimental flags

[SungJin1212]

#7892 should be merged to fix TestParquetStoreGateway_HybridModeFuzz .

[sandy2008]

LGTM. I traced the handover: the TSDB sub-store's filter drops converted blocks and records their IDs from the same bucket index snapshot, and the Parquet store reads the converter mark when its own index view doesn't have a block yet, so a block is servable whichever side it is routed to. The merge path cancels and closes both producers correctly, and the hybrid, filter and streaming-server tests pass with -race; generated docs and schema are in sync. The failing test-no-race (amd64) job is TestCompactor_ShouldSkipOutOrOrderBlocks in pkg/compactor, which this PR doesn't touch: the test checks the counter as soon as no-compact-mark.json exists, but Thanos uploads the mark before incrementing it.

[sandy2008]

Nit: since this changes what an existing bucket_store_type=parquet deployment does by default (it now keeps index-headers on local disk and requires the bucket index), a [CHANGE] line might help operators notice it on upgrade.