[client-v2] Every compressed read fails on ClickHouse 26.9+: response reader is hardcoded to LZ4 but the server default codec is now ZSTD(3)

[Marais]

Summary

ClickHouse #108786 ("Switch the default compression to ZSTD(3) for table data and network", merged 2026-09-05, Version info says merged into 26.9.1.810, so included in 26.9 and later) changes CompressionCodecFactory::getDefaultCodec from LZ4 to ZSTD(3).

The HTTP compress=1 framed output is one of the direct getDefaultCodec users. client-v2 requests compress=1 by default and pipes the response body through ClickHouseLZ4InputStream, which asserts the LZ4 block magic byte, so every compressed read fails against ClickHouse 26.9+:

com.clickhouse.client.api.ClientException: Invalid LZ4 magic byte: '-112'
	at com.clickhouse.client.api.internal.ClickHouseLZ4InputStream.refill(ClickHouseLZ4InputStream.java:112)
	at com.clickhouse.client.api.internal.ClickHouseLZ4InputStream.read(ClickHouseLZ4InputStream.java:62)
	...
	at com.clickhouse.client.api.internal.TableSchemaParser.readTSKV(TableSchemaParser.java:23)
	at com.clickhouse.client.api.Client.getTableSchemaImpl(Client.java:1872)
	at com.clickhouse.client.api.Client.getTableSchema(Client.java:1847)

getTableSchema is just where we hit it first — this is not specific to schema introspection, it affects any compressed response.

Per that PR's own changelog, the HTTP compress=1 path has no runtime rollback: it is not controlled by the compatibility setting, per-column CODEC, or the server <compression> config. So this cannot be worked around server-side.

Affected versions

• client-v2 0.9.5 — verified failing
• client-v2 0.10.0 (latest release) — same ClickHouseLZ4InputStream with the same Invalid LZ4 magic byte check, so also affected
• Server: ClickHouse 26.9+ (reproduced on clickhouse/clickhouse-server:head = 26.9.1.943). 26.8 and earlier are unaffected by this bug.

Root cause / evidence

-112 is 0x90 signed. Raw curl against 26.9.1.943 confirms the framing:

$ curl -s -u default:x 'localhost:8123/?compress=1' --data-binary 'DESCRIBE TABLE t FORMAT TSKV' | xxd | head -2
00000000: 151e 70e4 4490 8555 b0c4 8424 d0d8 731d  ..p.D..U...$..s.
00000010: 9065 0000 00c0 0000 0028 b52f fd20 c09d  .e.......(./. ..

After the 16-byte checksum, offset 0x10 is 0x90 — the compression method byte for ZSTD, exactly the byte reported in the exception — and offset 0x18 is 28 b5 2f fd, the ZSTD magic number. ClickHouseLZ4InputStream expects 0x82 there.

The same request without compress=1 returns plain readable TSKV.

Reproduce

docker run -d --name chhead -e CLICKHOUSE_PASSWORD=x -p 8123:8123 clickhouse/clickhouse-server:head
# then any client-v2 read, e.g.
# client.getTableSchema("t", "default")

Suggested fix

The compressed frame is self-describing — ClickHouse#108786 explicitly relies on "the receiver auto-detects the codec", which holds for the server's own readers. client-v2 does not auto-detect; it assumes LZ4. The response reader should dispatch on the compression method byte in the frame header (0x82 LZ4, 0x90 ZSTD, 0x02 none) rather than asserting LZ4, and gain a ZSTD decompression path.

Note on interaction with #3070 / #3094

On 26.9+ this bug currently masks the X-ClickHouse-Format precedence bug (#3070, #3094): the stream dies in refill before readTSKV ever sees bytes. I verified the header bug is still present on 26.9.1.943 — curl -H 'format: TSKV' ... --data-binary 'DESCRIBE TABLE t' returns TabSeparated there too. So fixing only this issue will move getTableSchema back to failing with Non-null columnName and columnType are required; both need to land.

Setting compressServerResponse(false) sidesteps this issue alone, but for the same reason it is not a usable workaround for getTableSchema, and it costs compression on every read.

Context

Found while running the ClickHouse Flink connector test suite against ClickHouse HEAD.

[mshustov]

@polyglotAI-bot fix this