topsql_v2: add detailed TiKV I/O dimensions

[jiong-nba]

TiKV TopSQL Detailed I/O Dimensions

Overview

This PR updates the 0.49 TopSQL v2 pipeline to consume and persist the detailed TiKV I/O dimensions introduced by tikv/tikv#19837 and pingcap/kvproto#1498.

It covers the dedicated cloud path implemented on the 0.49 branch:

TiKV -> topsql_v2 -> topsql_data_deltalake -> Delta Lake

Changes

Protocol

• Add rocksdb_block_read_count = 9 to GroupTagRecordItem.
• Keep old TiKV payloads compatible: the protobuf default remains 0 when the field is absent.

TopSQL v2 source

• Propagate rocksdb_block_read_count for both tikv_topsql and tikv_topregion events.
• Select the per-second candidate union independently by:
  • CPU
  • Combined network input/output
  • Logical reads
  • Logical writes
  • RocksDB block reads
• Preserve the new dimension when merging upstream/evicted others records and during downsampling.

Delta Lake sink

• Add nullable topsql_rocksdb_block_read_count to the TopSQL data schema.
• Keep historical Delta Lake data compatible when older partitions do not contain the new column.

Metric semantics

rocksdb_block_read_count is the number of foreground RocksDB block reads attributed to a request. It is useful for relative TopSQL ranking, but it is not device-level Read IOPS and should not be presented as an exact physical I/O count.

Validation

cargo test --lib sources::topsql_v2::upstream::tikv::parser::tests -- --nocapture
cargo test --lib --features nextgen sources::topsql_v2::upstream::tikv::parser::tests -- --nocapture
cargo test --lib sinks::topsql_data_deltalake::processor::tests::test_topsql_schema_contains_rocksdb_block_read_count -- --nocapture
cargo check --workspace --all-targets
cargo check --workspace --all-targets --features nextgen
git diff --check

Related changes:

• tikv/tikv#19837
• pingcap/kvproto#1498
• tidbcloud/vector-extensions#63

[pingcap-cla-assistant]

All committers have signed the CLA.

[Copilot]

🟡 Not ready to approve

The updated Top-N cutoff logic has a tie-handling bug and also risks u64 overflow when summing network bytes, which can lead to incorrect filtering/aggregation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR extends the TiKV TopSQL upstream parser to decode and emit additional resource-dimension metrics (network I/O bytes, logical read/write bytes, RocksDB block read count) while preserving existing behaviors like per-second Top-N filtering, others aggregation, and downsampling.

Changes:

• Extend GroupTagRecordItem (protobuf + Rust handling) to include 5 new TiKV resource usage fields.
• Emit new TopSQL metrics for the added resource dimensions.
• Update Top-N filtering, others merging behavior, and downsampling logic/tests to account for the new dimensions.

File summaries

File	Description
src/sources/topsql/upstream/tikv/parser.rs	Adds field merges, resource-aware Top-N selection, emits new metrics, and extends downsampling/tests.
src/sources/topsql/upstream/tikv/mock_upstream.rs	Extends mock TiKV metering records with the new resource fields.
src/sources/topsql/upstream/consts.rs	Adds metric name constants for the new TiKV resource dimensions.
proto/tikv.proto	Extends the protobuf message with new fields and field numbers (5–9).

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.