The issue can happen in the following scenario:
When the export pipeline decides to split a merge tree file into multiple parquet files, and the export_merge_tree_part_file_already_exists_policy setting is set to SKIP (the default) and clickhouse crashes after the first file has been exported.
In such scenario, when the task is resumed, the file writer will try to export that same part again. When it generates the first filename, it will realize it already exists and skip it without the knowledge that there would be more files.
It doesn't have that knowledge because file splitting is done on the fly and not pre-computed.
The export will eventually be marked as completed with missing files, leading to corrupted data.
The issue can happen in the following scenario:
When the export pipeline decides to split a merge tree file into multiple parquet files, and the
export_merge_tree_part_file_already_exists_policysetting is set toSKIP(the default) and clickhouse crashes after the first file has been exported.In such scenario, when the task is resumed, the file writer will try to export that same part again. When it generates the first filename, it will realize it already exists and skip it without the knowledge that there would be more files.
It doesn't have that knowledge because file splitting is done on the fly and not pre-computed.
The export will eventually be marked as completed with missing files, leading to corrupted data.