Skip to content

[To dev/1.3] Fix schema file I/O failure handling and short writes - #19214

Merged
Caideyipi merged 3 commits into
apache:dev/1.3from
Caideyipi:codex/cp-e3b60020-dev-1.3
Nov 11, 2026
Merged

Caideyipi merged 3 commits into
apache:dev/1.3from
Caideyipi:codex/cp-e3b60020-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Backport #19190 (e3b60020bbb34b60e1272dbad4f291635309a6f6) to dev/1.3, based on f1e915ae2b80513857b5c40f73aff85fbe20ffe2.

Schema-file initialization, shutdown, and writes can leave incomplete data or open resources after an I/O failure. This change completes short header and page writes, persists a valid initial header immediately, and attempts every cleanup step while retaining the first exception and suppressing later cleanup failures.

For example, when a page write consumes three bytes and the next write returns zero or throws an IOException, the operation now fails and the page remains dirty. A later flush retries the entire page from its original file offset. If initialization fails, resources are closed and the partial new file is truncated before a retry.

Compatibility adaptations

  • Keep the 1.3 stream-backed log writer. Use its channel for complete commit-marker writes, and inspect that same channel in the resource-close regression assertion.
  • Add the missing node-commons locale source selection and the required English/Chinese message class. Port the existing schema-version error constant into the DataNode locale classes.
  • Preserve the source regression scenarios and assertions, including the architecture rule against direct schema-file FileChannel.write calls.

Validation

  • mvn spotless:apply -P with-integration-tests -pl iotdb-core/datanode,iotdb-core/node-commons passed.
  • git diff --check passed; source-path coverage, locale-key parity, message hash, and source architecture test were checked.
  • Remote clean-source verification could not resolve Maven Central on one approved build host; the alternate approved host timed out on the same endpoint.
  • The local fallback built TsFile common, but its clean TsFile install exceeded the 20-minute verification budget during dependency/plugin resolution. IoTDB regression tests and Chinese-locale compilation were not run.
  • Cluster integration tests were not run.

This PR has:

  • added unit tests to cover the changed code paths.
Key changed classes

IOUtils, SchemaFile, SchemaPage, PageIOChannel, PageManager

Backport the requested schema-file I/O fix to the 1.3 branch, preserving its failure cleanup, complete writes and regression coverage.

(cherry picked from commit e3b6002)
@Caideyipi
Caideyipi merged commit dca11e3 into apache:dev/1.3 Nov 11, 2026
23 checks passed
@Caideyipi
Caideyipi deleted the codex/cp-e3b60020-dev-1.3 branch November 11, 2026 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant