Skip to content

fix(filesystem): preserve UTF-8 across head/tail chunks - #4703

Open
CryoThrust wants to merge 1 commit into
modelcontextprotocol:mainfrom
CryoThrust:fix/filesystem-utf8-chunk-decoding
Open

fix(filesystem): preserve UTF-8 across head/tail chunks#4703
CryoThrust wants to merge 1 commit into
modelcontextprotocol:mainfrom
CryoThrust:fix/filesystem-utf8-chunk-decoding

Conversation

@CryoThrust

Copy link
Copy Markdown

What does this PR do?

Fixes #4666.

headFile and tailFile previously decoded each 1024-byte read independently. A UTF-8 character split across a chunk boundary could therefore become replacement characters.

This change keeps tail reads as raw bytes until the selected range is assembled, then decodes once, and uses Node's StringDecoder for forward head reads so incomplete trailing sequences are buffered across reads.

Verification

  • git diff --check passes.
  • Local test execution was attempted, but the installed Node binary cannot start because Homebrew's simdjson dylib is missing (dyld: Library not loaded).

@CryoThrust
CryoThrust force-pushed the fix/filesystem-utf8-chunk-decoding branch from bf20578 to 7942545 Compare August 28, 2026 04:38
@CryoThrust
CryoThrust force-pushed the fix/filesystem-utf8-chunk-decoding branch from 7942545 to f2cbdfb Compare August 28, 2026 04:46
@CryoThrust

Copy link
Copy Markdown
Author

The updated branch now includes a regression test for a UTF-8 character split at the 1024-byte read boundary. Build and filesystem tests are green on the latest commit. Ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

headFile/tailFile in filesystem server corrupts multi-byte UTF-8 characters at 1024-byte chunk boundaries

1 participant