Skip to content

[WIP] Do not fsync on disk on indexers when compactors are enabled - #6752

Draft
Mallets wants to merge 2 commits into
mainfrom
mallets/unsynced-dir
Draft

[WIP] Do not fsync on disk on indexers when compactors are enabled#6752
Mallets wants to merge 2 commits into
mainfrom
mallets/unsynced-dir

Conversation

@Mallets

@Mallets Mallets commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add an unsynced filesystem-backed Tantivy directory for temporary index files
  • use it for indexing pipelines without a local merge planner

Test plan

  • make fmt
  • cargo test -p quickwit-directories test_unsynced_mmap_directory
  • cargo check -p quickwit-indexing

The compactor publishes splits only after remote upload succeeds, so its temporary Tantivy files do not need crash durability.
@Mallets Mallets changed the title [WIP] Avoid syncing standalone compactor scratch files [WIP] Do not fsync on disk on indexers when compactors are enabled Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant