SOLR-12239: add RESORTINDEX core-admin action to re-sort an existing index#4644
Open
serhiy-bzhezytskyy wants to merge 1 commit into
Open
Conversation
…index Enabling index sorting on a collection created without it currently fails on core reload with an IndexWriter.validateIndexSort error, forcing a delete + reindex from source. This adds a RESORTINDEX core-admin action that re-sorts the existing index into the requested sort order in place, using the LUCENE-9484 mechanism: each segment reader is wrapped in a SortingCodecReader and merged into a fresh sort-configured IndexWriter via addIndexes(CodecReader...), and the result is swapped in with modifyIndexProps (as RestoreCore/replication do), then the writer and searcher are reopened. Notes: - addIndexes does not auto-sort unsorted readers (LUCENE-8505 removed that), so the explicit SortingCodecReader wrap is what performs the merge-based re-sort. - Not supported in SolrCloud mode. - Indexes containing child/nested documents are rejected (re-sorting would break the parent-child blocks), matching UPGRADEINDEX's restriction. - On a failed swap the original index.properties is restored (RestoreCore's rollback). The target sort is given by the sort request parameter using the usual Solr sort syntax; the fields must have docValues.
github-actions
Bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/SOLR-12239
Enabling an index sort on an existing collection currently fails on core reload with
IndexWriter'svalidateIndexSort("cannot change previous indexSort") — Lucene rejects the transition from unsorted segments to a configured sort, so today you have to delete and reindex from source.This adds a
RESORTINDEXcore-admin action that re-sorts the existing index into a target sort order and swaps it in, without reindexing.Mechanism (LUCENE-9484, shipped in Lucene 9.0): each existing segment reader is wrapped in a
SortingCodecReaderand merged into a fresh sort-configuredIndexWriterviaaddIndexes(CodecReader...); the result is swapped in withmodifyIndexProps(the same path RestoreCore/replication use), then the writer and searcher are reopened. NoteaddIndexesdoes not auto-sort unsorted readers (LUCENE-8505 removed that), so the explicitSortingCodecReaderwrap is what performs the re-sort.Details:
action=RESORTINDEX&core=X&sort=...) and v2 (POST /api/cores/X/resort) APIs; v1 delegates to the v2 impl.sortparam (usual Solr sort syntax), or falls back to the core's configuredSortingMergePolicysort if omitted.SolrIndexConfig's parent-field handling so it works on child-doc-capable schemas.I see this as the migration piece that complements SOLR-13681 / #313 (making the index sort configurable) — resort an existing index, then declare the matching sort in config so new docs stay sorted. Filing as a PoC/for discussion: does this approach fit with how index sorting should evolve? Happy to adjust. I did not want to step on the config work.