Repository navigation
HDDS-17469. OMKeysRenameResponse writes renames whose sources and destinations overlap in map order, so the committed key table differs from the cache - #11780
Conversation
…tinations overlap in map order, so the committed key table differs from the cache Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Could you extend the regression tests with these scenarios, asserting
Please also cover and clarify the compatibility restriction for a batch that previously allowed a valid partial result: suppose I understand that rejecting overlaps is intentional, but this is broader than rejecting combinations that actually corrupt persistence. Please add a test documenting that behavior and mention the change from partial success to whole-request rejection in the PR description. Generated-by: Codex (GPT-6) |
…and partial rename cases in TestOMKeysRenameRequest Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, added all three as cases of The partial rename case is the new |
|
Thanks @jojochuang for the review. |
Generated-by: Claude Code (Opus 5.5)
What changes were proposed in this pull request?
OMKeysRenameRequest.validateAndUpdateCachechecks each pair against the key table, whose cache already holds the result of the earlier pairs of the same transaction, and stores the accepted pairs in aHashMapfrom source name to the newOmKeyInfo.OMKeysRenameResponse.addToDBBatchiterates that map and writes, per entry, a delete of the source and a put of the destination. When pairs of one transaction share key names, the batch depends on the map's iteration order, and a source named twice keeps only its last pair. The committed key table then differs from the cache: depending on the names, a key is committed twice on the same blocks, or a key reported as renamed disappears and its blocks leak. Once the cache entries are evicted after the flush, or on restart, the committed state is what every read sees.This change makes
OMKeysRenameRequest.preExecutereject the transaction withINVALID_REQUESTwhen a pair names a key that an earlier pair of the same transaction uses as source or destination. Pairs with an empty name are skipped, asvalidateAndUpdateCachenever applies them. The check needs only the key names, so it runs on the leader before the transaction is replicated.validateAndUpdateCacheand the response are unchanged, so entries already in the Raft log are applied as before.Behaviour change: such a transaction is now rejected as a whole with
INVALID_REQUESTand nothing in it is renamed. The check looks at names only, so it is broader than the combinations that leave the DB different from the cache. It also rejects transactions that commit correctly today (for examplektok_b, thenk_btok_c, for some key names) and transactions that end in a partial rename today: with onlyapresent,btocfails for its missing source andatobsucceeds (PARTIAL_RENAME), while the whole request is now rejected. Transactions whose pairs share no key behave as before.Keeping the pairs in their original order in the response was considered and not taken: the map keyed by source name loses the earlier of two pairs with one source, so it would need an ordered list of pairs passed from the request to the response, a larger change of both.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17469
How was this patch tested?
TestOMKeysRenameRequest.testCommittedKeyTableMatchesCache: runs a transaction throughpreExecuteandvalidateAndUpdateCache, commits its response, and compares each key in the DB with the cache. Six cases with pairs that share a key must be rejected withINVALID_REQUEST, among them a repeated source, a repeated destination and an overlap between nonadjacent pairs; a case with disjoint pairs is the control. Without the change, three cases leave the DB different from the cache and all six fail on the missing rejection.testPairsSharingKeyRejectedInsteadOfPartialRename:preExecuterejects the partial rename example above; the same request applied byvalidateAndUpdateCachealone still ends inPARTIAL_RENAMEwitharenamed tob, which is how such an entry already in the Raft log is applied.TestOMKeysRenameRequest,TestOMKeysRenameResponse,TestOMKeyRenameRequest,TestOMKeyRenameRequestWithFSO,TestOMKeyRenameResponseandTestOMKeyRenameResponseWithFSO(41 tests) pass locally on master; checkstyle is clean.