Repository navigation
fix(store): normalize trailing slashes in the S3 key prefix - #2
Merged
Merged
Conversation
PhysicalPrefix() appends "/objects/" to MOONCAKE_S3_KEY_PREFIX verbatim, so a prefix ending in "/" yields "<prefix>//objects/". OSS folds "//" on PUT and GET but not in ListObjectsV2, so ScanMeta finds nothing, providers never re-register LOCAL_DISK replicas, and readers pinned with MC_STORE_REQUIRED_OFFLOAD_ENDPOINT fail every cold read. Strip trailing '/' once, in FromEnvironment() and in the adapter constructor, so PhysicalPrefix, WriterPrefix, DeletionIntentKey and the durable-delete namespace all share one value. Validate() rejects a prefix that is only slashes. Objects already written under the folded single-slash path are discovered as-is; no migration. Tracked in MindLab-Research/mint#1775.
xjtu-ctgg
marked this pull request as ready for review
October 26, 2026 12:06
Contributor
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
12 of 13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
PhysicalPrefix()appends"/objects/"toMOONCAKE_S3_KEY_PREFIXas-is, so a prefix ending in/produces<prefix>//objects/. OSS folds//on PUT and GET but not inListObjectsV2. As a result:ScanMetafinds nothing.MC_STORE_REQUIRED_OFFLOAD_ENDPOINTfails.The fix strips trailing
/once, inS3ObjectStorageConfig::FromEnvironment()and in the adapter constructor.PhysicalPrefix,WriterPrefix,DeletionIntentKeyand the durable-delete namespace then share one value.Validate()rejects a prefix made only of slashes.No migration is needed: objects written earlier were stored under the folded single-slash path, and they are found as-is.
Upgrade note. The durable-delete namespace identity includes
key_prefix, and the Master journals it with each delete. A deployment that ran with a trailing-slash prefix and still has an unfinished durable delete at upgrade time will keep failing that delete withUNAVAILABLE_IN_CURRENT_STATUS. The reason: no provider reports the oldp/identity any more. Nothing is deleted or resurrected wrongly. To avoid it, finish or drain pending durable deletes before upgrading.Tracked in MindLab-Research/mint#1775 (this fork has issues disabled). The Mint deploy-side normalization is in a separate Mint PR.
Module
mooncake-store)Type of Change
How Has This Been Tested?
Built on
a26dedb1(this branch's base) with GCC 13,-O3.Unit tests:
object_storage_adapter_testpasses 22/22. It has two new cases:KeyPrefixTrailingSlashesNameOneNamespace:p,p/andp///give the same prefix, including throughFromEnvironment().RejectsPrefixThatIsOnlySlashesWith the normalization removed, both new cases fail.
Real OSS (Aliyun
cn-beijing, existing validation prefix with a trailing/added; read-only):ScanMetaa26dedb1total_keys=0rc=-704)total_keys=12Control:
a26dedb1with the prefix without a trailing/also findstotal_keys=12.Format and spelling:
scripts/code_format.sh --check --changed-lines --base origin/main(clang-format 20.1.8) andtyposare clean.Fork CI (run 40322874200): format, spell-check, Ruff, docs, build-flags and every wheel build (standard, cu13, EFA ×3, ROCm ×2) pass. Two jobs fail, neither in code touched here:
master_service_test.cppandmaster_service_group_test.cppdon't compile. They callWrappedMasterService::GetReplicaListas a plain return, but it returnsLazy<…>ona26dedb1. Unmodifiedmainfails identically (run 40329230656).