Repository navigation
feat(storage): managed folders - #102
Merged
Merged
Conversation
managedFolders insert, get, list and delete, with allowNonEmpty and metageneration preconditions, and uniform bucket-level access on buckets, which managed folders need. A bucket with managed folders left is not empty.
Accept maxResults beside pageSize, refuse predefined ACLs under uniform bucket-level access, record and report publicAccessPrevention, and cite sources for the managed folder rules.
uniformBucketLevelAccess wins over bucketPolicyOnly, object ACLs under uniform access answer Cloud Storage's 400 invalid, objects.restore refuses copySourceAcl under uniform access and needs storage.objects.setIamPolicy without it, PutBucket checks publicAccessPrevention, and a page size of 0 counts as unset.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Cloud Storage managed folders to the
gcp/storagestand-in, plus the uniform bucket-level access setting they depend on.Managed folders (
storage.managedFolders.*):insert,get,list(prefix,pageToken, and a page size of at most 1,000 aspageSize, which the discovery doc and Go client use, ormaxResults, which the reference page names; both, if set and not 0, must agree; in name order) anddelete(allowNonEmpty,ifMetagenerationMatch/ifMetagenerationNotMatch, whichgettakes too).kind,id,selfLink,name,bucket,metageneration,createTimeandupdateTime. Names end with/and follow the documented rules: 1 to 1,024 bytes, at most 15/s, no CR or LF, not.or.., and not under.well-known/acme-challenge/.conflict. A missing folder returns 404notFound.conflictunlessallowNonEmpty=true. Noncurrent generations count; soft-deleted ones don't. WithallowNonEmpty=truethe folder goes and the objects stay, as the reference's "a simulated folder with the same name is retained" implies.buckets.deletewith folders left fails with the existing 409conflict"The bucket you tried to delete is not empty."WithPolicythe methods needstorage.managedFolders.create/get/list/delete.allowNonEmpty=truealso needsstorage.managedFolders.setIamPolicy, per the delete reference.PutManagedFolderandManagedFoldersset up and inspect folders directly.Uniform bucket-level access (
iamConfiguration): managed folders work only in buckets that have it, sobuckets.insertandbuckets.patchnow accept it.uniformBucketLevelAccessand its old namebucketPolicyOnly. When a body carries both,uniformBucketLevelAccesswins, as the discovery doc and the Go client say, so a raw read-modify-write can turn it off. It is reported under both names with alockedTime90 days after it is turned on. Without this field the stand-in rejected the Go client'sUniformBucketLevelAccessbucket create with 400.lockedTime.predefinedAclon uploads,objects.patchandobjects.updatefails with what Cloud Storage answers: 400invalid"Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access". The status codes page lists this case underbadRequest, and the package doc notes the mismatch.objects.restorewithcopySourceAcl=truefails the same way (inferred). On bucket insert and patch,predefinedAclandpredefinedDefaultObjectAclfail with 400badRequest, per the status codes reference (message inferred).WithPolicy,copySourceAcl=trueon a bucket without uniform access needsstorage.objects.setIamPolicy, per the restore reference.aclanddefaultObjectAclbodies were already rejected with 400, and copy, rewrite and compose are not implemented.publicAccessPrevention(inheritedby default, orenforced, also checked inPutBucket) is recorded and reported, not enforced, likecorsandrpo.iamConfiguration, as Cloud Storage does.Left out:
managedFolders.update, the per-folder IAM policy methods andrapidCacheConfigreturn 404 or 400, because the stand-in has no IAM policies on buckets or objects to build on. The Storage Control client is also not covered: its managed folder calls are gRPC-only, and the stand-in serves the JSON API.Inferred (marked in the package doc)
The docs and client libraries don't give these, so they are inferred:
conflict. The Go client library's tests expect FAILED_PRECONDITION over gRPC. The 409 mirrors how the JSON API answers a non-empty bucket delete; the status and message are inferred.conflict(status, reason and message). Noncurrent generations count toward non-empty.invalidfor a bad name, and the 400badRequestfor bucket ACLs under uniform access.objects.restorewithcopySourceAcl=trueon a uniform bucket is refused.buckets.deletewith only managed folders left uses the same message as with objects left./gets one added, as the reference's examples name folders without it.id(bucket/name/) andselfLink(name escaped).allowNonEmptyalso leaves nested managed folders in place.lockedTimereturns 400invalid, and while managed folders remain it returns 409conflict(statuses and messages).Soft-deleted buckets hold no managed folders to restore, which fits the docs' "You can't recover managed folders."
Testing
gcp/storage/folders_test.gocover the response shape, error statuses and reasons, paging with either size parameter, the emptiness rules, the 90-day lock, ACL refusal under uniform access, reportediamConfiguration, and policy checks.sdktest/gcp/storage/folders_test.gogo throughgoogle.golang.org/api/storage/v1(ManagedFolders.*) andcloud.google.com/go/storage(creating, updating and deleting buckets with uniform bucket-level access, reading back public access prevention, and predefined ACLs refused).go vet,go test -race ./...in both modules,make lintandmake vulnpass.make benchforgcp/storage(3 runs each), canary vs this branch: about 185-199 µs/op vs 192-199 µs/op, with 219 allocs/op in both. This is within noise.Checklist
make checkpasses