[Michael McQuade (giraffesyo)]

Summary

The Cloud Storage stand-in now models the data protection features that decide whether a delete or an overwrite goes through:

• Retention policies and Bucket Lock. retentionPolicy on bucket insert and patch, with effectiveTime and isLocked, and buckets.lockRetentionPolicy, which needs ifMetagenerationMatch. Objects report retentionExpirationTime. A delete or overwrite before that time fails with 403 retentionPolicyNotMet and Cloud Storage's message. A locked policy can be lengthened, up to 3,155,760,000 seconds, but not shortened or removed.
• Object holds. temporaryHold and eventBasedHold can be set on upload, objects.patch and the new objects.update, and the bucket's defaultEventBasedHold applies them to new objects. A held generation can't be deleted, overwritten or made noncurrent, and fails with Cloud Storage's 403. Releasing an event-based hold starts the retention period again.
• Object retention. enableObjectRetention on bucket insert, and per-object retention in Unlocked or Locked mode, with overrideUnlockedRetention, which shortening, removing or locking Unlocked retention needs. The 400 and 403 errors match Cloud Storage's.
• Soft delete. softDeletePolicy defaults to 7 days and accepts 0 or 7 to 90 days. Deleted and replaced generations are kept with softDeleteTime and hardDeleteTime. They are listed with softDeleted=true, read with objects.get (softDeleted and generation) and brought back with objects.restore, until they expire on the server's clock. softDeletePolicy.effectiveTime follows the policy's history, as the discovery document defines it. Deleted buckets are kept the same way, at least until their last soft-deleted object expires, listed and read with softDeleted=true, and brought back with buckets.restore.
• Requester Pays. billing.requesterPays is recorded. Under WithPolicy, a call without userProject needs resourcemanager.projects.createBillingAssignment and fails with 400 "Bucket is a requester pays bucket but no user project provided." A call with userProject needs serviceusage.services.use.
• Test helpers to set up state directly: SetRetentionPolicy (locked or not), SetObjectHolds, SetObjectRetention, PutSoftDeletedObject, SoftDeletedObjects and SoftDeletedBuckets.
• objects.get (metadata only), objects.patch and objects.update are new. Bucket insert and patch, and object patch and update, ignore the fields the server sets (effectiveTime and isLocked in the bucket policies included), so a client can send back the body a get returned. A bucket patch accepts the insert-only name, location and customPlacementConfig only unchanged, and objectRetention only as the bucket has it (it is set by enableObjectRetention on insert); rpo can now be patched. A different object storageClass is rejected with 400.
• Uploads and restores with ifGenerationNotMatch and no live object fail with 412, per the reference.
• The permission checks cover storage.objects.setRetention (for any body that includes retention, null included, and for an objects.update that removes it), storage.objects.overrideUnlockedRetention, storage.objects.restore and storage.buckets.restore.

Statuses, reasons and messages follow the JSON API reference, the feature pages and Cloud Storage's own responses; where none gives the answer, the package documentation marks it inferred. Shortening or removing a locked policy answers 403: the bucket lock page says 400, but the retentionPolicy field description in the JSON API discovery document describes these as PERMISSION_DENIED, which is what Cloud Storage answers, and the package doc says so. Retention error messages give the time in UTC, where Cloud Storage writes Pacific time.

Testing

• The new behaviors are tested through cloud.google.com/go/storage in sdktest/gcp/storage/protection_test.go, plus a permission test. Bucket soft delete and objects.update go through the generated google.golang.org/api/storage/v1 client, because the Go storage client has no calls for them.
• HTTP-level tests cover the lock precondition and the routing of objects.restore for object names that contain /restore. New fuzz seeds were added, and FuzzServeHTTP ran for 30s.
• Ran go build, go vet and go test -race in both modules, plus make lint and make vuln.
• make bench, comparing BenchmarkUploadListDelete with canary (n=6):

                    │ canary      │ this branch                 │
                    │   sec/op    │   sec/op     vs base        │
UploadListDelete-18   181.7µ ± 2%   182.4µ ± 4%  ~ (p=0.818 n=6)
                    │    B/op     │    B/op      vs base        │
UploadListDelete-18  325.5Ki ± 1%  334.4Ki ± 1%  +2.74% (p=0.002 n=6)
                    │  allocs/op  │  allocs/op   vs base        │
UploadListDelete-18    215.0 ± 0%    215.0 ± 0%  ~ (p=1.000 n=6)

Checklist

• make check passes
• New methods are tested through the official client library
• Package documentation says where the stand-in differs from the real API