[Michael McQuade (giraffesyo)]

Summary

Adds three areas of the S3 API to aws/s3:

• Multipart uploads: CreateMultipartUpload, UploadPart, CompleteMultipartUpload, AbortMultipartUpload, ListMultipartUploads (prefix, delimiter, key and upload ID markers, max-uploads, URL encoding) and ListParts (pages). Completing an upload checks the parts the way S3 does: they must be in ascending order (InvalidPartOrder), uploaded with the ETags given (InvalidPart), and at least 5 MiB except for the last (EntityTooSmall). The new version's ETag takes S3's md5-of-md5s-N form. Completing an upload ends it, so a retry after a lost response gets NoSuchUpload. Server.MultipartUploads reports in-progress uploads and their parts, which S3 bills as storage. Server.CreateMultipartUpload starts an upload directly. S3 documents in-progress uploads as blocking DeleteBucket only for directory buckets, so deleting a general purpose bucket discards its uploads.
• Object Lock:
  • Bucket setup: CreateBucket with x-amz-bucket-object-lock-enabled turns on versioning, which then cannot be suspended. Get/PutObjectLockConfiguration handle default retention in Days or Years, and Put can enable Object Lock on a bucket that already has versioning on.
  • Per-version locks: PutObject and CreateMultipartUpload take lock mode, retain-until and legal hold headers. Get/PutObjectRetention and Get/PutObjectLegalHold read and set a version's lock.
  • Deletes: DeleteObject and DeleteObjects refuse to delete a locked version (403 AccessDenied, key by key in DeleteObjects). COMPLIANCE never yields. GOVERNANCE yields only to x-amz-bypass-governance-retention: true from a caller granted s3:BypassGovernanceRetention. A delete without a version ID adds a delete marker, and DeleteBucket stays BucketNotEmpty while any version remains.
  • Retention expiry follows the server's clock (WithClock). Server.SetRetention and Server.SetLegalHold lock a version directly.
  • A PutObject that sets a retention needs Content-MD5 or a checksum, as S3 documents.
• Request payment: Get/PutBucketRequestPayment record the payer. There is only one account, and S3 never asks a bucket owner for x-amz-request-payer, so there is nothing to enforce. The docs say so.

WithPolicy checks the documented actions for all of these, including the conditional ones (lock headers on PutObject, Object Lock on CreateBucket, and the bypass).

The package docs list every behavior that is inferred rather than documented. These are the messages for the object lock AccessDenied, InvalidBucketState and InvalidRequest errors, NoSuchObjectLockConfiguration, the MethodNotAllowed answer for a delete marker's version, the InvalidArgument messages for lock headers, and how ListMultipartUploads counts common prefixes toward a page. Changing a GOVERNANCE retention to COMPLIANCE also needs the bypass, which is stricter than the cases AWS documents (shortening and removal) and is marked inferred. Object Lock writes (PutObject with a retention, UploadPart of such an upload, and the lock configuration, retention and legal hold PUTs) need Content-MD5 or a checksum, retain-until dates are kept to the millisecond, max-uploads and max-parts above 1000 return 1000 like max-keys, and below 1 fail with InvalidArgument (inferred). GetBucketVersioning, GetBucketTagging and GetBucketRequestPayment responses now carry the S3 namespace. PutBucketRequestPayment and the Object Lock PUTs reject another root element, as the versioning and tagging PUTs already did, and need Content-MD5 or a checksum. Event holds (variable retention), UploadPartCopy, and conditional writes on CompleteMultipartUpload are rejected with 400 rather than ignored.

Testing

• sdktest/aws/s3: tests through aws-sdk-go-v2 cover:
  • multipart: completion and ETag, part errors, abort, ListParts pages, ListMultipartUploads pages with the SDK paginator (by key and initiation order, with delimiters and prefixes), DeleteBucket with only uploads, a lost completion response, and permissions
  • Object Lock: bucket configuration, simple and versioned deletes of COMPLIANCE, GOVERNANCE and legal-hold versions with and without bypass, expiry on the clock, extending and shortening retention, default retention on PutObject and CompleteMultipartUpload, and the bypass permission
  • request payment
• aws/s3: XML shapes and header validation through ServeHTTP, plus fuzz seeds for the new operations.
• go build, go vet, go test -p 2 -race ./... in both modules, make lint and make vuln all pass.
• make bench (aws/s3, -count 5), canary compared with this branch:
  • BenchmarkPutListDeleteVersions: 232 to 248 µs/op and 546 allocs/op on canary, 238 to 264 µs/op and 545 allocs/op on this branch (the timings were noisy)
  • BenchmarkListObjectsV2: 162 to 182 µs/op and 189 allocs/op on both, with one 282 µs outlier on this branch

Checklist

• make check passes
• New methods are tested through the official client library
• Package documentation says where the stand-in differs from the real API