feat: optional gzip upload for the s3 deploy target

[XtremeOwnageDotCom]

What & why

Note: this does not fix the reported "search stops working on S3" bug. That turned out to be
CORS, not size — see Root cause, for the record at the bottom. This PR
is a real but separate improvement, and the last commit documents the actual cause.

Chasing a report that search stops working on a site served from S3 once the index gets large.

First, what it is not: I drove Material's real search worker against a 13 MB index / 10,501 docs, and against a single 400k-word page. Both work — queries return results, worker initialises. There is no functional cliff in the index or in the search code.

What is real is the delivery. S3 returns an object as exactly the bytes it stores and will not compress on the fly — GitHub Pages does this automatically. So the same site ships far more data from S3, and search/search_index.json is the file that hurts, because every visitor downloads it in full the moment they open search:

	raw	stored gzipped
this docs site	529 KB	99 KB	5.4×
13 MB synthetic index	13,108,995 B	2,287,060 B	5.7×

The existing deploy is a plain aws s3 sync, which sets no Content-Encoding, so there was no way to serve that index compressed at all.

The change

deploy.gzip (opt-in) stores text assets compressed. The deploy becomes two syncs:

aws s3 sync <site>    s3://bucket --delete --exclude *.json --exclude *.css ...
aws s3 sync <staging> s3://bucket --delete --exclude * --include *.json ... --content-encoding gzip

Compressed copies are staged in a temp directory rather than gzipping in place, so the build output stays serveable locally. Extensions and relative paths are preserved so the AWS CLI still infers the same Content-Type.

Off by default. S3 returns the stored bytes and the gzip header regardless of whether the client sent Accept-Encoding: gzip. Browsers handle it; a script reading an object straight out of the bucket may not.

Type of change

• Bug fix
• New feature
• Documentation
• Refactor / chore

Checklist

• dotnet format Netdocs.slnx --verify-no-changes passes
• dotnet build Netdocs.slnx -c Release succeeds
• dotnet test Netdocs.slnx -c Release passes (516, +6 new)
• Added/updated tests for the change
• Updated docs under docs-site/docs/** if behavior changed

Verification

Beyond the unit tests, I ran a real deploy against a stubbed aws on PATH that records its argv, so the actual code path is exercised without touching AWS:

• gzip: true → exactly the two invocations above, 44 assets staged.
• gzip: false → a single sync, byte-identical to today's behaviour.
• Staged files verified to be real gzip that decompresses back to the original bytes.

One test pins that every extension excluded by pass 1 is re-included by pass 2 — a type filtered out of both would silently never be uploaded, which is the failure mode worth guarding.

That stub run also caught a regression I introduced: the non-gzip path had lost its Deployed to '<dest>' log line in the refactor. Restored, and confirmed on every path.

Notes for reviewers

InternalsVisibleTo added to Netdocs.Core for the test project — the staging and argument helpers are implementation details of the deploy target, not public API, and this matches what Netdocs.Plugins already does.

Worth knowing if you're behind CloudFront: its automatic compression only applies up to an object-size limit, so a growing search index can quietly stop being compressed. Storing it already compressed sidesteps that. If the S3 symptom turns out to be a hard error rather than slow transfer, that points somewhere else entirely and I'd chase it separately — the response headers on the deployed index would say which.

🤖 Generated with Claude Code

Root cause, for the record

The reported bug was not index size. A network trace showed the index request being 302-redirected to a presigned S3 URL on another origin, and the redirected response coming back from Server: AmazonS3 with no Access-Control-Allow-Origin — so the browser discarded it. Content-Length was 786 KB; size was never the problem.

Only search breaks because the index is the one asset fetched with XHR, which is CORS-checked. CSS, JS and images load through <link>/<script> tags, which follow cross-origin redirects without any CORS check — so the page looks perfectly healthy while search silently does nothing.

The fix is a bucket CORS rule, not a code change. The final commit here documents it in the S3 publishing section, since the symptom points nowhere near the cause.

This PR still stands on its own — 786 KB uncompressed vs ~145 KB gzipped for that same index — but it is an efficiency change, not the bug fix.