Cache rewritten npm and Composer metadata

[montehurd]

The npm and Composer handlers rewrite every metadata document they serve so that download URLs point at the proxy. That means decoding the whole document into generic maps, changing the URLs and encoding it again, and for Composer, expanding the minified format first. It ran on every request, including requests for metadata that was already cached.

I measured the cost on an 8 vCPU VM with cache_metadata on, so upstream was out of the picture:

• Serving cached npm packuments to 20 clients ran the proxy at 450% CPU for about 1,100 requests a second, about 4 ms of CPU per request.
• Composer metadata cost about 3 ms per request.
• A single cached typescript request allocates about 134 MB.

What this changes

• Cache: rewritten documents are kept in memory, so each distinct upstream document is rewritten once.
• Shared rewrite: requests that arrive while a document is being rewritten wait for that rewrite instead of running their own. A waiter leaves when its client does, as in the other coalescers, and the rewrite still completes and is cached.
• Key: ecosystem, proxy URL, package, and a SHA-256 of the raw document. New bytes from upstream are rewritten again, so nothing is served stale. The denylist is fixed at startup, so it has no place in the key.
• Cooldown: filtering depends on the current time, so with cooldown on the cache is bypassed. Otherwise a cached rewrite could keep hiding a version after its cooldown ended.
• Size: metadata_rewrite_cache_size bounds the cache, least recently used out first. It defaults to "256MB", and "0" rewrites on every request. It's also available as PROXY_METADATA_REWRITE_CACHE_SIZE.
• Opt-in: NewProxy callers such as tests and the mirror command keep the old behaviour unless they call SetMetadataRewriteCacheSize, as with BatchHits.
• Shared output: npm's rewriteMetadata returns its input unchanged when a document has no versions object. Caching that is safe because the input is already the shared read-only slice from fetchOrCacheMetadata.

Benchmarks

Whole cached requests (lookup, read, rewrite, response), measured locally on real documents:

Document	Before	After	Allocations
npm @babel/core (426 KB)	7.1 ms	0.38 ms	54,489 → 119
npm typescript (8.7 MB)	155 ms	10.5 ms	1.37M → 133
Composer symfony/console	17.4 ms	0.48 ms	165,909 → 123

What remains is reading the raw document from storage and hashing it. Hashing is most of what's left for typescript.

Tests cover:

• a repeat request reusing the rewrite
• new upstream bytes being rewritten again
• concurrent requests sharing one rewrite
• a waiter leaving while the rewrite still lands in the cache
• least-recently-used eviction
• output too large to cache
• errors not cached
• "0" disabling the cache
• cooldown bypassing the cache
• both handlers end to end through Routes()
• the config setting: parse, validate and environment variable

Tested with go test ./..., -race on internal/handler, and golangci-lint. I moved the new config check into validateComponents to keep Config.Validate under the gocognit limit.

Persisting rewritten documents across restarts would be possible later, but in-process caching gets nearly all the benefit with much less to review.

[andrew]

Thanks!