Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions changelog.d/20261102_l1-latency-sampling-fork-window.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
### SaaS API — L1 hit-latency sampling: the hookless-fork startup window

- [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency), **Sampling**
([API-81](spec/saas-api.md#api-81)): names one exception to the independent per-call draw.
A child that inherits a backend instance through a fork that runs none of the SDK's at-fork
handlers may repeat its parent's and siblings' draw sequence until it creates a report
accumulator, or first uses one it inherited (its first request other than the health check
on an instance that has one, or its first timed L1 hit). A backend instance with no
accumulator, such as one with telemetry off, does not end the window. Each call is still
timed with probability 1/N. No header, value format or range of N changes.
2 changes: 1 addition & 1 deletion spec/saas-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -596,7 +596,7 @@ An SDK MUST<sup id="api-78">API-78</sup> compute `X-CacheKit-L1-Hit-Rate` as `L1

The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store.

**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUST<sup id="api-81">API-81</sup> decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates the latency of every L1 hit of 60 s or less, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`.
**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUST<sup id="api-81">API-81</sup> decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. One exception applies to a child process that inherits a backend instance through a fork that runs none of the SDK's at-fork handlers, such as a fork made from C without the language runtime's fork hooks. Until that child creates a report accumulator for a backend instance, or first uses an accumulator it inherited (its first request other than `GET /v1/cache/health` on an instance that has one, or its first timed L1 hit), its draws MAY repeat the sequence that its parent and every other process forked from the same state draw. Each call is still timed with probability 1/N, but those processes time the same call positions. Building or using a backend instance that has no accumulator, such as one with telemetry turned off or on a clock that does not qualify, does not end the window. Every other draw, including every draw such a child makes after that point, is independent as above. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates the latency of every L1 hit of 60 s or less, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`.

**Clock.** An SDK MUST NOT<sup id="api-82">API-82</sup> send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify:

Expand Down
Loading