From fce944315bb4cbb7bff6247e05085d4008d62160 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Mon, 2 Nov 2026 01:09:27 +0000 Subject: [PATCH 1/2] docs(saas-api): name the hookless-fork startup window as an API-81 sampling exception --- changelog.d/20261102_l1-latency-sampling-fork-window.md | 9 +++++++++ spec/saas-api.md | 2 +- 2 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 changelog.d/20261102_l1-latency-sampling-fork-window.md diff --git a/changelog.d/20261102_l1-latency-sampling-fork-window.md b/changelog.d/20261102_l1-latency-sampling-fork-window.md new file mode 100644 index 0000000..f4e3862 --- /dev/null +++ b/changelog.d/20261102_l1-latency-sampling-fork-window.md @@ -0,0 +1,9 @@ +### SaaS API — L1 hit-latency sampling: the hookless-fork startup window + +- [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency), **Sampling** + ([API-81](spec/saas-api.md#api-81)): names one exception to the independent per-call draw. + A child that inherits a backend instance through a fork that runs none of the SDK's at-fork + handlers may repeat its parent's and siblings' draw sequence until it builds a backend + instance of its own, or first uses an inherited instance's report (its first request other + than the health check, or its first timed L1 hit). Each call is still timed with + probability 1/N. No header, value format or range of N changes. diff --git a/spec/saas-api.md b/spec/saas-api.md index b717d66..58119a9 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -596,7 +596,7 @@ An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1 The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store. -**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates the latency of every L1 hit of 60 s or less, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. +**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. One exception applies to a child process that inherits a backend instance through a fork that runs none of the SDK's at-fork handlers, such as a fork made from C without the language runtime's fork hooks. Until that child builds a backend instance of its own, or first uses an inherited instance's report accumulator (its first request other than `GET /v1/cache/health`, or its first timed L1 hit), its draws MAY repeat the sequence that its parent and every other process forked from the same state draw. Each call is still timed with probability 1/N, but those processes time the same call positions. Every other draw, including every draw such a child makes after that point, is independent as above. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates the latency of every L1 hit of 60 s or less, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. **Clock.** An SDK MUST NOTAPI-82 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify: From 63c87a640575132f968f1406e950bbfb9800f648 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Mon, 2 Nov 2026 01:15:23 +0000 Subject: [PATCH 2/2] docs(saas-api): end the fork window at accumulator creation or use, not backend construction --- changelog.d/20261102_l1-latency-sampling-fork-window.md | 9 +++++---- spec/saas-api.md | 2 +- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/changelog.d/20261102_l1-latency-sampling-fork-window.md b/changelog.d/20261102_l1-latency-sampling-fork-window.md index f4e3862..f714a42 100644 --- a/changelog.d/20261102_l1-latency-sampling-fork-window.md +++ b/changelog.d/20261102_l1-latency-sampling-fork-window.md @@ -3,7 +3,8 @@ - [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency), **Sampling** ([API-81](spec/saas-api.md#api-81)): names one exception to the independent per-call draw. A child that inherits a backend instance through a fork that runs none of the SDK's at-fork - handlers may repeat its parent's and siblings' draw sequence until it builds a backend - instance of its own, or first uses an inherited instance's report (its first request other - than the health check, or its first timed L1 hit). Each call is still timed with - probability 1/N. No header, value format or range of N changes. + handlers may repeat its parent's and siblings' draw sequence until it creates a report + accumulator, or first uses one it inherited (its first request other than the health check + on an instance that has one, or its first timed L1 hit). A backend instance with no + accumulator, such as one with telemetry off, does not end the window. Each call is still + timed with probability 1/N. No header, value format or range of N changes. diff --git a/spec/saas-api.md b/spec/saas-api.md index 58119a9..ecd8ded 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -596,7 +596,7 @@ An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1 The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store. -**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. One exception applies to a child process that inherits a backend instance through a fork that runs none of the SDK's at-fork handlers, such as a fork made from C without the language runtime's fork hooks. Until that child builds a backend instance of its own, or first uses an inherited instance's report accumulator (its first request other than `GET /v1/cache/health`, or its first timed L1 hit), its draws MAY repeat the sequence that its parent and every other process forked from the same state draw. Each call is still timed with probability 1/N, but those processes time the same call positions. Every other draw, including every draw such a child makes after that point, is independent as above. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates the latency of every L1 hit of 60 s or less, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. +**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. One exception applies to a child process that inherits a backend instance through a fork that runs none of the SDK's at-fork handlers, such as a fork made from C without the language runtime's fork hooks. Until that child creates a report accumulator for a backend instance, or first uses an accumulator it inherited (its first request other than `GET /v1/cache/health` on an instance that has one, or its first timed L1 hit), its draws MAY repeat the sequence that its parent and every other process forked from the same state draw. Each call is still timed with probability 1/N, but those processes time the same call positions. Building or using a backend instance that has no accumulator, such as one with telemetry turned off or on a clock that does not qualify, does not end the window. Every other draw, including every draw such a child makes after that point, is independent as above. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates the latency of every L1 hit of 60 s or less, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. **Clock.** An SDK MUST NOTAPI-82 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify: