Repository navigation
Commit f53397c
Read the rate-limit headers, and hold a 429 once per client (#48)
* feat: read the rate-limit headers, and hold a 429 once per client
Same treatment as the Python sibling, same reasoning.
The SDK read exactly one header, Retry-After, and only after a 429 had already
happened. It could tell you that you had run out, never that you were about to.
Both meters are now on the client. The per-minute REST one, and the hourly
history budget the API started publishing today:
tp.rateLimit.rest.remaining
tp.rateLimit.history.remaining
secondsUntilReset(tp.rateLimit.rest)
ABSENCE IS NOT ZERO, and the design turns on it. An unmetered plan advertises
no figures, and neither does a publicly cacheable response, because the numbers
are per-caller and a shared cache would hand one caller's budget to another --
so anonymous calls carry nothing. null means the server did not say.
isExhausted is true only when it said zero. Reading an unknown as zero would
stall every anonymous client permanently, which is the first thing the tests
pin.
reset is a relative countdown frozen when it was read, so secondsUntilReset
ages it. Using the raw value later is how a client waits an hour longer than it
needs to, and it is the same bug the server had in its cached headers.
The client acts on what it reads: a window the server said is spent is waited
out rather than walked into, because that request is a certain 429 that also
costs a unit of budget to refuse. retry: { respectRemaining: false } opts out.
A 429 IS NOW HELD ONCE FOR THE WHOLE CLIENT. The wait belongs to the caller,
not to whichever request met it. Ten concurrent requests each slept their own
Retry-After and then retried at the same instant, re-tripping the limit
together. It goes on a shared gate with a little jitter, taken once, and the
retry path no longer pays it a second time. A shorter wait arriving while a
longer one is in force no longer brings the gate forward. Past maxRetryAfterMs
the gate is deliberately left open: we throw instead, and blocking the next
call for most of an hour is the opposite of letting the caller checkpoint.
The cache wrapper gap that Python hit does not exist here -- the client holds
the inner transport directly -- but there is a test for it either way, because
caching is on by default and every other test turns it off.
Verified against production: reads limit=300 remaining=296 reset=40
policy="300;w=60" on a real anonymous call, with the history meter correctly
absent and isExhausted false.
123 tests, 21 new. Mutation-checked: treating an unknown remaining as exhausted
fails 3, letting a blank response erase what we knew fails 1, bringing the gate
forward fails 1, removing the jitter fails 1, and not ageing the reset fails 2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: paged history crashed on the default cache, plus three more
Four review agents, two of them independently found the same blocker.
PAGED HISTORY CRASHED FOR EVERY DEFAULT CALLER. I added #gate and #hold as
ECMAScript private fields on Transport. client.ts wraps the transport in a
Proxy to add caching, and a Proxy forwards methods with `this` bound to the
proxy, so the private brand check fails:
TypeError: Receiver must be an instance of class Transport
history.days() threw on page two for anyone who had not passed cache: false --
the paid feature, on the default configuration. 130 tests passed because the
paging test helper hardcodes cache: false, so every paging test took the one
path where the bug cannot fire. They are TS-private now, which compiles to a
plain property and forwards fine, and there is a paging test on the default
cache that fails if the # fields come back.
THE GATE TIMED OFF THE WALL CLOCK. closeFor stored Date.now() + ms and waitMs
subtracted Date.now(), so a backward NTP step turned a five-second wait into
however far the clock moved -- an hour, measured -- with nothing bounding it,
because the cap is applied when the gate is armed and never when it is served.
Monotonic now, which is what the Python sibling had from the start.
on429: false DID NOT OPT OUT. It threw the error the caller asked for and then
closed the shared gate anyway, so their NEXT call blocked for the full
Retry-After. An advertised switch that switches nothing is worse than none.
THE PREMISE WAS BACKWARDS. The docs said anonymous calls carry no figures.
Measured against production it is the other way round for the per-minute meter;
the hourly history set is the one withheld from cacheable responses. And a
cached response's figures belong to whoever populated the entry -- age: 9 with
an unmoving remaining: 285, served to everyone -- so a non-zero Age is now
treated as saying nothing. A cache MISS carries no Age and is still recorded.
Also: strict integer parsing, so "0.4" no longer reads as 0 and makes
isExhausted true; fifteen edge cases now agree byte for byte with Python. Two
timing tests made deterministic instead of asserting ranges around the real
clock. Gate, readRateLimits and the UNKNOWN_* sentinels unexported -- Gate had
no route to the client's instance and readRateLimits' parameter type was not
exported, so nobody could name what they were passing. A CHANGELOG Changed
section saying plainly that calls may now block before sending.
131 tests. tsc, eslint, prettier and the build clean. Mutation-checked:
restoring the # fields fails the new paging test, honouring cached figures
fails 1, and both opt-outs are pinned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: maxRetryAfterMs bounds the whole call, and three vacuous tests
Same per-call budget as the Python sibling, same measurement behind it: the
gate wait and the spent-window wait stack, so a 429 carrying both a
Retry-After and RateLimit-Remaining: 0 blocked for 180 seconds under a 120
second cap. Every leg was under the cap, so the per-leg check never fired.
maxRetryAfterMs is a per-call budget now.
Three tests could not fail, all found by mutation:
- The spent-window assertion was an upper bound only, on the single test
covering the flagship behaviour. `sleep(left)` in place of
`sleep(left * 1000)` -- seven milliseconds instead of seven seconds, a
thousandfold too short -- passed green. Both bounds now.
- The jitter test was sound against Date.now()'s millisecond granularity and
MY monotonic-clock change gutted it: performance.now() ticks between the 20
calls, so the set is distinct with or without jitter. It asserted that time
passes. It runs against a frozen clock now and bounds the spread.
- Nothing proved a sleep was AWAITED rather than merely requested. Dropping
every await in hold() passed all 135 tests. A fake sleep that resolves on a
later macrotask and counts itself pending now fails if a request starts
while a hold is in flight. Honest limit: dropping ONE await is still masked
by the other, because with one await remaining the hold does still block.
No test in either file had ever sent a 429 carrying rate-limit headers, which
is why none of this surfaced: both docstrings claimed to cover the gate and
every fixture was a 200.
136 tests. tsc, prettier and the build clean. Mutation-checked: removing the
budget fails 1, deleting the jitter fails 2, the thousandfold-short sleep
fails 1, dropping all awaits fails 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: Retry-After parsing, two release herds, and a NaN that disabled the gate
Same three fixes as the Python sibling, same measurements behind them.
RETRY-AFTER PARSING. Only null reaches the exponential backoff, so a header
parsing to zero meant no wait at all -- and `Retry-After: 0` is legal per RFC
9110, as is a negative and an already-past date. Measured in Python, which had
the identical shape: four requests in 3ms against a server that had just said
429, and 204 a second across ten threads.
Number() was also far too generous for a `delta-seconds = 1*DIGIT` field: it
read ' ' as 0 and spun, and '0x10' as 16 and slept 48 seconds across three
retries where Python correctly took 2. The two parsers in this file now follow
the same rule, which they did not after the last round tightened only one.
THE SPENT-WINDOW PATH WAS A PURE HERD: ten waiters left inside the SAME
MILLISECOND, measured. Every one derived its deadline from the same observedAt
and slept to the same absolute instant with no spread. It runs through the same
jitter as the gate now.
A WAITER THAT WOKE INTO A RE-CLOSED GATE SENT ANYWAY -- measured waking at
584ms with the gate shut for another two seconds. It re-reads now, but only
when the deadline actually MOVED. My first attempt re-read unconditionally and
spun 57 times against a frozen test clock, which is the correct behaviour of a
wrong loop.
A NaN jitter would have made setTimeout fire immediately and silently disable
the gate rather than fail loudly. Guarded.
The new spread test is measured against a frozen clock, because against a live
one `left` varies by itself and the assertion passes with the spread deleted.
137 tests. Mutation-checked: removing either spread fails, and so does
removing the budget.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 2068594 commit f53397c
9 files changed
Lines changed: 1055 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
8 | 75 | | |
9 | 76 | | |
10 | 77 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| |||
161 | 161 | | |
162 | 162 | | |
163 | 163 | | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
164 | 207 | | |
165 | 208 | | |
166 | 209 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | | - | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
51 | 52 | | |
52 | 53 | | |
53 | 54 | | |
| 55 | + | |
54 | 56 | | |
55 | 57 | | |
56 | 58 | | |
| |||
60 | 62 | | |
61 | 63 | | |
62 | 64 | | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
63 | 85 | | |
64 | 86 | | |
65 | 87 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
42 | 43 | | |
0 commit comments