Repository navigation
ruby/spec: raise the floor, 14200 -> 14900 - #419
Conversation
|
Holding this. Its premise is wrong. I argued the total was safe to pin tight because two runs of the same commit both reported 14878 to the example. That was two green samples, not determinism — the lane still OOMs, and this PR own ratchet run proved it: memory 13396 MB -> 214 MB, runner gone, same signature as before. There is a fifth blowup, the largest of them, which I catalogued in the first sweep and never diagnosed: o = Object.new
class << o
CONST = self
end
class << o
x = CONST # 14 GB. ruby: instant
endIt is the constant read alone — not the comparison the spec makes, and not A floor is only worth pinning to a measurement the lane can retake. This waits for that fix. |
The floor had not moved while the total climbed under it: 14303 on 17 August, 14567 and 14625 on the 18th, and 14949-14963 now that the five memory blowups are fixed and five more spec files load. A floor 750 below the measurement tolerates a regression of 750 examples in silence, which is the one thing a ratchet exists not to do. Not pinned to the measurement, and I was wrong to propose that the first time. I argued the total was deterministic because two runs of one commit both said 14878. Two more runs of the same tree then said 14949 and 14963 — fourteen apart. Two samples agreeing is not determinism. 14900 is 49 below the lowest of those, which absorbs a jitter observed at 14, and 700 above where the floor sat. What would drop it much further is a heavy spec file failing to load, worth about 150 examples. That is now unlikely rather than routine: with the blowups fixed, the sweep's lowest free memory is 13.2 GB of 15, where a fortnight of runs died at 214 MB. If this lane ever reds by roughly 150, read "failed to load" in the log before looking for a conformance regression — it is 74 now, and was 79 before this week.
5d0ecf5 to
3c09a0f
Compare
The floor had not moved while the total climbed under it:
A floor 750 below the measurement tolerates a regression of 750 examples in silence, which is the one thing a ratchet exists not to do.
Not pinned to the measurement, and I was wrong to propose that first
The first version of this PR set the floor to the exact total, arguing the total was deterministic because two runs of one commit both said 14878. Two more runs of the same tree then said 14949 and 14963 — fourteen apart. Two samples agreeing is not determinism, and that PR's own ratchet run then OOMed, which is what held it.
14900 is 49 below the lowest of those — absorbing a jitter observed at 14 — and 700 above where the floor sat.
Why the lane can be trusted to retake this measurement now
Five memory blowups are fixed (#415, #416, #417, #418, #421; #412 is the watcher that found them). The difference is not "it went green", it is what the watcher reads during a run:
The sweep no longer approaches the runner's memory at all.
If it ever reds by about 150
That is a heavy spec file failing to load, not a conformance regression — read
failed to loadin the log first. It is 74 now, and was 79 before this week; five more files load than did.🤖 Generated with Claude Code