docs(spec): section 7.6, what a reconnect costs - #3
Merged
Merged
Conversation
Section 7.2 already said the server's per-environment state goes with a closed connection, but it said it inside the resync case, as if resync were the only way a connection ends mid-run. It is not, and the consequence is the same however the connection died: the previous observation, the policy step state and the terminated/truncated flags are gone, so a feedback whose action arrived on an earlier connection describes a transition that can no longer be completed. Section 7.6 states that generally, adds the obligation on the client to drop held feedback across a reconnect, and adds the obligation on the server not to sever a healthy connection for a liveness reason section 7.3 already covers - WebSocket keepalive pings being the specific trap, since a CPU-bound learn step cannot answer one. Also a conformance checklist line, and a pointer from 7.5, which said a departing client costs nothing - true of one that is finished, not of one that means to come back. This is written up from a real failure: plugrl-server left the websockets default 20 s ping in place, and on a CPU-only machine the quickstart dropped twice in six minutes, feeding the learner an empty transition each time. Fixed in plugrl-server#4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 11, 2026
Merged
Section 7.6 told servers not to sever a healthy connection for a liveness reason section 7.3 already covers. It did not say that 7.3 does not cover everything, which made the advice look free. It is not free. The feedback timeout bounds the wait between an action and its feedback, and nothing bounds the wait for the next infer, so a peer that dies without closing its socket is now noticed only when TCP gives up. That is a fifth Gap and it belongs beside the other four. The alternative was a long ping timeout rather than none, and the reason it was rejected belongs in the record too: the value would have to exceed both the longest learn step and the longest legitimate pause before a client's first infer, and neither is the server's to know. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The section told servers not to sever a healthy connection for a liveness reason 7.3 already covers, named WebSocket keepalive pings as the trap, and added a fifth Gap describing what turning them off gives up. All three rested on the claim that a CPU-bound learn step holds the event loop past the ping timeout. It does not. plugrl-server runs learn through asyncio.to_thread, so the loop stays free, and five learn steps of about 190 seconds each - nine times the timeout - closed nothing. The incident that prompted the section was a machine suspending for nearly two hours, and the timeout that fired was the client library's, not the server's. What is measured and survives is the rest of the section: connection state is per connection, a client must drop held feedback across a reconnect, and a server should say something when feedback arrives with no step state. Those came from reading the handler and were reproduced in a unit test, not inferred from the incident, which is why they are unaffected. The corollary and the fifth Gap are replaced by a note saying reconnects come from outside the protocol and the rule is written in terms of the reconnect for that reason. The historical note now records what actually happened, including the diagnosis that measurement overturned. Measurement in plugrl-server experiments/e8-keepalive-hypothesis. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Section 7.2 already said the server's per-environment state goes with a closed connection, but it said it inside the resync case, as though resync were the only way a connection ends mid-run.
It is not, and the consequence does not depend on the cause. The previous observation, the policy step state and the done flags are gone, so a
feedbackwhoseactionarrived on an earlier connection describes a transition nothing can complete.§7.6 adds:
Also a conformance checklist line, and a pointer from §7.5, which said a departing client costs nothing. True of one that is finished, not of one that means to come back.
One thing this PR got wrong and then removed
An earlier revision told servers not to sever a healthy connection for a liveness reason §7.3 already covers, named WebSocket keepalive pings as the trap, and added a fifth Gap describing the price of turning them off.
All three rested on the claim that a CPU-bound learn step holds the event loop past the ping timeout. It does not.
plugrl-serverrunslearnthroughasyncio.to_thread, and five learn steps of 177–190 s each — nine times the timeout — closed nothing. The incident that prompted the section was a machine suspending for 1 h 53 min, and the timeout that fired came from the client library, not the server's.Measurement:
experiments/e8-keepalive-hypothesis/in PlugRL/plugrl-server#4.The rest of §7.6 is unaffected, because it was read out of the server's handler and reproduced in a unit test rather than inferred from the incident.
Client half in PlugRL/plugrl-env-client#5; site tour in PlugRL/plugrl.github.io#4.
🤖 Generated with Claude Code