Repository navigation
Net::FTP: the data connection releases the GVL too, and the backlog shrinks - #927
Merged
Merged
Conversation
…hrinks #921 bounded the control connection and #924 released the GVL around it. Both left the DATA connection holding it, and the gate #924 added said so in its seeded backlog -- four entries I wrote down rather than fixed, so the gate could land without a sprawling change. This is the change. dialData, acceptData, sendData, retrlines and retrbinary now run their blocking call inside ftpIO, and the four entries are DELETED from testdata/gvl-blocking-backlog.txt. That list may only shrink, and an entry that stops reporting fails the gate too, so a fix cannot leave its allowance behind. ⛔ acceptData is the one #916 is actually about. In ACTIVE mode the client listens and the server dials back, so a Ruby program that serves itself has both ends in one process -- and holding the GVL in that Accept parks every other Ruby thread, INCLUDING the one that would make the call it is waiting for. ruby/spec's net-ftp fixtures are exactly that shape, and their spec_helper.rb puts Net::FTP into active mode (see #916's comment: rbgo has no Net::FTP.default_passive, so today that helper is skipped in silence and the suite never reaches this path at all). Only the blocking call goes inside each closure. ftpConnErr raises, and a panic crossing a released GVL would unwind without re-acquiring it, so every raise stays outside. ⛔ NO BEHAVIOURAL TEST, AND THE REASON IS NOT LAZINESS. The in-repo FTP test server is Go-side, so it cannot reproduce this deadlock at all: nothing about rbgo's emulated GVL blocks a Go goroutine, and a test built on it would pass whether or not the fix is present. A real regression test needs BOTH ends in Ruby -- a dummy FTP server in a Ruby Thread, which is what ruby/spec's fixture is -- and that is worth doing properly rather than approximating with something that looks like proof and is not. What does guard the property is the gate: TestEveryBlockingSocketCallReleasesTheGVL fails in about a second if any of these five is unwrapped again, and the backlog deletion means it also fails if one silently stops being reported. The measurement that would show this moving -- the library/net-ftp sweep -- is deferred: this machine has been carrying a 1-minute load average between 23 and 86, and a per-file timeout measured under that cannot tell a hang from a slow file. Refs #916 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tannevaled
added a commit
that referenced
this pull request
Nov 15, 2026
…938) Every figure here was already measured and merged; none of it needed a new run. What it needed was someone to ask which published sentences the merges had made false, which is a different question from "what did we fix". ## The four 1. `library/socket` **8 killed -> 7**, and the headline **75 -> 9** is **75 -> 8**. #935 measured it in the sentence "nine files killed by the clock across `library/socket` + `library/timeout` before, eight after". 2. A withdrawn figure. The README said the remaining hangs were "275 passing against MRI's 371". That control scored **7** on those 190 files, because `library/socket/spec_helper.rb` calls `MSpec.enable_feature` and the shim defined no `MSpec` (#936). A number counted against a broken control is not a smaller number, it is not a number; it is removed rather than adjusted. 3. The corpus map's "the rest is concentrated: matrix 233, bigdecimal 180, ..." list was compiled while `library/socket` contributed a flat **zero on both sides**. Repaired, socket is MRI 1790 / rbgo 224 -- about **-1560**, larger than every module on that list put together (#937). The two totals are deliberately **not summed**: different runs, different MRI patch releases (4.0.7 and 4.0.6), and a sum across two measurements is how two errors cancel. The published gap is marked a **floor** instead. 4. `docs/SHARED-ENGINES.md` still concluded, at the top of its `net-ftp` section, that **"#916 is not closed, because the measurement did not move ... still 39 killed"**, and told the reader `net-ftp` was not available for extraction. That was true on 2026-11-13 and is the opposite of the outcome: #927 released the GVL on the **data** connection and #928 added `Net::FTP.default_passive`, and net-ftp is **0 killed, -128**. The section now records why the inference was wrong -- "the fix did not move the measurement" was read as "the fix is incomplete" when the mechanism was wired at only some of its doors -- and lifts the extraction blocker while naming the -128 the new home inherits. ## Why a doc gate did not catch any of them Seven README drift gates run in this repository's root package and all seven are green both before and after this change: they check the conformance total against `BASELINE`, the Ruby version against the runtime, the Go floor, the numeric surface, the wasm lane, and the shim line numbers the prose cites. Not one of them looks at a figure in a dated prose table, because there is nothing to compare such a figure against -- the run that produced it is gone. That is the honest limit, and it is stated in the README rather than papered over: the dated map is explicitly "not a gate", and the only number CI enforces is per-file. The gates caught the thing they can catch -- when a 22-line insertion moved two cited line numbers, every lane went red. Refs #916, #933, #935, #936, #937 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Net::FTP: the data connection releases the GVL too, and the backlog shrinks
#921 bounded the control connection; #924 released the GVL around it and added a
gate. Both left the data connection holding it — and the gate said so, in
four seeded backlog entries I wrote down rather than fixed, so that the gate
could land without a sprawling change nobody would review.
This is that change.
dialData,acceptData,sendData,retrlinesandretrbinarynow run their blocking call insideftpIO, and the four entries aredeleted from
testdata/gvl-blocking-backlog.txt. That list may only shrink,and an entry that stops reporting fails the gate too, so a fix cannot leave its
allowance behind for the next regression to hide in.
⛔
acceptDatais the one issue 916 is actually aboutIn active mode the client listens and the server dials back, so a Ruby
program that serves itself has both ends in one process. Holding the GVL in that
Acceptparks every other Ruby thread — including the one that would make thecall it is waiting for.
ruby/spec's net-ftp fixtures are exactly that shape. Their
spec_helper.rbputsNet::FTPinto active mode, and rbgo has noNet::FTP.default_passive, so todaythat helper is skipped in silence and the suite never reaches this path at all
(written up in the issue's latest comment). The order matters and it is the
opposite of the obvious one: release the GVL here first, add
default_passivesecond, re-measure third — otherwise today's fast errors turninto fresh hangs.
Only the blocking call goes inside each closure:
ftpConnErrraises, and a paniccrossing a released GVL would unwind without re-acquiring it, so every raise
stays outside.
⛔ No behavioural test, and the reason is not laziness
The in-repo FTP test server is Go-side. Nothing about rbgo's emulated GVL
blocks a Go goroutine, so that server cannot reproduce this deadlock at all: a
test built on it would pass whether or not this change is present. A real
regression test needs both ends in Ruby — a dummy FTP server in a Ruby
Thread, which is what ruby/spec's fixture is — and that is worth doing properlyrather than approximating with something that looks like proof and is not.
What does guard the property is the gate itself:
TestEveryBlockingSocketCallReleasesTheGVLfails in about a second if any ofthese five is unwrapped again, and the backlog deletion means it also fails if
one silently stops being reported.
The measurement that would show this moving — the
library/net-ftpsweep — isdeferred: this machine has been carrying a 1-minute load average between 23
and 86, and a per-file timeout measured under that cannot tell a hang from a slow
file. Reporting a number taken there would be worse than reporting none.
Refs #916, #921, #924
(#916 stays open on purpose — the measurement has not moved. #921 and #924 are
the merged pull requests this one continues.)
🤖 Generated with Claude Code