Skip to content

Net::FTP.default_passive, and the options a nil host was throwing away - #928

Merged
tannevaled merged 3 commits into
mainfrom
ftp-default-passive
Nov 13, 2026
Merged

tannevaled merged 3 commits into
mainfrom
ftp-default-passive

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Net::FTP.default_passive, and the options a nil host was throwing away

Stacked on #927 — the data-connection GVL release has to land first, and the
reason is in that pull request: this change moves the net-ftp specs into
active mode, which is where the Accept #927 wraps actually runs.

The gap

library/net-ftp/spec_helper.rb, which all 56 files require, opens with:

if defined?(Net::FTP.default_passive)
  Net::FTP.default_passive = false
end

rbgo had no such accessor, so that guard was skipped in silence and the
whole suite ran against a connection rbgo kept in passive mode — while the
fixture server implements PORT/EPRT and has no pasv method at all.
That is the 451 Exception: undefined method 'pasv' those specs report.

Implemented against the reference, not against expectations

net-ftp 0.3.9 lib/net/ftp.rb:96 (@@default_passive = true), :159-167 (the
accessors), :286-290 (if options[:passive].nil? → the default). The value
lives in the class variable MRI uses, so
Net::FTP.class_variable_get(:@@default_passive) answers here as it does
there, and one place holds it rather than two that can disagree.

⛔ Two defects the comparison found that reading our own code would not

Net::FTP.new(nil, passive: true) answered false. ftpInit returned
before reading the options Hash whenever the host was nil — so on that path
every option was dropped: binary, the timeouts, the credentials, all of it.
MRI applies the options first and connects separately, and ruby/spec drives
exactly that shape: @ftp.send(:initialize, nil, options) and then asserts
passive (initialize_spec.rb:201-226). A mechanism wired at only some of its
doors, again — the ninth this week.

passive: nil meant false. ftpApplyOpts tested the key's presence;
MRI tests the value, and nil is "no opinion" falling back to the default.
This is the opposite of read_timeout in the same Hash three lines away,
where nil genuinely is a request (for no bound). Both now say which they are,
in a comment, because that is precisely the pair someone will conflate next.

Tests and measurement

A 12-line table driven through rbgo, every line annotated with the reference
line it comes from — including that a connection built before
default_passive = keeps its mode, and that the writer returns its argument.
Plus ftpDefaultPassive's fallbacks (nil class, absent or nil class variable →
true), which no Ruby program can reach and which decide the transfer mode of
every connection.

⛔ Coverage measured before pushing, with the tool added in #925 — which
caught my own gap: covratchet -require ftpDefaultPassive,ftpApplyOpts,ftpInit
first reported ftpDefaultPassive at 66.7%, and reports all three at
100% now. That tool exists because twice in one day a commit written to
answer the coverage ratchet did not move the measurement.

Refs #916, #925, #927

(#916 stays open on purpose: the sweep that would show this moving is still
deferred — this machine has been carrying a 1-minute load average between 23
and 86, and a per-file timeout measured under that cannot tell a hang from a
slow file. #925 and #927 are the pull requests this one builds on.)

🤖 Generated with Claude Code

tannevaled and others added 2 commits November 13, 2026 13:42
…hrinks

#921 bounded the control connection and #924 released the GVL around it. Both
left the DATA connection holding it, and the gate #924 added said so in its
seeded backlog -- four entries I wrote down rather than fixed, so the gate
could land without a sprawling change.

This is the change. dialData, acceptData, sendData, retrlines and retrbinary
now run their blocking call inside ftpIO, and the four entries are DELETED from
testdata/gvl-blocking-backlog.txt. That list may only shrink, and an entry that
stops reporting fails the gate too, so a fix cannot leave its allowance behind.

⛔ acceptData is the one #916 is actually about. In ACTIVE mode the client
listens and the server dials back, so a Ruby program that serves itself has
both ends in one process -- and holding the GVL in that Accept parks every
other Ruby thread, INCLUDING the one that would make the call it is waiting
for. ruby/spec's net-ftp fixtures are exactly that shape, and their
spec_helper.rb puts Net::FTP into active mode (see #916's comment: rbgo has no
Net::FTP.default_passive, so today that helper is skipped in silence and the
suite never reaches this path at all).

Only the blocking call goes inside each closure. ftpConnErr raises, and a panic
crossing a released GVL would unwind without re-acquiring it, so every raise
stays outside.

⛔ NO BEHAVIOURAL TEST, AND THE REASON IS NOT LAZINESS. The in-repo FTP test
server is Go-side, so it cannot reproduce this deadlock at all: nothing about
rbgo's emulated GVL blocks a Go goroutine, and a test built on it would pass
whether or not the fix is present. A real regression test needs BOTH ends in
Ruby -- a dummy FTP server in a Ruby Thread, which is what ruby/spec's fixture
is -- and that is worth doing properly rather than approximating with something
that looks like proof and is not.

What does guard the property is the gate: TestEveryBlockingSocketCallReleasesTheGVL
fails in about a second if any of these five is unwrapped again, and the
backlog deletion means it also fails if one silently stops being reported.

The measurement that would show this moving -- the library/net-ftp sweep -- is
deferred: this machine has been carrying a 1-minute load average between 23 and
86, and a per-file timeout measured under that cannot tell a hang from a slow
file.

Refs #916

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ruby/spec's library/net-ftp/spec_helper.rb — which all 56 files require —
opens with:

    if defined?(Net::FTP.default_passive)
      Net::FTP.default_passive = false
    end

rbgo had no such accessor, so the guard was skipped in SILENCE and the whole
suite ran against a connection rbgo kept in PASSIVE mode, while the fixture
server implements PORT/EPRT and has no `pasv` method at all. That is the
`451 Exception: undefined method 'pasv'` those specs report (#916).

Implemented against the reference, not against expectations: net-ftp 0.3.9
lib/net/ftp.rb:96 (`@@default_passive = true`), :159-167 (the accessors) and
:286-290 (`if options[:passive].nil?` → the default). The value lives in the
CLASS VARIABLE MRI uses, so Net::FTP.class_variable_get(:@@default_passive)
answers here as it does there and there is one place holding it, not two that
can disagree.

Two defects the reference comparison turned up that reading our own code would
not have:

  - ⛔ `Net::FTP.new(nil, passive: true)` answered FALSE. ftpInit returned
    before reading the options Hash whenever the host was nil, so EVERY option
    was dropped on that path — binary, timeouts, credentials, all of it. MRI
    applies the options first and connects separately, and ruby/spec drives
    exactly that shape: `@ftp.send(:initialize, nil, options)` then asserts
    `passive` (initialize_spec.rb:201-226). A mechanism wired at only some of
    its doors, again.

  - ⛔ `passive: nil` meant FALSE. ftpApplyOpts tested the KEY's presence, but
    MRI tests the VALUE: nil is "no opinion" and falls back to the default.
    Note this is the OPPOSITE of read_timeout in the same Hash three lines
    away, where nil IS a request (for no bound) — easy to conflate, so both
    now say which they are in a comment.

Tests: a 12-line table run through rbgo, each line annotated with the
reference line it comes from, plus ftpDefaultPassive's fallbacks (nil class,
absent or nil class variable → true), which no Ruby program can reach and
which decide the transfer mode of every connection.

⛔ Measured before pushing with the tool added in #925, which caught my own
gap: `covratchet -require ftpDefaultPassive,ftpApplyOpts,ftpInit` first
reported ftpDefaultPassive at 66.7%, and reports all three at 100% now.

Refs #916

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 8c8ec0a into main Nov 13, 2026
26 checks passed
@tannevaled
tannevaled deleted the ftp-default-passive branch November 13, 2026 16:15
tannevaled added a commit that referenced this pull request Nov 15, 2026
…938)

Every figure here was already measured and merged; none of it needed a new run.
What it needed was someone to ask which published sentences the merges had made
false, which is a different question from "what did we fix".

## The four

1. `library/socket` **8 killed -> 7**, and the headline **75 -> 9** is **75 -> 8**.
   #935 measured it in the sentence "nine files killed by the clock across
   `library/socket` + `library/timeout` before, eight after".

2. A withdrawn figure. The README said the remaining hangs were "275 passing
   against MRI's 371". That control scored **7** on those 190 files, because
   `library/socket/spec_helper.rb` calls `MSpec.enable_feature` and the shim
   defined no `MSpec` (#936). A number counted against a broken control is not a
   smaller number, it is not a number; it is removed rather than adjusted.

3. The corpus map's "the rest is concentrated: matrix 233, bigdecimal 180, ..."
   list was compiled while `library/socket` contributed a flat **zero on both
   sides**. Repaired, socket is MRI 1790 / rbgo 224 -- about **-1560**, larger
   than every module on that list put together (#937). The two totals are
   deliberately **not summed**: different runs, different MRI patch releases
   (4.0.7 and 4.0.6), and a sum across two measurements is how two errors cancel.
   The published gap is marked a **floor** instead.

4. `docs/SHARED-ENGINES.md` still concluded, at the top of its `net-ftp` section,
   that **"#916 is not closed, because the measurement did not move ... still 39
   killed"**, and told the reader `net-ftp` was not available for extraction.
   That was true on 2026-11-13 and is the opposite of the outcome: #927 released
   the GVL on the **data** connection and #928 added `Net::FTP.default_passive`,
   and net-ftp is **0 killed, -128**. The section now records why the inference
   was wrong -- "the fix did not move the measurement" was read as "the fix is
   incomplete" when the mechanism was wired at only some of its doors -- and
   lifts the extraction blocker while naming the -128 the new home inherits.

## Why a doc gate did not catch any of them

Seven README drift gates run in this repository's root package and all seven are
green both before and after this change: they check the conformance total
against `BASELINE`, the Ruby version against the runtime, the Go floor, the
numeric surface, the wasm lane, and the shim line numbers the prose cites. Not
one of them looks at a figure in a dated prose table, because there is nothing
to compare such a figure against -- the run that produced it is gone.

That is the honest limit, and it is stated in the README rather than papered
over: the dated map is explicitly "not a gate", and the only number CI enforces
is per-file. The gates caught the thing they can catch -- when a 22-line
insertion moved two cited line numbers, every lane went red.

Refs #916, #933, #935, #936, #937

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant