Skip to content

library/socket is 1560 examples behind MRI: the largest module gap, invisible until the oracle could load it #937

Open

Description

@tannevaled

library/socket is 1 560 examples behind MRI — the largest module gap in the corpus, and it was invisible until today

Measured 2026-11-14 against MRI 4.0.6, same corpus, 30 s per file, with the
oracle repaired (#930) and MSpec.enable_feature defined (#936):

passing
MRI 4.0.6 1790
rbgo 224

(That figure covers library/socket + library/timeout, 190 files;
library/timeout is −6 of it, so socket is ≈ −1560.)

⛔ Why no one has seen this before

library/socket/spec_helper.rb calls MSpec.enable_feature, which the
conformance shim did not define, so every one of those 190 files raised
NameError at load and scored 0 — including under MRI.
A sweep then prints
control 0 passing, and a control of zero is indistinguishable from a module
with no gap. The same sweep that now reads −1566 read +217 in rbgo's favour
twenty minutes earlier.

This also means the corpus map in the README understates the picture: its
"the rest is concentrated: matrix 233, bigdecimal 180, stringscanner 154…"
list was compiled while socket contributed a flat zero. Socket is larger than
all of those together.

⛔ Checked, so the worry stops here: only four directories under library/
carry their own spec_helper.rb — matrix, socket, net-ftp, readline —
and of those, matrix tolerates a missing matcher with a rescue, net-ftp
guards its call behind defined?, and readline behind a LoadError that
fails on this machine. socket was the only one calling without a guard.
The blind spot was single, not diffuse.

What the gap is made of — surface, not hangs

The eight files still killed by the clock (#933) are a small part of this. The
bulk is unimplemented surface, and it clusters:

area examples behind
Addrinfo#initialize alone −151 (195 → 44)
BasicSocket#recvmsg + #recvmsg_nonblock −100 (together, 0 passing)
AncillaryData#initialize −40
BasicSocket address/option accessors — shutdown, remote_address, local_address, connect_address, setsockopt, getsockopt, recv −188 together
the _nonblock family — recvfrom_nonblock, accept_nonblock, UDPSocket#recvfrom_nonblock −79

156 files carry a negative delta. Two whole classes — Addrinfo and
AncillaryData — account for a quarter of the gap on their own, and
recvmsg/ancillary data is the next block.

Why this is worth its own issue

#916, #931 and #933 are all about files that hang; this is about files that
run and fail, which needs different work and a different order. The natural
first slice is Addrinfo, because it is self-contained (address parsing and
formatting, no I/O) and it is the single biggest line in the table.

Refs #916, #930, #933, #936

Prior art: priorart "library/socket Addrinfo AncillaryData recvmsg parity gap" at 2026-11-14 21:51 CET reported prior art; checked: priorart --repo go-embedded-ruby/ruby 'library/socket Addrinfo AncillaryData recvmsg parity gap' returned 259 broad matches. The adjacent ones are all about files that HANG (#916, #931, #933) or about the measurement itself (#930, #936); #922 is one missing constant. None covers the unimplemented socket SURFACE, which is what this measures: files that run and fail, needing different work and a different order.

Activity

  1. tannevaled commented on Nov 14, 2026

    @tannevaled
    ContributorAuthor

    The Addrinfo slice, measured: 16 methods against MRI's 50, and 19 of the 34 missing need no I/O at all

    Addrinfo is not absent — Addrinfo.tcp("127.0.0.1", 80) works and answers
    ip_address / ip_port correctly. The gap is surface. Asked of both engines
    directly (instance_methods(false) + singleton_methods(false)):

    methods
    rbgo 16
    MRI 4.0.6 50

    rbgo has: afamily getaddrinfo inspect ip ip_address ip_port ipv4? ipv6? new pfamily protocol socktype tcp to_s to_sockaddr udp.

    The 34 missing, grouped by what they need

    ⭐ Pure predicates on the address bytes — no I/O, no state (19):

    ipv4_loopback?  ipv4_multicast?  ipv4_private?
    ipv6_linklocal?  ipv6_loopback?  ipv6_multicast?  ipv6_sitelocal?
    ipv6_mc_global?  ipv6_mc_linklocal?  ipv6_mc_nodelocal?
    ipv6_mc_orglocal?  ipv6_mc_sitelocal?
    ipv6_unique_local?  ipv6_unspecified?  ipv6_v4compat?  ipv6_v4mapped?
    ipv6_to_ipv4  ip?  unix?
    

    Each is a bit test or prefix comparison against an address already held, and
    Go's net.IP already answers most of them (IsLoopback, IsMulticast,
    IsLinkLocalUnicast, IsPrivate, To4). They are the cheapest examples in
    the module and there are a lot of them.

    Introspection / conversion (5): canonname, family_addrinfo,
    inspect_sockaddr, ip_unpack, getnameinfo.

    Unix sockets (3): unix, unix_path, and the predicate above.

    Marshalling (2): marshal_dump, marshal_load.

    I/O — the only group needing sockets (7): bind, connect,
    connect_from, connect_internal, connect_to, listen, foreach.

    What this says about the order

    The first slice should be the 19 predicates: no I/O, no concurrency, each
    testable from a literal address, and together they are the bulk of what
    addrinfo/initialize_spec.rb and its neighbours exercise. The I/O group is
    seven methods and can wait — it also overlaps the blocking-call work in #933,
    so doing it first would tangle two unrelated concerns.

    ⛔ Not yet measured: how many of the −151 on initialize_spec.rb are the
    constructor's own argument forms rather than these methods. The split above is
    the method surface, which is a different question from where that file's
    examples fail, and I have not taken that second measurement.

  2. tannevaled commented on Nov 14, 2026

    @tannevaled
    ContributorAuthor

    The measurement I said I had not taken — and it changes the recommended first slice

    Above I proposed starting with the 19 I/O-free predicates, and flagged that I
    had not measured where addrinfo/initialize_spec.rb's examples actually
    fail. Taken now, it says to start somewhere else.

    RBGO_RESULT pass=44 fail=5 error=43 skip=0
    

    against MRI's 195 passing. Two things in that line are worth separating.

    The 43 errors are two defects, not forty-three

    kind count what it says
    NameError 15 uninitialized constant Socket::SOCK_RAW — also SOCK_SEQPACKET, SOCK_RDM
    TypeError 25 no implicit conversion of "smtp" into Integer
    ArgumentError 3 arity
    SpecFail 5 genuine behaviour differences

    So 40 of the 48 non-passing examples in that file come from two small,
    self-contained fixes
    — three constants and one coercion — not from the
    predicate surface I pointed at. I would have spent the first day on the wrong
    nineteen methods.

    ⛔ And 103 examples never run at all

    44 + 5 + 43 = 92 examples executed, against 195 passing under MRI. A
    hundred and three examples are not failing — they are absent: the file
    produces no result for them.

    That is a different kind of gap from everything above, and I do not know
    its cause yet. Candidates, unmeasured: describe blocks whose setup raises
    before any example runs, feature guards the shim resolves differently, or
    examples the shim's describe/it handling drops. Until that is measured, the
    −151 on this file cannot be attributed, and the method-surface table above
    accounts for only part of it.

    Whoever takes this first should measure that 103 before writing any code: it is
    a third of the file, and a fix aimed at the visible 48 would leave the larger
    half untouched and unexplained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions