From 242edff0267049c941125cee4953385541a204e7 Mon Sep 17 00:00:00 2001 From: tannevaled Date: Sat, 14 Nov 2026 13:21:30 +0100 Subject: [PATCH 1/2] conformance: define MSpec.enable_feature, without which the ORACLE scores zero MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit library/socket/spec_helper.rb opens with nine MSpec.enable_feature calls. The shim defines no MSpec, so that file raises NameError at LOAD and every file under library/socket contributes 0. ⛔ That is worst when the engine being scored is the REFERENCE. MRI then contributes 0 to all of them, and cmd/specdelta prints 190 file(s): control 7 passing, treatment 224 passing, +217 which reads as rbgo beating MRI by 217 examples. An oracle that cannot load is not a low score, it is no measurement at all — the same lesson as #930, one layer down, and I nearly published that +217 before reading the interpreter's stderr. enable_feature is the ONLY MSpec method ruby/spec calls. Measured, not assumed: grepping library/, core/ and language/ for `MSpec.` returns that one name and nothing else. It is a no-op because with_feature already runs every feature block unconditionally in this shim, which is also what the corpus asks for in that file's own comment: "we want a constant number of spec examples across Ruby implementations, even if they don't define these constants". Checked: MRI now scores 13 on library/socket/tcpserver/accept_spec.rb where it previously stopped at load. Refs #916 Co-Authored-By: Claude Opus 5 --- scripts/conformance/rubyspec/spec_helper.rb | 22 +++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/scripts/conformance/rubyspec/spec_helper.rb b/scripts/conformance/rubyspec/spec_helper.rb index ecef98ce..2b3b840e 100644 --- a/scripts/conformance/rubyspec/spec_helper.rb +++ b/scripts/conformance/rubyspec/spec_helper.rb @@ -879,6 +879,28 @@ def with_block_device(&b) end def with_feature(*a, &b); _guard_run(true, &b); end def without_feature(*a, &b); _guard_run(false, &b); end + +# ⛔ MSpec.enable_feature IS THE ONLY MSpec METHOD ruby/spec CALLS — measured by +# grepping library/, core/ and language/ for `MSpec.`, which returns +# that one name and nothing else. Without the constant, +# library/socket/spec_helper.rb raises NameError at LOAD, and every file under +# library/socket scores 0. +# +# That is worse than it sounds when the engine being scored is the REFERENCE: +# MRI then contributes 0 to all 56 of those files, a sweep prints +# `control 0 passing, treatment 224 passing, +217`, and the result reads as the +# implementation under test beating MRI. An oracle that cannot load is not a +# low score, it is no measurement at all (#930 is the same lesson, one layer +# down). +# +# It is a no-op because `with_feature` above already runs every feature block +# unconditionally, which is also what the corpus asks for in that file's own +# comment: "we want a constant number of spec examples across Ruby +# implementations, even if they don't define these constants". +module MSpec + def self.enable_feature(*) end + def self.disable_feature(*) end +end def ruby_exe(*a, **k); ::Kernel.raise(SpecSkip, "ruby_exe subprocess unsupported"); end def ruby_cmd(*a, **k); ::Kernel.raise(SpecSkip, "ruby_cmd unsupported"); end From e53c89e2d60ec4f5ca5113c9e5b6b08c89ba3448 Mon Sep 17 00:00:00 2001 From: tannevaled Date: Sat, 14 Nov 2026 22:40:43 +0100 Subject: [PATCH 2/2] conformance: the README cites shim line numbers, and the insertion moved two MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TestReadmeStubbedHelpersMatchTheShim failed on EVERY lane — all three test lanes and all three arch lanes — for a change that touches one Ruby file: README.md cites `ruby_exe` at spec_helper.rb:882; it is at :904 README.md cites `ruby_cmd` at spec_helper.rb:883; it is at :905 The MSpec block is 22 lines and both citations sit below it. 904 - 882 = 22. That test is a drift gate on PROSE, and it earned its keep here: nothing about adding a module to a Ruby file suggests that two line numbers in a README three hundred lines away are now wrong, and no reviewer reads for that. It is also why the failure showed up on the arch lanes, which build and vet rather than run specs — the test lives in the repository's root package, so every lane that runs `go test ./...` sees it. ⛔ The citation style is the brittle part, not the test: a line number in prose is a fact about the file's current layout, and any insertion above it is a silent edit to the documentation. The test converts that silence into a failure, which is the right trade until the README cites names instead of lines. Left as it is here — changing the convention is a separate change from making it true. Verified locally before pushing this time: the test passes. Co-Authored-By: Claude Opus 5 --- scripts/conformance/rubyspec/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/conformance/rubyspec/README.md b/scripts/conformance/rubyspec/README.md index dbbd1d12..a118de22 100644 --- a/scripts/conformance/rubyspec/README.md +++ b/scripts/conformance/rubyspec/README.md @@ -635,8 +635,8 @@ what a ratchet should be: | stubbed (raises `SpecSkip`) | where | |---|---| -| `ruby_exe` — no subprocess | `spec_helper.rb:882` | -| `ruby_cmd` — no subprocess command line | `spec_helper.rb:883` | +| `ruby_exe` — no subprocess | `spec_helper.rb:904` | +| `ruby_cmd` — no subprocess command line | `spec_helper.rb:905` | | `output_to_fd` — no real fd redirection | `spec_helper.rb:280` | ⛔ **That table is the whole list, and it is now derived rather than retyped.**