Skip to content

enumerator: Enumerator::Product class + Chain/ArithmeticSequence constructors + break-early take_while (core/enumerator +49) - #597

Merged
tannevaled merged 2 commits into
mainfrom
wave20/enummarshal
Sep 8, 2026
Merged

tannevaled merged 2 commits into
mainfrom
wave20/enummarshal

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Summary

Advances MRI-Ruby-4.0.x conformance of core/enumerator. Adds the full
Enumerator::Product class, the missing Enumerator::Chain and
Enumerator::ArithmeticSequence constructors, and fixes an infinite-loop hang
in Enumerator#take_while.

All work is in internal/vm/enumerator.go (+ a new whitebox test file). The
marshal files were investigated but left unchanged — see "Marshal: why no change".

core/enumerator: fail 22 → 16, error 40 → 11, produce_spec FILEFAIL cleared (pass 348 → 397, +49)

Reference: ruby/ruby v3_4_0 enumerator.c — enum_product_initialize,
enum_product_init_copy, enum_product_each / product_each_i,
enum_product_total_size, enum_product_rewind, inspect_enum_product;
arith_seq_eq / arith_seq_hash; and the fact that ArithmeticSequence defines
neither an allocator nor .new. take_while follows rb_iter_break.

  • Enumerator::Product is now a real class backed by a productSources
    slice on the enumerator:
    • .new / #initialize (private) store the enumerables; .allocate yields an
      uninitialized product; #initialize_copy (private) replaces the enumerables
      with MRI's OBJ_INIT_COPY guards (self-copy no-op even when frozen, then
      FrozenError, then TypeError "initialize_copy should take same class object", then ArgumentError "uninitialized product").
    • #each walks the Cartesian product through #each_entry, driven via
      enumRunEach so #to_a / #take / #next all work (infinite sources
      included). Enumerator.product builds the same object (top-level
      product_spec stays 12/12).
    • #size follows enum_product_total_size: 0 on an empty source, nil for a
      missing/nil/finite-non-Integer size, Float::INFINITY for an infinite one,
      otherwise the Integer product.
    • #rewind rewinds each source in order; #inspect renders the enumerables
      with a self-reference guard (...) via the shared repr guard.
  • Enumerator::Chain gains .allocate + a private #initialize and an
    "uninitialized" #inspect.
  • Enumerator::ArithmeticSequence: .new raises NoMethodError and
    .allocate raises TypeError "allocator undefined for Enumerator::ArithmeticSequence" (MRI defines neither); #== / #hash key on
    the begin/end/step/exclude_end? tuple.
  • Enumerator#take_while now breaks at the first falsy element, so it
    terminates on unbounded sources (Enumerator.produce, Array#cycle). The
    generic Enumerable#take_while loops to the end, which hung produce_spec.

Coverage / gates

  • go build ./... and GOWORK=off go test ./... — all packages green.
  • internal/vm/enumerator.go at 100% (go tool cover -func); new whitebox
    tests in enumerator_wave20_internal_test.go, every case byte-checked against
    ruby 4.0.6.
  • gofmt -l internal/vm/ empty; GOOS=wasip1 GOARCH=wasm build OK.

Marshal: why no change

core/marshal was measured at fail=49, error=9 and stays there. Every remaining
failure is blocked outside the four in-scope files:

  • ivar wrapping on built-ins (the I/e/C dump/load cases) — out of scope
    per the task; rbgo's object model can't attach ivars to built-in
    String/Array/Hash/Regexp/Time/Struct (this is the large majority of the
    failures).
  • Symbol encoding — object.Symbol carries no encoding, so a binary-encoded
    symbol can't be distinguished from a UTF-8 one on dump, and Symbol#encoding
    (needed by the load specs) lives in the symbol/object model, not marshal.
  • Time zone/offset — Time.local ignores TZ (always UTC in rbgo), so the
    zone/offset dump+load specs can't be satisfied from marshal alone; the marshal
    encoder was already correct for the offset-only case.
  • _dump result encoding, 'o'/'C' validation, wrapped C pointer (d
    TYPE_DATA), Random#marshal_load, UTF-16/32 encode, StringIO binmode
    — all
    need work in object, builtins.go, io*, random, or the encoding
    subsystem, which are outside this branch's editable files.

core/enumerator: remaining failures, all blocked outside scope

  • with_index (fail 3, error 7): the shim's it_behaves_like does not merge a
    shared before(:each) into the parent context the way real mspec's
    it_should_behave_like does, so @enum is nil in the parent examples. Fix
    belongs in spec_helper.rb (off-limits).
  • each (fail 2), chain/inspect "calls inspect on elements" (fail 1): the
    shim's mock and_yield/and_return chaining (spec_helper.rb).
  • each_with_object arity, product/each arity, lazy/grep+grep_v arity: a
    native method reports arity -1; setting a fixed arity needs a Method field
    in object_model.go.
  • arithmetic_sequence/==: the == operator uses the valueEqual fast path,
    which only dispatches a user #== for types listed in hasCustomEq
    (digest_bind.go); #== itself is correct (covered via #send).
  • arithmetic_sequence/inspect: MRI omits the step only when it was defaulted,
    which needs a "step given" flag recorded by Numeric#step/Range#step in
    builtins.go.
  • product/initialize_copy different-class TypeError: a Ruby subclass of a
    built-in Enumerator can't carry its own class identity (object model).
  • produce kwargs-as-initial-value: the shim declares version 3.4.1 and selects
    the pre-4.0 example, but rbgo (correctly for MRI 4.0.5) rejects the keywords.
  • lazy/take (loop enumerator), lazy/force+lazy/zip infinite: Kernel#loop
    without a block and Range#each over 0..Float::INFINITY (builtins.go).

🤖 Generated with Claude Code

tannevaled and others added 2 commits September 8, 2026 12:24
…ructors, break-early take_while (core/enumerator +49)

Reference: ruby/ruby v3_4_0 enumerator.c (enum_product_initialize,
enum_product_init_copy, enum_product_each/product_each_i,
enum_product_total_size, enum_product_rewind, inspect_enum_product;
arith_seq_eq/arith_seq_hash; the missing ArithmeticSequence allocator/new).

- Enumerator::Product is now a real class backed by productSources on the
  enumerator: .new/#initialize store the enumerables, .allocate yields an
  uninitialized product, #initialize/#initialize_copy are private,
  #each walks the Cartesian product via #each_entry (driven through
  enumRunEach so #to_a/#take/#next work, infinite sources included),
  #size follows enum_product_total_size (0/nil/INFINITY/Integer rules),
  #rewind rewinds each source in order, #inspect renders the enumerables
  with a self-reference guard ("..."). Enumerator.product now builds the
  same object (top-level product_spec stays 12/12).
- Enumerator::Chain gains .allocate + private #initialize and an
  "uninitialized" #inspect.
- Enumerator::ArithmeticSequence.new raises NoMethodError and .allocate
  raises TypeError (MRI defines neither); #== / #hash key on the
  begin/end/step/exclude_end? tuple.
- Enumerator#take_while breaks at the first falsy element (MRI's
  rb_iter_break), so it terminates on unbounded sources — the generic
  Enumerable#take_while loops to the end and hung produce_spec.

core/enumerator: fail 22->16, error 40->11, produce_spec FILEFAIL cleared
(pass 348->397). Remaining gaps are harness/object-model limits (shared
before-hook merge, native-method arity, subclass identity, the OpEq
fast-path gate) noted in the PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…quence/take_while surface (100% coverage)

Covers every new branch via eval() snippets checked byte-for-byte against
ruby 4.0.6: Enumerator::Product .new/.allocate/#initialize/#initialize_copy
(replace, self-copy, frozen, uninitialized-source, different-class and arity
guards)/#rewind/#size (Integer/empty-0/no-size-nil/finite-Float-nil/Symbol-nil/
Bignum/INFINITY)/#inspect (normal, uninitialized, self-referential);
Enumerator::Chain .allocate + private #initialize + "uninitialized" #inspect;
Enumerator::ArithmeticSequence .new (NoMethodError) / .allocate (TypeError) /
#== / #hash; Enumerator#take_while breaking early on unbounded sources.

Also simplifies #uninitialized (Inspect handles Product/Chain before it is
reached) and drops an unreachable Bignum-zero branch in #size (0 always
normalises to an Integer). go tool cover -func shows enumerator.go at 100%.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 61ce472 into main Sep 8, 2026
7 checks passed
@tannevaled
tannevaled deleted the wave20/enummarshal branch September 8, 2026 11:58
tannevaled added a commit that referenced this pull request Sep 8, 2026
…598)

Wave-20 merged three bibliography-first conformance PRs:
- #596 core/io: File.open/new trailing-Hash options, IO#read/readchar/gets
  external->internal transcoding, IO#set_encoding_by_bom, BOM stripping — +62
  (and core/file +8 incidental).
- #597 core/enumerator: Enumerator::Product class, Chain/ArithmeticSequence
  constructors, Enumerator#take_while break-early (clearing the produce_spec
  hang) — +49.
- #595 core/module: extend_object, const_source_location, const_set const_added
  + redefinition warning, class_variable ancestry search, Module.constants
  singleton, remove/undef_method hooks — +19.

Measured total on merged main: 19915 passing (2172 files load, +1 that was a
hang). Local measure at TIMEOUT=25; CI runs TIMEOUT=60 so its total is >= this floor.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant