Repository navigation
Match MRI for Encoding.compatible? across all operand types - #510
Merged
Merged
Conversation
Rewrite the Encoding.compatible? negotiation to follow MRI's rb_enc_compatible / enc_compatible_latter faithfully, including the crucial distinction between a real String (whose byte coderange participates) and a Symbol, Regexp or bare Encoding (whose declared encoding alone participates, with US-ASCII the "same as contents" special case). This fixes Encoding, Encoding pairs (e.g. UTF-8 vs US-ASCII now yields UTF-8) and every String/Regexp/Symbol/Encoding combination the spec exercises. Add Regexp operands to the negotiation and record the source String's encoding on a runtime Regexp.new, so Regexp#encoding reports EUC-JP / BINARY / UTF-8 as MRI does instead of collapsing every non-ASCII source to UTF-8. Honour a `# encoding:` / `# coding:` magic comment (binary / ascii-8bit / us-ascii) so string literals carry the source file's encoding, matching MRI: a literal in a `# encoding: binary` file is BINARY. A literal whose bytes are valid multi-byte UTF-8 is left UTF-8 regardless, mirroring MRI's rule that a \u escape forces UTF-8 even in a non-UTF-8 source (the parser has already decoded escapes, so this heuristic recovers the distinction the corpus relies on). Detection and tagging live in the compiler, wired in at the two source-compile seams. Make Encoding#to_s a true alias of #name (one shared Method record) so Encoding.instance_method(:to_s) == Encoding.instance_method(:name). Spec deltas (ruby/spec, MRI 4.0.6): - core/encoding/compatible_spec.rb 362 -> 384 (the last failure needs Symbol to carry its interned encoding, a Symbol-type change out of scope here) - core/encoding/to_s_spec.rb 0 -> 1 - core/string/encoding_spec.rb 9 -> 22 - core/regexp/compile_spec.rb 18 -> 19 - core/regexp/new_spec.rb 18 -> 19 No regressions across core/encoding, core/string, core/regexp, core/symbol. New code is 100% covered; go test ./... and the wasip1/wasm build pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Advances MRI 4.0.6 conformance of
Encoding.compatible?and related Encoding/Regexp behaviour.Fixes
rb_enc_compatiblenegotiation.encodingCompatiblenow follows MRI'senc_compatible_latterexactly, including the distinction between a real String (whose byte coderange participates) and a Symbol/Regexp/Encoding (whose declared encoding alone participates, with US-ASCII the "same as contents" special case). This fixesEncoding, Encodingpairs (e.g.compatible?(UTF_8, US_ASCII)now returns UTF-8) and every String/Regexp/Symbol/Encoding combination.Regexp.new(str)now records the source String's encoding, soRegexp#encodingreports EUC-JP / BINARY / UTF-8 instead of collapsing every non-ASCII source to UTF-8.# encoding:magic comment. String literals now carry the source file's encoding (binary / ascii-8bit / us-ascii), so a literal in a# encoding: binaryfile is BINARY, as in MRI. A literal whose bytes are valid multi-byte UTF-8 stays UTF-8 (mirroring MRI's rule that a\uescape forces UTF-8 even in a non-UTF-8 source; the parser has already decoded escapes, so the heuristic recovers the distinction the corpus relies on). Detection/tagging live in the compiler, wired in at both source-compile seams.Encoding#to_sis a true alias of#name(one shared Method record).Spec deltas (ruby/spec, MRI 4.0.6)
Total: +38 examples, no regressions across core/encoding, core/string, core/regexp, core/symbol.
Out of scope
The one remaining
compatible_specfailure (Symbol, Symbolwith a non-ASCII symbol) needsobject.Symbolto carry its interned encoding — a Symbol-type change beyond this cluster. The residualstring/encoding_specfailures need parser-level source-encoding support (ISO-8859-9, US-ASCII\xcoderange).Verification
encodingCompatible,encOperandOf,is7bit,Regexp.encodingName,MagicSourceEncoding,newStrLit,CompileWithEncoding).go test ./...green;GOOS=wasip1 GOARCH=wasm go build ./...green;gofmt -lclean on changed files.🤖 Generated with Claude Code