Skip to content

Match MRI for Encoding.compatible? across all operand types - #510

Merged
tannevaled merged 1 commit into
mainfrom
feat/agent-encoding
Aug 30, 2026
Merged

tannevaled merged 1 commit into
mainfrom
feat/agent-encoding

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Summary

Advances MRI 4.0.6 conformance of Encoding.compatible? and related Encoding/Regexp behaviour.

Fixes

  • Faithful rb_enc_compatible negotiation. encodingCompatible now follows MRI's enc_compatible_latter exactly, including the distinction between a real String (whose byte coderange participates) and a Symbol/Regexp/Encoding (whose declared encoding alone participates, with US-ASCII the "same as contents" special case). This fixes Encoding, Encoding pairs (e.g. compatible?(UTF_8, US_ASCII) now returns UTF-8) and every String/Regexp/Symbol/Encoding combination.
  • Regexp operands. Added to the negotiation; a runtime Regexp.new(str) now records the source String's encoding, so Regexp#encoding reports EUC-JP / BINARY / UTF-8 instead of collapsing every non-ASCII source to UTF-8.
  • # encoding: magic comment. String literals now carry the source file's encoding (binary / ascii-8bit / us-ascii), so a literal in a # encoding: binary file is BINARY, as in MRI. A literal whose bytes are valid multi-byte UTF-8 stays UTF-8 (mirroring MRI's rule that a \u escape forces UTF-8 even in a non-UTF-8 source; the parser has already decoded escapes, so the heuristic recovers the distinction the corpus relies on). Detection/tagging live in the compiler, wired in at both source-compile seams.
  • Encoding#to_s is a true alias of #name (one shared Method record).

Spec deltas (ruby/spec, MRI 4.0.6)

file before → after
core/encoding/compatible_spec.rb 362 → 384
core/encoding/to_s_spec.rb 0 → 1
core/string/encoding_spec.rb 9 → 22
core/regexp/compile_spec.rb 18 → 19
core/regexp/new_spec.rb 18 → 19

Total: +38 examples, no regressions across core/encoding, core/string, core/regexp, core/symbol.

Out of scope

The one remaining compatible_spec failure (Symbol, Symbol with a non-ASCII symbol) needs object.Symbol to carry its interned encoding — a Symbol-type change beyond this cluster. The residual string/encoding_spec failures need parser-level source-encoding support (ISO-8859-9, US-ASCII \x coderange).

Verification

  • New code 100% covered (encodingCompatible, encOperandOf, is7bit, Regexp.encodingName, MagicSourceEncoding, newStrLit, CompileWithEncoding).
  • go test ./... green; GOOS=wasip1 GOARCH=wasm go build ./... green; gofmt -l clean on changed files.

🤖 Generated with Claude Code

Rewrite the Encoding.compatible? negotiation to follow MRI's
rb_enc_compatible / enc_compatible_latter faithfully, including the
crucial distinction between a real String (whose byte coderange
participates) and a Symbol, Regexp or bare Encoding (whose declared
encoding alone participates, with US-ASCII the "same as contents"
special case). This fixes Encoding, Encoding pairs (e.g. UTF-8 vs
US-ASCII now yields UTF-8) and every String/Regexp/Symbol/Encoding
combination the spec exercises.

Add Regexp operands to the negotiation and record the source String's
encoding on a runtime Regexp.new, so Regexp#encoding reports EUC-JP /
BINARY / UTF-8 as MRI does instead of collapsing every non-ASCII source
to UTF-8.

Honour a `# encoding:` / `# coding:` magic comment (binary / ascii-8bit
/ us-ascii) so string literals carry the source file's encoding, matching
MRI: a literal in a `# encoding: binary` file is BINARY. A literal whose
bytes are valid multi-byte UTF-8 is left UTF-8 regardless, mirroring
MRI's rule that a \u escape forces UTF-8 even in a non-UTF-8 source (the
parser has already decoded escapes, so this heuristic recovers the
distinction the corpus relies on). Detection and tagging live in the
compiler, wired in at the two source-compile seams.

Make Encoding#to_s a true alias of #name (one shared Method record) so
Encoding.instance_method(:to_s) == Encoding.instance_method(:name).

Spec deltas (ruby/spec, MRI 4.0.6):
- core/encoding/compatible_spec.rb  362 -> 384 (the last failure needs
  Symbol to carry its interned encoding, a Symbol-type change out of
  scope here)
- core/encoding/to_s_spec.rb           0 -> 1
- core/string/encoding_spec.rb         9 -> 22
- core/regexp/compile_spec.rb         18 -> 19
- core/regexp/new_spec.rb             18 -> 19
No regressions across core/encoding, core/string, core/regexp, core/symbol.

New code is 100% covered; go test ./... and the wasip1/wasm build pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit b8282ef into main Aug 30, 2026
7 checks passed
@tannevaled
tannevaled deleted the feat/agent-encoding branch August 30, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant