Skip to content

Add String#encode :fallback option - #502

Merged
tannevaled merged 1 commit into
mainfrom
feat/encode-fallback
Aug 30, 2026
Merged

tannevaled merged 1 commit into
mainfrom
feat/encode-fallback

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

String#encode's :fallback keyword supplies substitutes for characters that cannot be represented in the target encoding. It accepts a Hash, a Proc, a Method, or any object queried with #[], each passed the offending character as a String.

Semantics verified byte-for-byte against MRI Ruby 4.0.6:

  • a nil result (a Hash miss included) leaves the character for the normal Encoding::UndefinedConversionError;
  • a non-String, non-nil result raises TypeError;
  • undef: :replace pre-empts the fallback entirely (it is never consulted once undef substitution is on);
  • a representable character is never routed through the fallback.

The fallback is applied as a pre-pass over the UTF-8 intermediate, rewriting only the genuinely unrepresentable characters (decided per target by charEncodableIn, covering the core encodings and the x/text-backed legacy ones), so representable characters flow straight to the encoder.

Conformance: lifts core/string/encode_spec.rb from 77 to 100 passing examples. ruby/spec ratchet FLOOR 16282 → 16305 (measured total 16364). go test ./... green, native + wasip1/wasm build, gofmt clean, 100% coverage of the new code.

🤖 Generated with Claude Code

@tannevaled
tannevaled force-pushed the feat/encode-fallback branch from 38ee9bd to 93105b5 Compare August 30, 2026 10:07
String#encode's :fallback keyword supplies substitutes for characters
that cannot be represented in the target encoding. It accepts a Hash, a
Proc, a Method, or any object queried with #[], each passed the offending
character as a String; a nil result (a Hash miss included) leaves the
character for the normal Encoding::UndefinedConversionError, and a
non-String, non-nil result raises TypeError — all matching MRI 4.0.6.

The fallback is applied as a pre-pass over the UTF-8 intermediate,
rewriting only the genuinely unrepresentable characters (decided per
target by charEncodableIn, which covers the core encodings and the
x/text-backed legacy ones) so representable characters flow straight to
the encoder. undef: :replace pre-empts the fallback entirely, as in MRI.

Lifts core/string/encode_spec.rb from 77 to 100 passing examples,
byte-for-byte cross-checked against MRI Ruby 4.0.6. ruby/spec ratchet
FLOOR 16282 -> 16305 (measured total 16364).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tannevaled
tannevaled force-pushed the feat/encode-fallback branch from 93105b5 to 9140a59 Compare August 30, 2026 10:57
@tannevaled
tannevaled merged commit 1d5bd39 into main Aug 30, 2026
7 checks passed
@tannevaled
tannevaled deleted the feat/encode-fallback branch August 30, 2026 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant