Repository navigation
MatchData: support \u escapes and duplicate / non-ASCII capture names (MRI 4.0.6) - #565
Merged
Merged
Conversation
…names
The regexp engine rejects Ruby's \uHHHH / \u{...} escapes and cannot compile
patterns whose capture groups share a name or carry a non-ASCII name — both
valid in Ruby (Onigmo). This crashed ~27 core/matchdata examples (begin, end,
offset, byte* variants, named_captures, names, element_reference) at compile
time before any MatchData method ran.
compileRegexp now translates \u escapes to the literal characters they denote
and, only when a source has duplicate or non-ASCII capture names, rewrites each
named group to a unique ASCII synthetic name (fixing up \k/\g references),
recording per original name the synthetic names in source order. Name
resolution goes through MatchData#indexOfName, which applies Ruby's shared-name
rule: the highest-indexed group with that name that participated in the match.
Ordinary patterns are untouched (nil nameMap, engine handles names directly).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…riter
Cover the new \u-escape translation and duplicate/non-ASCII capture-name
handling with black-box vm_test cases (every new branch exercised).
Also: reject an empty \u{} list (Ruby's "invalid Unicode list"), drop the
unreachable synthetic-name lookup guard in indexOfName, and simplify the
character-class scan in rewriteNamedGroups (Ruby forbids an empty class, so a
valid pattern never opens a class with ']').
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MatchData: advance MRI 4.0.6 conformance (core/matchdata residuals)
The
core/matchdatacluster carried ~27 ERRORs from a shared root cause that wasnot in the MatchData methods at all: the regexp engine rejects three pieces of
valid Ruby (Onigmo) regexp syntax at compile time, so every example that
built such a pattern crashed with
RegexpErrorbefore any MatchData method ran.The three engine limitations, all handled now inside
internal/vm/regexp.go:\uHHHH/\u{…}escapes — translated to the literal characters theydenote (an ASCII metacharacter is emitted backslash-escaped so it stays
literal; an invalid escape is left for the engine, which raises
RegexpErrorexactly as MRI does). Unblocked
offset,byteoffset.group is rewritten to a unique ASCII synthetic name; name resolution then
applies Ruby's rule (the highest-indexed group with that name that participated
in the match). Unblocked
named_captures,names,element_reference, and thebegin/end/bytebegin/byteendname variants.(?<æ>…)) — same rewrite path, keyed on anynon-ASCII byte in a name;
\k<name>/\g<name>references (both<>and''delimiters) are rewritten to match.
The rewrite is gated: it engages only when a source actually has duplicate or
non-ASCII names, and
\utranslation short-circuits when the source has no\u.For every other pattern the engine source is byte-identical to before and the
name map stays nil, so ordinary patterns are entirely unaffected.
Result (core/matchdata)
183 pass, 0 error, 2 fail(was ~2 fail + 27 error). Every fixed example waschecked byte-for-byte against
ruby 4.0.6.Two residual failures are out of scope (they need changes outside
regexp.go):dupneeds generic instance-variable storage on native-backedobjects (
object_model.go/builtins.go), and oneelement_referenceexample isan EUC-JP encoding-tracking gap.
Tests / checks
internal/vm/matchdata_residuals_test.go(packagevm_test) covers everynew branch; all changed/added
regexp.gofunctions are at 100.0% coverage.go test ./internal/vm/green,go test ./...green,GOOS=js GOARCH=wasmbuild OK,
gofmt -lclean.🤖 Generated with Claude Code