Skip to content

Add Criterion bench for native RLike (is_match loop) #5246

Description

@sam-1112

What is the problem the feature request solves?

Native RLike documents that is_match hand-rolls a loop over the input string array instead of calling Arrow's regexp_is_match(_scalar), which recompiles the pattern on every batch (see arrow-string regexp.rs). #5102 (and follow-up PRs) rewrote that loop (e.g. iter/map/collect). The rationale lives in docs, but nothing in native/spark-expr/benches/ measures it, so future changes to this hot path can evaporate without a baseline.

Describe the potential solution

Add native/spark-expr/benches/rlike.rs alongside the existing regexp_extract.rs Criterion bench, and register it in native/spark-expr/Cargo.toml ([[bench]], harness = false).

Suggested coverage (mirroring review discussion):

  • Scalar pattern, Utf8 subject array (common path through is_match)
  • Mix of matches / non-matches / nulls
  • Optionally Dictionary(Int32, Utf8), since that is the reachable
    dictionary shape in Comet today

This does not need to land in the string-layout / dictionary fix PR;
it is follow-up so the rewritten loop has a measurable baseline.

Additional context

Requested in review on the RLike string-layout PR: the null-check removal is likely lost in noise next to Regex::is_match, so this is not a merge blocker, but worth tracking rather than dropping.

Reference: native/spark-expr/benches/regexp_extract.rs
Related PR: #5215 (review)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions