Recheck remaining tokens after removing an overlapping match

[codewithfourtix]

When a longer overlapping token replaces the current token, the loop advances past its new position. This can leave overlapping or contained tokens in the result. Revisit that position before advancing.

Adds regression tests for an overlap chain and a contained token after replacement.

[Copilot]

🟢 Approval recommended

The logic change is narrowly scoped, addresses the stated defect, and is covered by targeted regression tests for the reported edge cases.

Pull request overview

Fixes filter_overlapping() so that when a longer overlapping token replaces the current token, the algorithm re-checks the new token at that position before advancing—preventing missed overlap/containment removals.

Changes:

• Adjust overlap-handling loop in filter_overlapping() to re-evaluate the replacement position after deleting the current token.
• Add regression tests covering (1) an overlap “chain” and (2) a contained token that should be removed after replacement.

File summaries

File	Description
src/license_expression/_pyahocorasick.py	Updates overlap-resolution loop to recheck the current index after deleting an overlapped token.
tests/test__pyahocorasick.py	Adds regression tests asserting correct filtering for overlap-chain and contained-token scenarios.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.