Skip to content

Speed up array deletes and add_line() by updating only the affected indexes - #711

Open
deadstrobe5 wants to merge 1 commit into
python-poetry:masterfrom
deadstrobe5:perf/array-index-shortcuts2
Open

deadstrobe5 wants to merge 1 commit into
python-poetry:masterfrom
deadstrobe5:perf/array-index-shortcuts2

Conversation

@deadstrobe5

Copy link
Copy Markdown
Contributor

Summary

Array.__delitem__ (used by del, pop() and remove()) and Array.add_line() rebuilt the whole position-to-group index map after every call, so emptying a long array with pop() or building one with add_line() per item was quadratic. This applies the approach from #666 (append) to these two methods.

After deleting a single item, only the entries after the removed position shift down by one. In add_line() the new groups always land after every existing value (at the end, or just before a trailing whitespace group, which is never indexed), so only the new values get an index. Both shortcuts keep the same guard as #666: when the map length does not match the array, or for slice deletes, they fall back to _reindex().

Same-process A/B on CPython 3.14, 7 alternating rounds, median:

Workload Before After Speedup
pop() until empty, 2000 items 816 ms 21.2 ms 38x
pop() until empty, 200 items 7.92 ms 1.76 ms 4.5x
add_line() x 2000 1033 ms 35.2 ms 29x
add_line() x 200 10.7 ms 3.80 ms 2.8x
pop() until empty, 10 items 0.190 ms 0.197 ms same

A differential fuzz of 3000 random edit sequences (pop, del with indexes and slices, remove, insert, append, add_line, setitem, extend, clear) on single-line, multiline, comment-heavy and comma-first arrays gave the same as_string(), values, index map and exceptions after every step. The new tests delete and add lines in commented arrays and then index into them.

Duplicate Check

Agent Drafting Metadata

  • Agent: Claude Code
  • Model: Claude Opus 5.5
  • Notes: the agent profiled the array paths, wrote the change and the tests, and ran the comparisons and benchmarks above. Human attestation is pending.

@human-attestation

Copy link
Copy Markdown

🛡️ Human Attestation required. A human needs to confirm this PR once before it can merge.

Attest this PR — sign in with GitHub, pass the challenge, and confirm one of the statements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant