Skip to content

perf(filters): parse a filter value in linear time - #121

Merged
Upd4ting merged 1 commit into
mainfrom
perf/linear-filter-parse
Nov 10, 2026
Merged

Upd4ting merged 1 commit into
mainfrom
perf/linear-filter-parse

Conversation

@Upd4ting

Copy link
Copy Markdown
Member

Why

The goal is to find and fix measurable performance problems in the routes DMS modules expose through the data API, without changing the exported surface (ANT-1929, wave 2 of the org-wide perf pass). Any client allowed to call a list route with a @Filter field can stall the whole Node process. Sending a long filter_<field>= value with no colon does it. parseFilter used the unanchored regex /([^:]+):(.*)/. On a colon-free value the regex retries from every start position and rescans the rest of the value each time, so its cost grows with the square of the value's length.

What changed

src/internal/filters.ts now matches /^:*([^:]+):(.*)/. Anchored, the regex tries one start position, so the match is linear. ^:* skips the leading colons the old regex also skipped, so every input parses to the same [value, comparison] pair as before. That held on 2M random strings here, and two independent reviewers found 0 differences across 3.5M and 5.5M generated strings, newlines, U+2028 and lone surrogates included.

filter_comparisons.test.ts gains two tests:

  • A 64,000-character colon-free value parses in under 250 ms. The old regex takes about 2,150 ms here, and the new one about 0.1 ms.
  • ::is:abc still reads comparison is and value abc. A plain ^([^:]+): anchor would break that.

Evidence

The evidence level is confirmed in an end-to-end benchmark. This is DoS hardening: normal requests do not get measurably faster.

Workload: the ajs module test harness (in-memory MongoDB, @antelopejs/api over HTTP). A list controller has one @Filter field over 100 rows. Each data point is the median of 9 requests. The before and after builds were interleaved 3 times per side on a shared, noisy server (load average about 12 on 16 cores). Numbers below are the median of the 3 runs, with the range in parentheses.

before after
list with a 15,000-char colon-free filter 120.29 ms (119.47–121.02) 1.31 ms (1.20–1.33)
list with an 8,000-char colon-free filter 36.08 ms (34.81–36.29) 1.29 ms (1.23–1.32)
normal list, idle 1.44 ms (1.40–1.50) 1.39 ms (1.35–1.42)
normal list while one client sends 15,000-char filters back to back 122.25 ms (121.90–123.93) 2.19 ms (2.14–2.23)

All responses were 200. Node's default 16 KB header limit answers 431 above about 16,000 characters, so 15,000 characters is close to the worst a single request reaches today, about 120 ms of blocked event loop. At that cost, about 8 such requests per second keep one core busy. Normal parsing goes from 58 ns to 42 ns per value in a microbenchmark, which does not matter.

Cumulative impact: none on normal traffic. A single cheap request that blocks every other request for about 120 ms is gone.

Readability: the regex gains ^:*. It needs no comment, and the leading-colon test pins down why it is there.

Exported surface

API: 1 packages, 26 entries unchanged against origin/main (committed HEAD).
API: skipped .agents/skills/formal-field-access, private.
API: skipped .agents/skills/verify-interface, private.
API: skipped test-helper entry ./dist/.d.ts in .; test helpers need a test runner and are outside both proof halves.
Runtime API: 1 packages, 23 entries unchanged against origin/main (committed HEAD).
Runtime API: skipped test-helper entry ./dist/
.js in .; test helpers need a test runner and are outside both proof halves.
No behavioural probe ran.

Suppressions in authored code files (origin/main and committed HEAD). A lint directive counts once per rule it names; * is a directive naming none.

Suppressions base HEAD
Lint suppressions, all rules 2 2
Lint suppressions, anti-slop/no-unsafe-dictionary-type 1 1
Lint suppressions, typescript/no-unsafe-return 1 1
@ts-expect-error 5 5
@ts-ignore 0 0
SAFETY: comments 4 4
Lint config rule-offs, all rules 0 0

pnpm run checks: Checks: 11 passed, 0 failed.

@Upd4ting
Upd4ting merged commit c2f0bea into main Nov 10, 2026
8 checks passed
@Upd4ting
Upd4ting deleted the perf/linear-filter-parse branch November 10, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant