Skip to content

perf(engine): parse the project once for the page and resource indexes - #192

Merged
Upd4ting merged 2 commits into
mainfrom
perf/shared-scan-project
Nov 10, 2026
Merged

Upd4ting merged 2 commits into
mainfrom
perf/shared-scan-project

Conversation

@Upd4ting

@Upd4ting Upd4ting commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Summary

The page index and the resource index each parsed the whole app into their own ts-morph project. They did it after every write, and again after every outside edit. The editor reads both right after each save: table usage, the page list and the table list. So each save paid two full parses.

 getSourceIndex()                  engine/source-index.ts
-  createModulesProject()          own parse
+  indexedProject()                engine/indexed-project.ts, shared
 getResourceIndex()                engine/resource-index.ts
-  createModulesProject()          own parse
+  indexedProject()

indexedProject() keeps the stamp check the two indexes had: a new parse once a file it read was written since, or after either index is invalidated. Each index rescans when the parse it read is replaced.

Evidence

Evidence level: benchmark of the engine operations, plus an end-to-end profile in the dashboard.

Workload: the repository playground in a fixture. An outside edit of the Board page is followed by what the editor reads after a save: table usage, ListPages and ListResources. Sides run interleaved, 5 runs each.

Workload main this PR
Rescan after an outside edit, 1 page 1063 ms (1042–1084) 584 ms (573–629)
Rescan after a write, 10 pages, 3 runs 1702 ms (1559–2025) 828 ms (782–1046)
Opening the editor, cold engine 2228 ms (2157–2289) 1700 ms (1644–1717)

In the dashboard, GET /api/builder/pages during the editor's open drops from about 550 ms to 15 ms. /resources has already parsed the shared project by then.

Cumulative impact: one parse less after every save and on every editor open, about 0.5 s of server CPU each time. The backend's event loop is blocked during a parse, so other requests wait too.

The engine specs (test:unit) pass: 523 passing.

Verification

  • pnpm run checks passes. Main fails sync:check on a stale compat row in packages/dms-builder/README.md (interface-dms-builder 0.4.0, published 0.4.1), so a separate commit syncs it. It is the same line build(deps): move to @antelopejs/tooling-configs 0.5.0 #190 carries, byte for byte, so whichever of the two lands second has an empty or trivial conflict there.
  • Verify harness on a stack of the three perf PRs: smoke, blocks (3), pages (3) and resources (4) journeys pass. Dashboard save, Save click to the server's answer, 15 saves per side: 2325 ms (2010–3792) on main, 630 ms (440–1102) with the three PRs.
  • formal-page-save: the same output on main and on this branch. The only finding is the recorded F2 (Moving a page with an unsaved draft lets the next Save overwrite an outside change without the stale banner #163), which none of these lines touch. The gate flags the covered code's emit as changed.

Merge Danger

Door: two-way.

Blast Radius: page and resource listings.

Invalidating one index now also rescans the other: RefreshSourceIndex reads resources again too. Neither scanner edits the project, and every write opens its nodes from the writable project.

@Upd4ting
Upd4ting merged commit 13fa89b into main Nov 10, 2026
13 checks passed
@Upd4ting
Upd4ting deleted the perf/shared-scan-project branch November 10, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant