perf: improve perf with more delegation - #1090
Conversation
🚀 Deploying Preview to Cloudflare 🚀Preview Deployments by commit
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1090 +/- ##
==========================================
+ Coverage 90.60% 90.70% +0.09%
==========================================
Files 217 220 +3
Lines 20802 21132 +330
Branches 1974 1991 +17
==========================================
+ Hits 18847 19167 +320
- Misses 1948 1958 +10
Partials 7 7 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| File | Main | PR | Change |
|---|---|---|---|
assets/style.css |
18.10 KB | 18.18 KB | +86.00 B (+0.5%) |
Performance estimate (single CI run)
- Generation time: 12.7% slower (35.62 s → 40.13 s)
- Peak memory: 3.2% lower (2.55 GB → 2.47 GB)
legacy-json Generator
Performance estimate (single CI run)
- Generation time: 62.9% faster (21.72 s → 8.06 s)
- Peak memory: 27.4% lower (2.05 GB → 1.49 GB)
llms-txt Generator
Performance estimate (single CI run)
- Generation time: 59.6% faster (21.63 s → 8.73 s)
- Peak memory: 21.0% lower (1.80 GB → 1.43 GB)
orama-db Generator
Performance estimate (single CI run)
- Generation time: 55.5% faster (20.69 s → 9.20 s)
- Peak memory: 16.8% lower (2.04 GB → 1.70 GB)
web Generator
Output size: 2 files changed · net -49.00 B
File size details
| File | Main | PR | Change |
|---|---|---|---|
all.html |
32.46 MB | 32.46 MB | -47.00 B (-0.0%) |
404.html |
21.90 KB | 21.90 KB | -2.00 B (-0.0%) |
Performance estimate (single CI run)
- Generation time: 62.3% faster (130.11 s → 49.04 s)
- Peak memory: 38.9% lower (5.50 GB → 3.36 GB)
Could you expand what these "various performance improvements" are? The PR descriptions should clearly state what the PR does. |
|
@nodejs/platform-riscv64 could you maybe break down this PR after looking at it for a while it feels that it is doing several things at the same time, which makes it harder to understand what is what (ie: what is being changed for performance improv and which piece of that, what is just refactoring) [...] imo even just the piece of threading tuning should be its own PR. Reviewing such large PRs is hard and reduces my ability (and of others) to properly review this PR. |
I'd maybe even go further and state that we need to slow down and do this in chunks.
You are doing good good work to reduce memory in pursuit of nodejs/node#62045 but it's hard to keep up with the pace |
|
I've updated the description for more information,
|
ovflowd
left a comment
ovflowd
left a comment
There was a problem hiding this comment.
I guess the part that makes this hard to review is since this has an agglomeration of a lot of different changes in several files, it makes it hard to know if for example, on vite.mjs are all changes there related to one thing? multiple things? It makes it harder to actually use Git as VCS and properly isolate changes.
I really appreciate if we could separate each change, PR stacks seems a good way to be able to get this done all at once reasonably, wdyt?
4 participants
Fixes #1008
htmlgenerator builds the component library once and the client assetsonce, then compiles, renders, minifies and writes each page on its own in the
worker pool, so memory scales with the largest page rather than with the
site. On the Node.js API docs with section pages, peak memory drops from
about 8 GB to 3.4 GB and the build runs in half the time.
all.htmlis assembled from the module pages' compiled content, in sidebarorder, instead of being built again from every module. The option moved from
jsx-ast.generateAllPagetohtml.generateAllPage, and the page shows noreading time.
jsx-astemits{ data, headings, readingTime, content }per page,contentbeing the page body as one JSX fragment;htmlwraps it in<Layout>.WebBundlercontract is nowbuildServer/compile/buildClient(see the
htmlREADME). Vite plugins no longer see the HTML pages; customizethem through the template, whose
${entrypoint}is replaced by${assets}.Synthetic pages load assets from the site root, so
404.htmlworks at nestedpaths.
@doc-kit/core:getRemarkRehypeWithShikimoved to@doc-kit/core/utils/remark-shiki.mjsandtypeAnnotationToHighlightedHastto
@doc-kit/core/utils/type-annotations/highlighted.mjs, so theastandmetadatastages no longer load Shiki. The defaultthreadsis capped at 4.Function-valued generator configuration (such as a custom
bundler) isdropped before it is sent to workers instead of failing the structured clone.