ci(studio): gate new useEffect call sites behind a per-file ratchet - #3604
ci(studio): gate new useEffect call sites behind a per-file ratchet#3604miguel-heygen wants to merge 4 commits into
Conversation
The ban on `useEffect` and `useLayoutEffect` in studio was prose only, so
nothing failed when it was broken. `check:no-use-effect` counts call sites on
the TypeScript AST and compares each file against a budget seeded from what
exists today: a new file with an effect fails, a budgeted file that grows
fails, and a budgeted file that shrinks fails until its number comes down.
Aliased (`import { useEffect as x }`) and namespaced (`React.useEffect`) calls
resolve to the same hook and count; `useMountEffect.ts` is the one sanctioned
caller and is excluded. No existing call site is changed.
jrusso1020
left a comment
There was a problem hiding this comment.
Approving at 830f1b677e07e50d820976f8cbfac716acaa2ca3. The ratchet is the strong form of this pattern, not the usual weak one, and the reasoning in the header about why neither oxlint shape would do is correct on both counts.
What I verified rather than took on trust
- The seed is real. 154 entries summing to 281, which is the number in the description. Counted off the file, not off the body.
- The gate actually runs.
check:no-use-effectis in thelintscript andci.yml:160runsbun run lint. It is wired, not just written. - The ratchet is two-sided, which is the part that matters. Over budget fails, and
if (actual >= allowed) continuemeans a file with slack also fails and is told to lower its number. So a budgeted file cannot quietly bank headroom, and effects cannot be shuffled from a tight file into a loose one -- the source drops toactualand the total allowance falls with it. That is a genuine debt register rather than an allowance list, and it is the specific thing most ratchets get wrong. - Aliasing really is resolved.
import { useEffect },import { useEffect as x },import React from "react"andimport * as Reactall land indirect/namespaces, andBANNEDcovering both hooks in one budget closes the obvious dodge of swapping one for the other. CountingCallExpressionnodes off the TS AST is also what makes the header's own quoting of the banned name safe, which a grep would have failed on immediately. - Deletion and rename are handled. A budgeted file that disappears reports
actual0 and asks for the entry to go; a rename reports both halves.SANCTIONEDis existence-checked inmainso the escape hatch cannot rot into a permanent exemption for a file nobody has. - The scope has no gap inside studio.
SCANNEDispackages/studio/src, andpackages/studio/frontendholds onlypackage.jsonandbun.lockwhilepackages/studio/testshas no banned call. So "the studio package" and "packages/studio/src" are the same set today.
The gate runs but does not gate
bun run lint is the only path to this script, and the Lint job is not in main's required status checks. Required today: Semantic PR title, Test: runtime contract, Typecheck, Build, regression, Test, Render on windows-latest, Tests on windows-latest. Lint is also conditional on needs.changes.outputs.code == 'true'.
So a PR that adds an effect gets a red Lint job and can still merge. Every other root check -- package cycles, tracked artifacts, workspace contracts -- sits in exactly the same position, so this is not a regression and not something to fix inside this PR. I raise it because this PR's own premise is that prose alone never failed a build, and there is a second version of that gap one level up: a check that fails a job nobody is required to pass. Worth closing on purpose rather than inheriting by default.
Fallow, which is not required either but is saying something specific
Four findings, all high-crap-score, and the numbers are worth reading rather than dismissing:
| function | cyclomatic | CRAP |
|---|---|---|
walk |
5 | 30 |
visit |
7 | 56 |
listBudgetIssues |
9 | 90 |
reactBindings |
14 | 210 |
CRAP is comp^2 * (1 - coverage)^3 + comp. Every one of those four is exactly comp^2 + comp, which is the value at coverage zero. So Fallow is not really complaining about complexity here -- it is reporting, precisely, that this file has no tests.
That is the note I would act on. The script is already built for them: callSites(file, text = readFileSync(...)) takes the source as a parameter and listBudgetIssues(found, budget = BUDGET) takes the register as one, and main is behind an import.meta.url guard so the module imports cleanly. Those seams only exist for a caller that does not exist yet. A handful of cases over callSites with inline sources -- the alias, the namespace call, the docblock that must not count, the .d.ts skip -- plus two over listBudgetIssues for the over-budget and stale-slack directions, would pin the behaviour this whole gate rests on and would take the critical finding with them. This file is now load-bearing for a rule the repo intends to enforce forever; it deserves the same standard it imposes.
Two narrow bypasses, worth knowing rather than fixing
- Only
from "react"counts. A hook imported through a local barrel or a compat shim resolves to nothing and is not counted. There is no such barrel in studio today, so this is latent, but it is the shape a future refactor could walk into without noticing. const { useEffect } = Reactis not a call site the walker sees. Destructuring off a namespace import produces an identifier that is not indirect. Obscure, and nobody writes it deliberately, but it is the one spelling the alias handling does not cover.
Neither is worth code today. Both are worth a line in the header beside the aliasing paragraph, so the next reader knows where the edges are rather than assuming the resolution is total.
The CLAUDE.md section reads well and, importantly, says what to do instead rather than only what not to do -- derive during render, work in handlers, key to reset a subtree. A ban that does not name the alternative is how the alternative becomes useMountEffect for everything.
Review by Rames
jerrai-bot-heygen
left a comment
There was a problem hiding this comment.
Request changes at 830f1b677e07e50d820976f8cbfac716acaa2ca3.
The per-file baseline is exact (281 calls across 154 files), and the NEW/OVER/STALE behavior is non-vacuous. Two requirements remain before this can serve as an enforcement gate:
- The diff causes a red
Fallow audit: the new scanner contains CRAP findings, includingreactBindingsat 210/30. Refactor the scanner enough to clear the repository's maintainability gate, rather than landing a knowingly red check. check:no-use-effectcurrently runs only through the non-requiredLintjob. A failing rule therefore need not block a merge. Run it in an existing required CI job (or supply the branch-protection change that makes its job required) and add adversarial tests for the resolved React call forms. In particular, the current scanner passes violations using dynamicimport("react"),require("react"), computed namespace access, barrel re-export,.js, and local aliases.
The allowed useMountEffect escape should also be constrained to its intended implementation rather than skipping the file wholesale, or be explicitly tested/justified as a supported boundary.
— Jerrai
…ates The Fallow audit flagged four functions in `check-no-use-effect.mjs` on CRAP score, which multiplies cyclomatic complexity by how much of the function no test reaches. Coverage is the half that cannot be paid here: fallow's static estimator gives every `.mjs` file in this repo a 0% estimate, tested or not, so a script written this way can only clear the threshold on complexity. So each function now makes one decision. Resolving a `react` import, reading a namespace name, following a `useEffect as x` alias and matching a callee are four separate named predicates instead of one nested condition, and the budget comparison splits into "what is wrong with this file" and "what is wrong with this entry", assembled by a flat map. No rule, message, budget number or exit code changes: the gate still fails a new call site, a file that grew, and an entry the file has since paid down. The helpers are exported and `check-no-use-effect.test.mjs` drives them the way the sibling check scripts are tested, so the split is verified rather than assumed, and `main()` now hoists the source walk out of its loop instead of rebuilding the file list per sanctioned entry.
A ban that one import spelling walks around is not a gate. The scanner now resolves the react namespace through `require` and dynamic `import` (awaited or not, bound or destructured), reads computed member access, follows chains of local aliases in any declaration order, and fails a barrel under the scanned tree that re-exports the hook, since the barrel is the file the laundering route needs and it is in scope. `.js` and `.jsx` are scanned on the same terms as `.ts` and `.tsx`. The script header now lists every spelling as either detected or out of scope with its reason, so nothing is silently unhandled: cross-package barrels (no file in the repo re-exports react, and catching one needs whole-program resolution) and runtime indirection a static pass cannot follow. The sanctioned file stops being skipped wholesale, which had made it the one place any effect could hide. It must hold exactly one banned-hook call and that call must pass an empty dependency array, so useMountEffect cannot quietly grow into a general effect. Each new detection has a test that fails when the detection is removed.
`Lint` is not required by the branch ruleset, so a gate that only runs there cannot block a merge. The required `Typecheck` context runs it too. No ruleset change needed.
jerrai-bot-heygen
left a comment
There was a problem hiding this comment.
Request changes at exact head d378a390884c954c92afa926f1e4f28e679918bc.
The scanner coverage and required-Typecheck wiring are now materially stronger, but the sanctioned-wrapper invariant still has a bypass. sanctionedProblem() accepts exactly one banned hook call and then calls isMountEffectCall(), which checks only the argument count and an empty dependency array. It never verifies that the callee is useEffect.
Consequently, replacing the real wrapper's useEffect(effect, []) with useLayoutEffect(effect, []) passes:
bannedCalls()returns that one call because both hooks are inBANNED.isMountEffectCall()accepts its two arguments and empty array.
That violates the stated sanctioned boundary—one mount useEffect—and permits a paint-blocking layout effect exactly where the escape hatch is meant to be narrow.
Require the one call to resolve specifically to useEffect (not merely any BANNED member) and add a regression that mutates the wrapper to useLayoutEffect(effect, []) and expects the gate to fail. The dynamic-import/require/computed/barrel/JS-alias coverage and Typecheck gate can remain as implemented.
Review by Jerrai
jrusso1020
left a comment
There was a problem hiding this comment.
At d378a390884c954c92afa926f1e4f28e679918bc. Holding the re-stamp. My 830f1b67 approval is stale anyway under this repo's last-push-approval rule, so treat it as withdrawn rather than merely old.
I confirmed the finding jerrai-bot-heygen raised, at source, rather than seconding it. It is right, and here is the exact mechanism so you do not have to take two reviewers' word for it:
const BANNED = new Set(["useEffect", "useLayoutEffect"]); // :68
function isMountEffectCall(node) { // :477
const [, deps] = node.arguments;
return node.arguments.length === 2
&& ts.isArrayLiteralExpression(deps)
&& deps.elements.length === 0;
}
isMountEffectCall reads node.arguments and never touches node.expression. So the callee is unconstrained beyond having already passed bannedCalls, which admits either member of BANNED. Swap the wrapper's body to useLayoutEffect(effect, []) and: bannedCalls returns exactly one node, the count check passes, isMountEffectCall sees two arguments and an empty array, and sanctionedProblem returns null.
The message the function would have printed is the tell -- must call useEffect(effect, []) and nothing else -- so the code does not enforce what its own error text claims. On a ratchet that is the defect that matters most, because the whole value of the file is that it enforces exactly what it says.
The new suite covers the sanctioned file four ways (baseline, a second useEffect(other, []), [effect] deps, and the missing array) and none of them is a callee swap, which is consistent with the hole being in the callee check.
A second hiding place of the same shape, which I do not think has been named
scan() skips the sanctioned file wholesale:
for (const file of sources()) {
if (SANCTIONED.has(file)) continue; // :504
const lines = violations(file);
...
}
and sanctionedProblem calls bannedCalls alone. But violations() -- the path that was just skipped -- checks two things, not one:
const found = [...bannedCalls(root), ...root.statements.filter(isReactReExport)];
So export { useEffect } from "react" inside the sanctioned file is checked by nobody. Your own suite proves the gate catches that statement in an ordinary file (assert.deepEqual(violations("a.ts", 'export { useEffect } from "react";'), [2])), and the one file exempt from that path is the one whose whole job is to be the narrow exception.
It is the same bypass class and worth fixing in the same pass, because sanctionedProblem's own docblock is the thing that argues against it: "Skipping the file wholesale would make it a hiding place: any effect, any dependency array, unchecked." Re-exports are the third item that sentence does not list, and they are strictly worse than a stray effect -- a re-export hands the banned hook to every other file under SCANNED through an import the scanner has no reason to distrust.
What I would want before re-stamping
- Require the one sanctioned call to resolve specifically to
useEffect, not merely to aBANNEDmember. - Have
sanctionedProblemalso reject a react re-export, so the exemption is from the BUDGET and not from the whole check. - Two regressions, each mutating the real wrapper:
useLayoutEffect(effect, []), and aexport { useEffect } from "react"added to it. Both should fail the gate.
Everything else in this revision holds up and I am not asking you to revisit it. The expanded spelling coverage is real -- dynamic import, require, computed namespace access, in-tree barrel re-export, .js/.jsx sources and local alias chains -- with a break-and-revert per spelling, and documenting the cross-package barrel as out of scope with the observation that none exists in the repo is the right way to bound it. useMountEffect.ts no longer being skipped is the change that makes the sanctioned entry mean anything at all, which is why these two holes are worth closing now rather than later: they are the residue of exactly the gap this revision set out to close.
Moving the gate into the required Typecheck job so it blocks without a ruleset change is the right lever, and worth saying out loud that it is now load-bearing: a gate that runs and does not block is a lint, and this one is meant to be a ratchet.
Not a second block -- jerrai-bot-heygen's request-changes at this head is live and gating, and this is deliberately additive to it. Point 2 is the part I would not want lost between the two reviews.
No merge or enqueue action taken.
Review by Rames
CLAUDE.mdbansuseEffectin studio, but nothing enforced it, so the call sites keptaccumulating while the rule read as absolute. This adds
check:no-use-effectto the rootlintchain (
.github/workflows/ci.ymlrunsbun run lintat line 160). No existing call site ischanged.
What the gate does
scripts/check-no-use-effect.mjswalkspackages/studio/src, parses each.ts/.tsxfile withthe TypeScript compiler, and counts calls to
useEffectanduseLayoutEffect. It compares eachfile against a per-file budget seeded from what exists today: 281 call sites across 154 files,
plus the one sanctioned caller. Three ways to fail:
The last one is what makes it a debt register rather than a permanent allowance: the budget can
only ever go down, and the script is the only place it lives.
Counting is done on the AST rather than with grep, because docblocks quote the banned pattern
while explaining it, and prose about a rule must not count as breaking it. Aliasing is resolved
rather than assumed away:
import { useEffect as x },React.useEffectand a namespace importall resolve to the same call and all count. Both hooks share one budget, so neither can be
smuggled in by spelling it the other way.
How to shrink a budget
Remove the effect, then lower that file's number in
BUDGET; delete the entry when it reacheszero. The check tells you the number to write. When every entry is gone, delete
BUDGETand thescript becomes a flat ban.
The allowed escape
useMountEffect()(packages/studio/src/hooks/useMountEffect.ts) is the one sanctioned way tosync once with an external system on mount. That file is excluded outright rather than budgeted,
because the budget must be able to reach zero and this entry never will. Everything else derives
during render, does the work in an event handler, uses a data-fetching library, or resets with a
key.Verification
Full
bun run lintis green with the step wired in:```
$ node scripts/check-no-use-effect.mjs
no-use-effect: no new useEffect or useLayoutEffect in packages/studio/src. 281 budgeted call site(s) across 154 file(s) remain.
```
Proved non-vacuous by adding one
useEffect(() => {}, [])to a studio file and re-running, thenreverting:
```
OVER BUDGET: packages/studio/src/hooks/useFileTree.ts has 2, budget allows 1
Lines: 18, 88. The budget is debt, not headroom.
1 problem(s). Budgeted debt is 281 across 154 files.
```
A new unbudgeted file using the aliased, namespaced and layout spellings fails the same way:
```
NEW banned effect hook: packages/studio/src/probeGate.ts:4, :5, :6
The ban is absolute. Use useMountEffect() for a one-time external sync, or derive
the value during render. See CLAUDE.md > React Rules.
```
Since review
The Fallow audit flagged four functions in the new script on CRAP score, which weighs cyclomatic
complexity by the share of the function no test reaches. Coverage is not the payable half here:
fallow's static estimator scores every
.mjsfile in this repo at 0% whether or not it has a test(
check-cli-process-ownership.mjsandcheck-package-cycles.mjssit at 42 and 132 today, and passonly because
--gate new-onlygrandfathers them). So the score came down on complexity, and theaudit is untouched: no suppression comment, no ignore entry, no raised
--max-crap.Each function now makes one decision. Resolving a
reactimport, reading a namespace name,following a
useEffect as xalias and matching a callee are four named predicates instead of onenested condition, and the budget comparison splits into "what is wrong with this file" and "what is
wrong with this entry".
main()also hoists the source walk out of its loop rather than rebuildingthe file list once per sanctioned entry.
The gate itself is unchanged: same rule, same messages, same budget numbers, same exit codes. The
helpers are now exported and
scripts/check-no-use-effect.test.mjsdrives them the way the siblingcheck scripts are tested (registered in
test:scripts, 15 cases). Break-and-revert still holdsafter the split, on both failure shapes:
```
$ node scripts/check-no-use-effect.mjs # a new aliased effect in an unbudgeted file
NEW banned effect hook: packages/studio/src/__probe.tsx:2
The ban is absolute. Use useMountEffect() for a one-time external sync, or derive
the value during render. See CLAUDE.md > React Rules.
exit=1
$ node scripts/check-no-use-effect.mjs # App.tsx budget nudged 1 -> 9
STALE budget entry: packages/studio/src/App.tsx now has 1, budget still says 9
Lower it to 1. Thanks for paying the debt down.
exit=1
```
The tests are non-vacuous: breaking alias resolution, the stale-budget comparison, and the module
specifier check each turns cases red (2, 1 and 2 failures), and all 15 pass on revert.
Local runs of the audit exactly as CI invokes it, before and after:
```
$ bunx fallow audit --base origin/main --fail-on-issues --format pr-comment-github
before: exit 1, 4 findings (reactBindings 210.0, listBudgetIssues 90.0, visit 56.0, walk 30.0)
after: exit 0, "No GitHub PR/MR findings."
```
bun run lint,bun run test:scripts(203 pass) andbun run format:checkare green.Since review
Head
d378a3908. Points 2 and 3 of the review, plus the branch-protection question.Every spelling the review named is now resolved, and nothing is silently unhandled. The script
header carries the full list, each entry either detected or out of scope with its reason:
importof the module, awaited or not, bound or destructuredrequireof the module, bound or destructuredReact["useEffect"]packages/studio/src.js/.jsxsources.ts/.tsxconst a = useEffect; const b = a;), any declaration orderpackages/re-exports react todayReact[flag ? "useEffect" : "useMemo"],eval, a hook read out of a data structure)The barrel case is closed where it is written rather than where it is imported: the re-export is
itself the file the laundering route needs, and it is in scope, so
export { useEffect } from "react"andexport * from "react"fail in the barrel. That is a root fix, not a per-consumer one.The sanctioned escape is no longer skipped wholesale. Skipping the file made it the one place
any effect could hide.
useMountEffect.tsmust now contain exactly one banned-hook call and thatcall must pass an empty dependency array; a second effect, a non-empty array, or no array at all
fails the check. Three tests cover those three shapes, and one holds the real file to it.
Branch protection: no ruleset change required.
Lintis not a required context, so the gatealso runs in the required
Typecheckjob (bun run check:no-use-effect, one added step). Afailing rule now blocks a merge without a repo-admin action.
Break-and-revert, once per new detection, each mutation reverted afterwards:
The baseline is unchanged by the wider resolution: still 281 calls across 154 files, so the new
spellings add no previously-missed debt.
bun run lint,bun run test:scripts(217 node cases + 42 vitest) andbun run format:checkaregreen.
One thing worth flagging for whoever reads the script next: the header deliberately writes the
module specifier as a
<react>placeholder insiderequire(...)/import(...)in prose. Spellingit literally in a comment makes the dependency audit read the comment as a real import and report
react as an unlisted dependency of the repo root.