Repository navigation
perf(hooks): cut what the agent waits on in hooks and pulls, and measure it - #1822
Merged
Merged
Conversation
The session-start MR hint runs `gh pr list` for a GitHub project. A world now puts a fake gh first on PATH: `pr list` prints fakes/gh-pr-list.json or [], `--version` answers, anything else fails the way a gh that is not signed in does, and every call is logged to fakes/gh.log.
`node scripts/agent-wait-budget.mjs` builds a verify world, runs the hooks and pulls of one Claude Code session in it, from the main checkout and from linked worktrees, and prints what the agent waits on as JSON: the git, Node and gh processes on the waited side and after the hook returned, files written, and the remotes reached, on a first run and a second. No elapsed time: the same hook took twice as long in one world as in another while every count stayed the same. git's trace2 counts every git process, the login shell's included. A Node preload logs each Node process and tags what runs after the hook returns (the --bg-only pass and anything started detached), which trace2 records per process. A row counts only when the hook added its line to debug.log, since a hook command exits 0 when teamai is not found. It exits non-zero when an invariant breaks: a second run costs more than the first, a warm fast-path hook waits on git started from Node, the session start does not reach gh exactly once, a row reaches a remote it is not known to reach or fetches the project, a new worktree lacks the team's resources when `git worktree add` returns or after the tool hook's background pass, or a linked worktree costs more than the main checkout once prepared. `--compare <main.json> [<head.json>]` prints the Markdown report CI shows.
A new job measures main with main's own measurer, when it has one, and this PR once, and puts the comparison in the job summary. A broken invariant of this PR's own measurement fails the job; the comparison never does, and a main that cannot be measured is reported, not failed. When the PR changes the measurer or the world, the report says so above its table. When a waited count grew, the report is uploaded and the review comment shows it beside the agent context budget. The review reads both budgets in one loop with the same row filter, plus the wait budget's header and its two sentences as exact matches: the report comes from the PR's own code and is posted by a workflow with write access.
The glossary gets a Budgets section: agent wait, the agent wait budget, and the agent context budget that CI already reports. AGENTS.md asks for work an agent does not need to be done after the hook returns, with the saving proved in counts, adds the first and second run to the surfaces to walk, and says CI reports the wait budget against main.
…ptied Uninstall deleted the files teamai delivered and left the directories that held them, so a project kept an empty `.claude/rules` after a workspace uninstall. It now prunes, once its rule, skill and agent removal is done, the directories those were in when nothing is left in them: above a rule, each folder up to the tool's rules directory. A directory that still holds a member's file stays, and so does the tool's own root. The verify ledger also knows what the hooks of a session leave today (events.jsonl, session-owners.jsonl, state.json and the sessions directory, tracked in Tencent#1808), so a world that ran hooks ends clean unless something else leaked.
A session start read the checkout's team delivery twice before its pull: once to observe the checkout and once more for the skills its capture records, each resolving the delivered revision with git. It now reads it once and hands the same answer to both. The background reads of the record before and after the session's own pull only need the record itself, so they take it without resolving its revision. Agent wait budget, git processes waited / after the hook: - SessionStart, first run: 10 / 39 -> 9 / 37 - SessionStart, second run: 7 / 24 -> 6 / 22 - SessionStart with a stale fetch: 7 / 26 -> 6 / 24 - SessionStart in a new worktree, first run: 7 / 47 -> 6 / 45 - SessionStart in a new worktree, second run: 7 / 32 -> 6 / 30
A pull's git exclude sync and the doctor check after it ask git the same two questions for each checkout: which delivered paths it tracks, and which it would still offer for a commit. They share one GitContext for the run, which now keeps both answers, keyed by the checkout root and the paths asked about, so a linked worktree gets its own. A failed answer is not kept. A sync that writes an exclude file forgets what git offers, since that answer depends on the file it just wrote. Agent wait budget, git processes waited (ls-files 6 -> 4 in each): - pull after init: 37 -> 35 first run, 25 -> 23 second run - pull of a new team commit: 26 -> 24 first run, 25 -> 23 second run
…trees exist While `git worktree add` waited, the git hook's preparation of the new checkout synced the delivered git exclude block across every checkout of the repository, asking git what each one tracks and offers, so the wait grew by two git processes with each worktree a member kept. The exclude file is one per repository and a line stays while any checkout needs it, so adding the new checkout's lines needs only the new checkout. The inline preparation now lists that checkout's delivered files and drops no line; the detached pull the hook already starts right after syncs every checkout's lines, as before. The tool-use preparation and every other pull are unchanged. A line no checkout needs any more now stays in the exclude file from the moment `git worktree add` returns until that detached pull finishes, or until the next full pull if it does not run. Before, the inline sync dropped it at once. The agent wait budget now fails when the waited ls-files count differs between the three `git worktree add` rows, or when the two second-run rows wait on a different number of git processes. Agent wait budget, git processes waited (ls-files waited -> 2 in each): - worktree add, two checkouts (first run): 14 -> 12 - worktree add, three checkouts: 14 -> 10 - worktree add, four checkouts: 16 -> 10
With `updatePolicy: skip`, or a team's `autoUpdate: false`, the Stop hook's automatic update still asked the npm registry for the latest version before it read the policy that skips the update. It now reads the policy first and returns before the check. `teamai update` run by a member still asks and still says when it is already up to date. The policy is now read before the registry is asked on every update, so a local config that cannot be loaded falls back to the running scope's, as the check for other registered copies already did: an update whose policy read rejected before now uses the running scope's config instead.
Every session start listed the team skills at the revision its checkout received and resolved that revision to its full commit, two git processes the agent waited on, for an answer that does not change: a pull that delivers something else records another revision. Both answers are now kept with the other git answers, keyed by the team clone and the revision, under the clone's repository stamp, so a clone removed and cloned again is read again. A failed read keeps nothing, and the listing still falls back to the clone's HEAD when the revision cannot be read. A linked worktree of the project reuses what its main checkout read, as both deliver from one team clone. Agent wait budget, git processes waited / after the hook: - SessionStart, second run: 6 / 22 -> 4 / 21 - SessionStart with a stale fetch: 6 / 24 -> 4 / 23 - SessionStart in a new worktree, second run: 6 / 30 -> 4 / 29 - SessionStart, first run: 9 / 37 -> 9 / 36
A Stop's background pass read the checkout three times: `git status` for the session's end record, then `rev-parse` and `symbolic-ref` for the dashboard's Stop event. It now runs one `git status --branch` and hands the head, its dirtiness and the branch to both readers in the payload, as the session start already hands its state. Each reader keeps its own read for when nothing is handed over. When that status fails (a bare repository) or passes its 2 s cap (a very large checkout), the Stop event still takes the head and branch from `rev-parse` and `symbolic-ref`, as before. The branch now comes from git status, so a branch with a tag of the same name is recorded by its name, not as `heads/<name>`. Nothing the agent waits on changes: both readers run after the hook returns. Agent wait budget: Stop, git processes after the hook 3 -> 1 on the first and second run; no waited count moves.
The MR hint ran `git remote get-url origin` and `gh pr list` (or a REST call, up to 8 s) in the foreground of every session start in a GitHub or TGit project. The session start's detached pass now looks for them and leaves the hint in the session's state directory; the session's next prompt shows it, once: the prompt that takes it moves the file aside before reading it, and the fast path sends that prompt to the full dispatch. The cache now records MRs as hinted when the hint is shown, not when it is computed, so a hint no prompt took leaves its MRs to the next session, and a prompt shows nothing when another session has shown all its MRs since. A session start that finds nothing new drops the hint its session had waiting. A tool teamai gives no prompt hook (Kiro, Hermes), or one whose team turned the prompt hook off, still gets the hint at session start. The remote is read through the async `getRemoteUrl` in place of a `spawnSync`.
A `bash -lc` runs the env loader twice in one process: from the profile block, then as the BASH_ENV that pass exported. Each pass asked git for the directory's repository. The second pass must still run, since it is what makes a team value win over a profile line after teamai's block, so it keeps loading the files and only takes the route the first pass found, once, while the directory, the registry stamp, PATH and every GIT_* variable are what they were. The record is never exported, so a child shell asks git itself, and it is taken only in a non-interactive bash.
A session start resolves the tool's pid in two handlers, and each walk asked ps once per lookup: ten processes the session start waited on. On macOS and the BSDs, one `ps -A -o pid=,ppid=,comm=` now answers every lookup in the process, as the Windows table already does, a failed read included; its 2 s cap applies once. `comm` goes last because an executable path may hold spaces. Linux keeps /proc and no longer falls back to a per-pid ps when a /proc read fails: procps reads the same /proc, so it could not answer either. Only short-lived hook processes resolve the pid; the dashboard checks liveness with isProcessAlive, which is unchanged.
Two optional per-tool facts in `TOOLS`: `unwaitedSince`, the first version whose command hooks take `"async": true` and are then not waited for, and `loginDeferrable`, a tool observed to run hook commands with a plain `sh -c` on POSIX. `builtinHookForm` turns them into the form of one hook file: a project file a repository may commit always keeps the waited login-shell form, and the member's own file is unwaited only when the installed version reads at or above the floor. `reconcileHooks` and `getHookStatus` take the form of the file they are given, so pull, `teamai hooks inject`, doctor and `teamai hooks list` agree on it. The unwaited form marks the PostToolUse entry `async`, which the Claude Code and Codex renderers write. Checks that match an entry on disk against teamai's own recognise every form, so an older or a newer teamai still finds its entries. No tool has either fact yet, so every hook file is written as before.
From 2.1.119, Claude Code takes "async": true on a command hook and no longer writes an empty transcript entry for an async PostToolUse hook that prints nothing. In the member's own ~/.claude/settings.json, the wildcard entry becomes two entries under the same key: an async one for every tool but TodoWrite, and a waited one for TodoWrite, whose hint must reach the model in the same turn. A project file, an older or unreadable version, and every other tool keep the single waited entry. A full pull decides the form of a hook file from the tool's version. An inline pull, which a git hook and a new checkout wait for, keeps the form the file already has and asks the version only when the file holds no teamai built-in to read it from, so it starts no tool. One function, builtinHookFormOf, owns that reading; getHookStatus takes where to read it from: the instruction readiness check asks about the file's own form, `teamai hooks list` about the version's. A file whose entries agree with no single form is not installed and is repaired in the form an entry shows. The cost: an upgrade or downgrade across 2.1.119 changes the file at the next full pull, not at the next git hook.
From 0.148.0, Codex runs a command handler with "async": true in the background, every event but SessionEnd; 0.134.0 to 0.147.0 skip such a handler instead. In the member's own ~/.codex/hooks.json, the PostToolUse entry takes "async": true beside command and commandWindows. It stays one entry with no matcher: Codex compiles matchers with the Rust regex crate, which has no look-ahead, and drops a group whose matcher does not compile. codex-internal and tcodex keep the waited entry. The fact rests on Codex's documentation and on its source at 0.148.0 to 0.167.0. An async handler was run on 0.167.0 for UserPromptSubmit and SessionStart; PostToolUse was not run. Codex trusts a hook by a hash of its handler, async included, so the changed entry reads as modified until trusted again. The trust pass after each write already trusts it: it writes back the hash Codex's own app-server lists, never one teamai computes. The agent wait budget measures Claude Code, so no row moves.
A hook command can now start teamai through ~/.teamai/bin with TEAMAI_HOOK_LOGIN=deferred. teamai then answers before the CLI bundle loads: a hook the fast path takes starts its background pass through a detached login shell and returns, and any other hook runs through a login shell in the foreground, as the command did before. A payload over 16 KB goes to the background pass as a file, removed again when the pass cannot start. When the Node or the entry script the wrapper records is gone, a deferred hook runs teamai as a login shell finds it, at most once. Without the marker the wrapper does what it did. The wrapper is now written to a temporary file and renamed, since a hook may run it at any moment. No hook file is written in this form yet.
…t is waited for Where Claude Code is older than 2.1.119, or its version cannot be read, its PostToolUse entry in ~/.claude/settings.json now starts teamai through ~/.teamai/bin/teamai with TEAMAI_HOOK_LOGIN=deferred, and the login shell runs around the background pass. Without the wrapper the entry runs today's command. Pull writes the wrapper for a tool that can take this form. A repository's project file, Windows, the TodoWrite entry and every other event keep today's command. The agent wait budget does not move: the Claude Code it measures is above 2.1.119, where the entry is not waited for at all.
…sion is known not to sync over HTTP Claude Code's UserPromptSubmit entry in ~/.claude/settings.json now starts teamai through ~/.teamai/bin/teamai, as the waited PostToolUse entry does. Whether a prompt must be answered in the foreground depends on an HTTP endpoint variable that only the profile or the team env may set, which this form's foreground cannot see. So every dispatch pass that runs inside the login environment records, per session, the cwd it ran in and whether it saw HTTP enrolment, writing only when that changes. A prompt takes the fast path only when that record, for its own cwd, says not enrolled. With no record, another cwd (a session that moved into a worktree), or enrolment, the prompt goes through the login shell in the foreground as before, and that pass writes the record. The record lives beside the session's other state and is removed with it.
A login-deferred hook entry starts teamai through ~/.teamai/bin/teamai. When that file is gone, or names a Node or a teamai that no longer exists (after a Node or teamai reinstall), the hook still runs, but waits for a login shell first, and nothing tells the member. When a tool's hook file holds such an entry, doctor now compares the wrapper with the one this build writes, its Node and entry paths aside, through the same comparison it uses for the adapters' saved launchers, and checks that both paths exist. The fix it names is `teamai pull`. Without such an entry it adds no check.
… waited for Codex runs a hook's command with the account's zsh, bash or sh and -c, never -lc, and falls back to one of those three for any other account shell. In the member's own ~/.codex/hooks.json on macOS and Linux, the PostToolUse entry below 0.148.0 or with an unreadable version, and the UserPromptSubmit entry at any version, now start teamai through ~/.teamai/bin/teamai and leave the login shell to the background pass, as Claude Code's do. From 0.148.0 the PostToolUse entry is async and keeps today's command. codex-internal and tcodex keep today's entries, and so does Windows. Codex's commandWindows was derived from a command in today's form only. One helper now gives the hook-dispatch text of a built-in in either form, so a deferred entry keeps today's commandWindows byte for byte. The fact rests on Codex's source at 0.134.0 to 0.167.0 and on a run of its hook executor on 0.167.0, macOS: /bin/zsh -c, no login flag, its parent the codex binary. teamai's own entry in this form was not run on Codex. The changed command changes the handler's trust hash, and the trust pass writes back the hash Codex lists for it. A Codex payload carries session_id and cwd, so a login pass records whether the session syncs over HTTP, and a later prompt takes the fast path as Claude Code's does. The agent wait budget measures Claude Code, so no row moves.
The PR description includes sufficient representative real-CLI verification records; no missing-e2e finding. Review was read-only—no PR code was run, built, or installed. |
A hint the session start left for the next prompt was shown, and its MRs recorded as shown, even when TEAMAI_MR_HINT_DISABLED or the scope's switch turned the hint off in between. Showing it now asks the same switch as computing it. A hint turned off is dropped unshown, so the next session start hints the same MRs again if the hint is back on.
…dger leak The measure kept counting when a measured command exited non-zero, was ended by a signal or did not start, and it only printed a ledger leak. Each is now a failure of the measure, so the budget job cannot pass on a pull that failed or a world that leaked.
SaulMoro
requested changes
Nov 10, 2026
SaulMoro
left a comment
SaulMoro
left a comment
Collaborator
There was a problem hiding this comment.
This was generated by AI during triage.
Reviewed at 9860b005 with the built CLI in verify worlds on macOS, on main and on the head. I then read the two commits in c47ea00f; the findings below hold at c47ea00f.
What checks out:
- Counts.
scripts/agent-wait-budget.mjsreproduces the body's table exactly, and its world ends with the ledger CLEAN. - Claude Code forms. Claude Code 2.1.322 gets the unwaited PostToolUse pair. A stub at 2.1.100 gets the login-deferred form. Both hooks took the fast path.
- Doctor check. The wrapper check fails on a broken wrapper, and the
teamai pullit names fixes it. - Downgrade. Main's
teamai pullrewrites every new form back to today's entries, and main'suninstallremoves them, with the ledger CLEAN. The door is two-way, as the body says. - Codex on Windows. Codex's
commandWindowsand timeouts are byte-identical to main. - Workflows. The budget job runs on
pull_requestwith read-only permissions. Thecodex-review-on-assignchange reads one more artifact through the same line filter the context budget uses. Neither gives code from a fork anything it could not reach before.
c47ea00f fixes the CI review's three points: a queued MR hint is dropped once hints are switched off, a ledger LEAK fails the budget job, and a measured command's exit status counts.
Blocking
- [P1 blocking] Codex never gets the unwaited PostToolUse entry.
DETECTORSinsrc/agent-version.tshascodex: null, so teamai reads no Codex version and theunwaitedSince: '0.148.0'gate never passes. With codex-cli 0.167.0 on PATH,init --agent codexwrites~/.codex/hooks.jsonwith no"async"line, anddebug.logshows[agent-version] codex → "". The body, both languages of the guide and thesrc/tools.tscomment say Codex 0.148.0+ gets this form. The unit tests pass only because they stub the version. Either add a Codex version detector, or drop that commit and its claims.
Non-blocking
-
[P2 non-blocking] The unwaited form reaches Windows too. Only the login-deferred form is limited to POSIX. Claude Code 2.1.119+ on Windows gets the new async pair, and the Codex test expects
asyncon Windows as well. The body says "Windows keeps today's entries", and nothing has run on Windows. Either keep the unwaited form off Windows, or run it there and correct the body. -
[P2 non-blocking] No CHANGELOG entry. A member will see several changes:
- the MR hint moves to the first prompt;
~/.claude/settings.jsonand~/.codex/hooks.jsonchange form;~/.teamai/bin/teamaiis new;doctorhas a new check;- Stop skips the registry under
skip; - uninstall removes the rules, skills and agents directories it emptied.
The squash title is
perf(hooks), which would hide the twofeatcommits. Entries under Features and Bug Fixes would cover it. -
[P3 nit] A first prompt can arrive before the background pass has stored the MR hint, so the hint shows one prompt later.
-
[P3 nit] Uninstall's removal of emptied
skills/agentsdirectories also takes an empty folder a member created there. -
[P3 nit]
AGENTS.md"Where code lives" still lists only the context budget underscripts/.
Not run here: Codex itself, which is not signed in inside the worlds, and Windows. CI is pending at c47ea00f. #1775 touches hook-fast-path.ts, mr-hint.ts and pid-monitor.ts too. The conflicts are mechanical: that PR owns Windows child processes, this one the POSIX hook forms and the macOS ps table.
|
No new findings in the specified PR diff. All three earlier findings are resolved:
The PR description includes sufficient representative real-CLI verification; no missing-e2e finding. Review was read-only. No PR code was run, built, or installed. |
…ds found Codex has no version detector, so the entry never took the unwaited form. The version compared with 0.148.0 is now the oldest of the Codex builds that read the member's hook file: codex on PATH and each desktop app's build. A build whose version cannot be read, or no build at all, keeps the entry waited. Each build is asked its version once per process, so the trust pass does not ask again.
debug.log now names each Codex build found and the version it printed, as it does for a tool's own version detector.
The hint moved from the SessionStart output to the first UserPromptSubmit after the session start's background pass leaves it. The offer test now waits for that pass's stash, reads the hint at the first prompt and nothing at the second. The disabled case asserted a marker the hint never prints; it now checks that no hint is left and none is shown at the prompt.
…e Code does A full pull asks Claude Code its version to choose the hook form. The fake logged that probe as an AI call, so the test that asks no AI tool saw one.
…aude Code does A full pull asks Claude Code its version to choose the hook form. The fake logged that probe as a model call for an unknown MR, so four cases saw an extra entry in its log.
…date policy" This reverts commit 82ad001. It stopped the one registry request that the shared update cache of Tencent#1751 needs under a skip policy to refresh a legacy null. That request runs in the Stop hook's background pass, which nothing waits on.
The unwaited form was written on Windows too, where neither Claude Code's nor Codex's hook was run with `async`. The member's own hook file now takes the unwaited and login-deferred forms on macOS and Linux only, and on Windows no tool is started to read its version. The guide also says that Codex's form follows every Codex build teamai finds.
…, as their originals' claude-internal and tclaude behave as Claude Code, and codex-internal and tcodex as Codex: each is assumed to run a hook as its original does. On macOS and Linux their own hook files now take the login-deferred form for PostToolUse and UserPromptSubmit, and the wrapper is written for them. They get no version floor, so their PostToolUse entry stays waited. Windows, a project file and every other event keep today's entries.
Also the wrapper and its doctor check, the MR hint at the first prompt, and the directories uninstall now removes.
… emptied The prune of the rules, skills and agents directories uninstall emptied also removed empty folders below them, which teamai never wrote: a member's empty `skills/drafts` went with it. Each directory is now removed only when nothing at all is left in it. AGENTS.md names the agent wait budget beside the context budget in `scripts/`.
…mber's Codex home `codex --version` first sets up its arg0 helpers under $CODEX_HOME/tmp, so each full pull, which now asks every Codex build its version, left a directory in ~/.codex/tmp per build. The probe now runs with a temporary CODEX_HOME, removed after it answers. The fake codex makes that directory as codex does.
…Codex does A full pull asks each Codex build its version to choose the hook form. The fake logged that probe as a trial's launch, so five benchmark cases saw an extra codex entry.
Each row now carries waitedMs, the milliseconds the tool waited on the row's commands in that one run; an unwaited entry adds nothing. It is not a count: no comparison, invariant or growth line reads it, and it shows only as the last column of the full table, both sides, below <details>.
|
No new findings in the specified PR diff. All three earlier findings are resolved:
The PR description includes sufficient representative real-CLI verification; no missing-e2e finding. Review was read-only. No PR code was run, built, or installed. |
SaulMoro
approved these changes
Nov 10, 2026
SaulMoro
left a comment
SaulMoro
left a comment
Collaborator
There was a problem hiding this comment.
This was generated by AI during triage.
Re-review at cdeec62b. Everything from the last round is resolved, so I'm approving. Merge waits for CI to finish green.
Resolved
- Codex now gets the unwaited
PostToolUseentry. With codex-cli 0.167.0 on PATH,init --agent codexwrites"async": trueinto~/.codex/hooks.json, anddebug.loglists every Codex build it found with its version. If any build's version can't be read, the hook stays waited. That is the right default. - The version probe leaves the member's Codex home alone. A
codexwrapper on PATH logged the call:--versionran with a throwawayCODEX_HOME, and nothing was left behind. Only the trust pass, which main already runs, wrote under~/.codex. - Windows keeps the waited entry (
builtinHookFormreturns early on win32), and the body, both guides and the glossary say so. The Windows job passed. - The CHANGELOG covers the wrapper, the doctor check, the earlier MR hint and the fixes, including the internal variants.
- Uninstall keeps a member's empty folder. A
draftsfolder made under.claude-internal/skillssurviveduninstall --force, and the emptied.tcodex/skillswas removed. AGENTS.mdnames the agent wait budget.- The revert puts
src/update.tsback exactly as on main (git diff main -- src/update.tsis empty).
Nits, not blocking
- [P3 nit] A workspace uninstall keeps
~/.tcodex/hooks.jsonfor other installs but removes~/.teamai/bin/teamai, which those kept entries start.sweepCreatedLeftoversdoesn't look atplan.keptGlobal. The entries fall back tobash -lc, so nothing breaks; they are only as slow as on main. A follow-up could keep the wrapper wheneverkeptGlobalholds a hook file. - [P3 nit] The internal variants get the login-deferred entry on the stated assumption that they run hooks through a POSIX shell. The waited form already assumes the same.
- [P3 nit] From the last round: the MR hint shows one prompt later on the first prompt. The CHANGELOG wording covers it.
Verified in an isolated verify world built from cdeec62b (macOS, claude 2.1.322, codex-cli 0.167.0):
init --agent codexwrites the unwaited entry.pull --forcewith the logging wrapper shows where each Codex call wrote.init --agent claude-internal,tcodexwrites login-deferred entries. APostToolUserun took the fast path, anddoctorreports the wrapper as current.uninstall --forcekeeps the member's empty folder.verifyshows the same leftovers as main's world for the same tools.
Collaborator
The Node 22 The code under test is fine. The assertion is what fails. A time in whole seconds survives the round trip exactly: const past = new Date(Math.floor(Date.now() / 1000) * 1000 - 60_000);My approval stands. I'll merge once CI is green on the new head and the review there is clean. |
The file time read back on Linux can sit a fraction of a millisecond below the one set, so the exact mtimeMs comparison failed there now and then.
All three earlier findings are resolved in their originally reported paths. The PR description includes sufficient representative real-CLI verification; no missing-e2e finding. Review was read-only. No PR code was run, built, or installed. |
Collaborator
The new automated finding is real, so one change before I approve this head:
if (flag === '--compare') {
if (!baseFile) throw new Error('--compare needs the JSON file a run on main wrote.');
const measured = headFile ? { budget: readBudget(headFile), failures: [] } : measureOrFail(false);
const { markdown, grew } = compareAgentWait(measured.budget, readBudget(baseFile));
process.stdout.write(`${markdown}\n`);
if (grew.length > 0) console.error(`Agent wait grew: ${grew.join('; ')}.`);
for (const failure of measured.failures) console.error(`Agent wait invariant broken: ${failure}`);
return measured.failures.length === 0 ? 0 : 1;
} |
Every init and pull wrote the wrapper as a new file. macOS takes longer over the first run of a new executable file, about 30 ms here for one with bytes it has seen, so the first hook after each pull that starts teamai through the wrapper waited that much more. The wrapper is now written only when its text differs or it is not executable.
The job log and a local run now show the time beside each row's counts, for reference; until main has the budget the comparison table is not written.
…n invariant `--compare <main.json>` with no head file measured the checkout and dropped what the measure found, so a failed command or a ledger leak still exited 0. It now reports them and exits 1, as the plain measure does. Growth alone stays a note.
|
No new findings in the specified PR diff. All four earlier findings are resolved:
The PR description includes sufficient representative real-CLI verification; no missing-e2e finding. Review was read-only. No PR code was run, built, or installed. |
SaulMoro
approved these changes
Nov 10, 2026
SaulMoro
left a comment
SaulMoro
left a comment
Collaborator
There was a problem hiding this comment.
This was generated by AI during triage.
Approving 1c6f2f9a.
1c6f2f9a: when--comparemeasures the head itself, it now reports that run's failures and exits 1. A head read from a file keeps the exit status it got when it was measured. This resolves the last finding.58ae5c21: an init or pull leaves the wrapper alone when the file already has the same content and is executable. A change in content, or a lost exec bit, still goes through the atomic rename. The new test covers all three cases: same content, lost exec bit and a changed Node path. The.cmdon Windows is still written every time, as before.17d9ba44: comparing the time as aDatefixes the failing assertion.
I'll merge once CI is green on this head.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every teamai hook and pull runs while a member's agent waits, and nothing measured that wait, so it grew unseen: on main a session start waits on 7 git processes, one
ghcall and, on macOS, 10ps. This PR adds the agent wait budget, a count of the processes each hook and pull makes the agent wait on, reported in CI beside the agent context budget, and then removes the waits it found, one commit per cut.Processes the agent waits on, second run, macOS:
gh/psRead): git / Nodegit worktree addwith four checkouts: gitteamai pullwith nothing new: git /claude --versionThe budget counts processes and files, never elapsed time: a count is the same in every run, and a time is not.
Type of Change
Test Plan
npx tsc --noEmitpassesnpm run lintpassesnpx vitest run <test files>); the fullnpx vitest runpasses if the change touchessrc/__tests__/helpers/, avitest*.config.ts,package.jsonorpackage-lock.json. CI runs the full suite on every push to a PR.Typecheck, lint and each commit's own tests pass at every runtime commit and at the head. Run once locally, on macOS only: the unit suite at the head (23868 tests, 23840 passed, 0 failed, 28 skipped) and the E2E suite at
14d16b35f(the one commit after it touches only the measurer; 2116 tests, 2081 passed, 0 failed, 35 skipped). The Linux and Windows runners, and the other repo providers and tools, are CI's.What the agent waits on, before and after
Every row, main / this PR
Counts are processes and files, never time: Claude Code 2.1.322, workspace scope, team repo mode git, darwin-arm64, git 2.56.0 (main: git 2.56.0), from the main checkout and linked worktrees. Not measured: a process other than git, Node and gh that the hook command's own shell starts (its login shell), Windows, tools other than Claude Code, user scope, self mode, HTTP mode, teamai's git hook on post-merge and post-rewrite, git worktree add when the team repo was last fetched more than 24 hours ago (it waits on one inline pull by design), a submodule, a workspace with workspace children, a bare repo.
Counts are processes, measured in a verify world on macOS 27 (arm64), git 2.56.0, Claude Code 2.1.322. No elapsed time is claimed.
mainhas no measurer, so CI cannot produce this comparison for this PR: the table is the PR's before and after.The four pull rows wait on one more process:
claude --version. A full pull reads the installed Claude Code's version to choose the form of the PostToolUse entry in~/.claude/settings.json. A hook andgit worktree addnever wait on it: their inline pull keeps the form the file already has, and the measurer now fails any row other than a pull that waits on the tool.git worktree addruns it once after it returns, in the full pull that follows.For a member with Codex, a full pull also asks each Codex build it finds for its version, once per build and process, and the trust pass reuses the answer: one build 0 -> 1
codex --version; two builds 2 (or 0 when the trust pass was skipped) -> 2. The probe runs with a temporaryCODEX_HOME, becausecodex --versionwrites under$CODEX_HOME/tmpbefore it reads its arguments. The measurer runs Claude Code, so no row shows it.Real-CLI record
fix(uninstall): a world ran every measured row, thenteamai uninstall; ledger CLEAN withproject/.claude/rulesno longer left behind.additionalContext, the second printed nothing.hook-dispatch; ledger CLEAN.pscut: the budget world at the head. Everygit worktree addrow andPostToolUse in a new worktreewait on 0claude(1 after), the pull rows on 1, and every SessionStart row on 1ps; no invariant broke; ledger CLEAN.teamai doctor: in a world, pull, then the wrapper's Node path replaced by hand:teamai pullrewrote it and doctor passed again.codexon PATH, codex-cli 0.167.0, and the ChatGPT desktop app's, 0.159.2), in a world built from6ade6477d:codex app-serverlists the entry aspost_tool_use:0:0 trustStatus=trusted. With a fakecodexthat answerscodex-cli 0.147.0first on PATH,teamai pull --forcewrote the login-deferred command with noasync, and it liststrusted. WithcodexTrustEnabled: falsethe async entry listsmodified; with the switch unset,trustedat the same hash. The ledger shows two leaks,home/.codexandproject/.codex; a world built from main shows the same two after the same commands (Codex's own app-server files, and the hook file a workspace uninstall keeps for other installs).Not verified
TodoWriteentry and its hint, in a real run:claude -p2.1.322 offers noTodoWritetool (it offersTaskCreate,TaskGet,TaskList,TaskUpdateandTaskStop). Unit and budget runs only.asyncline,codex execends without a hook warning, and apost-tool-use/codexline with its background pass; (2) a count of 2, thenprompt-submit/codexandpost-tool-use/codexlines; a ledger without LEAK.claude-internal,tclaude,codex-internalandtcodex: none was run. Their login-deferred form rests on the assumptionTOOLSalready records, that each runs a hook as its original does.-pfor async hooks: not run.lint-and-testand E2E jobs.windowsCI job.teamai updaterewrites the wrapper: read insrc/update.ts, not run.pull_request_target); the filter was run by hand.pstable on FreeBSD and the other BSDs: macOS local only.Related Issues
Builds on #1788 (the hook fast path) and #1807 (the verify world and the agent context budget).
Notes for Reviewers
Each commit stands alone and says which budget row it moves, so the PR reads best commit by commit.
Behaviour a member can see:
~/.claude/settings.jsonand~/.codex/hooks.jsonchange form on macOS and Linux, and so do the hook files ofclaude-internal,tclaude,codex-internalandtcodex(table below). A project file a repository commits keeps today's entries, and so does every file on Windows.teamai pullwrites~/.teamai/bin/teamai, andteamai doctorchecks it.git worktree addreturns, an exclude line no checkout needs any more stays until the full pull that follows finishes.git worktree add.Door: two-way. A revert, or a member going back to an older teamai, rewrites each hook file to today's entries at the next pull: every form keeps the literal
teamai hook-dispatchan older teamai recognises its entries by.Blast radius: on macOS and Linux, the hook file of every member who uses Claude Code, Codex or one of their four internal variants, and what each session start, prompt, tool call and Stop waits on. Windows keeps today's entries.
Commits
chore(verify): every world gets aghthat never reaches the network.test(verify):scripts/agent-wait-budget.mjsmeasures each hook and pull, waited and after, first and second run: git through trace2, Node and the other processes from a preload's spawn records, gh through the fake's log. It checks the invariants, among them that only a pull waits on the tool itself.ci: the budget job compares against main and the review shows the table.docs(dev): names the agent wait and its budget.fix(uninstall): removes the rules, skills and agents directories it emptied (the verify ledger needs it to end clean).perf(session): reads the team delivery once per session start. SessionStart git waited 10 -> 9 on the first run, 7 -> 6 on the others.perf(git-exclude): asks git what it tracks and offers once per pull. Pull git waited -2.perf(git-hook):git worktree addwaits the same however many worktrees exist: 14 / 14 / 16 -> 12 / 10 / 10. A line no checkout needs any more now stays in the exclude file from the momentgit worktree addreturns until the full pull that follows finishes (or the next full pull, if that one does not run). Before, the inline sync dropped it at once. This is the cost of the constant wait.perf(session): remembers what a delivered team revision names. SessionStart second run git waited 6 -> 4.perf(hooks): reads the checkout once at Stop for both of its readers. Stop git after 3 -> 1, first and second run (not waited).perf(hooks): looks for merged MRs after the session start returns, and shows the hint at the next prompt. SessionStart gh waited 1 -> 0, git waited -1.perf(env): asks git once in a login bash, not once per loader pass. Every login-shell hook git waited -1.perf(hooks): on macOS and the BSDs, oneps -Aper process answers every lookup of the tool's pid, as Windows' one table query already did. SessionStart waited ps 10 -> 1 on every row. Linux keeps/proc.refactor(hooks): each tool's hook file gets a built-in hook form (no behaviour change).perf(hooks): Claude Code 2.1.119+ runs PostToolUse without waiting, withTodoWritekept waited so its hint reaches the same turn. PostToolUse git waited 1 -> 0, Node 1 -> 0. A full pull reads the form from the version; an inline pull keeps the file's form and starts no tool. An upgrade or downgrade across 2.1.119 changes the file at the next full pull.perf(hooks): Codex 0.148.0+ runs PostToolUse without waiting: the single entry takes"async": true. No budget row moves (the measurer runs Claude Code).feat(hooks): a hook can start teamai through~/.teamai/bin/teamaiand run the login shell only around its background pass.perf(hooks): Claude Code below 2.1.119, or with no readable version, gets the waited PostToolUse entry in that form.perf(hooks): the prompt hook takes that form once a pass inside the login environment has seen that the session's directory does not sync over HTTP. UserPromptSubmit git waited 1 -> 0. The session's record adds files written on four first-run rows: SessionStart +2, PostToolUse outside any scope, PostToolUse in a new worktree and SessionStart in a new worktree +1 each; no second run writes more.feat(doctor): checks the wrapper a login-deferred hook starts, and namesteamai pullas the fix.perf(hooks): Codex gets the login-deferred form on macOS and Linux: PostToolUse below 0.148.0 or with no readable version, and UserPromptSubmit at any version. One helper gives the dispatch text of a built-in in either form, so the deferred entry keeps today'scommandWindowsbyte for byte. No budget row moves.From the reviews, on top:
fix(hooks): a waiting MR hint that was turned off before the prompt is dropped unshown.test(verify): the measure fails on a measured command that failed and on a ledger leak.fix(hooks): Codex's unwaited entry is gated on the Codex builds found. Codex has no version detector, so the entry never took the form. One hook file serves every Codex build on the machine, so the version compared with 0.148.0 is the oldest ofcodexon PATH and each desktop app's build; no build, or one whose version cannot be read, keeps the entry waited.chore(hooks)logs those versions indebug.log.fix(hooks): a Codex build is asked its version with a temporaryCODEX_HOME.fix(hooks): the unwaited form stays off Windows, where no tool is started to read its version.feat(hooks): the four internal variants get the login-deferred form.fix(uninstall): an empty folder a member made keeps its directory.Revert "fix(update)…": see Tried and left out.test(e2e), four commits: the MR hint is expected at the first prompt, and the fakeclaudeandcodexof the import and benchmark tests answer--versionas the tools do, without logging it as a model call.docs(changelog): entries under Features, Changed and Bug Fixes.test(verify), two commits: each row records how many milliseconds the tool waited in its one run, for reference. It shows in the measure's log line for the row and in the last column of the full table, and nothing compares it. Until main has the budget, the job log is where this PR's times are.perf(hooks): a pull that finds the wrapper unchanged keeps the file. macOS takes longer over the first run of a new executable file: on this machine about 150 ms for a script with bytes it has not seen and about 30 ms for a new file with bytes it has, against 4 ms after. Every pull wrote a new file, so the first hook through the wrapper after each pull waited those 30 ms; the first hook ever through a wrapper still waits the 150 ms, once.test(hooks): the enrolment record's time is compared as a date; the exactmtimeMscomparison failed on Linux in CI.Elapsed time of the prompt hook, for reference, since the budget claims none: 15 runs each in a world of each build on this machine, median 47 ms on this PR and 87 to 116 ms on main, with the world's temporary directory emptied or not. The measure's
UserPromptSubmit, first runshows about 190 ms on macOS because it is the first run ever of that world's wrapper.What changed in a session (the second
perf(session)commit), as the stored answer for a delivered revision:ls-treefails, listing falls back toHEAD,teamDeliverysaysunresolved_revisionrepositoryStampchanges, reads againHook forms per tool
claudecodexcodexon PATH and each desktop app's) is 0.148.0 or later: one entry,"async": true, no matcherasynchandler); run on 0.167.0 for UserPromptSubmit and SessionStart, PostToolUse not run. Login-deferred: source at 0.134.0 to 0.167.0 (zsh,bashorshwith-c, never-lc); run on 0.167.0, macOS (/bin/zsh -c, no login flag). The hook file, the build gate and the trust status: the record above. A Codex turn running teamai's entry: not runclaude-internal,tclaude,codex-internal,tcodexTOOLSalready records; not run. A detector and a floor are all the unwaited form needs laterqoderasyncOn a Claude Code at 2.1.119 or later,
~/.claude/settings.jsonholds the PostToolUse key as two entries:^(?!TodoWrite$).*$withasync, andTodoWritewaited. Both keep the keyHook dispatch post-tool-use wildcard, sobuiltin.disabledand a timeout override cover every tool call, as today. Cursor and Copilot CLI, which also read Claude's settings, see two disjoint matchers, so one call matches one entry and nothing is recorded twice (read in the source; neither tool run). Codex gets one entry: its matchers are the Rustregexcrate, which has no look-ahead, and Codex drops a group whose matcher does not compile.Codex trusts a hook by a hash of its handler,
asyncand the command included (hook_hash,codex-rs/hooks/src/engine/discovery.rs), so either change reads as modified until it is trusted again. teamai computes no hash: after each write it lists the hooks throughcodex app-serverand writes back the hash Codex lists for each (trustCodexHooks), so the changed entries are trusted by Codex's own computation. That trust pass runs forcodexonly, on main and here. Ifcodex-internalortcodexkeeps a hash per handler as Codex does, their changed command reads as modified until the member trusts it again, as after any release that changed an entry.The login-deferred command keeps the literal
teamai hook-dispatch, so an older teamai still recognises its entry:Without the wrapper it runs today's command. With a wrapper whose Node or teamai is gone, the wrapper hands the hook to a login shell once, and
teamai doctorreports it.Claude Code real runs: 2.1.322, 2026-11-10, macOS arm64,
claude -p --setting-sources project --model haikufrom a verify world's project, with the PostToolUse block teamai wrote copied into the project's settings and each command run inside the world:Read: onetool_useevent recorded,fast paththen the background pass.EnterWorktreethenReadinside it: both events, and the team rule delivered into the new worktree.claude -pexits.Hit every surface
claude-internal,tclaude,codex-internalandtcodexget the login-deferred form only; every other tool keeps today's entries byte for byte (existing goldens unchanged; new goldens for each new form).builtin.disabledand timeout overrides apply to both PostToolUse entries (same key).post-checkout,teamai pull,teamai hooks listandteamai doctor; the git hook onpost-mergeandpost-rewritenot measured. The instruction readiness check asks about the file's own form, so it never starts a tool;teamai hooks listcompares with the version's form.contribute-check.test.ts).commandWindows(test). Thepscut does not touch the Windows path.docs/guide/sharing.mdand its zh-CN page (the PostToolUse and UserPromptSubmit entries, Claude Code and Codex); the MR hint wording in knowledge, sharing and workflow guides and the hook's README; glossary entries for the agent wait, its budget, and the built-in hook form;AGENTS.mdnames the budget underscripts/; CHANGELOG. skill-data: no change needed.Always-loaded context: unchanged. Agent wait budget: every git, Node, gh and ps count that changed goes down. The pull rows wait on one
claude --version, for the reason above. Files written grow on four first-run rows, the prompt hook's per-session record (commit list above).Tried and left out
git --versiononce per process for a member whose git is a script. Left: replacing the spawn with reading the shebang must be strictly narrower than the kernel, for one process per session start.show <rev>:./manifest/members.yamlin a cold user-scope pull. Left: that scope is not measured.skipupdate policy. Reverted in this PR: it stopped the one request the shared update cache of [bug] Shared negative update cache hides older TeamAI installations on the same machine #1751 needs, in a pass nothing waits on, and it moved no budget row.