feat: trace judge calls in Braintrust

[mattrossman]

Diff note: 23 of the changed eval files only swap judge() for ctx.judge()

Scorers now ask for verdicts through ctx.judge(), which records each judge call's model, full prompt, output, tokens and timing in result.json. The uploader logs each call as an LLM span under the score span:

passed (score, purpose: scorer)
├─ gpt-6-sol (llm, purpose: scorer)
└─ gpt-6-sol (llm, purpose: scorer)

purpose: "scorer" is how Braintrust keeps judge cost out of preset cost charts. Core no longer exports judge() so folks are encouraged to use ctx.judge as the preferred way to access the judge in scorers similar ctx.query.

As an incidental fix, the root eval span no longer logs agent tokens, since its LLM spans carry them. Braintrust sums metrics across every span in a trace, so with the initial approach in #351 the experiment table double counts tokens (example). Note aiSdkAgent has no per-request usage so its rows now show no tokens, but it's unused and slated for removal anyway.

Ran the regression suite with codex-gpt-6-luna through the refresh workflow (snapshot, trace with 3 judge calls).

Closes AI-1271

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
evals	Ignored	Preview	Oct 5, 2026 9:18pm UTC