[yileicn]

Summary

Agent-testing UI updates: a new Template case type, a per-turn file editor, llmJudge spend on the run page, and a round of review fixes.

Case editor

• Template case type: calls one of the entry agent's templates directly, so it needs an entry agent and a template picked from it, and cannot have history. Changing the entry agent (by hand or from an AI draft) clears the template.
• Assertion types a template call cannot satisfy (routedToAgent, agentChain, toolCalled, toolNotCalled) are hidden from the type picker and rejected on save; the richContent output source is hidden too, since a template call returns text only.
• Turn files: each turn can add, edit and remove files (name + URL). Files are kept through load and save, so editing a case no longer clears files attached through the API.
• Entry agent, involved agents, template and the scope page's changed-agents pickers are searchable.
• The reviewed date is stamped in local time (toISOString() gave yesterday before 08:00 in UTC+8).
• If the form is edited while the authoring assistant is working, its draft still replaces the form, but a warning now says so and Undo brings the edits back.

Run page

• Reloads when only the run id changes (SvelteKit reuses the component), ignores stale responses, and keeps the last snapshot when a poll fails.
• "Re-run failed" queues one run per model set the failures ran on.
• Stats and pricing are keyed by provider + model, so the same model name under two providers no longer collides.
• llmJudge spend is shown separately from the model under test: the cost table has "Agent tokens / cost" and "Judge tokens / cost" columns, and each case has a Judge badge.
• A Judge row at the end of the cost table names the judge model the run was scored with, its total spend and its unit cost. It is read from the run, not the suite, so changing the suite's judge later does not relabel old runs.
• Progress shows as finished / planned ("10 / 30") on the run page and in the suite's run history, and the cost table fills in while the run goes. Runs without plannedCount show the finished count as before.

Lists

• Suite and case/run lists show a load error with Retry instead of an empty state.
• The suite agent filter applies on pick and ignores out-of-date responses.
• A new success or error notice restarts its timer instead of being hidden early by the previous one.

Backend dependency

Needs the matching BotSharp AgentTesting plugin changes (Template case type, judgeTokens / judgeCost on results, pricing snapshot, judgePricing and plannedCount on runs). Against an older plugin build the Judge column and Judge row stay hidden and unit cost shows as unknown.

Test plan

• Create a Template case: pick an entry agent and template, add turn files, save and reload
• Run a suite with an llmJudge assertion; check the Judge column, the Judge row and the per-case badge
• Re-run failed on a multi-model run
• Filter suites by agent; switch quickly between agents

🤖 Generated with Claude Code

[qodo-code-review]

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →