Repository navigation
fix(subagents): Tell a child that a missing-input reply is not a result - #2159
sentry-junior[bot] wants to merge 3 commits into
Conversation
Co-Authored-By: David Cramer <david@sentry.io>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
| }, | ||
| description: | ||
| "End this delegated task as blocked. Call this when missing access, credentials, input, or a decision stops the task, also when a search for the missing part found nothing. Do not guess, mock, or fake results to work around it.", | ||
| "End this delegated task as blocked. Call this instead of a final text reply when you cannot give the result that the task asks for because access, credentials, input, or a decision is missing, also when a search for the missing part found nothing. A reply that only says what is missing is not a result. Do not guess, mock, or fake results to work around it.", |
There was a problem hiding this comment.
the final text reply i dont think you get a choice on do you? given the assistant api contracts that exist? i might be forgetting something
There was a problem hiding this comment.
You do get a choice here. reportBlocked ends the run on the tool call. Its tool result returns terminate: true in packages/junior/src/chat/agent/index.ts (the FINISH_AUTOMATION_RUN_TOOL_NAME || REPORT_BLOCKED_TOOL_NAME branch), so no model call and no text reply follow it. finishAutomationRun works the same way. In the last step, the model either ends with text, which becomes the "finished" result, or it calls reportBlocked, which makes the agent invocation blocked. "Instead of a final text reply" means that choice.
This wording did not fix the eval. The case still failed live on both pushes: in each run the child replied with one sentence about the missing region and called no tools. I will revert the extra prompt-line edit, so that I do not stack prose that does not help. The root cause is still open.
…it has no result" This reverts commit 004013c.
|
Status: this change does not fix the eval yet. "Child Agents › when a child agent is missing input…" failed live in The last git recording (#2123) had a low-reasoning child that called |
A child agent that is missing input sends a text reply that says the input is missing, for example "No deploy region was specified." It does not call
reportBlocked. The parent then gets "Child agent finished its task", and the agent invocation ends ascompleted, notblocked.The integration eval
subagents.eval.ts› "when a child agent is missing input, it reports the block and the parent says what is missing" shows this. Live runs on #2156 failed 4 times in a row. In each run the child made no tool calls and replied with one sentence about the missing value. The parent now setsreasoning_level: "low"onspawnAgent, so the child answers in one step. Older recorded children searched first, and then they calledreportBlocked.The
reportBlockeddescription said to call it when missing input "stops the task". The model reads a reply that names the gap as a finished answer, so it did not think that the task had stopped. The description now says to call the tool instead of a final text reply when the child cannot give the result that the task asks for. It also says that a reply that only says what is missing is not a result. The tool description is the one home for this rule (policies/agent-steering.md). The system prompt and the eval are unchanged.How CI checks this: #2156 moved evals to the hosted Roach service, and it is merged into
main. Roach has no recording for this case, and it drops recordings from failed tests. So theintegration / shard 2/2check on this PR calls the model live with the write token. Before #2156,mainreplayed an old recording from git, and that replay passed whatever the product did. A pass here is real evidence.via Junior system actor
event.--
View Junior Session [Sentry]