Retry replay-safe MCP tool calls

[xeophon]

Overview

Teach the shared null/bash MCP client to retry a tool call after response loss only when the evaluator-configured server explicitly declares that repeating it is safe.

Details

• Add read_only and idempotent options to @vf.tool and publish the corresponding standard MCP tool annotations.
• Carry explicit readOnlyHint and idempotentHint values into the delivery-aware retry policy introduced by Fix MCP session lifecycle and replay safety #2134.
• Preserve delivery-unknown handling for unannotated calls and for annotations that do not guarantee safe repetition.

Scope

This pull request is stacked on #2138, which builds on #2134. It does not claim generic exactly-once execution: non-idempotent actions still need server-owned operation IDs, revisions, and result caching before opting into retry.

---

Note

Medium Risk
Changes retry behavior for MCP tools after transport loss; mis-declared idempotent/read-only tools could be executed twice, though the default remains no replay without explicit hints.

Overview
MCP tool retries after a lost response are now gated on explicit server hints, instead of treating all call_tool invocations as non-replayable.

The shared minimal harness reads each tool’s MCP readOnlyHint / idempotentHint at enumeration time, stores a per-tool replay_safe flag in dispatch metadata, and passes it into mcp_session for call_tool so the existing delivery-aware retry logic can replay only when repetition is declared safe. Unannotated tools keep the prior conservative behavior (no retry after the operation may have started).

Evaluator-authored toolsets can opt in via @vf.tool(read_only=…) / idempotent=…, which Toolset maps to standard MCP ToolAnnotations when registering tools with FastMCP.

^{Reviewed by Cursor Bugbot for commit 5ce6281. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Retry replay-safe MCP tool calls based on readOnlyHint and idempotentHint annotations

• Extends the @tool decorator in decorators.py with read_only and idempotent keyword parameters that attach metadata to decorated functions.
• Toolset._register reads this metadata and advertises matching ToolAnnotations (readOnlyHint/idempotentHint) when registering tools with MCP.
• connect_mcp inspects each tool's annotations to compute a replay_safe flag, stored in the dispatch map alongside the server and tool name.
• call_mcp passes the replay_safe flag to mcp_session, enabling retries on lost responses for read-only or idempotent tools; previously all calls used replay_safe=False.

^{Macroscope summarized 5ce6281.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR changes retry behavior for MCP tool calls from never-retry to conditionally-retry based on annotations. Changing retry/replay semantics has potential for non-obvious runtime impacts and warrants human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[mikasenghaas]

i dont love that the tool server has to opt-in. would be curious how mcp retries are handled in codex, etc. do they just blindly retry or is there any standard mechanism in mcp to interact with stateful tool servers?

[mikasenghaas]

isnt read only also idempotent? so could we just use mark idempotent = replay is safe?

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.