Skip to content

Add interception example environments - #2178

Open
xeophon wants to merge 8 commits into
agent/intercept-terminalfrom
agent/intercept-rules
Open

Add interception example environments#2178
xeophon wants to merge 8 commits into
agent/intercept-terminalfrom
agent/intercept-rules

Conversation

@xeophon

@xeophon xeophon commented Jul 30, 2026

Copy link
Copy Markdown
Member

Overview

This adds four small runnable environments that show the intended @vf.intercept interface directly on a task.

interception-v1 shows deterministic and ordinary LLM judge guards that rewrite matching messages, plus a termination guard that ends the rollout without a replacement message. The judge can use the canonical trace.

grayscale-interception-v1 shows typed user-message rewriting. It converts every inline image in the user message to grayscale with Pillow, preserves the other content parts, and returns a rewritten UserMessage before the request reaches the model.

bash-interception-v1 shows that an AssistantMessage interceptor can replace a Bash tool call before the Bash harness executes it. It records whether the call was intercepted and whether its sentinel side effect occurred.

web-search-interception-v1 enables Codex's provider-native web search. Its AssistantMessage interceptor detects a completed web_search_call, inspects the search-backed response, and terminates when the response contains a blocked word. Because intercepted responses are buffered, the blocked response does not reach the Codex harness. The provider has already executed the search at that point.

What this PR adds

  • Deterministic and judge-backed content guards.
  • A termination guard with a final reward.
  • A Bash example that blocks a local tool call before execution.
  • A Pillow example that rewrites user images before the model request.
  • A Codex example that blocks a native web-search response before the harness receives it.
  • A small Codex harness option for enabling native web search.
  • Example-only dependencies, with no root dependency change.
  • Editable example installation in the existing V1 live job.

Stack

This is PR 4 of 4. The stack should merge in order:

  1. Serialize model exchanges within a rollout #2164 — Serialize model exchanges and buffer intercepted streams. Establishes one active exchange per rollout and buffers streams only when response interception is active.
  2. Add typed model interception #2165 — Add typed @vf.intercept and graph-owned rewrites. Adds typed request and response interception and stores canonical rewrites in the trace graph.
  3. Make interception termination final #2166 — Add Terminate and make its reward final. Adds non-retryable early termination and prevents later scoring from changing the terminal reward.
  4. Add interception example environments #2178 — Add interception examples. Shows deterministic, judge-backed, termination, image-rewrite, Bash-tool, and Codex native-search interception.

Note

Add interception example environments for bash, grayscale, web search, and judge-based blocking

Adds four new example environments under environments/ to demonstrate the @vf.intercept() API across different interception patterns:

  • interception_v1: Shows three intercept handlers — deterministic string blocking, early termination on a marker, and async judge-based blocking with configurable OpenAI/PRIME credentials.
  • bash_interception_v1: Intercepts a specific bash tool call before it executes a sentinel command and rewards successful blocking.
  • grayscale_interception_v1: Rewrites inline base64 image data URLs to grayscale using Pillow before they reach the model.
  • web_search_interception_v1: Terminates rollouts with a positive reward when a Codex web-search response contains a blocked word.
  • Adds a search flag to CodexHarnessConfig that passes web_search="live" to the Codex process when enabled.
  • All four packages are installed as editable dependencies in CI via test.yml.

Macroscope summarized ff31809.

Comment thread verifiers/v1/intercepts/rules.py Outdated
Comment thread verifiers/v1/dialects/anthropic.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d41b94349e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/dialects/chat.py Outdated
Comment thread verifiers/v1/intercepts/rules.py Outdated
@macroscopeapp

macroscopeapp Bot commented Jul 30, 2026

Copy link
Copy Markdown

Approvability

Verdict: Approved ff31809

Adds 4 self-contained example environments demonstrating interception patterns, with minimal core changes: publicly exporting an existing function and adding an optional disabled-by-default search config flag to CodexHarness. Low-risk additive changes that don't affect existing runtime behavior.

You can customize Macroscope's approvability policy. Learn more.

Comment thread verifiers/v1/session.py
Comment thread verifiers/v1/interception/server.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7aae47f960

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/dialects/responses.py Outdated
Comment thread verifiers/v1/intercepts/rules.py Outdated
@xeophon
xeophon force-pushed the agent/intercept-rules branch from 7aae47f to b1f77a4 Compare July 30, 2026 09:39
Comment thread verifiers/v1/dialects/chat.py
@xeophon
xeophon force-pushed the agent/intercept-rules branch from b1f77a4 to 3113425 Compare July 30, 2026 09:47
Comment thread verifiers/v1/dialects/anthropic.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 311342505e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread environments/interception_v1/pyproject.toml
@xeophon
xeophon force-pushed the agent/intercept-rules branch from 3113425 to 4b92fe4 Compare July 30, 2026 09:56
@xeophon
xeophon force-pushed the agent/intercept-rules branch 3 times, most recently from 8dad0d9 to b12342b Compare July 30, 2026 10:25

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b12342b8be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/session.py Outdated
@xeophon
xeophon force-pushed the agent/intercept-rules branch from b12342b to ecb3b4b Compare July 30, 2026 10:36
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Jul 31, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

1 similar comment
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Comment thread environments/interception_v1/interception_v1/taskset.py Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

1 similar comment
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Comment thread environments/bash_interception_v1/bash_interception_v1/taskset.py Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Comment thread environments/web_search_interception_v1/web_search_interception_v1/taskset.py Outdated
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Jul 31, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

1 similar comment
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Comment thread environments/interception_v1/interception_v1/taskset.py Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1dde940. Configure here.

Comment thread environments/web_search_interception_v1/web_search_interception_v1/taskset.py Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

1 similar comment
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Comment thread environments/web_search_interception_v1/web_search_interception_v1/taskset.py Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

1 similar comment
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant