[andrewleesteele]

Stacked on #10 (retail-cart), which is stacked on #9.

Summary

Runs the 16 retail-cart guest add-to-cart tasks on Kernel, Browserbase, Steel, Hyperbrowser and Anchor browsers. Every arm uses the same agent (Codex, gpt-6-luna), the same tools and the same task container, so the only thing that differs between arms is the browser.

• remote-browser environment (RemoteBrowserEnvironment): runs the task container on the runner's Docker instead of inside a Kernel VM. Once the container is up, it creates a browser through harness/providers.py, writes the browser's CDP URL to /run/browser/cdp-url in the container, and releases the browser when the trial ends. A browser that can't be created fails the trial with BrowserCreationError, so it shows up in the infrastructure failures table. Arms choose it with environment = "remote-browser". The default is still kernel.
• Providers: Kernel through the SDK; Browserbase, Steel, Hyperbrowser and Anchor through their REST session APIs. An arm's [browser.config.params] is the provider's own session create body, so the arm file shows exactly which stealth, proxy and CAPTCHA settings each provider was given.
• playwright-mcp toolset: Playwright MCP (pinned to 0.0.84) attached over CDP. It works in the browser's existing context and page. The toolset ref sets the version.
• retail-cart:
  • open-start-url and score use a new cdp-execute script when the CDP URL file exists. It runs the same page-scoped code that Kernel's playwright/execute runs. Kernel-VM arms are unchanged.
  • The finalize tool's kernel_error description now just says "the browser", so agents on other providers can report browser failures. The outcome key is unchanged.
• Arms: codex-pw-{kernel,browserbase,steel,hyperbrowser,anchor}, each with that provider's anti-bot settings:

Arm	Settings
codex-pw-kernel	stealth: true
codex-pw-browserbase	Advanced Stealth (needs the Scale plan), solveCaptchas, proxies: true, keepAlive
codex-pw-steel	useProxy, solveCaptcha
codex-pw-hyperbrowser	useStealth, useProxy, solveCaptchas
codex-pw-anchor	US Anchor proxy, extra_stealth, captcha_solver

Sessions have to survive a CDP disconnect, because the start page, the agent and the verifier each connect separately. Browserbase gets keepAlive, Anchor a 30-minute idle timeout, and every provider a session timeout of 30 minutes or more.

• Workflow: retail-cart-providers.yml, 10 attempts by default. Concurrency defaults to 4 per arm, because every arm's containers run on the same runner. _run.yml passes BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID, STEEL_API_KEY, HYPERBROWSER_API_KEY and ANCHOR_API_KEY. A run checks for the keys of the providers its arms use before it starts.

Before a full run

• The repo needs the competitor keys above as secrets. I couldn't check the repo's secrets from here.
• The competitor adapters are tested against mocked responses only. Their first live call has not happened yet, so the first run should be one task per arm.
• Browserbase's Advanced Stealth needs the Scale plan. Without it, drop advancedStealth from the arm and say so next to the results.
• This compares add-to-cart on 16 US sites. It does not measure purchase completion and does not use UK merchants. A UK suite would reuse all of this with new tasks.

Testing

• ruff check, ruff format --check, pytest (39 passed). New tests cover the providers (mocked HTTP), the remote-browser environment's start and stop order, the toolset, and arm environment validation.
• bench prepare retail-cart-providers for every arm, added to CI.
• Live, on a Kernel browser:
  • Component check: the start page opened over CDP, Playwright MCP attached to that page and listed 25 tools, and the verifier read the page and the cart. The browser was deleted afterwards.
  • One full Harbor trial of codex-pw-kernel on costco-microfiber-towels, run locally: reward 1 and cart_verified 1 (the cart listed the 36-count Kirkland towel), 2 minutes, browser released.
• Not run: any competitor arm, because there are no keys yet.

🤖 Generated with Claude Code