Skip to content

Add the autowebcompat-etp agent for ETP Strict-mode breakage - #7582

Open
mozmraiber wants to merge 37 commits into
mozilla:masterfrom
mozmraiber:autowebcompat-etp-agent
Open

mozmraiber wants to merge 37 commits into
mozilla:masterfrom
mozmraiber:autowebcompat-etp-agent

Conversation

@mozmraiber

Copy link
Copy Markdown

Add the autowebcompat-etp agent for ETP Strict-mode breakage

This adds autowebcompat-etp, a hackbot agent that investigates site breakage reports that may be caused by Enhanced Tracking Protection (ETP) in Firefox. It also adds a second agent, autowebcompat-etp-compare, and a local runner for reviewing results before anything is posted to Bugzilla.

autowebcompat-etp

The agent drives Firefox Nightly through DevTools MCP. It follows the report's steps in Standard and Strict mode and uses the content blocking log to find the blocked resources responsible. It then proposes one of these fixes:

  • entries for the url-classifier-exceptions Remote Settings collection, after checking with the skip-URL prefs that they fix the breakage, or
  • a SmartBlock shim, when an exception would only be a convenience fix.

If it can't propose a fix, it returns a failure_reason instead, for example not_etp, not_reproducible, broken_in_standard, fixed_by_list_update, not_fixable_by_exception, anti_adblock, or blocked_captcha. For anti-adblock walls it always proposes the exceptions, but they are only added if a reviewer opts in.

The result also includes:

  • Before/after screenshots and the content blocking log for each ETP mode, published as run artifacts
  • Proposed whiteboard tags, user story lines, severity and priority
  • Exceptions and shims that already exist for the site
  • Tracker list history (links to the shavar-prod-lists commits that added the entries), so it can tell a stale list apart from a current block
  • A comparison against EasyPrivacy and EasyList+EasyPrivacy, with blocked tracker counts

The agent has a retest mode that reruns an old bug with Firefox's built-in ETP exceptions turned off, to check whether they are still needed.

The agent only returns a result. It doesn't write to Bugzilla itself. Its Bugzilla reads go through a broker, as the other autowebcompat agents.

autowebcompat-etp-compare

This agent loads a site in four configurations: off, strict, strict/easyprivacy and strict/easylist+easyprivacy. It takes stitched full-page screenshots of each and decides whether the differences are normal variation between page loads or real breakage. It tags the breakage for each configuration.

Local runner

agents/autowebcompat-etp/local_runner/ contains a small server (a PEP 723 script) and a React UI. It starts the agent through its compose.yml, streams the output, and shows the result next to a draft of the Bugzilla changes. After you edit and approve the draft, it posts the changes with your own API key. It can also close the bug, needinfo the reporter, and queue all open site reports. See its README for setup. This was used for testing, see e.g. https://bugzilla.mozilla.org/show_bug.cgi?id=2088845 .

Changes outside the agent

  • libs/autowebcompat-tools/task.py: if an agent finishes without calling submit_result, it now gets one retry turn instead of failing straight away. This affects every autowebcompat agent.
  • Registration: the agent is registered in hackbot-api (AutowebcompatEtpInputs with mode: investigate | retest), hackbot-ui, docker-compose.yml and the prod deploy workflow.
  • Docs: added to docs/hackbot/agents.md and security.md.
  • Pre-commit: codespell now skips pnpm-lock.yaml.

Deployment

hackbot.toml requests a Cloud Run job with 6 CPU, 24 GiB of memory and a 2 h timeout, plus a broker (same as e.g. autowebcompat-investigate).

Testing

  • Unit tests are in agents/autowebcompat-etp/tests/ (agent, compare, exceptions, result, tracker lists).

…istory

Add a wait_for_page_load tool to the patched DevTools MCP server: it polls
document.readyState until the load event instead of waiting inside
evaluate_script, which the bundle aborts after 10s, and reports a page that
never finishes loading rather than failing.

Add a check_tracker_lists tool backed by mozilla-services/shavar-prod-lists,
showing the list entries covering given hosts and the recent commits that
added or removed them. A site fixed by a recent removal works in the agent's
fresh profile but not for a reporter with older lists, which the agent now
reports as the new fixed_by_list_update outcome.
A FastAPI server (agents/autowebcompat-etp/local_runner/server.py) that runs
the agent through its compose.yml, queues runs, streams their output, and
lists untriaged Privacy: Site Reports bugs to queue in bulk. A React app
(web/, pnpm + Vite + Valtio) shows each result next to a draft Bugzilla
comment, screenshots and field changes, which are posted with the user's own
BUGZILLA_API_KEY after review.

Exclude pnpm-lock.yaml from codespell, as JSON lockfiles already are: its
package names (devlop, trough) read as typos.
Also publish the content blocking logs with the run.
@mozmraiber
mozmraiber requested review from a team as code owners November 19, 2026 12:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant