✅ 加固 UI 异步断言与 E2E 保存结果观察 (#1727)

* test: stabilize heavy network rules UI cases

* test: isolate UI files and await observable state

* ✅ 保留 UI 测试原有隔离配置

* ✅ harden async test observations

* 🐛 align E2E save expectations with failure cases

* 🔍 tighten test guard binding and toast observation

Author: cyfung1031

AGENTS.md

@@ -10,6 +10,7 @@ duplicating its rules.
 | Before you… | Read |
 | --- | --- |
 | write code | [`docs/develop.md`](docs/develop.md) |
+| modify tests, test helpers, or test runner configuration | [`docs/references/develop-testing.md`](docs/references/develop-testing.md) — apply the test-boundary, observation, and harness rules before editing |
 | review or report a branch/PR, or create/update a PR or publish its branch | [`docs/develop.md#revision-scope-and-publication-binding`](docs/develop.md#revision-scope-and-publication-binding) + [`docs/pull-request.md`](docs/pull-request.md) |
 | change a process/message/service/persistence boundary or add a subsystem | [`docs/architecture.md`](docs/architecture.md) + the relevant `docs/references/architecture-*.md` |
 | build or modify a page, dialog, or block | [`docs/design.md`](docs/design.md) — Core Constraints apply to every UI change |
@@ -48,6 +49,13 @@ downstream prose does not override it.
   Chinese or English titles. The two narrow, non-blanket exceptions are in
   [`docs/references/develop-testing.md`](docs/references/develop-testing.md#when-tdd-doesnt-apply); runner,
   mocks, and how to run tests are in [`docs/develop.md`](docs/develop.md).
+- **Test changes must follow the test route.** Before changing a test, shared test helper, or runner configuration,
+  identify the observable contract and test boundary, search existing coverage, capture a baseline or reproduction,
+  then make the smallest correction and rerun focused and relevant broader checks. A single passing run does not
+  establish a root-cause fix; report the trigger, evidence, and remaining uncertainty.
+- **Shared E2E helpers must model both outcomes.** A helper that drives a save, install, or other mutation must make
+  the expected success or failure explicit and wait for that operation's matching signal. Negative cases must opt into
+  the failure contract; never make them pass by accepting an arbitrary toast, an old notification, or a page shell.
 - **SOLID, high cohesion, low coupling.** Match existing extension points: persistence uses the small
   `Repo<T>` / `DAO<T>` / `OPFSRepo` / custom-repo taxonomy, matching an existing entity with the same needs;
   messages use `Group.on(...)`; service constructor shapes differ by context and Agent subsystem; depend on

docs/architecture.md

@@ -288,7 +288,8 @@ premature abstraction.
   test-first principle and [develop-testing.md § When TDD doesn't apply](./references/develop-testing.md#when-tdd-doesnt-apply)
   for the narrow exceptions — this section only covers architecture-specific test mechanics, not the policy
   itself.
-- **E2E (Playwright).** `e2e/*.spec.ts`, one worker, real Chromium. `pnpm run test:e2e` (first run:
+- **E2E (Playwright).** `e2e/*.spec.ts`, real Chromium; worker count and retries come from
+  [`playwright.config.ts`](../playwright.config.ts). `pnpm run test:e2e` (first run:
   `pnpm run test:e2e:install`).
 - **Before a PR:** lint + the relevant suite — owned by [references/develop-testing.md](./references/develop-testing.md) → *Testing*.
 

docs/develop.md

@@ -21,7 +21,7 @@ pnpm run coverage
 pnpm run typecheck        # tsc --noEmit
 
 pnpm run test:e2e:install # install Playwright Chromium (first run only)
-pnpm run test:e2e         # Playwright (e2e/*.spec.ts, 1 worker)
+pnpm run test:e2e         # Playwright (e2e/*.spec.ts; worker count comes from playwright.config.ts)
 pnpm run lint             # prettier --check + tsc --noEmit + check:i18n + check:issue-templates, then eslint
 pnpm run lint-fix         # prettier --write + tsc --noEmit + eslint --fix
 

docs/references/develop-testing.md

@@ -7,6 +7,57 @@ This guide owns how contributors design, write, review, clean up, and run automa
 merely because it raises coverage: it must protect an observable contract, fail for a relevant regression, and
 cost less to understand and maintain than the confidence it provides.
 
+## Test-change route and evidence
+
+Before modifying a test, shared test helper, or runner configuration, classify the contract and boundary first:
+
+1. State the trigger, observable outcome, and plausible regression.
+2. Search nearby unit, component, service, E2E, and lint coverage before adding or deleting a case.
+3. Run the narrowest baseline or reproduce the failure under the same runner, reporter, coverage, shard, and worker
+   conditions that exposed it.
+4. Change one cause at a time, then run the focused test and the relevant broader combination.
+5. Report exact commands and distinguish a passed assertion from an unobserved channel or unverified negative.
+
+One passing run is evidence for that run only. Do not treat a timeout increase, retry, deleted assertion, or arbitrary
+sleep as a root-cause repair.
+
+### Observation rules for asynchronous tests
+
+The test must observe completion of the contract under test. A request being called proves that work started; it does
+not prove that state, persistence, rendering, or the user-visible result completed. Use the narrowest primitive that
+matches the boundary:
+
+- Use direct assertions for synchronous effects and one `act` for a Promise-driven React update.
+- Use `findBy*` for a single element that appears asynchronously. Do not wrap `getBy*` in `waitFor` for a lone
+  `toBeInTheDocument` assertion.
+- Use `waitFor` for genuinely open-ended async state, multiple related assertions, or a non-DOM boundary that has no
+  dedicated completion signal. Keep the callback observational: do not fire events or call `userEvent` inside it,
+  because retries repeat the interaction.
+- Use a real timer only when elapsed time is the contract or the only bounded closure window proves a negative result
+  (for example, an observer timeout, a runaway retry check, a browser event-loop yield, or a library timer). Add a
+  local ESLint disable comment stating that contract. A fixed delay used merely to make a test pass is a defect.
+
+The mechanical guards `scriptcat/no-test-waitfor-interaction`, `scriptcat/no-test-waitfor-query`, and
+`scriptcat/no-test-fixed-sleep` cover reliably recognizable forms in committed page tests and E2E specs. They do not
+prove mock fidelity, the sufficiency of a negative observation window, or that coverage was not weakened; those remain
+semantic review duties. Do not disable a whole directory to silence them.
+
+The interaction and query guards follow actual Testing Library import bindings, including local aliases, and respect
+lexical shadowing; a same-named ordinary function or object is outside their contract. The sleep guard covers
+Playwright `waitForTimeout` and timer-backed `new Promise` forms, while finite observer timeouts remain valid only with
+a line-level disable comment that names the timeout contract. Scratch files remain excluded by the committed E2E
+configuration; inspect the effective ESLint configuration when a helper moves between tracks.
+
+### UI and Playwright examples
+
+For a UI mutation, assert the returned state, rendered result, or persisted collaborator result after completion;
+`expect(client.update).toHaveBeenCalled()` alone only proves dispatch. For Playwright, a helper that saves an editor
+must take an explicit success or failure expectation and wait for the matching, operation-specific signal. A negative
+case must request the failure contract; a helper that always waits for success turns a valid rejection into a harness
+failure. An arbitrary toast, an existing toast from an earlier action, or a page-shell anchor is not proof that the
+save completed. Keep real browser API, cross-context, and permission flows in E2E; do not replace them with mocks just
+to avoid waiting.
+
 ## Applicability gate — read this first
 
 Not every section below applies to every change. Before designing or reviewing tests, check which of these the
@@ -246,7 +297,9 @@ before/after in one environment with the JSON-report method below.
 - Co-locate `*.test.ts`/`*.test.tsx` next to source (or place in `tests`).
 - Use `describe.concurrent()` / `it.concurrent()` where independent.
 - Single file: `pnpm test -- --run path/to/file.test.ts`.
-- Playwright tests are `*.spec.ts` files in `e2e`; they run with one worker and retain failure artifacts. Run targeted tests while iterating, then run `pnpm run lint` plus the relevant full suite before a PR.
+- Playwright tests are `*.spec.ts` files in `e2e`; worker count, retries, and artifact settings come from
+  [`playwright.config.ts`](../../playwright.config.ts) and the CI matrix. Run targeted tests while iterating, then
+  run `pnpm run lint` plus the relevant full suite before a PR.
 
 ## Vitest Performance Hygiene
 

e2e/README.md

@@ -75,7 +75,10 @@ playwright config → fixture (launchPersistentContext, loads dist/ext)
 
 [`utils.ts`](./utils.ts) carries the page openers and script installer used by every track:
 `openOptionsPage`, `openPopupPage`, `openEditorPage`, `openAgentChatPage`, `openAgentProviderPage`,
-`saveCurrentEditor`, `installScriptByCode`, `runInlineTestScript`, and `autoApprovePermissions`.
+`saveCurrentEditor`, `installScriptByCode`, `runInlineTestScript`, and `autoApprovePermissions`. Save helpers
+default to the successful outcome; a test that intentionally rejects a script must pass
+`{ saveOutcome: "failure" }` so the helper waits for the matching save failure signal rather than accepting an
+unrelated notification.
 
 ### The two-phase launch
 

e2e/gm-api.spec.ts

@@ -664,6 +664,8 @@ test.describe("GM API", () => {
       const script = document.createElement("script");
       script.textContent = `window["${key}"] = true;`;
       document.head.appendChild(script);
+      // 页面脚本执行需要一次真实事件循环让步，才能观察 CSP 阻止后的最终状态。
+      // eslint-disable-next-line scriptcat/no-test-fixed-sleep -- page event-loop observation contract
       await new Promise((resolve) => setTimeout(resolve, 0));
       return Boolean((window as Record<string, unknown>)[key]);
     });

e2e/options-pages-smoke.spec.ts

@@ -61,7 +61,8 @@ test.describe("Options 各页加载冒烟", () => {
       await expect(route.anchor(page), `${route.name} (${route.path}) 未渲染稳定锚点`).toBeVisible({
         timeout: 20_000,
       });
-      // 给页面挂载副作用(数据加载/消息往返)一点时间触发可能的异常。
+      // 页面冒烟契约包含挂载后副作用的有限观察窗口；没有统一完成事件可等待。
+      // eslint-disable-next-line scriptcat/no-test-fixed-sleep -- finite post-mount error observation window
       await page.waitForTimeout(500);
     }
 

e2e/user-config-yaml.spec.ts

@@ -99,7 +99,7 @@ test.describe("UserConfig YAML prototype pollution (#1494)", () => {
     expect(before).toBeUndefined();
 
     // 尝试安装恶意脚本（parseUserConfig 应抛错并阻止安装）
-    await installScriptByCode(context, extensionId, evilCfg);
+    await installScriptByCode(context, extensionId, evilCfg, { saveOutcome: "failure" });
 
     const list = await openOptionsPage(context, extensionId);
     const evilInfo = await getScriptInfo(list, "UC Evil E2E");

e2e/utils.ts

@@ -144,38 +144,47 @@ async function focusMonacoEditor(page: Page): Promise<void> {
   await page.locator(".monaco-editor textarea.inputarea").focus();
 }
 
-async function waitForSavedScriptInList(context: BrowserContext, extensionId: string): Promise<void> {
-  const listPage = await openOptionsPage(context, extensionId);
-  try {
-    // new-ui 列表页加载完成的稳定信号（桌面工具栏搜索框 / 移动搜索栏）
-    await listPage
-      .getByTestId("script-search")
-      .or(listPage.getByTestId("mobile-search"))
-      .first()
-      .waitFor({ state: "visible", timeout: 30_000 });
-  } finally {
-    await listPage.close();
-  }
-}
-
-export async function saveCurrentEditor(context: BrowserContext, extensionId: string, page: Page): Promise<void> {
+export type SaveOutcome = "success" | "failure";
+
+const saveSuccessMessage =
+  /Saved successfully|successfully created|保存成功|新建成功|儲存成功|保存しました|作成に成功しました|저장되었습니다|새 스크립트가 생성되었습니다|Salvo com sucesso|Novo script criado com sucesso|Успешно сохранено|Создание успешно|Erfolgreich gespeichert|Erstellung erfolgreich|Başarıyla kaydedildi|Yeni betik başarıyla oluşturuldu|Đã lưu thành công|Script mới được tạo thành công/i;
+const saveFailureMessage =
+  /Save Failed|Speichern fehlgeschlagen|保存に失敗しました|저장에 실패했습니다|Falha ao salvar|Ошибка сохранения|Kaydetme Başarısız|Lưu thất bại|保存失败|儲存失敗/i;
+
+export async function saveCurrentEditor(
+  _context: BrowserContext,
+  _extensionId: string,
+  page: Page,
+  outcome: SaveOutcome = "success"
+): Promise<void> {
   await focusMonacoEditor(page);
+  const saveToast = page
+    .locator(`[data-sonner-toast][data-type="${outcome === "success" ? "success" : "error"}"]`)
+    .filter({
+      hasText: outcome === "success" ? saveSuccessMessage : saveFailureMessage,
+    });
+  // 先关闭同类旧通知的观察窗口，避免它在保存期间自动卸载后与本次通知共用计数。
+  await expect.poll(() => saveToast.count(), { timeout: 5_000 }).toBe(0);
   await page.keyboard.press("ControlOrMeta+s");
 
-  // new-ui 保存成功为 sonner toast
-  const toastAppeared = await page
-    .locator("[data-sonner-toast]")
-    .first()
-    .waitFor({ timeout: 10_000 })
-    .then(() => true)
-    .catch(() => false);
-  if (toastAppeared) return;
-
-  await waitForSavedScriptInList(context, extensionId);
+  // 只有保存后新出现且与结果匹配的通知能证明保存完成；任意 toast 和列表页挂载都不能替代它。
+  await expect
+    .poll(() => saveToast.count(), {
+      timeout: 10_000,
+      intervals: [100, 250, 500, 1_000],
+      message:
+        outcome === "success" ? "保存操作未产生成功通知，可能被错误通知或未完成状态掩盖" : "保存操作未产生失败通知",
+    })
+    .toBeGreaterThan(0);
 }
 
 /** Install a script by injecting code into the Monaco editor and saving */
-export async function installScriptByCode(context: BrowserContext, extensionId: string, code: string): Promise<void> {
+export async function installScriptByCode(
+  context: BrowserContext,
+  extensionId: string,
+  code: string,
+  options: { saveOutcome?: SaveOutcome } = {}
+): Promise<void> {
   const page = await openEditorPage(context, extensionId);
   // Wait for Monaco editor DOM and default template content to be ready
   await focusMonacoEditor(page);
@@ -190,7 +199,7 @@ export async function installScriptByCode(context: BrowserContext, extensionId:
     timeout: 5_000,
   });
   // Save
-  await saveCurrentEditor(context, extensionId, page);
+  await saveCurrentEditor(context, extensionId, page, options.saveOutcome);
   await page.close();
 }
 

e2e/vscode-connect.spec.ts

@@ -67,6 +67,8 @@ function createMockWSServer(): Promise<{
           }
         },
         waitForAction: (action: string, timeout = 10_000) =>
+          // 有限观察窗口：WebSocket action 可能永远不回传，超时用于清理监听器并传播失败。
+          // eslint-disable-next-line scriptcat/no-test-fixed-sleep -- WebSocket action observation timeout
           new Promise<unknown>((resolveAction, rejectAction) => {
             const timer = setTimeout(() => {
               const idx = messageListeners.indexOf(handler);