Skip to content

feat(test): add case-level executor scheduling - #3148

Open
quanru wants to merge 4 commits into
mainfrom
codex/review-test-case-executor-scheduler
Open

quanru wants to merge 4 commits into
mainfrom
codex/review-test-case-executor-scheduler

Conversation

@quanru

@quanru quanru commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add case-level scheduling to Midscene Test with configurable concurrency, resource locks, retries, and deterministic case selection
  • introduce a generic Test Executor API for delegating isolated cases to local processes, containers, device services, or remote runners without coupling the framework to a specific infrastructure provider
  • add CLI selection by file, directory, explicit case ID, and tag
  • persist per-case execution metadata and artifacts in the unified test result and HTML report
  • expose aiWaitFor as a built-in Midscene Test node
  • document the new scheduling and extension APIs in English and Chinese

Reliability and security

  • use a strict JSON-safe Executor DTO and validate the complete result and status invariants before hydration
  • prevent remote responses from supplying local report paths; reports must be materialized into a task-scoped directory and referenced by opaque IDs
  • isolate project-mode and case-mode orchestration while sharing result and progress helpers
  • reject unsafe case IDs and attempt IDs before they can be used as result paths
  • release resource locks correctly when an Executor throws synchronously
  • preserve project-mode behavior as the default for backward compatibility
  • document generated IDs as deterministic rather than stable; persistent selectors require explicit IDs

Report support

  • include Executor name, resources, attempts, lifecycle, failures, artifacts, and metadata in the HTML manifest
  • render Executor details, artifacts, and metadata in the Test report UI

Validation

  • pnpm run type-check:tests
  • pnpm exec nx test @midscene/test (343 passed)
  • pnpm exec nx build @midscene/test
  • pnpm exec nx test @midscene/report (148 passed)
  • pnpm exec nx build @midscene/report

Pre-release

Published as 1.12.7-beta-20260916022808.0 under the npm beta tag. A refreshed pre-release will be published after review fixes are accepted.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 16, 2026

Copy link
Copy Markdown

Deploying midscene with  Cloudflare Pages  Cloudflare Pages

Latest commit: bc3537a
Status: ✅  Deploy successful!
Preview URL: https://cbff5d84.midscene.pages.dev
Branch Preview URL: https://codex-review-test-case-execu.midscene.pages.dev

View logs

@quanru quanru changed the title fix(test): harden case executor scheduling feat(test): add case-level executor scheduling Sep 16, 2026
@quanru
quanru changed the base branch from codex/test-case-executor-scheduler to main September 16, 2026 02:52
@quanru

quanru commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the four review findings in bc3537a:

  1. Replaced the Executor/internal-runtime boundary with a strict JSON-safe DTO, full Zod validation, status/attempt invariants, and task-scoped opaque report references. Remote data can no longer supply local reportPaths.
  2. Added Executor/resources/attempts/failure/lifecycle/artifacts/metadata to the HTML report contract, manifest builder, report UI, and tests.
  3. Split project-mode orchestration into project-execution.ts and shared result/progress helpers in execution-result.ts; the main runner is now 597 lines instead of 929.
  4. Corrected English and Chinese docs: generated Case IDs are deterministic but not stable across insertion/reordering; persistent selectors must use explicit IDs.

Validation: pnpm run type-check:tests; 343 @midscene/test tests; @midscene/test build; 148 @midscene/report tests; @midscene/report build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant