Skip to content

ci: add native UI accessibility end-to-end gates - #193

Open
turinglambdaai wants to merge 15 commits into
mainfrom
test/native-ui-e2e
Open

turinglambdaai wants to merge 15 commits into
mainfrom
test/native-ui-e2e

Conversation

@turinglambdaai

Copy link
Copy Markdown
Owner

Summary

  • drive the packaged Taskboard through UI Automation, AXUIElement, and AT-SPI on the three native desktop stacks
  • assert stable semantics, one RPC/State mutation, one Event, the final large State, and orderly close
  • upload bounded accessibility snapshots, interaction traces, screenshots, and native logs even when a gate fails
  • document the product-level native test extension point without adding a shared UI DSL

Closes #184 after all three platform jobs are green.

Local validation

  • Python AST parse and bash syntax check
  • PowerShell parser validation
  • workflow parsed with PyYAML
  • raco test tests/documentation.rkt: 153 tests passed
  • git diff --check

@turinglambdaai

Copy link
Copy Markdown
Owner Author

Pushed e14f6fe to address both native UI failures. Linux now explicitly enables the GTK AT-SPI backend and always captures the discovered accessibility tree. Windows now virtualizes the 1,000-row list, assigns stable AutomationIds to realized containers, closes through the native WM_CLOSE path, stops the embedded backend deterministically, and releases the last WinUI Window without forcing Application::Exit during Closed teardown. Local full UIA evidence: all interactions and assertions passed; clean process exit in 117 ms. Also verified python -m py_compile and bash -n.

@turinglambdaai

Copy link
Copy Markdown
Owner Author

Pushed the root-cause fix in the native harness. Windows evidence showed Process.CloseMainWindow was selecting the lazily allocated diagnostic console, leaving the real WinUI window open; the gate now posts WM_CLOSE to the exact HWND exposed by the UIA root and releases provider proxies (local full 1,000-row run exits cleanly in 106 ms). Linux evidence showed GTK/AT-SPI exposes visible control names and the live status text rather than the requested GtkAccessible label property on Ubuntu 24.04, so the gate now asserts the actual assistive-technology tree and tracks the live status object. The uploaded failing trees were used directly for both corrections.

@turinglambdaai

Copy link
Copy Markdown
Owner Author

The latest native-E2E evidence exposed three platform-specific lifecycle details, now fixed in 40fa499: GTK 4.14 publishes actionable controls through AT-SPI SENSITIVE without the optional ENABLED bit; Swift 6 rejects conditional casts to Core Foundation accessibility types; and WinUI desktop applications own their dispatcher independently of their final window, so the reference app now ends the application lifecycle explicitly after its earlier-registered backend shutdown handler completes. The exact-HWND UIA path and the 1,000-row interaction assertions remain unchanged.

@turinglambdaai

Copy link
Copy Markdown
Owner Author

Rebased by merge onto current main after #198 and #199. Local Windows verification on the merged tree passed end to end: clean package, UI Automation RPC/Event/1,000-row State assertions, native close, backend shutdown, and process exit. The test remains strict; #199 removes the inherited diagnostics console that kept the process alive in CI. Also re-ran documentation and pending/diagnostics tests: 211 passed.

@turinglambdaai

Copy link
Copy Markdown
Owner Author

The macOS artifact proved all AX interactions and the final 1,000-row State succeeded. The remaining failure was lifecycle-specific: SwiftUI WindowGroup kept the application resident after its last window closed, and the failure cleanup then waited without a bound after SIGTERM.

This update makes the product behavior explicit: the AppKit delegate terminates after the last window closes, the root view stops the embedded backend on disappearance, and CI cleanup escalates from TERM to KILL within ten seconds so a future lifecycle regression produces a prompt failure plus artifacts instead of a 25-minute cancellation. The successful path now emits shutdown-trace.json. Current main (#202) is merged into the branch; YAML, diff checks, and 211 documentation/backend tests pass locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Native UI end-to-end, accessibility, and failure-artifact gates

1 participant