Summary
FlowSpec's step/assertion grammar (src/types.ts: actions visit | click | fill | select | wait_for, assertions url | visible | matches | not_visible) can't express several real, common bug classes. Hit this today while using FlowSpec as the QA driver for an A(i)-Team mission — two real bugs escaped the whole test pipeline and were only caught by a human-style manual browser walk, and neither could be turned into a regression spec afterward because the grammar has no way to say what actually needs asserting.
Gaps found
-
No key-press step. There's click, but no way to simulate pressing a specific key (Enter, Escape, Tab, etc.) on a focused element. Blocked writing specs for: submit-on-Enter in a text input, abandon-edit-on-Escape. Any keyboard-only interaction is currently unspecifiable.
-
No color/pixel (visual) assertion. visible/not_visible/matches all operate on DOM presence/text — none can assert on computed style or rendered appearance. This mattered concretely: a checkbox rendered browser-default blue/gray pixels (native UA widget styling) despite every CSS computed-style check reporting a clean design-system-compliant black-and-white result. getComputedStyle doesn't see UA widget rendering; only an actual pixel/screenshot-level check does. No way to pin that as a regression spec today.
-
No process-control step. Can't stop/start/restart a dependent process (e.g. the backing API) mid-flow. This blocked specifying an "app recovers correctly after the API comes back up" scenario — a real state-management bug (hasLoaded never reset after failure) was only reproducible by killing and restarting the API server by hand.
-
No checked (or general form-control-state) predicate. Toggling a checkbox can be driven with click, but there's no assertion for its resulting checked state — only text-based assertions exist, so toggle correctness can't actually be verified, only clicked.
Impact
Because these can't be expressed, the right call when a fix lands for a bug in one of these classes is to not graduate a spec for it (writing a spec that can't actually fail is worse than an acknowledged gap) — which means FlowSpec currently can't build a regression net against exactly the bug classes that are hardest to catch by other means (CSS-computed-style checks + unit tests both miss all four of the above).
Possibly useful additions
press: <key> step (or press: <key> as a modifier alongside fill)
- A
computed_style or pixel_match assertion (even a coarse one — e.g. assert a named CSS custom property or computed color value at a selector)
- A
run: <command> / stop_process / start_process step pair, or at minimum a documented pattern for driving process lifecycle from a flow
- A
checked (and maybe generic state: <attr>=<value>) assertion for form controls
Happy to share the full evidence bundle (screenshots + repro steps) from the mission that surfaced this if useful — everything above was found by an agent (Frankie, from the A(i)-Team plugin) doing a first-time-user browser walk against a React todo app, not synthetically constructed.
Summary
FlowSpec's step/assertion grammar (
src/types.ts: actionsvisit | click | fill | select | wait_for, assertionsurl | visible | matches | not_visible) can't express several real, common bug classes. Hit this today while using FlowSpec as the QA driver for an A(i)-Team mission — two real bugs escaped the whole test pipeline and were only caught by a human-style manual browser walk, and neither could be turned into a regression spec afterward because the grammar has no way to say what actually needs asserting.Gaps found
No key-press step. There's
click, but no way to simulate pressing a specific key (Enter, Escape, Tab, etc.) on a focused element. Blocked writing specs for: submit-on-Enter in a text input, abandon-edit-on-Escape. Any keyboard-only interaction is currently unspecifiable.No color/pixel (visual) assertion.
visible/not_visible/matchesall operate on DOM presence/text — none can assert on computed style or rendered appearance. This mattered concretely: a checkbox rendered browser-default blue/gray pixels (native UA widget styling) despite every CSS computed-style check reporting a clean design-system-compliant black-and-white result.getComputedStyledoesn't see UA widget rendering; only an actual pixel/screenshot-level check does. No way to pin that as a regression spec today.No process-control step. Can't stop/start/restart a dependent process (e.g. the backing API) mid-flow. This blocked specifying an "app recovers correctly after the API comes back up" scenario — a real state-management bug (
hasLoadednever reset after failure) was only reproducible by killing and restarting the API server by hand.No
checked(or general form-control-state) predicate. Toggling a checkbox can be driven withclick, but there's no assertion for its resultingcheckedstate — only text-based assertions exist, so toggle correctness can't actually be verified, only clicked.Impact
Because these can't be expressed, the right call when a fix lands for a bug in one of these classes is to not graduate a spec for it (writing a spec that can't actually fail is worse than an acknowledged gap) — which means FlowSpec currently can't build a regression net against exactly the bug classes that are hardest to catch by other means (CSS-computed-style checks + unit tests both miss all four of the above).
Possibly useful additions
press: <key>step (orpress: <key>as a modifier alongsidefill)computed_styleorpixel_matchassertion (even a coarse one — e.g. assert a named CSS custom property or computed color value at a selector)run: <command>/stop_process/start_processstep pair, or at minimum a documented pattern for driving process lifecycle from a flowchecked(and maybe genericstate: <attr>=<value>) assertion for form controlsHappy to share the full evidence bundle (screenshots + repro steps) from the mission that surfaced this if useful — everything above was found by an agent (Frankie, from the A(i)-Team plugin) doing a first-time-user browser walk against a React todo app, not synthetically constructed.