Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions bundles/rf-agentskills/skills/rf-browser/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ The browser and context opened in Suite Setup live until the suite ends; each te
| Popup, new tab, several pages | `Switch Page NEW`, `Get Page Ids`, `Close Page` | `references/browser-context-page.md` |
| Log in once, reuse the session | `Save Storage State`, then `New Context storageState=${state}` | `references/authentication-storage.md` |
| Upload or download a file | `Upload File By Selector`; `Promise To Wait For Download` + `Wait For` | `references/downloads-uploads.md` |
| Selector stopped matching after a UI change, or a test passes only some runs | role and accessible name first; measure before any rerun | `references/selector-drift.md` |
| Run JavaScript on the page | `Evaluate JavaScript ${None} () => document.title` | — |

## Agent workflow
Expand Down Expand Up @@ -118,6 +119,7 @@ Prefer, in order: test ids (`[data-testid="save"]`), accessible roles (`role=but
| `references/authentication-storage.md` | reusing a login, HTTP auth, cookies or local/session storage |
| `references/downloads-uploads.md` | a test uploads or downloads files |
| `references/troubleshooting.md` | a run fails with an error message you need to decode |
| `references/selector-drift.md` | a selector stopped matching after a UI change, matches several elements, or a test passes only some runs |

Examples in `assets/examples/`: `basic-web-test.robot`, `form-handling.robot`, `authentication-flow.robot`, `iframe-shadow-dom.robot`.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Selector drift and flaky UI tests

Use this when a UI test fails because an element is not found, is hidden or has moved, or passes on some runs and fails on others.

## Decide what changed

1. Read the failing step. A locator that matches nothing points at the selector or the page. A timeout on an element that is visible in the log points at timing.
2. Measure before any rerun. Read the element's position and size with `Get BoundingBox` twice, a short time apart, and check whether an animation runs on it.
3. Rerun only after you know what changed. Running unchanged code again only shows that the result is unstable.
4. Decide who owns the cause:
- the test: the selector is too weak, or the test does not wait for the state it needs;
- the app: the element is missing, has moved or shows the wrong content.

Only the app case becomes a defect report.

## Pick a locator that survives a redesign

- Prefer, in order: test ids (`[data-testid="save"]`), accessible roles (`role=button[name="Save"]`), short CSS, text (`text=Save`), XPath last. This is the order in `SKILL.md`.
- Check the match count with an operator in the getter, for example `Get Element Count role=button[name="Save"] == 1`.
- Avoid long CSS paths and `nth-child` chains. They break when the markup changes.
- The full order of locator strategies is in the locators reference listed in `SKILL.md`.

## Elements outside the viewport and menus

- A design-library menu can close when the page scrolls. Scroll the trigger into view first with `Scroll To Element`, then open the menu and click the option with no scroll in between.

## New and flaky tests

- To decide whether one failing test is flaky or a real defect, rerun that test three times with no code change. A test that passes on some runs and fails on others is flaky; one that fails every run points at the selector or the app.
- Quarantine a flaky test with a tag and a tracked issue. Do not add retries or longer timeouts to hide the cause.
2 changes: 2 additions & 0 deletions plugins/rf-web/skills/rf-browser/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ The browser and context opened in Suite Setup live until the suite ends; each te
| Popup, new tab, several pages | `Switch Page NEW`, `Get Page Ids`, `Close Page` | `references/browser-context-page.md` |
| Log in once, reuse the session | `Save Storage State`, then `New Context storageState=${state}` | `references/authentication-storage.md` |
| Upload or download a file | `Upload File By Selector`; `Promise To Wait For Download` + `Wait For` | `references/downloads-uploads.md` |
| Selector stopped matching after a UI change, or a test passes only some runs | role and accessible name first; measure before any rerun | `references/selector-drift.md` |
| Run JavaScript on the page | `Evaluate JavaScript ${None} () => document.title` | — |

## Agent workflow
Expand Down Expand Up @@ -118,6 +119,7 @@ Prefer, in order: test ids (`[data-testid="save"]`), accessible roles (`role=but
| `references/authentication-storage.md` | reusing a login, HTTP auth, cookies or local/session storage |
| `references/downloads-uploads.md` | a test uploads or downloads files |
| `references/troubleshooting.md` | a run fails with an error message you need to decode |
| `references/selector-drift.md` | a selector stopped matching after a UI change, matches several elements, or a test passes only some runs |

Examples in `assets/examples/`: `basic-web-test.robot`, `form-handling.robot`, `authentication-flow.robot`, `iframe-shadow-dom.robot`.

Expand Down
30 changes: 30 additions & 0 deletions plugins/rf-web/skills/rf-browser/references/selector-drift.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Selector drift and flaky UI tests

Use this when a UI test fails because an element is not found, is hidden or has moved, or passes on some runs and fails on others.

## Decide what changed

1. Read the failing step. A locator that matches nothing points at the selector or the page. A timeout on an element that is visible in the log points at timing.
2. Measure before any rerun. Read the element's position and size with `Get BoundingBox` twice, a short time apart, and check whether an animation runs on it.
3. Rerun only after you know what changed. Running unchanged code again only shows that the result is unstable.
4. Decide who owns the cause:
- the test: the selector is too weak, or the test does not wait for the state it needs;
- the app: the element is missing, has moved or shows the wrong content.

Only the app case becomes a defect report.

## Pick a locator that survives a redesign

- Prefer, in order: test ids (`[data-testid="save"]`), accessible roles (`role=button[name="Save"]`), short CSS, text (`text=Save`), XPath last. This is the order in `SKILL.md`.
- Check the match count with an operator in the getter, for example `Get Element Count role=button[name="Save"] == 1`.
- Avoid long CSS paths and `nth-child` chains. They break when the markup changes.
- The full order of locator strategies is in the locators reference listed in `SKILL.md`.

## Elements outside the viewport and menus

- A design-library menu can close when the page scrolls. Scroll the trigger into view first with `Scroll To Element`, then open the menu and click the option with no scroll in between.

## New and flaky tests

- To decide whether one failing test is flaky or a real defect, rerun that test three times with no code change. A test that passes on some runs and fails on others is flaky; one that fails every run points at the selector or the app.
- Quarantine a flaky test with a tag and a tracked issue. Do not add retries or longer timeouts to hide the cause.
Loading