diff --git a/bundles/rf-agentskills/skills/rf-browser/SKILL.md b/bundles/rf-agentskills/skills/rf-browser/SKILL.md index f945ee4..19254c5 100644 --- a/bundles/rf-agentskills/skills/rf-browser/SKILL.md +++ b/bundles/rf-agentskills/skills/rf-browser/SKILL.md @@ -77,6 +77,7 @@ The browser and context opened in Suite Setup live until the suite ends; each te | Popup, new tab, several pages | `Switch Page NEW`, `Get Page Ids`, `Close Page` | `references/browser-context-page.md` | | Log in once, reuse the session | `Save Storage State`, then `New Context storageState=${state}` | `references/authentication-storage.md` | | Upload or download a file | `Upload File By Selector`; `Promise To Wait For Download` + `Wait For` | `references/downloads-uploads.md` | +| Selector stopped matching after a UI change, or a test passes only some runs | role and accessible name first; measure before any rerun | `references/selector-drift.md` | | Run JavaScript on the page | `Evaluate JavaScript ${None} () => document.title` | — | ## Agent workflow @@ -118,6 +119,7 @@ Prefer, in order: test ids (`[data-testid="save"]`), accessible roles (`role=but | `references/authentication-storage.md` | reusing a login, HTTP auth, cookies or local/session storage | | `references/downloads-uploads.md` | a test uploads or downloads files | | `references/troubleshooting.md` | a run fails with an error message you need to decode | +| `references/selector-drift.md` | a selector stopped matching after a UI change, matches several elements, or a test passes only some runs | Examples in `assets/examples/`: `basic-web-test.robot`, `form-handling.robot`, `authentication-flow.robot`, `iframe-shadow-dom.robot`. diff --git a/bundles/rf-agentskills/skills/rf-browser/references/selector-drift.md b/bundles/rf-agentskills/skills/rf-browser/references/selector-drift.md new file mode 100644 index 0000000..e3867cf --- /dev/null +++ b/bundles/rf-agentskills/skills/rf-browser/references/selector-drift.md @@ -0,0 +1,30 @@ +# Selector drift and flaky UI tests + +Use this when a UI test fails because an element is not found, is hidden or has moved, or passes on some runs and fails on others. + +## Decide what changed + +1. Read the failing step. A locator that matches nothing points at the selector or the page. A timeout on an element that is visible in the log points at timing. +2. Measure before any rerun. Read the element's position and size with `Get BoundingBox` twice, a short time apart, and check whether an animation runs on it. +3. Rerun only after you know what changed. Running unchanged code again only shows that the result is unstable. +4. Decide who owns the cause: + - the test: the selector is too weak, or the test does not wait for the state it needs; + - the app: the element is missing, has moved or shows the wrong content. + + Only the app case becomes a defect report. + +## Pick a locator that survives a redesign + +- Prefer, in order: test ids (`[data-testid="save"]`), accessible roles (`role=button[name="Save"]`), short CSS, text (`text=Save`), XPath last. This is the order in `SKILL.md`. +- Check the match count with an operator in the getter, for example `Get Element Count role=button[name="Save"] == 1`. +- Avoid long CSS paths and `nth-child` chains. They break when the markup changes. +- The full order of locator strategies is in the locators reference listed in `SKILL.md`. + +## Elements outside the viewport and menus + +- A design-library menu can close when the page scrolls. Scroll the trigger into view first with `Scroll To Element`, then open the menu and click the option with no scroll in between. + +## New and flaky tests + +- To decide whether one failing test is flaky or a real defect, rerun that test three times with no code change. A test that passes on some runs and fails on others is flaky; one that fails every run points at the selector or the app. +- Quarantine a flaky test with a tag and a tracked issue. Do not add retries or longer timeouts to hide the cause. diff --git a/plugins/rf-web/skills/rf-browser/SKILL.md b/plugins/rf-web/skills/rf-browser/SKILL.md index f945ee4..19254c5 100644 --- a/plugins/rf-web/skills/rf-browser/SKILL.md +++ b/plugins/rf-web/skills/rf-browser/SKILL.md @@ -77,6 +77,7 @@ The browser and context opened in Suite Setup live until the suite ends; each te | Popup, new tab, several pages | `Switch Page NEW`, `Get Page Ids`, `Close Page` | `references/browser-context-page.md` | | Log in once, reuse the session | `Save Storage State`, then `New Context storageState=${state}` | `references/authentication-storage.md` | | Upload or download a file | `Upload File By Selector`; `Promise To Wait For Download` + `Wait For` | `references/downloads-uploads.md` | +| Selector stopped matching after a UI change, or a test passes only some runs | role and accessible name first; measure before any rerun | `references/selector-drift.md` | | Run JavaScript on the page | `Evaluate JavaScript ${None} () => document.title` | — | ## Agent workflow @@ -118,6 +119,7 @@ Prefer, in order: test ids (`[data-testid="save"]`), accessible roles (`role=but | `references/authentication-storage.md` | reusing a login, HTTP auth, cookies or local/session storage | | `references/downloads-uploads.md` | a test uploads or downloads files | | `references/troubleshooting.md` | a run fails with an error message you need to decode | +| `references/selector-drift.md` | a selector stopped matching after a UI change, matches several elements, or a test passes only some runs | Examples in `assets/examples/`: `basic-web-test.robot`, `form-handling.robot`, `authentication-flow.robot`, `iframe-shadow-dom.robot`. diff --git a/plugins/rf-web/skills/rf-browser/references/selector-drift.md b/plugins/rf-web/skills/rf-browser/references/selector-drift.md new file mode 100644 index 0000000..e3867cf --- /dev/null +++ b/plugins/rf-web/skills/rf-browser/references/selector-drift.md @@ -0,0 +1,30 @@ +# Selector drift and flaky UI tests + +Use this when a UI test fails because an element is not found, is hidden or has moved, or passes on some runs and fails on others. + +## Decide what changed + +1. Read the failing step. A locator that matches nothing points at the selector or the page. A timeout on an element that is visible in the log points at timing. +2. Measure before any rerun. Read the element's position and size with `Get BoundingBox` twice, a short time apart, and check whether an animation runs on it. +3. Rerun only after you know what changed. Running unchanged code again only shows that the result is unstable. +4. Decide who owns the cause: + - the test: the selector is too weak, or the test does not wait for the state it needs; + - the app: the element is missing, has moved or shows the wrong content. + + Only the app case becomes a defect report. + +## Pick a locator that survives a redesign + +- Prefer, in order: test ids (`[data-testid="save"]`), accessible roles (`role=button[name="Save"]`), short CSS, text (`text=Save`), XPath last. This is the order in `SKILL.md`. +- Check the match count with an operator in the getter, for example `Get Element Count role=button[name="Save"] == 1`. +- Avoid long CSS paths and `nth-child` chains. They break when the markup changes. +- The full order of locator strategies is in the locators reference listed in `SKILL.md`. + +## Elements outside the viewport and menus + +- A design-library menu can close when the page scrolls. Scroll the trigger into view first with `Scroll To Element`, then open the menu and click the option with no scroll in between. + +## New and flaky tests + +- To decide whether one failing test is flaky or a real defect, rerun that test three times with no code change. A test that passes on some runs and fails on others is flaky; one that fails every run points at the selector or the app. +- Quarantine a flaky test with a tag and a tracked issue. Do not add retries or longer timeouts to hide the cause.