feat(failure-intel): group failures by root cause in the report

CHANGELOG.md

@@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ### Unreleased
 
+- **Failure grouping** — the report's Failures tab now opens with a "N tests, same cause" panel: failed tests are grouped by a signature of category, exception type, normalised message (ids, numbers and quoted values stripped) and top application stack frame. The flat per-class list is unchanged below it. With `ai.failureAnalysis` on, one Claude call is made per group; the other tests in the group reuse its analysis. The metrics JSON gains `failureSignature`. (#140)
+
 - **Failure classification** — every failed test is tagged with a category (`LOCATOR_NOT_FOUND`, `ELEMENT_NOT_INTERACTABLE`, `STALE_ELEMENT`, `TIMEOUT`, `ASSERTION_MISMATCH`, `NAVIGATION_ERROR`, `BROWSER_CRASH`, or `UNKNOWN`) and the evidence for it, from the exception class chain and message alone. No AI, no API key, no network. Shown as a badge in the HTML report's failure panel, as `failureCategory` / `failureEvidence` in the metrics JSON, and as the `type` of `<failure>` in the JUnit XML. The raw error and stack trace are unchanged. (#139)
 
 ---

docs-site/docs/changelog.md

@@ -11,6 +11,8 @@ All notable changes to Selenium Boot are documented here.
 
 ## [Unreleased]
 
+- **Failure grouping** — the report's Failures tab now opens with a "N tests, same cause" panel: failed tests are grouped by a signature of category, exception type, normalised message (ids, numbers and quoted values stripped) and top application stack frame. The flat per-class list is unchanged below it. With `ai.failureAnalysis` on, one Claude call is made per group; the other tests in the group reuse its analysis. The metrics JSON gains `failureSignature`. (#140)
+
 - **`Locator` form actions: `clear()`, `check()`, `uncheck()`, `isChecked()`, `selectOption(text)`, `selectOptionByValue(value)`, `inputValue()`** — form controls can now be manipulated and inspected directly on `Locator` without falling back to `BasePage` helpers or raw `WebElement`s. `check()` and `uncheck()` are idempotent (no click if already in the target state), `isChecked()` queries checked state without waiting, and terminal actions (`clear`, `check`, `uncheck`, `selectOption`, `selectOptionByValue`, `inputValue`) auto-wait, re-resolving from scratch on every poll to survive stale or late-appearing elements. (#133)
 
 ---

docs-site/docs/reporting/html-report.md

@@ -78,6 +78,12 @@ The category is also written to the metrics JSON as `failureCategory` and `failu
 
 ---
 
+### Same-cause groups
+
+When two or more failures share a cause, the Failures tab lists them above the flat list as "N tests, same cause", largest group first, so one outage does not read as forty problems. Tests are grouped by a signature made of the failure category, the exception type, the first line of the message with ids, numbers and quoted values replaced by placeholders, and the top application stack frame (class and method, not line number). The flat list below still shows every failure. With `ai.failureAnalysis` on, one API call is made per group and the other tests in it reuse that analysis.
+
+---
+
 ## Self-contained format
 
 All screenshots are Base64-encoded and embedded inline. The report is a single file you can:

src/main/java/com/seleniumboot/ai/AiFailureAnalyzer.java

@@ -13,6 +13,9 @@
 import java.net.http.HttpResponse;
 import java.time.Duration;
 import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
 import java.util.logging.Logger;
 import java.util.stream.Collectors;
 
@@ -46,8 +49,17 @@ public final class AiFailureAnalyzer {
     private static final String API_URL = "https://api.anthropic.com/v1/messages";
     private static final String API_VERSION = "2023-06-01";
 
+    /** Analysis per failure signature, so one cause costs one API call. */
+    private static final ConcurrentHashMap<String, CompletableFuture<String>> BY_SIGNATURE =
+            new ConcurrentHashMap<>();
+
     private AiFailureAnalyzer() {}
 
+    /** Test hook: forget cached analyses. */
+    static void resetCache() {
+        BY_SIGNATURE.clear();
+    }
+
     // ------------------------------------------------------------------
     // Public API — called by TestExecutionListener
     // ------------------------------------------------------------------
@@ -74,8 +86,26 @@ public static void analyze(String testId, String pageUrl, String pageTitle) {
             TestTiming timing = ExecutionMetrics.getTiming(testId);
             if (timing == null) return;
 
-            String prompt = buildPrompt(timing, pageUrl, pageTitle);
-            String analysis = callApi(apiKey, aiCfg.getModel(), prompt, aiCfg.getTimeoutSeconds());
+            // One call per distinct cause: tests failing for the same reason share the analysis of
+            // whichever failed first. Concurrent same-cause failures wait for that one call.
+            String signature = com.seleniumboot.failure.FailureSignature.of(timing);
+            CompletableFuture<String> mine = new CompletableFuture<>();
+            CompletableFuture<String> existing = BY_SIGNATURE.putIfAbsent(signature, mine);
+            String analysis;
+            if (existing != null) {
+                analysis = existing.get(aiCfg.getTimeoutSeconds() + 5L, TimeUnit.SECONDS);
+                if (analysis != null) LOG.info("[AiFailureAnalyzer] Reusing analysis for same cause: " + testId);
+            } else {
+                String result = null;
+                try {
+                    result = callApi(apiKey, aiCfg.getModel(), buildPrompt(timing, pageUrl, pageTitle),
+                            aiCfg.getTimeoutSeconds());
+                } finally {
+                    if (result == null || result.isBlank()) BY_SIGNATURE.remove(signature, mine);
+                    mine.complete(result);
+                }
+                analysis = result;
+            }
             if (analysis != null && !analysis.isBlank()) {
                 ExecutionMetrics.recordAiAnalysis(testId, analysis.strip());
                 LOG.info("[AiFailureAnalyzer] Analysis recorded for: " + testId);

src/main/java/com/seleniumboot/failure/FailureClassifier.java

@@ -91,6 +91,10 @@ public static Result classify(String errorMessage, String stackTrace) {
         return unknown(outer != null ? "unrecognised " + simple(outer) : "unrecognised error");
     }
 
+    static List<String> causeChainOf(String stackTrace) {
+        return causeChain(stackTrace);
+    }
+
     /** Fully-qualified class names of the outer throwable followed by each cause, in order. */
     static List<String> causeChain(String stackTrace) {
         List<String> chain = new ArrayList<>();

src/main/java/com/seleniumboot/failure/FailureSignature.java

@@ -0,0 +1,78 @@
+package com.seleniumboot.failure;
+
+import com.seleniumboot.metrics.TestTiming;
+
+import java.util.List;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * A stable fingerprint of <em>why</em> a test failed, so that many failures with one cause
+ * collapse into one group.
+ *
+ * <p>The signature joins the {@link FailureCategory}, the outer exception type, the first line of
+ * the message with volatile parts (UUIDs, hex ids, numbers, quoted values) replaced by
+ * placeholders, and the top application stack frame (class and method, no line number).
+ * Two tests share a signature when all four match.
+ */
+public final class FailureSignature {
+
+    private static final Pattern UUID = Pattern.compile(
+            "\\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\\b");
+    private static final Pattern HEX = Pattern.compile("\\b(?:0x)?[0-9a-fA-F]{12,}\\b");
+    private static final Pattern QUOTED = Pattern.compile("'[^']*'|\"[^\"]*\"");
+    private static final Pattern NUMBER = Pattern.compile("\\d+");
+    private static final Pattern SPACES = Pattern.compile("\\s+");
+    private static final Pattern FRAME = Pattern.compile("^\\s+at ([^(\\s]+)\\(");
+
+    private static final List<String> FRAMEWORK_PREFIXES = List.of(
+            "java.", "javax.", "jdk.", "sun.", "org.openqa.", "org.testng.", "org.junit.",
+            "io.cucumber.", "com.seleniumboot.");
+
+    private static final int MESSAGE_MAX = 160;
+
+    private FailureSignature() { }
+
+    public static String of(TestTiming timing) {
+        return timing == null ? of(null, null) : of(timing.getErrorMessage(), timing.getStackTrace());
+    }
+
+    public static String of(String errorMessage, String stackTrace) {
+        FailureCategory category = FailureClassifier.classify(errorMessage, stackTrace).category();
+        List<String> chain = FailureClassifier.causeChainOf(stackTrace);
+        String type = chain.isEmpty() ? "?" : chain.get(0);
+        return category.name() + " | " + type + " | " + normalise(errorMessage) + " | " + topAppFrame(stackTrace);
+    }
+
+    /** First message line, with ids, numbers and quoted values replaced so recurring failures match. */
+    static String normalise(String message) {
+        if (message == null) return "";
+        String first = message.strip();
+        int nl = first.indexOf('\n');
+        if (nl >= 0) first = first.substring(0, nl);
+        first = UUID.matcher(first).replaceAll("<id>");
+        first = HEX.matcher(first).replaceAll("<id>");
+        first = QUOTED.matcher(first).replaceAll("<v>");
+        first = NUMBER.matcher(first).replaceAll("#");
+        first = SPACES.matcher(first).replaceAll(" ").strip();
+        return first.length() > MESSAGE_MAX ? first.substring(0, MESSAGE_MAX) : first;
+    }
+
+    /**
+     * Class and method of the first stack frame outside the JDK, Selenium, the test runner and the
+     * framework: the page object or test code that was running. Falls back to the first frame.
+     */
+    static String topAppFrame(String stackTrace) {
+        if (stackTrace == null) return "";
+        String firstFrame = null;
+        for (String line : stackTrace.split("\\R")) {
+            if (line.startsWith("Caused by: ")) break;
+            Matcher m = FRAME.matcher(line);
+            if (!m.find()) continue;
+            String frame = m.group(1);
+            if (firstFrame == null) firstFrame = frame;
+            if (FRAMEWORK_PREFIXES.stream().noneMatch(frame::startsWith)) return frame;
+        }
+        return firstFrame != null ? firstFrame : "";
+    }
+}

src/main/java/com/seleniumboot/metrics/ExecutionMetrics.java

@@ -353,6 +353,7 @@ public static void exportToJson() {
                         com.seleniumboot.failure.FailureClassifier.classify(timing);
                 testEntry.put("failureCategory", failure.category().name());
                 testEntry.put("failureEvidence", failure.evidence());
+                testEntry.put("failureSignature", com.seleniumboot.failure.FailureSignature.of(timing));
             }
             if (timing.getBrowser() != null) {
                 testEntry.put("browser", timing.getBrowser());

src/main/java/com/seleniumboot/reporting/HtmlReportGenerator.java

@@ -234,6 +234,7 @@ static String buildHtml(JsonNode root) {
         String slowestSection   = tests != null ? buildSlowestTests(tests) : "";
         String flakinessSection = buildFlakinessSection();
         String failureRows    = buildFailureRows(tests, isMatrixRun);
+        String failureGroups  = buildFailureGroups(tests);
         String failureBadge   = failedTests > 0
                 ? "<span class=\"nav-count nav-count-fail\">" + failedTests + "</span>"
                 : "";
@@ -265,6 +266,7 @@ static String buildHtml(JsonNode root) {
                 .replace("{{DONUT_DATA}}", donutData)
                 .replace("{{ROWS}}", rows.toString())
                 .replace("{{FAILURE_ROWS}}", failureRows)
+                .replace("{{FAILURE_GROUPS}}", failureGroups)
                 .replace("{{FAILURE_BADGE}}", failureBadge)
                 .replace("{{FAILURES_SHORTCUT}}", failuresShortcut)
                 .replace("{{EXECUTION_PERCENTILES}}", executionPercentiles.replace("'", "\\'"));
@@ -436,6 +438,55 @@ private static String buildFailureRows(JsonNode tests, boolean showBrowser) {
         return rows.toString();
     }
 
+    /**
+     * "N tests, same cause" panel shown above the flat failure list. Only causes shared by two or
+     * more tests are listed, largest first; the flat list below still shows every failure.
+     */
+    private static String buildFailureGroups(JsonNode tests) {
+        if (tests == null) return "";
+        java.util.Map<String, java.util.List<JsonNode>> bySignature = new java.util.LinkedHashMap<>();
+        for (JsonNode test : tests) {
+            if (!"FAILED".equals(test.has("status") ? test.get("status").asText() : "")) continue;
+            String signature = test.has("failureSignature")
+                    ? test.get("failureSignature").asText()
+                    : com.seleniumboot.failure.FailureSignature.of(
+                            test.has("errorMessage") ? test.get("errorMessage").asText() : null,
+                            test.has("stackTrace") ? test.get("stackTrace").asText() : null);
+            bySignature.computeIfAbsent(signature, k -> new java.util.ArrayList<>()).add(test);
+        }
+        java.util.List<java.util.List<JsonNode>> groups = new java.util.ArrayList<>();
+        for (java.util.List<JsonNode> g : bySignature.values()) {
+            if (g.size() >= 2) groups.add(g);
+        }
+        if (groups.isEmpty()) return "";
+        groups.sort((a, b) -> Integer.compare(b.size(), a.size()));
+
+        StringBuilder sb = new StringBuilder("<div class=\"card section-mb failure-groups\">")
+                .append("<div class=\"card-header\">Same cause (")
+                .append(groups.size()).append(groups.size() == 1 ? " group" : " groups").append(")</div>")
+                .append("<div class=\"card-body\">");
+        for (java.util.List<JsonNode> g : groups) {
+            JsonNode first = g.get(0);
+            sb.append("<div class=\"failure-group\"><div class=\"failure-group-title\">");
+            if (first.has("failureCategory")) {
+                sb.append(buildFailureWhy(first)).append("");
+            }
+            sb.append("<strong>").append(g.size()).append(" tests, same cause</strong></div>");
+            String msg = first.has("errorMessage") ? first.get("errorMessage").asText() : "";
+            int nl = msg.indexOf('\n');
+            if (nl >= 0) msg = msg.substring(0, nl);
+            if (!msg.isEmpty()) {
+                sb.append("<div class=\"failure-group-msg\">").append(escapeHtml(msg)).append("</div>");
+            }
+            sb.append("<ul class=\"failure-group-tests\">");
+            for (JsonNode t : g) {
+                sb.append("<li>").append(escapeHtml(t.has("testId") ? t.get("testId").asText() : "")).append("</li>");
+            }
+            sb.append("</ul></div>");
+        }
+        return sb.append("</div></div>").toString();
+    }
+
     private static String noFailuresRow() {
         return "<tr><td colspan=\"7\" class=\"no-failures-msg\">&#10003; No failures in this run</td></tr>";
     }

src/main/resources/report-template.html

@@ -254,6 +254,11 @@
         .detail-row td { padding: 0; }
         .detail-panel { padding: 16px 24px; background: var(--surface2); border-top: 2px solid var(--bad); }
         .error-msg { font-size: 13px; font-weight: 600; color: var(--bad); margin-bottom: 10px; }
+        .failure-group { padding: 10px 0; border-bottom: 1px solid var(--border); }
+        .failure-group:last-child { border-bottom: 0; }
+        .failure-group-title { font-size: 14px; margin-bottom: 4px; }
+        .failure-group-msg { font-size: 13px; color: var(--bad); margin-bottom: 6px; word-break: break-word; }
+        .failure-group-tests { margin: 0; padding-left: 20px; font-size: 12px; color: var(--text-secondary); }
         .failure-why { font-size: 12px; color: var(--muted, #6b7280); margin-bottom: 8px; }
         .failure-badge { display: inline-block; padding: 1px 8px; margin-right: 6px; border-radius: 10px; font-size: 11px; font-weight: 700; letter-spacing: .03em; color: var(--bad); border: 1px solid var(--bad); }
         .stack-trace {
@@ -479,6 +484,7 @@ <h1>Selenium Boot Report</h1>
         <!-- ═══════════════ FAILURES TAB ═══════════════ -->
         <div id="tab-failures" class="tab-pane">
             <div class="tab-heading">Failures <span style="font-size:14px;font-weight:400;color:var(--text-secondary)">({{FAILED}} failed)</span></div>
+            {{FAILURE_GROUPS}}
             <div class="card table-card">
                 <div class="table-wrap">
                     <table id="failureTable">

src/test/java/com/seleniumboot/unit/FailureGroupsReportTest.java

@@ -0,0 +1,43 @@
+package com.seleniumboot.unit;
+
+import org.testng.Assert;
+import org.testng.annotations.Test;
+
+import java.lang.reflect.Method;
+
+/** The failures tab lists shared causes above the flat list. */
+public class FailureGroupsReportTest {
+
+    private static String groups(String json) throws Exception {
+        com.fasterxml.jackson.databind.JsonNode tests = new com.fasterxml.jackson.databind.ObjectMapper().readTree(json);
+        Method m = com.seleniumboot.reporting.HtmlReportGenerator.class
+                .getDeclaredMethod("buildFailureGroups", com.fasterxml.jackson.databind.JsonNode.class);
+        m.setAccessible(true);
+        return (String) m.invoke(null, tests);
+    }
+
+    private static String failed(String id, String signature) {
+        return "{\"testId\":\"" + id + "\",\"status\":\"FAILED\",\"errorMessage\":\"gone\","
+                + "\"failureCategory\":\"LOCATOR_NOT_FOUND\",\"failureEvidence\":\"x\","
+                + "\"failureSignature\":\"" + signature + "\"}";
+    }
+
+    @Test
+    public void sharedCause_isListedWithCount() throws Exception {
+        String html = groups("[" + failed("A.a", "s1") + "," + failed("B.b", "s1") + "," + failed("C.c", "s2") + "]");
+        Assert.assertTrue(html.contains("2 tests, same cause"), html);
+        Assert.assertTrue(html.contains("A.a") && html.contains("B.b"), html);
+        Assert.assertFalse(html.contains("C.c"), html);
+    }
+
+    @Test
+    public void noSharedCause_rendersNothing() throws Exception {
+        Assert.assertEquals(groups("[" + failed("A.a", "s1") + "," + failed("C.c", "s2") + "]"), "");
+    }
+
+    @Test
+    public void passedTestsAreIgnored() throws Exception {
+        Assert.assertEquals(groups("[{\"testId\":\"A.a\",\"status\":\"PASSED\",\"failureSignature\":\"s1\"},"
+                + "{\"testId\":\"B.b\",\"status\":\"PASSED\",\"failureSignature\":\"s1\"}]"), "");
+    }
+}