Introduce Flaky-Test Detection - #798
Draft
castler wants to merge 3 commits into
Draft
Conversation
Introduce reusable composite actions to support CI, particularly the nightly flaky-test detection pipeline: - setup_qnx_environment: installs QEMU, enables KVM permissions and provisions the QNX license, with a post step that cleans the license up. Replaces the inline QNX/QEMU setup previously duplicated in build_and_test_qnx. - upload_bazel_testlogs_on_failure: collects and uploads bazel-testlogs as an artifact when a test step fails, wired into the gcc15, ASan/UBSan/ LSan, TSan and QNX workflows. - prepare_bazel_environment: add an optional repository-cache input so callers can opt into the Bazel repository cache. Also ignore the local .worktrees/ directory. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
castler
force-pushed
the
js_flaky_test_detection
branch
from
July 28, 2026 20:34
aaa089b to
617c082
Compare
Add a scheduled/dispatchable workflow that repeatedly runs the test suite across all supported configurations to surface flaky tests. - nightly_flaky_detection.yml orchestrates per-configuration runner jobs (gcc15 unit/integration, ASan/UBSan/LSan, TSan, QNX unit/integration) via the reusable _nightly_flaky_detection_runner.yml, using --runs_per_test with --runs_per_test_detects_flakes and collecting the Bazel build event protocol output per config. - collect_flaky_tests.py parses the per-config BEP into a structured report; merge_flaky_reports.py aggregates all configs into a single per-target summary (JSON + Markdown step summary) and exposes counts as workflow outputs. - Tests exclude themselves from detection via the no-flaky-test-detection tag; integration tests are tagged integration-test so the runner can select unit vs integration suites with --test_tag_filters. - README_flaky_detection.md documents the pipeline; the scripts are packaged as Bazel py_binary/py_test targets with unit tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the ephemeral report-only model with self-updating GitHub issues, so each flaky test target has a single durable, deduplicated issue that accumulates history across nightly runs. - sync_flaky_issues.py reconciles the merged flaky-target summary against existing issues: it creates one issue per target on first detection and, on recurrence, increments a cumulative counter, refreshes a managed stats region in the body and appends a run comment. Per-run comments are the source of truth (cumulative totals are recomputed from them), the run_id makes reruns idempotent, and closed issues are reopened rather than duplicated. Robust to human edits: a corrupted managed body block is rebuilt from the comment ledger. - The aggregate job gains issues: write permission and a sync step that runs after the report is built. - Unit tests cover creation, recurrence increment, idempotency, reopen and corrupted-body recovery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.