Skip to content

test-windows-guards: test_daemon_stability turns setup failures and timeouts into REGRESSION verdicts on unrelated PRs #2057

Description

@DeusData

Summary

test / test-windows-guards is a required check, and it goes red on pull requests whose diffs cannot touch it. The red always comes from tests/windows/test_daemon_stability.py, and it comes from three mechanisms in the guard's own harness rather than from the product. Until they are fixed the leg is a lottery: a green needs a rerun that changes nothing but the dice, which is exactly what the CI-determinism rule forbids.

Observed (unrelated diffs, same leg, same file)

PR diff run / job
#1875 Cypher job 99662589846 (line 820: SETUP FAIL: ASCII baseline did not index: {nodes: None, edges: None, definition_nodes: 5})
#1876 Windows Unicode paths reddened 2026-08-28
#1896 Swift extraction reddened 2026-09-01
#2002 Cypher SKIP/LIMIT section_start_status_port: "the daemon did not accept the UI configuration; browser was not opened"
#2055 scripts/run-test-wave.py + one contract test run 33892540012, job 101150446542: section_crash_recovery ("the daemon did not accept the UI configuration; browser was not opened") and section_cold_storm ("cold-storm client 2 failed (racing daemon spawn): secure CLI coordination could not be created (endpoint)")

Mechanisms (verified in the tree on 2026-09-03; line numbers as of that audit)

  1. A setup failure is reported as a regression. The file header of test_daemon_stability.py documents the exit contract 0 == all sections green, 1 == regression, 2 == setup error, and scripts/test-windows.ps1 (around line 190) treats exit 2 as a documented precondition skip. But every setup-failure branch in the stability guard (return False at roughly lines 206 / 218 / 265 / 319) exits 1, while its sibling guards return 2 for the same class of failure. The same daemon-start failure is therefore an invisible skip in one guard and a REGRESSION in this one. That asymmetry is why the red reads like a daemon bug.
  2. A timeout decides a verdict. _parallel_one_shots (around line 297) runs run_cli(timeout=120) on bare threads with no except; a TimeoutExpired leaves results[i] = None, and section_cold_storm (around line 361) then prints "cold-storm client N failed (racing daemon spawn)" for what was a 120 s timeout. A timeout must never be the thing that decides a test.
  3. The daemon cohort is machine-global. cbm_daemon_rendezvous_key() in src/daemon/service.c (around line 144) hashes a compile-time constant, by design. The only relocation hook is CBM_RUNTIME_DIR, and neither scripts/test-windows.ps1 nor anything under tests/windows/ sets it. Every section, and every other daemon-using guard on the same runner, shares one rendezvous.

What "fixed" looks like

  • Setup-failure branches in the stability guard exit 2, matching the documented contract and the sibling guards; a precondition that cannot be met is a skip with a printed reason, never a regression.
  • The cold-storm clients treat TimeoutExpired as a setup error with its own message (and their own budget), not as a client failure.
  • The guards run their daemons under a per-run CBM_RUNTIME_DIR so the cohort is isolated from anything else on the runner.
  • Every case that is red-tolerated or skipped carries an at-the-case explanation of WHY plus what was tried (the Memory leak: process grows to 50+ GB virtual memory over hours/days, crashes Windows #581 whitelist shape); an unexplained skip is a board defect.

#1796 ("a refused UI configuration is not a failed daemon") addresses the section_crash_recovery conflation and is being taken as CI-unblocking; it does not cover section_cold_storm or the exit-code contract, which is what this issue tracks.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcypherCypher query language parser/executor bugsparsing/qualityGraph extraction bugs, false positives, missing edgesstability/performanceServer crashes, OOM, hangs, high CPU/memoryux/behaviorDisplay bugs, docs, adoption UXwindowsWindows-specific issues

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions