Skip to content

test(guard): tests must not gate on paths outside the workspace - #2478

Open
noahgift wants to merge 1 commit into
mainfrom
fix/test-fixture-path-guard
Open

test(guard): tests must not gate on paths outside the workspace#2478
noahgift wants to merge 1 commit into
mainfrom
fix/test-fixture-path-guard

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

The monorepo consolidated 20 sibling repos in-tree. Tests written before the
merge still gate on the OLD sibling checkouts:

let has_q4k = file_exists("/home/noah/src/realizar/src/quantize.rs");
if !has_q4k { eprintln!("SKIP - realizar not found"); return; }

realizar IS crates/aprender-serve now. That path cannot exist again on any
machine, so every test behind such a gate is permanently and silently green.
Five of them remain, in falsification_2x_ollama_tests.rs and
falsification_correctness_tests.rs, and one runs a command with
.current_dir("/home/noah/src/realizar"). Not one reads the file it probes --
the path exists only to decide whether to skip. F102's gate is purely
decorative: its body times two inline loops and never touches realizar at all.

They are invisible twice over: workspace-test runs --lib, so these integration
targets are never compiled either. A skip nobody sees, in a test nobody runs,
behind a claim somebody trusts.

Scope: 85 out-of-workspace paths across 1921 test files. Includes gates on this
repo's OWN pre-monorepo layout (/home/noah/src/aprender/src/format/mod.rs --
that file is crates/aprender-core/src/format/ now).

This is a RATCHET, not a cleanup. Repointing the existing gates makes dormant
tests execute for the first time and will surface real failures; that is its own
change with room for the fallout. This stops the population growing and makes
the debt a number that can only fall.

Verified:

  • case table, 3 defect shapes + 4 legitimate paths (workspace-relative, /tmp,
    env var, relative). The table caught my first version being BLIND -- the
    find pruned tests directories, excluding the only thing it must scan.
    Fifth time a guard regex in this repo was wrong; fifth time a table caught it
    and review would not have.
  • mutation: add one new out-of-workspace gate -> RED, listing every violation.
    Restored -> PASS.
  • vacuity: fails if fewer than 200 test files are scanned, so a broken scan
    cannot report clean.

Refs #2474

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

…erywhere but one machine

The monorepo consolidated 20 sibling repos in-tree. Tests written before the
merge still gate on the OLD sibling checkouts:

    let has_q4k = file_exists("/home/noah/src/realizar/src/quantize.rs");
    if !has_q4k { eprintln!("SKIP - realizar not found"); return; }

realizar IS crates/aprender-serve now. That path cannot exist again on any
machine, so every test behind such a gate is permanently and silently green.
Five of them remain, in falsification_2x_ollama_tests.rs and
falsification_correctness_tests.rs, and one runs a command with
`.current_dir("/home/noah/src/realizar")`. Not one reads the file it probes --
the path exists only to decide whether to skip. F102's gate is purely
decorative: its body times two inline loops and never touches realizar at all.

They are invisible twice over: workspace-test runs `--lib`, so these integration
targets are never compiled either. A skip nobody sees, in a test nobody runs,
behind a claim somebody trusts.

Scope: 85 out-of-workspace paths across 1921 test files. Includes gates on this
repo's OWN pre-monorepo layout (`/home/noah/src/aprender/src/format/mod.rs` --
that file is `crates/aprender-core/src/format/` now).

This is a RATCHET, not a cleanup. Repointing the existing gates makes dormant
tests execute for the first time and will surface real failures; that is its own
change with room for the fallout. This stops the population growing and makes
the debt a number that can only fall.

Verified:
  * case table, 3 defect shapes + 4 legitimate paths (workspace-relative, /tmp,
    env var, relative). The table caught my first version being BLIND -- the
    `find` pruned `tests` directories, excluding the only thing it must scan.
    Fifth time a guard regex in this repo was wrong; fifth time a table caught it
    and review would not have.
  * mutation: add one new out-of-workspace gate -> RED, listing every violation.
    Restored -> PASS.
  * vacuity: fails if fewer than 200 test files are scanned, so a broken scan
    cannot report clean.

Refs #2474

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@noahgift
noahgift force-pushed the fix/test-fixture-path-guard branch from 879028c to 014ef9f Compare August 18, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant