Skip to content

test(research): bind baseline vectors to production detector - #119

Merged
stacknil merged 1 commit into
mainfrom
stacknil/v07-production-baseline-conformance
Aug 28, 2026
Merged

test(research): bind baseline vectors to production detector#119
stacknil merged 1 commit into
mainfrom
stacknil/v07-production-baseline-conformance

Conversation

@stacknil

Copy link
Copy Markdown
Owner

Summary

  • add production-side exact conformance vectors for both episode research fixtures
  • require the C++ detector to match committed baseline episode counts, identities, indexes, timestamps, and evidence IDs
  • keep the change test-only and leave runtime, CLI, report, and fixture bytes unchanged

Why

The research baseline validator replays v0.6 selection and finding identity in Python. Without a production-side exact vector, a later C++ detector change could drift while the Python replay and static artifacts still remained internally consistent. These two vectors cover distinct boundaries: continuous low-density background with first-maximum selection, and a gap greater than the rule window that preserves two isolated episodes.

How to validate

  • cmake --build build/ci-release --config Release -- /m:1
  • ctest --test-dir build/ci-release -C Release --output-on-failure --timeout 120
  • python -m ruff check scripts/episode_candidate_core.py scripts/episode_baseline_contract.py scripts/evaluate_episode_candidate.py tests/test_episode_candidate*.py
  • python -B -m unittest discover -s tests -p "test_episode_candidate*.py" -v
  • git diff --check origin/main...HEAD
  • gitleaks git . --redact --no-banner --log-opts="origin/main..HEAD"

Risk / rollout notes

  • Design decision: use exact production conformance vectors instead of a new test-only JSON driver; this is the smallest gate that detects detector algorithm or identity drift.
  • Main risk: the raw C++ vectors intentionally duplicate the normalized fixture inputs, so a deliberate fixture change must update both sides.
  • Compatibility impact: tests only; no source, runtime behavior, CLI, report schema, configuration, or release behavior changes.
  • Rollback path: revert the single test commit; no migration or persisted state exists.

@stacknil

Copy link
Copy Markdown
Owner Author

Final current-head review for e110c22 is complete.

  • Design decision: bind both normalized research baselines to exact production-side C++ detector vectors, avoiding a larger test-only JSON driver.
  • Main risk: the raw C++ inputs intentionally duplicate the normalized fixture events; a deliberate fixture revision must update both conformance surfaces.
  • Compatibility impact: test-only. Runtime source, CLI behavior, loglens.report.v3, configuration, fixture bytes, and released v0.6 behavior are unchanged.
  • Rollback path: revert the single test commit; no migration or persisted state exists.

Exact-head evidence:

  • GitHub checks: 11/11 successful; merge state CLEAN
  • MSVC warnings-as-errors build: passed
  • CTest: 7/7 passed
  • Episode-candidate research suite: 25/25 passed
  • Ruff: passed
  • git diff --check: passed
  • Gitleaks: no leaks
  • Final one-file / 79-line diff review: no unresolved reuse, quality, efficiency, clarity, privacy, or scope findings

@stacknil
stacknil merged commit 03089db into main Aug 28, 2026
11 checks passed
@stacknil
stacknil deleted the stacknil/v07-production-baseline-conformance branch August 28, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant