Fix Markdown scope and multilingual evidence routing with paired validation - #73
Merged
Merged
Conversation
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #61, #62; remaining semantic reliability work continues in #66.
Ordinary Markdown could lose its qualifying heading, and English/Russian or mixed-language queries could select the wrong evidence view. This PR fixes heading and list-container scope, repairs stale derived projections without rewriting authored bytes, and recognizes bounded report, speculation, discussion and rejected-proposal queries.
For example, a
## Detailsheading inside a list under## Hypothesismust not turn the enclosing scenario into facts. List boundaries now use a pinned CommonMark parser dependency. Code blocks, literal comments, unchecked tasks, backward qualifications and frame limits retain their meaning. Routing no longer lets an unrelated assistant-speculation sentence, conditional rejection, machine error message or English document-title token override the requested view.Production changes remain in
kb/prose.tsandmemory/intent.ts, plus the parser dependency. Retention, answer generation and answer verification are unchanged from the actual base. Luna remains the runtime model; no stronger fallback or additional model calls are introduced.Review and current validation
Four independent review/fix rounds followed by a clean final code review cover the reported regressions and nearby cases. The final reviewer ran 463 assertions with no remaining findings. The comparison harness also now rejects failed or incomplete runs, including an empty result index with an otherwise successful completion receipt.
Current production commit
558954dcompletes all 48/48 offline pairs against actual basece8fbe4. Complete arm observations are deeply equal to the previous offline run, with no fields excluded: reads, projections, routing, recall evidence, answer inputs/outcomes, factual controls and source preservation. Projection upgrades are separately checked against the recorded runtime version.All 88 retained explicit-view evidence controls remain identical. All four ordinary fixtures upgrade a base-created index correctly, and all 48 pairs preserve source bytes. Additional review regressions have dedicated tests without changing the frozen fixture denominator.
Typecheck/build, lint, knip, formatting, repository safety, 4,121 tests across 156 files, 34 smoke checks, and the installed-release-package smoke pass. The rebuilt CLI/client passes 61 assertions across two isolated service starts. Build-only redeploy succeeds; no pre-existing live service was restarted. A Changeset covers the additional published behavior.
Historical Luna evidence and limits
The earlier paired live experiment belongs to frozen runtime
f23c0da, not the reviewed558954druntime. No live generation was rerun during these review rounds.Independent review graded all 220 main/control outputs from 11 fixtures and two repetitions. There were 25 source-useful gains and zero useful-answer losses, but three candidate source errors replaced baseline nulls. Eight source-meaning errors and one qualification omission remained. Matching requests shared responses across arms; different requests sampled independently. Reconstructed retained fixtures lack the original complete provenance metadata. This is bounded downstream evidence, not a new retention trial or proof of general accuracy/noninferiority.
The 233/320 full Luna result and failed quality gate remain unchanged. Its corpus, fresh retention and denominator differ from the paired experiment. #61/#62 remain bounded implementation closures; #66 retains unresolved semantic requirements. These results do not establish a cheap-model intelligence ceiling.
The current review report, complete results, source hashes and independent audit are linked below. Earlier evaluation artifacts remain byte-identical.
Current review report · Independent final audit · Exact offline comparison · Historical paired Luna report · Unchanged full Luna evaluation