Skip to content

Improve agent-memory evidence and benchmark integrity - #225

Open
Coding-Dev-Tools wants to merge 42 commits into
mainfrom
codex/benchmark-expansion-20260915
Open

Coding-Dev-Tools wants to merge 42 commits into
mainfrom
codex/benchmark-expansion-20260915

Conversation

@Coding-Dev-Tools

@Coding-Dev-Tools Coding-Dev-Tools commented Sep 17, 2026

Copy link
Copy Markdown
Owner

Engraphis now preserves more useful, source-bound evidence within an explicit context budget, while benchmark artifacts and resumed runs remain attributable to the inputs and implementation that produced them.

  • Add opt-in coverage packing, typed exact values, and retrieval recipes. A literal inside an oversized sentence can retain nearby conditions without exceeding the rendered token budget; repeated literals retain the correct source coordinates. Multiline payloads preserve surrounding restrictions whenever the budget permits.
  • Validate structured action destinations, source identity, literal spans, Unicode values, and explicit authorization. The deterministic action diagnostic passes 14/14 cases. One 35-token fixture retains three sources in 33 tokens with coverage packing versus one in 35 with legacy packing. One oversized-sentence fixture now retains its literal and nearby conditions in 24 tokens where the previous implementation admitted no source.
  • Preserve peer evidence attribution and trust labels, apply candidate limits after mapping, score malformed candidate output as failure, and reject unsupported eligible completions. Paired comparisons bind normalized corpora and repair manifests.
  • Freeze producer, dataset, and repair snapshots through final artifact construction. Campaign publication validates manifest and loaded corpus identity; analysis and validation receipts retain digests from the exact bytes parsed. Related journey and storage/capacity exporters verify completed envelopes. Checkpoint v2 validates cached question coverage, scoring data, and final producer state, and binds actual embedder and runtime identities. Changed identities require new checkpoint directories; historical evidence remains intact. Unscored category metrics stay null.
  • Carry the synchronized Railway startup and ownership repairs from PR fix: harden Railway startup persistence and readiness #224, including rootless provisioning, restored-volume detection, and rejection of linked privileged inputs.
  • Select offline-fixtures-v45.json for the current public registry and regenerate its SVGs. All 578 producer hashes match committed source bytes; all 87 retained artifact checksums verify. The current action/packing diagnostic is evidence-contracts-20260919-v5.json.

Validation on 0c11ae58d04694ed4a615b22de7e89f33569bbe3: the full offline unit suite passed (5,659 passed, 59 skipped); all eight offline evaluation gates, Ruff, Pyright, commercial manifest and dashboard asset checks passed. All 29 GitHub checks are successful, including Python 3.9 through 3.14, coverage, browser accessibility, security analysis, packaging, and Docker smoke. Nineteen isolated container startup cases, eight POSIX regression cases, and actual UID 10001 startup/restart checks passed. The branch merges cleanly with current main and PR #224; all shared Railway files match.

These action and packing results are small deterministic development fixtures. They do not establish external QA, generated-answer quality, provider savings, full capacity qualification, or a held-out quality gain. Production retrieval and packing defaults remain unchanged.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants