Prove source-verifiable context compilation

[copyleftdev]

What changed

• positions Symgliph as an experimental research protocol and source-verifiable context compiler
• makes the optional SVG glyph a deterministic fingerprint rather than model input
• publishes a reproducible 30-task Multi-SWE-bench Rust comparison across full-context, BM25, embedding, and compiled-context arms
• adds study schemas, raw results, validation tooling, and CI enforcement
• restructures the public site around protocol, compiler, evidence, limitations, and the future enterprise direction

Why

The public identity previously blurred the visual glyph, the compiler mechanism, and broader enterprise aspirations. This change establishes a precise claim boundary: Symgliph verifies the provenance and integrity of selected excerpts under a token budget, while retrieval relevance and answer quality remain measured research questions.

Evidence

• 51 Rust tests passed across the library, binaries, and conformance suite
• strict Clippy passed with warnings denied
• SCGP-0.1 and ECEB-0.1 conformance passed
• the checked-in Multi-SWE-bench study contract passed
• static-site and HTML validation passed
• release package verification and warning-free rustdoc passed

Study boundary

The included study covers 30 Rust tasks and one model configuration. It supports an efficiency result, not a universal quality claim: compiled context had the lowest recorded API spend and highest exact predicted fix-path recall among the retrieval arms, while gold-path coverage remained incomplete.

Summary by CodeRabbit

• New Features

  • Added deterministic comparative benchmarking across 30 Rust software tasks, with reproducible results for full-context, lexical, embedding, and compiled retrieval strategies.
  • Context compilation can now use explicit file-path hints to improve source-verified results for broad queries.
  • Added a command-line study runner and public study artifacts.

• Documentation

  • Updated the website, README, and guides to present source-verifiable context, protocol guarantees, benchmark evidence, and limitations.

• Quality

  • Added automated validation and replay checks for study data and reported metrics.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1d0d9a6c-a5bf-4b46-952a-1d50075da225

📥 Commits

Reviewing files that changed from the base of the PR and between 933f1e0 and 38bc08b.

📒 Files selected for processing (28)

• .github/workflows/ci.yml
• .impeccable.md
• CHANGELOG.md
• Cargo.toml
• README.md
• benchmarks/eceb-multiswe-rust-v0.1.json
• benchmarks/results/eceb-multiswe-rust-v0.1/embedding_cosine.json
• benchmarks/results/eceb-multiswe-rust-v0.1/full_context.json
• benchmarks/results/eceb-multiswe-rust-v0.1/lexical_bm25.json
• benchmarks/results/eceb-multiswe-rust-v0.1/report.json
• benchmarks/results/eceb-multiswe-rust-v0.1/symgliph_compiled.json
• docs/context.md
• docs/eceb-multiswe-study.md
• docs/site-design-brief.md
• scripts/eceb-study-check.sh
• scripts/site-check.sh
• site/app.js
• site/index.html
• site/styles.css
• spec/README.md
• spec/schema/eceb-study-report-v0.1.schema.json
• spec/schema/eceb-study-run-v0.1.schema.json
• spec/schema/eceb-study-suite-v0.1.schema.json
• src/bin/symgliph-eceb-study.rs
• src/context.rs
• src/eceb_study.rs
• src/lib.rs
• src/main.rs

---

📝 Walkthrough

Walkthrough

The pull request adds a reproducible Multi-SWE-bench ECEB study profile, a Rust study runner, schemas, recorded results, deterministic evaluation, and CI replay validation. It also adds path-seeded context retrieval and updates project, documentation, and site positioning.

Changes

ECEB context compiler

Layer / File(s)	Summary
Study contracts and evaluation benchmarks/eceb-multiswe-rust-v0.1.json, spec/schema/*, src/eceb_study.rs, src/lib.rs	Defines study suites, runs, reports, strategies, observations, metrics, validation rules, and deterministic bootstrap intervals.
Study runner and retrieval execution Cargo.toml, src/bin/symgliph-eceb-study.rs, src/context.rs, src/main.rs	Adds deterministic task preparation, four retrieval strategies, model localization, caching, usage tracking, and path-seeded verified context retrieval.
Study records and replay gate benchmarks/results/eceb-multiswe-rust-v0.1/*, scripts/eceb-study-check.sh, .github/workflows/ci.yml, docs/eceb-multiswe-study.md, CHANGELOG.md	Adds recorded study runs and reports, schema validation, report comparison, expected-metric checks, negative validation, documentation, and CI replay execution.
Public protocol and site presentation README.md, site/index.html, site/app.js, site/styles.css, docs/site-design-brief.md, .impeccable.md, scripts/site-check.sh, docs/context.md	Reframes Symgliph as a source-verifiable context compiler and updates protocol, evidence, limits, navigation, responsive layouts, and public-content checks.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ReplayScript
  participant StudyCLI
  participant ContextEngine
  participant ReportJSON
  ReplayScript->>StudyCLI: Evaluate suite and recorded runs
  StudyCLI->>ContextEngine: Build retrieval contexts
  ContextEngine-->>StudyCLI: Return bounded verified context
  StudyCLI-->>ReplayScript: Generate comparative report
  ReplayScript->>ReportJSON: Compare generated report
  ReportJSON-->>ReplayScript: Confirm expected metrics and failures

Loading

Possibly related PRs

• copyleftdev/symgliph#1: Introduces the ECEB benchmark and validation concepts extended by this comparative study profile.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/research-protocol-context-compiler

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}