Add adaptive context compilation study

[copyleftdev]

What changed

• add deterministic adaptive context compilation to the Rust library, CLI, and JSON-RPC tool server
• define the experimental SACP-0.2 profile and machine-readable result schema
• add a policy-bound ECEB adaptive arm with complete per-round traces and tamper rejection
• publish the calibration record, 30-task OpenRouter observations, replayable report, and public-site evidence update

Why

Fixed context budgets spend the maximum even when an earlier verified packet may be sufficient. This experiment makes expansion explicit and auditable: walk a declared token/node ladder, evaluate a deterministic evidence heuristic, and retain every attempted context digest and stop reason.

The heuristic remains a routing estimate—not a claim of complete evidence or answer quality.

Measured result

On the existing 30 pinned Multi-SWE-bench Rust tasks, the adaptive arm used 4,345 mean prompt tokens versus 5,138 for fixed compilation, retained 51.85% evidence recall, and improved exact fix-path recall from 41.76% to 45.09%. It remained below BM25 and full-context recall. The threshold was calibrated on these same tasks and each arm has one model response per task, so this does not establish quality parity or generalization.

Validation

• cargo test --all-features --all-targets --locked
• cargo clippy --all-features --all-targets -- -D warnings
• spec/validate.sh
• scripts/eceb-conformance.sh
• scripts/eceb-study-check.sh
• scripts/eceb-adaptive-study-check.sh
• scripts/site-check.sh
• npx --yes html-validate site/index.html site/404.html
• rustdoc with warnings denied
• cargo package --locked --allow-dirty

The adaptive replay gate also proves that mismatched selected rounds and premature hard-limit claims fail closed.

Summary by CodeRabbit

• New Features
  • Added experimental adaptive context compilation that expands retrieval budgets until evidence is sufficient or limits are reached.
  • Added the adaptive-pack command and context.adaptive JSON-RPC method, with configurable token, node, coverage, and evidence requirements.
  • Added adaptive study support, trace reporting, schemas, and validation safeguards.
• Documentation
  • Added the SACP-0.2 specification, calibration findings, study methodology, API guidance, and published-site updates.
• Tests
  • Added replay checks and conformance validation for adaptive outputs, stopping behavior, and invalid configurations.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2822ad14-e504-44be-a28f-91c37ffc6b13

📥 Commits

Reviewing files that changed from the base of the PR and between fd0dd49 and 78fa895.

📒 Files selected for processing (30)

• .github/workflows/ci.yml
• .github/workflows/pages.yml
• CHANGELOG.md
• README.md
• benchmarks/results/eceb-multiswe-rust-adaptive-v0.2/full_context.json
• benchmarks/results/eceb-multiswe-rust-adaptive-v0.2/lexical_bm25.json
• benchmarks/results/eceb-multiswe-rust-adaptive-v0.2/report.json
• benchmarks/results/eceb-multiswe-rust-adaptive-v0.2/symgliph_adaptive.json
• benchmarks/results/eceb-multiswe-rust-adaptive-v0.2/symgliph_compiled.json
• docs/adaptive-calibration.md
• docs/context.md
• docs/eceb-adaptive-study.md
• docs/eceb-multiswe-study.md
• scripts/eceb-adaptive-study-check.sh
• scripts/site-check.sh
• site/index.html
• site/styles.css
• spec/README.md
• spec/adaptive-context-v0.2.md
• spec/schema/adaptive-context-v1.schema.json
• spec/schema/eceb-study-report-v0.1.schema.json
• spec/schema/eceb-study-run-v0.1.schema.json
• spec/validate.sh
• src/adaptive.rs
• src/bin/symgliph-eceb-study.rs
• src/eceb_study.rs
• src/error.rs
• src/lib.rs
• src/main.rs
• src/tool_server.rs

🔥 Files not summarized due to errors (1)

• benchmarks/results/eceb-multiswe-rust-adaptive-v0.2/symgliph_adaptive.json: Server error: no LLM provider could handle the message

---

📝 Walkthrough

Walkthrough

The PR adds experimental SACP-0.2 adaptive context compilation, CLI and JSON-RPC interfaces, adaptive ECEB study support, schemas, replay validation, benchmark results, and documentation.

Changes

Adaptive Context and ECEB Study

Layer / File(s)	Summary
Adaptive contracts and schemas spec/adaptive-context-v0.2.md, spec/schema/*, src/error.rs, src/lib.rs	Defines adaptive requests, expansion rounds, sufficiency results, evidence records, stopping states, study traces, and public Rust exports.
Adaptive compilation interfaces src/adaptive.rs, src/main.rs, src/tool_server.rs, spec/validate.sh	Adds deterministic adaptive packing with budget expansion, sufficiency estimation, CLI output, JSON-RPC dispatch, schema checks, and fail-closed validation.
Adaptive ECEB strategy src/eceb_study.rs, src/bin/symgliph-eceb-study.rs	Adds the adaptive retrieval strategy, option validation, context dispatch, serialized traces, and trace invariant checks.
Study replay and conformance validation scripts/eceb-adaptive-study-check.sh, .github/workflows/ci.yml, benchmarks/results/eceb-multiswe-rust-adaptive-v0.2/*	Adds replay validation, regenerated-report comparison, negative trace tests, CI execution, and committed benchmark artifacts.
Documentation and published evidence README.md, docs/*, CHANGELOG.md, site/*, spec/README.md, .github/workflows/pages.yml, scripts/site-check.sh	Documents the adaptive profile, calibration, study results, conformance gates, publication links, and updated site evidence. Estimated code review effort: 5 (Critical) | ~90 minutes

Possibly related PRs

• copyleftdev/symgliph#1: Introduces the ECEB benchmark infrastructure extended by this adaptive study support.
• copyleftdev/symgliph#2: Adds the ECEB study infrastructure extended here with adaptive compilation and traces.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/adaptive-context-compiler

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}