Add ECEB enterprise context benchmark

.github/workflows/ci.yml

@@ -33,6 +33,8 @@ jobs:
         run: cargo test --all-features --all-targets --locked
       - name: Validate SCGP-0.1
         run: spec/validate.sh
+      - name: Validate ECEB-0.1
+        run: scripts/eceb-conformance.sh
       - name: Build documentation
         run: cargo doc --all-features --no-deps --locked
       - name: Package

.github/workflows/pages.yml

@@ -29,6 +29,7 @@ jobs:
           mkdir -p _site/spec
           cp -a site/. _site/
           cp spec/symgliph-context-glyph-v0.1.md _site/spec/SCGP-0.1.md
+          cp spec/eceb-v0.1.md _site/spec/ECEB-0.1.md
       - name: Configure Pages
         uses: actions/configure-pages@v5
       - name: Upload Pages artifact

CHANGELOG.md

@@ -12,6 +12,8 @@ the project intends to use Semantic Versioning after its first stable release.
 - Bright kinetic public presentation site and GitHub Pages deployment workflow.
 - Public contribution, conduct, security, citation, and issue-reporting files.
 - Published semantic-routing gold seed and blind-discovery proof documentation.
+- ECEB-0.1 draft specification, JSON Schemas, multi-source enterprise
+  conformance seed, executable evaluator, and negative policy/provenance gates.
 
 ### Changed
 

README.md

@@ -153,6 +153,37 @@ provider token usage, billed credits, latency, answers, provenance, and every
 individual pass/fail assertion. This command spends API credit; its default
 report path is `.symglyph/openrouter-proof.json`.
 
+## Enterprise Context Efficiency Benchmark
+
+[`ECEB-0.1`](spec/eceb-v0.1.md) measures the minimum verified,
+policy-permitted context that preserves task quality. It treats provenance,
+access policy, token budget, answer constraints, and repeated context identity
+as mandatory gates, then reports tokens, cost, latency, and quality as a metric
+vector rather than hiding failures inside one score.
+
+The checked-in enterprise conformance seed crosses code, an RFC, an operations
+runbook, and pull-request history while a semantically similar payroll document
+acts as restricted evidence. Run the complete positive and negative gate:
+
+```console
+scripts/eceb-conformance.sh
+```
+
+Or evaluate any schema-valid recorded run:
+
+```console
+cargo run -- evaluate-eceb \
+  datasets/eceb-v0.1/benchmark.json \
+  datasets/eceb-v0.1/runs/conformance-pass.json \
+  --corpus datasets/eceb-v0.1/corpus
+```
+
+The seed deterministically reports 86.51% mean input-token reduction and 79.40%
+mean synthetic-cost reduction across two tasks. Those values prove the evaluator
+and its fail-closed gates; they are explicitly not empirical model-performance
+claims. See [`datasets/eceb-v0.1`](datasets/eceb-v0.1/) for the benchmark, run,
+expected report, source corpus, and claim boundary.
+
 ## Artifact contract
 
 For a corpus `X`, this release produces:

datasets/eceb-v0.1/README.md

@@ -0,0 +1,59 @@
+# ECEB enterprise seed v0.1
+
+This directory is the mandatory **conformance seed** for the Enterprise Context
+Efficiency Benchmark 0.1. It mixes five synthetic enterprise source types:
+
+- Rust authentication code;
+- an architecture RFC;
+- an operations runbook;
+- pull-request history; and
+- a restricted payroll distractor.
+
+The two tasks require evidence across code, design history, and operations while
+explicitly denying the semantically similar payroll artifact. Every source is
+synthetic and safe to publish.
+
+## What the fixture proves
+
+The fixture proves that an implementation can:
+
+- verify complete corpus bytes before evaluation;
+- derive and check the ECEB corpus root;
+- compare full permitted context with a budget ladder;
+- enforce a quality non-inferiority margin;
+- require exact evidence provenance;
+- reject restricted-source leakage;
+- reject stale evidence and token overflow;
+- require repeatable context construction; and
+- emit a schema-valid metric vector.
+
+The checked-in run uses deterministic fixture answers, synthetic costs, and
+declared conformance-only quality scores. Its 86.51% mean token reduction and
+79.40% mean cost reduction are **not empirical model results**.
+
+## Run the gate
+
+From the repository root:
+
+```console
+scripts/eceb-conformance.sh
+```
+
+Or evaluate the passing run directly:
+
+```console
+cargo run -- evaluate-eceb \
+  datasets/eceb-v0.1/benchmark.json \
+  datasets/eceb-v0.1/runs/conformance-pass.json \
+  --corpus datasets/eceb-v0.1/corpus
+```
+
+The normative contract is [`spec/eceb-v0.1.md`](../../spec/eceb-v0.1.md).
+
+## Replacing the fixture with a real run
+
+Keep `benchmark_id` and `corpus_root` immutable. Replace the model, judge,
+pricing snapshot, answers, evidence selections, context digests, and raw metrics
+with measurements from the declared provider. Record at least two repetitions
+for every baseline and budget under evaluation. Do not publish a quality claim
+without the judge rubric and raw judgments required by the specification.

datasets/eceb-v0.1/benchmark.json

@@ -0,0 +1,86 @@
+{
+  "schema": "symgliph.eceb/benchmark-v0.1",
+  "benchmark_id": "symgliph-eceb-enterprise-seed/v0.1",
+  "corpus_root": "9c6777f56c9ea2703950a3a09718a5ba90e253652540e0c7a4c7943ef8e8c8fe",
+  "required_repetitions": 2,
+  "artifacts": [
+    {
+      "artifact_id": "code:auth-verifier",
+      "path": "code/auth.rs",
+      "source_kind": "code",
+      "classification": "internal",
+      "source_blake3": "fb852a46df67d0a3e341439d21cf8acd6ab98ee33cd2fdb4282a6cc793a7708f"
+    },
+    {
+      "artifact_id": "history:pr-1842",
+      "path": "history/pr-1842.md",
+      "source_kind": "pull_request",
+      "classification": "internal",
+      "source_blake3": "1bfcc6f94d46c51d8cff149c53c066262958b9dd864667da5883c3e21270153c"
+    },
+    {
+      "artifact_id": "hr:payroll-token-rotation",
+      "path": "hr/payroll-token-rotation.md",
+      "source_kind": "policy",
+      "classification": "restricted",
+      "source_blake3": "fa2010a2fae489777b5ae23131c431c79f681a773af04990d13fb44adaaa5d3a"
+    },
+    {
+      "artifact_id": "ops:auth-runbook",
+      "path": "ops/auth-service-runbook.md",
+      "source_kind": "runbook",
+      "classification": "confidential",
+      "source_blake3": "b2dd2f55505e872fc573fe9c1bd0720c1b1fc5a777293f561e58949586a8e402"
+    },
+    {
+      "artifact_id": "rfc:jwt-retry-policy",
+      "path": "rfc/0007-jwt-retry-policy.md",
+      "source_kind": "rfc",
+      "classification": "internal",
+      "source_blake3": "2ffe428c37fdb630ea2472450568e94e7c763c0f802832120899ac09c3256b94"
+    }
+  ],
+  "tasks": [
+    {
+      "task_id": "auth:bounded-retries",
+      "query": "Explain why JWT issuer retries are limited, where the rule is enforced, and how operators respond after issuer failure.",
+      "required_artifact_ids": [
+        "code:auth-verifier",
+        "history:pr-1842",
+        "ops:auth-runbook",
+        "rfc:jwt-retry-policy"
+      ],
+      "allowed_classifications": ["internal", "confidential"],
+      "denied_artifact_ids": ["hr:payroll-token-rotation"],
+      "budget_ladder": [256, 512, 1024],
+      "minimum_quality_milli": 900,
+      "maximum_quality_regression_milli": 25,
+      "required_answer_terms": [
+        "AuthVerifier::verify",
+        "three attempts",
+        "exponential backoff",
+        "circuit breaker"
+      ]
+    },
+    {
+      "task_id": "auth:issuer-key-compromise",
+      "query": "What must the on-call engineer do after a suspected JWT issuer-key compromise, and what evidence permits recovery?",
+      "required_artifact_ids": [
+        "code:auth-verifier",
+        "ops:auth-runbook",
+        "rfc:jwt-retry-policy"
+      ],
+      "allowed_classifications": ["internal", "confidential"],
+      "denied_artifact_ids": ["hr:payroll-token-rotation"],
+      "budget_ladder": [256, 512, 1024],
+      "minimum_quality_milli": 900,
+      "maximum_quality_regression_milli": 25,
+      "required_answer_terms": [
+        "revoke the previous key identifier",
+        "invalidate every verifier cache",
+        "signed canary token",
+        "circuit breaker"
+      ]
+    }
+  ]
+}

datasets/eceb-v0.1/corpus/code/auth.rs

@@ -0,0 +1,38 @@
+pub struct AuthVerifier {
+    issuer: String,
+}
+
+impl AuthVerifier {
+    pub fn verify(&self, token: &str) -> Result<Claims, AuthError> {
+        let mut delay_ms = 50;
+        for attempt in 1..=3 {
+            match self.fetch_and_validate(token) {
+                Ok(claims) => return Ok(claims),
+                Err(AuthError::IssuerUnavailable) if attempt < 3 => {
+                    std::thread::sleep(std::time::Duration::from_millis(delay_ms));
+                    delay_ms *= 2;
+                }
+                Err(error) => return Err(error),
+            }
+        }
+        Err(AuthError::IssuerUnavailable)
+    }
+
+    fn fetch_and_validate(&self, token: &str) -> Result<Claims, AuthError> {
+        validate_signature_and_audience(token, &self.issuer)
+    }
+}
+
+pub struct Claims;
+
+pub enum AuthError {
+    IssuerUnavailable,
+    InvalidSignature,
+}
+
+fn validate_signature_and_audience(
+    _token: &str,
+    _issuer: &str,
+) -> Result<Claims, AuthError> {
+    Ok(Claims)
+}

datasets/eceb-v0.1/corpus/history/pr-1842.md

@@ -0,0 +1,11 @@
+# PR 1842: Bound issuer retries
+
+The authentication client previously retried discovery until its caller timed
+out. Reviewers rejected a five-attempt proposal because synchronized application
+instances could multiply an issuer outage. The accepted implementation caps the
+loop at three attempts, doubles the delay after transient availability errors,
+and immediately returns signature failures.
+
+The rollout test asserts that an unavailable issuer opens the circuit breaker
+after the third failure. The change intentionally does not add a verification
+bypass or a stale-key fallback.

datasets/eceb-v0.1/corpus/hr/payroll-token-rotation.md

@@ -0,0 +1,8 @@
+# Payroll export token rotation
+
+Classification: restricted
+
+The payroll export service rotates its bearer token after three failed attempts.
+The emergency credential and employee compensation export location are available
+only to the payroll operations group. These instructions are unrelated to JWT
+issuer verification and must never be included in an engineering support answer.

datasets/eceb-v0.1/corpus/ops/auth-service-runbook.md

@@ -0,0 +1,16 @@
+# Authentication service runbook
+
+## Issuer unavailable
+
+1. Confirm the `issuer_unavailable` alert and identify the affected issuer.
+2. Leave the authentication circuit breaker open; do not bypass verification.
+3. Check issuer discovery and key-set endpoints from the production network.
+4. After recovery, invalidate the verifier cache.
+5. Send a signed canary token and confirm signature, audience, and expiry checks.
+6. Close the circuit breaker and attach the canary result to the incident.
+
+## Suspected issuer-key compromise
+
+Rotate the issuer key, revoke the previous key identifier, invalidate every
+verifier cache, and keep the circuit breaker open until a signed canary token
+passes. Escalate any unexpected acceptance of the revoked key to Security.

datasets/eceb-v0.1/corpus/rfc/0007-jwt-retry-policy.md

@@ -0,0 +1,21 @@
+# RFC 0007: JWT issuer retry policy
+
+## Decision
+
+JWT verification may retry an unavailable issuer at most three attempts with
+exponential backoff beginning at 50 milliseconds. Invalid signatures are never
+retried. Exhausted availability retries open the authentication circuit breaker
+and fail the request closed.
+
+## Rationale
+
+Three attempts tolerate a brief discovery outage without allowing authentication
+traffic to amplify an issuer failure. The circuit breaker protects both systems
+and prevents an unavailable identity service from becoming an authorization
+bypass.
+
+## Recovery
+
+Issuer-key rotation requires the on-call engineer to invalidate the verifier
+cache, restore the circuit breaker only after a signed canary token succeeds,
+and retain the incident identifier in the deployment record.

datasets/eceb-v0.1/expected-report.json

@@ -0,0 +1,51 @@
+{
+  "schema": "symgliph.eceb/report-v0.1",
+  "benchmark_id": "symgliph-eceb-enterprise-seed/v0.1",
+  "system": "symgliph-conformance-fixture/v0.1",
+  "model": "deterministic-fixture/no-provider-call",
+  "judge": "required-terms-plus-declared-score/conformance-only",
+  "pricing_snapshot": "synthetic-microusd/conformance-only/2026-08-02",
+  "passed": true,
+  "qualified_tasks": 2,
+  "total_tasks": 2,
+  "mean_token_savings_bps": 8651,
+  "mean_cost_savings_bps": 7940,
+  "tasks": [
+    {
+      "task_id": "auth:bounded-retries",
+      "baseline_prompt_tokens": 3200,
+      "minimum_viable_budget": 512,
+      "compiled_prompt_tokens": 440,
+      "token_savings_bps": 8625,
+      "baseline_cost_microusd": 7000,
+      "compiled_cost_microusd": 1464,
+      "cost_savings_bps": 7908,
+      "quality_floor_milli": 930,
+      "baseline_quality_milli": 955,
+      "compiled_quality_milli": 944,
+      "provenance_coverage_bps": 10000,
+      "policy_violations": 0,
+      "repeatable": true,
+      "passed": true,
+      "failures": []
+    },
+    {
+      "task_id": "auth:issuer-key-compromise",
+      "baseline_prompt_tokens": 2800,
+      "minimum_viable_budget": 512,
+      "compiled_prompt_tokens": 370,
+      "token_savings_bps": 8678,
+      "baseline_cost_microusd": 6120,
+      "compiled_cost_microusd": 1240,
+      "cost_savings_bps": 7973,
+      "quality_floor_milli": 935,
+      "baseline_quality_milli": 960,
+      "compiled_quality_milli": 949,
+      "provenance_coverage_bps": 10000,
+      "policy_violations": 0,
+      "repeatable": true,
+      "passed": true,
+      "failures": []
+    }
+  ]
+}