Skip to content

feat(receipts): tool-execution receipts + epistemic-source classification (#251) - #254

Merged
tcconnally merged 2 commits into
mainfrom
feat/251-spec
Aug 16, 2026
Merged

feat(receipts): tool-execution receipts + epistemic-source classification (#251)#254
tcconnally merged 2 commits into
mainfrom
feat/251-spec

Conversation

@tcconnally

Copy link
Copy Markdown
Contributor

Summary

Implements #251: tool-execution receipts + epistemic-source classification (arXiv:2603.10060, NabaOS — "Tool Receipts, Not Zero-Knowledge Proofs"). Runtime-issued, HMAC-signed receipts per tool call (unforgeable by the model), a pramāṇa epistemic classifier, and a claim cross-check engine that flags fabricated executions, count misstatements, and false-absence claims. Extends the #235 evidence-level ladder and #224 external-artifact bindings with a receipt cross-check layer.

What changed

  • ledger_agent/tool_receipts.pybuild_tool_receipt/verify_tool_receipt (HMAC-SHA256 over id|tool_name|input_hash|output_hash|result_count|facts|timestamp, per the paper's field order; key resolution reuses the feat(receipts): evidence levels — separate signature authenticity from durable inclusion #235 key_registry path), a session-scoped ToolReceiptLedger with completeness (omitted-call) detection, pramāṇa classification (pratyakṣa/anumāna/upamāna/śabda/abhāva/ungrounded), six hallucination-type flags, five trust levels, and markdown epistemic-label rendering. receipt_to_evidence_hash() bridges tool receipts into evidence_hashes.
  • benchmark/nyaya_verify_bench.py — deterministic 1,800-scenario NyayaVerifyBench adaptation (4 languages × 6 hallucination types × 50 + 150 clean) with per-type detection rates, false-positive rate, and verification-overhead measurement; fails the run below the issue's 90% / 20ms gates.
  • docs/tool-receipts.md — receipt spec, pramāṇa table, 6-stage protocol, threat model, measured benchmark numbers vs the paper's 94.2% / <15ms, and the receipts-over-ZK cost argument (zkLLM ~180s/query vs receipts <20ms).
  • tests/test_tool_receipts.py — signature/tamper/forgery coverage, all six hallucination types, completeness, trust levels, benchmark-generator determinism.

Success criteria (#251)

  • Fabricated-tool-reference detection ≥ 90% on the adapted benchmark (paper: 94.2%) — measured 100.00% (0/600 false positives on clean claims; see PR comment / benchmark output).
  • Verification overhead < 20 ms per response (paper: <15 ms) — measured 0.032 ms average / 0.032 ms median.
  • Epistemic labels rendered on receipts and claim surfaces; absence claims verified against receipt sets.

Closes #251

Note on the benchmark numbers: the adapted NyayaVerifyBench is deterministic (seeded, programmatic ground truth rather than LLM-generated responses), so the 100% detection rate reflects the detector's completeness against exact injected fabrications rather than a claim of parity with the paper's 94.2% on their LLM-generated scenarios. The honest comparisons are: every fabricated reference is caught, false positives are zero, and overhead is ~0.03 ms vs the paper's <15 ms — receipts remain the cheapest practical verification layer for interactive agents.

@tcconnally
tcconnally merged commit ae1506e into main Aug 16, 2026
2 checks passed
@tcconnally
tcconnally deleted the feat/251-spec branch August 16, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(receipts): tool-execution receipts + epistemic-source classification (NabaOS 2603.10060)

1 participant