Skip to content

F5 digests bytes addressed by a range grammar the spec never defines #183

Description

@macanderson

F5 requires a digest over "the exact UTF-8 source bytes addressed by uri + range". range is never defined. Two conforming implementations cannot agree on what bytes a provenance digest covers, so the one guarantee that makes provenance tamper-evident is not interoperable.

In plain words

When a digest ("fingerprint") is computed over a range of text — like "line 120 to 160" of a file — the exact rules for what bytes that covers are never written down anywhere a provider author could read: where the range starts counting, whether the end line is included, whether trailing newlines count, what happens if the range runs past the end of the file. The reference code has already picked answers to all of these questions, silently, and a different (equally reasonable) implementation will compute a different fingerprint for the same underlying text — which the protocol then reports as tampering, not as a parsing disagreement.

SPEC.md mentions range five times and defines it zero times. It appears in the §6 frame example as "range": "L120-160" and in §6.2's digested-bytes clause. There is no grammar, no anchor, and no conformance check that two implementations produce the same digest for the same range.

The reference implementation already knows this. contextgraph-host/src/verify.rs:25:

//! `SPEC.md` §6.2 does not fix a `range` grammar; the only convention in this
//! codebase is line ranges, `L<start>` or `L<start>-<end>` (1-indexed,

So the grammar exists, is load-bearing, and is published nowhere a provider author would read it — while §1 states a provider "can be implemented from this document, the JSON Schema, and the examples alone".

Why this is not cosmetic

Every one of these is a decision extract_line_range (contextgraph-host/src/verify.rs:171) has already made, silently, on behalf of the whole ecosystem:

  • Inclusive or exclusive end? It is inclusive: L1-4 is four lines.
  • 1- or 0-indexed? 1-indexed; L0 is an error.
  • Does a line include its terminating \n? Yes — L1-4 on a \n-terminated file includes the newline after line 4. A verifier that excluded it computes a different digest for identical bytes on disk.
  • Is \r stripped? No (§6.2's no-normalization clause), but nothing states that range inherits it.
  • What if the end is past EOF? The end is clamped to EOF while a start past EOF is an error — an asymmetry no one would reproduce by guessing. A provider that treated an out-of-range end as an error instead would disagree with the reference host about L1-9999 on a 10-line file.
  • Is a byte range, a char range, or a column span ever legal? Unknown. unsupported_range rejects them, so the reference implementation has closed a vocabulary the spec never opened.

A provider that picks any other reasonable answer produces digests the host reports as mismatched — which §6.5.4 and the F5-bytes host check both treat as the signal for tampering. The failure mode of an undefined grammar here is a false tampering report, not a parse error.

Reproduce

rg -n 'range' SPEC.md          # five mentions, no grammar
rg -n 'unsupported_range' contextgraph-host/src/verify.rs

Then write a provider that emits range: "120-160" (no L), or "L120-160" meaning an exclusive end, and run contextgraph-inspect. Nothing catches it; the provenance-fixture-consistency check passes or fails depending on a convention the provider author had no way to learn.

Why this needs a decision rather than a patch

Publishing a grammar is a normative addition under §15 and pins choices the ecosystem then lives with inside contextgraph/1 (§13 U4 makes the field's meaning unchangeable afterwards). The open questions — whether the end clamps or errors, whether a non-line range vocabulary is reserved, whether range gets its own requirement anchor or rides F5 — are the maintainer's to settle, and one of them (clamp vs. error) means choosing which existing behaviour is the bug.

Pillar

Stability and reliability. F5 is the floor the attestation chain (§6.5) is built on: encode(link) length-prefixes range into the signed preimage, so two implementations that disagree about what L120-160 means also compute different chain_heads and therefore different signatures over identical evidence.

Done looks like

  • A range grammar in §6.2 with its own requirement anchor, ABNF alongside §3.1's version grammar, stating: prefix, indexing base, end inclusivity, newline inclusion, past-EOF behaviour, and whether non-line ranges are reserved or forbidden.
  • A statement of what an unrecognised range means for a verifier — an error, per current behaviour, and explicitly not a whole-resource fallback.
  • A published reference vector in tests/vectors/ pinning the digest of a known range over a known file, so a second implementation can check itself.
  • A conformance check, or an extension of provenance-fixture-consistency, that fails a provider whose range grammar disagrees with the spec.
  • If README's seven-guarantee table advertises this digest (content integrity / provenance), confirm its wording still matches the pinned grammar once resolved.
  • Lands across contextgraph-host (verify.rs) and contextgraph-conformance (reference vectors, checks) without growing either into a new god file under this workspace's size conventions.
  • Any new SPEC.md prose cites code by symbol (extract_line_range, unsupported_range) rather than a pinned line number, matching this org's citation convention.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions