Skip to content

docs(protocol): document the content-replication flywheel - #80

Merged
MichaelTaylor3d merged 4 commits into
mainfrom
loop/1425-flywheel-docs
Aug 24, 2026
Merged

docs(protocol): document the content-replication flywheel#80
MichaelTaylor3d merged 4 commits into
mainfrom
loop/1425-flywheel-docs

Conversation

@MichaelTaylor3d

@MichaelTaylor3d MichaelTaylor3d commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What changed

Adds docs/protocol/content-replication.md — the L7 protocol page specifying how content spreads across the DIG Network with no origin server and no CDN: the connect -> discover -> fetch+verify -> cache -> announce loop that turns every reader into a discoverable holder.

Closes the documentation gap for https://github.com/DIG-Network/dig_ecosystem/issues/1425. Before this PR the word "flywheel" appeared in zero files under docs/, and docs/protocol/peer-network.md was the only page carrying any resharing content at all.

Wired in everywhere a page has to be reachable:

  • the Protocol sidebar, immediately after protocol/peer-network
  • the protocol-deep-dive layer table and its Related list
  • static/llms.txt
  • the 13 locale protocol-deep-dive files, each getting exactly two added lines and zero deletions, translated per locale (verified: git diff --numstat origin/main is 2 0 for all 13)
  • static/knowledge-graph.json (regenerated by the build; it picked up the new node and its five see-also edges)

i18n: English fallback, deliberately, with no locale stubs

The new page ships English-only, with no locale copies. Docusaurus renders a present locale file as a completed translation, so an English file wearing a locale's name gives no fallback signal and no future job can detect the gap — a missing file is honest and self-correcting. Machine-produced prose for 13 languages on a normative protocol page would be worse still: a wrong normative sentence in a language nobody here reads is undetectable.

Only the two navigation lines per locale are translated, since those are short and checkable.

The constraint this surfaced, worth knowing: a .md link does not resolve across locale fallback. With no locale copy, Docusaurus falls back to the English page and routes it at /<locale>/docs/protocol/content-replication, but every relative ./peer-network.md inside it then breaks in all 13 non-English builds — onBrokenLinks: "throw" failed on exactly that. The page's internal links are therefore extensionless relative URLs (./peer-network, ./peer-network#dht), which resolve against the current locale's path.

Audience and scope

Protocol-developer tier, per the site's three-audience split — it explains a mechanism, not a task. It is placed in the Protocol section only and is not surfaced in any task-oriented audience category.

How the claims were verified

Every substantive claim was read off code on origin/main of the owning repo rather than taken from a ticket body:

Claim Verified against
find_providers is a distributed iterative lookup; a local hit is a fast path only dig-dht src/service.rs:204
Lookup-discovered records are hearsay, never re-served and never republished dig-dht src/service.rs — the cached_providers doc contract, plus handle_request_from reading the authoritative store only
Cache default is 1 GiB, LRU, pins exempt dig-store-cache src/config.rs:8 (DEFAULT_MAX_BYTES = 1 << 30), src/config.rs:46
Opcode 222 HoldingsAnnounce, dig:holdings:v1, fail-closed dig-gossip src/connection/inbound_limits.rs:21, src/connection/dig_rate_limiter.rs:366; dig-node crates/dig-node-core/src/seams/dig_peer/holdings.rs
The six announce-ingress threat classes and their guards dig-node holdings.rs module doc — reproduced as classes, not as individual attacks
Inventory gain announces at store and capsule granularity dig-node crates/dig-node-core/src/seams/dig_peer/dht.rs:1-25
Eviction uses an active retract_own_provider, not a passive withdraw dig-node dht.rs:546-550, with the regression test at dht.rs:1136
Half-specified proof/root binding fails closed dig-node crates/dig-node-core/src/download.rs:~425 (DigstoreProofVerifier)
Cache/announce/reshare fire only for first-party reads dig-node download.rs:403 (landing_origin), capsule_store.rs:190 (maybe_backfill_capsule is a no-op unless origin == Local)
Inbound-demand pull is operator opt-in and XOR-proximity confined dig-node lib.rs:4521, capsule_store.rs:167

Omitted rather than guessed: the concrete alpha and k Kademlia parameters, the advertised-TTL and republish-interval numbers (the page states the relation that binds them, which is what an implementer needs and what is enforced by a test, without pinning values that are configurable), and the gossip flood fan-out.

The DHT / recursive-hop distinction

The page carries an explicit section (2c) with a side-by-side table separating this iterative provider lookup from the transitive hop-by-hop ask, on four axes: shape, what comes back, who carries the bytes, and who bears the cost. The llms.txt entry repeats the distinction so a machine reader summarizing the page cannot conflate them either. No wording anywhere in the page describes the DHT as satisfying a recursive peer-ask.

Docs discipline

No internal project files, ticket numbers, agent or process commentary, or drift narration appear in the page. It is written in the normative present tense the surrounding protocol pages use.

Blast radius

Documentation and site configuration only. No code symbols were edited, so there are no callers to assess. The touched surfaces are the new page, sidebars.ts (one array insertion), protocol-deep-dive.md (two additive entries), llms.txt (one additive entry), the 13 locale mirrors, and the build-regenerated knowledge-graph.json / openrpc*.json version stamps (which were already stale at 0.15.0 against a 0.16.1 package version, and are now coherent at 0.17.0).

Evidence

  • npm run build green with onBrokenLinks: "throw" and onBrokenAnchors: "throw" — so every cross-page link and the peer-network#dht anchor resolve. 14 locales built, postbuild annotated 130 URLs per locale, 1820 total.
  • npm run build re-run green after the i18n fix, all 14 locales; dist/zh-CN/… and dist/de/docs/protocol/content-replication/index.html both present, and dist/de/docs/protocol-deep-dive/ still renders the original German description.
  • npm test28 passed, including @axe-core/playwright WCAG 2.2 AA checks on protocol pages in both en and de, both colour modes, and mobile viewport.
  • The page is present in dist/sitemap.xml and renders at dist/docs/protocol/content-replication/index.html.

Version

0.16.1 -> 0.17.0 — minor, per SemVer: additive new content and a new navigation entry, nothing removed or renamed.

Adds the L7 protocol page specifying how content spreads with no origin
server or CDN: the connect -> discover -> fetch+verify -> cache -> announce
loop that turns every reader into a discoverable holder.

Covers the two discovery halves (the distributed iterative Kademlia
find_providers lookup and the real-time signed gossip opcode-222 holdings
announce), why lookup-learned records are hearsay used only for fetching,
the announce-ingress guards per threat class, per-range merkle verification
bound to the chain-anchored root, the bounded LRU capsule cache and its two
landing paths, the first-party landing gate, the active retract on eviction,
and a conformance list.

States plainly that this iterative provider lookup is a different mechanism
from a recursive hop-by-hop ask.

Wires the page into the Protocol sidebar, the deep-dive index, llms.txt, and
all 13 locale doc trees.

@MichaelTaylor3d MichaelTaylor3d left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CHANGES-REQUIRED (correctness gate, docs tier — head 14699187, which has MOVED since dispatch quoted d434eaf2; findings below are resolved against 14699187).

Note: a same-identity PR 422s any verdict event, so this is recorded as a --comment review. The two inline threads are what bar the merge.

The English page itself is excellent and I am not blocking on it. Every technical claim I could check verified against origin/main crate source:

  • 1 GiB default cache capacity + pin-exempt eviction -> dig-store-cache/src/config.rs:8, cache.rs:175
  • opcode 222 = HoldingsAnnounce -> dig-gossip/src/connection/inbound_limits.rs:21
  • dig:holdings:v1 preimage, ECDSA-P256 TLS leaf key, peer_id == SHA-256(SPKI DER) verified on receive -> dig-gossip/src/service/holdings_announce.rs:75, module doc lines 18-50
  • distributed iterative find_providers, local record as fast path only, NC-12 "accepted because it verifies against the merkle root, never because a peer supplied it" -> dig-dht/src/service.rs:204 and its doc comment
  • discovered records are hearsay, used on the fetch path, never re-served/republished -> dig-dht/src/service.rs:247-250, provider_store.rs:317

Trap 1 (DHT vs recursive hop-by-hop): CLEAR. Section 2c is an explicit four-row table distinguishing iterative-lookup-returning-records from a transitive ask that holds each hop open, including who carries the bytes and who bears the cost, closing with "A statement about one is not a statement about the other." The llms.txt entry repeats the distinction. No conflation.

Trap 2 (NC-12): STATED, both halves. "Every peer is untrusted... accepted because it verifies against the merkle root and the chain-anchored generation, never because of who served it", and separately "Verified is not the same as safe."

§4.3: clean. Zero hits for SYSTEM.md / CLAUDE.md / SPEC.md / TASKS.md / DEVELOPMENT_LOG.md / PROTOCOL_GAPS.md / -per-systemmd / dig_ecosystem / "we decided" / "formerly" / "this is NOT", and zero #NNN ticket numbers in added lines (positive control on the same grep returned 72 find_providers hits, so the search works). Voice is direct and positive. Sits at protocol-developer tier, correctly.

Baseline: mostly good. sidebars.ts links protocol/content-replication (page is reachable); all three JSON files parse; llms.txt gains a matching entry; package.json 0.16.1 -> 0.17.0 with openrpc*.json realigned 0.15.0 -> 0.17.0; unique title + description + keywords + tags; heading structure is clean. No CLAUDE.md/AGENTS.md/SKILL.md or gitnexus strays in the 34-file list (read the list, not the count). No corrupted whitespace runs (control fired).

Locale count: 13 non-English files + the English source = 14, matching the configured set in docusaurus.config.ts (en, zh-CN, zh-TW, ko, ja, ru, es, pt-BR, fr, de, tr, vi, id, hi). The count is right; the CONTENT is not — see the two threads.

Merge is blocked on the two i18n findings only. Not merging, not undrafting.

Comment thread i18n/de/docusaurus-plugin-content-docs/current/protocol-deep-dive.md Outdated
Comment thread i18n/ja/docusaurus-plugin-content-docs/current/protocol/content-replication.md Outdated
Two regressions from the previous commit, both i18n:

1. All 13 locale copies of protocol-deep-dive.md had been regenerated from
   the English source, replacing real translated prose. Each is now reverted
   to its origin/main content with ONLY the two-line addition applied, and
   that addition is translated per locale rather than left in English.

2. The 13 locale copies of the new content-replication page were verbatim
   English. Docusaurus renders a present locale file as a COMPLETED
   translation, so an English file under a locale name gives no fallback
   signal and no future job can detect the gap. They are removed, so the
   page falls back to English visibly and self-correctingly.

The new page and the locale nav entries use extensionless relative links.
A .md link does not resolve across locale fallback, so the fallback page's
own cross-links broke in every non-English build; the URL form resolves
against the current locale's path and builds clean in all 14.
@MichaelTaylor3d
MichaelTaylor3d marked this pull request as ready for review August 24, 2026 15:55
@MichaelTaylor3d
MichaelTaylor3d merged commit 6a167e7 into main Aug 24, 2026
8 checks passed
@MichaelTaylor3d
MichaelTaylor3d deleted the loop/1425-flywheel-docs branch August 24, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant