Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/protocol/peer-network.md
Original file line number Diff line number Diff line change
Expand Up @@ -813,6 +813,12 @@ A content read (`dig.getContent`, `dig.fetchRange`, or a peer range stream) may
- **FETCH-THROUGH (opt-in, `DIG_NODE_ON_MISS=fetch`).** An operator-level setting: instead of redirecting, the node **pulls** the resource from the holder(s) via the [multi-source download](#multi-source) (verified against the chain-anchored root), caches it, and **serves it directly** — transparent to every caller, one round-trip, and the node now holds it. If the fetch fails the node falls back to the redirect (still never a silent 404 while a holder exists).

- **PROXY (opt-in per request, `params.proxy`, default `false`).** A caller-level alternative to fetch-through, for a caller that cannot itself reach any named holder (e.g. NAT asymmetry): setting `proxy: true` on `dig.getContent` / `dig.fetchRange` asks THIS node to fetch the resource, over the identical chain-anchored, merkle-verified path, and return the bytes directly instead of redirecting. Automatic fetch-on-miss stays off by default — the caller opts in per request. The node serves the bytes but does **not** become a holder of the content by doing so (a proxied read never triggers reshare/backfill), so `proxy: true` cannot be used to plant attacker-chosen inventory on a node. A miss already reveals the requested content id to the node you ask; setting `proxy: true` additionally reveals to the serving holder that you wanted that resource — the same disclosure a direct read from that holder would make.
- **FORWARDED ASK (opt-in, `DIG_NODE_FORWARD_ON_MISS`, default OFF).** An operator-level setting: on a miss, the node also asks its own connected peers whether THEY hold the content, merging any holders they name into the redirect. Unset, empty, or any value other than `on`/`1`/`true`/`yes` (case-insensitive) leaves it disabled — a misspelled value cannot turn it on. Off, a miss is answered from this node's own DHT lookup alone, exactly as if the setting did not exist.
Comment thread
MichaelTaylor3d marked this conversation as resolved.

**Enabling it is a real amplification decision, not a small one.** A relay token is charged once per admitted miss and buys several outbound asks; those asks recruit further peers who each ask further peers again, so **one admitted request can fan out to on the order of 1,300+ outbound dials and DHT lookups across the network**, not just at this node. That cost is borne by every node downstream of you, not only your own, which is why the setting defaults off and why turning it on is an operator decision to make deliberately, not a knob to flip for a marginal cache-hit improvement.

**It also tells your peers what you are looking for.** Asking a peer "do you hold X?" discloses X to that peer, the same disclosure a direct read from a holder already makes — but with the forwarded ask, that disclosure now reaches every peer recruited into the fan-out, not just the one peer you originally asked.

Comment thread
MichaelTaylor3d marked this conversation as resolved.

**The redirect is a READ-TIER response.** It carries only what the caller needs to reach a holder — it exposes no peer/write/control surface — so it is served on both the mTLS peer/control tier and, where the read path is public ([§0](#dual-transport)), the anonymous read tier.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,12 @@ A content read (`dig.getContent`, `dig.fetchRange`, or a peer range stream) may

- **FETCH-THROUGH (opt-in, `DIG_NODE_ON_MISS=fetch`).** Instead of redirecting, the node **pulls** the resource from the holder(s) via the [multi-source download](#multi-source) (verified against the chain-anchored root), caches it, and **serves it directly** — transparent to the caller, one round-trip, and the node now holds it. If the fetch fails the node falls back to the redirect (still never a silent 404 while a holder exists).

- **FORWARDED ASK (opt-in, `DIG_NODE_FORWARD_ON_MISS`, default OFF).** An operator-level setting: on a miss, the node also asks its own connected peers whether THEY hold the content, merging any holders they name into the redirect. Unset, empty, or any value other than `on`/`1`/`true`/`yes` (case-insensitive) leaves it disabled — a misspelled value cannot turn it on. Off, a miss is answered from this node's own DHT lookup alone, exactly as if the setting did not exist.

**Enabling it is a real amplification decision, not a small one.** A relay token is charged once per admitted miss and buys several outbound asks; those asks recruit further peers who each ask further peers again, so **one admitted request can fan out to on the order of 1,300+ outbound dials and DHT lookups across the network**, not just at this node. That cost is borne by every node downstream of you, not only your own, which is why the setting defaults off and why turning it on is an operator decision to make deliberately, not a knob to flip for a marginal cache-hit improvement.

**It also tells your peers what you are looking for.** Asking a peer "do you hold X?" discloses X to that peer, the same disclosure a direct read from a holder already makes — but with the forwarded ask, that disclosure now reaches every peer recruited into the fan-out, not just the one peer you originally asked.

**The redirect is a READ-TIER response.** It carries only what the caller needs to reach a holder — it exposes no peer/write/control surface — so it is served on both the mTLS peer/control tier and, where the read path is public ([§0](#dual-transport)), the anonymous read tier.

**Bounded hops (no loops).** The redirect carries a `redirect_depth` (the number of redirects already followed) and the `max_redirects` cap. A caller **echoes `redirect_depth`** in its re-request `params`; a node that receives a request already at/over the cap answers the plain not-found instead of another redirect, so a set of nodes can never bounce a caller in a loop. A node **never redirects a caller back to itself** (its own `peer_id` is excluded from the providers).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,12 @@ A content read (`dig.getContent`, `dig.fetchRange`, or a peer range stream) may

- **FETCH-THROUGH (opt-in, `DIG_NODE_ON_MISS=fetch`).** Instead of redirecting, the node **pulls** the resource from the holder(s) via the [multi-source download](#multi-source) (verified against the chain-anchored root), caches it, and **serves it directly** — transparent to the caller, one round-trip, and the node now holds it. If the fetch fails the node falls back to the redirect (still never a silent 404 while a holder exists).

- **FORWARDED ASK (opt-in, `DIG_NODE_FORWARD_ON_MISS`, default OFF).** An operator-level setting: on a miss, the node also asks its own connected peers whether THEY hold the content, merging any holders they name into the redirect. Unset, empty, or any value other than `on`/`1`/`true`/`yes` (case-insensitive) leaves it disabled — a misspelled value cannot turn it on. Off, a miss is answered from this node's own DHT lookup alone, exactly as if the setting did not exist.

**Enabling it is a real amplification decision, not a small one.** A relay token is charged once per admitted miss and buys several outbound asks; those asks recruit further peers who each ask further peers again, so **one admitted request can fan out to on the order of 1,300+ outbound dials and DHT lookups across the network**, not just at this node. That cost is borne by every node downstream of you, not only your own, which is why the setting defaults off and why turning it on is an operator decision to make deliberately, not a knob to flip for a marginal cache-hit improvement.

**It also tells your peers what you are looking for.** Asking a peer "do you hold X?" discloses X to that peer, the same disclosure a direct read from a holder already makes — but with the forwarded ask, that disclosure now reaches every peer recruited into the fan-out, not just the one peer you originally asked.

**The redirect is a READ-TIER response.** It carries only what the caller needs to reach a holder — it exposes no peer/write/control surface — so it is served on both the mTLS peer/control tier and, where the read path is public ([§0](#dual-transport)), the anonymous read tier.

**Bounded hops (no loops).** The redirect carries a `redirect_depth` (the number of redirects already followed) and the `max_redirects` cap. A caller **echoes `redirect_depth`** in its re-request `params`; a node that receives a request already at/over the cap answers the plain not-found instead of another redirect, so a set of nodes can never bounce a caller in a loop. A node **never redirects a caller back to itself** (its own `peer_id` is excluded from the providers).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,12 @@ A content read (`dig.getContent`, `dig.fetchRange`, or a peer range stream) may

- **FETCH-THROUGH (opt-in, `DIG_NODE_ON_MISS=fetch`).** Instead of redirecting, the node **pulls** the resource from the holder(s) via the [multi-source download](#multi-source) (verified against the chain-anchored root), caches it, and **serves it directly** — transparent to the caller, one round-trip, and the node now holds it. If the fetch fails the node falls back to the redirect (still never a silent 404 while a holder exists).

- **FORWARDED ASK (opt-in, `DIG_NODE_FORWARD_ON_MISS`, default OFF).** An operator-level setting: on a miss, the node also asks its own connected peers whether THEY hold the content, merging any holders they name into the redirect. Unset, empty, or any value other than `on`/`1`/`true`/`yes` (case-insensitive) leaves it disabled — a misspelled value cannot turn it on. Off, a miss is answered from this node's own DHT lookup alone, exactly as if the setting did not exist.

**Enabling it is a real amplification decision, not a small one.** A relay token is charged once per admitted miss and buys several outbound asks; those asks recruit further peers who each ask further peers again, so **one admitted request can fan out to on the order of 1,300+ outbound dials and DHT lookups across the network**, not just at this node. That cost is borne by every node downstream of you, not only your own, which is why the setting defaults off and why turning it on is an operator decision to make deliberately, not a knob to flip for a marginal cache-hit improvement.

**It also tells your peers what you are looking for.** Asking a peer "do you hold X?" discloses X to that peer, the same disclosure a direct read from a holder already makes — but with the forwarded ask, that disclosure now reaches every peer recruited into the fan-out, not just the one peer you originally asked.

**The redirect is a READ-TIER response.** It carries only what the caller needs to reach a holder — it exposes no peer/write/control surface — so it is served on both the mTLS peer/control tier and, where the read path is public ([§0](#dual-transport)), the anonymous read tier.

**Bounded hops (no loops).** The redirect carries a `redirect_depth` (the number of redirects already followed) and the `max_redirects` cap. A caller **echoes `redirect_depth`** in its re-request `params`; a node that receives a request already at/over the cap answers the plain not-found instead of another redirect, so a set of nodes can never bounce a caller in a loop. A node **never redirects a caller back to itself** (its own `peer_id` is excluded from the providers).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,12 @@ A content read (`dig.getContent`, `dig.fetchRange`, or a peer range stream) may

- **FETCH-THROUGH (opt-in, `DIG_NODE_ON_MISS=fetch`).** Instead of redirecting, the node **pulls** the resource from the holder(s) via the [multi-source download](#multi-source) (verified against the chain-anchored root), caches it, and **serves it directly** — transparent to the caller, one round-trip, and the node now holds it. If the fetch fails the node falls back to the redirect (still never a silent 404 while a holder exists).

- **FORWARDED ASK (opt-in, `DIG_NODE_FORWARD_ON_MISS`, default OFF).** An operator-level setting: on a miss, the node also asks its own connected peers whether THEY hold the content, merging any holders they name into the redirect. Unset, empty, or any value other than `on`/`1`/`true`/`yes` (case-insensitive) leaves it disabled — a misspelled value cannot turn it on. Off, a miss is answered from this node's own DHT lookup alone, exactly as if the setting did not exist.

**Enabling it is a real amplification decision, not a small one.** A relay token is charged once per admitted miss and buys several outbound asks; those asks recruit further peers who each ask further peers again, so **one admitted request can fan out to on the order of 1,300+ outbound dials and DHT lookups across the network**, not just at this node. That cost is borne by every node downstream of you, not only your own, which is why the setting defaults off and why turning it on is an operator decision to make deliberately, not a knob to flip for a marginal cache-hit improvement.

**It also tells your peers what you are looking for.** Asking a peer "do you hold X?" discloses X to that peer, the same disclosure a direct read from a holder already makes — but with the forwarded ask, that disclosure now reaches every peer recruited into the fan-out, not just the one peer you originally asked.

**The redirect is a READ-TIER response.** It carries only what the caller needs to reach a holder — it exposes no peer/write/control surface — so it is served on both the mTLS peer/control tier and, where the read path is public ([§0](#dual-transport)), the anonymous read tier.

**Bounded hops (no loops).** The redirect carries a `redirect_depth` (the number of redirects already followed) and the `max_redirects` cap. A caller **echoes `redirect_depth`** in its re-request `params`; a node that receives a request already at/over the cap answers the plain not-found instead of another redirect, so a set of nodes can never bounce a caller in a loop. A node **never redirects a caller back to itself** (its own `peer_id` is excluded from the providers).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,12 @@ A content read (`dig.getContent`, `dig.fetchRange`, or a peer range stream) may

- **FETCH-THROUGH (opt-in, `DIG_NODE_ON_MISS=fetch`).** Instead of redirecting, the node **pulls** the resource from the holder(s) via the [multi-source download](#multi-source) (verified against the chain-anchored root), caches it, and **serves it directly** — transparent to the caller, one round-trip, and the node now holds it. If the fetch fails the node falls back to the redirect (still never a silent 404 while a holder exists).

- **FORWARDED ASK (opt-in, `DIG_NODE_FORWARD_ON_MISS`, default OFF).** An operator-level setting: on a miss, the node also asks its own connected peers whether THEY hold the content, merging any holders they name into the redirect. Unset, empty, or any value other than `on`/`1`/`true`/`yes` (case-insensitive) leaves it disabled — a misspelled value cannot turn it on. Off, a miss is answered from this node's own DHT lookup alone, exactly as if the setting did not exist.

**Enabling it is a real amplification decision, not a small one.** A relay token is charged once per admitted miss and buys several outbound asks; those asks recruit further peers who each ask further peers again, so **one admitted request can fan out to on the order of 1,300+ outbound dials and DHT lookups across the network**, not just at this node. That cost is borne by every node downstream of you, not only your own, which is why the setting defaults off and why turning it on is an operator decision to make deliberately, not a knob to flip for a marginal cache-hit improvement.

**It also tells your peers what you are looking for.** Asking a peer "do you hold X?" discloses X to that peer, the same disclosure a direct read from a holder already makes — but with the forwarded ask, that disclosure now reaches every peer recruited into the fan-out, not just the one peer you originally asked.

**The redirect is a READ-TIER response.** It carries only what the caller needs to reach a holder — it exposes no peer/write/control surface — so it is served on both the mTLS peer/control tier and, where the read path is public ([§0](#dual-transport)), the anonymous read tier.

**Bounded hops (no loops).** The redirect carries a `redirect_depth` (the number of redirects already followed) and the `max_redirects` cap. A caller **echoes `redirect_depth`** in its re-request `params`; a node that receives a request already at/over the cap answers the plain not-found instead of another redirect, so a set of nodes can never bounce a caller in a loop. A node **never redirects a caller back to itself** (its own `peer_id` is excluded from the providers).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,12 @@ A content read (`dig.getContent`, `dig.fetchRange`, or a peer range stream) may

- **FETCH-THROUGH (opt-in, `DIG_NODE_ON_MISS=fetch`).** Instead of redirecting, the node **pulls** the resource from the holder(s) via the [multi-source download](#multi-source) (verified against the chain-anchored root), caches it, and **serves it directly** — transparent to the caller, one round-trip, and the node now holds it. If the fetch fails the node falls back to the redirect (still never a silent 404 while a holder exists).

- **FORWARDED ASK (opt-in, `DIG_NODE_FORWARD_ON_MISS`, default OFF).** An operator-level setting: on a miss, the node also asks its own connected peers whether THEY hold the content, merging any holders they name into the redirect. Unset, empty, or any value other than `on`/`1`/`true`/`yes` (case-insensitive) leaves it disabled — a misspelled value cannot turn it on. Off, a miss is answered from this node's own DHT lookup alone, exactly as if the setting did not exist.

**Enabling it is a real amplification decision, not a small one.** A relay token is charged once per admitted miss and buys several outbound asks; those asks recruit further peers who each ask further peers again, so **one admitted request can fan out to on the order of 1,300+ outbound dials and DHT lookups across the network**, not just at this node. That cost is borne by every node downstream of you, not only your own, which is why the setting defaults off and why turning it on is an operator decision to make deliberately, not a knob to flip for a marginal cache-hit improvement.

**It also tells your peers what you are looking for.** Asking a peer "do you hold X?" discloses X to that peer, the same disclosure a direct read from a holder already makes — but with the forwarded ask, that disclosure now reaches every peer recruited into the fan-out, not just the one peer you originally asked.

**The redirect is a READ-TIER response.** It carries only what the caller needs to reach a holder — it exposes no peer/write/control surface — so it is served on both the mTLS peer/control tier and, where the read path is public ([§0](#dual-transport)), the anonymous read tier.

**Bounded hops (no loops).** The redirect carries a `redirect_depth` (the number of redirects already followed) and the `max_redirects` cap. A caller **echoes `redirect_depth`** in its re-request `params`; a node that receives a request already at/over the cap answers the plain not-found instead of another redirect, so a set of nodes can never bounce a caller in a loop. A node **never redirects a caller back to itself** (its own `peer_id` is excluded from the providers).
Expand Down
Loading
Loading