Skip to content

feat(swarm): add swarm rendezvous messages for clients sharing records - #50

Open
grumbach wants to merge 5 commits into
WithAutonomi:mainfrom
grumbach:feat/peer-swarm
Open

grumbach wants to merge 5 commits into
WithAutonomi:mainfrom
grumbach:feat/peer-swarm

Conversation

@grumbach

@grumbach grumbach commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Wire types for clients that read the same file sharing its records over WebRTC, with nodes introducing them and staying the last seed (ADR-0026): the announce, answer and leave messages of a file's swarm room, their shared limits, and the SDP rule both a relaying node and an applying client use so a stranger cannot point readers' connectivity checks anywhere. A second commit makes ChunkMessage::decode refuse bytes after a message. The branch is rebased onto main as three commits, the swarm messages following its get-or-closer request (ant-node ADR-0020). A fourth commit, from review, classes addresses as the IANA special-purpose registries mark them globally reachable (some, such as 2001:4:112::/48 and NAT64's 64:ff9b::/96, had counted as not public, so a node's branch for members on its own network could relay them), drops every candidate in NAT64's well-known and local-use prefixes (64:ff9b::/96, 64:ff9b:1::/48), and adds is_local_ip, the allowlist a node and a client now hold local candidates to, so an address neither public nor local is kept on no network.

One change across six repositories. Merge order:

  1. ant-protocol feat(swarm): add swarm rendezvous messages for clients sharing records #50, saorsa-transport feat(webrtc): answer plain STUN Binding requests on the WebRTC Direct socket saorsa-transport#231 and saorsa-core feat(network): send a message on one named channel only saorsa-core#183, independent of each other
  2. ant-node feat(swarm): host a swarm rendezvous on every node ant-node#262
  3. ant-client feat(swarm): share records with other readers of a file, in every client ant-client#244
  4. Try https://github.com/WithAutonomi/try/pull/21, once a browser SDK release carries the swarm

This PR depends on: none.

Linear issue

Closes V2-1892

Risk tier

  • T0 — docs / tooling / CI / pure UX-output. Repo CI only.
  • T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
  • T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
  • T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Proposed; a human confirms at review.

Compatibility

  • Wire: six ChunkMessageBody variants appended (tags 16 to 21, after main's get-or-closer pair); released nodes (v0.23.1, and v0.24.0 of the 2026.11.1 release) cannot decode them, log a warning and leave them unanswered (no panic or ERROR line from v0.23.1 in the rollout lab; v0.24.0 handles an undecodable message the same way), and clients pass those nodes over. ChunkMessage::decode refuses bytes after a message, which no sender writes.
  • Storage: none
  • API: new swarm module, limits, constrain_swarm_sdp and its address classes (canonical_ip, is_global_ip, is_local_ip); ChunkMessageBody is #[non_exhaustive].

Semver impact

  • breaking
  • feature
  • fix

Test evidence

  • cargo test --lib: 132 passed at 4bf8dc9, on main 1179782 (the 2026.11.1 release, ant-protocol 6.0.0, which carries ADR-0020's get-or-closer request): three commits with the same code as the reviewed 5c5a1f1, and the address-class fix. cargo build --no-default-features builds. cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings, cargo clippy -- -D warnings and RUSTDOCFLAGS="-D warnings" cargo doc --no-deps: clean (Rust 1.99).
  • New tests pin every new wire tag, the shared limits, decoding that refuses lists past their limit and bytes after a message, and constrain_swarm_sdp: only the standard form of a candidate or fingerprint line stays (other spellings, Unicode whitespace, stray carriage returns, out-of-range components and priorities go), a mostly dropped SDP holds only what it keeps, and the address classes: public where the IANA special-purpose registries mark an address globally reachable (the first and last address of every prefix the fix changed, 2001:4:112::/48, 192.0.0.9 and NAT64's 64:ff9b::/96 among them, and each one's neighbours), with Teredo, which the registries leave undecided, not public and 6to4, also undecided, public; local only for private use, shared, link-local, loopback and unique local addresses; no listed address both; and no candidate in 64:ff9b::/96 or 64:ff9b:1::/48 kept whatever a consumer allows. The SDP and address tests fail with each fix reverted.
  • Adversarial: ant-core parses SDPs this filter let through with webrtc-rs's own SDP and ICE candidate parsers (fingerprints, candidate addresses, candidate priorities).
  • Consumers at their PRs' heads: ant-node 48e2761f 1311 lib tests and its WebRTC Direct devnet; ant-core b6f4a8c 989 lib tests, e2e_swarm and 261 WASM tests, all passing.
  • CI at 4bf8dc9: every build, test and lint job passes; Security Audit fails. It fails only on RUSTSEC-2026-0387 (crypto-bigint 0.5.5, through elliptic-curve 0.13), which cargo audit reports on this repository's main lockfile too, with the same dependents; its fixes wait on alloy and webrtc-rs releases. linear-link and pr-template, which check this description, pass.

New dependency

none

ADR

https://github.com/grumbach/ant-node/blob/feat/peer-swarm/docs/adr/ADR-0026-clients-share-records-nodes-introduce-them.md (new in the ant-node PR, status Proposed)

Mitigation / rollback

Revert: nothing sends the new messages until the node and client PRs land.

@grumbach
grumbach marked this pull request as ready for review November 2, 2026 19:27
@grumbach
grumbach marked this pull request as draft November 5, 2026 04:11
grumbach added a commit to grumbach/ant-node that referenced this pull request Nov 5, 2026
Patch ant-protocol, saorsa-core and saorsa-transport to their
feat/peer-swarm branches, which this change needs: the swarm messages
(WithAutonomi/ant-protocol#50), the send on one named channel
(WithAutonomi/saorsa-core#183) and the plain STUN answers
(WithAutonomi/saorsa-transport#231). Once those merge, these point at
their main branches, or at the released versions at the release train.
grumbach added a commit to grumbach/ant-client that referenced this pull request Nov 5, 2026
Patch ant-protocol, saorsa-core, saorsa-transport and ant-node to their
feat/peer-swarm branches, which this change needs: the swarm messages
(WithAutonomi/ant-protocol#50), the send on one named channel
(WithAutonomi/saorsa-core#183), the plain STUN answers
(WithAutonomi/saorsa-transport#231) and the rendezvous on every node
(WithAutonomi/ant-node#262). Once those merge, these point at their
main branches, or at the released versions at the release train.

Cargo.lock starts from main's lock and differs from it only where those
branches need it: the swarm's WebRTC and STUN crates, saorsa-transport's
main dropping its CLI (and its clap and reqwest dependencies), and
ant-node's main adding winapi-util on Windows. iana-time-zone keeps the
windows-core 0.62.2 main's lock has.
grumbach added a commit to grumbach/ant-node that referenced this pull request Nov 5, 2026
Patch ant-protocol, saorsa-core and saorsa-transport to their
feat/peer-swarm branches, which this change needs: the swarm messages
(WithAutonomi/ant-protocol#50), the send on one named channel
(WithAutonomi/saorsa-core#183) and the plain STUN answers
(WithAutonomi/saorsa-transport#231). Once those merge, these point at
their main branches, or at the released versions at the release train.
grumbach added a commit to grumbach/ant-client that referenced this pull request Nov 5, 2026
Patch ant-protocol, saorsa-core, saorsa-transport and ant-node to their
feat/peer-swarm branches, which this change needs: the swarm messages
(WithAutonomi/ant-protocol#50), the send on one named channel
(WithAutonomi/saorsa-core#183), the plain STUN answers
(WithAutonomi/saorsa-transport#231) and the rendezvous on every node
(WithAutonomi/ant-node#262). Once those merge, these point at their
main branches, or at the released versions at the release train.

Cargo.lock starts from main's lock and differs from it only where those
branches need it: the swarm's WebRTC and STUN crates, saorsa-transport's
main dropping its CLI (and its clap and reqwest dependencies), and
ant-node's main adding winapi-util on Windows. iana-time-zone keeps the
windows-core 0.62.2 main's lock has.
Clients reading the same file can share its records with each other over
WebRTC data channels, with nodes introducing them and staying the last
seed. This adds the wire types for that introduction.

- `swarm` module: `RoomKey`, `MemberId`, `OfferId`, `NodeId` and
  `Membership`, and the announce, answer and leave requests with their
  replies. An announce joins or refreshes a room, hands the node offers for
  other members and collects the signals waiting for this member,
  optionally waiting for one.
- Every request names the rendezvous node it is for, so a request one node
  saw cannot be replayed at another.
- A membership number in announce and leave lets a node keep each
  connection's newest membership of a room: a newer one replaces the old,
  an ended or older one is refused (`SwarmRefusal::Superseded`), and a
  leave ends only the membership it names. A node browsers cannot reach
  refuses with `SwarmRefusal::NotHosting`, which a client takes as "pass
  this node over now".
- Shared limits both sides enforce: offers per announce, SDP size, inbox
  size, reply size, wait time, ICE servers, offers counted as delivered,
  the request size a node refuses before decoding, and the length of a
  reply's error. The `validate` methods of the requests and of all three
  replies check them; the offers, inbox and STUN server lists are refused
  while decoding once past their limit, so nothing longer is ever built.
- `SWARM_SIGNAL_TTL_MS` (70 s): how long a node keeps a delivered offer for
  its answer and a signal for its member, which a client checks its own
  answering time against.
- `constrain_swarm_sdp`: the ICE candidates an SDP from another client may
  keep (UDP, an IP that is not unspecified, multicast or broadcast, or an
  mDNS name, at most 16), with a caller's rule on top. A node relaying an
  SDP and a client applying one both use it, so readers' connectivity
  checks cannot be pointed at a third party. Receivers read SDP leniently
  (webrtc-rs trims an attribute's value, Chrome takes a name followed by a
  space, Firefox names attributes in any case, webrtc-ice splits fields on
  any Unicode whitespace), so a line any of them may read as a candidate
  or a DTLS fingerprint stays only in its standard form (printable ASCII
  words separated by single spaces; a candidate's component 1 to 256 and
  its priority a 32-bit number, so no ICE agent's ranking of it
  overflows), and a line with a carriage return anywhere but before its
  line feed goes: what a receiver reads is exactly what was checked, and a
  link's binding covers every fingerprint a receiver uses. The result
  holds no more memory than its length needs. `is_global_ip` and
  `canonical_ip` help both apply their rules.
- Six `ChunkMessageBody` variants appended as discriminants 14 to 19, so
  existing messages keep their encoding. Tests pin every new tag.
`ChunkMessage::decode` took the message postcard decodes from the front of
the input and ignored whatever followed. A sender could therefore pad a
small message, such as a valid reply, out to the 5 MiB wire limit and still
have it taken, so a receiver's bound on a reply was really the wire limit.

Decoding now refuses any bytes after the message's end, as
`DeserializationFailed`. Every sender encodes a message exactly, so nothing
well formed changes; `test_decode_rejects_bytes_after_the_message` pins it.
The `RoomKey` doc said a public file's room is its address. A client
derives the room from the file's root DataMap, once a shrunk map is
resolved to its root, as ADR-0026 says: a public file published with a
shrunk map has a room other than its address, and a client that followed
the doc would wait in a room no other reader joins. The doc now says so.
grumbach added a commit to grumbach/ant-node that referenced this pull request Nov 6, 2026
Patch ant-protocol, saorsa-core and saorsa-transport to their
feat/peer-swarm branches, which this change needs: the swarm messages
(WithAutonomi/ant-protocol#50), the send on one named channel
(WithAutonomi/saorsa-core#183) and the plain STUN answers
(WithAutonomi/saorsa-transport#231). Once those merge, these point at
their main branches, or at the released versions at the release train.

@MaidSafe-Hermes MaidSafe-Hermes left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES — 1 P2 source finding

src/swarm.rs:690–695 misclassifies globally reachable IPv6 destinations as non-global. It excludes all of 2001::/23 and does not include the NAT64 well-known prefix 64:ff9b::/96 in the global-address range. The production consumer in ant-node#262 uses this predicate at src/swarm.rs:543–547: when the observed host is local, the local branch retains candidates classified as non-global. An announce can therefore retain 2001:4:112::1 (IANA marks 2001:4:112::/48 globally reachable) or a NAT64 address such as 64:ff9b::808:808, which represents public IPv4 8.8.8.8 under RFC 6052. A receiving client on a NAT64 network can then send ICE checks to that public destination.

Please classify using an explicit, current special-purpose policy, including globally reachable exceptions and translation prefixes, or use a strict allowlist for local candidates. Add regression cases for these addresses and the local-host filtering path.

Panel: 2/2 seats reviewed. sol-1 raised the address-classification issue; glm-1 did not. I confirmed the production call path and checked the IANA IPv6 Special-Purpose Address Registry and RFC 6052. CI checks were read, not rerun locally.

… allowlist

`is_global_ip` called all of the IETF protocol assignments (`2001::/23`,
`192.0.0.0/24`) and NAT64's well-known prefix (`64:ff9b::/96`)
non-global, although the IANA special-purpose registries mark some of
them globally reachable. A node keeps a member's non-global candidates
when it reaches that member at a local address, so such a member could
have it relay `2001:4:112::1`, or `64:ff9b::808:808` (public 8.8.8.8
through a reader's NAT64), to other readers.

- `is_global_ip` follows the registries: PCP, TURN and DNS-SD SRP
  anycast, AMT, AS112-v6, ORCHIDv2 and drone remote ID entity tags in
  `2001::/23`, PCP and TURN anycast `192.0.0.9` and `192.0.0.10`, and
  `64:ff9b::/96` are global. Teredo stays non-global and 6to4 global.
- `constrain_swarm_sdp` drops every candidate in a NAT64 prefix
  (`64:ff9b::/96`, `64:ff9b:1::/48`) whatever `keep` says, by policy:
  each stands for an IPv4 address that a translator on the receiver's
  network sends to. A client numbered in one by a stateless translator
  meets other readers by its other candidates.
- New `is_local_ip`: private use, shared, link-local, loopback and
  unique local. A node or client that keeps local candidates keeps only
  these, so an address that is neither global nor local (documentation,
  benchmarking, reserved, Teredo, NAT64's local-use prefix) is kept on
  no network, and a class this crate gets wrong cannot pass as local.

Tests check the first and last address of every prefix the change
touches and of every local one, their neighbours, that no listed address
is both global and local, and the NAT64 drop at both prefixes' edges.
@grumbach
grumbach marked this pull request as ready for review November 6, 2026 06:42

@MaidSafe-Hermes MaidSafe-Hermes left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES — reviewed commit 4bf8dc937bd3cb0049ae45fb01491b1c99d5bc0f

P2: shared CGNAT space is treated as proof of a local network. In src/swarm.rs:774–781, is_local_ip returns true for 100.64.0.0/10. The node and client use this result to authorize local SDP candidates. RFC 6598 §1 defines this as shared provider-CGN space between CGN and CPE, not a subscriber LAN; IANA also marks it not globally reachable. A peer's 100.64/10 source therefore does not prove that another member shares its LAN.

Execution path: when a node sees a member at 100.64.0.1, ant-node/src/swarm.rs:545–549 classifies that member as local and retains a 192.168.x.x or .local SDP candidate. A client reached through another shared address then treats reached as local at ant-core/src/swarm/room.rs:1345–1350 and applies that candidate; ICE can probe the receiving client's private network. The supplied tests cover documentation, benchmarking and NAT64 origins, but not shared-CGNAT origins.

Minimum fix: do not let 100.64.0.0/10 alone satisfy the local-network predicate used for candidate authorization. Add tests showing that shared-CGNAT source/reached addresses do not unlock private, ULA, link-local or mDNS candidates; if support is required, use a stronger locality signal than CGNAT membership.

Coverage: 2/2 seats reviewed this exact head's address-classification delta (sol-1 gpt-6.1-sol xhigh; glm-1 z-ai/glm-5.3). Both approved the IPv6/NAT64 classification changes. This shared-CGNAT trust-boundary finding was confirmed separately against the exact callers and RFC/IANA definitions.

`is_local_ip` counted shared address space (`100.64.0.0/10`, RFC 6598)
as local. A provider numbers the subscribers behind its carrier-grade
NAT in it, so a member a node reaches at `100.64.0.1`, or a node a
client reached at one, shares no network with the readers it would
introduce: a node would relay such a member's private, unique local,
link-local and mDNS candidates, and a client would apply them, pointing
other readers' checks at their own home networks.

Shared space is now neither local nor global, like documentation and
benchmarking addresses: it unlocks no local candidates and is kept on no
network. The tests list its first and last address, `100.64.0.1` and its
IPv4-mapped form as neither, and its neighbours as global.
MaidSafe-Hermes pushed a commit to grumbach/ant-client that referenced this pull request Nov 6, 2026
Patch ant-protocol, saorsa-core, saorsa-transport and ant-node to their
feat/peer-swarm branches, which this change needs: the swarm messages
(WithAutonomi/ant-protocol#50), the send on one named channel
(WithAutonomi/saorsa-core#183), the plain STUN answers
(WithAutonomi/saorsa-transport#231) and the rendezvous on every node
(WithAutonomi/ant-node#262). Once those merge, these point at their
main branches, or at the released versions at the release train.

Cargo.lock starts from main's lock and differs from it only where those
branches need it: the swarm's WebRTC and STUN crates, saorsa-transport's
main dropping its CLI (and its clap and reqwest dependencies), and
ant-node's main adding winapi-util on Windows. iana-time-zone keeps the
windows-core 0.62.2 main's lock has.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants