-
Notifications
You must be signed in to change notification settings - Fork 32
SWIP-060: BPS singlehop — brokered broadcast pub/sub, base protocol #104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
d0fd952
25f6f08
77f6088
7481286
22e8325
4ea5c9e
98e8918
10df5e9
87f6b71
7a59348
8e770b1
5ebfb18
be251a5
0b51ad4
5a5e0e9
8f7e720
0e4d835
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,201 @@ | ||
| // Broadcast Pub/Sub (BPS) — protocol messages and types. | ||
| // Spec: SWIP-60 (../../swip-60.md), extending the base wire of SWIP-74 (BPS-lite). | ||
| // | ||
| // Revision 14 (2026-10-01), per Viktor — the chunk is an ordinary single-owner chunk | ||
| // with its full id; the frame carries what the id was derived from (kind, challenge, | ||
| // index) beside it; the challenge is 32 bytes; service messages are kinds, not | ||
| // payloads; AUTH is the empty claim; a chunk that does not validate disconnects. | ||
| // | ||
| // SWIP-74 fixes the base: three frames (Join, Ack, Broadcast) and the two types they | ||
| // carry (CohortSpec, Kind with DATA and AUTH), for a single publisher over a feed at | ||
| // one broker, one hop, with every chunk signed under an id salted by the challenge | ||
| // the broker issued for its stream, so that the first valid frame claims the | ||
| // publisher role. This file adds what the full singlehop protocol needs and changes | ||
| // nothing SWIP-74 defines: | ||
| // - CohortSpec gains `publishers` (one value, ALL), `history` and `closed`; | ||
| // - Kind gains the admin's service kinds, EOS and ROSTER, and Roster is the | ||
| // payload of a ROSTER chunk. | ||
| // Field numbers follow SWIP-74's; the added fields come after. There is no envelope: | ||
| // what a frame is follows from the stream's direction and role, and what a chunk is | ||
| // from the frame's kind. Multihop (SWIP-61) adds its control frames as messages of | ||
| // its own. | ||
| // | ||
| // Enum zero values (*_UNSPECIFIED): proto3 requires a zero value; it is | ||
| // deliberately NOT a legitimate wire value. It exists so that an unset field is | ||
| // detectable and no implementation can silently rely on a default. Receivers MUST | ||
| // reject messages carrying it -- PublisherRegime excepted, where unset is a value. | ||
| // | ||
| // Implementation: bee PR #5626. | ||
|
|
||
| syntax = "proto3"; | ||
| package bps; | ||
|
|
||
| option go_package = "github.com/ethersphere/bee/v2/pkg/bps/pb"; | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // Cohort genesis — immutable policy, and the cohort's identity: cohorts are keyed | ||
| // by the spec's canonical serialisation (fields in number order, unset fields not | ||
| // emitted). The roster is NOT here (see Kind). | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| // What the topic binds to (see SWIP-60: binding semantics). SWIP-74 defines | ||
| // FEED_TOPIC alone; the numbers are shared. | ||
| enum TopicBinding { | ||
| TOPIC_BINDING_UNSPECIFIED = 0; // invalid on the wire (see header note) | ||
| ANCHOR = 1; // topic = full SOC/GSOC address; dedup on the wrapped CAC | ||
| SOC_ID = 2; // topic = SOC id; any owner with PO(addr, anchor) >= PO_MIN | ||
| OWNER = 3; // topic = keccak256(owner); any id, same PO constraint (MIC) | ||
| FEED_TOPIC = 4; // a feed on the topic; on the wire the session feed (see | ||
| // Broadcast) | ||
| MNEMONIC = 5; // the topic names the cohort and constrains nothing: under | ||
| // implicit authorship any SOC from any owner qualifies (dedup | ||
| // on chunk address), under ALL any owner's session chunk. What | ||
| // PublisherRegime.ALL needs -- authorship unrestricted, but | ||
| // never unattributable, since every message is SOC-signed. | ||
| } | ||
|
|
||
| // One value. Set: anyone attached may publish (group chat) -- no claim; a stream | ||
| // declares the address it publishes as, and every message it sends is validated | ||
| // against it. Unset, with an admin: the admin publishes, and whoever its roster | ||
| // ever names -- a cohort is multi-publisher iff a ROSTER is ever published, and | ||
| // nobody needs to know in advance. With no admin the cohort is implicit: | ||
| // authorship follows the binding's SOC shape and this does not apply. | ||
| enum PublisherRegime { | ||
| PUBLISHER_REGIME_UNSPECIFIED = 0; // unset: the admin and whoever its roster names | ||
| // -- the one zero value legitimate on the wire | ||
| ALL = 1; // anyone attached; needs MNEMONIC binding | ||
| } | ||
|
|
||
| // Fixed by whoever joins first; immutable; keyed as a whole -- two specs that | ||
| // differ in any field are two cohorts, even on one topic. | ||
| // NOTE: broker capacity is NOT a cohort parameter -- a cohort cannot dictate a | ||
| // remote node's connection count. Each broker enforces its own bounds and | ||
| // answers FULL when one is exhausted. | ||
| // NOTE: the proximity constraint for implicit bindings is a protocol constant, | ||
| // PO_MIN = 16 -- not a cohort parameter (a proto3 unset uint32 is | ||
| // indistinguishable from 0, which would silently disable the constraint; and | ||
| // no use case varies it). | ||
| message CohortSpec { | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the comment is misleading and partially incorrect. the seq diagram in the markdown file defines that actually both publisher and subscriber use the same type of message to connect to a broker. i'm not sure what are my feelings around this. it seems to be too elaborate for both sides to use symmetrically - why does a subscriber need to provide the whole so it begs the question, why shouldn't a state channel opening become its own specific message? and then equally |
||
| bytes topic = 1; // 32 bytes, meaning per binding | ||
| TopicBinding binding = 2; | ||
| bytes admin = 3; // 20-byte eth address: the cohort's authority | ||
| // and always a member of its publisher set. | ||
| // Absent (length 0) => implicit authorship, | ||
| // and `publishers`, `closed` do not apply. | ||
| PublisherRegime publishers = 4; // ALL, or unset (see the enum) | ||
| bool history = 5; // deliver matching chunks from the local store | ||
| bool closed = 6; // no audience: every stream is admitted silent, | ||
| // receiving nothing but a roster naming its addr, | ||
| // and is disconnected unless | ||
| // its first valid frame, from the admin's or a | ||
| // rostered address, arrives within the claim | ||
| // deadline. | ||
| // Unset = open, so that a SWIP-74 spec, which | ||
| // never sets it, reads as an open cohort. | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // Stream establishment, stream name "pubsub/1.0.0" — one stream per | ||
| // (peer, cohort, identity). The first and only handshake frame on a fresh stream | ||
| // is Join; the broker answers with Ack. The first frame settles the cohort; the | ||
| // first valid Broadcast settles the role. | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| // Peer -> broker: the first frame on a fresh stream. Creates the cohort if no live | ||
| // cohort has this spec, attaches to it otherwise. Nothing else is ever in it -- no | ||
| // cursor, no credential: the stream's role follows from what it sends after the | ||
| // Ack. | ||
| message Join { | ||
| CohortSpec cohort = 1; | ||
| bytes addr = 2; // 20 bytes, required: the stream's identity -- under | ||
| // explicit authorship the address whose first valid | ||
| // frame claims the stream; the admin's or a rostered | ||
| // one makes the stream pending (outside the fan-out | ||
| // bound, receiving nothing but the roster that names it | ||
| // until it claims), any other | ||
| // a spectator, which may not publish; under ALL and | ||
| // under implicit authorship the one every publication | ||
| // is validated against, no claim | ||
| } | ||
|
|
||
| enum Status { | ||
| STATUS_UNSPECIFIED = 0; // invalid on the wire (see header note) | ||
| OK = 1; | ||
| FULL = 2; // a capacity bound (per cohort, per broker, per peer | ||
| // connection); a singlehop broker refuses -- nothing | ||
| // else | ||
| REJECTED = 3; // a spec value outside this SWIP, or a malformed Join | ||
| // (addr not 20 bytes) | ||
| } | ||
|
|
||
| // Broker -> peer, answering Join. A non-OK Ack ends the stream. The latest roster | ||
| // is the first Broadcast on a stream when it enters a fan-out set -- at attach for | ||
| // a spectator, at upgrade for a pending or silent stream -- and reaches a pending | ||
| // or silent stream before that only if it names the stream's addr. | ||
| message Ack { | ||
| Status status = 1; | ||
| bytes challenge = 2; // iff OK: 32 bytes drawn at random for this stream -- the | ||
| // salt of its session feeds (SWIP-74); held for the | ||
| // stream's life, never persisted, never reused | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // Broadcast — SOC-only is a protocol feature | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| // What a chunk on this wire is. DATA and AUTH are SWIP-74's; EOS and ROSTER are | ||
| // the admin's control plane: ordinary SOCs on the ordinary path, owned by the | ||
| // admin, so a broker relays them and cannot author them, and a subscriber checks | ||
| // them with the same code as any broadcast. | ||
| enum Kind { | ||
| KIND_UNSPECIFIED = 0; // invalid on the wire (see header note) | ||
| DATA = 1; // a publication | ||
| AUTH = 2; // an empty chunk: claims the stream, says nothing, is | ||
| // never delivered | ||
| EOS = 3; // an empty chunk from the admin, at index 0: the channel | ||
| // is closed, attributably and for good | ||
| ROSTER = 4; // from the admin: the full publisher set as of this | ||
| // index (not a delta); payload = Roster | ||
| } | ||
|
|
||
| // The payload of a ROSTER chunk. | ||
| message Roster { | ||
| repeated bytes publishers = 1; // 20-byte eth addresses: the complete set excl. | ||
| // admin (who is always a publisher). Full state, | ||
| // not a delta, so a reader needs only the latest | ||
| // it can verify. | ||
| } | ||
|
|
||
| // Every frame after the handshake: publisher -> broker a publication, a claim or, | ||
| // from the admin, a service message; broker -> peer a delivery of the same frame. | ||
| // The chunk is an ordinary single-owner chunk, travelling as its chunk data with | ||
| // its full id: | ||
| // soc = id (32) || signature (65) || span (8, LE) || payload (<= 4096) | ||
| // Under explicit authorship and under ALL the frame carries what that id was | ||
| // derived from (SWIP-74): | ||
| // prefix = (empty) kind == DATA | ||
| // = "bps-service:v1" || kind any other kind, kind as one byte | ||
| // topic_s = keccak256(prefix || topic || challenge) | ||
| // id = keccak256(topic_s || index) index as a uint64 big-endian | ||
| // and the receiver derives the id, requires the chunk's to equal it, forms the | ||
| // address the chunk must have from the id and the owner it knows (the stream's | ||
| // claimed or declared address; at a subscriber, the owner it recovers and admits) | ||
| // and validates the chunk against it with the ordinary SOC code. For EOS and | ||
| // ROSTER the owner is the admin, in every configuration; an AUTH is held to the | ||
| // stream's declared address and is never delivered. Each kind is a | ||
| // feed of its own with its own index sequence; under explicit authorship and ALL a | ||
| // publisher's DATA indices increase and are never reused, and every receiver keeps | ||
| // a cursor per publisher. Under implicit authorship there is | ||
| // no challenge: the chunks are the binding's own, `challenge` and `index` are | ||
| // unset, and the chunk's id is whatever the binding's SOC shape says. | ||
| message Broadcast { | ||
| bytes soc = 1; | ||
| Kind kind = 2; | ||
| bytes challenge = 3; // 32 bytes: the challenge of the stream the chunk was | ||
| // signed for | ||
| uint64 index = 4; // the chunk's index on its feed | ||
| } | ||
|
|
||
| // Keepalive / RTT: none at the BPS level. Liveness is the transport's job | ||
| // (libp2p), and latency metrics for reorganisation policies (SWATCH) are | ||
| // sourced there as well. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: i find this whole thing really confusing and not very approachable and i wonder if this even makes sense to do in a first iteration. "pubsub" is very dumb in this sense - it usually does not give you different topic semantics. here, a topic could have different semantics and input validation according to its "type" which makes for a much more complex API surfaces for users later on...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i meant the concept of pubsub usually does not offer different semantics over the concept of a topic. i would appreciate you not hijacking my words and initial intention as this is really counter productive and aggressive. thanks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I quoted your words which indeed were unnecessarily agressive.
As for your original intention, what was it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure the semantics of topic or pubsub changes here, I thinkk the various bindings merely link the updates on a topic differently to each other as well as allow for multiple sources