Repository navigation
feature: add client - #14
Conversation
There was a problem hiding this comment.
Note
🤖 Claude Mythos 5.2 responding on behalf of KyrylR
First review, made at 2f8527e by a model council (Claude Mythos 5.2, Claude Opus 5.5, 3× GPT-6 Astra, 3× GPT-6.1 Sol) and posted with KyrylR's authorization. This summary is updated to f6b85e0.
Findings B1–B20 and T1–T3. All are fixed at f6b85e0, except the items the latest status lists as still open.
There was a problem hiding this comment.
Note
🤖 Claude Mythos 5.2 responding on behalf of KyrylR
Re-review, made at 5508511 by a model council (Claude Mythos 5.2, Claude Opus 5.5, 3× GPT-6 Astra, 3× GPT-6.1 Sol) and posted with KyrylR's authorization. This summary is updated to f6b85e0.
Findings RN1–RN20. All are fixed at f6b85e0, except the items the latest status lists as still open.
| /// 16 random bytes, base64url. | ||
| pub id: String, | ||
| /// Message id. | ||
| pub id: MessageId, |
There was a problem hiding this comment.
Note
🤖 Claude Opus 5.5 responding on behalf of KyrylR
RN17 · Low–Medium · needs a release note. The relay now rejects an outer-frame id that is not canonical base64url of 16 bytes, and disconnects the sender (relay/connection.rs:756); on master any string was forwarded. This matches spec.md:188, but it is a relay wire change inside a client PR.
Please call it out in the PR description, confirm the deploy order, and add an explicit relay rejection test.
Two relay tests lost specificity in the migration:
relay/server.rs:2287now accepts any handler error;relay/session.rs:1778lost its quote/backslash escaping cases.
Tests fixed in a7fd7c9.
The PR description still needs the release note and the deploy order.
There was a problem hiding this comment.
Note
🤖 Claude Mythos 5.2 responding on behalf of KyrylR
Second re-review, made at efc88bb by a model council (Claude Mythos 5.2, Claude Opus 5.5, 3× GPT-6 Astra, 3× GPT-6.1 Sol) and posted with KyrylR's authorization. This summary is updated to f6b85e0.
Findings S1–S9. All are fixed at f6b85e0, except the items the latest status lists as still open.
There was a problem hiding this comment.
Note
🤖 Claude Mythos 5.2 responding on behalf of KyrylR
Third re-review, made at 192a0c0 by a model council (Claude Mythos 5.2, Claude Opus 5.5, 3× GPT-6 Astra, 3× GPT-6.1 Sol) and posted with KyrylR's authorization. This summary is updated to f6b85e0.
Status at f6b85e0: every finding from the four reviews is fixed, except the items under "Still open".
| Finding | Fixed in |
|---|---|
| V1 | dcc821e |
| V2, V6, V7, RN9 | 5c13b51 |
V3, V9, RN15, B19 (frame v) |
7e245f4 |
| RN16, reconnect-budget residue | e8e1f06 |
| RN14 | 9a12e54 |
| V5, RN20 | 2159261 |
| V4, documentation | ea820d3 |
| RN4, RN13 | 756e13a |
| B16 | 245ddf2 |
| T1, T3 (low-order points, timing margins), RN17 tests | a7fd7c9 |
V8 / S8, B19 (sessionExpiry is advisory) |
f6b85e0 |
Everything else was fixed by 192a0c0.
Still open
- RN18: there is no handshake timeout or
cancel(). A proposer whose proposal is delivered but never answered staysProposed. - RN17: the PR description needs a release note and the deploy order. It should cover canonical frame ids,
proposalIdinapproveandreject, and the removed publicack(). - T3: the
live_relaytests are still ignored, and there is no captured-log redaction test. - RN13: the HKDF PRK is not wiped;
hkdf0.13 has no zeroize support. poll_eventreturns outcomes before lifecycle events. This is now documented, not changed.
After an unacknowledged `approve` the responder scheduled a timeout and, when it fired, dropped the session link if it was not ready at that instant. That was wrong in three ways: * It dropped the link without making the client terminal, so `recv()` then waited forever on an inbox that nothing would feed again. * "Not ready at that instant" also describes a working session whose peer is briefly away, so the timeout could tear down a session that had been established. * The timer started only when the application consumed the approve's `Failed` event, which an application that only calls `recv()` or `wait_session_ready()` never does. And an ack for the approve stopped it altogether, although anyone holding the pairing key can send one. An ack says nothing about the session topic; only `session_ready` on the session link shows that the proposer derived the same one. So: * The transport latches whether the relay ever reported `session_ready` on a link. * The join timeout is armed when the approve is sent, whatever happens to the approve afterwards. It fires at the end of the handshake retry window plus a new `session_join_timeout`, 30 seconds by default. * When it fires, only a link that was never joined is abandoned, and the client then closes with a new terminal error, `SessionNotEstablished`. A peer that joined and is briefly away is handled like any other disconnect, and so is one that joined after the timer fired but before we processed it: that session works. The tests cover a lost approve with and without the application consuming events, an acknowledged approve that nobody follows to the session topic, and a joined session whose peer is away when the timer fires.
A few places left secrets, or text an attacker controls, where they do not belong: * `chacha20poly1305` was built without its `zeroize` feature, so the cipher's copy of the session key was not wiped on drop. Enable it. * The cipher constructors took the key by value, which left a copy behind in the caller, and `PairingKey::from_base64` did not wipe the buffer it decoded into. Take the key by reference and wipe the buffer. * JSON parse errors were logged with serde's `Display`, which quotes unknown variants and mistyped strings verbatim. That let the relay, or the peer once a frame is decrypted, write into our logs. Log only the error category and position. * `Session` derived `Debug`, so formatting `client.session()` printed the pairing and session topics in full. A pairing topic is a bearer secret: whoever learns it can take the free role on it. Print the log tags instead, as everywhere else. Two things are only documented. `Session::peer_did` is the identity a holder of the pairing key declared, not one it proved, since envelopes are not signed with the identity key. And `tungstenite` logs the whole connection URL, token and topic included, at `trace` level, so that level must not be enabled for it where logs are kept or shared.
Every inbound frame is handed to the client with `try_send`, so that a full inbox never stops us servicing the socket. That is fine for message envelopes, which the peer retransmits until they are acked, but not for what is sent once: * If the inbox was full when the peer's grace window ran out, the only `peer_disconnected` was dropped. Readiness was cleared, but the application never got `PeerDisconnected` and just waited. * `Connected` had the opposite problem: we waited for inbox room before entering the socket loop, so an upgraded socket answered no pings while the application was not draining. Put both through a small queue in `run_connection` that hands items over via `Sender::reserve` as soon as there is room, alongside the other `select!` arms. An undelivered `session_ready` is replaced by whatever comes next, since only the latest relay state matters, but a queued `peer_disconnected` is kept even if the peer has returned since: the application must still learn that the peer was gone. Repeats collapse, so the queue stays at three entries at most. While anything is queued, message frames are dropped as they are on a full inbox, so the client never sees an envelope ahead of the lifecycle item that preceded it. Whatever is still queued when the socket ends is delivered, waiting for room, before `Disconnected`. Two ordering fixes go with that. Outbound frames are written ahead of control delivery, so a final message queued just before the client goes away is still sent. And when a connection ends, stale outbound frames are released before we wait for inbox room, because the retry task may be blocked on that queue and must not wait for the application. Finally, ignore a relay frame that names another topic or a version we do not speak. Each socket is bound to one topic, so such a frame is a relay bug or worse, and it must neither make us ready nor reach the client. I could not get a test to fail on the outbound-first ordering or on the early release of stale frames; the rest is tested, including a unit test that lifecycle sends wait for room rather than give up.
The reconnect budget was reset whenever a connection saw `session_ready`, or stayed up for `connection_stability` without a pong or idle timeout. Neither shows that the relay is healthy: * A relay that accepts, says ready and then stops responding kept us reconnecting forever, without ever giving up with a terminal `Transport` error. Write timeouts were not counted as a liveness failure at all. * A relay that says ready and then closes at once, as on a token expiry, did the same, and without even a backoff. Reset the budget only after a connection on which the relay kept responding for `connection_stability`. A connection that ended on a pong, idle or write deadline counts as healthy only until the relay last answered one of our pings; frames it sends on its own, pings included, do not show that it reads what we send. So a session that was fine for an hour and then lost its network still spends one attempt at most, and a one-way connection never resets the budget. A write that fails outright, as on a reset, is told apart from one that hits its deadline and is not counted as a stall. With that rule, a token that expires before a connection can count as stable would make every routine renewal spend an attempt. Tokens carry truncated whole-second `iat` and `exp`, and are signed before we connect, so require `connect_timeout` plus `connection_stability` to be shorter than the whole seconds of `token_ttl` less one. The defaults, 10 s and 30 s against 300 s, are well inside that. While in the config, reject what cannot work: a timeout above one day, `ack_max_attempts` of zero (a frame dispatched while offline would be reported failed without ever being written), a `token_ttl` under one second (the token would already be expired) and an initial reconnect delay above the maximum.
Since a failed proposal returns the proposer to `Idle`, it can propose again with a fresh keypair. But nothing tied an `approve` or a `reject` to a proposal, and nothing checked who sent what: * If P1's acks were lost, the dApp proposed P2 with a new keypair while the wallet still held P1. The wallet approved with the key it derived for P1, the dApp derived with P2's keypair, and both sides ended up `Established` on different session topics without an error. * A reject only changed the `session()` label. A later approve for the rejected proposal still established a session, and after a local `reject()` the derived session key stayed in memory and `approve()` still worked. * A pairing-key holder could send a proposal to a proposer, or an approve to a responder, and move its handshake state. Add a `proposalId` to the approve and reject payloads: the frame id of the proposal being answered. The proposer drops, unacknowledged, an answer for anything but its outstanding proposal. A reject now ends that proposal on both sides: the proposer discards the keypair and may propose again, and the responder forgets the pending proposal and its key. A handshake message is accepted only from the role that may send it, and `reject()` is responder-only like `approve()`. On the responder, a newer proposal from the counterpart replaces a pending one until we approve, so the wallet ends up answering the proposal the dApp is waiting on. That must not let a user approve terms they never saw: the application keeps calling `recv()` while it shows a proposal, so a dApp could send a harmless P1 and then a broader P2. `approve()` and `reject()` therefore take the `ProposalPayload` the application showed the user and return a new `ProposalSuperseded` error if it is no longer the pending one. They compare the whole payload, so a proposal re-sent unchanged under a new frame id is still answered. Smaller changes in the same code: * `propose()` and `approve()` return the envelope id that their `Delivered` or `Failed` event carries. * The responder builds the session cipher before it sends the approve. Once the approve is on the wire the proposer may establish the session, and we must then be able to as well. * If the proposer cannot build the session cipher after an approve has spent its keypair, the proposal is settled and the state returns to `Idle`, so the application can propose again. This changes the wire format of `approve` and `reject` and the signatures of `approve()` and `reject()`.
Several paths gave weight to envelopes that should have had none, or dropped ones that mattered: * With the pending queue full we dropped every non-ack envelope before looking at the replay memory. If our ack for a queued message was lost, its retransmissions were dropped without being re-acked, and the sender reported `Failed` for a message we were holding. Classify against the replay memory first: a retransmission is re-acked as usual and only a new message is held back. * The first envelope that merely authenticated on the pairing topic pinned its sender as the counterpart. A pairing-key holder could get itself pinned with a message we then refused, or with an ack for a made-up id, and every frame from the real peer was dropped from then on. Pin only on an envelope we accept, or an ack that completes one of our sends. * An ack was dropped when the replay memory was full, which turned a delivered message into `Failed` under sustained traffic. `mark_delivered` now says whether an ack completed one of our sends, and that decides everything about it. One that did is always processed, and recorded even when the replay memory is full, up to a fixed headroom of 256 records, so that a conflicting copy is still caught; past the headroom it is processed without being recorded. The headroom has to be fixed, since an application that consumes each outcome and sends again keeps completing sends without limit. An ack that completes nothing has no effect: it is not recorded, so made-up acks cannot fill the replay memory, it pins nobody, and one that names an approve we already gave up on does not retire the pairing link.
Ending a session had problems on both sides. On the sender, `disconnect()` dispatched a tracked frame and then closed at once, so the application always got `Failed` for its own disconnect. `send(&Disconnect)` and `send(&SessionDelete)` told the peer the session was over without closing anything locally, and there was no `session_delete()`. Now `disconnect()` and `session_delete()` queue an untracked frame and close, and `send()` refuses both kinds. `send()` also refuses a response that does not carry exactly one of `response` and `error`, which the receiver would drop unacknowledged. So a teardown is sent once and never retransmitted, and the receiver must not lose it where it would drop anything else for the peer to retry: * With the pending queue full it is queued anyway. * With the replay memory full it is accepted anyway. Replay conflicts are classified first and are still caught. * If the client inbox is full, or control items are queued, the transport reads the envelope kind and holds the teardown behind the control items instead of dropping it. Only the first one waits. One that replaced it at the back would keep repeated control frames from merging, and a relay alternating teardowns with control frames could grow the queue without limit. The kind is read before the envelope is authenticated, which is fine: the client drops a forged one like any other frame, and a relay that withheld the real one could have dropped it anyway. * If acking it fails, the client still closes. Each of these grows a queue by one entry at most, because the frame ends the session. The last point applies to handshake answers too. An `approve` or a `reject` has changed the handshake state by the time we ack it, and its retransmission is then only re-acked or refused. So it is delivered to the application even if its ack cannot be sent.
Once the client closes, every call should return the error that closed it, after handing over what was accepted before. Several paths did not: * `wait_session_ready` checked for a terminal state only on entry. A frame processed inside its loop could close the client, after which the next inbox wait never returned. Check before every wait. * Ending a session that had already closed replaced the cause, so `recv()` reported `Closed` instead of, say, `SessionNotEstablished`. Keep the first cause. Closing now also drops a pending proposal's session key and our proposal keypair instead of holding them until the client is dropped. * A link's transport closes its outbound queue once it has stopped and put its `Closed` in the inbox, behind the frames it received before. Acking one of those frames then failed with `Closed`: `recv()` lost the message and reported `Closed`, and only the next call gave the real cause. On the pairing link after establishment this even reported a live session as closed. Drop an ack that finds its link closed and let the queued `Closed` say what happened. * `next_event` returned the terminal error before the `Failed` outcomes that closing had just queued, so an event loop that stops at the error never saw them. `session().state` now follows suit: it is `Disconnected` whenever the client has closed, and `Active` again once the relay reports a peer back, whichever call processes that. Document all of this on `Client`: how the client is driven, what cancelling a call may lose, which errors are terminal, and that `Failed` means delivery is unconfirmed, not that it did not happen.
Three problems in the retry loop, all around waiting for outbox room: * It removed expired and exhausted entries from `pending` and then, if it also had a frame to resend, awaited outbox room before publishing their `Failed` events. A full outbox delayed them, and a link closing during the wait, which aborts the task, lost them for good. Publish first; the outcome channel is unbounded, so that never waits. * It published after releasing the pending lock. On a multi-threaded runtime the client could close in that gap, find nothing left to report and return the terminal error, with the `Failed` turning up after it. Publish under the lock, which `fail_all` also takes. * A retry set its next deadline before waiting for outbox room, so a long wait used up the ack window before the frame was written: a final attempt that waited four seconds was reported `Failed` 50 ms after it was queued, against a two-second window. Start the window, and the retransmission pacing, when the frame enters the outbox. I have no test for the second one: it needs the task preempted between two statements on another thread.
Tests that go with no single fix: * The relay tests lost some specific assertions when they moved to the `MessageId` type. Restore them, and test that a frame id that is too short, padded or has stray trailing bits closes the sender. * The low-order test only tried the all-zero point. Try every entry of libsodium's blocklist, and say in the comment which entries are canonical encodings and which are not. * Reflection, a kind mismatch and a replay conflict were only tested with the plaintext test cipher. Test them with the real one, and test separately that a frame bounced back to its sender fails on the role bound into the AAD. * Two adversarial tests could not fail: one checked before either frame had been processed, the other sent an ack, which is silent whether accepted or not. * A disconnect processed inside `wait_session_ready`, a superseded proposal that reuses the key, and `session_delete()` reaching the peer had no test. * Give two timing-sensitive tests more margin.
Bring the spec in line with what the client now does, and close gaps that would let two implementations disagree: * Roles are fixed: only the proposer proposes, only the responder approves or rejects, and a handshake message from the wrong role is dropped unacknowledged. * Binary fields use canonical unpadded base64url, and a fixed-length field has exactly one encoding. * `approve` and `reject` carry a `proposalId`. The spec says what a reject ends on each side, when a newer proposal replaces a pending one, and that a decision on a replaced proposal must not answer the newer one unless the two are identical. * A weak `proposal` leaves the responder's handshake as it was, which may be an earlier valid proposal still pending. * The pairing window gets a start point, and the responder's join timeout is counted from the end of that window. An ack for the approve does not stop it. * §10 says where the kind/topic and plaintext checks run relative to the replay steps, which envelope establishes the counterpart, and what a receiver with a full replay memory does: it still processes an ack that completes a send, and never drops a teardown. * A `disconnect` or `session_delete` is sent once: the sender closes without waiting for its ack, and a receiver must not drop one for lack of room. * §13: an ack that cannot be sent may be dropped, and before the counterpart is established an ack that completes a send establishes it. * `sessionExpiry` is advisory.
No description provided.