Repository navigation
[multicast] Bitmap-based multicast replication for sidecar-lite - #152
Open
zeeshanlakhani wants to merge 1 commit into
Open
zeeshanlakhani wants to merge 1 commit into
zeeshanlakhani wants to merge 1 commit into
Conversation
zeeshanlakhani
force-pushed
the
zl/multicast
branch
from
March 23, 2026 21:24
09d1d59 to
88b2180
Compare
zeeshanlakhani
marked this pull request as ready for review
March 24, 2026 00:40
zeeshanlakhani
force-pushed
the
zl/multicast
branch
from
March 25, 2026 15:37
88b2180 to
d0a41f9
Compare
zeeshanlakhani
marked this pull request as draft
March 26, 2026 15:54
zeeshanlakhani
marked this pull request as ready for review
March 30, 2026 05:48
zeeshanlakhani
added a commit
to oxidecomputer/dendrite
that referenced
this pull request
Mar 30, 2026
This wires up the softnpu ASIC backend to support multicast end-to-end by translating DPD's sidecar.p4 table operations into sidecar-lite.p4's simplified P4 pipeline. ## AsicMulticastOps We replace the stubbed AsicMulticastOps implementation (which returned "OperationUnsupported" for group creation and port addition with in-memory group tracking via McGroupData, following the tofino_stub pattern. Group membership is used by the table translation layer to build port bitmaps for sidecar-lite's Replicate extern. Ports >= 128 are rejected at add time to match sidecar-lite's 128-bit bitmap width. ## Table translation (asic/src/softnpu/table.rs) We map sidecar.p4 table names to sidecar-lite equivalents and translate action parameters where the designs differ for emulation. All multicast action arms are gated with #[cfg(feature = "multicast")]. ## References - [softnpu #183](oxidecomputer/softnpu#183) - [propolis #1093](oxidecomputer/propolis#1093) - [p4rs #240](oxidecomputer/p4#240) - [sidecar-lite #152](oxidecomputer/sidecar-lite#152) - tokio: 1.50 (due to softnpu) - oxide-tokio-rt: 0.1.3 (following-up from tokio's move to 1.50)
zeeshanlakhani
added a commit
to oxidecomputer/dendrite
that referenced
this pull request
Apr 1, 2026
This wires up the softnpu ASIC backend to support multicast end-to-end by translating DPD's sidecar.p4 table operations into sidecar-lite.p4's simplified P4 pipeline. ## AsicMulticastOps We replace the stubbed AsicMulticastOps implementation (which returned "OperationUnsupported" for group creation and port addition with in-memory group tracking via McGroupData, following the tofino_stub pattern. Group membership is used by the table translation layer to build port bitmaps for sidecar-lite's Replicate extern. Ports >= 128 are rejected at add time to match sidecar-lite's 128-bit bitmap width. ## Table translation (asic/src/softnpu/table.rs) We map sidecar.p4 table names to sidecar-lite equivalents and translate action parameters where the designs differ for emulation. All multicast action arms are gated with #[cfg(feature = "multicast")]. ## References - [softnpu #183](oxidecomputer/softnpu#183) - [propolis #1093](oxidecomputer/propolis#1093) - [p4rs #240](oxidecomputer/p4#240) - [sidecar-lite #152](oxidecomputer/sidecar-lite#152) - tokio: 1.50 (due to softnpu) - oxide-tokio-rt: 0.1.3 (following-up from tokio's move to 1.50)
zeeshanlakhani
added a commit
to oxidecomputer/dendrite
that referenced
this pull request
Apr 28, 2026
This wires up the softnpu ASIC backend to support multicast end-to-end by translating DPD's sidecar.p4 table operations into sidecar-lite.p4's simplified P4 pipeline. ## AsicMulticastOps We replace the stubbed AsicMulticastOps implementation (which returned "OperationUnsupported" for group creation and port addition with in-memory group tracking via McGroupData, following the tofino_stub pattern. Group membership is used by the table translation layer to build port bitmaps for sidecar-lite's Replicate extern. Ports >= 128 are rejected at add time to match sidecar-lite's 128-bit bitmap width. ## Table translation (asic/src/softnpu/table.rs) We map sidecar.p4 table names to sidecar-lite equivalents and translate action parameters where the designs differ for emulation. ## Fix update semantics We fix an issue with stale in-memory sources after update with absent source list(s). `modify_group_external` wrote back `new.sources.or(existing)` to the in-memory record while `update_external_tables` in the same call canonicalized `new.sources` directly to rewrite the P4 source-filter entries. A `None` update therefore cleared the P4 table but left a stale `Some([...])` in memory. All multicast action arms are gated with #[cfg(feature = "multicast")]. ## References and Deps - [softnpu #183](oxidecomputer/softnpu#183) - [propolis #1093](oxidecomputer/propolis#1093) - [p4rs #240](oxidecomputer/p4#240) - [sidecar-lite #152](oxidecomputer/sidecar-lite#152) - tokio: 1.50 (due to softnpu) - oxide-tokio-rt: 0.1.3 (following-up from tokio's move to 1.50) - bump oxide-tokio-rt to 0.1.4, tokio to 1.52.1 (matching Omicron)
zeeshanlakhani
force-pushed
the
zl/multicast
branch
2 times, most recently
from
June 4, 2026 03:42
8757f58 to
bcceb2d
Compare
This adds multicast replication to sidecar-lite, emulating DPD's
sidecar.p4 on dendrite @ zl/mcast-swadm, following RFD 488.
We manage 2 bitmaps per underlay group (external and underlay), for
which replication is driven by p4rs `replicate` (dep @ p4#240). Ingress
zeroes out at most one of these bitmaps determined by the geneve
multicast option tag (tag 0 zeroes out underlay; tag 1 zeroes out
external), and merges both.
Tag 0 (External) also decaps upon ingress. With no tag given, as on
external NAT ingress packets, both bitmaps remain in place.
Only geneve-encapsulated multicast commits to replication. Multicast
packets that miss ingress NAT drop on the empty bitmap, matching DPD's
`drop_mcastv{4,6}_no_group` semantics. Packet copies that
remain encapsulated lose a hop on the outer IPv6 header, as modelled in
`MulticastRouter6`.
External multicast is NAT-encapsulated into the underlay via
`nat_v{4,6}_mcast(_untagged)`, reusing `forward_to_sled`. Tagged and
untagged variants exist as separate tables since x4c cannot key
directly on header validity. The encap piece now zeroes out the outer
src, traffic class, flow label, and inner eth src, like DPD's
`add_encap_headers`.
Egress decap checks `ff04::/64` and `tag=2`, matching DPD's egress
`mcast_tag_check`, with per-port decap from a RID-keyed bitmap
(`tbl_decap_ports`) and optional VLAN. `tag=2` decapsulation also
rejects inner unicast. `tag=0` copies (already decapped in ingress)
use the table for the VLAN. Dst MAC derivation (RFC 1112, RFC 2464)
and per-port src MAC rewriting run on every packet copy.
Includes:
- The parser rejects `ff01` addresses outright, as well as `ff02` destined
packets with a hop limit of 0 (`ff02` *does* still go to the scrimlet).
- Multicast packets with a TTL/hop_limit <= 1 get dropped in ingress
after the `fwd_from_scrimlet` return, letting scrimlet transmissions
pass through.
- Both decapsulation paths drop inner TTL <= 1. IPv4 checksums use
RFC 1624 (eqn. 3): a checksum of 0xfeff turns into 0x0000 after
decrementing the TTL (XOR used here due to `~` not being parsable by x4c).
- scadm gets add/remove/dump for replication (v6), source filters, mcast
NAT, decap ports, src MAC, and asic-id-to-port (for logical mapping).
- Unicast modifications:
- The router drops TTL/hop_limit <= 1 and decrements remaining packets
on routes to nonzero ports, updating the IPv4 checksum.
- Port 0 (the scrimlet) skips both the TTL check and decrement.
- DPD's Router4/6 still decrement service-bound packets.
- The NAT encap zeroing described above is shared with unicast NAT.
- Packets with a parsed Geneve header skip ingress NAT, including
unicast.
### References
This work depends on the P4(rs) work in
[#240](oxidecomputer/p4#240) and softnpu
zl/multicast.
zeeshanlakhani
force-pushed
the
zl/multicast
branch
from
October 15, 2026 08:34
ff2aac0 to
8e55228
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds multicast replication to sidecar-lite, emulating DPD's sidecar.p4 on dendrite @ zl/mcast-swadm, following RFD 488.
We manage 2 bitmaps per underlay group (external and underlay), for which replication is driven by p4rs
replicate(dep @ p4#240). Ingress zeroes out at most one of these bitmaps determined by the geneve multicast option tag (tag 0 zeroes out underlay; tag 1 zeroes out external), and merges both.Tag 0 (External) also decaps upon ingress. With no tag given, as on external NAT ingress packets, both bitmaps remain in place.
Only geneve-encapsulated multicast commits to replication. Multicast packets that miss ingress NAT drop on the empty bitmap, matching DPD's
drop_mcastv{4,6}_no_groupsemantics. Packet copies that remain encapsulated lose a hop on the outer IPv6 header, as modelled inMulticastRouter6.External multicast is NAT-encapsulated into the underlay via
nat_v{4,6_mcast(_untagged), reusingforward_to_sled. Tagged and untagged variants exist as separate tables since x4c cannot key directly on header validity. The encap piece now zeroes out the outer src, traffic class, flow label, and inner eth src, like DPD'sadd_encap_headers.Egress decap checks
ff04::/64andtag=2, matching DPD's egressmcast_tag_check, with per-port decap from a RID-keyed bitmap (tbl_decap_ports) and optional VLAN.tag=2decapsulation also rejects inner unicast.tag=0copies (already decapped in ingress) use the table for the VLAN. Dst MAC derivation (RFC 1112, RFC 2464) and per-port src MAC rewriting run on every packet copy.Includes:
ff01addresses outright, as well asff02destined packets with a hop limit of 0 (ff02does still go to the scrimlet).fwd_from_scrimletreturn, letting scrimlet transmissions pass through.~not being parsable by x4c).References
This work depends on the P4(rs) work in #240 and softnpu's zl/multicast.