Skip to content

feat(spmc): add competing queues - #305

Merged
tisonkun merged 7 commits into
apache:mainfrom
mxsm:mxsm/212-spmc
Sep 20, 2026
Merged

tisonkun merged 7 commits into
apache:mainfrom
mxsm:mxsm/212-spmc

Conversation

@mxsm

@mxsm mxsm commented Sep 11, 2026

Copy link
Copy Markdown
Member

Summary

Add opt-in bounded and unbounded asyncband::spmc queues for distributing work from one producer to competing consumers. Senders are non-cloneable and require &mut self; receivers are cloneable, and each consumed value is delivered to exactly one receiver.

  • Preserve strict bounded capacity, synchronous unbounded sends, targeted notifications, cancellation handoff, and draining after sender disconnection.
  • Cover the public capability constraints with five compile-fail checks, and exercise delivery, cancellation, disconnection, backpressure, and concurrent 1P/8C consumption.
  • Add 1P/1C–1P/8C comparisons against Asyncband MPMC, async-channel, and flume. Keep methodology alongside the benchmark harness and measurements in this PR.

Fixes #212.

Design Notes

SPMC owns its queue implementation and its four public error types, following MPMC's single-lock coordination design. MPMC and SPMC have independent type identities and can evolve separately; there is no shared internal error API or error conversion layer. The queues reuse the existing private mutex and waiter-list primitives. SPMC tracks the presence of its unique sender with a boolean. The MPMC library implementation is unchanged relative to main at 7a4e000. Benchmark adapters, batch constants, and runtime setup live in a shared benchmarks/channels fixture; MPMC and SPMC retain separate workloads, and only the MPMC workload requires cloneable senders.

Validation

  • cargo x test: 634 tests and doctests passed, including 20 explicit SPMC tests and five SPMC compile-fail checks.
  • cargo x check: the feature matrix passed, including standalone mpmc and spmc.
  • cargo x lint: Clippy, rustfmt, Taplo, typos, HawkEye, and rustdoc passed.
  • cargo x bench --no-run: both benchmark targets compiled; all 167 MPMC/SPMC benchmark cases passed in test mode after the fixture reorganization. The performance report below records three serial runs of all 64 SPMC comparison cases at 687d2e0, with 100 samples per case.
  • Targeted SPMC Miri validation (cargo +nightly miri test --package tests-integration --test spmc_test): 18 passed; the two OS-backed Tokio concurrency tests were ignored, as intended.

Performance report

Measured on 2026-09-20 at 687d2e0 with an Apple M4 Max (14 CPU cores), 36 GiB RAM, macOS 26.6.2, and Rust 1.99.0-nightly (3d6c19bb9, aarch64-apple-darwin), using the default optimized bench profile. Peers: async-channel 2.5.0 and flume 0.12.0; runtime: Tokio 1.53.1; harness: Divan 0.1.21.

Each sample transfers 16,384 usize values from one producer to 1, 2, 4, or 8 competing consumers; bounded capacity is 64. The sender moves into one task without cloning or an added synchronization wrapper. Consumers drain until disconnection without fixed quotas, and each sample checks the total count and checksum. Runtime, channel, and task creation are outside the measured region; data operations, disconnection, and task completion are inside.

The table reports the median of three run medians, in milliseconds per batch, from three serial runs of 100 samples per case with one batch per sample. No builds or tests ran alongside these measurements. Lower elapsed time is better. current denotes Tokio current-thread; 4 denotes four worker threads. MPMC uses one producer. The peer ratio is SPMC time divided by the faster of async-channel and flume, calculated before rounding.

Queue Runtime Consumers SPMC MPMC async-channel flume Peer ratio
bounded current 1 0.246 0.282 0.591 0.373 0.66x
bounded current 2 0.248 0.291 0.605 0.365 0.68x
bounded current 4 0.270 0.304 0.650 0.386 0.70x
bounded current 8 0.309 0.335 0.739 0.456 0.68x
bounded 4 1 0.404 0.470 0.919 0.589 0.69x
bounded 4 2 0.722 0.882 1.691 1.092 0.66x
bounded 4 4 0.941 1.081 2.147 1.565 0.60x
bounded 4 8 1.065 1.231 2.751 2.001 0.53x
unbounded current 1 0.206 0.233 0.546 0.236 0.87x
unbounded current 2 0.215 0.235 0.548 0.243 0.89x
unbounded current 4 0.206 0.235 0.546 0.260 0.79x
unbounded current 8 0.221 0.236 0.549 0.255 0.87x
unbounded 4 1 0.252 0.275 0.637 0.294 0.86x
unbounded 4 2 0.581 0.539 0.733 0.579 1.00x
unbounded 4 4 0.834 0.894 1.193 0.989 0.84x
unbounded 4 8 0.868 0.946 1.225 1.053 0.82x

The largest SPMC/faster-peer ratio is 1.00x on this host; none of the measured configurations reaches the 3x investigation or 10x rejection thresholds from #208. These measurements do not establish performance across platforms or workloads.

Unbounded sends are synchronous for every implementation, so the producer can fill the queue before consumers run on a current-thread executor. Those rows measure draining rather than parallel consumer contention; use the four-worker results and targeted-wakeup tests to assess 1P/8C behavior.

Reproduce by running cargo x bench --no-run, then execute the ecosystem benchmark binary printed by Cargo with --bench --color never --sample-count 100 'spmc::' three times serially, without concurrent builds or tests. The default harness uses 20 samples with one batch per sample.

Comment thread asyncband/src/mpmc/mod.rs Outdated
Comment on lines +34 to +37
pub use crate::internal::competing_queue::RecvError;
pub use crate::internal::competing_queue::SendError;
pub use crate::internal::competing_queue::TryRecvError;
pub use crate::internal::competing_queue::TrySendError;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree this style just for "reusing code".

Duplicate the code a bit is helpful to decouple concepts for evolution.

At least we should never public an internal struct but have another place to hold it. Or else we don't do software engineering but putting code randomly.

@tisonkun tisonkun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments above.

mxsm and others added 2 commits September 16, 2026 06:54
mpmc::RecvError/SendError/TryRecvError/TrySendError and their spmc
counterparts were pub re-exports of the exact same
internal::competing_queue types, so the two modules' errors were
literally interchangeable and an internal implementation detail leaked
into the public API surface.

Give mpmc and spmc their own duplicated public error types, converted
from the internal errors at the endpoint boundary via From impls. The
internal Shared<T> competing-consumer core stays reused between the
two modules; only the public error surface is decoupled, per review
feedback on PR apache#305.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@tisonkun tisonkun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. But I think bounded and unbounded impls can use different queue for performance consideration.

Also I have to push some commits to apply the comments above about not reusing errors since they are logically different concept that happens to have similar/same structure. We should not reuse just because code looks the same.

@tisonkun
tisonkun merged commit 10ac73a into apache:main Sep 20, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(spmc): add a competing queue

2 participants