diff --git a/.github/workflows/verification.yml b/.github/workflows/verification.yml index c1e628da6..5162e3bf3 100644 --- a/.github/workflows/verification.yml +++ b/.github/workflows/verification.yml @@ -81,7 +81,7 @@ jobs: - name: Record supervisor lifecycle traces env: LIFECYCLE_TRACE_DIR: ${{ github.workspace }}/lifecycle-traces - run: cargo test --locked -p spawned-concurrency --features lifecycle-trace,failpoints --test supervision_trace --test cohort_trace --test tree_trace --test nested_supervision --test coverage_trace --test job_trace --test reply_trace --test disposal_trace --test lifecycle_trace --test coordinator_trace + run: cargo test --locked -p spawned-concurrency --features lifecycle-trace,failpoints --test supervision_trace --test cohort_trace --test tree_trace --test nested_supervision --test coverage_trace --test job_trace --test reply_trace --test disposal_trace --test lifecycle_trace --test start_trace --test coordinator_trace - name: Validate traces and negative controls run: nix run .#check-tla -- --traces-only lifecycle-traces --negative-controls # Which model actions, projected states and transitions the recorded diff --git a/.gitignore b/.gitignore index 177f35340..1f7aee18d 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,6 @@ __pycache__/ # Standalone experiments crate (benchmarks/README.md) benchmarks/experiments/target/ benchmarks/experiments/Cargo.lock + +# Standalone example crates build into their own directories +examples/thread_only/target/ diff --git a/CHANGELOG.md b/CHANGELOG.md index c8eb616f7..d11e25f27 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,40 @@ - ROADMAP: 0.9 is now soundness and introspection; the unified start API (G1, G2, G8) and `{shutdown, Term}` exits (G7) move into 0.9, and the admission package is designed during 0.9. +- **Breaking: unified start API** (rank 5h, gaps G1, G2, G8; design + `docs/design/unified-start.md`). OTP's verbs in both modes: `spawn` (no + wait), `start`, `start_link(&ctx)` and `start_monitor(&ctx)` (wait for + `started()` and answer `Started`, `Ignored` or a `StartError` after the + child's publication), `with_options(StartOptions)`. `started()` may fail: + it returns `()`, `Result<(), E>`, `Result, E>` or `Init` + (`ready`, `continue_with`, `error`, `stop`, `ignore`), resolved by the + design's precedence table; no failed start runs `stopped()` (also not + after `ctx.stop()` in `started()`, now `Init::stop`; a behaviour change); + an administrative stop during `started()` waits for it and runs + `stopped()`, as OTP's `gen_server:stop`. A parent killed (or hit by an + untrapped signal) while it waits for a `start_link`/`start_monitor` stops + waiting at once (`ParentExited`), as OTP's caller dies without waiting + for `init/1`. A dispatch or thread-spawn failure after the commit kills + the child (`InitExited { Killed }`) instead of a stage-1 error. The init + timer runs on the runtime clock in tasks mode (virtual under the + simulator) and on one shared thread in threads mode. `registry::register_owned` + takes the holder's `ChildHandle` (for `AlreadyStarted { existing }`). Stage 1 is one + atomic commit (group admission, the parent's link or monitor, the typed + `Name`); a refusal is never observable. `start_link`/`start_monitor` + inherit the parent's placement. A failed start consumes a trapping + parent's `Exit` and a monitor's `Down`; a non-trapping parent still exits + on a non-`Normal` failure. `Init::continue_with` runs before any mailbox + entry. `init_timeout`, `ActorRef::ready()` for every placement, + `Starting::actor_ref()`. Removed: `start_on`, `start_with_backend`, + `start_linked`, `ExecutionGroup::start`, `SystemError::StartupFailed`; + test kit `Mock::spawn`/`spawn_with`, `TestSystem::spawn`/`start`/ + `spawn_mock`/`options`. Every break is in `docs/MIGRATION.md` (0.8 to + 0.9). Start.tla is now trace-validated (`StartEvent`, `TraceStart.tla`, + `tests/start_trace.rs`, 15 recorded-trace negative controls, coverage 28/28 + arms); the model releases a group slot at scope completion, as the code + does. Not yet: scope ownership (S8), supervisor `ignore` and + `ChildSpec::options`. + - `model-sync`: a design-only TLA+ model (`design_only = true`: Start, Introspection, DynamicSupervisor) stays allowlisted as TLC-only only while no code maps to it; the PR that implements it must add its trace diff --git a/CLAUDE.md b/CLAUDE.md index 7b50c91aa..ca095c6b3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -73,10 +73,14 @@ Both provide identical Actor API. The `tasks` module has `Backend` enum: `Async` | `Exit` / `#[exit_received]` | Bidirectional links and trapped exits (`ctx.link`, `ctx.trap_exit`) | | `ActorSystem` / `ExecutionGroup` | Owned placement with worker budgets and ordered shutdown (v0.7) | | `Supervisor` / `ChildSpec` / `SupervisedChild` | OneForOne, OneForAll, RestForOne and nested supervision (v0.7) | +| `ActorStart` | `spawn` (no wait), `start` / `start_link(&ctx)` / `start_monitor(&ctx)` (wait for `started()`), `with_options(opts)` (v0.9) | +| `StartOptions` / `Name` | Placement (`group`, `backend`, `placement_of`; link/monitor inherit the parent's), typed start-time name, `init_timeout`, `label`, `stack_size`, per-instance bounds (v0.9) | +| `Init` | What `started()` returns: `ready`, `continue_with(msg)`, `error(e)`, `stop(r)`, `ignore` (also `()`, `Result`) (v0.9) | +| `Starting` / `StartOutcome` / `StartError` | A started instance's wait (`actor_ref()`, drop = abandon); `Started`/`Ignored`; stage-1 refusals and `InitFailed`/`InitExited`/`InitTimeout`, answered after publication (v0.9) | ## Actor Lifecycle -1. `started(&mut self, ctx: &Context)` - Setup before message loop +1. `started(&mut self, ctx: &Context) -> impl Into>` - Setup before message loop; may fail, stop, ignore or continue. A failed start never runs `stopped()` 2. `Handler::handle(&mut self, msg: M, reply: PendingReply<'_, M::Result>, ctx: &Context)` - Process messages 3. `stopped(&mut self, ctx: &Context)` - Cleanup after stop (skipped on kill or an untrapped exit signal) @@ -85,8 +89,11 @@ Death has one point (`actor_cleanup.rs`): every caller is answered and links/mon ## Common Patterns ```rust -// Start an actor (use ActorStart trait) -let ns = NameServer::new().start(); +// Start an actor (ActorStart trait; v0.9 OTP verbs) +let ns = NameServer::new().spawn(); // no wait +let ns = NameServer::new().start().await?.started().unwrap(); // waits for started() +let child = Worker::new().start_link(&ctx).await?; // linked, parent's placement +let worker = Worker::new().with_options(&group).spawn()?; // in an ExecutionGroup // Call protocol methods directly on ActorRef ns.add("key".into(), "value".into()).await.unwrap(); diff --git a/README.md b/README.md index 3c727da53..727d96e4f 100644 --- a/README.md +++ b/README.md @@ -92,7 +92,7 @@ use spawned_rt::tasks as rt; fn main() { rt::run(async { - let ns = NameServer::new().start(); + let ns = NameServer::new().spawn(); ns.add("Joe".into(), "At Home".into()).await.unwrap(); @@ -170,7 +170,9 @@ For developers familiar with Erlang/OTP: | `gen_server:call/2` | `ns.find(...)` | Direct method call on ActorRef | | `gen_server:cast/2` | `ns.notify(...)` | Direct method call (send variant) | | `Pid` | `ActorRef` | Handle to a running actor | -| `start_link/0` | `actor.start_linked(&ctx)` | Start the actor | +| `proc_lib:spawn/1` | `actor.spawn()` | Start without waiting for `started()` | +| `gen_server:start/3`, `start_link/3`, `start_monitor/3` | `actor.start()`, `actor.start_link(&ctx)`, `actor.start_monitor(&ctx)` | Start and wait for `started()` (`.await` in tasks mode) | +| `{ok, S}` / `{ok, S, {continue, C}}` / `{error, R}` / `{stop, R}` / `ignore` | `started()` returning `()`, `Init::continue_with(c)`, `Err(e)`, `Init::stop(r)`, `Init::ignore()` | Init results | | `register/2` | `registry::register(name, ref)` | Register an actor by name | | `whereis/1` | `registry::whereis(name)` | Look up an actor by name | @@ -295,7 +297,7 @@ with virtual time, so timers and restart backoff take no wall-clock time: ```rust #[spawned_concurrency::sim_test] async fn find_returns_what_add_stored() { - let ns = NameServer::new().start(); + let ns = NameServer::new().spawn(); ns.add("Joe".into(), "At Home".into()).await.unwrap(); assert_eq!( ns.find("Joe".into()).await.unwrap(), diff --git a/ROADMAP.md b/ROADMAP.md index 2d0980097..39af381ac 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -655,7 +655,7 @@ earlier [detailed acceptance criteria](https://github.com/lambdaclass/spawned/bl | Idea | Released capability | |---|---| -| ActorSystem and execution groups | Named parked async pools with explicit worker budgets, `start_on(&group)`, owned helpers, readiness and reverse teardown. Fixed native actor pools and parked Linux pinned shards are implemented; budgets exclude helper/owner/control/coordinator threads. | +| ActorSystem and execution groups | Named parked async pools with explicit worker budgets, `with_options(&group)` (0.8: `start_on`), owned helpers, readiness and reverse teardown. Fixed native actor pools and parked Linux pinned shards are implemented; budgets exclude helper/owner/control/coordinator threads. | | Automatic supervision trees | ChildSpec factories, Permanent/Transient/Temporary, rolling restart limits, capped backoff/jitter, ordered startup and reverse shutdown. Nested specs and OneForAll/RestForOne are implemented, with actual cleanup barriers and bounded formal models/real trace checks for all three strategies and nested ownership. | | Tokio-free synchronous builds | Disable async defaults on both crates; isolated fixture and graph CI cover protocols, lifecycle and metrics. ActorSystem/supervision remain async-only. | | Fallible startup / readiness | Fallible startup and readiness for supervised children. The `result_on_start` branch is not the implementation; unsupervised `started()` still cannot fail. | @@ -746,7 +746,7 @@ startup: OTP aborts `start_link` on any child start error, while Spawned retries a Permanent or Transient child's failed factory, panic or startup timeout during the first start (semantics §11; found reviewing #233). Behaviour change: extend the `Supervision`/`Cohorts` TLA+ models and their Apalache inductive checks (`BudgetBounded`) to the new rule, with a negative control for counting per child, and update the Kani restart-window proofs and the guide. | | 5g | One exit reason when the runtime drops an actor (v0.8) | Tasks mode reports `Kill` when the runtime drops an actor; threads mode falls back to `Panic("unexpected framework panic")` for an actor that never finishes cleanup. Pick one documented reason for both modes and test it in both. | -| 5h | Unified start API (v0.9, design approved for implementation after three review rounds, PR #259) | [Design](docs/design/unified-start.md) with the `Start.tla` model (8 configurations, 40 negative controls; TLC-only until implemented) and a [client mapping](docs/design/unified-start-client-mapping.md). OTP's `start`, `start_link` and `start_monitor` wait for `started()` and answer `Started`, `Ignored` or an error after the child's publication (with the final exit reason); `spawn` does not wait. One typed `StartOptions`. `started()` may fail, ignore or continue (`Init`); a failed init never runs `stopped()`. An immutable startup commitment per instance; readiness committed by the child under the global lock order; the init timer and abandonment arbitrate through the commitment. Linked and monitored children inherit the parent's group or backend (API-review footgun 5.11); linked children in the parent's system are owned by its scope (polite Shutdown at the parent's publication, absolute composing deadlines). Supervisors keep the whole-attempt `startup_timeout`, and `ignore` keeps the spec as OTP. Shared result types with #245 and #256. Client impact: 51 mechanical sites, no behaviour change. | +| 5h | Unified start API (v0.9; design PR #259; **implementation in progress**: verbs, `StartOptions`, fallible `started()`, continuations, names, init timeout, inheritance and Start.tla trace validation in the first PR; scope ownership (S8) and supervisor `ignore` follow) | [Design](docs/design/unified-start.md) (with its implementation status and deviations) with the `Start.tla` model (8 configurations, 40 negative controls; trace-validated by `TraceStart.tla`) and a [client mapping](docs/design/unified-start-client-mapping.md). OTP's `start`, `start_link` and `start_monitor` wait for `started()` and answer `Started`, `Ignored` or an error after the child's publication (with the final exit reason); `spawn` does not wait. One typed `StartOptions`. `started()` may fail, ignore or continue (`Init`); a failed init never runs `stopped()`. An immutable startup commitment per instance; readiness committed by the child under the global lock order; the init timer and abandonment arbitrate through the commitment. Linked and monitored children inherit the parent's group or backend (API-review footgun 5.11); linked children in the parent's system are owned by its scope (polite Shutdown at the parent's publication, absolute composing deadlines). Supervisors keep the whole-attempt `startup_timeout`, and `ignore` keeps the spec as OTP. Shared result types with #245 and #256. Client impact: 51 mechanical sites, no behaviour change. | | 6 | Test kit with `ActorRef::mock()` | **Phase A implemented** in [`spawned-test`](#testing-crate-spawned-test) (unreleased): closure-backed `Mock` actors that implement `Handler` for every message, so `ActorRef` satisfies any `#[protocol]` and replaces hand-stubbed protocol refs with no production change; recording `Probe` recipients (`expect`, `expect_no_message`); bounded harness waits; fault injection (`faults::exit`, `restart_child`, mock delay/discard/crash). Unmocked messages fail loudly. **Remaining:** mocking a field typed as a concrete `ActorRef` needs an `Envelope` hook that downcasts to the concrete message and yields message and reply sender (a core change, deferred); controllable time for timers; a generic idle check; core `ChildHandle::exit(reason)` would make injected exits exact (today `Panic` and `Kill` go through a linked helper, so a panic surfaces as `Panic("panic in handler: ...")`). **Client evidence:** hand-stubbed protocols in ethlambda tests (`RecordingChain` in `column_checks.rs`, also `envelope_checks.rs`, `verdict.rs`). **Research ([actor testing](docs/research/actor-testing.md)):** three gaps against Akka/Erlang kits. (1) A synchronous behavior test kit (Akka `BehaviorTestKit`): run one handler with a test `Context` and record its effects (sends, spawns, timers, stop) without a runtime; needs a core change. (2) Manual time (Akka `ManualTime`): document Tokio's `start_paused` for async actors; thread actors need the framework-owned clock, the same seam as the [simulation backend spike](#simulation-backend-spike) and `ctx.now()`, so build one clock, not two. (3) Concrete `ActorRef` mocking stays deferred (actix's `Mocker` is the precedent); until then users hold protocol refs or `Recipient`s. Also: the idle check should report total deadlock Go-`checkdead` style (every actor idle, a request outstanding, no timer pending), and a replay-equivalence check for event-sourced actors ([event sourcing](docs/research/event-sourcing.md)). | | 6b | Shared-state and abort lints in `#[actor]` | Zero runtime cost. The macro warns when an actor's fields hold shared mutable state (`Arc>`, `Arc>`, atomics behind `Arc`, `static mut`), with an opt-out attribute for deliberate cases, and a build script check warns when the profile uses `panic = "abort"`, under which a handler panic kills the process instead of one actor. Syntactic only (it cannot see sharing behind a type alias), but it catches the common pattern, including the one LLM-written code reaches for first. Erlang gap: memory isolation. | | 7 | Request deadlock detection | **Client evidence:** requests awaited inside handlers exist (ethrex peer table), no cycle found; keep as a cheap safety net. `ActorError::WouldDeadlock` naming the cycle instead of a timeout. `ActorRef::request` takes only `&self`, so set a thread-local `CURRENT_ACTOR` only while the actor's own handler future is polled (and around the synchronous handler in thread mode); detached helpers create no edges. Keep a small wait set per actor for `join!`. Measure cost before enabling outside debug builds. **Approach chosen ([deadlock and stall detection](docs/research/deadlock-and-stall-detection.md)):** eager detection. Add a wait-for edge when a handler sends a request, walk the per-actor wait sets for a cycle, fail the request that would close it with `WouldDeadlock` naming the cycle, remove the edge on reply or timeout; a request to self fails at once (Erlang's `calling_self`). Waits on raw channels, locks or `spawn_blocking` create no edges, so it is a safety net, not a proof. Edges are exported to rank 30. | diff --git a/benchmarks/experiments/benches/designs.rs b/benchmarks/experiments/benches/designs.rs index f96d67dca..c3aa36da2 100644 --- a/benchmarks/experiments/benches/designs.rs +++ b/benchmarks/experiments/benches/designs.rs @@ -693,10 +693,11 @@ fn main() { "blocking" => tasks::Backend::Blocking, _ => tasks::Backend::Thread, }; - Target::Tasks(tasks::ActorStart::start_with_backend( - Sink::new(&config, address), - backend, - )) + Target::Tasks( + tasks::ActorStart::with_options(Sink::new(&config, address), backend) + .spawn() + .expect("spawn"), + ) } "threads" => Target::Threads(threads::ActorStart::start(Sink::new(&config, address))), "tokio" => { diff --git a/benchmarks/experiments/benches/lifecycle_soak.rs b/benchmarks/experiments/benches/lifecycle_soak.rs index 716a24323..4c2538b39 100644 --- a/benchmarks/experiments/benches/lifecycle_soak.rs +++ b/benchmarks/experiments/benches/lifecycle_soak.rs @@ -284,9 +284,9 @@ fn main() { counts: counts.clone(), pending: HashMap::new(), } - .start(); + .spawn(); // Initialize every owned actor worker/runtime while the caller still inherits actor CPUs. - let seed = Worker::new(counts.clone()).start(); + let seed = Worker::new(counts.clone()).spawn(); runtime .block_on(seed.request(Step { expected: 0, @@ -334,7 +334,7 @@ fn main() { }); runtime.block_on(async { let residents: Vec<_> = (0..idle) - .map(|_| Worker::new(counts.clone()).start()) + .map(|_| Worker::new(counts.clone()).spawn()) .collect(); watched(&observer, &residents, false).await; for a in &residents { @@ -354,7 +354,7 @@ fn main() { let mut expected_down = 0; while started.elapsed() < Duration::from_secs(seconds) { let actors: Vec<_> = (0..churn) - .map(|_| Worker::new(counts.clone()).start()) + .map(|_| Worker::new(counts.clone()).spawn()) .collect(); let panic = counts.cycles.load(Ordering::Relaxed).is_multiple_of(16); watched(&observer, &actors, panic).await; diff --git a/benchmarks/experiments/benches/pinned_pipeline.rs b/benchmarks/experiments/benches/pinned_pipeline.rs index 2e1378d60..39a5a1454 100644 --- a/benchmarks/experiments/benches/pinned_pipeline.rs +++ b/benchmarks/experiments/benches/pinned_pipeline.rs @@ -190,7 +190,7 @@ mod linux { counts: counts.clone(), handled: 0, } - .start() + .spawn() }) .collect(); actor_runtime.block_on(async { diff --git a/benchmarks/experiments/benches/runtime_isolation.rs b/benchmarks/experiments/benches/runtime_isolation.rs index bd8c38ae1..952c82c60 100644 --- a/benchmarks/experiments/benches/runtime_isolation.rs +++ b/benchmarks/experiments/benches/runtime_isolation.rs @@ -138,7 +138,7 @@ mod linux { counts: counts.clone(), handled: 0, } - .start() + .spawn() } fn affine(n: usize) -> (u64, u64) { let (mut a, mut b) = (1u64, 0u64); diff --git a/benchmarks/experiments/benches/workload_research.rs b/benchmarks/experiments/benches/workload_research.rs index 124cf281b..56e102fc0 100644 --- a/benchmarks/experiments/benches/workload_research.rs +++ b/benchmarks/experiments/benches/workload_research.rs @@ -588,7 +588,7 @@ fn main() { children: Vec::new(), meter: meters[id].clone(), } - .start(), + .spawn(), ); } let nodes: Vec<_> = nodes.into_iter().map(Option::unwrap).collect(); diff --git a/benchmarks/experiments/tests/backend_experiments.rs b/benchmarks/experiments/tests/backend_experiments.rs index 7a231c8e2..bf036067e 100644 --- a/benchmarks/experiments/tests/backend_experiments.rs +++ b/benchmarks/experiments/tests/backend_experiments.rs @@ -62,7 +62,7 @@ fn experiment_contracts() { .build() .unwrap(); rt.block_on(async { - let actor = Worker.start_with_backend(Backend::Thread); + let actor = Worker.with_options(Backend::Thread).spawn().unwrap(); assert_eq!(actor.request(Probe).await.unwrap(), 42); let helper_progress = actor.request(Nested).await.unwrap(); let current = std::env::var("SPAWNED_EXPERIMENT_RUNTIME").as_deref() == Ok("current"); @@ -120,7 +120,7 @@ fn experiment_io_reactor() { .build() .unwrap(); rt.block_on(async { - let actor = Worker.start_with_backend(Backend::Thread); + let actor = Worker.with_options(Backend::Thread).spawn().unwrap(); assert_eq!(actor.request(Network).await.unwrap(), 42); actor.child_handle().stop(); within!(actor.join()).await; @@ -135,7 +135,7 @@ fn characterize_caller_runtime_lifetime() { .build() .unwrap(); let actor = rt.block_on(async { - let actor = Worker.start_with_backend(Backend::Thread); + let actor = Worker.with_options(Backend::Thread).spawn().unwrap(); assert_eq!(actor.request(Probe).await.unwrap(), 42); actor }); diff --git a/benchmarks/experiments/tests/descriptor_limit.rs b/benchmarks/experiments/tests/descriptor_limit.rs index 4df1511e5..fef9c0798 100644 --- a/benchmarks/experiments/tests/descriptor_limit.rs +++ b/benchmarks/experiments/tests/descriptor_limit.rs @@ -106,7 +106,7 @@ fn descriptor_limit_child() { }; let mut cleanup = Cleanup(Vec::new()); if mode != "sharded-cold" { - let seed = Worker(None).start_with_backend(backend); + let seed = Worker(None).with_options(backend).spawn().unwrap(); cleanup.0.push(seed.child_handle()); seed.request(Ping).await.unwrap(); } @@ -127,7 +127,7 @@ fn descriptor_limit_child() { 1 }; for _ in 0..count { - let actor = Worker(None).start_with_backend(backend); + let actor = Worker(None).with_options(backend).spawn().unwrap(); cleanup.0.push(actor.child_handle()); let outcome = actor.request_with_timeout(Ping, LIMIT).await; if mode == "sharded" || mode == "async" { @@ -167,7 +167,7 @@ fn descriptor_limit_child() { drop(descriptors); if mode == "sharded-cold" { // An initialization error must not poison the registry or leave workers behind. - let recovered = Worker(None).start(); + let recovered = Worker(None).spawn(); cleanup.0.push(recovered.child_handle()); recovered.request(Ping).await.unwrap(); recovered.child_handle().stop(); diff --git a/benchmarks/experiments/tests/full_actor_shards.rs b/benchmarks/experiments/tests/full_actor_shards.rs index b7651fdbd..16f0926b2 100644 --- a/benchmarks/experiments/tests/full_actor_shards.rs +++ b/benchmarks/experiments/tests/full_actor_shards.rs @@ -85,7 +85,7 @@ impl Handler for Worker { ) { let __reply_value: u64 = { use tokio::io::{AsyncReadExt, AsyncWriteExt}; - let child = Worker(Default::default()).start(); + let child = Worker(Default::default()).spawn(); let _ = child.request(Thread).await.unwrap(); child.child_handle().stop(); within!(child.join()).await; @@ -126,7 +126,7 @@ fn nested_spawn_helpers_timers_and_tcp_use_the_actor_reactor() { }); runtime().block_on(async { let events = Arc::new(Mutex::new(Vec::new())); - let actor = Worker(events.clone()).start(); + let actor = Worker(events.clone()).spawn(); let _cleanup = Cleanup(vec![actor.child_handle()]); assert_eq!( actor @@ -149,7 +149,7 @@ fn nested_spawn_helpers_timers_and_tcp_use_the_actor_reactor() { #[test] fn stop_finishes_in_flight_handler_and_rejects_queued_work() { runtime().block_on(async { - let actor = Worker(Default::default()).start(); + let actor = Worker(Default::default()).spawn(); let _cleanup = Cleanup(vec![actor.child_handle()]); let (started, rx) = tokio::sync::oneshot::channel(); let (release, gate) = tokio::sync::oneshot::channel(); @@ -196,7 +196,7 @@ fn suspended_actor_does_not_block_another_actor_on_its_shard() { let mut actors = Vec::new(); let mut names = Vec::new(); for _ in 0..=count { - let actor = Worker(Default::default()).start(); + let actor = Worker(Default::default()).spawn(); cleanup.0.push(actor.child_handle()); let name = actor.request_with_timeout(Thread, LIMIT).await.unwrap(); assert!(name.starts_with("spawned-shard-")); @@ -251,7 +251,7 @@ fn actors_outlive_their_caller_runtime() { } let rt = runtime(); let actor = rt.block_on(async { - let actor = Worker(Default::default()).start(); + let actor = Worker(Default::default()).spawn(); actor.request(Thread).await.unwrap(); actor }); @@ -298,7 +298,7 @@ fn blocking_handler_stalls_its_shard_while_another_shard_progresses() { let mut actors = Vec::new(); let mut names = Vec::new(); for _ in 0..=count { - let actor = Worker(Default::default()).start(); + let actor = Worker(Default::default()).spawn(); cleanup.0.push(actor.child_handle()); names.push(actor.request_with_timeout(Thread, LIMIT).await.unwrap()); actors.push(actor); @@ -383,7 +383,7 @@ impl Handler for Worker { ) { reply.reply( Worker(Default::default()) - .start_with_backend(spawned_concurrency::tasks::Backend::Blocking), + .with_options(spawned_concurrency::tasks::Backend::Blocking).spawn().unwrap(), ); } } @@ -408,7 +408,7 @@ impl Handler for Worker { #[test] fn detached_helper_and_blocking_child_outlive_parent_stop() { runtime().block_on(async { - let actor = Worker(Default::default()).start(); + let actor = Worker(Default::default()).spawn(); let mut cleanup = Cleanup(vec![actor.child_handle()]); let child = actor .request_with_timeout(SpawnBlockingChild, LIMIT) @@ -480,9 +480,9 @@ impl Handler for Worker { fn monitor_helper_survives_the_last_sharded_actor_stopping() { runtime().block_on(async { use spawned_concurrency::tasks::Backend; - let installer = Worker(Default::default()).start(); - let observer = Worker(Default::default()).start_with_backend(Backend::Thread); - let target = Worker(Default::default()).start_with_backend(Backend::Thread); + let installer = Worker(Default::default()).spawn(); + let observer = Worker(Default::default()).with_options(Backend::Thread).spawn().unwrap(); + let target = Worker(Default::default()).with_options(Backend::Thread).spawn().unwrap(); let _cleanup = Cleanup(vec![ installer.child_handle(), observer.child_handle(), diff --git a/benchmarks/experiments/tests/native_dispatcher.rs b/benchmarks/experiments/tests/native_dispatcher.rs index e0d5f6ec8..8fa801939 100644 --- a/benchmarks/experiments/tests/native_dispatcher.rs +++ b/benchmarks/experiments/tests/native_dispatcher.rs @@ -47,8 +47,8 @@ fn drop_panic_isolated_and_terminal_shutdown_rejects_new_actors() { .build() .unwrap(); runtime.block_on(async { - let bad = DropPanic.start(); - let healthy = Healthy.start(); + let bad = DropPanic.spawn(); + let healthy = Healthy.spawn(); assert_eq!(healthy.request_with_timeout(Ping, LIMIT).await.unwrap(), 42); bad.child_handle().stop(); assert_eq!( @@ -68,7 +68,7 @@ fn drop_panic_isolated_and_terminal_shutdown_rejects_new_actors() { spawned_rt::tasks::experimental::dispatcher::shutdown().unwrap(); spawned_rt::tasks::experimental::dispatcher::shutdown().unwrap(); runtime.block_on(async { - let refused = Healthy.start(); + let refused = Healthy.spawn(); assert_eq!( tokio::time::timeout(LIMIT, refused.wait_exit()) .await diff --git a/benchmarks/experiments/tests/support/deadline.rs b/benchmarks/experiments/tests/support/deadline.rs index 7c14ab612..e67f9aff9 100644 --- a/benchmarks/experiments/tests/support/deadline.rs +++ b/benchmarks/experiments/tests/support/deadline.rs @@ -9,7 +9,7 @@ //! #[macro_use] //! #[path = "support/deadline.rs"] //! mod deadline; -//! let tree = within!(tree.start()).await.unwrap(); +//! let tree = within!(tree.spawn()).await.unwrap(); //! ``` #![allow(dead_code, unused_macros)] use std::time::Duration; diff --git a/benchmarks/hotpath.py b/benchmarks/hotpath.py index c7bfc0be7..8a7820eef 100755 --- a/benchmarks/hotpath.py +++ b/benchmarks/hotpath.py @@ -56,6 +56,11 @@ def build(work, label, rev, features=None): shutil.copy(DRIVER, tree / "concurrency" / "examples" / "hotpath.rs") target = work / f"target-{label}" # one per side: identical crate hashes would collide env = dict(os.environ, CARGO_TARGET_DIR=str(target), CARGO_INCREMENTAL="0") + # Before the 0.9 unified start, `start()` was the non-waiting start: the + # driver's `spawn_actor!` uses it on such a tree (same workload). + actor_rs = tree / "concurrency" / "src" / "tasks" / "actor.rs" + if "fn start_with_backend" in actor_rs.read_text(): + env["RUSTFLAGS"] = (env.get("RUSTFLAGS", "") + " --cfg hotpath_legacy_start").strip() extra = ["--features", features] if features else [] sh(["cargo", "build", "--release", "--locked", "-q", "-p", "spawned-concurrency", "--example", "hotpath", *extra], cwd=tree, env=env) diff --git a/benchmarks/hotpath/driver.rs b/benchmarks/hotpath/driver.rs index d446b7eb6..cd33657d5 100644 --- a/benchmarks/hotpath/driver.rs +++ b/benchmarks/hotpath/driver.rs @@ -17,6 +17,11 @@ pub struct Hit(pub u64); impl Message for Hit { type Result = (); } +/// Stops the actor (the spawn benchmarks' full lifecycle). +pub struct Quit; +impl Message for Quit { + type Result = (); +} mod a { use super::*; @@ -33,6 +38,10 @@ mod a { async fn hit(&mut self, msg: Hit, _ctx: &Context) { self.0 = self.0.wrapping_add(msg.0); } + #[request_handler] + async fn quit(&mut self, _msg: Quit, ctx: &Context) { + ctx.stop(); + } } } @@ -51,6 +60,10 @@ mod t { fn hit(&mut self, msg: Hit, _ctx: &Context) { self.0 = self.0.wrapping_add(msg.0); } + #[request_handler] + fn quit(&mut self, _msg: Quit, ctx: &Context) { + ctx.stop(); + } } } @@ -67,6 +80,9 @@ const BENCHMARKS: &[(&str, bool)] = &[ ("async_ct_request_conc16", true), ("async_ct_send_p1", true), ("async_ct_send_p2", true), + // One actor's whole life per op: spawn, one request, a stop. + ("async_ct_spawn", true), + ("threads_spawn", false), ("threads_request_seq", true), ("threads_burst4", false), ("threads_send_p1", false), @@ -92,6 +108,22 @@ fn runtime(current_thread: bool) -> tokio::runtime::Runtime { builder.enable_all().build().unwrap() } +/// Start an actor without waiting for `started()`: `spawn()` since the 0.9 +/// unified start, `start()` before it (`hotpath.py` sets the cfg for an older +/// base), so both sides run the same workload. +#[cfg(not(hotpath_legacy_start))] +macro_rules! spawn_actor { + ($actor:expr) => { + $actor.spawn() + }; +} +#[cfg(hotpath_legacy_start)] +macro_rules! spawn_actor { + ($actor:expr) => { + $actor.start() + }; +} + /// Async actor benchmarks. Each returns the measured time; setup and warm-up /// are outside it (the gate cancels them by differencing two sizes). fn run_async(name: &str, n: u64) -> Duration { @@ -100,7 +132,28 @@ fn run_async(name: &str, n: u64) -> Duration { let rt = runtime(current_thread); let name = name.replacen("async_ct_", "async_", 1); let name = name.as_str(); - let actor = rt.block_on(async { a::Sink(0).start() }); + if name == "async_spawn" { + return rt.block_on(async { + let life = |i: u64| async move { + let actor = spawn_actor!(a::Sink(i)); + std::hint::black_box(actor.request(Q(i)).await.unwrap()); + actor.request(Quit).await.unwrap(); + // Let it reach its death point and publish. + for _ in 0..4 { + tokio::task::yield_now().await; + } + }; + for i in 0..WARMUP { + life(i).await; + } + let start = Instant::now(); + for i in 0..n { + life(i).await; + } + start.elapsed() + }); + } + let actor = rt.block_on(async { spawn_actor!(a::Sink(0)) }); let fence = |actor: &spawned_concurrency::tasks::ActorRef| { rt.block_on(actor.request(Q(0))).unwrap(); }; @@ -192,7 +245,16 @@ fn run_async(name: &str, n: u64) -> Duration { fn run_threads(name: &str, n: u64) -> Duration { use spawned_concurrency::threads::ActorStart; - let actor = t::Sink(0).start(); + if name == "threads_spawn" { + let start = Instant::now(); + for i in 0..n { + let actor = spawn_actor!(t::Sink(i)); + std::hint::black_box(actor.request(Q(i)).unwrap()); + actor.request(Quit).unwrap(); + } + return start.elapsed(); + } + let actor = spawn_actor!(t::Sink(0)); for i in 0..WARMUP { actor.request(Q(i)).unwrap(); } diff --git a/concurrency/Cargo.toml b/concurrency/Cargo.toml index 549d3ce60..0585d15f7 100644 --- a/concurrency/Cargo.toml +++ b/concurrency/Cargo.toml @@ -64,7 +64,9 @@ failpoints = [] loom = { version = "0.7", features = ["futures"] } [lints.rust] -unexpected_cfgs = { level = "warn", check-cfg = ['cfg(spawned_scheduler_loom)', 'cfg(spawned_sim)', 'cfg(kani)', 'cfg(kani_slow)'] } +# `started()` impls refine the trait's `impl Into>` on purpose. +refining_impl_trait = { level = "allow", priority = -1 } +unexpected_cfgs = { level = "warn", check-cfg = ['cfg(spawned_scheduler_loom)', 'cfg(spawned_sim)', 'cfg(kani)', 'cfg(kani_slow)', 'cfg(hotpath_legacy_start)'] } # clippy.toml bans direct std/parking_lot synchronization in the Loom-checked # cores; only the Loom harnesses deny it (on the included core), see there. diff --git a/concurrency/README.md b/concurrency/README.md index 2b5d9ec16..5d3f69b89 100644 --- a/concurrency/README.md +++ b/concurrency/README.md @@ -35,7 +35,7 @@ impl Handler for MyCounter { fn main() { rt::run(async { - let counter = MyCounter { count: 0 }.start(); + let counter = MyCounter { count: 0 }.spawn(); counter.increment(); let value = counter.get().await.unwrap(); assert_eq!(value, 1); diff --git a/concurrency/benches/baseline.rs b/concurrency/benches/baseline.rs index 2ace42c13..eae7890b7 100644 --- a/concurrency/benches/baseline.rs +++ b/concurrency/benches/baseline.rs @@ -282,9 +282,13 @@ fn main() { "blocking" => tasks::Backend::Blocking, _ => tasks::Backend::Thread, }; - Target::Tasks(tasks::ActorStart::start_with_backend(Sink, backend)) + Target::Tasks( + tasks::ActorStart::with_options(Sink, backend) + .spawn() + .unwrap(), + ) } - "threads" => Target::Threads(threads::ActorStart::start(Sink)), + "threads" => Target::Threads(threads::ActorStart::spawn(Sink)), "tokio" => { let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel::<( Work, diff --git a/concurrency/benches/metrics_overhead.rs b/concurrency/benches/metrics_overhead.rs index 7b67de52a..d8ca92a32 100644 --- a/concurrency/benches/metrics_overhead.rs +++ b/concurrency/benches/metrics_overhead.rs @@ -66,7 +66,7 @@ fn main() { .build() .unwrap(); rt.block_on(async { - let actors: Vec<_> = (0..actors).map(|_| Sink.start()).collect(); + let actors: Vec<_> = (0..actors).map(|_| Sink.spawn()).collect(); for actor in &actors { black_box(actor.request(Work(work)).await.unwrap()); } @@ -100,7 +100,7 @@ fn main() { } else { assert_eq!(mode, "threads"); use threads::ActorStart; - let actors: Vec<_> = (0..actors).map(|_| Sink.start()).collect(); + let actors: Vec<_> = (0..actors).map(|_| Sink.spawn()).collect(); for actor in &actors { black_box(actor.request(Work(work)).unwrap()); } diff --git a/concurrency/benches/soak/native.rs b/concurrency/benches/soak/native.rs index 085ead37d..61e63533c 100644 --- a/concurrency/benches/soak/native.rs +++ b/concurrency/benches/soak/native.rs @@ -247,7 +247,7 @@ fn watcher() -> ActorRef { downs: 0, exits: 0, } - .start(), + .spawn(), ) } @@ -323,7 +323,7 @@ pub const COUNT: usize = 5; fn churn(rng: &mut Rng) -> R { let actors: Vec<_> = (0..1 + rng.below(4)) - .map(|_| track(NWorker::new().start())) + .map(|_| track(NWorker::new().spawn())) .collect(); for actor in &actors { for expected in 1..=1 + rng.below(3) { @@ -339,7 +339,7 @@ fn churn(rng: &mut Rng) -> R { fn links_monitors(rng: &mut Rng) -> R { let watcher = watcher(); - let peer = track(NWorker::new().start()); + let peer = track(NWorker::new().spawn()); if rng.one_in(2) { let trap = rng.one_in(2); watcher.request(Trap(trap)).map_err(e("trap"))?; @@ -377,7 +377,7 @@ fn links_monitors(rng: &mut Rng) -> R { } fn timers_streams(rng: &mut Rng) -> R { - let actor = track(NWorker::new().start()); + let actor = track(NWorker::new().spawn()); if rng.one_in(2) { let once = 1 + rng.below(3); let period = Duration::from_millis(1 + rng.below(3)); @@ -399,7 +399,7 @@ fn timers_streams(rng: &mut Rng) -> R { } fn registry_requests(rng: &mut Rng, stats: &Stats) -> R { - let actor = track(NWorker::new().start()); + let actor = track(NWorker::new().spawn()); let id = stats.next_id(); let name = format!("soak-native-{id}"); registry::register(&name, actor.clone()).map_err(e("register"))?; @@ -427,7 +427,7 @@ fn registry_requests(rng: &mut Rng, stats: &Stats) -> R { } fn burst(rng: &mut Rng) -> R { - let actor = track(NWorker::new().start()); + let actor = track(NWorker::new().spawn()); let n = 1_000 + rng.below(10_000); for _ in 0..n { actor.send(Bump).map_err(e("burst send"))?; diff --git a/concurrency/benches/soak/workload.rs b/concurrency/benches/soak/workload.rs index 5e4c319a9..58789cc27 100644 --- a/concurrency/benches/soak/workload.rs +++ b/concurrency/benches/soak/workload.rs @@ -289,7 +289,7 @@ impl Handler for Watcher { reply: spawned_concurrency::PendingReply<'_, ActorRef>, ctx: &Context, ) { - let value = { Worker::new().start_linked(ctx) }; + let value = { Worker::new().start_link(ctx).actor_ref().unwrap() }; reply.reply(value); } } @@ -454,7 +454,7 @@ pub async fn run(index: usize, rng: &mut Rng, env: &Env) -> R { /// Start a few actors on one backend, exercise requests, end each one. async fn churn(rng: &mut Rng, backend: Backend) -> R { let actors: Vec<_> = (0..1 + rng.below(6)) - .map(|_| track(Worker::new().start_with_backend(backend))) + .map(|_| track(Worker::new().with_options(backend).spawn().unwrap())) .collect(); for actor in &actors { for expected in 1..=1 + rng.below(3) { @@ -478,9 +478,9 @@ async fn group(rng: &mut Rng, env: &Env) -> R { }; let mut actors = Vec::new(); for _ in 0..1 + rng.below(3) { - match group.start(Worker::new()) { + match Worker::new().with_options(group).spawn() { Ok(actor) => actors.push(track(actor)), - Err(SystemError::Capacity) => { + Err(spawned_concurrency::StartError::Capacity) => { env.stats .capacity .fetch_add(1, std::sync::atomic::Ordering::Relaxed); @@ -605,8 +605,13 @@ async fn supervisor(rng: &mut Rng, env: &Env, strategy: SupervisorStrategy, nest /// Bidirectional links: trapped exits, propagated exits, `start_linked`, unlink. async fn links(rng: &mut Rng) -> R { - let watcher = track(Watcher::new().start()); - let peer = track(Worker::new().start_with_backend(backend(rng))); + let watcher = track(Watcher::new().spawn()); + let peer = track( + Worker::new() + .with_options(backend(rng)) + .spawn() + .expect("spawn"), + ); match rng.below(4) { 0 => { watcher.request(Trap(true)).await.map_err(e("trap"))?; @@ -666,9 +671,16 @@ async fn links(rng: &mut Rng) -> R { /// Monitors, demonitors, monitoring an already-dead actor, and a watcher /// that stops before its targets. async fn monitors(rng: &mut Rng) -> R { - let watcher = track(Watcher::new().start()); + let watcher = track(Watcher::new().spawn()); let targets: Vec<_> = (0..1 + rng.below(4)) - .map(|_| track(Worker::new().start_with_backend(backend(rng)))) + .map(|_| { + track( + Worker::new() + .with_options(backend(rng)) + .spawn() + .expect("spawn"), + ) + }) .collect(); let mut expected = 0; for target in &targets { @@ -712,7 +724,12 @@ async fn monitors(rng: &mut Rng) -> R { /// `send_after` and `send_interval`, cancelled, fired, or live at stop. async fn timers(rng: &mut Rng) -> R { - let actor = track(Worker::new().start_with_backend(backend(rng))); + let actor = track( + Worker::new() + .with_options(backend(rng)) + .spawn() + .expect("spawn"), + ); let once = 1 + rng.below(4); let period = Duration::from_millis(1 + rng.below(4)); actor @@ -736,7 +753,7 @@ async fn timers(rng: &mut Rng) -> R { /// Unmanaged registry: register, look up, reject duplicates, unregister. async fn registry_names(rng: &mut Rng, env: &Env) -> R { - let actor = track(Worker::new().start()); + let actor = track(Worker::new().spawn()); let leak = leaks().registry; for _ in 0..1 + rng.below(4) { let id = env.stats.next_id(); @@ -762,7 +779,12 @@ async fn registry_names(rng: &mut Rng, env: &Env) -> R { /// Requests that time out, requests dropped mid-flight, requests to a stopped actor. async fn requests(rng: &mut Rng) -> R { - let actor = track(Worker::new().start_with_backend(backend(rng))); + let actor = track( + Worker::new() + .with_options(backend(rng)) + .spawn() + .expect("spawn"), + ); let timed_out = actor .request_with_timeout( Slow(Duration::from_millis(20 + rng.below(20))), @@ -799,7 +821,12 @@ async fn requests(rng: &mut Rng) -> R { /// Stream listeners: a finite stream drained into the actor, and an endless /// one that must end when the actor stops. async fn streams(rng: &mut Rng) -> R { - let actor = track(Worker::new().start_with_backend(backend(rng))); + let actor = track( + Worker::new() + .with_options(backend(rng)) + .spawn() + .expect("spawn"), + ); let n = 1 + rng.below(64); drop(spawn_listener( actor.context(), @@ -835,10 +862,20 @@ async fn system_cycle(rng: &mut Rng) -> R { }; let mut actors = Vec::new(); for _ in 0..1 + rng.below(4) { - actors.push(group.start(Worker::new()).map_err(e("start"))?); + actors.push( + Worker::new() + .with_options(&group) + .spawn() + .map_err(e("start"))?, + ); } if let Some(native) = &native { - actors.push(native.start(Worker::new()).map_err(e("native start"))?); + actors.push( + Worker::new() + .with_options(native) + .spawn() + .map_err(e("native start"))?, + ); } for actor in &actors { ensure!(actor.request(Inc).await.map_err(e("inc"))? == 1, "inc"); @@ -869,7 +906,12 @@ async fn system_cycle(rng: &mut Rng) -> R { /// Fill a mailbox with thousands of messages, then drain it or stop with it full. async fn burst(rng: &mut Rng) -> R { - let actor = track(Worker::new().start_with_backend(backend(rng))); + let actor = track( + Worker::new() + .with_options(backend(rng)) + .spawn() + .expect("spawn"), + ); let n = 1_000 + rng.below(20_000); for _ in 0..n { actor.send(Bump).map_err(e("burst send"))?; diff --git a/concurrency/examples/panic_abort.rs b/concurrency/examples/panic_abort.rs index 79aecdbec..9d5e71a4e 100644 --- a/concurrency/examples/panic_abort.rs +++ b/concurrency/examples/panic_abort.rs @@ -53,7 +53,7 @@ mod native { } } pub fn handler_panic() { - let actor = Victim.start(); + let actor = Victim.spawn(); println!("request sent"); let result = actor.request_with_timeout(Boom, LIMIT); println!("caller observed {result:?}"); @@ -82,7 +82,7 @@ mod tasks { } pub fn handler_panic() { tokio::runtime::Runtime::new().unwrap().block_on(async { - let actor = Victim.start(); + let actor = Victim.spawn(); println!("request sent"); let result = actor.request_with_timeout(Boom, LIMIT).await; println!("caller observed {result:?}"); diff --git a/concurrency/src/actor_cleanup.rs b/concurrency/src/actor_cleanup.rs index ed1605116..0a24608b2 100644 --- a/concurrency/src/actor_cleanup.rs +++ b/concurrency/src/actor_cleanup.rs @@ -64,6 +64,11 @@ pub(crate) struct ActorCleanup { skip_log: Option, /// [`discharged`](Self::discharged) ran. discharged: bool, + /// The startup cell and the start edge's parent (unified start): the + /// edge's notice is disposed of through the cell at publication. + startup: Option<(std::sync::Arc, Option)>, + /// The start-time name, released at publication before any notice. + name: Option, } impl ActorCleanup { pub(crate) fn new( @@ -89,8 +94,24 @@ impl ActorCleanup { jobs: None, skip_log: None, discharged: false, + startup: None, + name: None, } } + /// The instance's startup cell and its start edge's parent. + pub(crate) fn with_startup( + mut self, + cell: std::sync::Arc, + edge_peer: Option, + ) -> Self { + self.startup = Some((cell, edge_peer)); + self + } + /// The start-time name, held until publication. + pub(crate) fn with_name(mut self, name: Option) -> Self { + self.name = name; + self + } /// Seal `replies` with the final reason at D, inside the exit-table /// critical section: callers then get `ActorStopped(final reason)` and /// later sends `NoProc`. @@ -172,6 +193,11 @@ impl ActorCleanup { if skipped { self.skip_log = Some(reason.clone()); } + // A start never decided (a runtime drop, an unwound lifecycle) fails + // with the final reason. + if let Some((cell, _)) = &self.startup { + cell.fail_at_death(&reason); + } fail_point!("actor.at_death"); // Every open deferred reply (and every exit-pending one) is answered // with the final reason, before any destructor runs. Their wake-ups @@ -272,13 +298,40 @@ impl Drop for ActorCleanup { // The trace's `Publish` is this point: destruction is over and the // notices are delivered after it. let id = self.id.raw(); + // The start-time name is released first, after destruction and + // before any notice, wait_exit or answer (unified start 5.2, 7). + drop(self.name.take()); trace::life_section(id, || { trace::life_emit(id, || trace::LifeEvent::new("Publish", id)); }); - if let Some(notices) = self.notices.take() { - notices.deliver(); + let mut notices = self.notices.take(); + trace::start_section(|| { + #[cfg(feature = "lifecycle-trace")] + if crate::link::lock_request(&self.cause).trace == trace::StartRole::Parent { + trace::start_emit_env(|| trace::StartEvent::new("PPublish", 0)); + } + // The start edge's notice is disposed of by the commitment and + // the waiter; the others are ordinary. + let edge = match &self.startup { + Some((cell, peer)) => { + let notice = + peer.and_then(|peer| notices.as_mut().and_then(|n| n.take_notice(peer))); + cell.publish(&reason, notice) + } + None => None, + }; + if let Some(notices) = notices.take() { + notices.deliver(); + } + if let Some(edge) = edge { + edge.deliver(); + } + }); + self.completion.send_replace(Some(reason.clone())); + // Answers become available only now that wait_exit is published. + if let Some((cell, _)) = &self.startup { + cell.published(&reason); } - self.completion.send_replace(Some(reason)); } } diff --git a/concurrency/src/child_handle.rs b/concurrency/src/child_handle.rs index 3b94d2136..6ab2bc7f8 100644 --- a/concurrency/src/child_handle.rs +++ b/concurrency/src/child_handle.rs @@ -170,7 +170,7 @@ impl ChildHandle { /// runs. `stop_with_reason(Kill)` ends `Kill`; use [`kill`](Self::kill) /// for the untrappable kill signal that ends `Killed`. pub fn stop_with_reason(&self, reason: ExitReason) { - crate::link::stop_with_reason(self.target.id, &self.target.request, reason, || { + crate::link::admin_stop_with_reason(self.target.id, &self.target.request, reason, || { (self.target.wake)() }); } @@ -391,6 +391,7 @@ pub(crate) mod tests { send_exit: Arc::new(|_| Ok(())), send_parent_exit: Arc::new(|_| Ok(())), send_held: Arc::new(|_, _| Ok(())), + startup: crate::start::StartupCell::new(), }) } @@ -503,7 +504,7 @@ pub(crate) mod tests { } } - let actor = Worker.start(); + let actor = Worker.spawn(); let handle = actor.child_handle(); assert!(handle.is_alive()); @@ -547,7 +548,7 @@ pub(crate) mod tests { let runtime = spawned_rt::tasks::Runtime::new().unwrap(); runtime.block_on(async { - let actor = Worker.start(); + let actor = Worker.spawn(); let handle = actor.child_handle(); assert!(handle.is_alive()); @@ -574,7 +575,7 @@ pub(crate) mod tests { let runtime = spawned_rt::tasks::Runtime::new().unwrap(); runtime.block_on(async { - let actor = Idler.start(); + let actor = Idler.spawn(); let handle = actor.child_handle(); assert!(handle.is_alive()); @@ -601,7 +602,7 @@ pub(crate) mod tests { // Multi-thread runtime — wait_exit_blocking should work via block_in_place let runtime = spawned_rt::tasks::Runtime::new().unwrap(); runtime.block_on(async { - let actor = Idler.start(); + let actor = Idler.spawn(); let handle = actor.child_handle(); handle.stop(); @@ -631,7 +632,7 @@ pub(crate) mod tests { .unwrap(); runtime.block_on(async { - let actor = Idler.start(); + let actor = Idler.spawn(); let handle = actor.child_handle(); // Calling wait_exit_blocking from within a current-thread runtime // would deadlock the actor task — we panic with a clear message instead. @@ -661,7 +662,7 @@ pub(crate) mod tests { // Fast path: if the actor has already exited, wait_exit_blocking is safe // even on a current-thread runtime (no actual blocking happens). runtime.block_on(async { - let actor = Idler.start(); + let actor = Idler.spawn(); let handle = actor.child_handle(); handle.stop(); // Wait async first so the actor actually exits @@ -706,7 +707,7 @@ pub(crate) mod tests { } } - let actor = Boomer.start(); + let actor = Boomer.spawn(); let handle = actor.child_handle(); let _ = actor.send(Boom); @@ -755,7 +756,7 @@ pub(crate) mod tests { let runtime = spawned_rt::tasks::Runtime::new().unwrap(); runtime.block_on(async { - let actor = Boomer.start(); + let actor = Boomer.spawn(); let handle = actor.child_handle(); let _ = actor.send(Boom); @@ -792,7 +793,7 @@ pub(crate) mod tests { let runtime = spawned_rt::tasks::Runtime::new().unwrap(); runtime.block_on(async { - let actor = PanicStart.start(); + let actor = PanicStart.spawn(); let handle = actor.child_handle(); let reason = handle.wait_exit_async().await; diff --git a/concurrency/src/failpoints.rs b/concurrency/src/failpoints.rs index 16ebc8b0e..fa5ddf1c7 100644 --- a/concurrency/src/failpoints.rs +++ b/concurrency/src/failpoints.rs @@ -188,6 +188,21 @@ pub const SITES: &[Site] = &[ description: "Admission of a new scope on an execution group (actor start or \ supervised generation). Error reports `SystemError::Capacity`.", }, + Site { + name: "system.dispatch_actor", + has_error: true, + description: "Dispatch of a committed managed actor to its worker. Error models \ + a group dropped mid-shutdown or a closed native worker channel: \ + the child is killed before started() (`KilledEarly`) and its \ + start answers `InitExited { Killed }`.", + }, + Site { + name: "actor.thread_spawn", + has_error: true, + description: "OS thread spawn of a committed thread-backed actor. Error models \ + a failed spawn: the child is killed before started() and its \ + start answers `InitExited { Killed }`.", + }, Site { name: "registry.before_register", has_error: true, diff --git a/concurrency/src/lib.rs b/concurrency/src/lib.rs index a452fdaa1..5023f4cfa 100644 --- a/concurrency/src/lib.rs +++ b/concurrency/src/lib.rs @@ -105,6 +105,7 @@ pub mod registry; #[doc(hidden)] pub mod reply; pub mod response; +mod start; #[cfg(feature = "async")] mod stop_token; #[cfg(feature = "async")] @@ -124,6 +125,7 @@ pub use reply::{ ReplyReceiver, ReplyTo, }; pub use response::{Response, DEFAULT_REQUEST_TIMEOUT}; +pub use start::{StartError, StartOutcome, Timeout}; // send_handler also supports manual Handler impls. request_handler and // deferred_handler are method markers consumed by #[actor], not standalone // proc macros to re-export. diff --git a/concurrency/src/lifecycle_trace.rs b/concurrency/src/lifecycle_trace.rs index 1c069c144..a2a071f6c 100644 --- a/concurrency/src/lifecycle_trace.rs +++ b/concurrency/src/lifecycle_trace.rs @@ -275,6 +275,71 @@ impl LifeEvent { } } +/// One action of `verification/tla/Start.tla` (unified start, design +/// 16.2), recorded for the starts made while `record_start` is on. `c` is +/// the child (an `ActorId` number, normalized by the checker; 0 for an +/// action of the parent, a group or a lookup miss); `x`, `y`, `z` and `f` +/// are the action's parameters (see `TraceStart.tla`). +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct StartEvent { + pub action: &'static str, + pub c: u64, + pub x: &'static str, + pub y: &'static str, + pub z: &'static str, + pub f: bool, +} + +impl StartEvent { + #[allow(dead_code)] // recording is compiled out without the feature + pub(crate) fn new(action: &'static str, c: u64) -> Self { + Self { + action, + c, + x: "", + y: "", + z: "", + f: false, + } + } + #[allow(dead_code)] + pub(crate) fn x(mut self, x: &'static str) -> Self { + self.x = x; + self + } + #[allow(dead_code)] + pub(crate) fn y(mut self, y: &'static str) -> Self { + self.y = y; + self + } + #[allow(dead_code)] + pub(crate) fn z(mut self, z: &'static str) -> Self { + self.z = z; + self + } + #[allow(dead_code)] + pub(crate) fn f(mut self, f: bool) -> Self { + self.f = f; + self + } + /// JSON object understood by `verification/tla/trace_check.py`. + pub fn to_json(&self) -> String { + format!( + r#"{{"a":"{}","c":{},"x":"{}","y":"{}","z":"{}","f":{}}}"#, + self.action, self.c, self.x, self.y, self.z, self.f + ) + } +} + +/// An instance's role in the start trace. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub(crate) enum StartRole { + #[default] + None, + Child(u64), + Parent, +} + /// Model name of an exit reason (`Lifecycle.tla` `Reasons`). #[allow(dead_code)] // recording is compiled out without the feature pub(crate) fn life_reason(reason: &crate::ExitReason) -> &'static str { @@ -614,13 +679,22 @@ mod imp { /// keyed by the actor whose locks it takes first, and nested sections use /// the same key, so the section is always the outermost lock. pub(crate) fn life_section(id: u64, f: impl FnOnce() -> R) -> R { - if life_traced(id) { + // Also for the start trace's actors: a message's admission (`SSend`) + // and its dispatch (`SHandle`) are then one step each of the same + // linearization (the item is receivable only after `SSend`). + if life_traced(id) || start_member(id) { atomic(f) } else { f() } } + /// Whether `id` is the start trace's parent or one of its children. + fn start_member(id: u64) -> bool { + start_on() + && (START_PARENT.load(Ordering::Acquire) == id || lock(&START_KIDS).contains(&id)) + } + /// Record an event of actor `t` (inside its section). pub(crate) fn life_emit(t: u64, event: impl FnOnce() -> super::LifeEvent) { if life_traced(t) { @@ -640,6 +714,225 @@ mod imp { }); } + // Unified start (`Start.tla`): one process-wide recording at a time, of + // the starts made while it is on and of their parent P. Events are + // pushed under the lock of the state they describe (a leaf push); + // multi-lock steps run in the trace section (`start_section`). + static START_ON: AtomicBool = AtomicBool::new(false); + static START_PARENT: AtomicU64 = AtomicU64::new(0); + static START_KIDS: Mutex> = + Mutex::new(std::collections::BTreeSet::new()); + static START: Mutex> = Mutex::new(Vec::new()); + thread_local! { + /// A child's notice is being applied to P (the model's Deliver). + static IN_NOTICE: Cell = const { Cell::new(false) }; + } + + /// Start recording the unified-start events of every start made from + /// now on, with `parent` (an `ActorId` number, 0 for none) as the + /// model's P. Tests that record must not run concurrently. + pub fn record_start(parent: u64) { + atomic(|| { + lock(&START_KIDS).clear(); + lock(&START).clear(); + START_PARENT.store(parent, Ordering::Release); + START_ON.store(true, Ordering::Release); + }); + } + + /// Stop recording and return the events in their recorded order. + pub fn take_start() -> Vec { + atomic(|| { + START_ON.store(false, Ordering::Release); + START_PARENT.store(0, Ordering::Release); + lock(&START_KIDS).clear(); + std::mem::take(&mut *lock(&START)) + }) + } + + /// How many recorded events named `action` (with `x`, if non-empty) the + /// start trace holds (lets a test wait for an ordering point). + pub fn start_count(action: &str, x: &str) -> usize { + lock(&START) + .iter() + .filter(|e| e.action == action && (x.is_empty() || e.x == x)) + .count() + } + + pub(crate) fn start_on() -> bool { + START_ON.load(Ordering::Acquire) + } + + /// A start begins: its child `id` is recorded from now on. + pub(crate) fn start_child(id: u64) -> u64 { + if !start_on() { + return 0; + } + lock(&START_KIDS).insert(id); + id + } + + thread_local! { + /// The stage-1 commit in progress on this thread: (child, placement, + /// already recorded). + static COMMIT: std::cell::Cell<(u64, &'static str, bool)> = + const { std::cell::Cell::new((0, "", false)) }; + } + + /// A start's stage-1 commit begins on this thread. Its `SCommit` or + /// `SRefuse` is recorded at the decision point ([`commit_point`]), + /// under the innermost lock held there (the registry for a named start, + /// else the parent's exit request or the group), so it is ordered with + /// `SReleaseName`, `SLookup` and `SReleaseSlot`, recorded under the same + /// locks. + pub(crate) fn commit_begin(c: u64, placement: &'static str) { + COMMIT.set((c, placement, false)); + } + + /// The commit is decided here: `None` committed, `Some(refusal)` refused. + /// Only the first decision of a commit is recorded. + pub(crate) fn commit_point(refusal: Option<&'static str>) { + let (c, placement, done) = COMMIT.get(); + if c == 0 || done { + return; + } + COMMIT.set((c, placement, true)); + start_emit(c, || match refusal { + None => super::StartEvent::new("SCommit", c).x(placement), + Some(refusal) => super::StartEvent::new("SRefuse", c).x(refusal), + }); + } + + /// The commit returned: records its decision if no lock holder did (a + /// start with no shared lock to order against), and ends it. + pub(crate) fn commit_end(refusal: Option<&'static str>) { + commit_point(refusal); + COMMIT.set((0, "", false)); + } + + /// `id` if it is a recorded child, else 0. + pub(crate) fn start_child_of(id: u64) -> u64 { + if start_on() && lock(&START_KIDS).contains(&id) { + id + } else { + 0 + } + } + + /// Start recording the unified-start events with `parent` as the + /// model's P (its exit begin, death point and publication are recorded + /// as `PExitBegin`, `PDie` and `PPublish`). Tests that record must not run + /// concurrently. + pub fn record_start_parent(parent: &crate::ChildHandle) { + crate::link::lock_request(&parent.signal_target().request).trace = super::StartRole::Parent; + record_start(parent.id().raw()); + } + + /// Record an event of recorded child `c` (0: not recorded). + pub(crate) fn start_emit(c: u64, event: impl FnOnce() -> super::StartEvent) { + if c != 0 && start_on() { + lock(&START).push(event()); + } + } + + /// Record an event of the parent, a group or the environment. + pub(crate) fn start_emit_env(event: impl FnOnce() -> super::StartEvent) { + if start_on() { + lock(&START).push(event()); + } + } + + /// Run `f` as one step of the start trace's linearization when + /// recording (it must not await, and no framework lock may be held). + pub(crate) fn start_section(f: impl FnOnce() -> R) -> R { + if start_on() { + atomic(f) + } else { + f() + } + } + + pub(crate) fn start_admin(role: super::StartRole) { + if let super::StartRole::Child(c) = role { + start_emit(c, || super::StartEvent::new("SAdminStop", c)); + } + } + + pub(crate) fn start_kill(role: super::StartRole, _starting: bool) { + if let super::StartRole::Child(c) = role { + start_emit(c, || super::StartEvent::new("SKill", c)); + } + } + + pub(crate) fn start_exit(role: super::StartRole, reason: &crate::ExitReason) { + match role { + super::StartRole::Child(c) => start_emit(c, || { + super::StartEvent::new("SExit", c).x(super::life_reason(reason)) + }), + super::StartRole::Parent if !IN_NOTICE.with(Cell::get) => start_emit_env(|| { + super::StartEvent::new("PExitBegin", 0).x(super::life_reason(reason)) + }), + _ => {} + } + } + + pub(crate) fn start_die(role: super::StartRole, reason: &crate::ExitReason, frozen: bool) { + match role { + super::StartRole::Child(c) => start_emit(c, || { + super::StartEvent::new("SDie", c) + .x(super::life_reason(reason)) + .f(frozen) + }), + super::StartRole::Parent => start_emit_env(|| super::StartEvent::new("PDie", 0)), + super::StartRole::None => {} + } + } + + /// A recorded child's run loop took a message of kind `kind` (`Handle`; + /// exit envelopes are recorded by the exit they begin). + pub(crate) fn start_dispatch(id: u64, kind: &'static str) { + if matches!(kind, "req" | "msg") { + let c = start_child_of(id); + start_emit(c, || super::StartEvent::new("SHandle", c)); + } + } + + /// A recorded child unlinked from P (`ChildUnlink`). + pub(crate) fn start_unlink(own: u64, peer: u64) { + if start_on() && START_PARENT.load(Ordering::Acquire) == peer { + let c = start_child_of(own); + start_emit(c, || super::StartEvent::new("SUnlink", c)); + } + } + + /// A child's notice to P is applied or enqueued (the model's Deliver): + /// P's own exit that it begins is part of that step. + pub(crate) fn start_notice( + receiver: u64, + from: u64, + kind: &'static str, + reason: &crate::ExitReason, + f: impl FnOnce() -> R, + ) -> R { + if !start_on() || START_PARENT.load(Ordering::Acquire) != receiver { + return f(); + } + let c = if lock(&START_KIDS).contains(&from) { + from + } else { + 0 + }; + start_emit(c, || { + super::StartEvent::new("PNotice", c) + .x(kind) + .z(super::life_reason(reason)) + }); + let previous = IN_NOTICE.with(|n| n.replace(true)); + let result = f(); + IN_NOTICE.with(|n| n.set(previous)); + result + } + /// The model's mailbox item kind of a message of type `M`. pub(crate) fn life_kind() -> &'static str { if std::any::TypeId::of::() == std::any::TypeId::of::() { @@ -663,6 +956,10 @@ mod imp { .s(if admitted { from } else { 0 }) .x(if admitted { "queued" } else { "noproc" }) }); + if admitted { + let c = start_child_of(owner); + start_emit(c, || super::StartEvent::new("SSend", c)); + } } /// The recorded actor whose turn (or hook) is sending a plain message @@ -1202,6 +1499,52 @@ mod imp { } #[allow(dead_code)] #[inline(always)] + pub(crate) fn start_on() -> bool { + false + } + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn start_child(_: u64) -> u64 { + 0 + } + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn commit_begin(_: u64, _: &'static str) {} + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn commit_point(_: Option<&'static str>) {} + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn commit_end(_: Option<&'static str>) {} + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn start_child_of(_: u64) -> u64 { + 0 + } + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn start_emit(_: u64, _: impl FnOnce() -> super::StartEvent) {} + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn start_emit_env(_: impl FnOnce() -> super::StartEvent) {} + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn start_section(f: impl FnOnce() -> R) -> R { + f() + } + #[allow(dead_code)] + #[inline(always)] + pub(crate) fn start_notice( + _: u64, + _: u64, + _: &'static str, + _: &crate::ExitReason, + f: impl FnOnce() -> R, + ) -> R { + f() + } + #[allow(dead_code)] + #[inline(always)] pub(crate) fn disposal(_: &'static str, _: u64, _: u64, _: u64, _: &'static str) {} #[inline(always)] pub(crate) fn disposal_section(f: impl FnOnce() -> R) -> R { @@ -1216,6 +1559,7 @@ mod imp { #[cfg(feature = "lifecycle-trace")] pub use imp::{ - record_disposal, record_lifecycle, recorded, recorded_with, take, take_disposal, take_jobs, - take_lifecycle, take_replies, take_tree, watch_deferred, + record_disposal, record_lifecycle, record_start, record_start_parent, recorded, recorded_with, + start_count, take, take_disposal, take_jobs, take_lifecycle, take_replies, take_start, + take_tree, watch_deferred, }; diff --git a/concurrency/src/link.rs b/concurrency/src/link.rs index c5db4d08f..542fceec5 100644 --- a/concurrency/src/link.rs +++ b/concurrency/src/link.rs @@ -76,6 +76,21 @@ pub(crate) struct ExitRequest { /// A parent exit is queued behind earlier messages of a trapping actor /// (E3): when dispatched it ends the actor whatever its reason. parent_exit_queued: bool, + /// The first administrative stop that arrived during `started()`: as + /// OTP's `gen_server:stop` (a system message handled after `init/1` + /// returns), it is applied after a ready start commits, and `stopped()` + /// runs (unified start 2.3). + admin: Option, + /// A non-ready startup decision fixed the reason (unified start 2.3): + /// the resolution and D are one step, so nothing changes it any more. + frozen: bool, + /// The `start_link`/`start_monitor` waits of this actor (as a parent): + /// an exit begun by a kill or an untrapped exit signal abandons them, + /// as in OTP the caller dies at once. + waiting_starts: Vec>, + /// The start-trace role of this instance (0: not recorded). + #[cfg(feature = "lifecycle-trace")] + pub(crate) trace: crate::lifecycle_trace::StartRole, } impl ExitRequest { @@ -93,6 +108,37 @@ impl ExitRequest { pub(crate) fn exit_pending(&self) -> bool { self.reason.is_some() || self.parent_exit_queued } + /// The exit was begun by a kill or an untrapped exit signal: in OTP the + /// process is dead at once (Spawned: once its running turn returns). + fn exit_by_signal(&self) -> bool { + self.killed || (self.reason.is_some() && self.skip_hook) + } + /// Register a start this actor waits for; `false` if its exit by a + /// signal already began (the wait is abandoned at once). + pub(crate) fn add_waiting_start( + &mut self, + cell: std::sync::Weak, + ) -> bool { + if self.dead || self.exit_by_signal() { + return false; + } + self.waiting_starts.retain(|w| w.strong_count() > 0); + self.waiting_starts.push(cell); + true + } + /// The start was answered or abandoned: forget it. + pub(crate) fn remove_waiting_start(&mut self, cell: *const crate::start::StartupCell) { + self.waiting_starts + .retain(|w| w.strong_count() > 0 && w.as_ptr() != cell); + } + /// The waits to abandon once a signal began this actor's exit. + fn take_waiting_starts(&mut self) -> Vec> { + if self.exit_by_signal() { + std::mem::take(&mut self.waiting_starts) + } else { + Vec::new() + } + } /// What began an exit that skips `stopped()`, for its debug log. pub(crate) fn skip_cause(&self) -> &'static str { if self.killed { @@ -108,33 +154,61 @@ impl ExitRequest { /// A self-exit or an administrative stop: the first requested reason /// wins. Returns whether the run loop must be woken. pub(crate) fn request(&mut self, reason: ExitReason) -> bool { - if self.dead { + if self.dead || self.frozen { return false; } if self.reason.is_none() { + self.note_exit(&reason); self.reason = Some(reason); } true } + /// An administrative stop (`ChildHandle::stop*`, a scope's stop). One + /// that arrives while `started()` runs is held: it decides nothing about + /// the start, and is applied once the start commits ready (OTP: a + /// `gen_server:stop` waits for `init/1` to return); a failed start + /// discards it. Returns whether the run loop must be woken. + pub(crate) fn admin_stop(&mut self, reason: ExitReason) -> bool { + if self.dead || self.frozen { + return false; + } + if self.starting { + if self.admin.is_none() && self.reason.is_none() { + #[cfg(feature = "lifecycle-trace")] + crate::lifecycle_trace::start_admin(self.trace); + self.admin = Some(reason); + } + return true; + } + self.request(reason) + } /// An untrapped non-`Normal` exit signal or link notice: begins exit /// without `stopped()`, or replaces the reason of an actor already /// exiting (decision 8), unless a kill was received. pub(crate) fn untrapped(&mut self, reason: ExitReason) -> bool { - if self.dead || self.killed { + if self.dead || self.killed || self.frozen { return false; } if self.reason.is_none() && !self.hook_begun { self.skip_hook = true; } + if self.reason.is_none() { + self.note_exit(&reason); + } self.reason = Some(reason); true } /// An untrappable kill: `Killed` supersedes any reason until D and /// skips a `stopped()` that has not begun. pub(crate) fn kill(&mut self) -> bool { - if self.dead { + if self.dead || self.frozen { return false; } + #[cfg(feature = "lifecycle-trace")] + crate::lifecycle_trace::start_kill(self.trace, self.starting); + if self.reason.is_none() { + self.note_exit(&ExitReason::Killed); + } self.reason = Some(ExitReason::Killed); self.killed = true; if !self.hook_begun { @@ -145,10 +219,21 @@ impl ExitRequest { /// A panic in `started()`, a handler or `stopped()`: begins exit, or /// replaces the requested reason, unless a kill was received. pub(crate) fn panic(&mut self, reason: ExitReason) { - if !self.dead && !self.killed { + if !self.dead && !self.killed && !self.frozen { + if self.reason.is_none() { + self.note_exit(&reason); + } self.reason = Some(reason); } } + /// A ready instance's exit begins (or a start-trace parent's). + #[inline(always)] + fn note_exit(&self, _reason: &ExitReason) { + #[cfg(feature = "lifecycle-trace")] + if !self.starting { + crate::lifecycle_trace::start_exit(self.trace, _reason); + } + } /// `stopped()` is about to start: returns whether it runs. pub(crate) fn begin_hook(&mut self) -> bool { self.hook_begun = true; @@ -156,13 +241,21 @@ impl ExitRequest { } /// The death point D: the requested reason becomes final. pub(crate) fn die(&mut self) -> ExitReason { + let first = !self.dead; self.dead = true; - self.reason.get_or_insert(ExitReason::Normal).clone() + let reason = self.reason.get_or_insert(ExitReason::Normal).clone(); + #[cfg(feature = "lifecycle-trace")] + if first { + crate::lifecycle_trace::start_die(self.trace, &reason, self.frozen); + } + let _ = first; + reason } /// The runtime dropped the actor before D (G1): `Killed` unless a /// non-`Normal` reason was already requested; then D. pub(crate) fn runtime_drop(&mut self) -> ExitReason { - if !self.dead { + // A non-ready startup decision fixed the reason for good (L9). + if !self.dead && !self.frozen { self.reason = match self.reason.take() { None | Some(ExitReason::Normal) => Some(ExitReason::Killed), Some(reason) => Some(reason), @@ -177,6 +270,7 @@ impl ExitRequest { /// now. A non-trapping actor begins exit with the first non-`Normal` /// one; a trapping actor dispatches each held envelope as an `Exit` in /// its own place in the queue. Returns whether the run loop must stop. + #[cfg(test)] pub(crate) fn end_starting(&mut self, trapping: bool) -> bool { self.starting = false; let held = std::mem::take(&mut self.held); @@ -195,6 +289,88 @@ impl ExitRequest { pub(crate) fn held_trapped(&self) -> bool { self.held_trapped } + /// `started()` has not run yet and a kill came first (`KilledEarly`). + pub(crate) fn killed_early(&self) -> bool { + self.killed + } + /// The startup decision (unified start, design 2.3), when `started()` + /// returns (or instead of it). `timed_out`: the init timer committed; + /// `open`: the child's admission is open (read under its gate). A + /// non-ready row fixes the reason for good (the resolution and D are one + /// step); a ready one resolves the held signals as phase 0 always did + /// (E2, E3: a trapping child queues them, `Normal` ones are dropped). + pub(crate) fn resolve_start( + &mut self, + trapping: bool, + timed_out: bool, + returned: &crate::start::Returned, + open: bool, + ) -> crate::start::Row { + use crate::start::{Returned, Row}; + self.starting = false; + let held = std::mem::take(&mut self.held); + // Row 3: the first held non-Normal signal to a non-trapping child. + // (An administrative stop is not a row: it is applied after a ready + // commitment, below.) + let row3 = if trapping { + None + } else { + held.iter().find(|r| **r != ExitReason::Normal).cloned() + }; + let admin = self.admin.take(); + let row = if timed_out { + self.reason = Some(ExitReason::Killed); + Row::Timer + } else if self.killed { + self.reason = Some(ExitReason::Killed); + Row::Kill + } else if let Some(reason) = row3 { + self.reason = Some(reason); + Row::Signal + } else { + match returned { + Returned::Panic(message) => { + self.reason = Some(ExitReason::Panic(format!("panic in started(): {message}"))); + Row::Panic + } + _ if self.reason.is_some() => Row::Stop, + Returned::Stop(reason) => { + self.reason = Some(reason.clone()); + Row::Stop + } + Returned::Error(_) => { + self.reason = Some(ExitReason::Normal); + Row::Error + } + Returned::Ignore => { + self.reason = Some(ExitReason::Normal); + Row::Ignore + } + Returned::Ready | Returned::NotBegun if !open => { + self.reason = Some(ExitReason::Shutdown(None)); + Row::Closed + } + Returned::Ready | Returned::NotBegun => Row::Ready, + } + }; + if row == Row::Ready { + self.held_trapped = trapping; + if trapping && std::mem::take(&mut self.held_parent) { + self.parent_exit_queued = true; + } + // A held administrative stop now begins an ordinary exit: the + // run loop (already woken) dispatches nothing and runs stopped(). + // (Its `SExit` is recorded right after the ready `SResolve`.) + if let Some(reason) = admin { + self.reason = Some(reason); + } + } else { + self.frozen = true; + self.skip_hook = true; + self.hook_begun = true; + } + row + } } /// Per-actor exit request, shared by every handle of the instance (and by its @@ -228,6 +404,8 @@ pub(crate) struct SignalTarget { /// Queues a signal held during `started()` in its place in the mailbox /// (the flag marks a parent signal). pub(crate) send_held: SendHeldFn, + /// The instance's startup result (unified start, design 5.1). + pub(crate) startup: Arc, } /// Where an exit signal comes from. @@ -269,8 +447,12 @@ fn apply_signal( from_parent: bool, ) { if origin == Origin::Direct && reason == ExitReason::Kill { - if lock_request(&target.request).kill() { + let mut request = lock_request(&target.request); + if request.kill() { + let waits = request.take_waiting_starts(); + drop(request); (target.wake)(); + abandon_waits(waits); } return; } @@ -311,8 +493,20 @@ fn apply_signal( return; } if request.untrapped(exit.reason) { + let waits = request.take_waiting_starts(); drop(request); (target.wake)(); + abandon_waits(waits); + } +} + +/// The actor's exit by a signal began: its pending start waits are +/// abandoned (outside its exit-request lock). +fn abandon_waits(waits: Vec>) { + for cell in waits { + if let Some(cell) = cell.upgrade() { + cell.parent_exit(); + } } } @@ -331,6 +525,19 @@ pub(crate) fn new_trap_exit_flag() -> TrapExitFlag { static NEXT_EDGE: AtomicU64 = AtomicU64::new(1); +/// The edge of a `start_link`, in the start's commit (see +/// [`table::install_start_edge`]). Returns whether it was installed. +pub(crate) fn install_start_edge( + parent: &Arc, + parent_links: &LinkTable, + child: &Arc, + child_links: &LinkTable, + admit: impl FnOnce() -> bool, +) -> bool { + let gen = NEXT_EDGE.fetch_add(1, Ordering::Relaxed); + table::install_start_edge(parent, parent_links, child, child_links, gen, admit) +} + /// Link `own` and `peer` (OTP `link/1`). An existing record (live or /// pending) makes it a no-op. Past the peer's D, `try_only` returns /// `NoProc` without a record; otherwise a pending late record is created and @@ -409,6 +616,8 @@ pub(crate) fn unregister_link( trace::life_emit(own_id.raw(), || { trace::LifeEvent::new("Unlink", own_id.raw()).s(peer_id.raw()) }); + #[cfg(feature = "lifecycle-trace")] + trace::start_unlink(own_id.raw(), peer_id.raw()); } removed }); @@ -470,7 +679,9 @@ fn deliver_notice_in_section( // One failing internal callback must not prevent other peers from // being notified, or panic again during actor-future destruction. panic = catch_unwind(AssertUnwindSafe(|| { - deliver_signal(receiver, from, reason, Origin::Link, false) + trace::start_notice(receiver.id.raw(), from.raw(), "link", &reason, || { + deliver_signal(receiver, from, reason.clone(), Origin::Link, false) + }) })) .err(); }); @@ -483,6 +694,47 @@ impl LinkNotices { pub(crate) fn deliver(self) { self.deliver_with(deliver_notice); } + /// Take the notice to `peer` (a unified start's parent edge) out of the + /// set, for its startup cell to dispose of. + pub(crate) fn take_notice(&mut self, peer: ActorId) -> Option { + self.take_peer(peer) + .map(|(from, reason, entry)| LinkNotice { + from, + reason, + entry, + }) + } +} + +/// One death notice taken at D (a unified start's parent edge), held by the +/// child's startup cell until it is disposed of (design 5.2). +pub(crate) struct LinkNotice { + from: ActorId, + reason: ExitReason, + entry: table::LinkEntry, +} + +impl LinkNotice { + /// Deliver it as an ordinary notice (fenced by the receiver's record). + pub(crate) fn deliver(self) { + if let Some(peer_links) = self.entry.peer_links.upgrade() { + deliver_notice( + &peer_links, + &self.entry.peer, + self.from, + self.entry.gen, + self.reason, + ); + } + } + /// Consumed by a failed start's answer: the receiver's record goes, so + /// no notice of this edge takes effect. + pub(crate) fn discard(self) { + if let Some(peer_links) = self.entry.peer_links.upgrade() { + let removed = table::take_record(&peer_links, self.from, self.entry.gen, |_| {}); + drop(removed); + } + } } /// Close the exit table at D: set the final reason and take one notice per @@ -515,15 +767,33 @@ pub(crate) fn propagate_exit(own_id: ActorId, own: &LinkTable, reason: &ExitReas /// first requested reason wins over later self-exits and stops; a later /// untrapped signal or kill can still replace it until D. pub(crate) fn normal_stop(id: ActorId, reason: &LinkedExitReason, cancel: impl FnOnce()) { - stop_with_reason(id, reason, ExitReason::Normal, cancel); + admin_stop_with_reason(id, reason, ExitReason::Normal, cancel); } -/// A self-exit or administrative stop of actor `id` (recorded as `Stop`). +/// A self-exit of actor `id` (recorded as `Stop`). pub(crate) fn stop_with_reason( id: ActorId, reason: &LinkedExitReason, cause: ExitReason, cancel: impl FnOnce(), +) { + trace::life_section(id.raw(), || { + trace::life_emit(id.raw(), || { + trace::LifeEvent::new("Stop", id.raw()).x(trace::life_reason(&cause)) + }); + if lock_request(reason).request(cause) { + cancel(); + } + }); +} + +/// An administrative stop of actor `id` (`ChildHandle::stop*`, recorded as +/// `Stop`): during `started()` it decides the start (unified start row 3). +pub(crate) fn admin_stop_with_reason( + id: ActorId, + reason: &LinkedExitReason, + cause: ExitReason, + cancel: impl FnOnce(), ) { trace::life_section(id.raw(), || { trace::life_emit(id.raw(), || { @@ -533,10 +803,10 @@ pub(crate) fn stop_with_reason( }); } -/// As [`stop_with_reason`], unrecorded: a parent exit being dispatched (part -/// of the model's `Dispatch`) or a managed scope's stop. +/// As [`admin_stop_with_reason`], unrecorded: a parent exit being +/// dispatched (part of the model's `Dispatch`) or a managed scope's stop. pub(crate) fn request_stop(reason: &LinkedExitReason, cause: ExitReason, cancel: impl FnOnce()) { - if lock_request(reason).request(cause) { + if lock_request(reason).admin_stop(cause) { cancel(); } } @@ -761,6 +1031,7 @@ mod tests { Ok(()) }), send_held: Arc::new(|_, _| Ok(())), + startup: crate::start::StartupCell::new(), } } fn pair( @@ -930,6 +1201,18 @@ mod tests { let mut r = ExitRequest::default(); r.request(ExitReason::Shutdown(None)); assert_eq!(r.runtime_drop(), ExitReason::Shutdown(None)); + // Review #291 item 8: a non-ready startup decision froze the + // reason (Init::error: Normal); a runtime drop before D keeps it. + let mut r = ExitRequest::default(); + r.begin_starting(); + let row = r.resolve_start( + false, + false, + &crate::start::Returned::Error("no config".into()), + true, + ); + assert_eq!(row, crate::start::Row::Error); + assert_eq!(r.runtime_drop(), ExitReason::Normal); }, ); } diff --git a/concurrency/src/link/table.rs b/concurrency/src/link/table.rs index b54cf03e4..b0da3cd47 100644 --- a/concurrency/src/link/table.rs +++ b/concurrency/src/link/table.rs @@ -145,6 +145,43 @@ pub(crate) fn install( }) } +/// The parent's half of a `start_link` edge (unified start, stage 1): +/// installed in the commit, together with the child's half, only if +/// `admit` (the parent's exit has not begun, read under its exit-request +/// lock, which ranks after the tables) holds. The child's table is still +/// private. Returns whether the edge was installed. +pub(crate) fn install_start_edge( + parent: &Arc, + parent_links: &LinkTable, + child: &Arc, + child_links: &LinkTable, + gen: u64, + admit: impl FnOnce() -> bool, +) -> bool { + with_pair(parent.id, parent_links, child.id, child_links, |p, c| { + if p.exited.is_some() || !admit() { + return false; + } + p.entries.push(LinkEntry { + peer_id: child.id, + gen, + peer: child.clone(), + peer_links: Arc::downgrade(child_links), + late: false, + _alive: Default::default(), + }); + c.entries.push(LinkEntry { + peer_id: parent.id, + gen, + peer: parent.clone(), + peer_links: Arc::downgrade(parent_links), + late: false, + _alive: Default::default(), + }); + true + }) +} + fn remove(table: &mut LinkState, id: ActorId) -> Option { table .entries @@ -198,6 +235,18 @@ pub(crate) struct LinkNotices { } impl LinkNotices { + /// Take the notice to `peer` out of the set (the unified start's edge, + /// disposed of through its startup cell). `None` if this actor holds no + /// live record of `peer`. + pub(crate) fn take_peer(&mut self, peer: ActorId) -> Option<(ActorId, ExitReason, LinkEntry)> { + let (from, reason) = (self.from?, self.reason.clone()?); + let i = self + .entries + .iter() + .position(|e| e.peer_id == peer && !e.late)?; + Some((from, reason, self.entries.swap_remove(i))) + } + /// Deliver every notice (publication) through `notice(receiver table, /// receiver, from, gen, reason)`, which applies it only if the receiver /// still holds the record (E6 fence). Late records carry no notice of diff --git a/concurrency/src/link_regression_tests.rs b/concurrency/src/link_regression_tests.rs index fa86bbbba..d454b26c1 100644 --- a/concurrency/src/link_regression_tests.rs +++ b/concurrency/src/link_regression_tests.rs @@ -35,7 +35,7 @@ fn linking_between_death_point_and_publication_delivers_noproc() { let events = Arc::new(Mutex::new(Vec::new())); let observer = { let _entered = rt.enter(); - Observer(events.clone()).start() + Observer(events.clone()).spawn() }; observer.context().trap_exit(true); let peer_id = ActorId::next(); @@ -98,11 +98,11 @@ fn never_polled_actor_propagates_runtime_shutdown_to_peer() { let survivor = runtime(); let target = { let _entered = owner.enter(); - Idler.start() + Idler.spawn() }; let peer = { let _entered = survivor.enter(); - Idler.start() + Idler.spawn() }; peer.context().link(&target.child_handle()); // The owner runtime never polls its actor. Dropping it must still close links. @@ -148,12 +148,12 @@ fn polled_actor_propagates_runtime_shutdown_to_peer() { let survivor = runtime(); let target = { let _entered = owner.enter(); - Idler.start() + Idler.spawn() }; owner.block_on(target.request(Ping)).unwrap(); let peer = { let _entered = survivor.enter(); - Idler.start() + Idler.spawn() }; peer.context().link(&target.child_handle()); drop(owner); @@ -192,12 +192,12 @@ fn never_polled_actor_notifies_monitor_on_runtime_shutdown() { let survivor = runtime(); let target = { let _entered = owner.enter(); - Idler.start() + Idler.spawn() }; let (tx, rx) = spawned_rt::tasks::oneshot::channel(); let observer = { let _entered = survivor.enter(); - MonitorObserver(Some(tx)).start() + MonitorObserver(Some(tx)).spawn() }; { let _entered = survivor.enter(); @@ -276,9 +276,9 @@ fn late_link_failure_relabels_an_already_requested_normal_stop() { entered: Some(entered_tx), release: Some(release_rx), } - .start(), - Idler.start(), - Idler.start(), + .spawn(), + Idler.spawn(), + Idler.spawn(), ) }; stopping.context().link(&failing.child_handle()); @@ -331,7 +331,7 @@ fn start_linked_registers_before_async_started() { .unwrap(); let parent = { let _entered = rt.enter(); - Idler.start() + Idler.spawn() }; for _ in 0..32 { let (tx, rx) = spawned_rt::tasks::oneshot::channel(); @@ -341,7 +341,9 @@ fn start_linked_registers_before_async_started() { parent: parent.id(), seen: Some(tx), } - .start_linked(&parent.context()) + .start_link(&parent.context()) + .actor_ref() + .unwrap() }; assert!(rt.block_on(rx).unwrap()); assert_eq!(rt.block_on(child.wait_exit()), ExitReason::Normal); @@ -365,11 +367,11 @@ fn never_polled_monitor_owner_releases_waiter_and_refuses_late_registration() { let (tx, _rx) = spawned_rt::tasks::oneshot::channel(); let owner = { let _entered = owner_rt.enter(); - MonitorObserver(Some(tx)).start() + MonitorObserver(Some(tx)).spawn() }; let target = { let _entered = survivor.enter(); - Idler.start() + Idler.spawn() }; let handle = target.child_handle(); let probe = handle.lifetime_probe(); @@ -416,11 +418,11 @@ fn linked_runtime_abort_is_trappable() { let events = Arc::new(Mutex::new(Vec::new())); let target = { let _entered = owner.enter(); - Idler.start() + Idler.spawn() }; let peer = { let _entered = survivor.enter(); - Observer(events.clone()).start() + Observer(events.clone()).spawn() }; peer.context().trap_exit(true); peer.context().link(&target.child_handle()); @@ -476,7 +478,7 @@ fn never_polled_completion_waits_for_actor_payload_destruction() { entered: entered_tx, release: release_rx, } - .start() + .spawn() }; let shutdown = std::thread::spawn(move || drop(owner)); let entered = entered_rx.recv_timeout(Duration::from_secs(2)); @@ -518,9 +520,9 @@ fn linked_runtime_abort_cancels_an_idle_threads_actor() { let waiter = runtime(); let target = { let _entered = owner.enter(); - Idler.start() + Idler.spawn() }; - let peer = ThreadPeer.start(); + let peer = ThreadPeer.spawn(); peer.context().link(&target.child_handle()); drop(owner); let result = waiter.block_on(async { @@ -570,7 +572,7 @@ fn dropping_runtime_with_large_polled_link_cycle_reclaims_every_actor() { let dropped = Arc::new(AtomicUsize::new(0)); let actors: Vec<_> = { let _entered = owner.enter(); - (0..4096).map(|_| Node(dropped.clone()).start()).collect() + (0..4096).map(|_| Node(dropped.clone()).spawn()).collect() }; for i in 0..actors.len() { actors[i] diff --git a/concurrency/src/monitor/fence.rs b/concurrency/src/monitor/fence.rs index 9b1992cfc..1b7006f05 100644 --- a/concurrency/src/monitor/fence.rs +++ b/concurrency/src/monitor/fence.rs @@ -63,12 +63,14 @@ pub(crate) fn register( active: impl FnOnce(), ) -> Registration { let mut guard = lock(table); - if observer_exiting() { - return Registration::Inert; - } + // The table's closure first: `observer_exiting` may take effects (a + // start registers its name there) that a closed table must not see. let Some(entries) = guard.as_mut() else { return Registration::Inert; }; + if observer_exiting() { + return Registration::Inert; + } if target_dead() { late(); entries.insert(key, Entry::Queued); diff --git a/concurrency/src/native_link_regression_tests.rs b/concurrency/src/native_link_regression_tests.rs index 6fbc625de..76f86fc08 100644 --- a/concurrency/src/native_link_regression_tests.rs +++ b/concurrency/src/native_link_regression_tests.rs @@ -26,19 +26,24 @@ fn start_linked_registers_before_threads_started() { ctx.stop(); } } - let parent = Parent.start(); + let parent = Parent.spawn(); for _ in 0..32 { let (tx, rx) = std::sync::mpsc::channel(); - let child = Child { + // The child stops itself in started(): a failed start, answered + // after its publication with its final reason. + let result = Child { parent: parent.id(), seen: tx, } - .start_linked(&parent.context()); + .start_link(&parent.context()); assert!(rx.recv_timeout(Duration::from_secs(1)).unwrap()); - assert_eq!( - child.child_handle().wait_exit_blocking(), - ExitReason::Normal - ); + assert!(matches!( + result, + Err(crate::StartError::InitExited { + reason: ExitReason::Normal, + pending: None + }) + )); } parent.child_handle().stop(); parent.child_handle().wait_exit_blocking(); diff --git a/concurrency/src/registry.rs b/concurrency/src/registry.rs index 69510735c..df7b3f695 100644 --- a/concurrency/src/registry.rs +++ b/concurrency/src/registry.rs @@ -6,6 +6,9 @@ use std::sync::{OnceLock, RwLock}; struct Entry { value: Box, owner: Option<(crate::ActorId, u64)>, + /// The holder's handle, for a name registered by a start + /// (`StartError::AlreadyStarted::existing`). + handle: Option, /// `spawned_registered_names`. _alive: crate::instrumentation::Live, } @@ -56,6 +59,7 @@ pub fn register( e.insert(Entry { value: Box::new(value), owner: None, + handle: None, _alive: Default::default(), }); Ok(()) @@ -84,12 +88,18 @@ pub fn unregister(name: &str) { pub struct RegistrationGuard { name: String, owner: (crate::ActorId, u64), + /// Start-trace child id (0: not recorded). + trace: u64, } impl Drop for RegistrationGuard { fn drop(&mut self) { fail_point!("registry.before_unregister"); let removed = { let mut store = global_store().write().unwrap_or_else(|p| p.into_inner()); + let trace = self.trace; + crate::lifecycle_trace::start_emit(trace, || { + crate::lifecycle_trace::StartEvent::new("SReleaseName", trace) + }); if store .get(&self.name) .is_some_and(|entry| entry.owner == Some(self.owner)) @@ -107,6 +117,7 @@ pub fn register_owned( name: &str, owner: crate::ActorId, value: T, + handle: Option, ) -> Result { use std::collections::hash_map::Entry as MapEntry; let mut store = global_store().write().unwrap_or_else(|p| p.into_inner()); @@ -117,16 +128,80 @@ pub fn register_owned( slot.insert(Entry { value: Box::new(value), owner: Some(owner), + // The holder, for a start's `AlreadyStarted { existing }`. + handle, _alive: Default::default(), }); Ok(RegistrationGuard { name: name.into(), owner, + trace: 0, }) } } } +/// A start's name (unified start, stage 1): registered in the commit, +/// with the holder's handle, under the registry's write lock; `admit` runs +/// under it too (the commit's other checks are already held). A taken name +/// reports the holder's handle (`None` for a value that is not an actor). +/// The guard releases it at publication. +pub(crate) fn register_start( + name: &str, + owner: crate::ActorId, + value: T, + handle: crate::ChildHandle, + trace: u64, + admit: impl FnOnce() -> Result<(), E>, +) -> Result>, E> { + use std::collections::hash_map::Entry as MapEntry; + let mut store = global_store().write().unwrap_or_else(|p| p.into_inner()); + // The registry is a start's innermost lock: its commit is decided (and + // recorded) here, ordered with `SReleaseName` and `SLookup`. + match store.entry(name.to_string()) { + MapEntry::Occupied(e) => { + crate::lifecycle_trace::commit_point(Some("already")); + Ok(Err(e.get().handle.clone())) + } + MapEntry::Vacant(slot) => { + admit()?; + crate::lifecycle_trace::commit_point(None); + let owner = (owner, NEXT_REGISTRATION.fetch_add(1, Ordering::Relaxed)); + slot.insert(Entry { + value: Box::new(value), + owner: Some(owner), + handle: Some(handle), + _alive: Default::default(), + }); + Ok(Ok(RegistrationGuard { + name: name.into(), + owner, + trace, + })) + } + } +} + +/// `whereis` for a start's typed name, recorded in the start trace. +pub(crate) fn whereis_start(name: &str) -> Option { + let store = global_store().read().unwrap_or_else(|p| p.into_inner()); + let entry = store.get(name); + #[cfg(feature = "lifecycle-trace")] + { + let holder = entry.and_then(|e| e.owner).map_or(0, |(id, _)| { + crate::lifecycle_trace::start_child_of(id.raw()) + }); + crate::lifecycle_trace::start_emit_env(|| { + crate::lifecycle_trace::StartEvent::new("SLookup", holder).x(if holder != 0 { + "hit" + } else { + "miss" + }) + }); + } + entry?.value.downcast_ref::().cloned() +} + /// List all registered names, sorted. (The store is a `HashMap`, whose /// iteration order depends on per-process random state; sorting keeps /// callers deterministic, which the simulation backend relies on.) @@ -213,9 +288,9 @@ mod tests { #[test] fn old_guard_preserves_new_registration_even_with_same_actor_identity() { let actor = crate::ActorId::next(); - let old = register_owned("owned_generation_guard_test", actor, 1u64).unwrap(); + let old = register_owned("owned_generation_guard_test", actor, 1u64, None).unwrap(); unregister("owned_generation_guard_test"); - let new = register_owned("owned_generation_guard_test", actor, 2u64).unwrap(); + let new = register_owned("owned_generation_guard_test", actor, 2u64, None).unwrap(); drop(old); assert_eq!(whereis::("owned_generation_guard_test"), Some(2)); drop(new); diff --git a/concurrency/src/reply/jobs.rs b/concurrency/src/reply/jobs.rs index cf9f0836b..d0552326e 100644 --- a/concurrency/src/reply/jobs.rs +++ b/concurrency/src/reply/jobs.rs @@ -27,11 +27,16 @@ use std::sync::{Arc, Condvar, Mutex, MutexGuard}; pub(crate) const DEFAULT_EMIT_CREDITS: u16 = 16; /// Default `Actor::MAX_NATIVE_JOBS`: `min(4, available_parallelism)`. +/// Read once per process: `available_parallelism` parses cgroup files on +/// Linux (about a third of a spawn's instructions when called per spawn). pub(crate) fn default_max_native_jobs() -> usize { - std::thread::available_parallelism() - .map(|n| n.get()) - .unwrap_or(1) - .min(4) + static DEFAULT: std::sync::OnceLock = std::sync::OnceLock::new(); + *DEFAULT.get_or_init(|| { + std::thread::available_parallelism() + .map(|n| n.get()) + .unwrap_or(1) + .min(4) + }) } /// Per-instance overrides of `Actor::MAX_OUTSTANDING` and @@ -46,6 +51,13 @@ pub(crate) struct Bounds { #[cfg(feature = "async")] impl Bounds { + /// These bounds, each falling back to the given one when unset. + pub(crate) fn or(self, max_outstanding: Option, max_native_jobs: Option) -> Self { + Self { + max_outstanding: self.max_outstanding.or(max_outstanding), + max_native_jobs: self.max_native_jobs.or(max_native_jobs), + } + } /// The effective `(max_outstanding, max_native_jobs)` for an actor whose /// constants are `outstanding` and `native`. pub(crate) fn resolve(self, outstanding: usize, native: Option) -> (usize, usize) { diff --git a/concurrency/src/reply/registry.rs b/concurrency/src/reply/registry.rs index 4bdad4b36..153c44673 100644 --- a/concurrency/src/reply/registry.rs +++ b/concurrency/src/reply/registry.rs @@ -852,7 +852,7 @@ mod tests { reply.reply(()); } } - let actor = std::thread::spawn(|| Native.start()).join().unwrap(); + let actor = std::thread::spawn(|| Native.spawn()).join().unwrap(); let runtime = spawned_rt::tasks::sim::Runtime::with_seed(29); let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { runtime @@ -870,7 +870,7 @@ mod tests { let runtime = spawned_rt::tasks::sim::Runtime::with_seed(31); let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { - runtime.block_on(async { Native.start() }) + runtime.block_on(async { Native.spawn() }) })); drop(runtime); let panic = match result { diff --git a/concurrency/src/start.rs b/concurrency/src/start.rs new file mode 100644 index 000000000..5554f45de --- /dev/null +++ b/concurrency/src/start.rs @@ -0,0 +1,1229 @@ +//! Unified start (0.9, design `docs/design/unified-start.md`): the start +//! errors and outcomes and the per-instance startup cell, shared by both +//! execution modes. +//! +//! Every instance has one [`StartupCell`] (in its `SignalTarget`): an +//! immutable commitment (ready, failed, ignored or timed out) written once, +//! by the child at the end of `started()` or by the init timer. A waiting +//! start (`start`, `start_link`, `start_monitor`) adds a waiter to it. The +//! child's start edge (the `start_link`/`start_monitor` parent) carries its +//! notice through the cell: at publication the cell disposes of it by the +//! commitment and the waiter state (design 5.2): delivered, diverted to the +//! waiter (consumed by its error, handed on if the wait is abandoned), +//! applied, or dropped. +//! +//! Lock order (design section 3; this code's ranks): an admission gate and +//! group lock, then link tables, then an actor's exit request, then the +//! registry; the cell lock is always last. Nothing under the cell lock takes +//! another framework lock; deliveries decided under it run after it is +//! released. +use crate::child_handle::ChildHandle; +use crate::error::ExitReason; +use crate::lifecycle_trace as trace; +use crate::link::LinkNotice; +use spawned_rt::time::Instant; +use std::sync::atomic::{AtomicU8, Ordering}; +use std::sync::{Arc, Condvar, Mutex, MutexGuard}; +use std::time::Duration; + +// --------------------------------------------------------------------------- +// Public types +// --------------------------------------------------------------------------- + +/// The result of a start that did not fail: OTP's `{ok, Pid}` or `ignore`. +/// +/// `pending` is `Some` when the ignoring instance was not yet published at +/// its cleanup deadline (a hung destructor): it still holds its name and +/// slot until it is (`wait_exit` on the handle reaches its real end). +#[must_use] +#[derive(Debug)] +pub enum StartOutcome { + /// `started()` returned ready (or a continuation). + Started(T), + /// `started()` returned `Init::ignore()`: the + /// actor ended `Normal` without becoming ready. + Ignored { pending: Option }, +} + +impl StartOutcome { + /// The started value, or `None` for an ignored start. + /// + /// `x.start().await?.started().unwrap()` panics if the actor's + /// `started()` can return `Init::ignore()`; for such actors `match` the + /// outcome (or use [`is_ignored`](Self::is_ignored)) instead. + pub fn started(self) -> Option { + match self { + Self::Started(value) => Some(value), + Self::Ignored { .. } => None, + } + } + /// Whether `started()` returned `Init::ignore()`. + pub fn is_ignored(&self) -> bool { + matches!(self, Self::Ignored { .. }) + } + /// Map the started value. + pub fn map(self, f: impl FnOnce(T) -> U) -> StartOutcome { + match self { + Self::Started(value) => StartOutcome::Started(f(value)), + Self::Ignored { pending } => StartOutcome::Ignored { pending }, + } + } +} + +/// Why a start failed. +/// +/// Stage 1 (nothing was created): `Closed`, `Capacity`, `AlreadyStarted`, +/// `ParentExited`, `InvalidOptions`, `System`. Stage 2 (the child existed +/// and is gone, or still cleaning up when `pending` is `Some`): `InitFailed`, +/// `InitExited`, `InitTimeout`. A stage-2 error with `pending: None` is +/// returned only after the child's publication: its state is dropped, its +/// notices are sent and its name is free. +#[derive(Debug, Clone, thiserror::Error)] +#[non_exhaustive] +pub enum StartError { + /// The group, the system or the parent's scope no longer admits actors. + #[error("the group, system or parent scope no longer admits actors")] + Closed, + /// A thread group has no free slot. + #[error("the execution group has no free actor slot")] + Capacity, + /// The start's name is registered. `existing` is the holder's handle, or + /// `None` for a value that is not an actor (OTP `{already_started, Pid}`). + #[error("name '{name}' is already registered")] + AlreadyStarted { + name: String, + existing: Option, + }, + /// `start_link`/`start_monitor` through a `Context` whose actor's exit + /// has begun (stage 1: nothing created). Also the answer of a waiting + /// `start_link`/`start_monitor` whose caller's exit was begun by a kill + /// or an untrapped exit signal while it waited: the wait is abandoned + /// (OTP: the caller dies at once, without waiting for the child's + /// `init/1`); the child, if created, gets the caller's exit through its + /// link or monitor. + #[error("the parent's exit has begun")] + ParentExited, + /// The options are invalid (validated before anything is created). + #[error("invalid start options: {0}")] + InvalidOptions(&'static str), + /// Executor or runtime construction failed. + #[cfg(feature = "async")] + #[error(transparent)] + System(crate::system::SystemError), + /// `started()` returned an error (`Err(e)` or `Init::error(e)`); the + /// actor ended `Normal` (OTP 26 `init_fail/3`). + #[error("started() failed: {cause}")] + InitFailed { + cause: String, + pending: Option, + }, + /// The actor ended during `started()`: a kill, an administrative stop, an + /// untrapped exit signal, a panic, `ctx.stop*` or `Init::stop`. + /// `reason` is the final exit reason (the one `wait_exit` reports). + #[error("the actor exited during started(): {reason:?}")] + InitExited { + reason: ExitReason, + pending: Option, + }, + /// `started()` did not return within the start's `init_timeout`. + #[error("started() did not return within the init timeout")] + InitTimeout { pending: Option }, +} + +impl StartError { + /// A stage-1 refusal's name in the start trace (`SRefuse`). + #[allow(dead_code)] // trace only + pub(crate) fn trace_refusal(&self) -> &'static str { + match self { + Self::Closed => "closed", + Self::Capacity => "capacity", + Self::AlreadyStarted { .. } => "already", + Self::ParentExited => "parentexited", + _ => "system", + } + } +} + +#[cfg(feature = "async")] +impl From for StartError { + fn from(error: crate::system::SystemError) -> Self { + use crate::system::SystemError; + match error { + SystemError::Closed => Self::Closed, + SystemError::Capacity => Self::Capacity, + other => Self::System(other), + } + } +} + +/// A bound that may be infinite (`init_timeout`, `startup_timeout`). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum Timeout { + After(Duration), + #[default] + Infinity, +} + +impl From for Timeout { + fn from(duration: Duration) -> Self { + Self::After(duration) + } +} + +// --------------------------------------------------------------------------- +// The startup decision (design 2.3) +// --------------------------------------------------------------------------- + +/// What `started()` returned, mode-independent (the continuation lives in +/// the mode's `Init`). +#[derive(Debug)] +pub(crate) enum Returned { + /// `started()` was not run: a kill or the init timer came first + /// (`KilledEarly`), or a managed child's admission closed first. + NotBegun, + Panic(String), + Stop(ExitReason), + Error(String), + Ignore, + Ready, +} + +impl Returned { + #[allow(dead_code)] // trace only + pub(crate) fn trace_name(&self, continued: bool) -> &'static str { + match self { + Self::NotBegun => "none", + Self::Panic(_) => "panic", + Self::Stop(ExitReason::Normal) => "stopN", + Self::Stop(_) => "stopS", + Self::Error(_) => "error", + Self::Ignore => "ignore", + Self::Ready if continued => "continue", + Self::Ready => "ready", + } + } +} + +/// The row of the design's 2.3 table that decided a start. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) enum Row { + /// 1: the init timer committed `TimedOut`. + Timer, + /// 2: a kill arrived. + Kill, + /// 3: an administrative stop or a held untrapped non-`Normal` signal. + Signal, + /// 4: `started()` panicked. + Panic, + /// 5, 6: `ctx.stop*` in `started()`, or `Init::stop`. + Stop, + /// 7: `Init::error`. + Error, + /// 8: `Init::ignore`. + Ignore, + /// 9: ready, but the child's admission closed. + Closed, + /// 10: ready. + Ready, +} + +impl Row { + pub(crate) fn commit(&self) -> Commit { + match self { + Self::Timer => Commit::TimedOut, + Self::Ignore => Commit::Ignored, + Self::Ready => Commit::Ready, + _ => Commit::Failed, + } + } + #[allow(dead_code)] // trace only + pub(crate) fn trace_name(&self) -> &'static str { + match self { + Self::Timer => "timer", + Self::Kill => "kill", + Self::Signal => "signal", + Self::Panic => "panic", + Self::Stop => "stop", + Self::Error => "error", + Self::Ignore => "ignore", + Self::Closed => "closed", + Self::Ready => "ready", + } + } +} + +// --------------------------------------------------------------------------- +// The startup cell (design 5.1) +// --------------------------------------------------------------------------- + +/// The immutable startup commitment. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[repr(u8)] +pub(crate) enum Commit { + Pending = 0, + Ready = 1, + Failed = 2, + Ignored = 3, + TimedOut = 4, +} + +impl Commit { + fn from_u8(value: u8) -> Self { + match value { + 1 => Self::Ready, + 2 => Self::Failed, + 3 => Self::Ignored, + 4 => Self::TimedOut, + _ => Self::Pending, + } + } + #[allow(dead_code)] // trace only + pub(crate) fn trace_name(self) -> &'static str { + match self { + Self::Pending => "none", + Self::Ready => "ready", + Self::Failed => "failed", + Self::Ignored => "ignored", + Self::TimedOut => "timedout", + } + } +} + +/// The waiter of a waiting start. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Waiter { + /// `spawn` (or a refused start): nobody waits. + None, + Waiting, + /// The waiter took its answer. + Consumed, + /// The `Starting` was dropped unanswered. + #[cfg_attr(not(feature = "async"), allow(dead_code))] // tasks mode only + Abandoned, +} + +/// The kind of a start edge. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum EdgeKind { + Link, + Monitor, +} + +/// What publication decided for the start edge's notice (design 5.2). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Disposition { + /// Ordinary: delivered (applied to a non-trapping parent by the usual + /// signal rules). + Deliver, + /// Held for the waiter: its error consumes it, abandonment hands it on. + Divert, + /// Suppressed by a failure the waiter already consumed. + Drop, +} + +impl Disposition { + #[allow(dead_code)] // trace only + fn trace_name(self) -> &'static str { + match self { + Self::Deliver => "deliver", + Self::Divert => "divert", + Self::Drop => "drop", + } + } +} + +/// A diverted `Down`: the parent's fenced delivery (`true`) or the removal +/// of its monitor entry (`false`). Built by the parent's monitor waiter. +pub(crate) type DownNotice = Box; + +/// The start edge (design 5.2), set at the commit for `start_link` and +/// `start_monitor`. +pub(crate) struct StartEdge { + pub(crate) kind: EdgeKind, + /// The parent, for its trap flag (read at publication). + pub(crate) parent: Arc, +} + +struct CellInner { + waiter: Waiter, + edge: Option, + /// `InitFailed`'s diagnostic. + cause: Option, + /// The decided (final) reason of a non-ready commitment. + requested: Option, + /// When the commitment was made: anchors the cleanup deadline. + committed_at: Option, + /// The cleanup interval of the instance's placement. + cleanup: Duration, + /// The final reason, once published. + published: Option, + /// Publication has begun (its dispositions are decided): observers wait + /// for it rather than answer `pending`. + publishing: bool, + /// The disposition of the edge's notice at publication. + disposition: Option, + /// Diverted notices, waiting for the waiter's answer or abandonment. + diverted_link: Option, + diverted_down: Option, + /// The monitor waiter has not yet run (its `Down` is not yet built). + down_pending: bool, + /// The waiting parent's exit began by a signal: the wait is abandoned. + parent_exited: bool, + /// The init timer is armed and has not fired. + timer: Option>, + /// Trace id of the instance (0 when not recorded). + trace: u64, + /// Bumped at every change (blocking observers wait on it). + version: u64, + /// The cleanup deadline was observed expired (recorded once). + deadline_seen: bool, +} + +type OnCancel = Box; + +/// Cancels a pending init timer at the commitment. +pub(crate) struct TimerCancel { + /// `None` once cancelled; else what cancelling does (if armed yet). + state: Mutex>>, +} + +impl Default for TimerCancel { + fn default() -> Self { + Self { + state: Mutex::new(Some(None)), + } + } +} + +impl TimerCancel { + fn cancel(&self) { + let action = self + .state + .lock() + .unwrap_or_else(|p| p.into_inner()) + .take() + .flatten(); + if let Some(action) = action { + action(); + } + } + /// What cancelling does; run at once if already cancelled. + fn on_cancel(&self, action: Box) { + let mut state = self.state.lock().unwrap_or_else(|p| p.into_inner()); + match state.as_mut() { + Some(slot) => *slot = Some(action), + None => { + drop(state); + action(); + } + } + } +} + +/// One instance's startup result and waiter (design 5.1). +pub(crate) struct StartupCell { + state: AtomicU8, + inner: Mutex, + /// Blocking observers. + changed: Condvar, + /// Async observers: bumped at every change. + /// Created by the first async observer: a start nobody awaits (`spawn`) + /// allocates no channel. + version: std::sync::OnceLock>, +} + +impl std::fmt::Debug for StartupCell { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("StartupCell") + .field("commit", &self.commit()) + .finish_non_exhaustive() + } +} + +/// An observation of the cell by `ready()` or the waiter. +pub(crate) enum Observed { + /// Nothing to answer yet; wait for a change or the deadline (if any). + Wait(Option), + Answer(Result, StartError>), +} + +impl StartupCell { + pub(crate) fn new() -> Arc { + Arc::new(Self { + state: AtomicU8::new(Commit::Pending as u8), + inner: Mutex::new(CellInner { + waiter: Waiter::None, + edge: None, + cause: None, + requested: None, + committed_at: None, + cleanup: Duration::from_secs(5), + published: None, + publishing: false, + disposition: None, + diverted_link: None, + diverted_down: None, + down_pending: false, + parent_exited: false, + timer: None, + trace: 0, + version: 0, + deadline_seen: false, + }), + changed: Condvar::new(), + version: std::sync::OnceLock::new(), + }) + } + + fn lock(&self) -> MutexGuard<'_, CellInner> { + self.inner.lock().unwrap_or_else(|p| p.into_inner()) + } + + fn notify(&self) { + { + let mut inner = self.lock(); + inner.version = inner.version.wrapping_add(1); + } + self.changed.notify_all(); + // An observer subscribes before it observes: one that subscribes + // after this sees the change when it observes. + if let Some(version) = self.version.get() { + let next = version.borrow().wrapping_add(1); + version.send_replace(next); + } + } + + pub(crate) fn commit(&self) -> Commit { + Commit::from_u8(self.state.load(Ordering::Acquire)) + } + + /// Prepare (stage 1, private): the waiter, the edge, the cleanup + /// interval and the trace id. + pub(crate) fn prepare( + &self, + waiting: bool, + edge: Option, + cleanup: Duration, + trace: u64, + ) { + let mut inner = self.lock(); + inner.waiter = if waiting { + Waiter::Waiting + } else { + Waiter::None + }; + inner.down_pending = edge.as_ref().is_some_and(|e| e.kind == EdgeKind::Monitor); + inner.edge = edge; + inner.cleanup = cleanup; + inner.trace = trace; + } + + /// The cleanup interval may be set after the commit (a managed scope's). + #[cfg_attr(not(feature = "async"), allow(dead_code))] + pub(crate) fn set_cleanup(&self, cleanup: Duration) { + self.lock().cleanup = cleanup; + } + + #[allow(dead_code)] // trace only + pub(crate) fn trace_id(&self) -> u64 { + self.lock().trace + } + + /// Commit the startup decision (the child, under its exit-request lock + /// and, when managed, its admission gate; design 5.1). `decide` runs + /// under the cell lock with whether the init timer already committed + /// `TimedOut` (row 1), and returns the row, the requested reason and an + /// `InitFailed` cause. Exactly one commitment wins `Pending`. + pub(crate) fn decide( + &self, + returned: &Returned, + continued: bool, + trapping: bool, + decide: impl FnOnce(bool) -> (Row, Option, Option), + ) -> Row { + let mut inner = self.lock(); + let current = self.commit(); + let (row, reason, cause) = decide(current == Commit::TimedOut); + if current == Commit::Pending { + self.state.store(row.commit() as u8, Ordering::Release); + inner.committed_at = Some(Instant::now()); + inner.requested = reason.clone(); + inner.cause = cause; + if let Some(timer) = inner.timer.take() { + timer.cancel(); + } + } + let id = inner.trace; + trace::start_emit(id, || { + trace::StartEvent::new("SResolve", id) + .x(returned.trace_name(continued)) + .y(row.trace_name()) + .z(reason.as_ref().map_or("", trace::life_reason)) + .f(trapping) + }); + // A held administrative stop begins the ready child's exit + // (`AdminStopApply`). + if row == Row::Ready { + if let Some(reason) = &reason { + trace::start_emit(id, || { + trace::StartEvent::new("SExit", id).x(trace::life_reason(reason)) + }); + } + } + drop(inner); + self.notify(); + row + } + + /// The death point of an instance whose start was never decided: its + /// runtime dropped it before (or while) `started()` ran, or its lifecycle + /// unwound. The start fails with the final reason, answered at + /// publication like any failed start. Not a step of `Start.tla` (runtime + /// drops are `Lifecycle.tla`'s). + pub(crate) fn fail_at_death(&self, reason: &ExitReason) { + let mut inner = self.lock(); + if self.commit() != Commit::Pending { + return; + } + self.state.store(Commit::Failed as u8, Ordering::Release); + inner.committed_at = Some(Instant::now()); + inner.requested = Some(reason.clone()); + if let Some(timer) = inner.timer.take() { + timer.cancel(); + } + drop(inner); + self.notify(); + } + + /// Arm the init timer (stage 1, after the commit). + pub(crate) fn arm_timer(&self) -> Option> { + let mut inner = self.lock(); + if self.commit() != Commit::Pending { + return None; + } + let timer = Arc::new(TimerCancel::default()); + inner.timer = Some(timer.clone()); + Some(timer) + } + + /// The init timer fired: commit `TimedOut` if nothing is committed + /// (design 6). Returns whether it won. + pub(crate) fn time_out(&self) -> bool { + let mut inner = self.lock(); + inner.timer = None; + let won = self.commit() == Commit::Pending; + if won { + self.state.store(Commit::TimedOut as u8, Ordering::Release); + inner.committed_at = Some(Instant::now()); + inner.requested = Some(ExitReason::Killed); + } + trace::start_emit(inner.trace, || { + trace::StartEvent::new("STimer", inner.trace).x(if won { "won" } else { "lost" }) + }); + drop(inner); + if won { + self.notify(); + } + won + } + + /// Publication (design 5.2 step 5), from the child's cleanup: dispose of + /// the start edge's link notice (taken at D) and decide the `Down`'s + /// disposition. Returns the link notice to deliver now, if any; the + /// caller delivers it and the other notices after the lock is released. + pub(crate) fn publish( + &self, + reason: &ExitReason, + link: Option, + ) -> Option { + let mut inner = self.lock(); + inner.publishing = true; + let commit = self.commit(); + let mut deliver = link; + let mut discard = None; + let mut names = ("none", ""); + if let Some((kind, trapping)) = inner + .edge + .as_ref() + .map(|edge| (edge.kind, edge.parent.trap.load(Ordering::Acquire))) + { + let disposition = dispose(commit, inner.waiter, kind, trapping, reason); + inner.disposition = Some(disposition); + if kind == EdgeKind::Link { + match disposition { + Disposition::Divert => inner.diverted_link = deliver.take(), + Disposition::Drop => discard = deliver.take(), + Disposition::Deliver => {} + } + } + names = ( + disposition.trace_name(), + if kind == EdgeKind::Link { + "link" + } else { + "monitor" + }, + ); + } + let id = inner.trace; + trace::start_emit(id, || { + trace::StartEvent::new("SPublish", id) + .x(names.0) + .y(names.1) + .z(trace::life_reason(reason)) + }); + drop(inner); + if let Some(notice) = discard { + notice.discard(); + } + deliver + } + + /// `wait_exit` is published: final answers become available. + pub(crate) fn published(&self, reason: &ExitReason) { + self.lock().published = Some(reason.clone()); + self.notify(); + } + + /// The parent's monitor waiter, after the child's publication: whether to + /// deliver the `Down` now; otherwise `down` is parked for the waiter or + /// discarded. + pub(crate) fn dispose_down(&self, down: DownNotice) -> Option { + let mut inner = self.lock(); + inner.down_pending = false; + match inner.disposition { + Some(Disposition::Divert) => match inner.waiter { + Waiter::Waiting => { + inner.diverted_down = Some(down); + None + } + // The waiter answered before the waiter task ran: consumed. + Waiter::Consumed => { + drop(inner); + down(false); + None + } + // Abandoned since: handed on. + Waiter::Abandoned | Waiter::None => Some(down), + }, + Some(Disposition::Drop) => { + drop(inner); + down(false); + None + } + _ => Some(down), + } + } + + /// Observe the result (design 5.5). `waiter` consumes the answer and the + /// diverted notices; `handle` builds the instance's handle for `pending`. + pub(crate) fn observe(&self, waiter: bool, handle: impl FnOnce() -> ChildHandle) -> Observed { + let mut inner = self.lock(); + let commit = self.commit(); + let id = inner.trace; + let answer = match commit { + Commit::Pending => return self.wait_or_abandon(inner, waiter, None), + Commit::Ready => Ok(StartOutcome::Started(())), + _ => { + let (pending, reason) = match &inner.published { + Some(final_reason) => (None, final_reason.clone()), + None if inner.publishing => return self.wait_or_abandon(inner, waiter, None), + None => { + let deadline = inner + .committed_at + .map(|at| at + inner.cleanup) + .expect("a commitment has its instant"); + if Instant::now() < deadline { + return self.wait_or_abandon(inner, waiter, Some(deadline)); + } + if !std::mem::replace(&mut inner.deadline_seen, true) { + trace::start_emit(id, || trace::StartEvent::new("SDeadline", id)); + } + ( + Some(handle()), + inner.requested.clone().unwrap_or(ExitReason::Killed), + ) + } + }; + match commit { + Commit::Ignored => Ok(StartOutcome::Ignored { pending }), + Commit::TimedOut => Err(StartError::InitTimeout { pending }), + _ => Err(match &inner.cause { + Some(cause) => StartError::InitFailed { + cause: cause.clone(), + pending, + }, + None => StartError::InitExited { reason, pending }, + }), + } + } + }; + if !waiter { + trace::start_emit(id, || { + trace::StartEvent::new("SReady", id).x(answer_name(&answer)) + }); + return Observed::Answer(answer); + } + if inner.waiter != Waiter::Waiting { + return Observed::Answer(answer); + } + inner.waiter = Waiter::Consumed; + let link = inner.diverted_link.take(); + let down = inner.diverted_down.take(); + trace::start_emit(id, || { + let (reason, pending) = match &answer { + Err(StartError::InitExited { reason, pending }) => { + (trace::life_reason(reason), pending.is_some()) + } + Err(StartError::InitFailed { pending, .. }) + | Err(StartError::InitTimeout { pending }) + | Ok(StartOutcome::Ignored { pending }) => ("Normal", pending.is_some()), + _ => ("", false), + }; + trace::StartEvent::new("SAnswer", id) + .x(answer_name(&answer)) + .z(reason) + .f(pending) + }); + drop(inner); + self.unwatch_parent(); + // Consumed: the parent's records of the diverted notices go. + if let Some(link) = link { + link.discard(); + } + if let Some(down) = down { + down(false); + } + Observed::Answer(answer) + } + + /// No answer yet: wait, unless the waiting parent's exit began by a + /// signal, which abandons the wait (`ParentExited`). + fn wait_or_abandon( + &self, + inner: MutexGuard<'_, CellInner>, + waiter: bool, + deadline: Option, + ) -> Observed { + // Only an undecided start (the child still in started()) is + // abandoned: a decided one is answered at its publication, bounded + // by the cleanup deadline. (The failed child's own notice may be + // what kills a non-trapping parent; it must still get the answer.) + if waiter + && inner.parent_exited + && inner.waiter == Waiter::Waiting + && self.commit() == Commit::Pending + { + drop(inner); + self.abandon(); + return Observed::Answer(Err(StartError::ParentExited)); + } + Observed::Wait(deadline) + } + + /// The waiter was dropped unanswered (design 5.4): a diverted notice is + /// handed on as an ordinary one, exactly once, after the lock is released. + #[cfg_attr(not(feature = "async"), allow(dead_code))] + pub(crate) fn abandon(&self) { + let mut inner = self.lock(); + if inner.waiter != Waiter::Waiting { + return; + } + inner.waiter = Waiter::Abandoned; + let link = inner.diverted_link.take(); + let down = inner.diverted_down.take(); + let id = inner.trace; + // A Down not yet built by the monitor waiter is handed on by it. + let handed = link.is_some() + || down.is_some() + || (inner.down_pending && inner.disposition == Some(Disposition::Divert)); + trace::start_emit(id, || trace::StartEvent::new("SAbandon", id).f(handed)); + drop(inner); + self.unwatch_parent(); + if let Some(link) = link { + link.deliver(); + } + if let Some(down) = down { + down(true); + } + } + + /// A waiting `start_link`/`start_monitor` registers with its parent: an + /// exit of the parent begun by a kill or an untrapped exit signal + /// abandons the wait (OTP: the caller dies at once, without waiting for + /// the child's `init/1`). Called after stage 1. + pub(crate) fn watch_parent(self: &Arc) { + let parent = { + let inner = self.lock(); + if inner.waiter != Waiter::Waiting { + return; + } + match &inner.edge { + Some(edge) => edge.parent.clone(), + None => return, + } + }; + let registered = + crate::link::lock_request(&parent.request).add_waiting_start(Arc::downgrade(self)); + if !registered { + self.parent_exit(); + } + } + + /// The answer was taken or the wait abandoned: unregister from the + /// parent (no cell lock held). + fn unwatch_parent(&self) { + let parent = self.lock().edge.as_ref().map(|edge| edge.parent.clone()); + if let Some(parent) = parent { + crate::link::lock_request(&parent.request).remove_waiting_start(self); + } + } + + /// The waiting parent's exit began by a signal: wake the waiter, which + /// abandons the wait unless the answer is ready. + pub(crate) fn parent_exit(&self) { + { + let mut inner = self.lock(); + if inner.waiter != Waiter::Waiting { + return; + } + inner.parent_exited = true; + } + self.notify(); + } + + /// The start_monitor's waiter ended without a `Down` to dispose (its + /// monitor cancelled, or it never ran): an abandonment hands on nothing. + pub(crate) fn down_gone(&self) { + self.lock().down_pending = false; + } + + /// The change counter, read before an observation. + pub(crate) fn version(&self) -> u64 { + self.lock().version + } + + /// Wait (blocking) until the cell changes after `seen`, or `deadline`. + pub(crate) fn wait_blocking(&self, seen: u64, deadline: Option) { + let mut inner = self.lock(); + while inner.version == seen { + match deadline { + None => { + inner = self.changed.wait(inner).unwrap_or_else(|p| p.into_inner()); + } + Some(deadline) => { + let now = Instant::now(); + if deadline <= now { + return; + } + inner = self + .changed + .wait_timeout(inner, deadline - now) + .unwrap_or_else(|p| p.into_inner()) + .0; + } + } + } + } + + /// A receiver bumped at every change (async observers). + #[cfg_attr(not(feature = "async"), allow(dead_code))] + pub(crate) fn subscribe(&self) -> spawned_rt::sync::watch::Receiver { + self.version + .get_or_init(|| spawned_rt::sync::watch::channel(0).0) + .subscribe() + } +} + +#[allow(dead_code)] // trace only +fn answer_name(answer: &Result, StartError>) -> &'static str { + match answer { + Ok(StartOutcome::Started(())) => "ok", + Ok(StartOutcome::Ignored { .. }) => "ignored", + Err(StartError::InitTimeout { .. }) => "timeout", + Err(_) => "failed", + } +} + +/// The disposition table of design 5.2. +fn dispose( + commit: Commit, + waiter: Waiter, + kind: EdgeKind, + trapping: bool, + reason: &ExitReason, +) -> Disposition { + if commit == Commit::Ready || matches!(waiter, Waiter::None | Waiter::Abandoned) { + return Disposition::Deliver; + } + match waiter { + Waiter::Waiting => match kind { + EdgeKind::Monitor => Disposition::Divert, + EdgeKind::Link if trapping || commit == Commit::TimedOut => Disposition::Divert, + // A non-trapping parent: applied if non-Normal (OTP: the link + // kills the caller); a Normal one is ignored, so it is dropped + // here (the model's legitimate drop) rather than delivered. + EdgeKind::Link if *reason == ExitReason::Normal => Disposition::Drop, + EdgeKind::Link => Disposition::Deliver, + }, + // Answered at the cleanup deadline before publication. + _ => { + if kind == EdgeKind::Link + && !trapping + && commit == Commit::Failed + && *reason != ExitReason::Normal + { + Disposition::Deliver + } else { + Disposition::Drop + } + } + } +} + +/// Held by a start_monitor's waiter: unless it disposes a `Down` +/// ([`take`](Self::take)), its drop tells the cell no `Down` is coming, so +/// an abandonment records only the hand-on that happens. +pub(crate) struct DownWatch(pub(crate) Option>); +impl DownWatch { + pub(crate) fn take(&mut self) -> Option> { + self.0.take() + } +} +impl Drop for DownWatch { + fn drop(&mut self) { + if let Some(cell) = self.0.take() { + cell.down_gone(); + } + } +} + +/// A committed child whose lifecycle cannot be dispatched (its group or +/// worker is gone, its thread cannot be spawned): kill it, then drive its +/// lifecycle here. The kill arrived before `started()` began, so the +/// lifecycle runs no user code and never blocks (`KilledEarly`): it +/// resolves the start `InitExited { Killed }`, publishes and returns. The +/// caller's `Starting` answers as for any other failed start. +pub(crate) fn run_killed(child: &ChildHandle, run: impl FnOnce()) { + child.kill(); + run(); +} + +/// Poll a future that never blocks (a killed lifecycle) to completion on +/// this thread, without a runtime. +#[cfg_attr(not(feature = "async"), allow(dead_code))] // tasks mode only +pub(crate) fn run_inline>(future: F) { + struct Unpark(std::thread::Thread); + impl std::task::Wake for Unpark { + fn wake(self: Arc) { + self.0.unpark(); + } + } + let waker = std::task::Waker::from(Arc::new(Unpark(std::thread::current()))); + let mut cx = std::task::Context::from_waker(&waker); + let mut future = std::pin::pin!(future); + while future.as_mut().poll(&mut cx).is_pending() { + std::thread::park(); + } +} + +/// Run `payload` with `run` on a new OS thread; if the thread cannot be +/// spawned, kill `child` and finish it with `killed` here ([`run_killed`]). +pub(crate) fn spawn_thread_or_kill( + builder: std::thread::Builder, + child: &ChildHandle, + payload: T, + run: impl FnOnce(T) + Send + 'static, + killed: impl FnOnce(T), +) { + let slot = Arc::new(Mutex::new(Some(payload))); + let theirs = slot.clone(); + let spawn = || -> std::io::Result<()> { + fail_point!("actor.thread_spawn" => return Err(std::io::Error::other("failpoint"))); + builder + .spawn(move || { + let payload = theirs.lock().unwrap_or_else(|p| p.into_inner()).take(); + if let Some(payload) = payload { + run(payload); + } + }) + .map(drop) + }; + if let Err(error) = spawn() { + tracing::error!(%error, actor = ?child.id(), "actor thread spawn failed; killing the child"); + let payload = slot.lock().unwrap_or_else(|p| p.into_inner()).take(); + if let Some(payload) = payload { + run_killed(child, || killed(payload)); + } + } +} + +/// What the init timer does at expiry: commit `TimedOut` unless the child +/// committed first, then kill it (the kill is applied when started() +/// returns; the timer's commitment already decided the start, row 1). +fn fire_init_timer(cell: &std::sync::Weak, handle: &ChildHandle) { + let Some(cell) = cell.upgrade() else { + return; + }; + if cell.time_out() { + crate::link::lock_request(&handle.signal_target().request).kill(); + (handle.signal_target().wake)(); + } +} + +/// The init timer of a tasks-mode start (design 6): a runtime sleep task +/// (virtual time under the simulator), cancelled when the child commits. +/// Only a start with `init_timeout` pays for it. Without a current runtime +/// it falls back to the shared timer thread. +#[cfg(feature = "async")] +pub(crate) fn spawn_init_timer_async( + cell: &Arc, + handle: ChildHandle, + after: Duration, +) { + let Ok(runtime) = spawned_rt::tasks::Handle::try_current() else { + return spawn_init_timer(cell, handle, after); + }; + let Some(cancel) = cell.arm_timer() else { + return; + }; + let token = spawned_rt::tasks::CancellationToken::new(); + let cell = Arc::downgrade(cell); + let theirs = token.clone(); + drop(runtime.spawn(async move { + let sleep = std::pin::pin!(spawned_rt::tasks::sleep(after)); + let cancelled = std::pin::pin!(theirs.cancelled()); + if let futures::future::Either::Left(_) = futures::future::select(sleep, cancelled).await { + fire_init_timer(&cell, &handle); + } + })); + cancel.on_cancel(Box::new(move || token.cancel())); +} + +/// The init timer of a threads-mode start (design 6): an entry on the one +/// shared timer thread, removed when the child commits. Only a start with +/// `init_timeout` pays for it. +pub(crate) fn spawn_init_timer(cell: &Arc, handle: ChildHandle, after: Duration) { + let Some(cancel) = cell.arm_timer() else { + return; + }; + let Some(timers) = shared_timers() else { + tracing::warn!("init timer thread could not be spawned; the start has no init timeout"); + return; + }; + let cell = Arc::downgrade(cell); + let key = ( + std::time::Instant::now() + after, + TIMER_SEQ.fetch_add(1, Ordering::Relaxed), + ); + timers + .queue + .lock() + .unwrap_or_else(|p| p.into_inner()) + .insert(key, Box::new(move || fire_init_timer(&cell, &handle))); + timers.wake.notify_one(); + cancel.on_cancel(Box::new(move || { + let removed = timers + .queue + .lock() + .unwrap_or_else(|p| p.into_inner()) + .remove(&key); + // Dropped outside the lock. + drop(removed); + })); +} + +type TimerKey = (std::time::Instant, u64); +static TIMER_SEQ: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0); + +/// The shared init-timer thread's queue, by deadline. +struct Timers { + queue: Mutex>>, + wake: Condvar, +} + +/// The one init-timer thread of the process, started on first use. +fn shared_timers() -> Option<&'static Timers> { + static TIMERS: std::sync::OnceLock> = std::sync::OnceLock::new(); + *TIMERS.get_or_init(|| { + let timers: &'static Timers = Box::leak(Box::new(Timers { + queue: Mutex::new(std::collections::BTreeMap::new()), + wake: Condvar::new(), + })); + std::thread::Builder::new() + .name("spawned-init-timer".into()) + .spawn(move || run_timers(timers)) + .ok() + .map(|_| timers) + }) +} + +fn run_timers(timers: &'static Timers) { + let mut queue = timers.queue.lock().unwrap_or_else(|p| p.into_inner()); + loop { + let now = std::time::Instant::now(); + match queue.first_key_value().map(|(key, _)| *key) { + Some(key) if key.0 <= now => { + let fire = queue.remove(&key); + drop(queue); + if let Some(fire) = fire { + fire(); + } + queue = timers.queue.lock().unwrap_or_else(|p| p.into_inner()); + } + Some(key) => { + queue = timers + .wake + .wait_timeout(queue, key.0 - now) + .unwrap_or_else(|p| p.into_inner()) + .0; + } + None => { + queue = timers.wake.wait(queue).unwrap_or_else(|p| p.into_inner()); + } + } + } +} + +#[cfg(test)] +mod parent_exit_tests { + use super::*; + + fn no_handle() -> ChildHandle { + unreachable!("no pending handle in these answers") + } + + /// Review #291 item 7 follow-up: a parent whose exit a signal begins + /// while the start is already decided (the failed child's own notice + /// killing a non-trapping parent during publication) gets the start's + /// answer, not `ParentExited`. Only an undecided start (the child still + /// in `started()`) is abandoned. + #[test] + fn a_decided_start_is_answered_not_abandoned() { + let cell = StartupCell::new(); + cell.prepare(true, None, Duration::from_secs(5), 0); + cell.fail_at_death(&ExitReason::Shutdown(None)); + let _ = cell.publish(&ExitReason::Shutdown(None), None); + cell.parent_exit(); + assert!(matches!(cell.observe(true, no_handle), Observed::Wait(_))); + cell.published(&ExitReason::Shutdown(None)); + assert!(matches!( + cell.observe(true, no_handle), + Observed::Answer(Err(StartError::InitExited { + reason: ExitReason::Shutdown(None), + pending: None + })) + )); + } + + #[test] + fn an_undecided_start_is_abandoned() { + let cell = StartupCell::new(); + cell.prepare(true, None, Duration::from_secs(5), 0); + cell.parent_exit(); + assert!(matches!( + cell.observe(true, no_handle), + Observed::Answer(Err(StartError::ParentExited)) + )); + } +} diff --git a/concurrency/src/supervision/policy_tests.rs b/concurrency/src/supervision/policy_tests.rs index ece00e1ea..cee0d855e 100644 --- a/concurrency/src/supervision/policy_tests.rs +++ b/concurrency/src/supervision/policy_tests.rs @@ -74,11 +74,13 @@ async fn initial_placement_failure_has_deterministic_failed_slots() { for _ in 0..8 { let system = ActorSystem::new().unwrap(); let group = system.thread_group("placement-failure", 1).unwrap(); - let occupied = CoordinatorWorker.start_on(&group).unwrap(); + let occupied = CoordinatorWorker.with_options(&group).spawn().unwrap(); rt::timeout(Duration::from_secs(10), occupied.ready()) .await .unwrap() - .unwrap(); + .unwrap() + .started() + .expect("started"); let mut tree = system.supervisor(SupervisorConfig::default()).unwrap(); let first = tree .add(ChildSpec::new("first", &group, |_| async { @@ -281,11 +283,14 @@ async fn admission_closure_after_readiness_preserves_fixed_system_order() { entered: None, release: None, } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); } let mut builder = system.supervisor(SupervisorConfig::default()).unwrap(); let child_log = log.clone(); @@ -312,11 +317,14 @@ async fn admission_closure_after_readiness_preserves_fixed_system_order() { entered: None, release: None, } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); let (entered, arrival) = rt::oneshot::channel(); let (release, gate) = rt::oneshot::channel(); Ordered { @@ -325,11 +333,14 @@ async fn admission_closure_after_readiness_preserves_fixed_system_order() { entered: Some(entered), release: Some(gate), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); let shutdown = system.shutdown(limit); tokio::pin!(shutdown); assert!(futures::poll!(&mut shutdown).is_pending()); @@ -395,11 +406,14 @@ async fn startup_completion_or_drop_after_readiness_preserves_system_order() { entered: None, release: None, } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); } let mut builder = system.supervisor(SupervisorConfig::default()).unwrap(); let child_log = log.clone(); @@ -439,11 +453,14 @@ async fn startup_completion_or_drop_after_readiness_preserves_system_order() { entered: None, release: None, } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); let (entered, arrival) = rt::oneshot::channel(); let (release, gate) = rt::oneshot::channel(); Ordered { @@ -452,11 +469,14 @@ async fn startup_completion_or_drop_after_readiness_preserves_system_order() { entered: Some(entered), release: Some(gate), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); let shutdown = system.shutdown(limit); tokio::pin!(shutdown); assert!(futures::poll!(&mut shutdown).is_pending()); diff --git a/concurrency/src/supervision/run.rs b/concurrency/src/supervision/run.rs index c0f227de3..05b19fd2c 100644 --- a/concurrency/src/supervision/run.rs +++ b/concurrency/src/supervision/run.rs @@ -687,11 +687,12 @@ pub(super) async fn run_child(spec: ChildSpec, view: Arc>, future: construction, lease, }); + let ready_scope = scope.clone(); let startup = async { let actor = task .await .map_err(|e| format!("construction task failed: {e}"))??; - actor.ready().await.map_err(|e| e.to_string())?; + ready_scope.wait_ready().await?; Ok::<_, String>(actor) }; let mut startup_timed_out = false; diff --git a/concurrency/src/system.rs b/concurrency/src/system.rs index 845d31589..8665816de 100644 --- a/concurrency/src/system.rs +++ b/concurrency/src/system.rs @@ -1,14 +1,9 @@ //! Owned execution groups and lifecycle-owned helper scopes. //! -//! Ordinary `.start()` remains ambient and unmanaged. Explicit group admission +//! Ordinary `.spawn()` remains ambient and unmanaged. Explicit group admission //! owns actors/helpers and keeps their runtime alive until managed cleanup ends. use crate::stop_token::StopToken; -use crate::{ - child_handle::ChildHandle, - error::ExitReason, - link, - tasks::{Actor, ActorRef}, -}; +use crate::{child_handle::ChildHandle, error::ExitReason, link}; use futures::{ future::{select, BoxFuture, Either}, FutureExt, @@ -112,10 +107,8 @@ pub enum SystemError { Runtime(String), #[error("execution group belongs to a different actor system")] ForeignGroup, - #[error("this actor has no managed helper scope; use start_on")] + #[error("this actor has no managed helper scope; start it in a group")] UnmanagedActor, - #[error("actor stopped before becoming ready: {0}")] - StartupFailed(ExitReason), } #[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] @@ -869,37 +862,80 @@ impl ExecutionGroup { .map(|g| lock(&g.state).closing) .unwrap_or(true) } - pub fn start(&self, actor: A) -> Result, SystemError> { - let (scope, lease) = self.reserve(None)?; - ActorRef::spawn_managed(actor, scope, lease) - } - pub(crate) fn reserve( + /// Whether `other` places actors exactly as this group does (the same + /// group and shard selection). + pub(crate) fn same_as(&self, other: &ExecutionGroup) -> bool { + self.inner.ptr_eq(&other.inner) && self.selection == other.selection + } + /// A unified start's admission (stage 1, design section 3): under this + /// group's admission lock, check that it is open and has a slot, then run + /// `commit` (the start's edge and name, under their own locks); only if + /// it succeeds is the scope created with the instance's prepared stop + /// token and exit request. A refusal applies nothing. + pub(crate) fn reserve_with( &self, - parent: Option>, - ) -> Result<(Arc, Lease), SystemError> { - fail_point!("system.reserve" => return Err(SystemError::Capacity)); - let _parent = parent.as_ref().map(|p| lock(&p.0)); - if _parent.as_deref() == Some(&false) { - return Err(SystemError::Closed); - } - let group = self.inner.upgrade().ok_or(SystemError::Closed)?; + token: StopToken, + cause: crate::link::LinkedExitReason, + commit: impl FnOnce() -> Result, + ) -> Result<(Arc, Lease, R), crate::start::StartError> { + use crate::start::StartError; + fail_point!("system.reserve" => return Err(StartError::Capacity)); + let group = self.inner.upgrade().ok_or(StartError::Closed)?; let mut state = lock(&group.state); + // A refusal here is decided under the group lock (also its trace + // event: ordered with `SReleaseSlot`). + let refuse = |error: StartError| { + crate::lifecycle_trace::commit_point(Some(error.trace_refusal())); + Err(error) + }; if state.closing { - return Err(SystemError::Closed); + return refuse(StartError::Closed); } - let id = next_admission()?; - let target = group.choose(&mut state, self.selection)?; + let id = match next_admission() { + Ok(id) => id, + Err(error) => return refuse(error.into()), + }; + let target = match group.choose(&mut state, self.selection) { + Ok(target) => target, + Err(error) => return refuse(error.into()), + }; + let committed = match commit() { + Ok(committed) => { + crate::lifecycle_trace::commit_point(None); + committed + } + Err(error) => { + // Undo the slot choice under the same lock: invisible. + if let Some(free) = &mut state.free { + free.insert(target); + } + return Err(error); + } + }; + let scope = self.new_scope(&group, id, target, None, token, cause); + state.scopes.insert(id, scope.clone()); + Ok((scope.clone(), Lease { scope, root: true }, committed)) + } + fn new_scope( + &self, + group: &Arc, + id: u64, + target: usize, + parent: Option>, + token: StopToken, + cause: crate::link::LinkedExitReason, + ) -> Arc { let (ready, _) = watch::channel(StartupState::Pending); let (done, _) = watch::channel(false); - let scope = Arc::new(Scope { + Arc::new(Scope { id, - group: Arc::downgrade(&group), + group: Arc::downgrade(group), handle: group.handle(target), target, selection: self.selection, - parent: parent.clone(), - token: StopToken::new(), - cause: crate::link::new_linked_exit_reason(), + parent, + token, + cause, state: Mutex::new(ScopeState { open: true, leases: 1, @@ -920,7 +956,34 @@ impl ExecutionGroup { ), jobs: Mutex::new(None), bounds: Mutex::new(Default::default()), - }); + #[cfg(feature = "lifecycle-trace")] + start_trace: std::sync::atomic::AtomicU64::new(0), + }) + } + pub(crate) fn reserve( + &self, + parent: Option>, + ) -> Result<(Arc, Lease), SystemError> { + fail_point!("system.reserve" => return Err(SystemError::Capacity)); + let _parent = parent.as_ref().map(|p| lock(&p.0)); + if _parent.as_deref() == Some(&false) { + return Err(SystemError::Closed); + } + let group = self.inner.upgrade().ok_or(SystemError::Closed)?; + let mut state = lock(&group.state); + if state.closing { + return Err(SystemError::Closed); + } + let id = next_admission()?; + let target = group.choose(&mut state, self.selection)?; + let scope = self.new_scope( + &group, + id, + target, + parent.clone(), + StopToken::new(), + crate::link::new_linked_exit_reason(), + ); state.scopes.insert(id, scope.clone()); Ok((scope.clone(), Lease { scope, root: true })) } @@ -965,6 +1028,9 @@ pub(crate) struct Scope { /// Per-`ChildSpec` overrides of `Actor::MAX_OUTSTANDING` and /// `Actor::MAX_NATIVE_JOBS`, read when the actor instance is created. bounds: Mutex, + /// The start-trace child this scope admitted (0: not recorded). + #[cfg(feature = "lifecycle-trace")] + start_trace: std::sync::atomic::AtomicU64, } impl Scope { pub(crate) fn set_bounds(&self, bounds: crate::reply::jobs::Bounds) { @@ -1012,17 +1078,21 @@ impl Scope { pub(crate) fn ready_rx(&self) -> watch::Receiver { self.ready.subscribe() } - pub(crate) async fn wait_ready(&self) -> Result<(), SystemError> { + /// The supervisor's readiness barrier: ready, or the reason the actor + /// ended before it. + pub(crate) async fn wait_ready(&self) -> Result<(), String> { let mut ready = self.ready_rx(); loop { let state = ready.borrow_and_update().clone(); match state { StartupState::Ready => return Ok(()), - StartupState::Failed(reason) => return Err(SystemError::StartupFailed(reason)), + StartupState::Failed(reason) => { + return Err(format!("actor stopped before becoming ready: {reason}")) + } StartupState::Pending => (), } if ready.changed().await.is_err() { - return Err(SystemError::Closed); + return Err(SystemError::Closed.to_string()); } } } @@ -1081,6 +1151,7 @@ impl Scope { name: &str, id: crate::ActorId, actor: impl Clone + Send + Sync + 'static, + handle: crate::ChildHandle, ) -> Result<(), crate::registry::ManagedRegistrationError> { fail_point!("registry.before_register" => { return Err(crate::registry::ManagedRegistrationError::OwnerClosed) @@ -1098,9 +1169,12 @@ impl Scope { { return Err(crate::registry::ManagedRegistrationError::OwnerClosed); } - state - .registrations - .push(crate::registry::register_owned(name, id, actor)?); + state.registrations.push(crate::registry::register_owned( + name, + id, + actor, + Some(handle), + )?); if state.name.is_none() { state.name = Some(name.to_owned()); } @@ -1151,7 +1225,10 @@ impl Scope { let scope = lock(&self.state); !group.closing && scope.open && !self.token.is_cancelled() } - pub(crate) fn mark_ready(&self) -> bool { + /// Whether the scope still admits its actor (its owner's gate, the group + /// and the scope), not counting a stop request: an administrative stop + /// before `started()` waits for the start (OTP), it does not close it. + pub(crate) fn admission_open(&self) -> bool { let _parent = self.parent.as_ref().map(|p| lock(&p.0)); if _parent.as_deref() == Some(&false) { return false; @@ -1160,13 +1237,40 @@ impl Scope { return false; }; let group = lock(&g.state); + let scope = lock(&self.state); + !group.closing && scope.open + } + /// The readiness commit of a unified start (design 5.1): under this + /// scope's admission (its owner's gate, the group, the scope), `decide` + /// gets whether admission is open and returns its result and whether the + /// instance became ready, which then marks the scope ready. + pub(crate) fn ready_commit(&self, decide: impl FnOnce(bool) -> (R, bool)) -> R { + let parent = self.parent.as_ref().map(|p| lock(&p.0)); + let group = self.group.upgrade(); + let group_state = group.as_ref().map(|g| lock(&g.state)); let mut scope = lock(&self.state); - if group.closing || !scope.open || self.token.is_cancelled() { - return false; + let open = parent.as_deref() != Some(&false) + && group_state.as_ref().is_some_and(|g| !g.closing) + && scope.open; + let (result, ready) = decide(open); + if ready { + scope.ready_at = Some(Instant::now()); + self.ready.send_replace(StartupState::Ready); + } + result + } + /// The start-trace child this scope admitted. + #[cfg(feature = "lifecycle-trace")] + pub(crate) fn set_start_trace(&self, c: u64) { + self.start_trace + .store(c, std::sync::atomic::Ordering::Release); + } + /// A start's label: the diagnostic name until a name is registered. + pub(crate) fn set_label(&self, label: &str) { + let mut state = lock(&self.state); + if state.name.is_none() { + state.name = Some(label.to_owned()); } - scope.ready_at = Some(Instant::now()); - self.ready.send_replace(StartupState::Ready); - true } pub(crate) fn end_running(&self) { let mut state = lock(&self.state); @@ -1219,14 +1323,20 @@ impl Scope { self.lease(false) } /// Dispatch the actor root to its reserved worker or runtime shard. - pub(crate) fn dispatch_actor(&self, future: F) -> Result<(), SystemError> + /// Dispatch a committed actor to its worker. The future comes back if + /// the group or its worker is gone (dropped mid-shutdown). + pub(crate) fn dispatch_actor( + &self, + future: F, + ) -> Result<(), std::pin::Pin + Send>>> where F: Future + Send + 'static, { - self.group - .upgrade() - .ok_or(SystemError::Closed)? - .dispatch(self.target, future) + fail_point!("system.dispatch_actor" => return Err(Box::pin(future))); + match self.group.upgrade() { + Some(group) => group.dispatch(self.target, future), + None => Err(Box::pin(future)), + } } /// Internal lifecycle cleanup is deliberately not cancelled with ordinary /// helpers: its lease must survive until the owned subtree actually joins. @@ -1382,8 +1492,20 @@ impl Drop for Lease { } if last { drop(registrations); + // The scope's owned work is done (`Start.tla` WorkDoneC), then its + // slot is released under the group lock (ReleaseSlot). + #[cfg(feature = "lifecycle-trace")] + let c = self + .scope + .start_trace + .load(std::sync::atomic::Ordering::Acquire); + #[cfg(not(feature = "lifecycle-trace"))] + let c = 0; + crate::lifecycle_trace::start_emit(c, || { + crate::lifecycle_trace::StartEvent::new("SWorkDone", c) + }); if let Some(g) = self.scope.group.upgrade() { - g.remove_scope(self.scope.id); + g.remove_scope(self.scope.id, c); } self.scope.done.send_replace(true); } @@ -1411,6 +1533,7 @@ impl Future for Tracked { #[cfg(test)] mod coordinator_tests { use super::*; + use crate::tasks::Actor; use std::sync::atomic::{AtomicBool, AtomicUsize}; const LIMIT: Duration = Duration::from_secs(10); struct Worker(Arc); @@ -1459,10 +1582,16 @@ mod coordinator_tests { let system = ActorSystem::new().unwrap(); let group = system.async_group("coordinator-fault", 1).unwrap(); let stopped = Arc::new(AtomicUsize::new(0)); - let first = Worker(stopped.clone()).start_on(&group).unwrap(); - let second = Worker(stopped.clone()).start_on(&group).unwrap(); - first.ready().await.unwrap(); - second.ready().await.unwrap(); + let first = Worker(stopped.clone()) + .with_options(&group) + .spawn() + .unwrap(); + let second = Worker(stopped.clone()) + .with_options(&group) + .spawn() + .unwrap(); + first.ready().await.unwrap().started().expect("started"); + second.ready().await.unwrap().started().expect("started"); let (done, _) = watch::channel(false); let fault = Arc::new(FaultUnit { allow_emergency: AtomicBool::new(false), diff --git a/concurrency/src/system/group.rs b/concurrency/src/system/group.rs index 131fa9c2c..c78281df3 100644 --- a/concurrency/src/system/group.rs +++ b/concurrency/src/system/group.rs @@ -237,14 +237,18 @@ impl GroupInner { pub(super) fn shard_count(&self) -> usize { self.targets.len() } - pub(super) fn dispatch(&self, target: usize, future: F) -> Result<(), SystemError> + /// Hands the future back when the worker is gone. + pub(super) fn dispatch(&self, target: usize, future: F) -> Result<(), WorkFuture> where F: Future + Send + 'static, { match &self.targets[target].native { Some(sender) => sender .send(Work::Actor(Box::pin(future))) - .map_err(|_| SystemError::Runtime("native worker is unavailable".into())), + .map_err(|refused| match refused.0 { + Work::Actor(future) => future, + Work::Stop => unreachable!("dispatch sends actors only"), + }), None => { drop(self.targets[target].handle.spawn(future)); Ok(()) @@ -255,10 +259,13 @@ impl GroupInner { lock(&self.state).closing = true; self.maybe_finish(); } - pub(super) fn remove_scope(&self, id: u64) { + pub(super) fn remove_scope(&self, id: u64, _trace: u64) { let removed = { let mut state = lock(&self.state); let removed = state.scopes.remove(&id); + crate::lifecycle_trace::start_emit(_trace, || { + crate::lifecycle_trace::StartEvent::new("SReleaseSlot", _trace) + }); if let Some(scope) = &removed { if let Some(free) = &mut state.free { free.insert(scope.target); diff --git a/concurrency/src/tasks/actor.rs b/concurrency/src/tasks/actor.rs index 34b6c48ac..0bab4adbe 100644 --- a/concurrency/src/tasks/actor.rs +++ b/concurrency/src/tasks/actor.rs @@ -94,7 +94,15 @@ pub trait Actor: Send + Sized + 'static { /// with `#[actor(max_native_jobs = N)]` or `ChildSpec::max_native_jobs`. const MAX_NATIVE_JOBS: Option = None; - fn started(&mut self, _ctx: &Context) -> impl Future + Send { + /// Initialisation, before any message (OTP `init/1`). Returns `()`, + /// `Result<(), E>`, `Result, E>` or [`Init`]: ready, a + /// continuation, an error, a stop or ignore. The waiting starts + /// (`start`, `start_link`, `start_monitor`) and [`ActorRef::ready`] + /// answer with its result. A failed start never runs `stopped()`. + fn started( + &mut self, + _ctx: &Context, + ) -> impl Future> + Send> + Send { async {} } @@ -191,7 +199,10 @@ struct ActorShared { /// This instance as the target of exit signals and link notices. signal: Arc, mailbox: crate::instrumentation::Mailbox, - managed: Option>, + /// The managed scope, set at the commit (unified start, stage 1). + managed: std::sync::OnceLock>, + /// The backend of an unmanaged instance (inherited by `start_link`). + backend: Backend, /// This instance's reply registry: deferred tokens, credits, turn phase. replies: Arc, /// This instance's pipe-to-self jobs. @@ -313,7 +324,15 @@ fn test_context( let links = new_link_table(); let request = new_linked_exit_reason(); let monitors: MonitorTable = Arc::new(crate::monitor::fence::open()); - let signal = signal_target::(id, None, tx, trap.clone(), request.clone(), token.clone()); + let signal = signal_target::( + id, + None, + tx, + trap.clone(), + request.clone(), + token.clone(), + crate::start::StartupCell::new(), + ); let ctx = Context { shared: Arc::new(ActorShared { id, @@ -326,7 +345,8 @@ fn test_context( linked_reason: request.clone(), signal, mailbox: crate::instrumentation::Mailbox::default(), - managed: None, + managed: std::sync::OnceLock::new(), + backend: Backend::Async, replies: Registry::new(id, DEFAULT_MAX_OUTSTANDING), jobs: Arc::new(crate::reply::jobs::JobSet::new(id, 1)), cleaned: watch::channel(false).1, @@ -340,47 +360,20 @@ fn test_context( } // --------------------------------------------------------------------------- -// Actor::start +// Starting actors (unified start, 0.9) // --------------------------------------------------------------------------- -/// Extension trait for starting an actor. Automatically implemented for all [`Actor`] types. -pub trait ActorStart: Actor { - /// Start the actor with the default backend ([`Backend::Async`]). - fn start(self) -> ActorRef { - self.start_with_backend(Backend::default()) - } - - /// Admit this actor to an owned parked async execution group. - fn start_on( - self, - group: &crate::system::ExecutionGroup, - ) -> Result, crate::system::SystemError> { - group.start(self) - } - - /// Start the actor with a specific [`Backend`]. - fn start_with_backend(self, backend: Backend) -> ActorRef { - ActorRef::spawn(self, backend, None) - } +mod start; +pub use start::{ActorStart, Init, Name, StartOptions, Starter, Starting}; - /// Start linked to `parent_ctx`'s actor, which becomes this instance's - /// parent (E3). The link is registered before `started()`. The parent's - /// exit ends a trapping child normally (its `stopped()` runs; it never - /// reaches `exit_received`), a non-trapping child as any link notice. A - /// parent already past its death point gives the child a `NoProc` parent - /// exit. - /// - /// The child is always an unmanaged `Backend::Async` actor, even when - /// the parent is managed: it is outside the parent's execution group and - /// its ordered shutdown (`spawn_helper` on it returns `UnmanagedActor`). - /// The unified start API (planned for 0.9) will inherit the placement. - fn start_linked(self, parent_ctx: &Context

) -> ActorRef { - ActorRef::spawn(self, Backend::Async, Some(parent_ctx.child_handle())) +impl ActorShared { + /// The managed scope (set once, at the commit). + #[inline] + pub(crate) fn managed(&self) -> Option<&Arc> { + self.managed.get() } } -impl ActorStart for A {} - // --------------------------------------------------------------------------- // send_message_on (utility) // --------------------------------------------------------------------------- diff --git a/concurrency/src/tasks/actor/actor_ref.rs b/concurrency/src/tasks/actor/actor_ref.rs index c32dc8d30..ee5d5a633 100644 --- a/concurrency/src/tasks/actor/actor_ref.rs +++ b/concurrency/src/tasks/actor/actor_ref.rs @@ -9,15 +9,14 @@ impl ActorRef { crate::response::RequestOptions::new(self.clone(), timeout) } - /// Await this managed instance's initial readiness barrier. Success does not - /// guarantee it remains alive afterward; ordinary request errors still apply. - pub async fn ready(&self) -> Result<(), crate::system::SystemError> { - self.shared - .managed - .as_ref() - .ok_or(crate::system::SystemError::UnmanagedActor)? - .wait_ready() - .await + /// This instance's committed startup result, for every placement and + /// verb (a `spawn`ed actor included): `Started` once `started()` + /// returned ready, `Ignored`, or the start's error. A failure is + /// answered after the instance's publication (or with `pending` after + /// its cleanup deadline), as the waiting starts answer. Success does not + /// guarantee the actor remains alive afterward. + pub async fn ready(&self) -> Result, crate::start::StartError> { + start::observe(self, false).await } pub(crate) fn signal_failure(&self, error: String) { let _ = self diff --git a/concurrency/src/tasks/actor/api_coverage.rs b/concurrency/src/tasks/actor/api_coverage.rs index 967630076..260313803 100644 --- a/concurrency/src/tasks/actor/api_coverage.rs +++ b/concurrency/src/tasks/actor/api_coverage.rs @@ -72,7 +72,7 @@ fn context_recipient_reaches_its_actor() { concat!(module_path!(), "::context_recipient_reaches_its_actor"), move || { rt::Runtime::new().unwrap().block_on(async { - let actor = Echo.start(); + let actor = Echo.spawn(); let (recipient, id) = actor.request(OwnRecipient).await.unwrap(); assert_eq!(id, actor.id()); assert!(matches!(request(&*recipient, Ask(7), WAIT).await, Ok(7))); @@ -95,7 +95,7 @@ fn requests_to_a_stopped_actor_fail_without_waiting() { ), move || { rt::Runtime::new().unwrap().block_on(async { - let actor = Echo.start(); + let actor = Echo.spawn(); let recipient = actor.recipient::(); actor.child_handle().stop(); actor.join().await; @@ -122,7 +122,7 @@ fn admitted_requests_fail_when_the_actor_stops_before_replying() { ), move || { rt::Runtime::new().unwrap().block_on(async { - let actor = Echo.start(); + let actor = Echo.spawn(); let recipient = actor.recipient::(); let (release, held) = rt::oneshot::channel(); actor.send(StopAfter(held)).unwrap(); @@ -150,7 +150,7 @@ fn erased_request_times_out() { concat!(module_path!(), "::erased_request_times_out"), move || { rt::Runtime::new().unwrap().block_on(async { - let actor = Echo.start(); + let actor = Echo.spawn(); let recipient = actor.recipient::(); let (release, held) = rt::oneshot::channel(); actor.send(StopAfter(held)).unwrap(); @@ -190,9 +190,9 @@ fn default_exit_received_keeps_a_trapping_actor_alive() { } } rt::Runtime::new().unwrap().block_on(async { - let trapper = Trapper.start(); + let trapper = Trapper.spawn(); assert!(matches!(trapper.request(Ask(0)).await, Ok(0))); - let peer = Echo.start(); + let peer = Echo.spawn(); trapper.context().link(&peer.child_handle()); peer.send(Boom).unwrap(); assert!(matches!(peer.wait_exit().await, ExitReason::Panic(_))); @@ -214,7 +214,7 @@ fn debug_output_names_the_handle() { concat!(module_path!(), "::debug_output_names_the_handle"), move || { rt::Runtime::new().unwrap().block_on(async { - let actor = Echo.start(); + let actor = Echo.spawn(); assert!(format!("{actor:?}").starts_with("ActorRef")); assert!(format!("{:?}", actor.context()).starts_with("Context")); actor.child_handle().stop(); diff --git a/concurrency/src/tasks/actor/context.rs b/concurrency/src/tasks/actor/context.rs index ebaad3056..9e9d9f5a6 100644 --- a/concurrency/src/tasks/actor/context.rs +++ b/concurrency/src/tasks/actor/context.rs @@ -281,14 +281,31 @@ impl Context { if !registered { return monitor_ref; } + self.spawn_monitor_waiter(target.clone(), monitor_ref, active, None); + monitor_ref + } - let target = target.clone(); + /// The monitor's waiter: at the target's publication, the `Down` is + /// delivered (receiver-side fenced). A `start_monitor`'s waiter disposes + /// of it through the child's startup cell instead (design 5.2): diverted + /// to the start's waiter, dropped once its error consumed it, or handed + /// on after abandonment. + fn spawn_monitor_waiter( + &self, + target: ChildHandle, + monitor_ref: MonitorRef, + active: spawned_rt::tasks::CancellationToken, + cell: Option>, + ) where + A: Handler, + { + let observer = self.shared.id.raw(); let actor_ref = self.actor_ref(); let monitors = self.shared.monitors.clone(); let registration = crate::monitor::MonitorRegistration(monitors.clone(), monitor_ref); + let mut watch = crate::start::DownWatch(cell); self.spawn_framework(async move { - let _registration = registration; let reason = match futures::future::select( Box::pin(active.cancelled()), Box::pin(target.wait_exit_async()), @@ -301,31 +318,86 @@ impl Context { // Receiver-side fence (E6): the entry is removed and the Down // enqueued in one critical section, so once `demonitor` returns // no Down of this monitor is enqueued. - crate::lifecycle_trace::life_section(observer, || { - crate::monitor::fence::deliver(&monitors, &monitor_ref, || { - crate::lifecycle_trace::life_emit(observer, || { - crate::lifecycle_trace::LifeEvent::new("Deliver", observer) - .s(target.id().raw()) - .m(monitor_ref.raw()) - .x(crate::lifecycle_trace::life_reason(&reason)) - .k("down") - .o("death") + let deliver = move |deliver: bool| { + let _registration = registration; + if !deliver { + return; + } + crate::lifecycle_trace::life_section(observer, || { + crate::monitor::fence::deliver(&monitors, &monitor_ref, || { + crate::lifecycle_trace::life_emit(observer, || { + crate::lifecycle_trace::LifeEvent::new("Deliver", observer) + .s(target.id().raw()) + .m(monitor_ref.raw()) + .x(crate::lifecycle_trace::life_reason(&reason)) + .k("down") + .o("death") + }); + crate::lifecycle_trace::start_notice( + observer, + target.id().raw(), + "monitor", + &reason, + || { + let _ = enqueue_down( + &actor_ref.shared.sender, + &actor_ref.shared.mailbox, + &actor_ref.shared.replies, + Down { + monitor_ref, + actor: target.id(), + reason: reason.clone(), + }, + ); + }, + ); }); - let _ = enqueue_down( - &actor_ref.shared.sender, - &actor_ref.shared.mailbox, - &actor_ref.shared.replies, - Down { - monitor_ref, - actor: target.id(), - reason, - }, - ); }); - }); + }; + match watch.take() { + None => deliver(true), + Some(cell) => { + if let Some(deliver) = cell.dispose_down(Box::new(deliver)) { + deliver(true); + } + } + } }); + } - monitor_ref + /// The start edge of a `start_monitor` (committed by the caller's stage + /// 1): arm the waiter before the child is dispatched. + pub(super) fn arm_start_monitor( + &self, + child: ChildHandle, + monitor_ref: MonitorRef, + active: spawned_rt::tasks::CancellationToken, + cell: Arc, + ) where + A: Handler, + { + self.spawn_monitor_waiter(child, monitor_ref, active, Some(cell)); + } + + /// This actor's placement (inherited by `start_link`/`start_monitor`). + pub(super) fn placement(&self) -> start::Placement { + match self.shared.managed() { + Some(scope) => start::Placement::Group(scope.group()), + None => start::Placement::Backend(self.shared.backend), + } + } + + /// This actor as the parent of a `start_link`/`start_monitor`. + pub(super) fn parent_parts(&self) -> lifecycle::ParentParts { + lifecycle::ParentParts { + id: self.shared.id, + signal: self.shared.signal.clone(), + links: self.shared.links.clone(), + request: self.shared.linked_reason.clone(), + monitors: self.shared.monitors.clone(), + token: self.shared.cancellation_token.clone(), + placement: self.placement(), + } } /// Cancel a previously-set monitor. @@ -458,6 +530,7 @@ impl Context { } /// Type-erased handle to this actor, without an intermediate `ActorRef`. + #[allow(dead_code)] // kept for crate-internal callers and tests pub(crate) fn child_handle(&self) -> ChildHandle { self.shared.child_handle() } @@ -466,7 +539,7 @@ impl Context { /// Construct runtime-bound work inside the async block. Closure of the scope /// cancels the helper and waits for its future to drop before cleanup ends. /// Returns [`crate::system::SystemError::UnmanagedActor`] for actors started - /// with `.start()`; use `.start_on(&group)` to establish cleanup ownership. + /// with `.spawn()`; use `.with_options(&group).spawn()` to establish cleanup ownership. pub fn spawn_helper( &self, future: F, @@ -476,8 +549,7 @@ impl Context { F::Output: Send + 'static, { self.shared - .managed - .as_ref() + .managed() .ok_or(crate::system::SystemError::UnmanagedActor)? .spawn_helper(future) } @@ -486,7 +558,7 @@ impl Context { /// Cancellation cannot preempt it. Raw nested spawn calls are not implicitly /// owned; use this method rather than detaching a blocking task inside a /// cancellable async helper when recovery must await its resource cleanup. - /// Requires `.start_on(&group)`, like [`Self::spawn_helper`]. + /// Requires `.with_options(&group).spawn()`, like [`Self::spawn_helper`]. pub fn spawn_blocking_helper( &self, work: F, @@ -496,14 +568,13 @@ impl Context { T: Send + 'static, { self.shared - .managed - .as_ref() + .managed() .ok_or(crate::system::SystemError::UnmanagedActor)? .spawn_blocking_helper(work) } pub fn execution_group(&self) -> Option { - self.shared.managed.as_ref().map(|s| s.group()) + self.shared.managed().map(|s| s.group()) } /// Register this managed actor until its owned cleanup completes. Removal @@ -513,20 +584,24 @@ impl Context { name: &str, ) -> Result<(), crate::registry::ManagedRegistrationError> { self.shared - .managed - .as_ref() + .managed() .ok_or(crate::registry::ManagedRegistrationError::OwnerClosed)? - .register(name, self.shared.id, self.actor_ref()) + .register( + name, + self.shared.id, + self.actor_ref(), + self.actor_ref().child_handle(), + ) } pub(crate) fn managed_scope(&self) -> Option<&Arc> { - self.shared.managed.as_ref() + self.shared.managed() } pub(crate) fn spawn_framework(&self, future: F) -> JoinHandle<()> where F: Future + Send + 'static, { - match &self.shared.managed { + match self.shared.managed() { Some(scope) => scope.spawn_framework(future), None => rt::spawn(future), } diff --git a/concurrency/src/tasks/actor/early_stop_regression.rs b/concurrency/src/tasks/actor/early_stop_regression.rs index ef7add532..738e3747a 100644 --- a/concurrency/src/tasks/actor/early_stop_regression.rs +++ b/concurrency/src/tasks/actor/early_stop_regression.rs @@ -5,20 +5,39 @@ impl Actor for EarlyStop { ctx.stop(); } async fn stopped(&mut self, _: &Context) { - panic!("cleanup failed"); + panic!("stopped() must not run after a failed start"); } } +/// `ctx.stop()` in `started()` is a failed start (unified start, 2.3 row +/// 5, OTP `{stop, normal}` from `init/1`): `stopped()` does not run, the +/// actor ends `Normal`, and the start's result reports it. #[test] -fn panic_during_early_stop_is_reported() { +fn stop_in_started_fails_the_start_without_stopped() { crate::test_wait::bounded( - concat!(module_path!(), "::panic_during_early_stop_is_reported"), + concat!( + module_path!(), + "::stop_in_started_fails_the_start_without_stopped" + ), move || { let rt = rt::Runtime::new().unwrap(); let _enter = rt.enter(); - let actor = EarlyStop.start(); - assert!( - matches!(rt.block_on(actor.wait_exit()), ExitReason::Panic(message) if message.contains("cleanup failed")) - ); + let actor = EarlyStop.spawn(); + assert_eq!(rt.block_on(actor.wait_exit()), ExitReason::Normal); + assert!(matches!( + rt.block_on(actor.ready()), + Err(crate::StartError::InitExited { + reason: ExitReason::Normal, + pending: None + }) + )); + let waited = rt.block_on(async { EarlyStop.start().await }); + assert!(matches!( + waited, + Err(crate::StartError::InitExited { + reason: ExitReason::Normal, + pending: None + }) + )); }, ); } diff --git a/concurrency/src/tasks/actor/envelope.rs b/concurrency/src/tasks/actor/envelope.rs index 281228080..3eee233b0 100644 --- a/concurrency/src/tasks/actor/envelope.rs +++ b/concurrency/src/tasks/actor/envelope.rs @@ -515,6 +515,7 @@ pub(super) fn signal_target( trap: TrapExitFlag, request: LinkedExitReason, token: StopToken, + startup: Arc, ) -> Arc { let exit_tx = sender.clone(); let parent_tx = sender.clone(); @@ -544,5 +545,6 @@ pub(super) fn signal_target( send_exit, send_parent_exit, send_held, + startup, }) } diff --git a/concurrency/src/tasks/actor/lifecycle.rs b/concurrency/src/tasks/actor/lifecycle.rs index e487ea0f1..c3cb4b999 100644 --- a/concurrency/src/tasks/actor/lifecycle.rs +++ b/concurrency/src/tasks/actor/lifecycle.rs @@ -5,12 +5,35 @@ use super::*; // Actor startup + main loop // --------------------------------------------------------------------------- -impl ActorRef { - pub(super) fn spawn(actor: A, backend: Backend, parent: Option) -> Self { - Self::spawn_inner(actor, backend, parent, None, None) - .expect("unmanaged actor dispatch failed") +/// The parent side of a `start_link`/`start_monitor` (type-erased). +pub(super) struct ParentParts { + pub(super) id: ActorId, + pub(super) signal: Arc, + pub(super) links: LinkTable, + pub(super) request: LinkedExitReason, + pub(super) monitors: MonitorTable, + pub(super) token: StopToken, + pub(super) placement: start::Placement, +} + +/// What the commit applied (stage 1, design section 3). +struct Committed { + managed: Option<(Arc, crate::system::Lease)>, + name: Option, + monitor: Option<(MonitorRef, spawned_rt::tasks::CancellationToken)>, +} + +/// Whether an explicit placement equals the parent's (`placement_of`). +fn same_placement(a: &start::Placement, b: &start::Placement) -> bool { + match (a, b) { + (start::Placement::Backend(x), start::Placement::Backend(y)) => x == y, + (start::Placement::Group(x), start::Placement::Group(y)) => x.same_as(y), + _ => false, } +} +impl ActorRef { + /// A supervised generation, admitted to `scope` by its supervisor. pub(crate) fn spawn_managed( actor: A, scope: Arc, @@ -19,48 +42,143 @@ impl ActorRef { if !scope.can_start() { return Err(crate::system::SystemError::Closed); } - Self::spawn_inner(actor, Backend::Async, None, Some(scope), Some(lease)) + Self::launch_inner( + actor, + start::StartOptions::new(), + start::Verb::Spawn, + None, + Some((scope, lease)), + ) + .map(|(actor, _)| actor) + .map_err(|error| match error { + crate::start::StartError::System(error) => error, + _ => crate::system::SystemError::Closed, + }) } - pub(super) fn spawn_inner( + /// Stage 1 of every start (design section 3): validate, prepare the + /// instance privately, commit it atomically (group admission, the + /// parent's edge, the name), then dispatch it. A refused start was never + /// observable. + pub(super) fn launch( actor: A, - backend: Backend, - parent: Option, - managed: Option>, - lease: Option, - ) -> Result { + options: start::StartOptions, + verb: start::Verb, + arm: Option, + ) -> Result<(Self, Option), crate::start::StartError> { + Self::launch_inner(actor, options, verb, arm, None) + } + + fn launch_inner( + actor: A, + options: start::StartOptions, + verb: start::Verb, + arm: Option, + reserved: Option<(Arc, crate::system::Lease)>, + ) -> Result<(Self, Option), crate::start::StartError> { + use crate::lifecycle_trace as trace; + use crate::start::{EdgeKind, StartEdge, StartError}; + let id = ActorId::next(); + let c = trace::start_child(id.raw()); + let parent = match &verb { + start::Verb::Link(p) | start::Verb::Monitor(p) => Some(p), + _ => None, + }; + // The placement: explicit, else the parent's for link/monitor + // (design section 4), else unmanaged Async. + let explicit = options.placement.clone(); + let placement = match (&explicit, parent) { + (Some(placement), _) => placement.clone(), + (None, Some(p)) => p.placement.clone(), + (None, None) => start::Placement::Backend(Backend::Async), + }; + trace::start_emit(c, || { + trace::StartEvent::new("SBegin", c) + .x(verb.trace_name()) + .y(match (&explicit, parent) { + (None, _) => "none", + (Some(p), Some(parent)) if same_placement(p, &parent.placement) => "same", + (Some(p), _) => p.trace_name(), + }) + .f(options.name.is_some()) + }); + if let Err(error) = options.validate(&placement) { + trace::start_emit(c, || trace::StartEvent::new("SPrepareFail", c)); + return Err(error); + } + let (backend, group) = match (&placement, &reserved) { + (_, Some(_)) => (Backend::Async, None), + (start::Placement::Group(group), None) => (Backend::Async, Some(group.clone())), + (start::Placement::Backend(backend), None) => (Backend::to_owned(backend), None), + }; + + // Prepare: the instance's state, private until the commit. A managed + // instance's stop token and exit request are its scope's, created + // here and handed to the scope the commit creates. let (tx, rx) = mpsc::channel:: + Send>>(); - let cancellation_token = managed + let cancellation_token = reserved .as_ref() - .map(|s| s.token.clone()) + .map(|(s, _)| s.token.clone()) .unwrap_or_default(); + let linked_reason = reserved + .as_ref() + .map(|(s, _)| s.cause.clone()) + .unwrap_or_else(new_linked_exit_reason); let (completion_tx, completion_rx) = watch::channel(None); let monitors: MonitorTable = Arc::new(crate::monitor::fence::open()); let trap_exit = new_trap_exit_flag(); let links = new_link_table(); - let linked_reason = managed - .as_ref() - .map(|s| s.cause.clone()) - .unwrap_or_else(new_linked_exit_reason); - - let id = ActorId::next(); - // Phase 0: signals that arrive before started() returns are held. - link::lock_request(&linked_reason).begin_starting(); + { + // Phase 0: signals that arrive before started() returns are held. + let mut request = link::lock_request(&linked_reason); + request.begin_starting(); + #[cfg(feature = "lifecycle-trace")] + if c != 0 { + request.trace = trace::StartRole::Child(c); + } + } + let cell = crate::start::StartupCell::new(); + cell.prepare( + verb.waiting(), + parent.map(|p| StartEdge { + kind: if matches!(verb, start::Verb::Link(_)) { + EdgeKind::Link + } else { + EdgeKind::Monitor + }, + parent: p.signal.clone(), + }), + reserved + .as_ref() + .map_or(DEFAULT_STOP_DEADLINE, |(s, _)| s.cleanup_timeout()), + c, + ); + let link_parent = match &verb { + start::Verb::Link(p) => Some(p.id), + _ => None, + }; let signal = signal_target::( id, - parent.as_ref().map(ChildHandle::id), + link_parent, &tx, trap_exit.clone(), linked_reason.clone(), cancellation_token.clone(), + cell.clone(), ); let (cleaned_tx, cleaned_rx) = watch::channel(false); - // A supervised child's `ChildSpec` may override the actor's bounds. - let (max_outstanding, max_native) = managed + // A supervised child's `ChildSpec` may override the actor's bounds; + // so may the start's options. + let bounds = reserved .as_ref() - .map(|scope| scope.bounds()) + .map(|(scope, _)| scope.bounds()) .unwrap_or_default() - .resolve(A::MAX_OUTSTANDING, A::MAX_NATIVE_JOBS); + .or(options.max_outstanding, options.max_native_jobs); + let (max_outstanding, max_native) = bounds.resolve(A::MAX_OUTSTANDING, A::MAX_NATIVE_JOBS); + let managed_cell = std::sync::OnceLock::new(); + if let Some((scope, _)) = &reserved { + let _ = managed_cell.set(scope.clone()); + } let actor_ref = ActorRef { shared: Arc::new(ActorShared { id, @@ -76,22 +194,163 @@ impl ActorRef { linked_reason: linked_reason.clone(), signal, mailbox: crate::instrumentation::Mailbox::default(), - managed: managed.clone(), + managed: managed_cell, + backend, }), }; - let ctx = actor_ref.context(); + let child_handle = actor_ref.child_handle(); + // Commit (design section 3): the group's admission, the parent's + // edge, the name, all or nothing, each check and its trace event + // under the locks. Lock order: the group's admission, the parent's + // table, the parent's exit request, the registry. + let register = || -> Result, StartError> { + let Some(name) = &options.name else { + return Ok(None); + }; + match crate::registry::register_start( + name.key(), + id, + actor_ref.clone(), + child_handle.clone(), + c, + || Ok::<(), StartError>(()), + )? { + Ok(guard) => Ok(Some(guard)), + Err(existing) => Err(StartError::AlreadyStarted { + name: name.key().to_owned(), + existing, + }), + } + }; + let edge_commit = || -> Result { + let mut committed = Committed { + managed: None, + name: None, + monitor: None, + }; + match &verb { + start::Verb::Link(p) => { + let mut inner = Ok(None); + let installed = link::install_start_edge( + &p.signal, + &p.links, + &actor_ref.shared.signal, + &links, + || { + let request = link::lock_request(&p.request); + if request.reason().is_some() || request.is_dead() { + trace::commit_point(Some("parentexited")); + return false; + } + // The registry is the innermost lock. + inner = register(); + if inner.is_ok() { + trace::commit_point(None); + } + inner.is_ok() + }, + ); + match (installed, inner) { + (_, Err(error)) => return Err(error), + (false, Ok(_)) => return Err(StartError::ParentExited), + (true, Ok(name)) => committed.name = name, + } + } + start::Verb::Monitor(p) => { + let monitor_ref = MonitorRef::next(); + let active = spawned_rt::tasks::CancellationToken::new(); + let mut inner = Ok(None); + // The parent's monitor table is checked open first (the + // fence), so a name is never registered for a start the + // closed table then refuses. + let registered = crate::monitor::fence::register( + &p.monitors, + monitor_ref, + active.clone(), + || { + let request = link::lock_request(&p.request); + if request.reason().is_some() + || request.is_dead() + || p.token.is_cancelled() + { + trace::commit_point(Some("parentexited")); + return true; + } + inner = register(); + if inner.is_ok() { + trace::commit_point(None); + } + inner.is_err() + }, + || false, + || {}, + || {}, + ); + match (registered, inner) { + (_, Err(error)) => return Err(error), + (crate::monitor::fence::Registration::Active, Ok(name)) => { + committed.name = name; + committed.monitor = Some((monitor_ref, active)); + } + // The parent is exiting, or its monitor table closed. + _ => return Err(StartError::ParentExited), + } + } + _ => committed.name = register()?, + } + Ok(committed) + }; + let committed = trace::start_section(|| { + trace::commit_begin(c, placement.trace_name()); + let result = match &group { + Some(group) => group + .reserve_with(cancellation_token.clone(), linked_reason.clone(), || { + edge_commit() + }) + .map(|(scope, lease, mut committed)| { + committed.managed = Some((scope, lease)); + committed + }), + None => edge_commit(), + }; + trace::commit_end(result.as_ref().err().map(StartError::trace_refusal)); + result + }); + // A refusal applied nothing: the prepared state is dropped (no hook + // ran, nothing was published or registered). + let mut committed = committed?; + // A waiting start_link/start_monitor: the parent's exit by a signal + // abandons the wait (OTP: the caller dies at once). + cell.watch_parent(); + let (managed, lease) = match (committed.managed.take(), reserved) { + (Some((scope, lease)), _) | (None, Some((scope, lease))) => (Some(scope), Some(lease)), + (None, None) => (None, None), + }; if let Some(scope) = &managed { + // Admitted by the commit: the instance adopts the scope. + let _ = actor_ref.shared.managed.set(scope.clone()); + cell.set_cleanup(scope.cleanup_timeout()); scope.attach::(actor_ref.child_handle()); scope.set_jobs(Arc::downgrade(&actor_ref.shared.jobs)); + #[cfg(feature = "lifecycle-trace")] + scope.set_start_trace(c); + if let Some(label) = &options.label { + scope.set_label(label); + } } - - let parent_id = parent.as_ref().map_or(0, |p| p.id().raw()); - if let Some(parent) = parent { - ctx.link(&parent); + let ctx = actor_ref.context(); + let monitor_ref = committed.monitor.take().map(|(monitor_ref, active)| { + if let Some(arm) = arm { + arm(child_handle.clone(), monitor_ref, active, cell.clone()); + } + monitor_ref + }); + // Recorded from here: the start_link link is the model's initial one. + crate::lifecycle_trace::life_spawn(id.raw(), link_parent.map_or(0, |p| p.raw())); + if let start::Timeout::After(after) = options.init_timeout { + crate::start::spawn_init_timer_async(&cell, child_handle.clone(), after); } - // Recorded from here: the start_linked link is the model's initial one. - crate::lifecycle_trace::life_spawn(id.raw(), parent_id); let cleanup_token = cancellation_token.clone(); let cleanup = crate::actor_cleanup::ActorCleanup::new( @@ -104,7 +363,9 @@ impl ActorRef { ) .with_mailbox_failures(actor_ref.shared.sender.failures()) .with_replies(actor_ref.shared.replies.clone()) - .with_jobs(actor_ref.shared.jobs.clone()); + .with_jobs(actor_ref.shared.jobs.clone()) + .with_startup(cell, link_parent) + .with_name(committed.name.take()); let startup = crate::actor_cleanup::ActorStartup::new( (actor, ctx, rx, cancellation_token), cleanup, @@ -131,13 +392,21 @@ impl ActorRef { // running then does not undo death and is reported unfinished. if job_owner.jobs.running() > 0 { let timeout = job_owner - .managed - .as_ref() + .managed() .map_or(DEFAULT_STOP_DEADLINE, |scope| scope.cleanup_timeout()); let remaining = job_owner.replies.cleanup_remaining(timeout); let unfinished = job_owner.jobs.join(remaining).await; crate::reply::jobs::report_unfinished(job_owner.id, unfinished); } + // An unowned child's owned work is done (`Start.tla` WorkDoneC; + // a managed one's is its scope's completion). + if job_owner.managed().is_none() { + let id = job_owner.id.raw(); + let c = crate::lifecycle_trace::start_child_of(id); + crate::lifecycle_trace::start_emit(c, || { + crate::lifecycle_trace::StartEvent::new("SWorkDone", c) + }); + } drop(job_owner); let _ = cleaned_tx.send(true); }; @@ -145,14 +414,19 @@ impl ActorRef { if let Some(lease) = lease { let scope = actor_ref .shared - .managed - .as_ref() + .managed() .expect("managed lease without scope"); - scope.dispatch_actor(crate::system::Tracked { + // Committed: a dispatch failure (the group dropped mid-shutdown, + // its worker gone) is not a stage-1 refusal. The child is killed + // before started() and its start answers `InitExited { Killed }`. + if let Err(future) = scope.dispatch_actor(crate::system::Tracked { future: inner_future, lease, - })?; - return Ok(actor_ref); + }) { + tracing::error!(actor = ?id, "actor dispatch failed after commit; killing the child"); + crate::start::run_killed(&child_handle, || crate::start::run_inline(future)); + } + return Ok((actor_ref, monitor_ref)); } match backend { @@ -165,11 +439,24 @@ impl ActorRef { let _handle = rt::spawn_blocking(move || rt::block_on(inner_future)); } Backend::Thread => { - let _handle = threads::spawn(move || threads::block_on(inner_future)); + let mut builder = std::thread::Builder::new(); + if let Some(bytes) = options.stack_size { + builder = builder.stack_size(bytes); + } + if let Some(label) = &options.label { + builder = builder.name(label.to_string()); + } + crate::start::spawn_thread_or_kill( + builder, + &child_handle, + inner_future, + threads::block_on, + crate::start::run_inline, + ); } } - Ok(actor_ref) + Ok((actor_ref, monitor_ref)) } } @@ -366,64 +653,19 @@ pub(super) async fn run_actor( owned.finish(false); } -pub(super) async fn run_actor_lifecycle( +/// The message loop, its own future: the start phase's state and code stay +/// out of each message's poll (as compact as 0.8's loop). +#[allow(clippy::too_many_arguments)] +async fn message_loop( actor: &mut A, - ctx: Context, + ctx: &Context, rx: &mut mpsc::Receiver + Send>>, cleanup: &mut crate::actor_cleanup::ActorCleanup, - cancellation_token: StopToken, + cancellation_token: &StopToken, + request: &crate::link::LinkedExitReason, + trace_id: u64, + continuation: Option + Send>>, ) { - let _monitor_cleanup = MonitorCleanup(ctx.shared.monitors.clone()); - let request = ctx.shared.linked_reason.clone(); - let trace_id = ctx.shared.id.raw(); - if let Some(scope) = &ctx.shared.managed { - if !scope.can_start() { - scope.stop(ExitReason::Shutdown(None)); - return; - } - } - let start_result = AssertUnwindSafe(async { - fail_point!("actor.before_started", async); - DeathFirst::new( - crate::lifecycle_trace::in_actor(ctx.shared.id.raw(), Box::pin(actor.started(&ctx))), - &mut *rx, - &mut *cleanup, - None, - ) - .await - .unwrap_or_else(|panic| std::panic::resume_unwind(panic)) - }) - .catch_unwind() - .await; - if let Err(panic) = start_result { - let msg = panic_message(&*panic); - crate::actor_cleanup::log_safely(|| { - tracing::error!("Panic in started() callback: {msg}"); - }); - // No stopped() after a failed start (L5). - crate::lifecycle_trace::life_section(trace_id, || { - crate::lifecycle_trace::life_emit(trace_id, || { - crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("started") - }); - link::lock_request(&request) - .panic(ExitReason::Panic(format!("panic in started(): {msg}"))); - }); - cancellation_token.cancel(); - return; - } - // Phase 0 ends: resolve signals held during started() with the trap - // flag read now. - if link::lock_request(&request).end_starting(ctx.shared.trap_exit.load(Ordering::Acquire)) { - cancellation_token.cancel(); - } - - fail_point!("actor.after_started_before_ready", async); - if let Some(scope) = &ctx.shared.managed { - if !scope.mark_ready() { - scope.stop(ExitReason::Shutdown(None)); - } - } - // One cancellation future for the whole loop: polled only when the // mailbox has nothing, so a busy actor registers no waiter per message. let mut cancelled = pin!(cancellation_token.cancelled()); @@ -431,131 +673,299 @@ pub(super) async fn run_actor_lifecycle( // after stop must not slip through an earlier snapshot. A rejected receive // stays in the mailbox wrapper until the death point answers its caller. // The fast check is one atomic load; only parked loops register a waiter. - loop { - let msg = { - let recv = pin!(future::poll_fn(|cx| { - // Stop and closed-mailbox both take the normal lifecycle exit, - // just like the cancellation arm below. Check after the - // mailbox acquire and retain a rejected item for shutdown. - // The receive, check and trace event are one step - // of the lifecycle trace: a recorded stop is seen here. - crate::lifecycle_trace::life_section(trace_id, || { - #[cfg(all(test, not(feature = "lifecycle-trace"), not(spawned_sim)))] - stop_receive_probe::before_receive(); - let received = rx.poll_recv(cx); - let result = cancellation_token.after_receive(received, |item| { - rx.hold_for_shutdown(item); - }); - #[cfg(feature = "lifecycle-trace")] - if let std::task::Poll::Ready(Some(envelope)) = &result { - crate::lifecycle_trace::life_emit(trace_id, || { - crate::lifecycle_trace::LifeEvent::new("Dispatch", trace_id) - .k(envelope.trace_kind()) - }); - } - result - }) - })); - let mut recv = recv; - future::poll_fn(|cx| match recv.as_mut().poll(cx) { - std::task::Poll::Pending => match cancelled.as_mut().poll(cx) { - std::task::Poll::Ready(()) => std::task::Poll::Ready(None), - std::task::Poll::Pending => std::task::Poll::Pending, - }, - ready => ready, - }) - .await - }; - match msg { - Some(envelope) => { - // Simulator turn budget (rank 8b); compiled only under the cfg. - #[cfg(spawned_sim)] - let _sim_turn = rt::sim::turn(std::any::type_name::(), envelope.message_type()); - #[cfg(feature = "diagnostics")] - let _turn = ctx - .shared - .managed - .as_ref() - .map(|scope| scope.begin_turn(envelope.message_type())); - #[cfg(feature = "metrics")] - let mut envelope = envelope; + // One turn (a mailbox entry or the continuation): true when the actor + // must stop (its turn panicked). The loop's body, also run once for + // the continuation before the loop, so the loop itself is unchanged. + macro_rules! turn { + ($envelope:expr) => {{ + let envelope = $envelope; - #[cfg(feature = "metrics")] - let observation = envelope - .observation() - .map(|observation| observation.begin()); - ctx.shared.replies.begin_turn(); - // The handler's future is built inside the guarded future: a - // manual `Handler` may panic before it returns one, and that - // is a handler panic like any other (threads mode catches the - // same call). The envelope outlives the turn and lends it the - // request's slot. - let mut envelope = envelope; - let result = match dispatch(&mut *envelope, &mut *actor, &ctx) { - // Dispatch completed the turn synchronously. The caller - // hold is only needed if an unfinished future is dropped; - // turn accounting and observation bracket either branch. - Ok((None, _)) => Ok(()), - Ok((Some(turn), caller)) => { - let mut turn = - Some(crate::lifecycle_trace::in_actor(ctx.shared.id.raw(), turn)); - // Most turns finish in their first poll, which runs - // here, with no await point before it returns: only - // a turn that stays pending is guarded by - // `DeathFirst` against the runtime dropping it. - let first = future::poll_fn(|cx| { - std::task::Poll::Ready(poll_caught( - turn.as_mut().expect("polled once"), - cx, - )) - }) - .await; - match first { - Ok(std::task::Poll::Ready(())) => { - drop(turn); - Ok(()) - } - Ok(std::task::Poll::Pending) => { - let turn = turn.take().expect("pending turn"); - DeathFirst::pending(turn, &mut *rx, &mut *cleanup, caller).await - } - Err(panic) => { - drop_panicked(turn.take()); - Err(panic) - } + // Simulator turn budget (rank 8b); compiled only under the cfg. + #[cfg(spawned_sim)] + let _sim_turn = rt::sim::turn(std::any::type_name::(), envelope.message_type()); + #[cfg(feature = "diagnostics")] + let _turn = ctx + .shared + .managed() + .map(|scope| scope.begin_turn(envelope.message_type())); + #[cfg(feature = "metrics")] + let mut envelope = envelope; + + #[cfg(feature = "metrics")] + let observation = envelope + .observation() + .map(|observation| observation.begin()); + ctx.shared.replies.begin_turn(); + // The handler's future is built inside the guarded future: a + // manual `Handler` may panic before it returns one, and that + // is a handler panic like any other (threads mode catches the + // same call). The envelope outlives the turn and lends it the + // request's slot. + let mut envelope = envelope; + let result = match dispatch(&mut *envelope, &mut *actor, &ctx) { + // Dispatch completed the turn synchronously. The caller + // hold is only needed if an unfinished future is dropped; + // turn accounting and observation bracket either branch. + Ok((None, _)) => Ok(()), + Ok((Some(turn), caller)) => { + let mut turn = + Some(crate::lifecycle_trace::in_actor(ctx.shared.id.raw(), turn)); + // Most turns finish in their first poll, which runs + // here, with no await point before it returns: only + // a turn that stays pending is guarded by + // `DeathFirst` against the runtime dropping it. + let first = future::poll_fn(|cx| { + std::task::Poll::Ready(poll_caught(turn.as_mut().expect("polled once"), cx)) + }) + .await; + match first { + Ok(std::task::Poll::Ready(())) => { + drop(turn); + Ok(()) + } + Ok(std::task::Poll::Pending) => { + let turn = turn.take().expect("pending turn"); + DeathFirst::pending(turn, &mut *rx, &mut *cleanup, caller).await + } + Err(panic) => { + drop_panicked(turn.take()); + Err(panic) } } - Err(panic) => Err(panic), - }; - ctx.shared.replies.end_turn(result.is_err()); - #[cfg(feature = "diagnostics")] - drop(_turn); - #[cfg(feature = "metrics")] - if let Some(observation) = observation { - observation.finish(result.is_err()); } - if let Err(panic) = result { - let msg = panic_message(&*panic); - crate::actor_cleanup::log_safely(|| { - tracing::error!("Panic in message handler: {msg}"); + Err(panic) => Err(panic), + }; + ctx.shared.replies.end_turn(result.is_err()); + #[cfg(feature = "diagnostics")] + drop(_turn); + #[cfg(feature = "metrics")] + if let Some(observation) = observation { + observation.finish(result.is_err()); + } + if let Err(panic) = result { + let msg = panic_message(&*panic); + crate::actor_cleanup::log_safely(|| { + tracing::error!("Panic in message handler: {msg}"); + }); + // A panic begins exit, or replaces the requested reason + // unless a kill was received. + crate::lifecycle_trace::life_section(trace_id, || { + crate::lifecycle_trace::life_emit(trace_id, || { + crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("turn") }); - // A panic begins exit, or replaces the requested reason - // unless a kill was received. + link::lock_request(&request) + .panic(ExitReason::Panic(format!("panic in handler: {msg}"))); + }); + true + } else { + false + } + }}; + } + { + // The continuation is the first turn, before any mailbox entry; one + // copy of the turn body keeps the message loop as compact as 0.8's. + let mut continuation = continuation; + loop { + let msg = if let Some(envelope) = continuation.take() { + Some(envelope) + } else { + let recv = pin!(future::poll_fn(|cx| { + // Stop and closed-mailbox both take the normal lifecycle exit, + // just like the cancellation arm below. Check after the + // mailbox acquire and retain a rejected item for shutdown. + // The receive, check and trace event are one step + // of the lifecycle trace: a recorded stop is seen here. crate::lifecycle_trace::life_section(trace_id, || { - crate::lifecycle_trace::life_emit(trace_id, || { - crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("turn") + #[cfg(all(test, not(feature = "lifecycle-trace"), not(spawned_sim)))] + stop_receive_probe::before_receive(); + let received = rx.poll_recv(cx); + let result = cancellation_token.after_receive(received, |item| { + rx.hold_for_shutdown(item); }); - link::lock_request(&request) - .panic(ExitReason::Panic(format!("panic in handler: {msg}"))); - }); - break; + #[cfg(feature = "lifecycle-trace")] + if let std::task::Poll::Ready(Some(envelope)) = &result { + crate::lifecycle_trace::life_emit(trace_id, || { + crate::lifecycle_trace::LifeEvent::new("Dispatch", trace_id) + .k(envelope.trace_kind()) + }); + crate::lifecycle_trace::start_dispatch(trace_id, envelope.trace_kind()); + } + result + }) + })); + let mut recv = recv; + future::poll_fn(|cx| match recv.as_mut().poll(cx) { + std::task::Poll::Pending => match cancelled.as_mut().poll(cx) { + std::task::Poll::Ready(()) => std::task::Poll::Ready(None), + std::task::Poll::Pending => std::task::Poll::Pending, + }, + ready => ready, + }) + .await + }; + match msg { + Some(envelope) => { + if turn!(envelope) { + break; + } } + None => break, } - None => break, } } +} + +pub(super) async fn run_actor_lifecycle( + actor: &mut A, + ctx: Context, + rx: &mut mpsc::Receiver + Send>>, + cleanup: &mut crate::actor_cleanup::ActorCleanup, + cancellation_token: StopToken, +) { + let _monitor_cleanup = MonitorCleanup(ctx.shared.monitors.clone()); + let request = ctx.shared.linked_reason.clone(); + let trace_id = ctx.shared.id.raw(); + let cell = ctx.shared.signal.startup.clone(); + // A kill or the init timer before started() began (`KilledEarly`), or + // a managed admission already closed: started() is not run. + let (killed, closed) = crate::lifecycle_trace::start_section(|| { + let killed = link::lock_request(&request).killed_early() + || cell.commit() == crate::start::Commit::TimedOut; + let closed = ctx + .shared + .managed() + .is_some_and(|scope| !scope.admission_open()); + if !killed { + // A closed admission skips started() but resolves as if it + // returned ready (row 9, or row 3 for a scope's stop): the + // model's StartedBegin and StartedEnd. + let c = cell.trace_id(); + crate::lifecycle_trace::start_emit(c, || { + crate::lifecycle_trace::StartEvent::new("SStartedBegin", c) + }); + } + (killed, closed) + }); + let (returned, continuation) = if killed { + (crate::start::Returned::NotBegun, None) + } else if closed { + (crate::start::Returned::Ready, None) + } else { + let mut init: Option> = None; + let start_result = AssertUnwindSafe(async { + fail_point!("actor.before_started", async); + let started = actor.started(&ctx); + let slot = &mut init; + DeathFirst::new( + crate::lifecycle_trace::in_actor( + ctx.shared.id.raw(), + Box::pin(async move { + *slot = Some(started.await.into()); + }), + ), + &mut *rx, + &mut *cleanup, + None, + ) + .await + .unwrap_or_else(|panic| std::panic::resume_unwind(panic)) + }) + .catch_unwind() + .await; + match start_result { + Err(panic) => { + let msg = panic_message(&*panic); + crate::actor_cleanup::log_safely(|| { + tracing::error!("Panic in started() callback: {msg}"); + }); + (crate::start::Returned::Panic(msg), None) + } + Ok(()) => init.map_or((crate::start::Returned::Ready, None), |init| { + init.into_returned() + }), + } + }; + fail_point!("actor.after_started_before_ready", async); + // The startup decision (design 2.3) and, when ready, the readiness + // commit (5.1): under the child's admission gate, its exit-request lock + // and the cell lock, one step. No failed start runs stopped(). + let trapping = ctx.shared.trap_exit.load(Ordering::Acquire); + let continued = continuation.is_some(); + let decide = |open: bool| { + let mut req = link::lock_request(&request); + cell.decide(&returned, continued, trapping, |timed_out| { + let row = req.resolve_start(trapping, timed_out, &returned, open); + let cause = match (&row, &returned) { + (crate::start::Row::Error, crate::start::Returned::Error(cause)) => { + Some(cause.clone()) + } + _ => None, + }; + (row, req.reason().cloned(), cause) + }) + }; + let row = crate::lifecycle_trace::start_section(|| { + crate::lifecycle_trace::life_section(trace_id, || { + let row = match ctx.shared.managed() { + Some(scope) => scope.ready_commit(|open| { + let row = decide(open); + let ready = row == crate::start::Row::Ready; + (row, ready) + }), + None => decide(true), + }; + if row == crate::start::Row::Panic { + crate::lifecycle_trace::life_emit(trace_id, || { + crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("started") + }); + } + row + }) + }); + if row != crate::start::Row::Ready { + if let (crate::start::Returned::Error(cause), false) = + (&returned, row == crate::start::Row::Error) + { + let id = ctx.shared.id; + crate::actor_cleanup::log_safely(|| { + tracing::debug!(actor = %id, cause, ?row, "init error superseded by a higher-precedence exit"); + }); + } + cancellation_token.cancel(); + return; + } + // A trapping child's held signals are queued; a non-trapping child's + // held Normal ones were dropped. A continuation runs first (L10); a stop + // observed before it begins discards it, as queued messages. + let continuation = match continuation { + Some(continuation) if cancellation_token.is_cancelled() => { + rx.hold_for_shutdown(continuation); + None + } + Some(continuation) => { + let c = cell.trace_id(); + crate::lifecycle_trace::start_emit(c, || { + crate::lifecycle_trace::StartEvent::new("SContinue", c) + }); + Some(continuation) + } + None => None, + }; + + message_loop( + &mut *actor, + &ctx, + &mut *rx, + &mut *cleanup, + &cancellation_token, + &request, + trace_id, + continuation, + ) + .await; - if let Some(scope) = &ctx.shared.managed { + if let Some(scope) = ctx.shared.managed() { scope.end_running(); } cancellation_token.cancel(); diff --git a/concurrency/src/tasks/actor/monitor_cleanup_regressions.rs b/concurrency/src/tasks/actor/monitor_cleanup_regressions.rs index 9df8703e3..7595390b2 100644 --- a/concurrency/src/tasks/actor/monitor_cleanup_regressions.rs +++ b/concurrency/src/tasks/actor/monitor_cleanup_regressions.rs @@ -21,8 +21,8 @@ fn closed_monitor_table_rejects_late_registration() { move || { let rt = rt::Runtime::new().unwrap(); let _entered = rt.enter(); - let observer = Idle.start(); - let target = Idle.start(); + let observer = Idle.spawn(); + let target = Idle.spawn(); // Model cleanup during an unexpected unwind before token cancellation. drop(MonitorCleanup(observer.shared.monitors.clone())); let child = target.child_handle(); @@ -52,8 +52,8 @@ fn released(rt: &rt::Runtime, done: impl Fn() -> bool) -> bool { fn check_cleanup(stop_owner: bool) { let rt = rt::Runtime::new().unwrap(); let _entered = rt.enter(); - let target = Idle.start(); - let observer = Idle.start(); + let target = Idle.spawn(); + let observer = Idle.spawn(); let child = target.child_handle(); let waiter = child.lifetime_probe(); let monitor = observer.context().monitor(&child); @@ -109,8 +109,8 @@ fn observer_cancels_its_monitors_before_publishing_its_exit() { .build() .unwrap(); rt.block_on(async { - let target = Idle.start(); - let observer = Idle.start(); + let target = Idle.spawn(); + let observer = Idle.spawn(); let child = target.child_handle(); let waiter = child.lifetime_probe(); observer.context().monitor(&child); diff --git a/concurrency/src/tasks/actor/shared_state_release.rs b/concurrency/src/tasks/actor/shared_state_release.rs index 224fa339b..da37bc466 100644 --- a/concurrency/src/tasks/actor/shared_state_release.rs +++ b/concurrency/src/tasks/actor/shared_state_release.rs @@ -41,7 +41,7 @@ fn freed_after_stop_and_last_handle_dropped() { concat!(module_path!(), "::freed_after_stop_and_last_handle_dropped"), move || { rt::Runtime::new().unwrap().block_on(async { - let actor = Plain.start(); + let actor = Plain.spawn(); let weak = actor.shared_weak(); let extra = (actor.clone(), actor.context(), actor.child_handle()); actor.context().stop(); @@ -63,7 +63,7 @@ fn freed_when_context_was_stored_in_actor_state() { ), move || { rt::Runtime::new().unwrap().block_on(async { - let actor = Holder(None).start(); + let actor = Holder(None).spawn(); let weak = actor.shared_weak(); actor.context().stop(); actor.join().await; @@ -80,14 +80,14 @@ fn freed_with_links_and_monitors() { concat!(module_path!(), "::freed_with_links_and_monitors"), move || { rt::Runtime::new().unwrap().block_on(async { - let a = Plain.start(); - let b = Plain.start(); - let watcher = Plain.start(); + let a = Plain.spawn(); + let b = Plain.spawn(); + let watcher = Plain.spawn(); a.context().trap_exit(true); a.context().link(&b.child_handle()); let _monitor = watcher.context().monitor(&a.child_handle()); watcher.context().monitor(&b.child_handle()); - let child = Plain.start_linked(&a.context()); + let child = Plain.start_link(&a.context()).actor_ref().unwrap(); let weaks = [ a.shared_weak(), b.shared_weak(), diff --git a/concurrency/src/tasks/actor/start.rs b/concurrency/src/tasks/actor/start.rs new file mode 100644 index 000000000..435aa0d4f --- /dev/null +++ b/concurrency/src/tasks/actor/start.rs @@ -0,0 +1,590 @@ +//! Unified start (0.9, design `docs/design/unified-start.md`): the start +//! verbs, `StartOptions`, typed names, `Init` and `Starting`. +use super::*; +pub(super) use crate::start::Timeout; +use crate::start::{Observed, StartError, StartOutcome, StartupCell}; +use std::borrow::Cow; +use std::fmt::Display; +use std::marker::PhantomData; + +// --------------------------------------------------------------------------- +// Init: what started() returns +// --------------------------------------------------------------------------- + +/// What `started()` returns (OTP `init/1`'s results). `started()` may +/// return `()`, `Result<(), E>`, `Result, E>` or `Init`. +/// +/// | `Init` | OTP | The start answers | +/// |---|---|---| +/// | `ready()` (`()`, `Ok(())`) | `{ok, S}` | `Ok(Started)` | +/// | `continue_with(m)` | `{ok, S, {continue, C}}` | `Ok(Started)`; `m` is handled before any mailbox message | +/// | `error(e)` (`Err(e)`) | `{error, R}` | `InitFailed { cause }`; the actor ends `Normal` | +/// | `stop(r)` | `{stop, R}` | `InitExited { reason: r }` | +/// | `ignore()` | `ignore` | `Ok(Ignored)`; the actor ends `Normal` | +/// +/// No failed start runs `stopped()` (OTP skips `terminate/2` when `init/1` +/// fails): release resources acquired in `started()` before returning an +/// error, or hold them in RAII guards. +#[must_use] +pub struct Init { + pub(super) kind: InitKind, +} + +pub(super) enum InitKind { + Ready, + Continue(Box + Send>), + Error(String), + Stop(ExitReason), + Ignore, +} + +impl std::fmt::Debug for Init { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let kind = match &self.kind { + InitKind::Ready => "ready", + InitKind::Continue(_) => "continue", + InitKind::Error(_) => "error", + InitKind::Stop(_) => "stop", + InitKind::Ignore => "ignore", + }; + f.debug_tuple("Init").field(&kind).finish() + } +} + +impl Init { + /// Ready (`{ok, S}`). + pub fn ready() -> Self { + Self { + kind: InitKind::Ready, + } + } + /// Ready, then handle `msg` as the first turn, before any mailbox entry + /// (`{ok, S, {continue, C}}`, `handle_continue`). Readiness is committed + /// before it runs, so it is not a barrier against termination: a kill, + /// an untrapped exit signal or a supervisor's shutdown can end the actor + /// before or during it. + pub fn continue_with(msg: M) -> Self + where + A: Handler, + { + Self { + kind: InitKind::Continue(Box::new(SendEnvelope { + msg: Some(msg), + #[cfg(feature = "metrics")] + observation: None, + #[cfg(feature = "lifecycle-trace")] + from: 0, + })), + } + } + /// A failed initialisation (`{error, R}`, OTP 26 `init_fail/3`): the + /// actor ends `Normal`, the start answers `InitFailed { cause }`. + pub fn error(cause: impl Display) -> Self { + Self { + kind: InitKind::Error(cause.to_string()), + } + } + /// Stop with `reason` (`{stop, R}`): links see it. The start answers + /// `InitExited { reason }`. + pub fn stop(reason: ExitReason) -> Self { + Self { + kind: InitKind::Stop(reason), + } + } + /// Opt out (`ignore`): the actor ends `Normal`; the start answers + /// `Ok(StartOutcome::Ignored)`. + pub fn ignore() -> Self { + Self { + kind: InitKind::Ignore, + } + } + pub(super) fn into_returned( + self, + ) -> (crate::start::Returned, Option + Send>>) { + use crate::start::Returned; + match self.kind { + InitKind::Ready => (Returned::Ready, None), + InitKind::Continue(envelope) => (Returned::Ready, Some(envelope)), + InitKind::Error(cause) => (Returned::Error(cause), None), + InitKind::Stop(reason) => (Returned::Stop(reason), None), + InitKind::Ignore => (Returned::Ignore, None), + } + } +} + +impl From<()> for Init { + fn from(_: ()) -> Self { + Self::ready() + } +} + +impl From> for Init { + fn from(result: Result<(), E>) -> Self { + match result { + Ok(()) => Self::ready(), + Err(error) => Self::error(error), + } + } +} + +impl From, E>> for Init { + fn from(result: Result, E>) -> Self { + result.unwrap_or_else(Self::error) + } +} + +// --------------------------------------------------------------------------- +// Typed names +// --------------------------------------------------------------------------- + +/// A registered name for actors of type `A` (OTP `{local, Name}`), committed +/// with the start and released at publication. A `Name` cannot name an +/// `A` start. Names are process-global. +pub struct Name { + key: Cow<'static, str>, + _actor: PhantomData A>, +} + +impl Name { + /// A name usable in a `static`. + pub const fn new(key: &'static str) -> Self { + Self { + key: Cow::Borrowed(key), + _actor: PhantomData, + } + } + pub fn from_string(key: String) -> Self { + Self { + key: Cow::Owned(key), + _actor: PhantomData, + } + } + pub fn key(&self) -> &str { + &self.key + } + /// The live instance holding this name, if it is an `A`. + pub fn whereis(&self) -> Option> { + crate::registry::whereis_start::>(&self.key) + } +} + +impl Clone for Name { + fn clone(&self) -> Self { + Self { + key: self.key.clone(), + _actor: PhantomData, + } + } +} + +impl std::fmt::Debug for Name { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_tuple("Name").field(&self.key).finish() + } +} + +impl From<&'static str> for Name { + fn from(key: &'static str) -> Self { + Self::new(key) + } +} + +impl From for Name { + fn from(key: String) -> Self { + Self::from_string(key) + } +} + +// --------------------------------------------------------------------------- +// StartOptions +// --------------------------------------------------------------------------- + +/// Where an actor runs. +#[derive(Clone)] +pub(super) enum Placement { + Group(crate::system::ExecutionGroup), + Backend(Backend), +} + +impl Placement { + pub(super) fn trace_name(&self) -> &'static str { + match self { + Self::Group(_) => "G", + Self::Backend(_) => "U", + } + } +} + +/// How to start an actor of type `A`: placement, a typed name, the init +/// timeout, a label, the stack size and per-instance bounds. The last of +/// `group`/`backend`/`placement_of` wins; without one, `start_link` and +/// `start_monitor` use the parent's placement and the other verbs an +/// unmanaged `Backend::Async`. +#[non_exhaustive] +pub struct StartOptions { + pub(super) placement: Option, + pub(super) name: Option>, + pub(super) label: Option>, + pub(super) init_timeout: Timeout, + pub(super) stack_size: Option, + pub(super) max_outstanding: Option, + pub(super) max_native_jobs: Option, +} + +impl Clone for StartOptions { + fn clone(&self) -> Self { + Self { + placement: self.placement.clone(), + name: self.name.clone(), + label: self.label.clone(), + init_timeout: self.init_timeout, + stack_size: self.stack_size, + max_outstanding: self.max_outstanding, + max_native_jobs: self.max_native_jobs, + } + } +} + +impl Default for StartOptions { + fn default() -> Self { + Self::new() + } +} + +impl std::fmt::Debug for StartOptions { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("StartOptions") + .field("name", &self.name) + .field("init_timeout", &self.init_timeout) + .finish_non_exhaustive() + } +} + +impl StartOptions { + pub fn new() -> Self { + Self { + placement: None, + name: None, + label: None, + init_timeout: Timeout::Infinity, + stack_size: None, + max_outstanding: None, + max_native_jobs: None, + } + } + /// Admit the actor to an execution group of an `ActorSystem`. + pub fn group(mut self, group: &crate::system::ExecutionGroup) -> Self { + self.placement = Some(Placement::Group(group.clone())); + self + } + /// Run the actor unmanaged on `backend`. + pub fn backend(mut self, backend: Backend) -> Self { + self.placement = Some(Placement::Backend(backend)); + self + } + /// Use the placement of `ctx`'s actor (its group, or its backend). + pub fn placement_of(mut self, ctx: &Context

) -> Self { + self.placement = Some(ctx.placement()); + self + } + /// Register the actor under `name` atomically with the start; the name + /// is released at publication, before any notice or answer. + pub fn name(mut self, name: impl Into>) -> Self { + self.name = Some(name.into()); + self + } + /// A label for diagnostics: a managed actor's diagnostic name when it + /// has no registered name, and the OS thread name of a + /// `Backend::Thread` actor. An unmanaged `Async` or `Blocking` actor has + /// no place to show it, and ignores it. + pub fn label(mut self, label: impl Into>) -> Self { + self.label = Some(label.into()); + self + } + /// Bound `started()` (OTP `{timeout, T}`; default `Infinity`). At + /// expiry the start commits `InitTimeout` and the actor is killed when + /// `started()` returns (it is never interrupted). + pub fn init_timeout(mut self, timeout: impl Into) -> Self { + self.init_timeout = timeout.into(); + self + } + /// Stack size of the actor's own thread (`Backend::Thread` only). + pub fn stack_size(mut self, bytes: usize) -> Self { + self.stack_size = Some(bytes); + self + } + /// Per-instance override of `Actor::MAX_OUTSTANDING`. + pub fn max_outstanding(mut self, max: usize) -> Self { + self.max_outstanding = Some(max); + self + } + /// Per-instance override of `Actor::MAX_NATIVE_JOBS`. + pub fn max_native_jobs(mut self, max: usize) -> Self { + self.max_native_jobs = Some(max); + self + } + /// The checks every start makes before anything exists. + pub(super) fn validate(&self, placement: &Placement) -> Result<(), StartError> { + if self.init_timeout == Timeout::After(Duration::ZERO) { + return Err(StartError::InvalidOptions("init_timeout must be nonzero")); + } + if self.max_outstanding == Some(0) { + return Err(StartError::InvalidOptions( + "max_outstanding must be nonzero", + )); + } + if self.max_native_jobs == Some(0) { + return Err(StartError::InvalidOptions( + "max_native_jobs must be nonzero", + )); + } + if self.stack_size == Some(0) { + return Err(StartError::InvalidOptions("stack_size must be nonzero")); + } + if self.stack_size.is_some() && !matches!(placement, Placement::Backend(Backend::Thread)) { + return Err(StartError::InvalidOptions( + "stack_size needs an actor-owned thread (Backend::Thread)", + )); + } + Ok(()) + } +} + +impl From<&crate::system::ExecutionGroup> for StartOptions { + fn from(group: &crate::system::ExecutionGroup) -> Self { + Self::new().group(group) + } +} + +impl From for StartOptions { + fn from(backend: Backend) -> Self { + Self::new().backend(backend) + } +} + +// --------------------------------------------------------------------------- +// The verbs +// --------------------------------------------------------------------------- + +/// How a start was requested. +pub(super) enum Verb { + Spawn, + Start, + Link(lifecycle::ParentParts), + Monitor(lifecycle::ParentParts), +} + +impl Verb { + pub(super) fn trace_name(&self) -> &'static str { + match self { + Self::Spawn => "spawn", + Self::Start => "start", + Self::Link(_) => "link", + Self::Monitor(_) => "monitor", + } + } + pub(super) fn waiting(&self) -> bool { + !matches!(self, Self::Spawn) + } +} + +/// Starting an actor (OTP `proc_lib:spawn` and `gen_server:start*`). +/// Implemented for every [`Actor`]. +pub trait ActorStart: Actor { + /// Start without waiting for `started()` (`proc_lib:spawn`), unmanaged + /// on `Backend::Async`. [`ActorRef::ready`] reports the start's result + /// later. Panics if the executor cannot be constructed (as + /// `std::thread::spawn`); `with_options(..).spawn()` returns the error. + fn spawn(self) -> ActorRef { + Starter::new(self, StartOptions::new()) + .spawn() + .unwrap_or_else(|error| panic!("actor spawn failed: {error}")) + } + /// Start and wait for `started()` (`gen_server:start`): `Started`, + /// `Ignored`, or the error, which comes only after the actor's + /// publication (its state dropped, its name free). + fn start(self) -> Starting> { + Starter::new(self, StartOptions::new()).start() + } + /// Start linked to `parent`'s actor (`gen_server:start_link`), in its + /// placement unless one is given. The link exists before `started()`. + /// A failed start's `Exit` is consumed by the error for a trapping + /// parent; a non-trapping parent still exits on a non-`Normal` failure. + /// The parent's turn waits: a request from the child's `started()` to + /// its parent waits for its timeout (as OTP's `call` from `init`). + fn start_link(self, parent: &Context

) -> Starting> { + Starter::new(self, StartOptions::new()).start_link(parent) + } + /// Start monitored by `parent`'s actor (`gen_server:start_monitor`), in + /// its placement unless one is given. A failed start's `Down` is + /// consumed by the error. + fn start_monitor>( + self, + parent: &Context

, + ) -> Starting, MonitorRef)> { + Starter::new(self, StartOptions::new()).start_monitor(parent) + } + /// Start with options: `a.with_options(&group).spawn()?`, + /// `a.with_options(StartOptions::new().name("x")).start().await?`. + fn with_options(self, options: impl Into>) -> Starter { + Starter::new(self, options.into()) + } +} + +impl ActorStart for A {} + +/// An actor and its start options; choose a verb. +#[must_use = "call spawn, start, start_link or start_monitor"] +pub struct Starter { + actor: A, + options: StartOptions, +} + +impl Starter { + fn new(actor: A, options: StartOptions) -> Self { + Self { actor, options } + } + /// Start without waiting (`proc_lib:spawn_opt`). + pub fn spawn(self) -> Result, StartError> { + ActorRef::launch(self.actor, self.options, Verb::Spawn, None).map(|(actor, _)| actor) + } + /// Start and wait for `started()`. + pub fn start(self) -> Starting> { + Starting::new( + ActorRef::launch(self.actor, self.options, Verb::Start, None), + |a, _| a, + ) + } + /// Start linked to `parent`'s actor and wait for `started()`. + pub fn start_link(self, parent: &Context

) -> Starting> { + let verb = Verb::Link(parent.parent_parts()); + Starting::new( + ActorRef::launch(self.actor, self.options, verb, None), + |a, _| a, + ) + } + /// Start monitored by `parent`'s actor and wait for `started()`. + /// + /// If the returned `Starting` is dropped unanswered, the monitor stays: + /// the child's `Down` is still delivered to `parent`, carrying a + /// [`MonitorRef`](crate::MonitorRef) the caller never received. Match it + /// by `down.actor` (`Starting::actor_ref` gives the id before awaiting). + pub fn start_monitor>( + self, + parent: &Context

, + ) -> Starting, MonitorRef)> { + let verb = Verb::Monitor(parent.parent_parts()); + let parent = parent.clone(); + let arm: ArmMonitor = Box::new(move |child, monitor_ref, active, cell| { + parent.arm_start_monitor(child, monitor_ref, active, cell) + }); + Starting::new( + ActorRef::launch(self.actor, self.options, verb, Some(arm)), + |a, m| (a, m.expect("start_monitor commits a monitor")), + ) + } +} + +/// Arms the parent's monitor waiter (typed by the parent) before dispatch. +pub(super) type ArmMonitor = Box< + dyn FnOnce(ChildHandle, MonitorRef, spawned_rt::tasks::CancellationToken, Arc) + + Send, +>; + +// --------------------------------------------------------------------------- +// Starting +// --------------------------------------------------------------------------- + +/// A start whose actor already exists (stage 1 ran in the call). Awaiting it +/// reads the startup result; dropping it abandons the wait (the actor runs +/// on, and a diverted notice is handed to the parent as an ordinary one). +#[must_use = "the actor runs; await this to learn whether it started"] +pub struct Starting { + inner: Result, StartError>, +} + +struct Waiting { + actor: ActorRef, + value: Option, + guard: AbandonGuard, +} + +/// Abandons the wait if the `Starting` (or its future) is dropped +/// unanswered. +struct AbandonGuard { + cell: Option>, +} + +impl Drop for AbandonGuard { + fn drop(&mut self) { + if let Some(cell) = self.cell.take() { + cell.abandon(); + } + } +} + +impl Starting { + fn new( + launched: Result<(ActorRef, Option), StartError>, + value: impl FnOnce(ActorRef, Option) -> T, + ) -> Self { + Self { + inner: launched.map(|(actor, monitor)| { + let cell = actor.shared.signal.startup.clone(); + Waiting { + value: Some(value(actor.clone(), monitor)), + actor, + guard: AbandonGuard { cell: Some(cell) }, + } + }), + } + } + /// The unready instance, without waiting or consuming the result (`Err` + /// only for a stage-1 refusal): for wiring during initialisation, or to + /// keep control of a stuck `started()`. + pub fn actor_ref(&self) -> Result, &StartError> { + match &self.inner { + Ok(waiting) => Ok(waiting.actor.clone()), + Err(error) => Err(error), + } + } +} + +impl std::future::IntoFuture for Starting { + type Output = Result, StartError>; + type IntoFuture = Pin + Send>>; + fn into_future(self) -> Self::IntoFuture { + Box::pin(async move { + let mut waiting = self.inner?; + let outcome = observe(&waiting.actor, true).await; + // Answered: nothing to abandon. + waiting.guard.cell = None; + outcome.map(|o| o.map(|()| waiting.value.take().expect("value taken once"))) + }) + } +} + +/// Wait for the startup result of `actor` (`waiter`: the start's own wait, +/// which consumes it). +pub(super) async fn observe( + actor: &ActorRef, + waiter: bool, +) -> Result, StartError> { + let cell = actor.shared.signal.startup.clone(); + loop { + let mut changed = cell.subscribe(); + match cell.observe(waiter, || actor.child_handle()) { + Observed::Answer(answer) => return answer, + Observed::Wait(None) => { + let _ = changed.changed().await; + } + Observed::Wait(Some(deadline)) => { + let _ = futures::future::select( + Box::pin(changed.changed()), + Box::pin(rt::sleep_until(deadline)), + ) + .await; + } + } + } +} diff --git a/concurrency/src/tasks/actor/tests.rs b/concurrency/src/tasks/actor/tests.rs index ac6e18db2..f27dd351f 100644 --- a/concurrency/src/tasks/actor/tests.rs +++ b/concurrency/src/tasks/actor/tests.rs @@ -109,7 +109,7 @@ pub fn backend_async_handles_send_and_request() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let counter = Counter { count: 0 }.start(); + let counter = Counter { count: 0 }.spawn(); let result = counter.request(GetCount).await.unwrap(); assert_eq!(result, 0); @@ -141,7 +141,10 @@ pub fn backend_blocking_handles_send_and_request() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let counter = Counter { count: 0 }.start_with_backend(Backend::Blocking); + let counter = Counter { count: 0 } + .with_options(Backend::Blocking) + .spawn() + .unwrap(); let result = counter.request(GetCount).await.unwrap(); assert_eq!(result, 0); @@ -168,7 +171,10 @@ pub fn backend_thread_handles_send_and_request() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let counter = Counter { count: 0 }.start_with_backend(Backend::Thread); + let counter = Counter { count: 0 } + .with_options(Backend::Thread) + .spawn() + .unwrap(); let result = counter.request(GetCount).await.unwrap(); assert_eq!(result, 0); @@ -195,9 +201,15 @@ pub fn multiple_backends_concurrent() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let async_counter = Counter { count: 0 }.start(); - let blocking_counter = Counter { count: 100 }.start_with_backend(Backend::Blocking); - let thread_counter = Counter { count: 200 }.start_with_backend(Backend::Thread); + let async_counter = Counter { count: 0 }.spawn(); + let blocking_counter = Counter { count: 100 } + .with_options(Backend::Blocking) + .spawn() + .unwrap(); + let thread_counter = Counter { count: 200 } + .with_options(Backend::Thread) + .spawn() + .unwrap(); async_counter.request(Increment).await.unwrap(); blocking_counter.request(Increment).await.unwrap(); @@ -244,7 +256,7 @@ pub fn request_timeout() { } } - let actor = SlowActor.start(); + let actor = SlowActor.spawn(); let result = actor .request_with_timeout(SlowOp, Duration::from_millis(50)) .await; @@ -260,7 +272,7 @@ pub fn recipient_type_erasure() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let counter = Counter { count: 42 }.start(); + let counter = Counter { count: 42 }.spawn(); let recipient: Recipient = counter.recipient(); let rx = recipient @@ -384,7 +396,7 @@ pub fn thread_backend_join_does_not_block_runtime() { runtime.block_on(async move { let (held, release) = HeldThread::new(); let released = held.released.clone(); - let actor = held.start_with_backend(Backend::Thread); + let actor = held.with_options(Backend::Thread).spawn().unwrap(); actor.send(StopSlow).unwrap(); let _releaser = rt::spawn(async move { let _ = release.send(()); @@ -408,7 +420,7 @@ pub fn multiple_join_callers_all_notified() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let actor = SlowShutdownActor.start(); + let actor = SlowShutdownActor.spawn(); let actor_clone1 = actor.clone(); let actor_clone2 = actor.clone(); @@ -524,9 +536,9 @@ pub fn badly_behaved_thread_non_blocking() { .build() .unwrap(); runtime.block_on(async move { - let badboy = BadlyBehavedTask.start(); + let badboy = BadlyBehavedTask.spawn(); badboy.send(DoBlock).unwrap(); - let goodboy = WellBehavedTask { count: 0 }.start(); + let goodboy = WellBehavedTask { count: 0 }.spawn(); goodboy.send(IncrementWell).unwrap(); rt::sleep(Duration::from_secs(1)).await; let count = goodboy.request(GetCount).await.unwrap(); @@ -550,7 +562,7 @@ fn blocked_handler_does_not_stall_runtime(backend: Backend) { runtime.block_on(async move { let (held, release) = HeldThread::new(); let entered = held.entered.clone(); - let badboy = held.start_with_backend(backend); + let badboy = held.with_options(backend).spawn().unwrap(); badboy.send(DoBlock).unwrap(); let entered = crate::test_wait::wait_for_async( || async { entered.load(atomic::Ordering::SeqCst) }, @@ -559,7 +571,7 @@ fn blocked_handler_does_not_stall_runtime(backend: Backend) { .await; assert!(entered, "blocking handler never started"); - let goodboy = WellBehavedTask { count: 0 }.start(); + let goodboy = WellBehavedTask { count: 0 }.spawn(); goodboy.send(IncrementWell).unwrap(); let count = crate::test_wait::wait_for_async( || goodboy.request(GetCount), @@ -615,7 +627,7 @@ pub fn panic_in_started_stops_actor() { } } - let actor = PanicOnStart.start(); + let actor = PanicOnStart.spawn(); actor.join().await; let result = actor.send(Ping); assert!(result.is_err()); @@ -664,7 +676,7 @@ pub fn panic_in_handler_stops_actor() { } } - let actor = PanicOnMsg.start(); + let actor = PanicOnMsg.spawn(); actor.send(Explode).unwrap(); actor.join().await; let result = actor.request(Check).await; @@ -714,7 +726,7 @@ pub fn panic_building_the_handler_future_is_a_handler_panic() { } let stopped = Arc::new(atomic::AtomicBool::new(false)); - let actor = Eager(stopped.clone()).start(); + let actor = Eager(stopped.clone()).spawn(); let reply = actor.request(Explode).await; let reason = actor.wait_exit().await; let ExitReason::Panic(message) = &reason else { @@ -763,7 +775,7 @@ pub fn panic_in_stopped_still_completes() { } } - let actor = PanicOnStop.start(); + let actor = PanicOnStop.spawn(); actor.send(StopMe).unwrap(); actor.join().await; }); @@ -778,7 +790,7 @@ pub fn send_message_on_delivers() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let counter = Counter { count: 0 }.start(); + let counter = Counter { count: 0 }.spawn(); let ctx = counter.context(); send_message_on(ctx, rt::sleep(Duration::from_millis(10)), Increment); let count = crate::test_wait::wait_for_async( @@ -800,7 +812,7 @@ pub fn send_message_on_cancelled() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let counter = Counter { count: 0 }.start(); + let counter = Counter { count: 0 }.spawn(); let ctx = counter.context(); send_message_on(ctx, rt::sleep(Duration::from_millis(200)), Increment); // Stop actor before the future resolves @@ -821,7 +833,7 @@ pub fn exit_reason_normal_on_clean_stop() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let actor = Counter { count: 0 }.start(); + let actor = Counter { count: 0 }.spawn(); actor.request(StopCounter).await.unwrap(); let reason = actor.wait_exit().await; assert!(matches!(reason, ExitReason::Normal)); @@ -858,7 +870,7 @@ pub fn exit_reason_panic_in_started() { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let actor = PanicStart.start(); + let actor = PanicStart.spawn(); let reason = actor.wait_exit().await; assert!( matches!(reason, ExitReason::Panic(ref msg) if msg.contains("boom in started")) @@ -894,7 +906,7 @@ pub fn exit_reason_panic_in_handler() { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let actor = PanicHandler.start(); + let actor = PanicHandler.spawn(); let _ = actor.send(Explode); let reason = actor.wait_exit().await; assert!( @@ -915,7 +927,7 @@ pub fn exit_reason_poll_returns_none_while_running() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let actor = Counter { count: 0 }.start(); + let actor = Counter { count: 0 }.spawn(); assert!(actor.exit_reason().is_none()); actor.request(StopCounter).await.unwrap(); actor.join().await; @@ -1023,13 +1035,13 @@ pub fn monitor_running_actor_delivers_down_on_exit() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); let target_handle = target.child_handle(); let watcher = Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start(); + .spawn(); let monitor_ref = watcher.request(StartMonitor(target_handle)).await.unwrap(); @@ -1061,13 +1073,13 @@ pub fn demonitor_flushes_a_queued_down() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); target.request(StopCounter).await.unwrap(); target.join().await; let watcher = Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start(); + .spawn(); watcher .request(MonitorThenDemonitor(target.child_handle())) .await @@ -1090,7 +1102,7 @@ pub fn monitor_already_dead_actor_delivers_down_immediately() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); target.request(StopCounter).await.unwrap(); target.join().await; let target_handle = target.child_handle(); @@ -1098,7 +1110,7 @@ pub fn monitor_already_dead_actor_delivers_down_immediately() { let watcher = Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start(); + .spawn(); let _ = watcher.request(StartMonitor(target_handle)).await.unwrap(); @@ -1124,13 +1136,13 @@ pub fn demonitor_before_target_dies_suppresses_down() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); let target_handle = target.child_handle(); let watcher = Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start(); + .spawn(); let monitor_ref = watcher.request(StartMonitor(target_handle)).await.unwrap(); watcher.request(CallDemonitor(monitor_ref)).await.unwrap(); @@ -1157,13 +1169,13 @@ pub fn multiple_monitors_each_get_own_ref_and_down() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); let target_handle = target.child_handle(); let watcher = Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start(); + .spawn(); let r1 = watcher .request(StartMonitor(target_handle.clone())) @@ -1223,11 +1235,11 @@ pub fn monitor_table_is_cleaned_up_after_target_dies() { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); let watcher = Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start(); + .spawn(); let _ = watcher .request(StartMonitor(target.child_handle())) @@ -1278,13 +1290,13 @@ pub fn monitor_observes_panic_reason() { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let target = PanicMe.start(); + let target = PanicMe.spawn(); let target_handle = target.child_handle(); let watcher = Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start(); + .spawn(); let _ = watcher.request(StartMonitor(target_handle)).await.unwrap(); let _ = target.send(PanicMsg); @@ -1316,13 +1328,13 @@ pub fn monitoring_actor_stops_before_target_does_not_panic() { // watcher exits cleanly. let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); let target_handle = target.child_handle(); let watcher = Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start(); + .spawn(); let _ = watcher.request(StartMonitor(target_handle)).await.unwrap(); @@ -1396,7 +1408,7 @@ fn make_trapper(trap: bool) -> ActorRef { exits: Arc::new(Mutex::new(Vec::new())), trap, } - .start() + .spawn() } #[test] @@ -1507,7 +1519,7 @@ pub fn link_normal_exit_not_propagated_to_non_trapping_peer() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let a = Counter { count: 0 }.start(); + let a = Counter { count: 0 }.spawn(); let b = make_trapper(false); b.request(LinkTo(a.child_handle())).await.unwrap(); @@ -1591,7 +1603,7 @@ pub fn unlink_prevents_signal_delivery() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let a = Counter { count: 0 }.start(); + let a = Counter { count: 0 }.spawn(); let b = make_trapper(true); b.request(LinkTo(a.child_handle())).await.unwrap(); @@ -1652,7 +1664,7 @@ pub fn duplicate_link_is_idempotent() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let a = Counter { count: 0 }.start(); + let a = Counter { count: 0 }.spawn(); let b = make_trapper(true); b.request(LinkTo(a.child_handle())).await.unwrap(); b.request(LinkTo(a.child_handle())).await.unwrap(); // duplicate @@ -1843,8 +1855,8 @@ pub fn start_linked_links_atomically() { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let parent = Parent.start(); - let child = Child.start_linked(&parent.context()); + let parent = Parent.spawn(); + let child = Child.start_link(&parent.context()).actor_ref().unwrap(); // Parent panics — child should die too let _ = parent.send(PanicParent); diff --git a/concurrency/src/tasks/jobs.rs b/concurrency/src/tasks/jobs.rs index fa83ae99a..c0138cf48 100644 --- a/concurrency/src/tasks/jobs.rs +++ b/concurrency/src/tasks/jobs.rs @@ -609,7 +609,7 @@ impl Context { runner: None, }; #[cfg(not(spawned_sim))] - let handle = match &self.shared.managed { + let handle = match self.shared.managed() { Some(scope) => Some(scope.handle.clone()), None => spawned_rt::tasks::Handle::try_current().ok(), }; @@ -645,7 +645,7 @@ impl Context { opts: &JobOptions, native: &mut NativeSlot, ) -> bool { - let lease = match &self.shared.managed { + let lease = match self.shared.managed() { Some(scope) => match scope.job_lease() { Ok(lease) => Some(lease), Err(_) => return false, @@ -1077,7 +1077,7 @@ mod tests { "::pipe_reply_retires_the_external_token_hold" ), async { - let owner = Owner.start(); + let owner = Owner.spawn(); let ctx = owner.context(); let registry = ctx.shared.replies.clone(); let (mut handle, receiver) = @@ -1125,7 +1125,7 @@ mod tests { #[tokio::test] async fn refused_job_payload_panics_are_contained() { - let owner = Owner.start(); + let owner = Owner.spawn(); let ctx = owner.context(); let failures = ctx.shared.sender.failures(); for finish in [false, true] { @@ -1168,7 +1168,7 @@ mod tests { #[tokio::test] async fn held_job_result_panics_do_not_interrupt_cancellation() { for action in ["cancel", "deadline", "replace", "owner_exit"] { - let owner = Owner.start(); + let owner = Owner.spawn(); let ctx = owner.context(); let failures = ctx.shared.sender.failures(); let set = crate::reply::jobs::JobSet::new(owner.id(), 1); @@ -1243,7 +1243,7 @@ mod tests { crate::test_wait::within( concat!(module_path!(), "::a_refused_emit_returns_its_credit_once"), async move { - let owner = Owner.start(); + let owner = Owner.spawn(); for credits in [1, u16::MAX] { let state = JobState::new(owner.id(), credits); let job = JobCtx { @@ -1273,7 +1273,7 @@ mod tests { "::cancel_on_caller_gone_on_a_plain_job_is_a_bug" ), async move { - let owner = Owner.start(); + let owner = Owner.spawn(); let ctx = owner.context(); let refused = std::panic::catch_unwind(AssertUnwindSafe(|| { ctx.reserve_blocking_job(JobOptions::new().cancel_on_caller_gone()) diff --git a/concurrency/src/tasks/mod.rs b/concurrency/src/tasks/mod.rs index 074d221d6..aea6e0e04 100644 --- a/concurrency/src/tasks/mod.rs +++ b/concurrency/src/tasks/mod.rs @@ -15,10 +15,11 @@ pub use crate::reply::{ ReplyFuture, ReplyReceiver, ReplyTo, }; pub use crate::response::Response; +pub use crate::start::{StartError, StartOutcome, Timeout}; pub use crate::Instant; pub use actor::{ - request, send_message_on, Actor, ActorRef, ActorStart, Backend, Context, Handler, Receiver, - Recipient, DEFAULT_REQUEST_TIMEOUT, + request, send_message_on, Actor, ActorRef, ActorStart, Backend, Context, Handler, Init, Name, + Receiver, Recipient, StartOptions, Starter, Starting, DEFAULT_REQUEST_TIMEOUT, }; pub use actor::{BlockingJobPermit, JobCtx, JobPermit, DEFAULT_STOP_DEADLINE}; pub use stream::spawn_listener; diff --git a/concurrency/src/tasks/stream_tests.rs b/concurrency/src/tasks/stream_tests.rs index 98b411dd8..f7e6a6c21 100644 --- a/concurrency/src/tasks/stream_tests.rs +++ b/concurrency/src/tasks/stream_tests.rs @@ -87,7 +87,7 @@ pub fn test_sum_numbers_from_stream() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let summatory = Summatory::new(0).start(); + let summatory = Summatory::new(0).spawn(); let stream = stream::iter(vec![1u16, 2, 3, 4, 5].into_iter().map(Ok::)); let ctx = summatory.context(); @@ -115,7 +115,7 @@ pub fn test_sum_numbers_from_channel() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let summatory = Summatory::new(0).start(); + let summatory = Summatory::new(0).spawn(); let (tx, rx) = spawned_rt::tasks::mpsc::channel::>(); spawned_rt::tasks::spawn(async move { @@ -150,7 +150,7 @@ pub fn test_sum_numbers_from_broadcast_channel() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let summatory = Summatory::new(0).start(); + let summatory = Summatory::new(0).spawn(); let (tx, rx) = tokio::sync::broadcast::channel::(5); spawned_rt::tasks::spawn(async move { @@ -189,7 +189,7 @@ pub fn test_stream_cancellation() { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let summatory = Summatory::new(0).start(); + let summatory = Summatory::new(0).spawn(); let (tx, rx) = spawned_rt::tasks::mpsc::channel::>(); spawned_rt::tasks::spawn(async move { @@ -229,7 +229,7 @@ pub fn test_halting_on_stream_error() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let summatory = Summatory::new(0).start(); + let summatory = Summatory::new(0).spawn(); let stream = tokio_stream::iter(vec![Ok(1u16), Ok(2), Ok(3), Err(()), Ok(4), Ok(5)]); let msg_stream = stream.filter_map(|value| async move { @@ -259,7 +259,7 @@ pub fn test_skipping_on_stream_error() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let summatory = Summatory::new(0).start(); + let summatory = Summatory::new(0).spawn(); let stream = tokio_stream::iter(vec![Ok(1u16), Ok(2), Ok(3), Err(()), Ok(4), Ok(5)]); let msg_stream = stream.filter_map(|value| async move { diff --git a/concurrency/src/tasks/timer_tests.rs b/concurrency/src/tasks/timer_tests.rs index 894affde4..46403709a 100644 --- a/concurrency/src/tasks/timer_tests.rs +++ b/concurrency/src/tasks/timer_tests.rs @@ -89,7 +89,7 @@ pub fn test_send_interval_and_cancellation() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let repeater = Repeater::new(0).start(); + let repeater = Repeater::new(0).spawn(); // The interval keeps firing. let count = wait_for_async( @@ -188,7 +188,7 @@ pub fn test_send_after_and_cancellation() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let repeater = Delayed::new(0).start(); + let repeater = Delayed::new(0).spawn(); let ctx = repeater.context(); let _ = send_after(Duration::from_millis(100), ctx, Inc); @@ -222,7 +222,7 @@ pub fn test_send_after_gen_server_teardown() { move || { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async move { - let repeater = Delayed::new(0).start(); + let repeater = Delayed::new(0).spawn(); let ctx = repeater.context(); let _ = send_after(Duration::from_millis(100), ctx, Inc); diff --git a/concurrency/src/threads/actor.rs b/concurrency/src/threads/actor.rs index 47aeeb96c..4d13268aa 100644 --- a/concurrency/src/threads/actor.rs +++ b/concurrency/src/threads/actor.rs @@ -65,7 +65,10 @@ pub trait Actor: Send + Sized + 'static { /// `#[actor(max_native_jobs = N)]`. const MAX_NATIVE_JOBS: Option = None; - fn started(&mut self, _ctx: &Context) {} + /// Initialisation, before any message (OTP `init/1`). Returns `()`, + /// `Result<(), E>`, `Result, E>` or [`Init`]. A failed + /// start never runs `stopped()`. + fn started(&mut self, _ctx: &Context) -> impl Into> {} fn stopped(&mut self, _ctx: &Context) {} /// Called when a linked actor stops, if this actor has called @@ -219,23 +222,9 @@ impl Clone for ActorRef { // Actor::start // --------------------------------------------------------------------------- -/// Extension trait for starting an actor. Automatically implemented for all [`Actor`] types. -pub trait ActorStart: Actor { - /// Start the actor on a dedicated OS thread. - fn start(self) -> ActorRef { - ActorRef::spawn(self, None) - } - - /// Start linked to `parent_ctx`'s actor, which becomes this instance's - /// parent (E3). The link is registered before `started()`. The parent's - /// exit ends a trapping child normally (its `stopped()` runs; it never - /// reaches `exit_received`), a non-trapping child as any link notice. - fn start_linked(self, parent_ctx: &Context

) -> ActorRef { - ActorRef::spawn(self, Some(parent_ctx.actor_ref().child_handle())) - } -} - -impl ActorStart for A {} +mod start; +pub use start::{ActorStart, Init, Name, StartOptions, Starter}; +use start::{ArmMonitor, ParentParts, Verb}; // --------------------------------------------------------------------------- // send_message_on (utility) diff --git a/concurrency/src/threads/actor/actor_ref.rs b/concurrency/src/threads/actor/actor_ref.rs index bbaedc039..260e2366a 100644 --- a/concurrency/src/threads/actor/actor_ref.rs +++ b/concurrency/src/threads/actor/actor_ref.rs @@ -173,6 +173,13 @@ impl ActorRef { pub fn child_handle(&self) -> ChildHandle { self.shared.child_handle() } + + /// This instance's committed startup result (blocks until `started()` + /// returned; a failure until the instance is published, or its cleanup + /// deadline): the same answer the waiting starts give. + pub fn ready(&self) -> Result, crate::StartError> { + start::wait(self, false) + } } impl ActorShared { diff --git a/concurrency/src/threads/actor/api_coverage.rs b/concurrency/src/threads/actor/api_coverage.rs index 24606a605..033b070d9 100644 --- a/concurrency/src/threads/actor/api_coverage.rs +++ b/concurrency/src/threads/actor/api_coverage.rs @@ -69,7 +69,7 @@ fn context_recipient_and_identity_reach_their_actor() { "::context_recipient_and_identity_reach_their_actor" ), move || { - let actor = Echo.start(); + let actor = Echo.spawn(); let (recipient, id) = actor.request(OwnRecipient).unwrap(); assert_eq!(id, actor.id()); assert!(matches!(request(&*recipient, Ask(7), WAIT), Ok(7))); @@ -90,7 +90,7 @@ fn requests_to_a_stopped_actor_fail_without_waiting() { "::requests_to_a_stopped_actor_fail_without_waiting" ), move || { - let actor = Echo.start(); + let actor = Echo.spawn(); let recipient = actor.recipient::(); actor.child_handle().stop(); actor.join(); @@ -115,12 +115,12 @@ fn admitted_requests_fail_when_the_handler_unwinds_before_replying() { "::admitted_requests_fail_when_the_handler_unwinds_before_replying" ), move || { - let actor = Echo.start(); + let actor = Echo.spawn(); assert!(matches!( actor.request_with_timeout(Boom, WAIT), Err(ActorError::ActorStopped(ExitReason::Panic(_))) )); - let other = Echo.start(); + let other = Echo.spawn(); let recipient = other.recipient::(); assert!(matches!( request(&*recipient, Boom, WAIT), @@ -138,7 +138,7 @@ fn requests_time_out_while_the_actor_is_busy() { "::requests_time_out_while_the_actor_is_busy" ), move || { - let actor = Echo.start(); + let actor = Echo.spawn(); let recipient = actor.recipient::(); let (release, held) = std_mpsc::channel(); actor.send(StopAfter(held)).unwrap(); @@ -190,9 +190,9 @@ fn default_exit_received_keeps_a_trapping_actor_alive() { "::default_exit_received_keeps_a_trapping_actor_alive" ), move || { - let trapper = Trapper { trap: true }.start(); + let trapper = Trapper { trap: true }.spawn(); assert!(matches!(trapper.request(Ask(0)), Ok(0))); - let peer = Echo.start(); + let peer = Echo.spawn(); trapper.context().link(&peer.child_handle()); peer.send(Boom).unwrap(); assert!(matches!(peer.wait_exit(), ExitReason::Panic(_))); @@ -213,8 +213,8 @@ fn unlink_prevents_signal_delivery() { concat!(module_path!(), "::unlink_prevents_signal_delivery"), move || { // Not trapping: a linked peer's panic would take this actor down. - let trapper = Trapper { trap: false }.start(); - let peer = Echo.start(); + let trapper = Trapper { trap: false }.spawn(); + let peer = Echo.spawn(); let peer_handle = peer.child_handle(); trapper.context().link(&peer_handle); trapper.request(Unlink(peer_handle.clone())).unwrap(); @@ -243,8 +243,8 @@ fn opposite_links_between_the_same_pair_do_not_deadlock() { move || { // Each link/unlink locks both link tables. Both directions must take // the locks in the same global order, or these two loops deadlock. - let a = Trapper { trap: true }.start(); - let b = Trapper { trap: true }.start(); + let a = Trapper { trap: true }.spawn(); + let b = Trapper { trap: true }.spawn(); let (done, finished) = std_mpsc::channel(); let workers: Vec<_> = [ (a.context(), b.child_handle()), @@ -285,7 +285,7 @@ fn debug_output_names_the_handle() { crate::test_wait::bounded( concat!(module_path!(), "::debug_output_names_the_handle"), move || { - let actor = Echo.start(); + let actor = Echo.spawn(); assert!(format!("{actor:?}").starts_with("ActorRef")); assert!(format!("{:?}", actor.context()).starts_with("Context")); actor.child_handle().stop(); diff --git a/concurrency/src/threads/actor/context.rs b/concurrency/src/threads/actor/context.rs index e6eec023c..1f2265e65 100644 --- a/concurrency/src/threads/actor/context.rs +++ b/concurrency/src/threads/actor/context.rs @@ -256,14 +256,29 @@ impl Context { if !registered { return monitor_ref; } + self.spawn_monitor_waiter(target.clone(), monitor_ref, active, None); + monitor_ref + } - let target = target.clone(); + /// The monitor's waiter thread (see the tasks-mode `Context`): a + /// `start_monitor`'s `Down` is disposed of through the child's startup + /// cell. + fn spawn_monitor_waiter( + &self, + target: ChildHandle, + monitor_ref: MonitorRef, + active: spawned_rt::sync::CancellationToken, + cell: Option>, + ) where + A: Handler, + { + let observer = self.shared.id.raw(); let actor_ref = self.actor_ref(); let monitors = self.shared.monitors.clone(); let registration = crate::monitor::MonitorRegistration(monitors.clone(), monitor_ref); + let mut watch = crate::start::DownWatch(cell); rt::spawn(move || { - let _registration = registration; let reason = futures::executor::block_on(async { match futures::future::select( Box::pin(active.cancelled()), @@ -279,31 +294,76 @@ impl Context { // Receiver-side fence (E6): the entry is removed and the Down // enqueued in one critical section, so once `demonitor` returns // no Down of this monitor is enqueued. - crate::lifecycle_trace::life_section(observer, || { - crate::monitor::fence::deliver(&monitors, &monitor_ref, || { - crate::lifecycle_trace::life_emit(observer, || { - crate::lifecycle_trace::LifeEvent::new("Deliver", observer) - .s(target.id().raw()) - .m(monitor_ref.raw()) - .x(crate::lifecycle_trace::life_reason(&reason)) - .k("down") - .o("death") + let deliver = move |deliver: bool| { + let _registration = registration; + if !deliver { + return; + } + crate::lifecycle_trace::life_section(observer, || { + crate::monitor::fence::deliver(&monitors, &monitor_ref, || { + crate::lifecycle_trace::life_emit(observer, || { + crate::lifecycle_trace::LifeEvent::new("Deliver", observer) + .s(target.id().raw()) + .m(monitor_ref.raw()) + .x(crate::lifecycle_trace::life_reason(&reason)) + .k("down") + .o("death") + }); + crate::lifecycle_trace::start_notice( + observer, + target.id().raw(), + "monitor", + &reason, + || { + let _ = enqueue_down( + &actor_ref.shared.sender, + &actor_ref.shared.mailbox, + &actor_ref.shared.replies, + Down { + monitor_ref, + actor: target.id(), + reason: reason.clone(), + }, + ); + }, + ); }); - let _ = enqueue_down( - &actor_ref.shared.sender, - &actor_ref.shared.mailbox, - &actor_ref.shared.replies, - Down { - monitor_ref, - actor: target.id(), - reason, - }, - ); }); - }); + }; + match watch.take() { + None => deliver(true), + Some(cell) => { + if let Some(deliver) = cell.dispose_down(Box::new(deliver)) { + deliver(true); + } + } + } }); + } - monitor_ref + /// The start edge of a `start_monitor`: arm the waiter before dispatch. + pub(super) fn arm_start_monitor( + &self, + child: ChildHandle, + monitor_ref: MonitorRef, + active: spawned_rt::sync::CancellationToken, + cell: Arc, + ) where + A: Handler, + { + self.spawn_monitor_waiter(child, monitor_ref, active, Some(cell)); + } + + /// This actor as the parent of a `start_link`/`start_monitor`. + pub(super) fn parent_parts(&self) -> ParentParts { + ParentParts { + id: self.shared.id, + signal: self.shared.signal.clone(), + links: self.shared.links.clone(), + request: self.shared.linked_reason.clone(), + monitors: self.shared.monitors.clone(), + token: self.shared.cancellation_token.clone(), + } } /// Cancel a previously-set monitor. diff --git a/concurrency/src/threads/actor/early_stop_regression.rs b/concurrency/src/threads/actor/early_stop_regression.rs index a66e19e0f..73378f553 100644 --- a/concurrency/src/threads/actor/early_stop_regression.rs +++ b/concurrency/src/threads/actor/early_stop_regression.rs @@ -5,18 +5,35 @@ impl Actor for EarlyStop { ctx.stop(); } fn stopped(&mut self, _: &Context) { - panic!("cleanup failed"); + panic!("stopped() must not run after a failed start"); } } +/// `ctx.stop()` in `started()` is a failed start (unified start, 2.3 row +/// 5): `stopped()` does not run and the start reports `InitExited(Normal)`. #[test] -fn panic_during_early_stop_is_reported() { +fn stop_in_started_fails_the_start_without_stopped() { crate::test_wait::bounded( - concat!(module_path!(), "::panic_during_early_stop_is_reported"), + concat!( + module_path!(), + "::stop_in_started_fails_the_start_without_stopped" + ), move || { - let actor = EarlyStop.start(); - assert!( - matches!(actor.wait_exit(), ExitReason::Panic(message) if message.contains("cleanup failed")) - ); + let actor = EarlyStop.spawn(); + assert_eq!(actor.wait_exit(), ExitReason::Normal); + assert!(matches!( + actor.ready(), + Err(crate::StartError::InitExited { + reason: ExitReason::Normal, + pending: None + }) + )); + assert!(matches!( + EarlyStop.start(), + Err(crate::StartError::InitExited { + reason: ExitReason::Normal, + pending: None + }) + )); }, ); } diff --git a/concurrency/src/threads/actor/envelope.rs b/concurrency/src/threads/actor/envelope.rs index 5ec5a1ebd..5cc60f6e8 100644 --- a/concurrency/src/threads/actor/envelope.rs +++ b/concurrency/src/threads/actor/envelope.rs @@ -322,6 +322,7 @@ pub(super) fn signal_target( trap: TrapExitFlag, request: LinkedExitReason, token: CancellationToken, + startup: Arc, ) -> Arc { let exit_tx = sender.clone(); let parent_tx = sender.clone(); @@ -353,6 +354,7 @@ pub(super) fn signal_target( send_exit, send_parent_exit, send_held, + startup, }) } diff --git a/concurrency/src/threads/actor/idle_stop_regression.rs b/concurrency/src/threads/actor/idle_stop_regression.rs index a105fb17e..77dbd122c 100644 --- a/concurrency/src/threads/actor/idle_stop_regression.rs +++ b/concurrency/src/threads/actor/idle_stop_regression.rs @@ -73,6 +73,7 @@ fn stop_while_receiving_does_not_dispatch_a_later_request() { trap.clone(), request.clone(), token.clone(), + crate::start::StartupCell::new(), ), mailbox: crate::instrumentation::Mailbox::default(), replies: Registry::new_native(ActorId::next(), DEFAULT_MAX_OUTSTANDING), diff --git a/concurrency/src/threads/actor/lifecycle.rs b/concurrency/src/threads/actor/lifecycle.rs index 3e59a60ea..f28dda265 100644 --- a/concurrency/src/threads/actor/lifecycle.rs +++ b/concurrency/src/threads/actor/lifecycle.rs @@ -6,27 +6,78 @@ use super::*; // --------------------------------------------------------------------------- impl ActorRef { - pub(super) fn spawn(actor: A, parent: Option) -> Self { + /// Stage 1 of every start (design section 3; threads mode): validate, + /// prepare the instance privately, commit it atomically (the parent's + /// edge and the name), then start its thread. + pub(super) fn launch( + actor: A, + options: StartOptions, + verb: Verb, + arm: Option, + ) -> Result<(Self, Option), crate::start::StartError> { + use crate::lifecycle_trace as trace; + use crate::start::{EdgeKind, StartEdge, StartError}; #[cfg(all(spawned_sim, feature = "async"))] assert_native_clock_context(); + let id = ActorId::next(); + let c = trace::start_child(id.raw()); + let parent = match &verb { + Verb::Link(p) | Verb::Monitor(p) => Some(p), + Verb::Spawn | Verb::Start => None, + }; + trace::start_emit(c, || { + trace::StartEvent::new("SBegin", c) + .x(verb.trace_name()) + .y("none") + .f(options.name.is_some()) + }); + if let Err(error) = options.validate() { + trace::start_emit(c, || trace::StartEvent::new("SPrepareFail", c)); + return Err(error); + } let (tx, rx) = mpsc::channel::>(); let cancellation_token = CancellationToken::new(); register_stop_wakeup(&cancellation_token, tx.clone()); let (completion_tx, completion) = spawned_rt::sync::watch::channel(None); - let id = ActorId::next(); let monitors: MonitorTable = Arc::new(crate::monitor::fence::open()); let trap_exit = new_trap_exit_flag(); let links = new_link_table(); let linked_reason = new_linked_exit_reason(); - // Phase 0: signals that arrive before started() returns are held. - link::lock_request(&linked_reason).begin_starting(); + { + // Phase 0: signals that arrive before started() returns are held. + let mut request = link::lock_request(&linked_reason); + request.begin_starting(); + #[cfg(feature = "lifecycle-trace")] + if c != 0 { + request.trace = trace::StartRole::Child(c); + } + } + let cell = crate::start::StartupCell::new(); + cell.prepare( + verb.waiting(), + parent.map(|p| StartEdge { + kind: if matches!(verb, Verb::Link(_)) { + EdgeKind::Link + } else { + EdgeKind::Monitor + }, + parent: p.signal.clone(), + }), + DEFAULT_STOP_DEADLINE, + c, + ); + let link_parent = match &verb { + Verb::Link(p) => Some(p.id), + _ => None, + }; let signal = signal_target::( id, - parent.as_ref().map(ChildHandle::id), + link_parent, &tx, trap_exit.clone(), linked_reason.clone(), cancellation_token.clone(), + cell.clone(), ); let shared = Arc::new(ActorShared { @@ -40,24 +91,134 @@ impl ActorRef { linked_reason: linked_reason.clone(), signal, mailbox: crate::instrumentation::Mailbox::default(), - replies: Registry::new_native(id, A::MAX_OUTSTANDING), + replies: Registry::new_native( + id, + options.max_outstanding.unwrap_or(A::MAX_OUTSTANDING), + ), jobs: Arc::new(crate::reply::jobs::JobSet::new( id, - A::MAX_NATIVE_JOBS.unwrap_or_else(crate::reply::jobs::default_max_native_jobs), + options + .max_native_jobs + .or(A::MAX_NATIVE_JOBS) + .unwrap_or_else(crate::reply::jobs::default_max_native_jobs), )), cleaned: Arc::default(), }); let actor_ref = ActorRef { shared: shared.clone(), }; + let child_handle = actor_ref.child_handle(); + + // Commit: the parent's edge and the name, all or nothing (lock + // order: the parent's table, its exit request, the registry). + let register = || -> Result, StartError> { + let Some(name) = &options.name else { + return Ok(None); + }; + match crate::registry::register_start( + name.key(), + id, + actor_ref.clone(), + child_handle.clone(), + c, + || Ok::<(), StartError>(()), + )? { + Ok(guard) => Ok(Some(guard)), + Err(existing) => Err(StartError::AlreadyStarted { + name: name.key().to_owned(), + existing, + }), + } + }; + let committed = trace::start_section(|| { + trace::commit_begin(c, "U"); + let result = (|| -> Result<_, StartError> { + match &verb { + Verb::Link(p) => { + let mut inner = Ok(None); + let installed = link::install_start_edge( + &p.signal, + &p.links, + &actor_ref.shared.signal, + &links, + || { + let request = link::lock_request(&p.request); + if request.reason().is_some() || request.is_dead() { + trace::commit_point(Some("parentexited")); + return false; + } + inner = register(); + if inner.is_ok() { + trace::commit_point(None); + } + inner.is_ok() + }, + ); + match (installed, inner) { + (_, Err(error)) => Err(error), + (false, Ok(_)) => Err(StartError::ParentExited), + (true, Ok(name)) => Ok((name, None)), + } + } + Verb::Monitor(p) => { + let monitor_ref = MonitorRef::next(); + let active = spawned_rt::sync::CancellationToken::new(); + let mut inner = Ok(None); + let registered = crate::monitor::fence::register( + &p.monitors, + monitor_ref, + active.clone(), + || { + let request = link::lock_request(&p.request); + if request.reason().is_some() + || request.is_dead() + || p.token.is_cancelled() + { + trace::commit_point(Some("parentexited")); + return true; + } + inner = register(); + if inner.is_ok() { + trace::commit_point(None); + } + inner.is_err() + }, + || false, + || {}, + || {}, + ); + match (registered, inner) { + (_, Err(error)) => Err(error), + (crate::monitor::fence::Registration::Active, Ok(name)) => { + Ok((name, Some((monitor_ref, active)))) + } + _ => Err(StartError::ParentExited), + } + } + Verb::Spawn | Verb::Start => Ok((register()?, None)), + } + })(); + trace::commit_end(result.as_ref().err().map(StartError::trace_refusal)); + result + }); + let (name, monitor) = committed?; + // A waiting start_link/start_monitor: the parent's exit by a signal + // abandons the wait (OTP: the caller dies at once). + cell.watch_parent(); + let monitor_ref = monitor.map(|(monitor_ref, active)| { + if let Some(arm) = arm { + arm(child_handle.clone(), monitor_ref, active, cell.clone()); + } + monitor_ref + }); + let ctx = Context { shared }; - let parent_id = parent.as_ref().map_or(0, |p| p.id().raw()); - if let Some(parent) = parent { - actor_ref.context().link(&parent); + // Recorded from here: the start_link link is the model's initial one. + crate::lifecycle_trace::life_spawn(id.raw(), link_parent.map_or(0, |p| p.raw())); + if let crate::start::Timeout::After(after) = options.init_timeout { + crate::start::spawn_init_timer(&cell, child_handle.clone(), after); } - // Recorded from here: the start_linked link is the model's initial one. - crate::lifecycle_trace::life_spawn(id.raw(), parent_id); let cleanup_token = cancellation_token.clone(); let cleanup = crate::actor_cleanup::ActorCleanup::new( @@ -70,7 +231,9 @@ impl ActorRef { ) .with_mailbox_failures(actor_ref.shared.sender.failures()) .with_replies(actor_ref.shared.replies.clone()) - .with_jobs(actor_ref.shared.jobs.clone()); + .with_jobs(actor_ref.shared.jobs.clone()) + .with_startup(cell, link_parent) + .with_name(name); let startup = crate::actor_cleanup::ActorStartup::new( (actor, ctx, rx, cancellation_token), cleanup, @@ -79,7 +242,7 @@ impl ActorRef { let replies = actor_ref.shared.replies.clone(); let jobs = actor_ref.shared.jobs.clone(); let cleaned = crate::reply::jobs::CleanedGuard(actor_ref.shared.cleaned.clone()); - let _thread_handle = rt::spawn(move || { + let body = move || { let _cleaned = cleaned; let ((mut actor, ctx, mut rx, cancellation_token), mut cleanup) = startup.into_parts(); let request = ctx.shared.linked_reason.clone(); @@ -122,9 +285,29 @@ impl ActorRef { let unfinished = jobs.join_blocking(std::time::Instant::now() + remaining); crate::reply::jobs::report_unfinished(jobs.owner(), unfinished); } - }); - - actor_ref + // The child's owned work is done (`Start.tla` WorkDoneC). + let c = crate::lifecycle_trace::start_child_of(jobs.owner().raw()); + crate::lifecycle_trace::start_emit(c, || { + crate::lifecycle_trace::StartEvent::new("SWorkDone", c) + }); + }; + let mut builder = std::thread::Builder::new(); + if let Some(bytes) = options.stack_size { + builder = builder.stack_size(bytes); + } + if let Some(label) = &options.label { + builder = builder.name(label.to_string()); + } + // A spawn failure after the commit kills the child before started() + // (`InitExited { Killed }`), not a stage-1 refusal. + crate::start::spawn_thread_or_kill( + builder, + &child_handle, + body, + |body| body(), + |body| body(), + ); + Ok((actor_ref, monitor_ref)) } } @@ -157,93 +340,178 @@ pub(super) fn run_actor_receiving( let request = ctx.shared.linked_reason.clone(); let trace_id = ctx.shared.id.raw(); let _monitor_cleanup = MonitorCleanup(ctx.shared.monitors.clone()); - let start_result = catch_unwind(AssertUnwindSafe(|| { - fail_point!("actor.before_started"); - crate::lifecycle_trace::with_actor(trace_id, || actor.started(&ctx)); - })); - if let Err(panic) = start_result { - let msg = panic_message(&*panic); - crate::actor_cleanup::log_safely(|| { - tracing::error!("Panic in started() callback: {msg}"); - }); - // No stopped() after a failed start (L5). + let cell = ctx.shared.signal.startup.clone(); + // A kill or the init timer before started() began: it is not run. + let early = crate::lifecycle_trace::start_section(|| { + let killed = link::lock_request(&request).killed_early() + || cell.commit() == crate::start::Commit::TimedOut; + if !killed { + let c = cell.trace_id(); + crate::lifecycle_trace::start_emit(c, || { + crate::lifecycle_trace::StartEvent::new("SStartedBegin", c) + }); + } + killed + }); + let (returned, continuation) = if early { + (crate::start::Returned::NotBegun, None) + } else { + let start_result = catch_unwind(AssertUnwindSafe(|| { + fail_point!("actor.before_started"); + crate::lifecycle_trace::with_actor(trace_id, || actor.started(&ctx).into()) + })); + match start_result { + Err(panic) => { + let msg = panic_message(&*panic); + crate::actor_cleanup::log_safely(|| { + tracing::error!("Panic in started() callback: {msg}"); + }); + (crate::start::Returned::Panic(msg), None) + } + Ok(init) => { + let init: Init = init; + init.into_returned() + } + } + }; + // The startup decision (design 2.3) and the readiness commit (5.1), + // under the exit-request and cell locks. No failed start runs stopped(). + let trapping = ctx.shared.trap_exit.load(Ordering::Acquire); + let continued = continuation.is_some(); + let row = crate::lifecycle_trace::start_section(|| { crate::lifecycle_trace::life_section(trace_id, || { - crate::lifecycle_trace::life_emit(trace_id, || { - crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("started") + let mut req = link::lock_request(&request); + let row = cell.decide(&returned, continued, trapping, |timed_out| { + let row = req.resolve_start(trapping, timed_out, &returned, true); + let cause = match (&row, &returned) { + (crate::start::Row::Error, crate::start::Returned::Error(cause)) => { + Some(cause.clone()) + } + _ => None, + }; + (row, req.reason().cloned(), cause) }); - link::lock_request(&request) - .panic(ExitReason::Panic(format!("panic in started(): {msg}"))); - }); + drop(req); + if row == crate::start::Row::Panic { + crate::lifecycle_trace::life_emit(trace_id, || { + crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("started") + }); + } + row + }) + }); + if row != crate::start::Row::Ready { + if let (crate::start::Returned::Error(cause), false) = + (&returned, row == crate::start::Row::Error) + { + let id = ctx.shared.id; + crate::actor_cleanup::log_safely(|| { + tracing::debug!(actor = %id, cause, ?row, "init error superseded by a higher-precedence exit"); + }); + } cancellation_token.cancel(); return None; } - // Phase 0 ends: resolve signals held during started() with the trap - // flag read now. - if link::lock_request(&request).end_starting(ctx.shared.trap_exit.load(Ordering::Acquire)) { - cancellation_token.cancel(); - } - - while !cancellation_token.is_cancelled() { - let msg = match receive() { - Ok(MailboxItem::Message(msg)) => Some(msg), - Ok(MailboxItem::Shutdown) => None, - Err(std::sync::mpsc::RecvTimeoutError::Timeout) => continue, - Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => None, - }; - match msg { - Some(envelope) => { - #[cfg(feature = "metrics")] - let mut envelope = envelope; - - // A stop observed after receive wins over dispatch, whether - // the channel waited or was buffered. Defer the message drop. - // The check and the trace event are one lifecycle-trace step. - let stopped = crate::lifecycle_trace::life_section(trace_id, || { - if cancellation_token.is_cancelled() { - return true; - } - #[cfg(feature = "lifecycle-trace")] + // A continuation runs first (L10); a stop observed before it begins + // discards it, as queued messages. + let mut panicked = false; + if let Some(envelope) = continuation { + if cancellation_token.is_cancelled() { + discarded = Some(envelope); + } else { + let c = cell.trace_id(); + crate::lifecycle_trace::start_emit(c, || { + crate::lifecycle_trace::StartEvent::new("SContinue", c) + }); + ctx.shared.replies.begin_turn(); + let result = catch_unwind(AssertUnwindSafe(|| { + crate::lifecycle_trace::with_actor(trace_id, || envelope.handle(actor, &ctx)); + })); + ctx.shared.replies.end_turn(result.is_err()); + if let Err(panic) = result { + let msg = panic_message(&*panic); + crate::actor_cleanup::log_safely(|| { + tracing::error!("Panic in message handler: {msg}"); + }); + crate::lifecycle_trace::life_section(trace_id, || { crate::lifecycle_trace::life_emit(trace_id, || { - crate::lifecycle_trace::LifeEvent::new("Dispatch", trace_id) - .k(envelope.trace_kind()) + crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("turn") }); - false + link::lock_request(&request) + .panic(ExitReason::Panic(format!("panic in handler: {msg}"))); }); - if stopped { - discarded = Some(envelope); - break; - } - #[cfg(feature = "metrics")] - let observation = envelope - .observation() - .map(|observation| observation.begin()); - ctx.shared.replies.begin_turn(); - let result = catch_unwind(AssertUnwindSafe(|| { - crate::lifecycle_trace::with_actor(trace_id, || envelope.handle(actor, &ctx)); - })); - ctx.shared.replies.end_turn(result.is_err()); - #[cfg(feature = "metrics")] - if let Some(observation) = observation { - observation.finish(result.is_err()); - } - if let Err(panic) = result { - let msg = panic_message(&*panic); - crate::actor_cleanup::log_safely(|| { - tracing::error!("Panic in message handler: {msg}"); - }); - // A panic begins exit, or replaces the requested reason - // unless a kill was received. - crate::lifecycle_trace::life_section(trace_id, || { + panicked = true; + } + } + } + + if !panicked { + while !cancellation_token.is_cancelled() { + let msg = match receive() { + Ok(MailboxItem::Message(msg)) => Some(msg), + Ok(MailboxItem::Shutdown) => None, + Err(std::sync::mpsc::RecvTimeoutError::Timeout) => continue, + Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => None, + }; + match msg { + Some(envelope) => { + #[cfg(feature = "metrics")] + let mut envelope = envelope; + + // A stop observed after receive wins over dispatch, whether + // the channel waited or was buffered. Defer the message drop. + // The check and the trace event are one lifecycle-trace step. + let stopped = crate::lifecycle_trace::life_section(trace_id, || { + if cancellation_token.is_cancelled() { + return true; + } + #[cfg(feature = "lifecycle-trace")] crate::lifecycle_trace::life_emit(trace_id, || { - crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("turn") + crate::lifecycle_trace::LifeEvent::new("Dispatch", trace_id) + .k(envelope.trace_kind()) }); - link::lock_request(&request) - .panic(ExitReason::Panic(format!("panic in handler: {msg}"))); + #[cfg(feature = "lifecycle-trace")] + crate::lifecycle_trace::start_dispatch(trace_id, envelope.trace_kind()); + false }); - break; + if stopped { + discarded = Some(envelope); + break; + } + #[cfg(feature = "metrics")] + let observation = envelope + .observation() + .map(|observation| observation.begin()); + ctx.shared.replies.begin_turn(); + let result = catch_unwind(AssertUnwindSafe(|| { + crate::lifecycle_trace::with_actor(trace_id, || { + envelope.handle(actor, &ctx) + }); + })); + ctx.shared.replies.end_turn(result.is_err()); + #[cfg(feature = "metrics")] + if let Some(observation) = observation { + observation.finish(result.is_err()); + } + if let Err(panic) = result { + let msg = panic_message(&*panic); + crate::actor_cleanup::log_safely(|| { + tracing::error!("Panic in message handler: {msg}"); + }); + // A panic begins exit, or replaces the requested reason + // unless a kill was received. + crate::lifecycle_trace::life_section(trace_id, || { + crate::lifecycle_trace::life_emit(trace_id, || { + crate::lifecycle_trace::LifeEvent::new("Panic", trace_id).x("turn") + }); + link::lock_request(&request) + .panic(ExitReason::Panic(format!("panic in handler: {msg}"))); + }); + break; + } } + None => break, } - None => break, } } diff --git a/concurrency/src/threads/actor/monitor_cleanup_regressions.rs b/concurrency/src/threads/actor/monitor_cleanup_regressions.rs index b5c66b8e2..060a367d5 100644 --- a/concurrency/src/threads/actor/monitor_cleanup_regressions.rs +++ b/concurrency/src/threads/actor/monitor_cleanup_regressions.rs @@ -14,8 +14,8 @@ fn closed_monitor_table_rejects_late_registration() { "::closed_monitor_table_rejects_late_registration" ), move || { - let observer = Idle.start(); - let target = Idle.start(); + let observer = Idle.spawn(); + let target = Idle.spawn(); // Model cleanup during an unexpected unwind before token cancellation. drop(MonitorCleanup(observer.shared.monitors.clone())); let child = target.child_handle(); @@ -32,8 +32,8 @@ fn closed_monitor_table_rejects_late_registration() { ); } fn check_cleanup(stop_owner: bool) { - let target = Idle.start(); - let observer = Idle.start(); + let target = Idle.spawn(); + let observer = Idle.spawn(); let child = target.child_handle(); let waiter = child.lifetime_probe(); let monitor = observer.context().monitor(&child); diff --git a/concurrency/src/threads/actor/start.rs b/concurrency/src/threads/actor/start.rs new file mode 100644 index 000000000..b30ad36a2 --- /dev/null +++ b/concurrency/src/threads/actor/start.rs @@ -0,0 +1,410 @@ +//! Unified start (0.9, design `docs/design/unified-start.md`), threads mode: +//! the same verbs as tasks mode; the waiting forms block. Every actor owns +//! a thread, so `StartOptions` has no placement. +use super::*; +use crate::start::{Observed, StartError, StartOutcome, Timeout}; +use std::borrow::Cow; +use std::fmt::Display; +use std::marker::PhantomData; + +/// What `started()` returns (OTP `init/1`'s results); see the tasks-mode +/// `Init` (`spawned_concurrency::tasks::Init`) for the table. No failed start runs +/// `stopped()`. +#[must_use] +pub struct Init { + pub(super) kind: InitKind, +} + +pub(super) enum InitKind { + Ready, + Continue(Box + Send>), + Error(String), + Stop(ExitReason), + Ignore, +} + +impl std::fmt::Debug for Init { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let kind = match &self.kind { + InitKind::Ready => "ready", + InitKind::Continue(_) => "continue", + InitKind::Error(_) => "error", + InitKind::Stop(_) => "stop", + InitKind::Ignore => "ignore", + }; + f.debug_tuple("Init").field(&kind).finish() + } +} + +impl Init { + /// Ready (`{ok, S}`). + pub fn ready() -> Self { + Self { + kind: InitKind::Ready, + } + } + /// Ready, then handle `msg` before any mailbox entry (`handle_continue`). + /// Not a barrier against termination. + pub fn continue_with(msg: M) -> Self + where + A: Handler, + { + Self { + kind: InitKind::Continue(Box::new(MessageEnvelope { + slot: SlotHandle::detached(), + msg, + #[cfg(feature = "metrics")] + observation: None, + #[cfg(feature = "lifecycle-trace")] + from: 0, + })), + } + } + /// A failed initialisation (`{error, R}`): the actor ends `Normal`. + pub fn error(cause: impl Display) -> Self { + Self { + kind: InitKind::Error(cause.to_string()), + } + } + /// Stop with `reason` (`{stop, R}`). + pub fn stop(reason: ExitReason) -> Self { + Self { + kind: InitKind::Stop(reason), + } + } + /// Opt out (`ignore`): the actor ends `Normal`. + pub fn ignore() -> Self { + Self { + kind: InitKind::Ignore, + } + } + pub(super) fn into_returned( + self, + ) -> (crate::start::Returned, Option + Send>>) { + use crate::start::Returned; + match self.kind { + InitKind::Ready => (Returned::Ready, None), + InitKind::Continue(envelope) => (Returned::Ready, Some(envelope)), + InitKind::Error(cause) => (Returned::Error(cause), None), + InitKind::Stop(reason) => (Returned::Stop(reason), None), + InitKind::Ignore => (Returned::Ignore, None), + } + } +} + +impl From<()> for Init { + fn from(_: ()) -> Self { + Self::ready() + } +} + +impl From> for Init { + fn from(result: Result<(), E>) -> Self { + match result { + Ok(()) => Self::ready(), + Err(error) => Self::error(error), + } + } +} + +impl From, E>> for Init { + fn from(result: Result, E>) -> Self { + result.unwrap_or_else(Self::error) + } +} + +/// A registered name for actors of type `A`, committed with the start and +/// released at publication. +pub struct Name { + key: Cow<'static, str>, + _actor: PhantomData A>, +} + +impl Name { + pub const fn new(key: &'static str) -> Self { + Self { + key: Cow::Borrowed(key), + _actor: PhantomData, + } + } + pub fn from_string(key: String) -> Self { + Self { + key: Cow::Owned(key), + _actor: PhantomData, + } + } + pub fn key(&self) -> &str { + &self.key + } + /// The live instance holding this name, if it is an `A`. + pub fn whereis(&self) -> Option> { + crate::registry::whereis_start::>(&self.key) + } +} + +impl Clone for Name { + fn clone(&self) -> Self { + Self { + key: self.key.clone(), + _actor: PhantomData, + } + } +} + +impl std::fmt::Debug for Name { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_tuple("Name").field(&self.key).finish() + } +} + +impl From<&'static str> for Name { + fn from(key: &'static str) -> Self { + Self::new(key) + } +} + +impl From for Name { + fn from(key: String) -> Self { + Self::from_string(key) + } +} + +/// How to start an actor of type `A` (threads mode: no placement). +#[non_exhaustive] +pub struct StartOptions { + pub(super) name: Option>, + pub(super) label: Option>, + pub(super) init_timeout: Timeout, + pub(super) stack_size: Option, + pub(super) max_outstanding: Option, + pub(super) max_native_jobs: Option, +} + +impl Clone for StartOptions { + fn clone(&self) -> Self { + Self { + name: self.name.clone(), + label: self.label.clone(), + init_timeout: self.init_timeout, + stack_size: self.stack_size, + max_outstanding: self.max_outstanding, + max_native_jobs: self.max_native_jobs, + } + } +} + +impl Default for StartOptions { + fn default() -> Self { + Self::new() + } +} + +impl std::fmt::Debug for StartOptions { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("StartOptions") + .field("name", &self.name) + .field("init_timeout", &self.init_timeout) + .finish_non_exhaustive() + } +} + +impl StartOptions { + pub fn new() -> Self { + Self { + name: None, + label: None, + init_timeout: Timeout::Infinity, + stack_size: None, + max_outstanding: None, + max_native_jobs: None, + } + } + /// Register the actor under `name` atomically with the start. + pub fn name(mut self, name: impl Into>) -> Self { + self.name = Some(name.into()); + self + } + /// A label for diagnostics (the actor thread's name). + pub fn label(mut self, label: impl Into>) -> Self { + self.label = Some(label.into()); + self + } + /// Bound `started()` (default `Infinity`). + pub fn init_timeout(mut self, timeout: impl Into) -> Self { + self.init_timeout = timeout.into(); + self + } + /// Stack size of the actor's thread. + pub fn stack_size(mut self, bytes: usize) -> Self { + self.stack_size = Some(bytes); + self + } + /// Per-instance override of `Actor::MAX_OUTSTANDING`. + pub fn max_outstanding(mut self, max: usize) -> Self { + self.max_outstanding = Some(max); + self + } + /// Per-instance override of `Actor::MAX_NATIVE_JOBS`. + pub fn max_native_jobs(mut self, max: usize) -> Self { + self.max_native_jobs = Some(max); + self + } + pub(super) fn validate(&self) -> Result<(), StartError> { + if self.init_timeout == Timeout::After(Duration::ZERO) { + return Err(StartError::InvalidOptions("init_timeout must be nonzero")); + } + if self.max_outstanding == Some(0) { + return Err(StartError::InvalidOptions( + "max_outstanding must be nonzero", + )); + } + if self.max_native_jobs == Some(0) { + return Err(StartError::InvalidOptions( + "max_native_jobs must be nonzero", + )); + } + if self.stack_size == Some(0) { + return Err(StartError::InvalidOptions("stack_size must be nonzero")); + } + Ok(()) + } +} + +/// How a start was requested. +pub(super) enum Verb { + Spawn, + Start, + Link(ParentParts), + Monitor(ParentParts), +} + +impl Verb { + pub(super) fn trace_name(&self) -> &'static str { + match self { + Self::Spawn => "spawn", + Self::Start => "start", + Self::Link(_) => "link", + Self::Monitor(_) => "monitor", + } + } + pub(super) fn waiting(&self) -> bool { + !matches!(self, Self::Spawn) + } +} + +/// The parent side of a `start_link`/`start_monitor` (type-erased). +pub(super) struct ParentParts { + pub(super) id: ActorId, + pub(super) signal: Arc, + pub(super) links: LinkTable, + pub(super) request: LinkedExitReason, + pub(super) monitors: MonitorTable, + pub(super) token: CancellationToken, +} + +/// Arms the parent's monitor waiter (typed by the parent) before dispatch. +pub(super) type ArmMonitor = Box< + dyn FnOnce( + ChildHandle, + MonitorRef, + spawned_rt::sync::CancellationToken, + Arc, + ) + Send, +>; + +/// Starting an actor; the waiting forms block until `started()` returns +/// (and, for a failure, until the actor is published). +pub trait ActorStart: Actor { + /// Start without waiting (`proc_lib:spawn`). + fn spawn(self) -> ActorRef { + Starter::new(self, StartOptions::new()) + .spawn() + .unwrap_or_else(|error| panic!("actor spawn failed: {error}")) + } + /// Start and wait for `started()` (`gen_server:start`). + fn start(self) -> Result>, StartError> { + Starter::new(self, StartOptions::new()).start() + } + /// Start linked to `parent`'s actor and wait (`gen_server:start_link`). + fn start_link( + self, + parent: &Context

, + ) -> Result>, StartError> { + Starter::new(self, StartOptions::new()).start_link(parent) + } + /// Start monitored by `parent`'s actor and wait + /// (`gen_server:start_monitor`). + fn start_monitor>( + self, + parent: &Context

, + ) -> Result, MonitorRef)>, StartError> { + Starter::new(self, StartOptions::new()).start_monitor(parent) + } + /// Start with options. + fn with_options(self, options: impl Into>) -> Starter { + Starter::new(self, options.into()) + } +} + +impl ActorStart for A {} + +/// An actor and its start options; choose a verb. +#[must_use = "call spawn, start, start_link or start_monitor"] +pub struct Starter { + actor: A, + options: StartOptions, +} + +impl Starter { + fn new(actor: A, options: StartOptions) -> Self { + Self { actor, options } + } + /// Start without waiting. + pub fn spawn(self) -> Result, StartError> { + ActorRef::launch(self.actor, self.options, Verb::Spawn, None).map(|(actor, _)| actor) + } + /// Start and wait for `started()`. + pub fn start(self) -> Result>, StartError> { + let (actor, _) = ActorRef::launch(self.actor, self.options, Verb::Start, None)?; + wait(&actor, true).map(|o| o.map(|()| actor)) + } + /// Start linked to `parent`'s actor and wait. + pub fn start_link( + self, + parent: &Context

, + ) -> Result>, StartError> { + let verb = Verb::Link(parent.parent_parts()); + let (actor, _) = ActorRef::launch(self.actor, self.options, verb, None)?; + wait(&actor, true).map(|o| o.map(|()| actor)) + } + /// Start monitored by `parent`'s actor and wait. + pub fn start_monitor>( + self, + parent: &Context

, + ) -> Result, MonitorRef)>, StartError> { + let verb = Verb::Monitor(parent.parent_parts()); + let parent = parent.clone(); + let arm: ArmMonitor = Box::new(move |child, monitor_ref, active, cell| { + parent.arm_start_monitor(child, monitor_ref, active, cell) + }); + let (actor, monitor) = ActorRef::launch(self.actor, self.options, verb, Some(arm))?; + let monitor = monitor.expect("start_monitor commits a monitor"); + wait(&actor, true).map(|o| o.map(|()| (actor, monitor))) + } +} + +/// Block for the startup result of `actor` (`waiter`: the start's own +/// wait, which consumes it). +pub(super) fn wait( + actor: &ActorRef, + waiter: bool, +) -> Result, StartError> { + let cell = actor.shared.signal.startup.clone(); + loop { + let seen = cell.version(); + match cell.observe(waiter, || actor.child_handle()) { + Observed::Answer(answer) => return answer, + Observed::Wait(deadline) => cell.wait_blocking(seen, deadline), + } + } +} diff --git a/concurrency/src/threads/actor/stop_wakeup_regression.rs b/concurrency/src/threads/actor/stop_wakeup_regression.rs index 995417159..2184189ac 100644 --- a/concurrency/src/threads/actor/stop_wakeup_regression.rs +++ b/concurrency/src/threads/actor/stop_wakeup_regression.rs @@ -79,6 +79,7 @@ fn cancellation_wakes_the_real_blocking_receive_boundary() { trap.clone(), request.clone(), token.clone(), + crate::start::StartupCell::new(), ), mailbox: crate::instrumentation::Mailbox::default(), replies: Registry::new_native(ActorId::next(), DEFAULT_MAX_OUTSTANDING), @@ -137,11 +138,11 @@ fn every_stop_route_wakes_an_idle_thread() { move || { for route in 0..4 { let (stopped_tx, stopped) = mpsc::channel(); - let actor = Idle(stopped_tx).start(); + let actor = Idle(stopped_tx).spawn(); // A reply ensures startup is complete before the cancellation race. actor.request(Noop).unwrap(); let failing = if route == 3 { - let peer = Failing.start(); + let peer = Failing.spawn(); actor.context().link(&peer.child_handle()); Some(peer) } else { @@ -219,7 +220,7 @@ fn repeated_stop_during_a_handler_discards_queued_requests() { release: release_rx, handled: handled.clone(), } - .start(); + .spawn(); let first = actor.send_request(Work, DEFAULT_REQUEST_TIMEOUT).unwrap(); entered.recv_timeout(Duration::from_secs(2)).unwrap(); let replies: Vec<_> = (0..256) diff --git a/concurrency/src/threads/actor/tests.rs b/concurrency/src/threads/actor/tests.rs index 0dbc1f31b..a7074eb2b 100644 --- a/concurrency/src/threads/actor/tests.rs +++ b/concurrency/src/threads/actor/tests.rs @@ -69,7 +69,7 @@ fn basic_send_and_request() { crate::test_wait::bounded( concat!(module_path!(), "::basic_send_and_request"), move || { - let actor = Counter { count: 0 }.start(); + let actor = Counter { count: 0 }.spawn(); assert_eq!(actor.request(GetCount).unwrap(), 0); assert_eq!(actor.request(Increment).unwrap(), 1); // Same mailbox: the request is handled after the send. @@ -107,7 +107,7 @@ fn join_waits_for_completion() { } } - let actor = SlowStop.start(); + let actor = SlowStop.spawn(); actor.send(StopSlow).unwrap(); actor.join(); // If join() returned, stopped() has completed @@ -142,7 +142,7 @@ fn join_multiple_callers() { } } - let actor = SlowStop2.start(); + let actor = SlowStop2.spawn(); let a1 = actor.clone(); let a2 = actor.clone(); let t1 = thread::spawn(move || { @@ -186,7 +186,7 @@ fn panic_in_started_stops_actor() { } } - let actor = PanicOnStart.start(); + let actor = PanicOnStart.spawn(); actor.join(); let result = actor.send(PingThread); assert!(result.is_err()); @@ -232,7 +232,7 @@ fn panic_in_handler_stops_actor() { } } - let actor = PanicOnMsg.start(); + let actor = PanicOnMsg.spawn(); actor.send(ExplodeThread).unwrap(); actor.join(); let result = actor.request(CheckThread); @@ -268,7 +268,7 @@ fn panic_in_stopped_still_completes() { } } - let actor = PanicOnStop.start(); + let actor = PanicOnStop.spawn(); actor.send(StopMeThread).unwrap(); actor.join(); }, @@ -280,7 +280,7 @@ fn recipient_type_erasure() { crate::test_wait::bounded( concat!(module_path!(), "::recipient_type_erasure"), move || { - let actor = Counter { count: 42 }.start(); + let actor = Counter { count: 42 }.spawn(); let recipient: Recipient = actor.recipient(); let result = request(&*recipient, GetCount, Duration::from_secs(5)).unwrap(); assert_eq!(result, 42); @@ -293,7 +293,7 @@ fn send_message_on_delivers() { crate::test_wait::bounded( concat!(module_path!(), "::send_message_on_delivers"), move || { - let actor = Counter { count: 0 }.start(); + let actor = Counter { count: 0 }.spawn(); let ctx = actor.context(); send_message_on(ctx, || rt::sleep(Duration::from_millis(10)), Increment); let count = @@ -311,7 +311,7 @@ fn exit_reason_normal_on_clean_stop() { crate::test_wait::bounded( concat!(module_path!(), "::exit_reason_normal_on_clean_stop"), move || { - let actor = Counter { count: 0 }.start(); + let actor = Counter { count: 0 }.spawn(); actor.request(StopCounter).unwrap(); let reason = actor.wait_exit(); assert!(matches!(reason, ExitReason::Normal)); @@ -345,7 +345,7 @@ fn exit_reason_panic_in_started() { } } - let actor = PanicStartThread.start(); + let actor = PanicStartThread.spawn(); let reason = actor.wait_exit(); assert!( matches!(reason, ExitReason::Panic(ref msg) if msg.contains("boom in started")) @@ -378,7 +378,7 @@ fn exit_reason_panic_in_handler() { } } - let actor = PanicHandlerThread.start(); + let actor = PanicHandlerThread.spawn(); let _ = actor.send(ExplodeThread2); let reason = actor.wait_exit(); assert!( @@ -396,7 +396,7 @@ fn exit_reason_poll_returns_none_while_running() { "::exit_reason_poll_returns_none_while_running" ), move || { - let actor = Counter { count: 0 }.start(); + let actor = Counter { count: 0 }.spawn(); assert!(actor.exit_reason().is_none()); actor.request(StopCounter).unwrap(); actor.join(); @@ -495,7 +495,7 @@ fn make_watcher() -> ActorRef { Watcher { downs: Arc::new(Mutex::new(Vec::new())), } - .start() + .spawn() } #[test] @@ -506,7 +506,7 @@ fn monitor_running_actor_delivers_down_on_exit() { "::monitor_running_actor_delivers_down_on_exit" ), move || { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); let target_handle = target.child_handle(); let watcher = make_watcher(); @@ -534,7 +534,7 @@ fn demonitor_flushes_a_queued_down() { crate::test_wait::bounded( concat!(module_path!(), "::demonitor_flushes_a_queued_down"), move || { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); target.request(StopCounter).unwrap(); target.join(); let watcher = make_watcher(); @@ -556,7 +556,7 @@ fn monitor_already_dead_actor_delivers_down_immediately() { "::monitor_already_dead_actor_delivers_down_immediately" ), move || { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); target.request(StopCounter).unwrap(); target.join(); let target_handle = target.child_handle(); @@ -581,7 +581,7 @@ fn demonitor_before_target_dies_suppresses_down() { "::demonitor_before_target_dies_suppresses_down" ), move || { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); let target_handle = target.child_handle(); let watcher = make_watcher(); @@ -606,7 +606,7 @@ fn multiple_monitors_each_get_own_ref_and_down() { "::multiple_monitors_each_get_own_ref_and_down" ), move || { - let target = Counter { count: 0 }.start(); + let target = Counter { count: 0 }.spawn(); let target_handle = target.child_handle(); let watcher = make_watcher(); @@ -655,7 +655,7 @@ fn monitor_observes_panic_reason() { } } - let target = PanicMe.start(); + let target = PanicMe.spawn(); let target_handle = target.child_handle(); let watcher = make_watcher(); @@ -726,7 +726,7 @@ fn make_trapper(trap: bool) -> ActorRef { exits: Arc::new(Mutex::new(Vec::new())), trap, } - .start() + .spawn() } #[test] @@ -842,8 +842,12 @@ fn start_linked_threads() { struct Child; impl Actor for Child {} - let parent = Parent.start(); - let child = Child.start_linked(&parent.context()); + let parent = Parent.spawn(); + let child = Child + .start_link(&parent.context()) + .expect("start_link") + .started() + .expect("ready"); let _ = parent.send(PanicParent); parent.wait_exit(); diff --git a/concurrency/src/threads/jobs.rs b/concurrency/src/threads/jobs.rs index 19a85441f..1e712f14c 100644 --- a/concurrency/src/threads/jobs.rs +++ b/concurrency/src/threads/jobs.rs @@ -700,7 +700,7 @@ mod tests { impl Actor for One { const MAX_NATIVE_JOBS: Option = Some(1); } - let owner = One.start(); + let owner = One.spawn(); let ctx = owner.context(); for i in 0..2000 { let permit = ctx @@ -729,7 +729,7 @@ mod tests { #[test] fn refused_job_payload_panics_are_contained() { - let owner = Owner.start(); + let owner = Owner.spawn(); let ctx = owner.context(); let failures = ctx.shared.sender.failures(); for finish in [false, true] { @@ -772,7 +772,7 @@ mod tests { #[test] fn held_job_result_panics_do_not_interrupt_cancellation() { for action in ["cancel", "deadline", "replace", "owner_exit"] { - let owner = Owner.start(); + let owner = Owner.spawn(); let ctx = owner.context(); let failures = ctx.shared.sender.failures(); let set = crate::reply::jobs::JobSet::new(owner.id(), 1); @@ -843,7 +843,7 @@ mod tests { crate::test_wait::bounded( concat!(module_path!(), "::a_refused_emit_returns_its_credit_once"), move || { - let owner = Owner.start(); + let owner = Owner.spawn(); for credits in [1, u16::MAX] { let state = JobState::new(owner.id(), credits); let job = JobCtx { @@ -872,7 +872,7 @@ mod tests { "::cancel_on_caller_gone_on_a_plain_job_is_a_bug" ), move || { - let owner = Owner.start(); + let owner = Owner.spawn(); let ctx = owner.context(); let refused = std::panic::catch_unwind(AssertUnwindSafe(|| { ctx.reserve_blocking_job(JobOptions::new().cancel_on_caller_gone()) diff --git a/concurrency/src/threads/mod.rs b/concurrency/src/threads/mod.rs index 37f44979f..95e9df5d7 100644 --- a/concurrency/src/threads/mod.rs +++ b/concurrency/src/threads/mod.rs @@ -16,10 +16,11 @@ pub use crate::reply::{ ReplyReceiver, ReplyTo, }; pub use crate::response::Response; +pub use crate::start::{StartError, StartOutcome, Timeout}; pub use crate::Instant; pub use actor::{ - request, send_message_on, Actor, ActorRef, ActorStart, Context, Handler, Receiver, Recipient, - DEFAULT_REQUEST_TIMEOUT, + request, send_message_on, Actor, ActorRef, ActorStart, Context, Handler, Init, Name, Receiver, + Recipient, StartOptions, Starter, DEFAULT_REQUEST_TIMEOUT, }; pub use actor::{BlockingJobPermit, JobCtx, DEFAULT_STOP_DEADLINE}; pub use stream::spawn_listener; diff --git a/concurrency/src/threads/stream_tests.rs b/concurrency/src/threads/stream_tests.rs index e16004936..505e278bb 100644 --- a/concurrency/src/threads/stream_tests.rs +++ b/concurrency/src/threads/stream_tests.rs @@ -61,7 +61,7 @@ fn listener_consumes_iterator() { items: vec![], done: done.clone(), } - .start(); + .spawn(); let ctx = actor.context(); let items = vec![ @@ -94,7 +94,7 @@ fn listener_stops_on_cancellation() { items: vec![], done: done.clone(), } - .start(); + .spawn(); let ctx = actor.context(); @@ -137,7 +137,7 @@ fn stopped_actor_does_not_poll_iterator() { items: vec![], done: Default::default(), } - .start(); + .spawn(); actor.child_handle().stop(); actor.join(); let count = polls.clone(); diff --git a/concurrency/src/threads/timer_tests.rs b/concurrency/src/threads/timer_tests.rs index 765651599..1c081eef0 100644 --- a/concurrency/src/threads/timer_tests.rs +++ b/concurrency/src/threads/timer_tests.rs @@ -87,7 +87,7 @@ pub fn test_send_interval_and_cancellation() { crate::test_wait::bounded( concat!(module_path!(), "::test_send_interval_and_cancellation"), move || { - let repeater = Repeater::new(0).start(); + let repeater = Repeater::new(0).spawn(); // The interval keeps firing. let count = wait_for( @@ -183,7 +183,7 @@ pub fn test_send_after_and_cancellation() { crate::test_wait::bounded( concat!(module_path!(), "::test_send_after_and_cancellation"), move || { - let actor = Delayed::new(0).start(); + let actor = Delayed::new(0).spawn(); let ctx = actor.context(); let _ = send_after(Duration::from_millis(100), ctx, Inc); @@ -213,7 +213,7 @@ pub fn test_send_after_actor_shutdown() { crate::test_wait::bounded( concat!(module_path!(), "::test_send_after_actor_shutdown"), move || { - let actor = Delayed::new(0).start(); + let actor = Delayed::new(0).spawn(); let ctx = actor.context(); let _ = send_after(Duration::from_millis(100), ctx, Inc); @@ -300,7 +300,7 @@ pub fn threads_send_after_uses_the_monotonic_clock_over_long_delays() { handled: handled.clone(), stall: None, } - .start(); + .spawn(); let delay = Duration::from_millis(300); let long = Duration::from_millis(1_500); let armed = std::time::Instant::now(); @@ -356,7 +356,7 @@ pub fn threads_send_interval_is_fixed_delay_while_the_actor_stalls() { handled: handled.clone(), stall: Some(period * 7 / 2), } - .start(); + .spawn(); let timer = send_interval(period, actor.context(), Stamp(sends.clone())); assert!( wait_for(|| handled.lock().unwrap().len(), |n| *n >= 10) >= 10, @@ -395,7 +395,7 @@ pub fn threads_timers_release_their_stop_token_callbacks_while_the_actor_runs() handled: handled.clone(), stall: None, } - .start(); + .spawn(); let token = actor.context().cancellation_token(); let baseline = token.callbacks_registered(); let fired = 50; diff --git a/concurrency/tests/actor_scenarios.rs b/concurrency/tests/actor_scenarios.rs index d88414da5..edcc539e4 100644 --- a/concurrency/tests/actor_scenarios.rs +++ b/concurrency/tests/actor_scenarios.rs @@ -132,12 +132,27 @@ impl Handle { use spawned_concurrency::threads::ActorStart as _; match _backend { #[cfg(feature = "async")] - 0 => Self::Tasks(tasks::TestActor(worker).start_with_backend(Backend::Async)), + 0 => Self::Tasks( + tasks::TestActor(worker) + .with_options(Backend::Async) + .spawn() + .unwrap(), + ), #[cfg(feature = "async")] - 1 => Self::Tasks(tasks::TestActor(worker).start_with_backend(Backend::Blocking)), + 1 => Self::Tasks( + tasks::TestActor(worker) + .with_options(Backend::Blocking) + .spawn() + .unwrap(), + ), #[cfg(feature = "async")] - 2 => Self::Tasks(tasks::TestActor(worker).start_with_backend(Backend::Thread)), - _ => Self::Threads(threads::TestActor(worker).start()), + 2 => Self::Tasks( + tasks::TestActor(worker) + .with_options(Backend::Thread) + .spawn() + .unwrap(), + ), + _ => Self::Threads(threads::TestActor(worker).spawn()), } } fn child(&self) -> ChildHandle { diff --git a/concurrency/tests/backend_resource_limits.rs b/concurrency/tests/backend_resource_limits.rs index 9adf35f10..4adaa693f 100644 --- a/concurrency/tests/backend_resource_limits.rs +++ b/concurrency/tests/backend_resource_limits.rs @@ -58,13 +58,19 @@ fn persistent_blocking_actors_queue_until_a_worker_is_released() { // Await both started hooks so the finite pool is occupied deterministically. for _ in 0..2 { let (tx, rx) = tokio::sync::oneshot::channel(); - let actor = Worker(Some(tx)).start_with_backend(Backend::Blocking); + let actor = Worker(Some(tx)) + .with_options(Backend::Blocking) + .spawn() + .unwrap(); cleanup.0.push(actor.child_handle()); tokio::time::timeout(LIMIT, rx).await.unwrap().unwrap(); actors.push(actor); } let (tx, mut started) = tokio::sync::oneshot::channel(); - let queued = Worker(Some(tx)).start_with_backend(Backend::Blocking); + let queued = Worker(Some(tx)) + .with_options(Backend::Blocking) + .spawn() + .unwrap(); cleanup.0.push(queued.child_handle()); assert!(matches!( started.try_recv(), diff --git a/concurrency/tests/borrowed_runtime.rs b/concurrency/tests/borrowed_runtime.rs index ba513bbd2..ad160caec 100644 --- a/concurrency/tests/borrowed_runtime.rs +++ b/concurrency/tests/borrowed_runtime.rs @@ -36,7 +36,7 @@ impl Handler for Worker { reply: spawned_concurrency::PendingReply<'_, ActorRef>, _: &Context, ) { - reply.reply(Worker.start()); + reply.reply(Worker.spawn()); } } @@ -58,11 +58,13 @@ fn borrowed_groups_join_owned_work_and_leave_the_runtime_and_ambient_tasks_alive system.async_group_on("borrowed", runtime.handle()), Err(SystemError::DuplicateName(_)) )); - let actor = Worker.start_on(&group).unwrap(); + let actor = Worker.with_options(&group).spawn().unwrap(); tokio::time::timeout(LIMIT, actor.ready()) .await .unwrap() - .unwrap(); + .unwrap() + .started() + .expect("started"); if current_thread { assert_eq!( actor.request(Identity).await.unwrap(), @@ -93,7 +95,10 @@ fn borrowed_groups_join_owned_work_and_leave_the_runtime_and_ambient_tasks_alive .unwrap() .unwrap(); assert!(report.wait(LIMIT).await.complete); - assert!(matches!(Worker.start_on(&group), Err(SystemError::Closed))); + assert!(matches!( + Worker.with_options(&group).spawn(), + Err(spawned_concurrency::StartError::Closed) + )); // Ambient work spawned from the managed handler was never adopted. tokio::time::timeout(LIMIT, ambient.request(Identity)) .await diff --git a/concurrency/tests/canary_feedback.rs b/concurrency/tests/canary_feedback.rs index 2ef391fda..131e70a91 100644 --- a/concurrency/tests/canary_feedback.rs +++ b/concurrency/tests/canary_feedback.rs @@ -119,8 +119,12 @@ async fn unfinished_report_identifies_child_and_active_turn() { async fn completed_turn_is_not_reported_while_helper_cleanup_remains() { let system = ActorSystem::new().unwrap(); let group = system.async_group("canary-helper", 1).unwrap(); - let actor = Worker.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Worker.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); actor.context().register_name("canary-standalone").unwrap(); let (entered, arrival) = oneshot::channel(); let (release, gate) = std::sync::mpsc::channel(); @@ -381,7 +385,7 @@ async fn supervision_logs_restart_budget_exhaustion_and_give_up() { #[spawned_macros::sim_test] async fn unmanaged_helpers_return_a_specific_error_without_executing_work() { - let actor = Worker.start(); + let actor = Worker.spawn(); assert!(matches!( actor.context().spawn_helper(async {}), Err(spawned_concurrency::system::SystemError::UnmanagedActor) @@ -428,9 +432,14 @@ async fn panicked_turn_is_cleared_before_stopped_hook() { entered: Some(entered), release: Some(gate), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - within!(actor.ready()).await.unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); assert!(actor.request(Quit(ExitReason::Kill)).await.is_err()); spawned_rt::tasks::timeout(LIMIT, arrival) .await @@ -478,11 +487,20 @@ async fn trapped_exit_diagnostics_use_the_public_signal_name() { entered: Some(entered), release: Some(gate), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - within!(observer.ready()).await.unwrap(); - let source = Worker.start_on(&group).unwrap(); - within!(source.ready()).await.unwrap(); + within!(observer.ready()) + .await + .unwrap() + .started() + .expect("started"); + let source = Worker.with_options(&group).spawn().unwrap(); + within!(source.ready()) + .await + .unwrap() + .started() + .expect("started"); observer.context().link(&source.child_handle()); source .child_handle() diff --git a/concurrency/tests/cancellation_safety.rs b/concurrency/tests/cancellation_safety.rs index adbe5342d..d4922ad9d 100644 --- a/concurrency/tests/cancellation_safety.rs +++ b/concurrency/tests/cancellation_safety.rs @@ -322,12 +322,13 @@ impl Fx { } fn spawn(&self, probe: Probe) -> ActorRef { match self.place { - Place::Async => probe.start_with_backend(Backend::Async), - Place::Blocking => probe.start_with_backend(Backend::Blocking), - Place::Thread => probe.start_with_backend(Backend::Thread), - Place::GroupAsync | Place::GroupThread => { - self.group.as_ref().unwrap().start(probe).unwrap() - } + Place::Async => probe.with_options(Backend::Async).spawn().unwrap(), + Place::Blocking => probe.with_options(Backend::Blocking).spawn().unwrap(), + Place::Thread => probe.with_options(Backend::Thread).spawn().unwrap(), + Place::GroupAsync | Place::GroupThread => probe + .with_options(self.group.as_ref().unwrap()) + .spawn() + .unwrap(), } } fn probe(&self) -> Probe { @@ -565,7 +566,9 @@ async fn group_ready_wait_is_cancel_safe() { spawned_rt::tasks::timeout(LIMIT, actor.ready()) .await .unwrap() - .unwrap(); + .unwrap() + .started() + .expect("started"); assert!(registry::registered().contains(&name)); usable(&actor).await; fx.finish(&[&actor]).await; @@ -775,7 +778,11 @@ async fn system_shutdown_is_cancel_safe() { let mut probe = fx.probe(); probe.name = Some(name.clone()); let actor = fx.spawn(probe); - within!(actor.ready()).await.unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let helper_gate = Arc::new(Notify::new()); actor.request(Cmd::Helper(helper_gate)).await.unwrap(); let system = fx.system.as_ref().unwrap(); @@ -811,7 +818,11 @@ async fn system_shutdown_is_cancel_safe() { let mut probe = fx.probe(); probe.stop_gate = Some(gate.clone()); let actor = fx.spawn(probe); - within!(actor.ready()).await.unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); // A short grace, not zero: the trapping probe must dispatch the // parent's exit and enter stopped() before the kill at grace // (E3). With zero grace the kill can land first and skip @@ -850,7 +861,11 @@ async fn helper_join_is_cancel_safe() { async move { let fx = Fx::new(place); let actor = fx.spawn(fx.probe()); - within!(actor.ready()).await.unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let gate = Arc::new(Notify::new()); let waiting = gate.clone(); let task = actor diff --git a/concurrency/tests/cfg_gated_protocol.rs b/concurrency/tests/cfg_gated_protocol.rs index 8134f5430..225b8eedb 100644 --- a/concurrency/tests/cfg_gated_protocol.rs +++ b/concurrency/tests/cfg_gated_protocol.rs @@ -57,7 +57,7 @@ impl Handler for MixedActor { fn cfg_gated_method_works_via_blanket_impl() { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async { - let actor = MixedActor { value: 10 }.start(); + let actor = MixedActor { value: 10 }.spawn(); let val = actor.get_value().await.unwrap(); assert_eq!(val, 10); @@ -80,7 +80,7 @@ fn cfg_gated_method_works_via_blanket_impl() { fn cfg_gated_method_works_via_protocol_ref() { let runtime = rt::Runtime::new().unwrap(); runtime.block_on(async { - let actor = MixedActor { value: 0 }.start(); + let actor = MixedActor { value: 0 }.spawn(); let proto_ref: MixedRef = actor.to_mixed_ref(); let val = proto_ref.get_value().await.unwrap(); diff --git a/concurrency/tests/context_exit.rs b/concurrency/tests/context_exit.rs index 70a9fd781..e87e0d4e4 100644 --- a/concurrency/tests/context_exit.rs +++ b/concurrency/tests/context_exit.rs @@ -33,7 +33,7 @@ fn native_explicit_exit_is_cooperative_and_first_reason_wins() { } for reason in [ExitReason::Shutdown(None), ExitReason::Kill] { let cleaned = Arc::new(AtomicBool::new(false)); - let actor = Worker(cleaned.clone()).start(); + let actor = Worker(cleaned.clone()).spawn(); actor.request(Exit(reason.clone())).unwrap(); actor.join(); assert_eq!(actor.child_handle().exit_reason(), Some(reason)); @@ -69,7 +69,7 @@ async fn async_explicit_exit_is_cooperative_and_first_reason_wins() { } for reason in [ExitReason::Shutdown(None), ExitReason::Kill] { let cleaned = Arc::new(AtomicBool::new(false)); - let actor = Worker(cleaned.clone()).start(); + let actor = Worker(cleaned.clone()).spawn(); actor.request(Exit(reason.clone())).await.unwrap(); spawned_rt::tasks::timeout(std::time::Duration::from_secs(5), actor.join()) .await diff --git a/concurrency/tests/coordinator_trace.rs b/concurrency/tests/coordinator_trace.rs index 7da893a4c..95cbfdddd 100644 --- a/concurrency/tests/coordinator_trace.rs +++ b/concurrency/tests/coordinator_trace.rs @@ -76,16 +76,24 @@ async fn trace_coordinator_ordered_shutdown() { let _serial = SERIAL.lock().await; let system = ActorSystem::new().unwrap(); let group = system.async_group("coordinator-ordered", 2).unwrap(); - let first = Worker.start_on(&group).unwrap(); - within!(first.ready()).await.unwrap(); + let first = Worker.with_options(&group).spawn().unwrap(); + within!(first.ready()) + .await + .unwrap() + .started() + .expect("started"); let mut tree = system.supervisor(SupervisorConfig::default()).unwrap(); tree.add(ChildSpec::new("child", &group, |_| async { Ok::<_, String>(Worker) })) .unwrap(); let supervisor = within!(tree.start()).await.unwrap(); - let last = Worker.start_on(&group).unwrap(); - within!(last.ready()).await.unwrap(); + let last = Worker.with_options(&group).spawn().unwrap(); + within!(last.ready()) + .await + .unwrap() + .started() + .expect("started"); assert!(system.shutdown(LIMIT).await.complete); assert_eq!( write_trace("ordered", &system, 3), @@ -105,8 +113,12 @@ async fn trace_coordinator_grace_expiry_kills_remaining() { let _serial = SERIAL.lock().await; let system = ActorSystem::new().unwrap(); let group = system.async_group("coordinator-grace", 1).unwrap(); - let first = Worker.start_on(&group).unwrap(); - within!(first.ready()).await.unwrap(); + let first = Worker.with_options(&group).spawn().unwrap(); + within!(first.ready()) + .await + .unwrap() + .started() + .expect("started"); let (entered_tx, entered) = tokio::sync::oneshot::channel(); let (release, release_rx) = tokio::sync::oneshot::channel(); struct Stuck { @@ -126,9 +138,14 @@ async fn trace_coordinator_grace_expiry_kills_remaining() { entered: Some(entered_tx), release: Some(release_rx), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - within!(last.ready()).await.unwrap(); + within!(last.ready()) + .await + .unwrap() + .started() + .expect("started"); let report = system.shutdown(Duration::from_millis(20)).await; assert!(!report.complete); within!(entered).await.unwrap(); @@ -169,8 +186,12 @@ async fn trace_coordinator_panic_takes_emergency_fallback() { let group = system.async_group("coordinator-crash", 1).unwrap(); let mut actors = Vec::new(); for _ in 0..3 { - let actor = Worker.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Worker.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); actors.push(actor); } failpoints::configure( diff --git a/concurrency/tests/cross_backend.rs b/concurrency/tests/cross_backend.rs index 3eb77bb5b..e209a606b 100644 --- a/concurrency/tests/cross_backend.rs +++ b/concurrency/tests/cross_backend.rs @@ -619,7 +619,7 @@ impl Env { peers, interval: None, } - .start(), + .spawn(), ) } else { let peers = peers @@ -636,14 +636,19 @@ impl Env { }; let _entered = handle.enter(); let actor = match self.placement { - Placement::TasksAsync => node.start_with_backend(Backend::Async), - Placement::TasksBlocking => node.start_with_backend(Backend::Blocking), - Placement::TasksThread => node.start_with_backend(Backend::Thread), + Placement::TasksAsync => node.with_options(Backend::Async).spawn().unwrap(), + Placement::TasksBlocking => node.with_options(Backend::Blocking).spawn().unwrap(), + Placement::TasksThread => node.with_options(Backend::Thread).spawn().unwrap(), _ => { - let actor = self.group.as_ref().unwrap().start(node).unwrap(); + let actor = node + .with_options(self.group.as_ref().unwrap()) + .spawn() + .unwrap(); self.bounded(actor.ready()) .expect("managed actor readiness timed out") - .unwrap(); + .unwrap() + .started() + .expect("started"); actor } }; @@ -1387,12 +1392,12 @@ mod early_stop { } } -/// The intended contract for an actor stopped before its first poll (GUIDE.md, -/// "Startup"): a managed actor whose admission closes before `started()` begins -/// skips both hooks and drops its state; legacy unmanaged tasks actors and -/// threads actors retain their existing behavior of running `started()` then -/// `stopped()`. All report Normal and every actor's state is dropped exactly -/// once. Listed as an allowed difference in TESTING.md. +/// The contract for an actor stopped (`ChildHandle::stop`) before its first +/// poll, on every placement: as OTP's `gen_server:stop`, the stop waits for +/// `started()`; the start commits ready, then the actor exits `Normal` +/// through `stopped()` (v0.9, unified start 2.3). Only an admission that +/// closes before `started()` begins (group or system shutdown) skips both +/// hooks (`startup_admission`). Every state is dropped exactly once. // Not simulated: compares Tokio, Thread, Blocking and native placements. #[cfg(not(spawned_sim))] #[test] @@ -1406,7 +1411,7 @@ fn stop_before_first_poll_follows_managed_and_legacy_contract() { let hooks = Recorder::new(); let actor = { let _entered = rt.enter(); - spawned_concurrency::tasks::ActorStart::start(Hooks(hooks.clone())) + spawned_concurrency::tasks::ActorStart::spawn(Hooks(hooks.clone())) }; actor.child_handle().stop(); let reason = rt.block_on(async { tokio::time::timeout(LIMIT, actor.wait_exit()).await }); @@ -1417,13 +1422,13 @@ fn stop_before_first_poll_follows_managed_and_legacy_contract() { // Legacy threads mode: no admission check either. let hooks = Recorder::new(); - let actor = spawned_concurrency::threads::ActorStart::start(Hooks(hooks.clone())); + let actor = spawned_concurrency::threads::ActorStart::spawn(Hooks(hooks.clone())); actor.child_handle().stop(); assert_eq!(actor.wait_exit(), ExitReason::Normal); assert_eq!(hooks.wait_until(LIMIT, "threads drop", done), legacy); // Managed: the group's single worker is blocked, so the actor is admitted - // but not yet polled when it is stopped. Neither hook runs; state drops. + // but not yet polled when it is stopped. The stop waits for started(). let rt = tokio::runtime::Builder::new_multi_thread() .worker_threads(1) .enable_all() @@ -1432,10 +1437,15 @@ fn stop_before_first_poll_follows_managed_and_legacy_contract() { let system = ActorSystem::new().unwrap(); let group = system.async_group("early-stop", 1).unwrap(); let blocker_hooks = Recorder::new(); - let blocker = group.start(Hooks(blocker_hooks.clone())).unwrap(); + let blocker = + spawned_concurrency::tasks::ActorStart::with_options(Hooks(blocker_hooks.clone()), &group) + .spawn() + .unwrap(); rt.block_on(async { tokio::time::timeout(LIMIT, blocker.ready()).await }) .unwrap() - .unwrap(); + .unwrap() + .started() + .expect("started"); let (release, wait) = std::sync::mpsc::channel(); let (entered, blocked) = std::sync::mpsc::channel(); blocker @@ -1443,12 +1453,14 @@ fn stop_before_first_poll_follows_managed_and_legacy_contract() { .unwrap(); blocked.recv_timeout(LIMIT).unwrap(); let hooks = Recorder::new(); - let actor = group.start(Hooks(hooks.clone())).unwrap(); + let actor = spawned_concurrency::tasks::ActorStart::with_options(Hooks(hooks.clone()), &group) + .spawn() + .unwrap(); actor.child_handle().stop(); release.send(()).unwrap(); let reason = rt.block_on(async { tokio::time::timeout(LIMIT, actor.wait_exit()).await }); assert_eq!(reason.unwrap(), ExitReason::Normal); - assert_eq!(hooks.wait_until(LIMIT, "managed drop", done), ["dropped"]); + assert_eq!(hooks.wait_until(LIMIT, "managed drop", done), legacy); let report = rt.block_on(system.shutdown(LIMIT)); assert!(report.complete); } diff --git a/concurrency/tests/deadline_service.rs b/concurrency/tests/deadline_service.rs index 63d0c6f70..eae42e651 100644 --- a/concurrency/tests/deadline_service.rs +++ b/concurrency/tests/deadline_service.rs @@ -72,7 +72,7 @@ fn deadline_service_child() { .build() .unwrap(); runtime.block_on(async { - let owner = Owner::default().start(); + let owner = Owner::default().spawn(); spawned_concurrency::__private::in_turn(&owner, |_, ctx| { let opts = JobOptions::new().deadline(Duration::from_millis(20)); ctx.pipe_blocking( @@ -94,7 +94,7 @@ fn deadline_service_child() { tokio::time::sleep(Duration::from_millis(100)).await; // Another actor's deadlines, serviced by the same thread. - let other = Owner::default().start(); + let other = Owner::default().spawn(); let parked = other .send_request(Park, Deadline::after(Duration::from_millis(50))) .unwrap(); @@ -223,7 +223,7 @@ fn deadline_service_isolates_a_blocking_destructor_threads() { /// Another actor's raw request and job deadlines, serviced by the heap. async fn other_actor_deadlines_fire(blocker: &Blocker) -> Result<(), String> { - let other = Owner::default().start(); + let other = Owner::default().spawn(); let parked = other .send_request(Park, Deadline::after(Duration::from_millis(50))) .unwrap(); @@ -281,7 +281,7 @@ fn deadline_service_blocking_destructor_child() { } let blocker = std::sync::Arc::new(Blocker::default()); tokio_runtime().block_on(async { - let owner = Owner::default().start(); + let owner = Owner::default().spawn(); let armed = blocker.clone(); spawned_concurrency::__private::in_turn(&owner, move |_, ctx| { let opts = JobOptions::new().deadline(Duration::from_millis(20)); @@ -332,7 +332,7 @@ fn deadline_service_same_actor_child() { } let blocker = std::sync::Arc::new(Blocker::default()); tokio_runtime().block_on(async { - let owner = Owner::default().start(); + let owner = Owner::default().spawn(); let deadline = Deadline::after(Duration::from_millis(100)); let first = owner.send_request(Park, deadline).unwrap(); let second = owner.send_request(Park, deadline).unwrap(); @@ -373,7 +373,7 @@ fn deadline_service_blocking_waker_child() { } let blocker = std::sync::Arc::new(Blocker::default()); tokio_runtime().block_on(async { - let owner = Owner::default().start(); + let owner = Owner::default().spawn(); // A raw receiver polled once with a waker that blocks: its deadline // goes to the owner's heap, whose expiry wakes it. let receiver = owner @@ -414,7 +414,7 @@ mod threads_backend { } pub fn run(blocker: &std::sync::Arc) -> Result<(), String> { - let owner = Owner.start(); + let owner = Owner.spawn(); let armed = blocker.clone(); spawned_concurrency::__private::in_turn_threads(&owner, move |_, ctx| { let opts = JobOptions::new().deadline(Duration::from_millis(20)); @@ -437,7 +437,7 @@ mod threads_backend { "the scheduled emit was never discarded" ); // Another actor's job deadline, serviced by the same deadline service. - let other = Owner.start(); + let other = Owner.spawn(); let fired = spawned_concurrency::__private::in_turn_threads(&other, |_, ctx| { let (tx, rx) = std::sync::mpsc::channel(); let opts = JobOptions::new().deadline(Duration::from_millis(20)); diff --git a/concurrency/tests/death_boundary.rs b/concurrency/tests/death_boundary.rs index 7d10bf46a..6b5d85e3a 100644 --- a/concurrency/tests/death_boundary.rs +++ b/concurrency/tests/death_boundary.rs @@ -172,7 +172,7 @@ mod tasks_mode { _state: inert(), parked: Vec::new(), } - .start(); + .spawn(); let caller = Arc::new(Mutex::new(None)); let answered = Arc::new(AtomicBool::new(false)); let payload = WaitsForAnswer { @@ -270,7 +270,7 @@ mod tasks_mode { _state: inert(), parked: Vec::new(), } - .start(); + .spawn(); let turn_caller = actor .send_request(Hang(payload, entered_tx), never) .unwrap(); @@ -331,7 +331,7 @@ mod tasks_mode { _state: inert(), parked: Vec::new(), } - .start(); + .spawn(); let (unblock, blocked) = mpsc::channel(); let (entered_tx, entered) = mpsc::channel(); actor.send(Block(blocked, entered_tx)).unwrap(); @@ -394,7 +394,7 @@ mod tasks_mode { _state: inert(), parked: Vec::new(), } - .start(); + .spawn(); let (tokens, token) = mpsc::channel(); let receiver = actor .send_request(HandOut(tokens), spawned_concurrency::Deadline::Never) @@ -488,7 +488,7 @@ mod tasks_mode { _state: inert(), parked: Vec::new(), } - .start(); + .spawn(); let (export, exported) = mpsc::channel(); runtime.block_on(actor.request(ExportJob(export))).unwrap(); let job = exported.recv_timeout(LIMIT).unwrap(); @@ -556,7 +556,7 @@ mod tasks_mode { _state: inert(), parked: Vec::new(), } - .start(); + .spawn(); let (unblock, blocked) = mpsc::channel(); let (entered_tx, entered) = mpsc::channel(); actor.send(Block(blocked, entered_tx)).unwrap(); @@ -606,7 +606,7 @@ mod tasks_mode { _state: state, parked: Vec::new(), } - .start() + .spawn() }; let mut queued = actor .send_request( @@ -640,7 +640,7 @@ mod tasks_mode { _state: state, parked: Vec::new(), } - .start(); + .spawn(); let parked = actor .send_request(Park, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -690,7 +690,7 @@ mod threads_mode { use spawned_concurrency::threads::JobOptions; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::{Arc, Mutex}; - let actor = Owner.start(); + let actor = Owner.spawn(); let caller = Arc::new(Mutex::new(None)); let answered = Arc::new(AtomicBool::new(false)); let payload = WaitsForAnswer { @@ -734,7 +734,7 @@ mod threads_mode { /// Review P0 (Codex 1), threads mode. #[test] fn queued_requests_are_answered_before_payload_destructors() { - let actor = Owner.start(); + let actor = Owner.spawn(); let (unblock, blocked) = mpsc::channel(); let (entered_tx, entered) = mpsc::channel(); actor.send(Block(blocked, entered_tx)).unwrap(); @@ -851,7 +851,7 @@ mod subscriber_after_discharge { // Never polled: dropping the runtime drops it before `stopped()`. let actor = { let _guard = runtime.enter(); - Idle.start() + Idle.spawn() }; let queued = actor .send_request(Queued(inert()), spawned_concurrency::Deadline::Never) diff --git a/concurrency/tests/dedicated_runtime.rs b/concurrency/tests/dedicated_runtime.rs index b8a52873f..2bd5526b8 100644 --- a/concurrency/tests/dedicated_runtime.rs +++ b/concurrency/tests/dedicated_runtime.rs @@ -182,7 +182,7 @@ impl Handler for Probe { let __reply_value: u64 = { self.assert_root(); // Default Async startup uses this actor's isolated runtime. - let child = Child.start(); + let child = Child.spawn(); let _cleanup = StopOnDrop(child.child_handle()); let answer = child.request_with_timeout(Ping, LIMIT).await.unwrap(); child.child_handle().stop(); @@ -258,7 +258,10 @@ async fn finish( fn dedicated_actor_and_helpers_progress_without_an_ambient_runtime() { assert!(tokio::runtime::Handle::try_current().is_err()); let (stopped, stop_rx) = oneshot::channel(); - let actor = Probe::new(stopped).start_with_backend(Backend::Thread); + let actor = Probe::new(stopped) + .with_options(Backend::Thread) + .spawn() + .unwrap(); let _cleanup = StopOnDrop(actor.child_handle()); observer().block_on(async { let first = actor @@ -285,7 +288,10 @@ fn dedicated_actor_and_helpers_progress_without_an_ambient_runtime() { fn dedicated_actor_drives_async_tcp() { use tokio::io::{AsyncReadExt, AsyncWriteExt}; let (stopped, stop_rx) = oneshot::channel(); - let actor = Probe::new(stopped).start_with_backend(Backend::Thread); + let actor = Probe::new(stopped) + .with_options(Backend::Thread) + .spawn() + .unwrap(); let _cleanup = StopOnDrop(actor.child_handle()); observer().block_on(async { let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); @@ -317,7 +323,10 @@ fn dedicated_actor_outlives_its_callers_runtime() { let (stopped, stop_rx) = oneshot::channel(); let caller = observer(); let (actor, root, _cleanup) = caller.block_on(async { - let actor = Probe::new(stopped).start_with_backend(Backend::Thread); + let actor = Probe::new(stopped) + .with_options(Backend::Thread) + .spawn() + .unwrap(); let cleanup = StopOnDrop(actor.child_handle()); let first = actor .request_with_timeout(Inspect::Yield, LIMIT) diff --git a/concurrency/tests/deferred_api.rs b/concurrency/tests/deferred_api.rs index a09fe8344..4158158cb 100644 --- a/concurrency/tests/deferred_api.rs +++ b/concurrency/tests/deferred_api.rs @@ -87,7 +87,7 @@ mod tasks_mode { fn start() -> (ActorRef, Arc) { let log = Arc::new(Log::default()); - (Owner { log: log.clone() }.start(), log) + (Owner { log: log.clone() }.spawn(), log) } async fn wait_for(log: &Log, n: usize) -> Vec { @@ -495,7 +495,7 @@ mod threads_mode { #[test] fn in_actor_permits_and_emit_at() { let log = Arc::new(Log::default()); - let actor = Owner { log: log.clone() }.start(); + let actor = Owner { log: log.clone() }.spawn(); let refused = in_actor(&actor, |_, ctx| { let permit = ctx .reserve_blocking_job(JobOptions::new()) @@ -523,7 +523,7 @@ mod threads_mode { #[test] fn a_deadline_overtakes_a_result_held_for_scheduled_emits() { let log = Arc::new(Log::default()); - let actor = Owner { log: log.clone() }.start(); + let actor = Owner { log: log.clone() }.spawn(); in_actor(&actor, |_, ctx| { let opts = JobOptions::new().deadline(Duration::from_millis(40)); ctx.pipe_blocking( @@ -549,7 +549,7 @@ mod threads_mode { #[test] fn a_held_result_keeps_its_credit_until_the_terminal_message() { let log = Arc::new(Log::default()); - let actor = Owner { log: log.clone() }.start(); + let actor = Owner { log: log.clone() }.spawn(); let job = in_actor(&actor, |_, ctx| { ctx.pipe_blocking( JobOptions::new(), @@ -581,7 +581,7 @@ mod threads_mode { #[test] fn emit_at_a_past_instant_is_delivered_immediately() { let log = Arc::new(Log::default()); - let actor = Owner { log: log.clone() }.start(); + let actor = Owner { log: log.clone() }.spawn(); in_actor(&actor, |_, ctx| { ctx.pipe_blocking( JobOptions::new(), @@ -603,7 +603,7 @@ mod threads_mode { #[test] fn send_handler_impls_answer_requests_with_unit() { let log = Arc::new(Log::default()); - let actor = Owner { log: log.clone() }.start(); + let actor = Owner { log: log.clone() }.spawn(); assert!(matches!(actor.request(Poke(false)), Ok(()))); assert!(matches!(actor.request(Poke(true)), Ok(()))); assert_eq!(log.snapshot(), ["poked"]); @@ -717,7 +717,7 @@ mod caller_gone_vs_shortening_tasks { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn shortening_the_deadline_does_not_cancel_a_caller_gone_job() { - let actor = Delegator::default().start(); + let actor = Delegator::default().spawn(); let (deadline, cancelled) = actor .request_with_timeout(Ask, Duration::from_secs(60)) .await @@ -842,7 +842,7 @@ mod caller_gone_vs_shortening_threads { #[test] fn shortening_the_deadline_does_not_cancel_a_caller_gone_job() { - let actor = Delegator::default().start(); + let actor = Delegator::default().spawn(); let (deadline, cancelled) = actor .request_with_timeout(Ask, Duration::from_secs(60)) .unwrap(); diff --git a/concurrency/tests/deferred_jobs.rs b/concurrency/tests/deferred_jobs.rs index 9c493c5c0..a47fda3b0 100644 --- a/concurrency/tests/deferred_jobs.rs +++ b/concurrency/tests/deferred_jobs.rs @@ -211,7 +211,7 @@ mod tasks_mode { log: log.clone(), gates: Vec::new(), } - .start(); + .spawn(); (actor, log) } @@ -572,11 +572,11 @@ mod tasks_mode { #[spawned_macros::sim_test] async fn router_delegates_with_pipe_reply_while_serving_its_mailbox() { - let backend = Backend.start(); + let backend = Backend.spawn(); let router = Router { backend: backend.clone(), } - .start(); + .spawn(); let slow = router .send_request(Route(200), spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -592,12 +592,12 @@ mod tasks_mode { #[spawned_macros::sim_test] async fn pipe_reply_refuses_a_token_of_another_actor() { - let backend = Backend.start(); + let backend = Backend.spawn(); let router = Router { backend: backend.clone(), } - .start(); - let holder = Holder(None).start(); + .spawn(); + let holder = Holder(None).spawn(); let rx = holder.send_request(Hold, Deadline::Never).unwrap(); let token = holder.request(Give).await.unwrap().unwrap(); assert!(router.request(Foreign(token)).await.unwrap()); @@ -686,7 +686,7 @@ mod threads_mode { #[test] fn native_jobs_answer_deferred_requests() { let log = Arc::new(Log::default()); - let actor: ActorRef = Owner { log: log.clone() }.start(); + let actor: ActorRef = Owner { log: log.clone() }.spawn(); let rx = actor.send_request(Compute(7), Deadline::Never).unwrap(); assert_eq!(rx.recv().unwrap(), 49); assert!(matches!( diff --git a/concurrency/tests/deferred_replies.rs b/concurrency/tests/deferred_replies.rs index 0c9cfbea6..cf1bf4fc3 100644 --- a/concurrency/tests/deferred_replies.rs +++ b/concurrency/tests/deferred_replies.rs @@ -161,7 +161,7 @@ mod tasks_mode { fn start() -> (spawned_concurrency::tasks::ActorRef, Arc) { let seen = Arc::new(Seen::default()); - (Parker::new(seen.clone()).start(), seen) + (Parker::new(seen.clone()).spawn(), seen) } async fn settle(actor: &spawned_concurrency::tasks::ActorRef) -> usize { @@ -413,7 +413,7 @@ mod tasks_mode { reply.reply(closed); } } - let actor: ActorRef = Watcher { closed: None }.start(); + let actor: ActorRef = Watcher { closed: None }.spawn(); let rx = actor.send_request(Watch, Deadline::Never).unwrap(); actor.request(Ping).await.unwrap(); // `Watch` was handled (FIFO) drop(rx); @@ -462,7 +462,7 @@ mod tasks_mode { self.0 = Some(token); } } - let actor = Shortener(None).start(); + let actor = Shortener(None).spawn(); let began = tokio::time::Instant::now(); let result = actor .request_with_timeout(Shorten, Duration::from_secs(60)) @@ -497,7 +497,7 @@ mod tasks_mode { } // otherwise dropped unanswered } } - let actor = Picky.start(); + let actor = Picky.spawn(); let dropped = actor.request(Ask(false)).await.unwrap_err(); assert!(matches!(dropped, ActorError::ReplyDropped)); assert!(!dropped.is_transient() && !dropped.target_gone()); @@ -525,7 +525,7 @@ mod tasks_mode { async fn reply_pair_drives_a_manual_handler_directly() { let seen = Arc::new(Seen::default()); let mut parker = Parker::new(seen); - let actor = Parker::new(Arc::new(Seen::default())).start(); + let actor = Parker::new(Arc::new(Seen::default())).spawn(); let ctx = actor.context(); let (reply, rx) = spawned_concurrency::__private::reply_pair::(Deadline::Never); parker.handle(Park(5), reply, &ctx).await; @@ -619,7 +619,7 @@ mod threads_mode { answer_in_stopped: false, seen: seen.clone(), } - .start(); + .spawn(); (actor, seen) } diff --git a/concurrency/tests/deferred_sim.rs b/concurrency/tests/deferred_sim.rs index c3b19d472..1e3e5970f 100644 --- a/concurrency/tests/deferred_sim.rs +++ b/concurrency/tests/deferred_sim.rs @@ -50,7 +50,7 @@ fn owner(log: &Log) -> ActorRef { stored: Vec::new(), job: None, } - .start() + .spawn() } struct Hold; diff --git a/concurrency/tests/disposal_trace.rs b/concurrency/tests/disposal_trace.rs index da19314ba..6f1457987 100644 --- a/concurrency/tests/disposal_trace.rs +++ b/concurrency/tests/disposal_trace.rs @@ -169,7 +169,7 @@ async fn stop(owner: ActorRef) { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn expired_callers_are_woken_by_workers() { let _serial = SERIAL.lock().await; - let owner = Owner::default().start(); + let owner = Owner::default().spawn(); record().await; expire_callers(&owner).await; tokio::time::sleep(Duration::from_millis(100)).await; @@ -186,7 +186,7 @@ async fn expired_callers_are_woken_by_workers() { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn released_payloads_are_dropped_by_workers() { let _serial = SERIAL.lock().await; - let owner = Owner::default().start(); + let owner = Owner::default().spawn(); let drops = Arc::new(AtomicUsize::new(0)); record().await; discard_an_emit(&owner, &drops).await; @@ -225,7 +225,7 @@ mod with_failpoints { gate: Some((gate, stopping_tx)), ..Owner::default() } - .start(); + .spawn(); let drops = Arc::new(AtomicUsize::new(0)); record().await; failpoints::configure( @@ -272,7 +272,7 @@ mod with_failpoints { failpoints::configure("disposal.keep_alive", FailPoint::new(Action::Error)).unwrap(); // Let the workers left by earlier scenarios retire first. tokio::time::sleep(Duration::from_millis(100)).await; - let owner = Owner::default().start(); + let owner = Owner::default().spawn(); record().await; expire_callers(&owner).await; tokio::time::sleep(Duration::from_millis(100)).await; @@ -293,7 +293,7 @@ mod with_failpoints { async fn batch_serial_control() { let _serial = SERIAL.lock().await; let _scenario = FailScenario::setup(); - let owner = Owner::default().start(); + let owner = Owner::default().spawn(); record().await; failpoints::configure("disposal.batch_serial", FailPoint::new(Action::Error)).unwrap(); expire_callers(&owner).await; diff --git a/concurrency/tests/execution_groups.rs b/concurrency/tests/execution_groups.rs index b4c3f1606..aa9660e4a 100644 --- a/concurrency/tests/execution_groups.rs +++ b/concurrency/tests/execution_groups.rs @@ -58,10 +58,18 @@ async fn owned_groups_communicate_and_close_with_stale_handles() { let system = ActorSystem::new().unwrap(); let a = system.async_group("critical", 1).unwrap(); let b = system.async_group("general", 2).unwrap(); - let first = Worker.start_on(&a).unwrap(); - let second = Worker.start_on(&b).unwrap(); - within!(first.ready()).await.unwrap(); - within!(second.ready()).await.unwrap(); + let first = Worker.with_options(&a).spawn().unwrap(); + let second = Worker.with_options(&b).spawn().unwrap(); + within!(first.ready()) + .await + .unwrap() + .started() + .expect("started"); + within!(second.ready()) + .await + .unwrap() + .started() + .expect("started"); assert!(first .request(Ping) .await @@ -99,7 +107,10 @@ async fn owned_groups_communicate_and_close_with_stale_handles() { first.child_handle().wait_exit_async().await, ExitReason::Shutdown(None) ); - assert!(matches!(Worker.start_on(&a), Err(SystemError::Closed))); + assert!(matches!( + Worker.with_options(&a).spawn(), + Err(spawned_concurrency::StartError::Closed) + )); assert!(first.request(Ping).await.is_err()); assert!(first.context().spawn_helper(async {}).is_err()); // Legacy infallible helper APIs remain inert from a plain thread after the @@ -144,7 +155,8 @@ async fn expired_shutdown_retains_closed_runtime_until_startup_cleanup_finishes( release: Some(release_rx), stopped: stopped.clone(), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); spawned_rt::tasks::timeout(LIMIT, entered) .await @@ -154,7 +166,7 @@ async fn expired_shutdown_retains_closed_runtime_until_startup_cleanup_finishes( assert!(!report.complete); assert_eq!(report.unfinished_groups, ["slow-start"]); assert!(group.is_closed()); - assert!(Worker.start_on(&group).is_err()); + assert!(Worker.with_options(&group).spawn().is_err()); // The system kills what remains at grace (E3); let the kill land before // started() returns, so the outcome does not race it. spawned_rt::tasks::sleep(Duration::from_millis(100)).await; @@ -202,10 +214,15 @@ async fn shutdown_joins_helper_drops_and_generation_owned_registration() { dropped: dropped.clone(), registered: Some(tx), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); rx.await.unwrap(); - within!(actor.ready()).await.unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); assert!(spawned_concurrency::registry::whereis::< spawned_concurrency::tasks::ActorRef, >("execution-group-owned-helper-test") @@ -226,8 +243,12 @@ async fn dropping_system_requests_shutdown_without_an_ambient_runtime() { assert!(system.async_group("bad", 0).is_err()); let group = system.async_group("drop", 1).unwrap(); assert!(system.async_group("drop", 2).is_err()); - let actor = Worker.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Worker.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); std::thread::spawn(move || drop(system)).join().unwrap(); assert_eq!( spawned_rt::tasks::timeout(LIMIT, actor.child_handle().wait_exit_async()) @@ -243,8 +264,12 @@ async fn dropping_system_requests_shutdown_without_an_ambient_runtime() { async fn registered_blocking_work_keeps_runtime_owned_until_native_completion() { let system = ActorSystem::new().unwrap(); let group = system.async_group("blocking-helper", 1).unwrap(); - let actor = Worker.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Worker.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let (entered_tx, entered) = tokio::sync::oneshot::channel(); let (release, release_rx) = std::sync::mpsc::channel(); let dropped = Arc::new(AtomicUsize::new(0)); @@ -277,8 +302,12 @@ async fn registered_blocking_work_keeps_runtime_owned_until_native_completion() async fn scoped_helpers_report_panics_and_framework_abort_drops_payload() { let system = ActorSystem::new().unwrap(); let group = system.async_group("helper-errors", 1).unwrap(); - let actor = Worker.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Worker.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let async_error = actor .context() .spawn_helper(async { panic!("async helper panic") }) @@ -318,11 +347,15 @@ async fn scoped_helpers_report_panics_and_framework_abort_drops_payload() { async fn ambient_actor_started_inside_group_is_unmanaged_and_aborted_at_teardown() { let system = ActorSystem::new().unwrap(); let group = system.async_group("ambient-child", 1).unwrap(); - let managed = Worker.start_on(&group).unwrap(); - within!(managed.ready()).await.unwrap(); + let managed = Worker.with_options(&group).spawn().unwrap(); + within!(managed.ready()) + .await + .unwrap() + .started() + .expect("started"); let ambient = managed .context() - .spawn_helper(async { Worker.start() }) + .spawn_helper(async { Worker.spawn() }) .unwrap() .join() .await @@ -330,7 +363,7 @@ async fn ambient_actor_started_inside_group_is_unmanaged_and_aborted_at_teardown assert!(ambient.context().execution_group().is_none()); assert!(matches!( within!(ambient.ready()).await, - Err(SystemError::UnmanagedActor) + Ok(spawned_concurrency::StartOutcome::Started(())) )); assert!(ambient.context().spawn_helper(async {}).is_err()); assert!(ambient @@ -393,7 +426,8 @@ async fn failed_startup_discards_queued_payloads_and_reply_channels_without_disp handled: handled.clone(), stopped: stopped.clone(), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); spawned_rt::tasks::timeout(LIMIT, entered) .await @@ -412,7 +446,10 @@ async fn failed_startup_discards_queued_payloads_and_reply_channels_without_disp spawned_rt::tasks::timeout(LIMIT, actor.ready()) .await .unwrap(), - Err(SystemError::StartupFailed(ExitReason::Panic(_))) + Err(spawned_concurrency::StartError::InitExited { + reason: ExitReason::Panic(_), + pending: None + }) )); assert!(matches!( spawned_rt::tasks::timeout(LIMIT, actor.wait_exit()) @@ -431,8 +468,12 @@ async fn failed_startup_discards_queued_payloads_and_reply_channels_without_disp async fn old_scope_cleanup_cannot_remove_a_replacement_registration() { let system = ActorSystem::new().unwrap(); let group = system.async_group("registry-replacement", 1).unwrap(); - let actor = Worker.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Worker.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let name = "execution-groups-stale-registration"; actor.context().register_name(name).unwrap(); assert!(matches!( @@ -445,7 +486,7 @@ async fn old_scope_cleanup_cannot_remove_a_replacement_registration() { )); spawned_concurrency::registry::unregister(name); let replacement = - spawned_concurrency::registry::register_owned(name, actor.id(), 42u64).unwrap(); + spawned_concurrency::registry::register_owned(name, actor.id(), 42u64, None).unwrap(); assert!(system.shutdown(LIMIT).await.complete); assert!(matches!( actor.context().register_name(name), @@ -475,8 +516,15 @@ impl ManagedProtocolActor { async fn generated_protocols_work_unchanged_with_explicit_group_placement() { let system = ActorSystem::new().unwrap(); let group = system.async_group("generated-protocol", 1).unwrap(); - let actor = ManagedProtocolActor(42).start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = ManagedProtocolActor(42) + .with_options(&group) + .spawn() + .unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); assert_eq!(actor.value().await.unwrap(), 42); assert!(system.shutdown(LIMIT).await.complete); assert!(actor.value().await.is_err()); @@ -486,8 +534,12 @@ async fn generated_protocols_work_unchanged_with_explicit_group_placement() { async fn stuck_standalone_cleanup_does_not_leave_earlier_actor_running_forever() { let system = ActorSystem::new().unwrap(); let group = system.async_group("stuck-standalone-order", 1).unwrap(); - let first = Worker.start_on(&group).unwrap(); - within!(first.ready()).await.unwrap(); + let first = Worker.with_options(&group).spawn().unwrap(); + within!(first.ready()) + .await + .unwrap() + .started() + .expect("started"); let (entered_tx, entered) = tokio::sync::oneshot::channel(); let (release, release_rx) = tokio::sync::oneshot::channel(); struct Stuck { @@ -509,9 +561,14 @@ async fn stuck_standalone_cleanup_does_not_leave_earlier_actor_running_forever() entered: Some(entered_tx), release: Some(release_rx), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - within!(last.ready()).await.unwrap(); + within!(last.ready()) + .await + .unwrap() + .started() + .expect("started"); let report = system.shutdown(Duration::from_millis(20)).await; assert!(!report.complete); spawned_rt::tasks::timeout(LIMIT, entered) @@ -552,19 +609,31 @@ async fn readiness_reports_actual_panic_after_startup_stop_instead_of_early_norm for in_stopped in [false, true] { for _ in 0..16 { let actor = StartupPanicAfterStop { in_stopped } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); let ready = spawned_rt::tasks::timeout(LIMIT, actor.ready()) .await .unwrap(); + // A panic after ctx.stop() in started() wins (row 4 over row 5); + // without it the start fails Normal and stopped() never runs. + let expected_panic = !in_stopped; + assert!( + matches!( + &ready, + Err(spawned_concurrency::StartError::InitExited { + reason: ExitReason::Panic(_), + pending: None + }) + ) == expected_panic, + "{ready:?}" + ); assert!(matches!( ready, - Err(SystemError::StartupFailed(ExitReason::Panic(_))) - )); - assert!(matches!( - within!(actor.wait_exit()).await, - ExitReason::Panic(_) + Err(spawned_concurrency::StartError::InitExited { pending: None, .. }) )); + let reason = within!(actor.wait_exit()).await; + assert_eq!(matches!(reason, ExitReason::Panic(_)), expected_panic); } } assert!(system.shutdown(LIMIT).await.complete); @@ -602,9 +671,14 @@ async fn concurrent_zero_deadline_reports_include_scopes_during_shutdown_initial let actor = SnapshotHeld { release: Some(release_rx), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - within!(actor.ready()).await.unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let (entered, entered_rx) = tokio::sync::oneshot::channel(); actor.send(Hold(entered)).unwrap(); entered_rx.await.unwrap(); @@ -659,18 +733,25 @@ impl Actor for Dependent { async fn ordered_shutdown_preserves_ordinary_calls_but_closes_new_helper_admission() { let system = ActorSystem::new().unwrap(); let group = system.async_group("shutdown-admission", 1).unwrap(); - let earlier = HelperAware.start_on(&group).unwrap(); - within!(earlier.ready()).await.unwrap(); + let earlier = HelperAware.with_options(&group).spawn().unwrap(); + within!(earlier.ready()) + .await + .unwrap() + .started() + .expect("started"); let observed = Arc::new(AtomicUsize::new(0)); Dependent { earlier, observed: observed.clone(), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); let report = system.shutdown(LIMIT).await; assert!(report.complete); assert_eq!(observed.load(Ordering::SeqCst), 1); @@ -688,8 +769,12 @@ async fn ordered_shutdown_preserves_ordinary_calls_but_closes_new_helper_admissi async fn aborted_scoped_helper_joins_as_cancelled() { let system = ActorSystem::new().unwrap(); let group = system.async_group("helper-abort", 1).unwrap(); - let actor = Worker.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Worker.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let helper = actor .context() .spawn_helper(std::future::pending::<()>()) @@ -751,7 +836,10 @@ async fn closed_system_rejects_new_groups_and_completed_reports_stay_complete() Err(SystemError::Closed) )); assert!(group.is_closed()); - assert!(matches!(Worker.start_on(&group), Err(SystemError::Closed))); + assert!(matches!( + Worker.with_options(&group).spawn(), + Err(spawned_concurrency::StartError::Closed) + )); } // Not simulated: shards belong to native thread groups. diff --git a/concurrency/tests/execution_profiles.rs b/concurrency/tests/execution_profiles.rs index b91c0cd7f..769f7fa01 100644 --- a/concurrency/tests/execution_profiles.rs +++ b/concurrency/tests/execution_profiles.rs @@ -9,6 +9,7 @@ use spawned_concurrency::{ supervision::{Backoff, ChildSpec, SupervisorConfig, SupervisorStrategy}, system::{ActorSystem, ExecutionProfile, SystemError}, tasks::{Actor, ActorStart, Context, Handler}, + StartError, }; use std::{ sync::{ @@ -43,7 +44,7 @@ impl Handler for Probe { } struct StartPeer; impl Message for StartPeer { - type Result = Result, SystemError>; + type Result = Result, StartError>; } impl Handler for Probe { async fn handle( @@ -53,7 +54,7 @@ impl Handler for Probe { ctx: &Context, ) { let __reply_value: ::Result = - { ctx.execution_group().unwrap().start(Probe) }; + { Probe.with_options(&ctx.execution_group().unwrap()).spawn() }; reply.reply(__reply_value); } } @@ -78,14 +79,25 @@ async fn native_context_placement_uses_a_free_slot_and_validates_helper_budgets( helper_workers: 2 }) ); - let actor = Probe.start_on(&group.shard(0).unwrap()).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Probe + .with_options(&group.shard(0).unwrap()) + .spawn() + .unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let peer = actor.request(StartPeer).await.unwrap().unwrap(); - within!(peer.ready()).await.unwrap(); + within!(peer.ready()) + .await + .unwrap() + .started() + .expect("started"); assert!(peer.request(Audit).await.unwrap().0.ends_with("actor-1")); assert!(matches!( actor.request(StartPeer).await.unwrap(), - Err(SystemError::Capacity) + Err(StartError::Capacity) )); assert!(system.shutdown(LIMIT).await.complete); } @@ -139,11 +151,28 @@ async fn persistent_native_slots_bound_admission_and_helpers_progress_during_blo helper_workers: 1 }) ); - let a = Probe.start_on(&group.shard(0).unwrap()).unwrap(); - let b = Probe.start_on(&group.shard(1).unwrap()).unwrap(); - within!(a.ready()).await.unwrap(); - within!(b.ready()).await.unwrap(); - assert!(matches!(Probe.start_on(&group), Err(SystemError::Capacity))); + let a = Probe + .with_options(&group.shard(0).unwrap()) + .spawn() + .unwrap(); + let b = Probe + .with_options(&group.shard(1).unwrap()) + .spawn() + .unwrap(); + within!(a.ready()) + .await + .unwrap() + .started() + .expect("started"); + within!(b.ready()) + .await + .unwrap() + .started() + .expect("started"); + assert!(matches!( + Probe.with_options(&group).spawn(), + Err(StartError::Capacity) + )); assert!(a.request(Audit).await.unwrap().0.ends_with("actor-0")); let (entered_tx, entered) = tokio::sync::oneshot::channel(); let (release, release_rx) = std::sync::mpsc::channel(); @@ -184,7 +213,10 @@ async fn persistent_native_slots_bound_admission_and_helpers_progress_during_blo .is_ok()); release.send(()).unwrap(); assert!(system.shutdown(LIMIT).await.complete); - assert!(matches!(Probe.start_on(&group), Err(SystemError::Closed))); + assert!(matches!( + Probe.with_options(&group).spawn(), + Err(StartError::Closed) + )); } // Not simulated: native execution groups need OS threads. #[cfg(not(spawned_sim))] @@ -192,8 +224,12 @@ async fn persistent_native_slots_bound_admission_and_helpers_progress_during_blo async fn native_capacity_is_released_only_after_registered_helper_cleanup() { let system = ActorSystem::new().unwrap(); let group = system.thread_group("slot-cleanup", 1).unwrap(); - let actor = Probe.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Probe.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let (entered_tx, entered) = tokio::sync::oneshot::channel(); let (release, release_rx) = std::sync::mpsc::channel(); let helper = actor @@ -209,21 +245,28 @@ async fn native_capacity_is_released_only_after_registered_helper_cleanup() { .unwrap(); actor.context().stop(); actor.child_handle().wait_exit_async().await; - assert!(matches!(Probe.start_on(&group), Err(SystemError::Capacity))); + assert!(matches!( + Probe.with_options(&group).spawn(), + Err(StartError::Capacity) + )); release.send(()).unwrap(); drop(helper); let replacement = spawned_rt::tasks::timeout(LIMIT, async { loop { - match Probe.start_on(&group) { + match Probe.with_options(&group).spawn() { Ok(actor) => break actor, - Err(SystemError::Capacity) => spawned_rt::tasks::yield_now().await, + Err(StartError::Capacity) => spawned_rt::tasks::yield_now().await, Err(e) => panic!("{e}"), } } }) .await .unwrap(); - within!(replacement.ready()).await.unwrap(); + within!(replacement.ready()) + .await + .unwrap() + .started() + .expect("started"); assert!(system.shutdown(LIMIT).await.complete); } // Not simulated: native execution groups need OS threads. @@ -318,12 +361,25 @@ async fn pinned_shards_preserve_explicit_cpu_placement_and_cross_shard_requests( let group = system.pinned_group("pipeline", cpus.clone()).unwrap(); assert!(group.shard(cpus.len()).is_err()); let actors: Vec<_> = (0..cpus.len()) - .map(|i| Probe.start_on(&group.shard(i).unwrap()).unwrap()) + .map(|i| { + Probe + .with_options(&group.shard(i).unwrap()) + .spawn() + .unwrap() + }) .collect(); for (i, actor) in actors.iter().enumerate() { - within!(actor.ready()).await.unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let peer_on_same_shard = actor.request(StartPeer).await.unwrap().unwrap(); - within!(peer_on_same_shard.ready()).await.unwrap(); + within!(peer_on_same_shard.ready()) + .await + .unwrap() + .started() + .expect("started"); assert_eq!( peer_on_same_shard.request(Audit).await.unwrap().1, Some(cpus[i]) @@ -387,8 +443,12 @@ mod tls_join { let system = ActorSystem::new().unwrap(); let cpu = spawned_rt::tasks::affinity::available_cpus().unwrap()[0]; let group = system.pinned_group("tls-owner", vec![cpu]).unwrap(); - let actor = Probe.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Probe.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let (entered_tx, entered) = tokio::sync::oneshot::channel(); let (release, release_rx) = std::sync::mpsc::channel(); actor @@ -522,8 +582,12 @@ fn partial_pinned_construction_rolls_back_before_returning_error() { let group = system.async_group("after-rollback", 1).unwrap(); let runtime = spawned_rt::tasks::control_runtime().unwrap(); runtime.block_on(async { - let actor = Probe.start_on(&group).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Probe.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); assert!(actor.request(Audit).await.is_ok()); assert!(system.shutdown(LIMIT).await.complete); }); diff --git a/concurrency/tests/exit_signal_matrix.rs b/concurrency/tests/exit_signal_matrix.rs index 14db2ccf4..c340dce4c 100644 --- a/concurrency/tests/exit_signal_matrix.rs +++ b/concurrency/tests/exit_signal_matrix.rs @@ -68,9 +68,9 @@ mod native { for relation in [Relation::Link, Relation::Monitor] { for trapping in [false, true] { for reason in reasons() { - let source = Idle.start(); + let source = Idle.spawn(); let (send, notices) = mpsc::channel(); - let observer = Observer(send).start(); + let observer = Observer(send).spawn(); source.request(Ping).unwrap(); observer.request(Ping).unwrap(); observer.context().trap_exit(trapping); @@ -127,9 +127,9 @@ mod asynchronous { for relation in [Relation::Link, Relation::Monitor] { for trapping in [false, true] { for reason in reasons() { - let source = Idle.start(); + let source = Idle.spawn(); let (send, notices) = mpsc::channel(); - let observer = Observer(send).start(); + let observer = Observer(send).spawn(); source.request(Ping).await.unwrap(); observer.request(Ping).await.unwrap(); observer.context().trap_exit(trapping); diff --git a/concurrency/tests/exit_signals_sim.rs b/concurrency/tests/exit_signals_sim.rs index ecc2aea3a..192c0ca1c 100644 --- a/concurrency/tests/exit_signals_sim.rs +++ b/concurrency/tests/exit_signals_sim.rs @@ -185,8 +185,8 @@ async fn kill_vs_stopped(seed: u64) { let (log, watched) = (Log::default(), Log::default()); let mut node = Node::new(&log, false); node.slow_stop = 2; - let target = node.start(); - let watcher = Node::new(&watched, false).start(); + let target = node.spawn(); + let watcher = Node::new(&watched, false).spawn(); let handle = target.child_handle(); let h = handle.clone(); on(&watcher, move |ctx| { @@ -251,8 +251,8 @@ async fn late_signal_vs_death(seed: u64) { let (log, observed) = (Log::default(), Log::default()); let mut node = Node::new(&log, false); node.slow_stop = 2; - let target = node.start(); - let observer = Node::new(&observed, true).start(); + let target = node.spawn(); + let observer = Node::new(&observed, true).spawn(); let handle = target.child_handle(); let h = handle.clone(); on(&observer, move |ctx| { @@ -272,12 +272,25 @@ async fn late_signal_vs_death(seed: u64) { signalling.await.unwrap(); let log = entries(&log); let signal = position(&log, "signal").unwrap(); - let end = position(&log, "stopped:end").expect("a self-exit runs stopped()"); - outcome(format!("late signal: {reason:?}")); - match reason { - ExitReason::Shutdown(_) => {} - ExitReason::Normal => assert!(end < signal, "a signal before D was lost: {log:?}"), - other => panic!("unexpected reason {other:?}: {log:?}"), + // The administrative stop waits for started() (OTP gen_server:stop). If + // the untrapped signal arrives while started() still runs, it fails the + // start without stopped() (an exit signal kills an initialising + // process); otherwise the stop's exit runs stopped(). + let end = position(&log, "stopped:end"); + outcome(format!( + "late signal: {reason:?}{}", + if end.is_none() { + " (during started)" + } else { + "" + } + )); + match (reason.clone(), end) { + (ExitReason::Shutdown(None), _) => {} + (ExitReason::Normal, Some(end)) => { + assert!(end < signal, "a signal before D was lost: {log:?}") + } + (other, _) => panic!("unexpected reason {other:?}: {log:?}"), } let downs = downs(&observer, &observed).await; assert_eq!( @@ -294,8 +307,8 @@ async fn late_signal_vs_death(seed: u64) { /// agree; `try_link` after D gets `Err(NoProc)` and no notice. async fn link_vs_death(seed: u64) { let observed = Log::default(); - let target = Node::new(&Log::default(), false).start(); - let observer = Node::new(&observed, true).start(); + let target = Node::new(&Log::default(), false).spawn(); + let observer = Node::new(&observed, true).spawn(); let handle = target.child_handle(); let stopper = handle.clone(); let stop_at = seed % 3; @@ -363,8 +376,8 @@ async fn unlink_demonitor_fence(seed: u64) { } } let observed = Log::default(); - let target = Node::new(&Log::default(), false).start(); - let observer = Node::new(&observed, true).start(); + let target = Node::new(&Log::default(), false).spawn(); + let observer = Node::new(&observed, true).spawn(); let handle = target.child_handle(); let monitor: Arc>> = Arc::default(); let (h, m) = (handle.clone(), monitor.clone()); diff --git a/concurrency/tests/failpoints.rs b/concurrency/tests/failpoints.rs index 39e8a8988..3c57b1db9 100644 --- a/concurrency/tests/failpoints.rs +++ b/concurrency/tests/failpoints.rs @@ -545,10 +545,18 @@ async fn slow_system_coordinator_reports_then_completes() { let system = ActorSystem::new().unwrap(); let group = system.async_group("fp-system", 2).unwrap(); let started = Started::default(); - let a = group.start(Worker::new(&started)).unwrap(); - let b = group.start(Worker::new(&started)).unwrap(); - within!(a.ready()).await.unwrap(); - within!(b.ready()).await.unwrap(); + let a = Worker::new(&started).with_options(&group).spawn().unwrap(); + let b = Worker::new(&started).with_options(&group).spawn().unwrap(); + within!(a.ready()) + .await + .unwrap() + .started() + .expect("started"); + within!(b.ready()) + .await + .unwrap() + .started() + .expect("started"); let report = system.shutdown(Duration::from_millis(50)).await; assert!(!report.complete); let report = report.wait(LIMIT).await; @@ -574,18 +582,24 @@ async fn failed_group_start_rolls_back_and_frees_the_name() { let group = system.async_group("fp-group", 2).unwrap(); let threads = system.thread_group("fp-threads", 2).unwrap(); let started = Started::default(); - group - .start(Worker::new(&started)) + Worker::new(&started) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); - threads - .start(Worker::new(&started)) + .unwrap() + .started() + .expect("started"); + Worker::new(&started) + .with_options(&threads) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); assert!(system.shutdown(LIMIT).await.complete); } @@ -597,8 +611,8 @@ async fn placement_failure_is_reported_and_fails_the_supervisor() { let started = Started::default(); failpoints::configure("system.reserve", FailPoint::new(Action::Error).times(1)).unwrap(); assert!(matches!( - group.start(Worker::new(&started)), - Err(SystemError::Capacity) + Worker::new(&started).with_options(&group).spawn(), + Err(spawned_concurrency::StartError::Capacity) )); failpoints::configure("system.reserve", Action::Error).unwrap(); let mut tree = system.supervisor(config(5)).unwrap(); @@ -621,12 +635,15 @@ async fn placement_failure_is_reported_and_fails_the_supervisor() { assert_eq!(child.snapshot().state, ChildState::Failed); assert_eq!(child.snapshot().generation, 0); failpoints::remove("system.reserve"); - group - .start(Worker::new(&started)) + Worker::new(&started) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); assert!(started.handles().len() == 1); assert!(system.shutdown(LIMIT).await.complete); } @@ -647,7 +664,7 @@ async fn registration_racing_shutdown_leaves_no_entry() { let mut worker = Worker::new(&started); worker.name = Some("fp-register-race"); let registration = worker.registration.clone(); - let actor = group.start(worker).unwrap(); + let actor = worker.with_options(&group).spawn().unwrap(); reached("registry.before_register", 1).await; assert!(system.shutdown(LIMIT).await.complete); assert_eq!(*registration.lock().unwrap(), Some(false)); @@ -665,7 +682,15 @@ async fn injected_registration_error_leaves_no_entry() { let mut worker = Worker::new(&started); worker.name = Some("fp-register-error"); let registration = worker.registration.clone(); - group.start(worker).unwrap().ready().await.unwrap(); + worker + .with_options(&group) + .spawn() + .unwrap() + .ready() + .await + .unwrap() + .started() + .expect("started"); assert_eq!(*registration.lock().unwrap(), Some(false)); assert!(registry::whereis::>("fp-register-error").is_none()); assert!(system.shutdown(LIMIT).await.complete); @@ -781,8 +806,8 @@ async fn peer_exit_during_link_install_still_propagates() { ) .unwrap(); let (downs, _) = tokio::sync::mpsc::unbounded_channel(); - let linker = Linker { downs }.start(); - let target = Worker::new(&Started::default()).start(); + let linker = Linker { downs }.spawn(); + let target = Worker::new(&Started::default()).spawn(); linker.send(LinkTo(target.child_handle())).unwrap(); reached("link.before_install", 1).await; target.send(Crash).unwrap(); @@ -803,8 +828,8 @@ async fn target_exit_during_monitor_install_delivers_noproc() { ) .unwrap(); let (downs, mut received) = tokio::sync::mpsc::unbounded_channel(); - let watcher = Linker { downs }.start(); - let target = Worker::new(&Started::default()).start(); + let watcher = Linker { downs }.spawn(); + let target = Worker::new(&Started::default()).spawn(); watcher.send(Watch(target.child_handle())).unwrap(); reached("monitor.before_install", 1).await; target.send(Crash).unwrap(); @@ -826,7 +851,7 @@ async fn target_exit_during_monitor_install_delivers_noproc() { async fn injected_started_panic_is_an_abnormal_exit_seen_by_monitors() { let _scenario = FailScenario::setup(); let (downs, mut received) = tokio::sync::mpsc::unbounded_channel(); - let watcher = Linker { downs }.start(); + let watcher = Linker { downs }.spawn(); // The watcher must be past started() before the site is armed. watcher.request(Ping).await.unwrap(); failpoints::configure( @@ -835,7 +860,7 @@ async fn injected_started_panic_is_an_abnormal_exit_seen_by_monitors() { ) .unwrap(); let started = Started::default(); - let target = Worker::new(&started).start(); + let target = Worker::new(&started).spawn(); watcher.send(Watch(target.child_handle())).unwrap(); match within!(target.wait_exit()).await { ExitReason::Panic(message) => assert!(message.contains("actor.before_started")), @@ -866,7 +891,10 @@ async fn readiness_racing_shutdown_is_not_published() { .unwrap(); let system = ActorSystem::new().unwrap(); let group = system.async_group("fp-ready", 1).unwrap(); - let actor = group.start(Worker::new(&Started::default())).unwrap(); + let actor = Worker::new(&Started::default()) + .with_options(&group) + .spawn() + .unwrap(); reached("actor.after_started_before_ready", 1).await; assert!(system.shutdown(LIMIT).await.complete); assert!(within!(actor.ready()).await.is_err()); @@ -885,8 +913,12 @@ async fn slow_stopped_reports_at_deadline_then_completes() { let group = system.async_group("fp-stopped", 1).unwrap(); let mut worker = Worker::new(&Started::default()); worker.trap = true; - let actor = group.start(worker).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = worker.with_options(&group).spawn().unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let report = system.shutdown(Duration::from_millis(100)).await; assert!(!report.complete); assert_eq!(report.unfinished_actors, [actor.id()]); @@ -975,8 +1007,15 @@ async fn system_shutdown_coordinator_panic_is_reported_and_contained() { .unwrap(); let system = ActorSystem::new().unwrap(); let group = system.async_group("fp-gap-system", 1).unwrap(); - let actor = group.start(Worker::new(&Started::default())).unwrap(); - within!(actor.ready()).await.unwrap(); + let actor = Worker::new(&Started::default()) + .with_options(&group) + .spawn() + .unwrap(); + within!(actor.ready()) + .await + .unwrap() + .started() + .expect("started"); let report = system.shutdown(LIMIT).await; assert!(report.complete); assert!( @@ -1028,7 +1067,7 @@ impl Handler for Holder { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn no_reply_succeeds_after_noproc_is_observable() { let _scenario = FailScenario::setup(); - let holder = Holder.start(); + let holder = Holder.spawn(); let (tokens, token) = std::sync::mpsc::channel(); let caller = holder .send_request(Hold(tokens), spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) @@ -1108,9 +1147,9 @@ impl Handler for SignalSender { async fn an_exit_signal_is_not_overtaken_by_its_senders_death_notice() { let _scenario = FailScenario::setup(); let seen = Arc::new(Mutex::new(Vec::new())); - let recorder = Recorder(seen.clone()).start(); + let recorder = Recorder(seen.clone()).spawn(); recorder.request(Ping).await.unwrap(); // trapping now - let sender = SignalSender.start(); + let sender = SignalSender.spawn(); sender .request(LinkTo(recorder.child_handle())) .await @@ -1201,8 +1240,8 @@ impl Actor for Quiet {} async fn demonitor_flushes_a_down_in_dispatch() { let _scenario = FailScenario::setup(); let handled = Arc::new(AtomicUsize::new(0)); - let observer = Observer(handled.clone()).start(); - let target = Quiet.start(); + let observer = Observer(handled.clone()).spawn(); + let target = Quiet.spawn(); let monitor = observer .request(WatchDown(target.child_handle())) .await @@ -1309,7 +1348,7 @@ impl Handler for Discharger { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn an_unread_reply_is_destroyed_after_every_caller_is_answered() { let _scenario = FailScenario::setup(); - let actor = Discharger.start(); + let actor = Discharger.spawn(); let caller: SharedReceiver = Arc::default(); let answered = Arc::new(AtomicUsize::new(0)); let unread = actor @@ -1369,7 +1408,7 @@ async fn parked_receivers_wake_before_any_waker() { } } let _scenario = FailScenario::setup(); - let actor = Discharger.start(); + let actor = Discharger.spawn(); let (release, gate) = std::sync::mpsc::channel(); let (entered_tx, entered) = std::sync::mpsc::channel(); actor.send(Gate(gate, entered_tx)).unwrap(); @@ -1503,7 +1542,7 @@ async fn emit_due_during_exit_is_destroyed_after_every_caller_is_answered() { let _scenario = FailScenario::setup(); let (stop_release, stop_gate) = std::sync::mpsc::channel(); let (stopping_tx, stopping) = std::sync::mpsc::channel(); - let actor = SlowStop(stop_gate, stopping_tx).start(); + let actor = SlowStop(stop_gate, stopping_tx).spawn(); let caller: SharedReceiver = Arc::default(); let answered = Arc::new(AtomicUsize::new(0)); let (started_tx, started) = std::sync::mpsc::channel(); @@ -1591,7 +1630,7 @@ mod native { #[test] fn no_reply_succeeds_after_noproc_is_observable() { let _scenario = FailScenario::setup(); - let holder = NativeHolder.start(); + let holder = NativeHolder.spawn(); let (tokens, token) = std::sync::mpsc::channel(); let caller = holder .send_request(Hold(tokens), spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) @@ -1655,8 +1694,8 @@ mod native { fn an_exit_signal_is_not_overtaken_by_its_senders_death_notice() { let _scenario = FailScenario::setup(); let seen = Arc::new(Mutex::new(Vec::new())); - let recorder = NativeRecorder(seen.clone()).start(); - let sender = NativeSender.start(); + let recorder = NativeRecorder(seen.clone()).spawn(); + let sender = NativeSender.spawn(); sender.request(LinkTo(recorder.child_handle())).unwrap(); failpoints::configure( "actor.exit_before_deliver", @@ -1712,7 +1751,7 @@ mod native { FailPoint::new(Action::Panic).times(1), ) .unwrap(); - let actor = Target.start(); + let actor = Target.spawn(); match actor.wait_exit() { ExitReason::Panic(message) => assert!(message.contains("actor.before_started")), other => panic!("expected panic exit, got {other:?}"), @@ -1727,7 +1766,10 @@ mod native { FailPoint::new(Action::Delay(Duration::from_millis(100))).times(1), ) .unwrap(); - let actor = Target.start(); + let actor = Target.spawn(); + // A ctx.stop() inside started() is Init::stop (no stopped()): stop + // the running actor. + actor.ready().unwrap().started().expect("started"); let began = Instant::now(); actor.context().stop(); actor.join(); @@ -1743,8 +1785,8 @@ mod native { FailPoint::new(Action::Delay(Duration::from_millis(200))).times(1), ) .unwrap(); - let peer = Peer.start(); - let target = Target.start(); + let peer = Peer.spawn(); + let target = Target.spawn(); peer.send(LinkTo(target.child_handle())).unwrap(); let began = Instant::now(); while failpoints::stats("link.before_install").hits < 1 { @@ -1802,8 +1844,8 @@ mod native { fn demonitor_flushes_a_down_in_dispatch() { let _scenario = FailScenario::setup(); let handled = Arc::new(AtomicUsize::new(0)); - let observer = NativeObserver(handled.clone()).start(); - let target = NativeQuiet.start(); + let observer = NativeObserver(handled.clone()).spawn(); + let target = NativeQuiet.spawn(); let monitor = observer.request(WatchDown(target.child_handle())).unwrap(); failpoints::configure( "monitor.down_before_dispatch", @@ -1874,7 +1916,7 @@ mod native { #[test] fn an_unread_reply_is_destroyed_after_every_caller_is_answered() { let _scenario = FailScenario::setup(); - let actor = NativeDischarger.start(); + let actor = NativeDischarger.spawn(); let caller: SharedReceiver = Arc::default(); let answered = Arc::new(AtomicUsize::new(0)); let unread = actor @@ -1930,7 +1972,7 @@ mod native { } } let _scenario = FailScenario::setup(); - let actor = NativeDischarger.start(); + let actor = NativeDischarger.spawn(); let (release, gate) = std::sync::mpsc::channel(); let (entered_tx, entered) = std::sync::mpsc::channel(); actor.send(Gate(gate, entered_tx)).unwrap(); @@ -1982,7 +2024,7 @@ mod native { FailPoint::new(Action::Delay(Duration::from_millis(500))).times(1), ) .unwrap(); - let owner = One.start(); + let owner = One.spawn(); let ctx = owner.context(); let opts = spawned_concurrency::threads::JobOptions::new; drop(ctx.reserve_blocking_job(opts()).unwrap()); @@ -2066,7 +2108,7 @@ mod native { let _scenario = FailScenario::setup(); let (stop_release, stop_gate) = std::sync::mpsc::channel(); let (stopping_tx, stopping) = std::sync::mpsc::channel(); - let actor = NativeSlowStop(stop_gate, stopping_tx).start(); + let actor = NativeSlowStop(stop_gate, stopping_tx).spawn(); let caller: SharedReceiver = Arc::default(); let answered = Arc::new(AtomicUsize::new(0)); let (started_tx, started) = std::sync::mpsc::channel(); diff --git a/concurrency/tests/failure_windows.rs b/concurrency/tests/failure_windows.rs index 0580af370..f4bcc7b18 100644 --- a/concurrency/tests/failure_windows.rs +++ b/concurrency/tests/failure_windows.rs @@ -118,6 +118,8 @@ fn controlled_failures_reach_every_required_window_on_every_backend() { reached.insert(REQUIRED[3]); } { + // The stop arrives while started() runs: it is applied once the + // start commits ready, and stopped() runs (OTP gen_server:stop). let (started, mut control) = gate(); let actor = Handle::start( backend, diff --git a/concurrency/tests/foreign_panics.rs b/concurrency/tests/foreign_panics.rs index 27f23bac3..13a0bb63d 100644 --- a/concurrency/tests/foreign_panics.rs +++ b/concurrency/tests/foreign_panics.rs @@ -408,18 +408,19 @@ mod env { pub fn start(&self, victim: Victim) -> Started { let _entered = self.rt.enter(); match self.place { - Place::Ambient => { - Started::Task(TaskVictim(victim).start_with_backend(Backend::Async)) - } - Place::Native => Started::Native(spawned_concurrency::threads::ActorStart::start( + Place::Ambient => Started::Task( + TaskVictim(victim) + .with_options(Backend::Async) + .spawn() + .unwrap(), + ), + Place::Native => Started::Native(spawned_concurrency::threads::ActorStart::spawn( super::thread_actors::ThreadVictim(victim), )), _ => { - let actor = self - .group - .as_ref() - .unwrap() - .start(TaskVictim(victim)) + let actor = TaskVictim(victim) + .with_options(self.group.as_ref().unwrap()) + .spawn() .unwrap(); let _ = self.bounded(actor.ready()); Started::Task(actor) diff --git a/concurrency/tests/future_layout.rs b/concurrency/tests/future_layout.rs index f9b7ac093..a6f90b8da 100644 --- a/concurrency/tests/future_layout.rs +++ b/concurrency/tests/future_layout.rs @@ -87,7 +87,7 @@ async fn configured_protocol(actor: ActorRef) -> Result } #[spawned_macros::sim_test] async fn owned_send_request_and_protocol_futures_fit_the_pre_guard_layout() { - let actor = Worker.start(); + let actor = Worker.spawn(); let baseline = Baseline([0; BASELINE_WORDS]); assert_eq!( size_of_val(&actor), diff --git a/concurrency/tests/immediate_dispatch.rs b/concurrency/tests/immediate_dispatch.rs index fd4bfdcd3..c6f24baa4 100644 --- a/concurrency/tests/immediate_dispatch.rs +++ b/concurrency/tests/immediate_dispatch.rs @@ -60,7 +60,7 @@ async fn direct_dispatch_keeps_patterns_returns_question_mark_and_name_hygiene() let actor = Worker { calls: calls.clone(), } - .start(); + .spawn(); assert_eq!(actor.request(Compute { value: 2 }).await.unwrap(), Ok(23)); assert_eq!(actor.request(Compute { value: 0 }).await.unwrap(), Ok(11)); assert_eq!( @@ -78,7 +78,7 @@ async fn inherent_methods_and_public_handler_futures_stay_lazy() { let actor = Worker { calls: calls.clone(), } - .start(); + .spawn(); let context = actor.context(); let mut direct = Worker { calls: calls.clone(), @@ -105,7 +105,7 @@ async fn eligible_handler_uses_the_immediate_hook() { let actor = Worker { calls: calls.clone(), } - .start(); + .spawn(); let context = actor.context(); let mut direct = Worker { calls: calls.clone(), @@ -185,7 +185,7 @@ mod local_items { // test uses the real runtime: statics are not reset between simulator seeds. #[tokio::test] async fn local_items_have_one_identity_across_direct_and_mailbox_calls() { - let actor = ItemWorker.start(); + let actor = ItemWorker.spawn(); let context = actor.context(); let mut direct = ItemWorker; assert_eq!(direct.count(ItemCounter, &context).await, 0); @@ -242,7 +242,7 @@ mod fairness { calls, observed: observed.clone(), } - .start(); + .spawn(); for _ in 0..1024 { actor.send(Tick).unwrap(); } @@ -277,7 +277,7 @@ impl Opaque { } #[spawned_macros::sim_test] async fn opaque_macros_can_still_suspend() { - let actor = Opaque.start(); + let actor = Opaque.spawn(); assert_eq!(actor.request(Yielding).await.unwrap(), 9); actor.context().stop(); actor.join().await; @@ -313,7 +313,7 @@ async fn direct_dispatch_panic_answers_queued_callers_and_runs_stopped() { let actor = Panicker { stopped: stopped.clone(), } - .start(); + .spawn(); let first = actor.send_request(Explode, Deadline::Never).unwrap(); let queued = actor.send_request(After, Deadline::Never).unwrap(); for result in [first.await, queued.await] { @@ -341,7 +341,7 @@ impl<'__spawned_reply_turn: 'static> Generic<'__spawned_reply_turn> { } #[spawned_macros::sim_test] async fn generic_lifetimes_do_not_shadow_generated_lifetimes() { - let actor = Generic::<'static>(std::marker::PhantomData).start(); + let actor = Generic::<'static>(std::marker::PhantomData).spawn(); assert_eq!(actor.request(GenericGet).await.unwrap(), 4); actor.context().stop(); actor.join().await; @@ -380,7 +380,7 @@ impl BorrowViews { } #[spawned_macros::sim_test] async fn shared_receiver_and_declared_context_types_keep_their_borrow_rules() { - let actor = BorrowViews(Arc::new(AtomicUsize::new(4))).start(); + let actor = BorrowViews(Arc::new(AtomicUsize::new(4))).spawn(); assert_eq!(actor.request(SharedRead).await.unwrap(), 8); assert_eq!(actor.request(ViewRead).await.unwrap(), 123); actor.context().stop(); @@ -447,7 +447,7 @@ async fn ignored_parameters_drop_after_the_body_in_mailbox_and_inherent_calls() body_ran: body_ran.clone(), stopped: stopped.clone(), } - .start(); + .spawn(); let context = actor.context(); let mut direct = DropWorker { body_ran: body_ran.clone(), @@ -505,7 +505,7 @@ async fn ignored_parameter_destructor_panics_after_body_and_remains_contained() body_ran: body_ran.clone(), stopped: stopped.clone(), } - .start(); + .spawn(); let result = actor .request(Ignored(DropOrder { body_ran, @@ -530,7 +530,7 @@ async fn payload_destructors_keep_the_original_future_drop_boundary() { body_ran: body_ran.clone(), stopped: Arc::new(AtomicBool::new(false)), } - .start(); + .spawn(); let mut direct = DropWorker { body_ran: body_ran.clone(), stopped: Arc::new(AtomicBool::new(false)), @@ -583,7 +583,7 @@ mod shadowed_prelude { } #[spawned_macros::sim_test] async fn generated_paths_do_not_capture_shadowed_prelude_names() { - let actor = shadowed_prelude::ShadowActor.start(); + let actor = shadowed_prelude::ShadowActor.spawn(); assert_eq!(actor.request(shadowed_prelude::Shadow).await.unwrap(), 19); actor.context().stop(); actor.join().await; @@ -616,7 +616,7 @@ impl Handler for Discarder { } #[spawned_macros::sim_test] async fn direct_dispatch_commits_an_unanswered_obligation_on_normal_return() { - let actor = Discarder.start(); + let actor = Discarder.spawn(); assert!(matches!( actor.request(Discard).await, Err(ActorError::ReplyDropped) @@ -659,7 +659,7 @@ async fn a_rejected_result_destructor_has_the_same_handler_panic_boundary() { dropped: dropped.clone(), stopped: stopped.clone(), } - .start(); + .spawn(); let receiver = actor.send_request(GetRejected, Deadline::Never).unwrap(); drop(receiver); actor.join().await; diff --git a/concurrency/tests/job_trace.rs b/concurrency/tests/job_trace.rs index f42efa959..0fa380089 100644 --- a/concurrency/tests/job_trace.rs +++ b/concurrency/tests/job_trace.rs @@ -371,7 +371,7 @@ fn start_with(old: OldJob) -> (ActorRef, Arc) { handles: Vec::new(), gates: Vec::new(), } - .start(); + .spawn(); lifecycle_trace::watch_deferred(actor.id().to_string().parse().unwrap()); (actor, log) } @@ -656,7 +656,7 @@ mod threads_mode { #[test] fn delayed_emit_releases_the_held_result_threads() { let log = Arc::new(Log::default()); - let actor = TOwner(log.clone()).start(); + let actor = TOwner(log.clone()).spawn(); let id: u64 = actor.id().to_string().parse().unwrap(); lifecycle_trace::watch_deferred(id); actor.request(Go).unwrap(); diff --git a/concurrency/tests/kill_links.rs b/concurrency/tests/kill_links.rs index 729347ead..81b879727 100644 --- a/concurrency/tests/kill_links.rs +++ b/concurrency/tests/kill_links.rs @@ -38,10 +38,10 @@ fn native_propagated_kill_is_trappable_but_direct_kill_is_not() { recv.recv_timeout(LIMIT).unwrap() } let (send, events) = mpsc::channel(); - let a = Node(send.clone()).start(); - let b = Node(send.clone()).start(); - let c = Node(send.clone()).start(); - let d = Node(send).start(); + let a = Node(send.clone()).spawn(); + let b = Node(send.clone()).spawn(); + let c = Node(send.clone()).spawn(); + let d = Node(send).spawn(); b.context().link(&a.child_handle()); c.context().trap_exit(true); c.context().link(&b.child_handle()); @@ -93,10 +93,10 @@ async fn async_propagated_kill_is_trappable_but_direct_kill_is_not() { } } let (send, events) = mpsc::channel(); - let a = Node(send.clone()).start(); - let b = Node(send.clone()).start(); - let c = Node(send.clone()).start(); - let d = Node(send).start(); + let a = Node(send.clone()).spawn(); + let b = Node(send.clone()).spawn(); + let c = Node(send.clone()).spawn(); + let d = Node(send).spawn(); b.context().link(&a.child_handle()); c.context().trap_exit(true); c.context().link(&b.child_handle()); diff --git a/concurrency/tests/leaks.rs b/concurrency/tests/leaks.rs index db7ca83f8..55791d54c 100644 --- a/concurrency/tests/leaks.rs +++ b/concurrency/tests/leaks.rs @@ -82,7 +82,7 @@ fn native_actor_thread_and_state_released_after_stop() { let state = Arc::new(()); let thread = Arc::new(()); let (state_weak, thread_weak) = (Arc::downgrade(&state), Arc::downgrade(&thread)); - let actor = threads::ActorStart::start(NativeProbe { + let actor = threads::ActorStart::spawn(NativeProbe { _state: state, thread: Some(thread), }); @@ -138,11 +138,11 @@ impl threads::Handler>> for NativeB { fn native_actor_reference_cycle_released_after_stop() { let (sa, sb) = (Arc::new(()), Arc::new(())); let (wa, wb) = (Arc::downgrade(&sa), Arc::downgrade(&sb)); - let a = threads::ActorStart::start(NativeA { + let a = threads::ActorStart::spawn(NativeA { _state: sa, peer: None, }); - let b = threads::ActorStart::start(NativeB { + let b = threads::ActorStart::spawn(NativeB { _state: sb, peer: None, }); @@ -159,12 +159,12 @@ fn native_actor_reference_cycle_released_after_stop() { #[test] fn owned_registration_removed_with_its_guard_after_native_stop() { - let actor = threads::ActorStart::start(NativeProbe { + let actor = threads::ActorStart::spawn(NativeProbe { _state: Arc::new(()), thread: None, }); let name = "leaks-native-owned"; - let guard = registry::register_owned(name, actor.id(), actor.clone()).unwrap(); + let guard = registry::register_owned(name, actor.id(), actor.clone(), None).unwrap(); actor.context().stop(); actor.join(); drop(guard); @@ -272,7 +272,7 @@ mod async_leaks { if backend == Backend::Thread { probe.thread = Some(thread); } - let actor = probe.start_with_backend(backend); + let actor = probe.with_options(backend).spawn().unwrap(); actor.request(Ping).await.unwrap(); if backend == Backend::Thread { assert_eq!(thread_weak.strong_count(), 1, "probe bound to actor thread"); @@ -294,8 +294,8 @@ mod async_leaks { fn monitor_waiters_released_on_demonitor_owner_exit_and_target_exit() { let rt = runtime(); rt.block_on(async { - let owner = Probe::new(Arc::new(())).start(); - let target = Probe::new(Arc::new(())).start(); + let owner = Probe::new(Arc::new(())).spawn(); + let target = Probe::new(Arc::new(())).spawn(); owner.request(Ping).await.unwrap(); target.request(Ping).await.unwrap(); let baseline = rt.metrics().num_alive_tasks(); @@ -315,7 +315,7 @@ mod async_leaks { tasks_drain_to(&rt, baseline - 1, "monitor waiters after owner exit"); // Target exit delivers Down and releases the waiter. - let watcher = Probe::new(Arc::new(())).start(); + let watcher = Probe::new(Arc::new(())).spawn(); watcher.context().monitor(&target.child_handle()); target.request(Crash).await.unwrap_err(); within!(target.join()).await; @@ -400,13 +400,17 @@ mod async_leaks { peer: None, recipient: None, } - .start_with_backend(backend); + .with_options(backend) + .spawn() + .unwrap(); let b = AsyncB { _state: sb, peer: None, recipient: None, } - .start_with_backend(backend); + .with_options(backend) + .spawn() + .unwrap(); let ra = a.recipient::(); let rb = b.recipient::(); let weaks = (Arc::downgrade(&ra), Arc::downgrade(&rb)); @@ -445,13 +449,13 @@ mod async_leaks { peer: None, recipient: None, } - .start(); + .spawn(); let b = AsyncB { _state: sb, peer: None, recipient: None, } - .start(); + .spawn(); a.request(SetPeer((b.clone(), b.recipient::()))) .await .unwrap(); @@ -566,7 +570,8 @@ mod async_leaks { thread: Some(worker), name: Some("leaks-system-actor"), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); actor.request(Ping).await.unwrap(); assert_eq!(worker_weak.strong_count(), 1, "probe bound to worker"); @@ -711,19 +716,19 @@ mod sim_leaks { let weak = [&state, &sa, &sb].map(Arc::downgrade); rt.block_on(async { let baseline = rt.stats().tasks; - let probe = Probe { _state: state }.start(); + let probe = Probe { _state: state }.spawn(); probe.request(Ping).await.unwrap(); let timer = tasks::send_interval(Duration::from_secs(3600), probe.context(), Ping); let a = A { _state: sa, peer: None, } - .start(); + .spawn(); let b = B { _state: sb, peer: None, } - .start(); + .spawn(); let (ra, rb) = (a.recipient::(), b.recipient::()); let recipients = [Arc::downgrade(&ra), Arc::downgrade(&rb)]; a.request(SetPeer((b.clone(), rb))).await.unwrap(); @@ -760,11 +765,11 @@ mod sim_leaks { let owner = Probe { _state: Arc::new(()), } - .start(); + .spawn(); let target = Probe { _state: Arc::new(()), } - .start(); + .spawn(); owner.request(Ping).await.unwrap(); target.request(Ping).await.unwrap(); let baseline = rt.stats().tasks; @@ -799,7 +804,10 @@ mod sim_leaks { rt.block_on(async { let system = ActorSystem::new().unwrap(); let group = system.async_group("leaks-sim", 1).unwrap(); - let actor = Probe { _state: state }.start_on(&group).unwrap(); + let actor = Probe { _state: state } + .with_options(&group) + .spawn() + .unwrap(); actor.request(Ping).await.unwrap(); actor.context().register_name(name).unwrap(); let _parked = actor @@ -849,11 +857,11 @@ mod sim_leaks { let watcher = Probe { _state: Arc::new(()), } - .start(); + .spawn(); let target = Probe { _state: Arc::new(()), } - .start(); + .spawn(); watcher.request(Ping).await.unwrap(); target.request(Ping).await.unwrap(); let baseline = rt.stats().tasks; diff --git a/concurrency/tests/lifecycle_trace.rs b/concurrency/tests/lifecycle_trace.rs index e9c60eda3..a695f37d8 100644 --- a/concurrency/tests/lifecycle_trace.rs +++ b/concurrency/tests/lifecycle_trace.rs @@ -238,7 +238,7 @@ async fn kill_answers_queued_request_at_death() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::new(&log).start(); + let a1 = Node::new(&log).spawn(); let (release, gate) = tokio::sync::oneshot::channel(); let blocked = a1 .send_request(Block(gate), spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) @@ -280,7 +280,7 @@ async fn kill_during_stopped() { lifecycle_trace::record_lifecycle(); let log = Log::default(); let (release, gate) = tokio::sync::oneshot::channel(); - let a1 = Node::holding_stopped(&log, gate).start(); + let a1 = Node::holding_stopped(&log, gate).spawn(); a1.request(Sync()).await.unwrap(); a1.child_handle().stop(); wait_for(&log, "stopped").await; @@ -307,7 +307,7 @@ async fn untrapped_signal_skips_stopped() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::new(&log).start(); + let a1 = Node::new(&log).spawn(); a1.request(Sync()).await.unwrap(); a1.child_handle().exit(ExitReason::Normal); a1.request(Sync()).await.unwrap(); @@ -335,8 +335,8 @@ async fn trapped_exit_reaches_exit_received() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::trapping(&log).start(); - let a2 = Node::new(&log).start(); + let a1 = Node::trapping(&log).spawn(); + let a2 = Node::new(&log).spawn(); a1.request(Sync()).await.unwrap(); a2.request(Sync()).await.unwrap(); a1.child_handle().exit(ExitReason::Shutdown(None)); @@ -369,7 +369,7 @@ async fn trapped_exit_then_stop() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::trapping(&log).start(); + let a1 = Node::trapping(&log).spawn(); a1.request(Sync()).await.unwrap(); a1.child_handle().exit(ExitReason::Shutdown(None)); wait_for(&log, "exit:Shutdown(None)").await; @@ -399,8 +399,8 @@ async fn late_registrations_get_noproc() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::trapping(&log).start(); - let a2 = Node::new(&Log::default()).start(); + let a1 = Node::trapping(&log).spawn(); + let a2 = Node::new(&Log::default()).spawn(); a2.child_handle().stop(); assert_eq!(exit_of(&a2).await, ExitReason::Normal); a1.request(Link(a2.child_handle())).await.unwrap(); @@ -433,8 +433,8 @@ async fn late_link_ends_a_non_trapping_caller() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::new(&log).start(); - let a2 = Node::new(&Log::default()).start(); + let a1 = Node::new(&log).spawn(); + let a2 = Node::new(&Log::default()).spawn(); a2.child_handle().stop(); exit_of(&a2).await; a1.request(Link(a2.child_handle())).await.unwrap(); @@ -459,9 +459,12 @@ async fn unlink_fences_a_notice_taken_at_death() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::trapping(&log).start(); + let a1 = Node::trapping(&log).spawn(); let (release, gate) = std::sync::mpsc::channel(); - let a2 = Node::holding_drop(gate).start_with_backend(Backend::Thread); + let a2 = Node::holding_drop(gate) + .with_options(Backend::Thread) + .spawn() + .unwrap(); a1.request(Link(a2.child_handle())).await.unwrap(); a2.request(Sync()).await.unwrap(); a2.context().stop_with_reason(ExitReason::Shutdown(None)); @@ -498,9 +501,12 @@ async fn demonitor_fences_a_down() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::new(&log).start(); + let a1 = Node::new(&log).spawn(); let (release, gate) = std::sync::mpsc::channel(); - let a2 = Node::holding_drop(gate).start_with_backend(Backend::Thread); + let a2 = Node::holding_drop(gate) + .with_options(Backend::Thread) + .spawn() + .unwrap(); let monitor = a1.request(Monitor(a2.child_handle())).await.unwrap(); a2.child_handle().stop(); past_death(&a2).await; @@ -547,8 +553,8 @@ async fn monitor_delivers_the_final_reason() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::new(&log).start(); - let a2 = Node::new(&Log::default()).start(); + let a1 = Node::new(&log).spawn(); + let a2 = Node::new(&Log::default()).spawn(); a1.request(Monitor(a2.child_handle())).await.unwrap(); a2.child_handle() .stop_with_reason(ExitReason::Shutdown(None)); @@ -578,9 +584,12 @@ async fn parent_exit_ends_a_trapping_child_through_stopped() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::new(&Log::default()).start(); + let a1 = Node::new(&Log::default()).spawn(); a1.request(Sync()).await.unwrap(); - let a2 = Node::trapping(&log).start_linked(&a1.context()); + let a2 = Node::trapping(&log) + .start_link(&a1.context()) + .actor_ref() + .unwrap(); a2.request(Sync()).await.unwrap(); a1.child_handle() .stop_with_reason(ExitReason::Shutdown(None)); @@ -610,8 +619,8 @@ async fn link_notice_ends_a_non_trapping_peer() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::new(&log).start(); - let a2 = Node::new(&Log::default()).start(); + let a1 = Node::new(&log).spawn(); + let a2 = Node::new(&Log::default()).spawn(); a1.request(Link(a2.child_handle())).await.unwrap(); assert!(matches!( a2.request(Boom()).await, @@ -647,7 +656,7 @@ fn runtime_drop_ends_killed() { .build() .unwrap(); let a1 = runtime.block_on(async { - let a1 = Node::new(&log).start(); + let a1 = Node::new(&log).spawn(); a1.request(Sync()).await.unwrap(); a1 }); @@ -690,7 +699,7 @@ fn threads_trapped_exit_then_kill() { } use threads::ActorStart as _; let log = Log::default(); - let a1 = Native(log.clone()).start(); + let a1 = Native(log.clone()).spawn(); a1.request(Ping).unwrap(); a1.child_handle().exit(ExitReason::Shutdown(None)); a1.request(Ping).unwrap(); @@ -722,8 +731,8 @@ async fn message_then_death_notice() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let a1 = Node::trapping(&log).start(); - let a2 = Node::new(&Log::default()).start(); + let a1 = Node::trapping(&log).spawn(); + let a2 = Node::new(&Log::default()).spawn(); a1.request(Sync()).await.unwrap(); a2.request(Link(a1.child_handle())).await.unwrap(); a2.request(NoteThenExit(a1.clone())).await.unwrap(); @@ -795,8 +804,8 @@ fn threads_actor_message() { } use threads::ActorStart as _; let log = Log::default(); - let a1 = Native(log.clone()).start(); - let a2 = Native(Log::default()).start(); + let a1 = Native(log.clone()).spawn(); + let a2 = Native(Log::default()).spawn(); a2.request(Forward(a1.clone())).unwrap(); a1.request(Ping).unwrap(); assert_eq!(logged(&log), ["hello"]); @@ -836,8 +845,8 @@ async fn direct_dispatch_preserves_causal_sender_and_lifecycle_trace() { let _serial = SERIAL.lock().await; lifecycle_trace::record_lifecycle(); let log = Log::default(); - let peer = Node::new(&log).start(); - let sender = DirectForwarder(peer.clone()).start(); + let peer = Node::new(&log).spawn(); + let sender = DirectForwarder(peer.clone()).spawn(); sender.request(ForwardNow).await.unwrap(); peer.request(Sync()).await.unwrap(); assert_eq!(exit_of(&sender).await, ExitReason::Shutdown(None)); diff --git a/concurrency/tests/link_fence_loom.rs b/concurrency/tests/link_fence_loom.rs index 605b167a4..336b66486 100644 --- a/concurrency/tests/link_fence_loom.rs +++ b/concurrency/tests/link_fence_loom.rs @@ -332,6 +332,50 @@ fn mutual_death_delivers_at_most_one_notice() { }); } +/// The commit of a `start_link` (unified start, stage 1) racing the +/// parent's exit begin and death point: the edge is installed only while +/// the parent's exit has not begun (read under the parent's exit request, +/// inside the table transaction), and an installed edge always gets its +/// notice at the parent's D. A refused commit leaves no record on either +/// side (`EdgeOnlyToRunningParent`, `NoEdgeToDeadParent`, `AbortInvisible`). +#[test] +fn start_edge_commit_races_parent_exit_begin() { + model("start_edge_commit_races_parent_exit_begin", || { + let (parent, child) = (Arc::new(Actor::new(1)), Arc::new(Actor::new(2))); + let exiting = Arc::new(loom::sync::Mutex::new(false)); + let exit = { + let (parent, exiting) = (parent.clone(), exiting.clone()); + spawn(move || { + *exiting.lock().unwrap() = true; + die(&parent, ExitReason::Shutdown, &[]); + }) + }; + let installed = table::install_start_edge( + &parent.target, + &parent.links, + &child.target, + &child.links, + 1, + || !*exiting.lock().unwrap(), + ); + exit.join().unwrap(); + if installed { + assert_eq!( + child.log(), + vec![Notice { + from: parent.id(), + gen: 1, + reason: ExitReason::Shutdown + }] + ); + } else { + assert!(child.log().is_empty()); + assert_eq!(child.records(), 0, "a refused commit left a record"); + } + assert_eq!(parent.records(), 0); + }); +} + /// Concurrent `link` from both ends: one edge, symmetric records. #[test] fn concurrent_links_install_one_edge() { diff --git a/concurrency/tests/linked_exit_macro.rs b/concurrency/tests/linked_exit_macro.rs index ddec401d1..c39e6f0ad 100644 --- a/concurrency/tests/linked_exit_macro.rs +++ b/concurrency/tests/linked_exit_macro.rs @@ -35,9 +35,9 @@ mod asynchronous { fn macro_receives_linked_exit() { spawned_rt::tasks::sim_test(|_| async { let (tx, rx) = tokio::sync::oneshot::channel(); - let observer = Observer { seen: Some(tx) }.start(); + let observer = Observer { seen: Some(tx) }.spawn(); observer.context().trap_exit(true); - let child = Failing.start_linked(&observer.context()); + let child = Failing.start_link(&observer.context()).actor_ref().unwrap(); let event = spawned_rt::tasks::timeout(std::time::Duration::from_secs(2), rx) .await .unwrap() @@ -51,7 +51,7 @@ mod asynchronous { } mod synchronous { use super::*; - use spawned_concurrency::threads::{Actor, ActorStart, Context}; + use spawned_concurrency::threads::{Actor, ActorStart, Context, Handler, Init}; struct Observer { seen: std::sync::mpsc::Sender, } @@ -67,20 +67,40 @@ mod synchronous { ctx.stop(); } } + // A failed start_link consumes the trapping parent's Exit (OTP), so the + // child fails right after it is ready, in its continuation. + struct Boom; + impl spawned_concurrency::message::Message for Boom { + type Result = (); + } struct Failing; #[actor] impl Failing { #[started] - fn begin(&mut self, _: &Context) { - panic!("linked startup failure"); + fn begin(&mut self, _: &Context) -> Init { + Init::continue_with(Boom) + } + } + impl Handler for Failing { + fn handle( + &mut self, + _: Boom, + _: spawned_concurrency::PendingReply<'_, ()>, + _: &Context, + ) { + panic!("linked failure after start"); } } #[test] fn macro_receives_linked_exit() { let (tx, rx) = std::sync::mpsc::channel(); - let observer = Observer { seen: tx }.start(); + let observer = Observer { seen: tx }.spawn(); observer.context().trap_exit(true); - let child = Failing.start_linked(&observer.context()); + let child = Failing + .start_link(&observer.context()) + .expect("start_link") + .started() + .expect("ready"); let event = rx.recv_timeout(std::time::Duration::from_secs(2)).unwrap(); assert_eq!(event.from, child.id()); assert!(matches!(event.reason, ExitReason::Panic(_))); diff --git a/concurrency/tests/mailbox_drop_panic.rs b/concurrency/tests/mailbox_drop_panic.rs index 0dc543ba5..4363ead88 100644 --- a/concurrency/tests/mailbox_drop_panic.rs +++ b/concurrency/tests/mailbox_drop_panic.rs @@ -131,17 +131,21 @@ fn mailbox_drop_child() { let handled = Arc::new(AtomicUsize::new(0)); let dropped = Arc::new(AtomicUsize::new(0)); let victim = match &group { - Some(group) => Victim(handled.clone(), false).start_on(group).unwrap(), - None => { - Victim(handled.clone(), false).start_with_backend(match placement { + Some(group) => Victim(handled.clone(), false) + .with_options(group) + .spawn() + .unwrap(), + None => Victim(handled.clone(), false) + .with_options(match placement { 0 => Backend::Async, 1 => Backend::Blocking, _ => Backend::Thread, }) - } + .spawn() + .unwrap(), }; let (send, mut events) = mpsc::unbounded_channel(); - let observer = Observer(send).start(); + let observer = Observer(send).spawn(); let ctx = observer.context(); ctx.trap_exit(true); ctx.link(&victim.child_handle()); @@ -257,7 +261,7 @@ fn abandoned_receiver_child() { let runtime = builder.worker_threads(1).enable_all().build().unwrap(); { let _entered = runtime.enter(); - let actor = Victim(Arc::default(), state_panics).start(); + let actor = Victim(Arc::default(), state_panics).spawn(); if polled { actor.send(Hold(entered, gate, false)).unwrap(); } @@ -310,7 +314,7 @@ fn abandoned_receiver_child() { .unwrap(); let actor = { let _entered = owner.enter(); - Victim(Arc::default(), true).start() + Victim(Arc::default(), true).spawn() }; let dropped = Arc::new(AtomicUsize::new(0)); for _ in 0..2 { @@ -337,10 +341,11 @@ fn abandoned_receiver_child() { let group = system.async_group("preserve-cause", 1).unwrap(); let actor = if managed { Victim(Arc::default(), state_panics) - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() } else { - Victim(Arc::default(), state_panics).start() + Victim(Arc::default(), state_panics).spawn() }; let (entered, arrival) = oneshot::channel(); let (release, gate) = oneshot::channel(); @@ -351,7 +356,7 @@ fn abandoned_receiver_child() { actor.send(Bomb(dropped.clone())).unwrap(); } if matches!(committed, ExitReason::Panic(_)) { - let peer = Victim(Arc::default(), false).start(); + let peer = Victim(Arc::default(), false).spawn(); actor.context().link(&peer.child_handle()); peer.context().stop_with_reason(committed.clone()); tokio::time::timeout(LIMIT, peer.wait_exit()).await.unwrap(); @@ -375,7 +380,7 @@ fn abandoned_receiver_child() { } // An actor-state panic must not unwind through an unguarded mailbox. // After the death point it is a cleanup failure; the reason stays Normal. - let actor = Victim(Arc::default(), true).start(); + let actor = Victim(Arc::default(), true).spawn(); let (entered, arrival) = oneshot::channel(); let (release, gate) = oneshot::channel(); actor.send(Hold(entered, gate, false)).unwrap(); @@ -417,9 +422,9 @@ fn primary_panic_child() { .build() .unwrap(); runtime.block_on(async { - let actor = Victim(Arc::default(), true).start(); + let actor = Victim(Arc::default(), true).spawn(); let (send, mut events) = mpsc::unbounded_channel(); - let observer = Observer(send).start(); + let observer = Observer(send).spawn(); observer.context().trap_exit(true); observer.context().link(&actor.child_handle()); let _monitor = observer.context().monitor(&actor.child_handle()); diff --git a/concurrency/tests/metrics.rs b/concurrency/tests/metrics.rs index 3b87e52a1..c0fe77165 100644 --- a/concurrency/tests/metrics.rs +++ b/concurrency/tests/metrics.rs @@ -97,7 +97,7 @@ fn metrics_conserve_work_across_all_lifecycle_paths() { metrics::configure(metrics::Config::default().timing_sample_every(1)).unwrap(); runtime().block_on(async { let (release, gate) = oneshot::channel(); - let actor = Probe { gate: Some(gate) }.start(); + let actor = Probe { gate: Some(gate) }.spawn(); let replies: Vec<_> = (0..10) .map(|_| { actor @@ -162,7 +162,7 @@ fn metrics_conserve_work_across_all_lifecycle_paths() { )); let (release, gate) = oneshot::channel(); - let actor = Probe { gate: Some(gate) }.start(); + let actor = Probe { gate: Some(gate) }.spawn(); let raw = actor .send_request(Ping, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -179,7 +179,7 @@ fn metrics_conserve_work_across_all_lifecycle_paths() { within!(actor.join()).await; assert_eq!(actor.mailbox_depth(), 0); - let actor = Probe { gate: None }.start(); + let actor = Probe { gate: None }.spawn(); assert!(matches!( actor.request(Crash).await, Err(ActorError::ActorStopped(ExitReason::Panic(_))) @@ -221,7 +221,7 @@ fn metrics_conserve_work_across_all_lifecycle_paths() { let owner = runtime(); let actor = { let _entered = owner.enter(); - Probe { gate: None }.start() + Probe { gate: None }.spawn() }; let (entered_tx, entered) = oneshot::channel(); let (_release, release) = oneshot::channel(); @@ -246,7 +246,7 @@ fn metrics_conserve_work_across_all_lifecycle_paths() { .unwrap(); rt.block_on(async { for backend in [Backend::Blocking, Backend::Thread] { - let actor = Probe { gate: None }.start_with_backend(backend); + let actor = Probe { gate: None }.with_options(backend).spawn().unwrap(); actor.request(Ping).await.unwrap(); actor.context().stop(); within!(actor.join()).await; @@ -290,7 +290,7 @@ fn metrics_conserve_work_across_all_lifecycle_paths() { panic!("native metric panic"); } } - let actor = Native.start(); + let actor = Native.spawn(); actor.request(Ping).unwrap(); let before = metrics::snapshot().requests_started_total; assert_eq!(actor.query().unwrap(), 9); diff --git a/concurrency/tests/metrics_composable.rs b/concurrency/tests/metrics_composable.rs index ee0e9b394..8d57d1dfd 100644 --- a/concurrency/tests/metrics_composable.rs +++ b/concurrency/tests/metrics_composable.rs @@ -25,7 +25,7 @@ impl Handler for Probe { #[test] fn embedded_components_can_add_sampling_requirements_after_actors_start() { - let actor = Probe.start(); + let actor = Probe.spawn(); actor.request(Ping).unwrap(); assert_eq!(metrics::snapshot().queue_wait.count, 1); // The implicit default did not register an application requirement. diff --git a/concurrency/tests/metrics_counts_only.rs b/concurrency/tests/metrics_counts_only.rs index 01c067962..f37270266 100644 --- a/concurrency/tests/metrics_counts_only.rs +++ b/concurrency/tests/metrics_counts_only.rs @@ -23,7 +23,7 @@ impl Handler for Probe { #[test] fn zero_timing_rate_retains_counters_without_timing_samples() { metrics::configure(metrics::Config::default().timing_sample_every(0)).unwrap(); - let actor = Probe.start(); + let actor = Probe.spawn(); for _ in 0..16 { actor.request(Ping).unwrap(); } diff --git a/concurrency/tests/metrics_deferred.rs b/concurrency/tests/metrics_deferred.rs index f75fedfd9..c20faf451 100644 --- a/concurrency/tests/metrics_deferred.rs +++ b/concurrency/tests/metrics_deferred.rs @@ -171,7 +171,7 @@ async fn settle(actor: &ActorRef) { async fn deferred_and_job_families_count_each_outcome_once() { metrics::configure(Config::default().timing_sample_every(1)).unwrap(); let before = metrics::snapshot(); - let actor = Owner::default().start(); + let actor = Owner::default().spawn(); // Replied, and Overloaded at the bound. let first = actor diff --git a/concurrency/tests/metrics_retained.rs b/concurrency/tests/metrics_retained.rs index 0db267858..3b9a84ad8 100644 --- a/concurrency/tests/metrics_retained.rs +++ b/concurrency/tests/metrics_retained.rs @@ -229,8 +229,8 @@ impl tasks::Handler for AsyncWorker { async fn async_pair(round: usize, i: usize) { use tasks::ActorStart; - let peer = AsyncWorker::new(format!("retained-a-{round}-{i}-peer")).start(); - let worker = AsyncWorker::new(format!("retained-a-{round}-{i}")).start(); + let peer = AsyncWorker::new(format!("retained-a-{round}-{i}-peer")).spawn(); + let worker = AsyncWorker::new(format!("retained-a-{round}-{i}")).spawn(); worker.request(Watch(peer.child_handle())).await.unwrap(); for _ in 0..4 { assert_eq!(worker.request(Ask).await.unwrap(), 1); @@ -257,9 +257,11 @@ async fn async_pair(round: usize, i: usize) { } else { (&peer, &worker) }; - first.context().stop(); + // An administrative stop (not the actor's own ctx.stop(), which inside a + // started() still running is Init::stop: no stopped(), so no unregister). + first.child_handle().stop(); first.join().await; - second.context().stop(); + second.child_handle().stop(); second.join().await; assert!(orphan.await.is_err()); } @@ -354,8 +356,8 @@ impl threads::Handler for NativeWorker { fn native_pair(round: usize, i: usize) { use threads::ActorStart; - let peer = NativeWorker::new(format!("retained-n-{round}-{i}-peer")).start(); - let worker = NativeWorker::new(format!("retained-n-{round}-{i}")).start(); + let peer = NativeWorker::new(format!("retained-n-{round}-{i}-peer")).spawn(); + let worker = NativeWorker::new(format!("retained-n-{round}-{i}")).spawn(); worker.request(Watch(peer.child_handle())).unwrap(); for _ in 0..4 { assert_eq!(worker.request(Ask).unwrap(), 1); @@ -375,9 +377,11 @@ fn native_pair(round: usize, i: usize) { } else { (&peer, &worker) }; - first.context().stop(); + // An administrative stop (not the actor's own ctx.stop(), which inside a + // started() still running is Init::stop: no stopped(), so no unregister). + first.child_handle().stop(); first.join(); - second.context().stop(); + second.child_handle().stop(); second.join(); assert!(orphan.recv().is_err()); } @@ -482,7 +486,7 @@ fn threads_timers_do_not_accumulate_on_a_running_actor() { // `send_interval` (lives until it stops) itself. let before = quiet()[11]; let baseline = before + 1; - let actor = NativeWorker::new("retained-timer-host".into()).start(); + let actor = NativeWorker::new("retained-timer-host".into()).spawn(); for _ in 0..100 { threads::send_after(Duration::from_millis(1), actor.context(), Tick); } @@ -635,7 +639,7 @@ impl tasks::Handler for AsyncExpirer { async fn async_expiry(dropped: std::sync::Arc) { use tasks::ActorStart; - let owner = AsyncExpirer::default().start(); + let owner = AsyncExpirer::default().spawn(); owner.request(StallJob(dropped)).await.unwrap(); let expiring: Vec<_> = (0..3) .map(|_| owner.send_request(Park, Deadline::after(EXPIRE)).unwrap()) @@ -693,7 +697,7 @@ impl threads::Handler for NativeExpirer { fn native_expiry(dropped: std::sync::Arc) { use threads::ActorStart; - let owner = NativeExpirer::default().start(); + let owner = NativeExpirer::default().spawn(); owner.request(StallJob(dropped)).unwrap(); let expiring: Vec<_> = (0..3) .map(|_| owner.send_request(Park, Deadline::after(EXPIRE)).unwrap()) diff --git a/concurrency/tests/metrics_sampling.rs b/concurrency/tests/metrics_sampling.rs index 28efcd77f..d6511d75d 100644 --- a/concurrency/tests/metrics_sampling.rs +++ b/concurrency/tests/metrics_sampling.rs @@ -32,7 +32,7 @@ fn sampling_retains_every_counter_and_scales_only_timing_observations() { .build() .unwrap(); rt.block_on(async { - let actor = Probe.start(); + let actor = Probe.spawn(); for _ in 0..16 { actor.request(Ping).await.unwrap(); } diff --git a/concurrency/tests/native_jobs.rs b/concurrency/tests/native_jobs.rs index 7ff3545d3..3910a132c 100644 --- a/concurrency/tests/native_jobs.rs +++ b/concurrency/tests/native_jobs.rs @@ -62,15 +62,17 @@ async fn shutdown_waits_for_the_blocking_job(borrowed: bool) { } else { system.async_group("owned-jobs", 1).unwrap() }; + use spawned_concurrency::tasks::ActorStart as _; let (release, gate) = mpsc::channel(); let running = Arc::new(AtomicBool::new(false)); - let actor = group - .start(Gated { - gate: Mutex::new(Some(gate)), - running: running.clone(), - }) - .unwrap(); - actor.ready().await.unwrap(); + let actor = Gated { + gate: Mutex::new(Some(gate)), + running: running.clone(), + } + .with_options(&group) + .spawn() + .unwrap(); + actor.ready().await.unwrap().started().expect("started"); for _ in 0..500 { if running.load(Ordering::SeqCst) { break; @@ -205,7 +207,7 @@ async fn a_job_admitted_after_its_owner_exited_never_runs() { impl Handler for Owner { async fn handle(&mut self, _: Done, _: PendingReply<'_, ()>, _: &Context) {} } - let actor = Owner.start(); + let actor = Owner.spawn(); let ctx = actor.context(); let owner = actor.clone(); let ran = Arc::new(AtomicBool::new(false)); @@ -260,8 +262,8 @@ async fn an_unused_permit_does_not_hold_the_scope(borrowed: bool) { } else { system.async_group("unused-permit-owned", 1).unwrap() }; - let actor = Owner.start_on(&group).unwrap(); - actor.ready().await.unwrap(); + let actor = Owner.with_options(&group).spawn().unwrap(); + actor.ready().await.unwrap().started().expect("started"); // Reserved through a cloned context and kept outside the actor, unused. let permit: BlockingJobPermit = actor .context() @@ -330,7 +332,10 @@ fn job_thread_spawn_failure_child() { } let (started, ready) = mpsc::channel(); // The actor runs on its own thread; this thread has no Tokio runtime. - let actor = ReadyOwner(started).start_with_backend(Backend::Thread); + let actor = ReadyOwner(started) + .with_options(Backend::Thread) + .spawn() + .unwrap(); ready .recv_timeout(Duration::from_secs(30)) .expect("actor runtime did not finish starting before thread fault injection"); diff --git a/concurrency/tests/native_mailbox_drop.rs b/concurrency/tests/native_mailbox_drop.rs index 73220a2ef..42202f5b2 100644 --- a/concurrency/tests/native_mailbox_drop.rs +++ b/concurrency/tests/native_mailbox_drop.rs @@ -93,7 +93,7 @@ fn native_drop_child() { release: gate, state_panics, } - .start(); + .spawn(); arrival.recv_timeout(LIMIT).unwrap(); let dropped = Arc::new(AtomicUsize::new(0)); let replies: Vec<_> = (0..2) @@ -138,7 +138,7 @@ fn native_drop_child() { release: gate, state_panics: true, } - .start(); + .spawn(); arrival.recv_timeout(LIMIT).unwrap(); actor.send(Crash).unwrap(); let dropped = Arc::new(AtomicUsize::new(0)); diff --git a/concurrency/tests/nested_supervision.rs b/concurrency/tests/nested_supervision.rs index d05677509..407efc417 100644 --- a/concurrency/tests/nested_supervision.rs +++ b/concurrency/tests/nested_supervision.rs @@ -511,8 +511,15 @@ async fn aborted_initial_startup_rolls_back_before_a_later_stuck_shutdown_target .unwrap(); let release = Arc::new(tokio::sync::Notify::new()); use spawned_concurrency::tasks::ActorStart; - let later = HeldStop(release.clone()).start_on(&group).unwrap(); - within!(later.ready()).await.unwrap(); + let later = HeldStop(release.clone()) + .with_options(&group) + .spawn() + .unwrap(); + within!(later.ready()) + .await + .unwrap() + .started() + .expect("started"); let owner = system.clone(); let shutdown = spawned_rt::tasks::spawn(async move { owner.shutdown(Duration::from_secs(30)).await }); diff --git a/concurrency/tests/otp_exit_signals.rs b/concurrency/tests/otp_exit_signals.rs index 9f44219cd..467fa4962 100644 --- a/concurrency/tests/otp_exit_signals.rs +++ b/concurrency/tests/otp_exit_signals.rs @@ -207,10 +207,10 @@ mod tasks_mode { Seen::default(), Seen::default(), ); - let target = Node::new(&target_seen, true).start(); - let peer = Node::new(&peer_seen, false).start(); - let trapper = Node::new(&trapper_seen, true).start(); - let watcher = Node::new(&watcher_seen, false).start(); + let target = Node::new(&target_seen, true).spawn(); + let peer = Node::new(&peer_seen, false).spawn(); + let trapper = Node::new(&trapper_seen, true).spawn(); + let watcher = Node::new(&watcher_seen, false).spawn(); settle(&target).await; settle(&trapper).await; let t = target.child_handle(); @@ -263,7 +263,7 @@ mod tasks_mode { let mut node = Node::new(&seen, false); node.in_stopped = Some((entered_tx, release_rx)); node.panic_in_stopped = true; - let actor = node.start(); + let actor = node.spawn(); settle(&actor).await; let child = actor.child_handle(); child.stop_with_reason(ExitReason::Shutdown(None)); @@ -279,7 +279,7 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn administrative_kill_reason_runs_stopped_and_ends_kill() { let seen = Seen::default(); - let actor = Node::new(&seen, true).start(); + let actor = Node::new(&seen, true).spawn(); settle(&actor).await; actor.child_handle().stop_with_reason(ExitReason::Kill); assert_eq!(ended(&actor.child_handle()).await, ExitReason::Kill); @@ -292,7 +292,7 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn exit_signals_follow_exit_signal_2() { let plain_seen = Seen::default(); - let plain = Node::new(&plain_seen, false).start(); + let plain = Node::new(&plain_seen, false).spawn(); settle(&plain).await; plain.child_handle().exit(ExitReason::Normal); settle(&plain).await; @@ -302,7 +302,7 @@ mod tasks_mode { ); let trap_seen = Seen::default(); - let trapper = Node::new(&trap_seen, true).start(); + let trapper = Node::new(&trap_seen, true).spawn(); settle(&trapper).await; trapper.child_handle().exit(ExitReason::Normal); trapper.child_handle().exit(ExitReason::Shutdown(None)); @@ -333,8 +333,8 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn ctx_exit_sends_from_the_calling_actor() { let (a_seen, b_seen) = (Seen::default(), Seen::default()); - let a = Node::new(&a_seen, false).start(); - let b = Node::new(&b_seen, true).start(); + let a = Node::new(&a_seen, false).spawn(); + let b = Node::new(&b_seen, true).spawn(); settle(&b).await; let bh = b.child_handle(); let bh2 = bh.clone(); @@ -360,7 +360,7 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn registrations_after_death_report_noproc() { let dead_seen = Seen::default(); - let dead = Node::new(&dead_seen, false).start(); + let dead = Node::new(&dead_seen, false).spawn(); dead.child_handle() .stop_with_reason(ExitReason::Panic("gone".into())); assert!(matches!( @@ -370,7 +370,7 @@ mod tasks_mode { let d = dead.child_handle(); let trap_seen = Seen::default(); - let trapper = Node::new(&trap_seen, true).start(); + let trapper = Node::new(&trap_seen, true).spawn(); let (d1, d2) = (d.clone(), d.clone()); on(&trapper, move |ctx| { ctx.link(&d1); @@ -387,7 +387,7 @@ mod tasks_mode { assert_eq!(downs[0].reason, ExitReason::NoProc); let try_seen = Seen::default(); - let trier = Node::new(&try_seen, false).start(); + let trier = Node::new(&try_seen, false).spawn(); let result = Arc::new(Mutex::new(None)); let (d3, r) = (d.clone(), result.clone()); on(&trier, move |ctx| { @@ -399,7 +399,7 @@ mod tasks_mode { assert!(trier.child_handle().is_alive()); let plain_seen = Seen::default(); - let plain = Node::new(&plain_seen, false).start(); + let plain = Node::new(&plain_seen, false).spawn(); let d4 = d.clone(); on(&plain, move |ctx| ctx.link(&d4)).await; assert_eq!(ended(&plain.child_handle()).await, ExitReason::NoProc); @@ -416,8 +416,8 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn link_from_a_dead_actor_is_inert() { let (dead_seen, peer_seen) = (Seen::default(), Seen::default()); - let dead = Node::new(&dead_seen, true).start(); - let peer = Node::new(&peer_seen, true).start(); + let dead = Node::new(&dead_seen, true).spawn(); + let peer = Node::new(&peer_seen, true).spawn(); settle(&peer).await; let kept = dead.context(); dead.child_handle().stop(); @@ -425,7 +425,7 @@ mod tasks_mode { kept.link(&peer.child_handle()); assert_eq!(kept.try_link(&peer.child_handle()), Ok(())); // The live peer gets no edge: its own exit notifies nobody. - let gone = Node::new(&Seen::default(), false).start(); + let gone = Node::new(&Seen::default(), false).spawn(); kept.link(&gone.child_handle()); gone.child_handle().stop(); ended(&gone.child_handle()).await; @@ -526,7 +526,10 @@ mod tasks_mode { reply: spawned_concurrency::PendingReply<'_, ChildHandle>, ctx: &Context, ) { - let child = Node::new(&self.child_seen, true).start_linked(ctx); + let child = Node::new(&self.child_seen, true) + .start_link(ctx) + .actor_ref() + .unwrap(); reply.reply(child.child_handle()); self.child = Some(child); } @@ -542,7 +545,7 @@ mod tasks_mode { child: None, child_seen: child_seen.clone(), } - .start(); + .spawn(); let child = parent.request(Spawn).await.unwrap(); parent .child_handle() @@ -582,7 +585,7 @@ mod tasks_mode { } let seen = Seen::default(); let (go, gate) = tokio::sync::oneshot::channel(); - let actor = Slow(seen.clone(), Some(gate)).start(); + let actor = Slow(seen.clone(), Some(gate)).spawn(); actor.child_handle().exit(ExitReason::Shutdown(None)); go.send(()).unwrap(); assert_eq!(actor.request(Ping).await.unwrap(), 7); @@ -603,7 +606,7 @@ mod tasks_mode { let (release, release_rx) = tokio::sync::oneshot::channel(); let mut node = Node::new(&seen, false); node.in_stopped = Some((entered_tx, release_rx)); - let actor = node.start(); + let actor = node.spawn(); settle(&actor).await; let child = actor.child_handle(); child.stop(); @@ -724,7 +727,7 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn begun_exits_are_pending() { let seen = Seen::default(); - let actor = Node::new(&seen, false).start(); + let actor = Node::new(&seen, false).spawn(); let pending = Arc::new(Mutex::new(Vec::new())); let p = pending.clone(); on(&actor, move |ctx| { @@ -735,7 +738,7 @@ mod tasks_mode { .await; assert_eq!(ended(&actor.child_handle()).await, ExitReason::Normal); assert_eq!(*pending.lock().unwrap(), vec![false, true]); - let actor = Node::new(&seen, false).start(); + let actor = Node::new(&seen, false).spawn(); let handle = actor.child_handle(); let p = pending.clone(); on(&actor, move |ctx| { @@ -837,11 +840,11 @@ mod tasks_mode { .unwrap(); let (killed, signalled, trapping) = runtime.block_on(async { let seen = Seen::default(); - let killed = Node::new(&seen, true).start(); + let killed = Node::new(&seen, true).spawn(); settle(&killed).await; killed.child_handle().kill(); assert_eq!(ended(&killed.child_handle()).await, ExitReason::Killed); - let signalled = Node::new(&seen, false).start(); + let signalled = Node::new(&seen, false).spawn(); settle(&signalled).await; signalled.child_handle().exit(ExitReason::Shutdown(None)); assert_eq!( @@ -850,7 +853,7 @@ mod tasks_mode { ); assert!(!seen.stopped()); // stopped() runs: nothing logged. - let trapping = Node::new(&seen, true).start(); + let trapping = Node::new(&seen, true).spawn(); settle(&trapping).await; trapping.child_handle().stop(); assert_eq!(ended(&trapping.child_handle()).await, ExitReason::Normal); @@ -868,7 +871,7 @@ mod tasks_mode { assert!(capture.skipped(trapping).is_empty()); // Runtime drop before stopped() began. let dropped = runtime.block_on(async { - let actor = Node::new(&Seen::default(), true).start(); + let actor = Node::new(&Seen::default(), true).spawn(); settle(&actor).await; actor.id() }); @@ -916,7 +919,7 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn queued_and_stored_requests_get_killed_after_a_kill() { let seen = Seen::default(); - let actor = Node::new(&seen, false).start(); + let actor = Node::new(&seen, false).spawn(); let stored = actor .send_request(Hold, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -937,7 +940,7 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn queued_request_gets_the_untrapped_signals_reason() { let seen = Seen::default(); - let actor = Node::new(&seen, false).start(); + let actor = Node::new(&seen, false).spawn(); settle(&actor).await; let release = block(&actor).await; let queued = actor @@ -962,7 +965,7 @@ mod tasks_mode { let (release, release_rx) = tokio::sync::oneshot::channel(); let mut node = Node::new(&seen, false); node.in_stopped = Some((entered_tx, release_rx)); - let actor = node.start(); + let actor = node.spawn(); settle(&actor).await; let mut crashed = actor .send_request(Boom, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) @@ -998,7 +1001,7 @@ mod tasks_mode { let mut node = Node::new(&seen, false); node.in_stopped = Some((entered_tx, release_rx)); node.drop_held_in_stopped = true; - let actor = node.start(); + let actor = node.spawn(); let mut stored = actor .send_request(Hold, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -1026,7 +1029,7 @@ mod tasks_mode { let (release, release_rx) = tokio::sync::oneshot::channel(); let mut node = Node::new(&seen, true); node.in_stopped = Some((entered_tx, release_rx)); - let actor = node.start(); + let actor = node.spawn(); settle(&actor).await; let child = actor.child_handle(); child.stop_with_reason(ExitReason::Shutdown(None)); @@ -1299,8 +1302,8 @@ mod threads_mode { fn kill_exit_and_noproc_in_threads_mode() { // Kill: untrappable, Killed, no stopped(); the trapping peer gets Killed. let (target_seen, peer_seen) = (Seen::default(), Seen::default()); - let target = Node(target_seen.clone(), true).start(); - let peer = Node(peer_seen.clone(), true).start(); + let target = Node(target_seen.clone(), true).spawn(); + let peer = Node(peer_seen.clone(), true).spawn(); assert_eq!(target.request(Ping).unwrap(), 7); assert_eq!(peer.request(Ping).unwrap(), 7); peer.context().link(&target.child_handle()); @@ -1320,7 +1323,7 @@ mod threads_mode { // exit(Normal) ignored by a non-trapping actor; exit(Shutdown) ends it // without stopped(). let plain_seen = Seen::default(); - let plain = Node(plain_seen.clone(), false).start(); + let plain = Node(plain_seen.clone(), false).spawn(); plain.child_handle().exit(ExitReason::Normal); assert_eq!(plain.request(Ping).unwrap(), 7); plain.child_handle().exit(ExitReason::Shutdown(None)); @@ -1406,7 +1409,7 @@ mod threads_mode { #[test] fn queued_and_stored_requests_get_killed_after_a_kill() { - let actor = Server::default().start(); + let actor = Server::default().spawn(); let stored = actor .send_request(Hold, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -1426,7 +1429,7 @@ mod threads_mode { #[test] fn panicked_request_and_dropped_token_wait_for_death() { let (server, entered, release) = gated(); - let actor = server.start(); + let actor = server.spawn(); let mut stored = actor .send_request(Hold, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -1453,7 +1456,7 @@ mod threads_mode { #[test] fn admitted_while_exiting_noproc_after_death() { let (server, entered, release) = gated(); - let actor = server.start(); + let actor = server.spawn(); assert_eq!(actor.request(Ping).unwrap(), 7); actor .child_handle() @@ -1537,14 +1540,20 @@ mod threads_mode { reply: spawned_concurrency::PendingReply<'_, ActorRef>, ctx: &Context, ) { - reply.reply(Late(self.0.clone()).start_linked(ctx)); + reply.reply( + Late(self.0.clone()) + .start_link(ctx) + .expect("start_link") + .started() + .expect("ready"), + ); } } #[test] fn queued_work_sees_the_parents_pending_exit() { let log: Log = Arc::default(); - let parent = Parent(log.clone()).start(); + let parent = Parent(log.clone()).spawn(); let child = parent.request(Spawn).unwrap(); child.request(Probe).unwrap(); let (release, gate) = mpsc::channel(); diff --git a/concurrency/tests/panic_safety.rs b/concurrency/tests/panic_safety.rs index 5ebbabe86..82ba23499 100644 --- a/concurrency/tests/panic_safety.rs +++ b/concurrency/tests/panic_safety.rs @@ -626,7 +626,7 @@ mod threads_mode { fn spawn_trigger(&self) -> ChildHandle { let gate = Arc::new(Gate::default()); let trigger = ThreadVictim(Victim::new(None, &gate)) - .start() + .spawn() .child_handle(); self.triggers.lock().unwrap().push(trigger.clone()); trigger @@ -661,8 +661,8 @@ mod threads_mode { let observer = ThreadObserver(Observer { events: events.clone(), }) - .start(); - let victim = ThreadVictim(Victim::new(Some(fault), &gate)).start(); + .spawn(); + let victim = ThreadVictim(Victim::new(Some(fault), &gate)).spawn(); let h = ThreadsHarness { victim, observer, @@ -747,10 +747,13 @@ mod tasks_mode { fn start(&self, actor: A) -> ActorRef { let _entered = self.rt.enter(); match self.place { - Place::Async => actor.start_with_backend(Backend::Async), - Place::Blocking => actor.start_with_backend(Backend::Blocking), - Place::Thread => actor.start_with_backend(Backend::Thread), - _ => self.group.as_ref().unwrap().start(actor).unwrap(), + Place::Async => actor.with_options(Backend::Async).spawn().unwrap(), + Place::Blocking => actor.with_options(Backend::Blocking).spawn().unwrap(), + Place::Thread => actor.with_options(Backend::Thread).spawn().unwrap(), + _ => actor + .with_options(self.group.as_ref().unwrap()) + .spawn() + .unwrap(), } } fn bounded(&self, f: F) -> F::Output { diff --git a/concurrency/tests/process_failures.rs b/concurrency/tests/process_failures.rs index 2967c30ba..6d0ee9c6d 100644 --- a/concurrency/tests/process_failures.rs +++ b/concurrency/tests/process_failures.rs @@ -67,7 +67,7 @@ fn isolated_process_fixture() { path: path.into(), abort: std::env::var("SPAWNED_PROCESS_MODE").unwrap() == "abort", } - .start() + .spawn() .request_with_timeout(Crash, Duration::from_secs(60)) .await .unwrap(); diff --git a/concurrency/tests/protocol_timeouts.rs b/concurrency/tests/protocol_timeouts.rs index 5fc61f449..85a7d7928 100644 --- a/concurrency/tests/protocol_timeouts.rs +++ b/concurrency/tests/protocol_timeouts.rs @@ -103,7 +103,7 @@ mod asynchronous { #[cfg_attr(not(spawned_sim), tokio::test(start_paused = true))] #[cfg_attr(spawned_sim, spawned_macros::sim_test)] async fn protocol_method_and_handle_overrides_have_explicit_precedence() { - let actor = Worker.start(); + let actor = Worker.spawn(); // Virtual time: larger than the old five-second default. assert_eq!(actor.work(6000).await.unwrap(), 6000); assert_eq!( @@ -127,7 +127,7 @@ mod asynchronous { actor.context().stop(); actor.join().await; - let actor = Worker.start(); + let actor = Worker.spawn(); assert!(matches!( actor .with_request_timeout(Duration::from_millis(1)) @@ -138,7 +138,7 @@ mod asynchronous { actor.context().stop(); actor.join().await; - let actor = Worker.start(); + let actor = Worker.spawn(); assert!(matches!( actor.legacy(6000).await, Err(ActorError::RequestTimeout) @@ -183,7 +183,7 @@ mod synchronous { } #[test] fn method_override_and_configured_type_erasure_work_without_async() { - let actor = Worker.start(); + let actor = Worker.spawn(); let configured = actor.with_request_timeout(Duration::from_secs(1)); assert_eq!(configured.brief(25).unwrap(), 25); let erased = configured.to_timed_ref(); @@ -258,7 +258,7 @@ mod synchronous { } #[test] fn configured_handles_do_not_shadow_protocol_method_names() { - let actor = Worker.start(); + let actor = Worker.spawn(); let configured = actor.with_request_timeout(Duration::from_secs(1)); assert_eq!(configured.timeout().unwrap(), 31); assert_eq!(configured.actor_ref().unwrap(), 32); @@ -283,7 +283,7 @@ mod synchronous { } #[test] fn timeout_expression_precedes_argument_moves_without_capturing_user_names() { - let actor = Worker.start(); + let actor = Worker.spawn(); assert_eq!( actor .owned(Job { @@ -314,7 +314,7 @@ mod synchronous { entered, release: wait, } - .start(); + .spawn(); ready.recv_timeout(Duration::from_secs(5)).unwrap(); assert!(matches!( futures::executor::block_on(actor.zero()), diff --git a/concurrency/tests/reply_trace.rs b/concurrency/tests/reply_trace.rs index 7159647e0..6925b35c1 100644 --- a/concurrency/tests/reply_trace.rs +++ b/concurrency/tests/reply_trace.rs @@ -263,7 +263,7 @@ impl Handler for Backend { /// Start an owner whose reply slots are traced. fn start() -> ActorRef { - let actor = Owner::default().start(); + let actor = Owner::default().spawn(); lifecycle_trace::watch_deferred(id(&actor)); actor } @@ -572,12 +572,12 @@ async fn owner_exit_cancels_the_job() { /// Start an owner with a backend B for its jobs. fn start_with_backend() -> (ActorRef, ActorRef) { - let backend = Backend::default().start(); + let backend = Backend::default().spawn(); let actor = Owner { backend: Some(backend.clone()), ..Owner::default() } - .start(); + .spawn(); lifecycle_trace::watch_deferred(id(&actor)); (actor, backend) } @@ -754,8 +754,8 @@ mod threads_mode { #[test] fn blocking_job_asks_downstream() { - let backend = Backend.start(); - let actor = Owner(backend).start(); + let backend = Backend.spawn(); + let actor = Owner(backend).spawn(); let owner: u64 = actor.id().to_string().parse().unwrap(); lifecycle_trace::watch_deferred(owner); assert!(matches!(actor.request(Piped), Ok(4))); diff --git a/concurrency/tests/request_histories.rs b/concurrency/tests/request_histories.rs index 3bd6e5225..0a71d54b6 100644 --- a/concurrency/tests/request_histories.rs +++ b/concurrency/tests/request_histories.rs @@ -495,7 +495,10 @@ fn run(rt: &tokio::runtime::Runtime, mode: Mode, bug: Bug, program: &Program) -> call.complete(reply); } } - let actor = store::AsyncStore(Store::new(bug)).start_with_backend(backend); + let actor = store::AsyncStore(Store::new(bug)) + .with_options(backend) + .spawn() + .unwrap(); for &cmd in prefix { issue(&actor, &recorder, 0, cmd).await; } @@ -532,7 +535,7 @@ fn run(rt: &tokio::runtime::Runtime, mode: Mode, bug: Bug, program: &Program) -> call.complete(reply); } } - let actor = NativeStore(Store::new(bug)).start(); + let actor = NativeStore(Store::new(bug)).spawn(); for &cmd in prefix { issue(&actor, &recorder, 0, cmd); } diff --git a/concurrency/tests/request_timer.rs b/concurrency/tests/request_timer.rs index 5466266b7..69549f5c8 100644 --- a/concurrency/tests/request_timer.rs +++ b/concurrency/tests/request_timer.rs @@ -28,7 +28,7 @@ impl Handler for Idle { async fn requests_after_virtual_time_advance_keep_their_full_timeout() { // An absolute wall-clock sleep_until would incorrectly be in the past. tokio::time::advance(Duration::from_secs(120)).await; - let actor = Idle(None).start(); + let actor = Idle(None).spawn(); for duration in [Duration::from_millis(20), Duration::from_millis(40)] { let began = tokio::time::Instant::now(); assert!(matches!( @@ -49,7 +49,7 @@ async fn wall_time_while_paused_cannot_expire_a_new_request() { // would. Tokio's paused clock does not advance during synchronous work. let _ = Deadline::after(Duration::from_secs(1)); std::thread::sleep(Duration::from_millis(100)); - let actor = Idle(None).start(); + let actor = Idle(None).spawn(); let duration = Duration::from_millis(20); let began = tokio::time::Instant::now(); assert!(matches!( @@ -67,7 +67,7 @@ async fn wall_time_while_paused_cannot_expire_a_new_request() { #[spawned_macros::sim_test] async fn overflowing_relative_timeout_remains_unbounded() { - let actor = Idle(None).start(); + let actor = Idle(None).spawn(); let answer = spawned_rt::tasks::timeout( Duration::from_secs(5), actor.request_with_timeout(Park, Duration::MAX), diff --git a/concurrency/tests/resource_cleanup.rs b/concurrency/tests/resource_cleanup.rs index 2fad0fee3..2cb3a479d 100644 --- a/concurrency/tests/resource_cleanup.rs +++ b/concurrency/tests/resource_cleanup.rs @@ -63,7 +63,9 @@ fn timers_release_payloads_on_actor_exit() { ] { let rt = tokio::runtime::Runtime::new().unwrap(); let _entered = rt.enter(); - let actor = tasks::ActorStart::start_with_backend(Sink, backend); + let actor = tasks::ActorStart::with_options(Sink, backend) + .spawn() + .unwrap(); let probe = Arc::new(()); let weak = Arc::downgrade(&probe); let once = tasks::send_after( @@ -78,7 +80,7 @@ fn timers_release_payloads_on_actor_exit() { released(&weak); drop((once, interval)); } - let actor = threads::ActorStart::start(Sink); + let actor = threads::ActorStart::spawn(Sink); let probe = Arc::new(()); let weak = Arc::downgrade(&probe); let once = threads::send_after( @@ -105,7 +107,9 @@ fn pending_async_listener_is_dropped_on_actor_exit() { ] { let rt = tokio::runtime::Runtime::new().unwrap(); rt.block_on(async { - let actor = tasks::ActorStart::start_with_backend(Sink, backend); + let actor = tasks::ActorStart::with_options(Sink, backend) + .spawn() + .unwrap(); let probe = Arc::new(()); let weak = Arc::downgrade(&probe); let stream = futures::stream::unfold(probe, |probe| async move { @@ -133,7 +137,7 @@ fn cancelled_thread_exit_wait_does_not_occupy_blocking_pool() { .enable_all() .build() .unwrap(); - let actor = threads::ActorStart::start(Sink); + let actor = threads::ActorStart::spawn(Sink); let child = actor.child_handle(); let available = rt.block_on(async { assert!( @@ -158,7 +162,7 @@ fn cancelled_thread_exit_wait_does_not_occupy_blocking_pool() { #[test] fn thread_exit_waits_work_inside_futures_executor() { for already_exited in [false, true] { - let actor = threads::ActorStart::start(Sink); + let actor = threads::ActorStart::spawn(Sink); let child = actor.child_handle(); child.stop(); if already_exited { diff --git a/concurrency/tests/semantics_gaps.rs b/concurrency/tests/semantics_gaps.rs index da90882ff..ee0640c50 100644 --- a/concurrency/tests/semantics_gaps.rs +++ b/concurrency/tests/semantics_gaps.rs @@ -168,8 +168,8 @@ mod native { for reason in reasons() { for _ in 0..ROUNDS { let (tx, rx) = mpsc::channel(); - let observer = Observer(tx).start(); - let source = Source.start(); + let observer = Observer(tx).spawn(); + let source = Source.spawn(); observer .request_with_timeout(Watch(source.child_handle()), LIMIT) .unwrap(); @@ -206,7 +206,7 @@ mod native { #[test] fn panicking_request_handler_gives_the_caller_actor_stopped() { - let actor = Panicker.start(); + let actor = Panicker.spawn(); let result = actor.request_with_timeout(Boom, LIMIT); assert!( matches!(result, Err(ActorError::ActorStopped(ExitReason::Panic(_)))), @@ -214,7 +214,7 @@ mod native { ); assert!(matches!(actor.wait_exit(), ExitReason::Panic(_))); - let erased = Panicker.start(); + let erased = Panicker.spawn(); let recipient: Recipient = erased.recipient(); let result = request(&*recipient, Boom, LIMIT); assert!( @@ -222,7 +222,7 @@ mod native { "{result:?}" ); - let raw = Panicker.start(); + let raw = Panicker.spawn(); let receiver = raw .send_request(Boom, spawned_concurrency::Deadline::after(LIMIT)) .unwrap(); @@ -330,8 +330,8 @@ mod tasks { for reason in reasons() { for _ in 0..ROUNDS { let (tx, mut rx) = unbounded_channel(); - let observer = Observer(tx).start_with_backend(backend); - let source = Source.start_with_backend(backend); + let observer = Observer(tx).with_options(backend).spawn().unwrap(); + let source = Source.with_options(backend).spawn().unwrap(); observer .request_with_timeout(Watch(source.child_handle()), LIMIT) .await @@ -380,7 +380,7 @@ mod tasks { #[tokio::test(start_paused = true)] async fn send_after_fires_once_no_earlier_than_its_delay() { let (tx, mut rx) = unbounded_channel(); - let actor = Ticker(tx).start(); + let actor = Ticker(tx).spawn(); let start = Instant::now(); let _timer = send_after(PERIOD, actor.context(), Tick); let fired = next(&mut rx).await; @@ -399,7 +399,7 @@ mod tasks { #[tokio::test(start_paused = true)] async fn send_interval_is_fixed_delay_and_never_catches_up() { let (tx, mut rx) = unbounded_channel(); - let actor = Ticker(tx).start(); + let actor = Ticker(tx).spawn(); let start = Instant::now(); let _timer = send_interval(PERIOD, actor.context(), Tick); tokio::task::yield_now().await; @@ -561,9 +561,9 @@ mod tasks { let group = system.async_group("panics", 1).unwrap(); let mut actors = BACKENDS .iter() - .map(|backend| Panicker.start_with_backend(*backend)) + .map(|backend| Panicker.with_options(*backend).spawn().unwrap()) .collect::>(); - actors.push(Panicker.start_on(&group).unwrap()); + actors.push(Panicker.with_options(&group).spawn().unwrap()); for actor in actors { let result = actor.request_with_timeout(Boom, LIMIT).await; assert!( @@ -574,7 +574,7 @@ mod tasks { assert!(matches!(reason, ExitReason::Panic(_)), "{reason:?}"); } - let erased = Panicker.start(); + let erased = Panicker.spawn(); let recipient: Recipient = erased.recipient(); let result = request(&*recipient, Boom, LIMIT).await; assert!( @@ -582,7 +582,7 @@ mod tasks { "{result:?}" ); - let raw = Panicker.start(); + let raw = Panicker.spawn(); let receiver = raw .send_request(Boom, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -775,7 +775,7 @@ mod native_contract { drops: drops.clone(), send_in_stopped: send_in_stopped.clone(), } - .start(); + .spawn(); let (entered, entered_rx) = mpsc::channel(); let (release, release_rx) = mpsc::channel(); actor @@ -877,7 +877,7 @@ mod native_contract { counter: 0, final_counter: final_counter.clone(), } - .start(); + .spawn(); let replied = Arc::new(AtomicUsize::new(0)); let senders: Vec<_> = (0..4) .map(|t| { @@ -953,7 +953,7 @@ mod native_contract { __reply.reply(__value); } } - let actor = Backlog(Vec::new()).start(); + let actor = Backlog(Vec::new()).spawn(); let (entered, entered_rx) = mpsc::channel(); let (release, release_rx) = mpsc::channel(); actor @@ -1009,7 +1009,7 @@ mod native_contract { #[test] fn dropping_every_actor_ref_keeps_the_actor_running() { let (reports, rx) = mpsc::channel(); - let actor = Looper { reports, count: 0 }.start(); + let actor = Looper { reports, count: 0 }.spawn(); let handle = actor.child_handle(); actor.send(Loop).unwrap(); drop(actor); @@ -1028,7 +1028,7 @@ mod native_contract { impl Actor for Idle {} let mut ids = std::collections::HashSet::new(); for _ in 0..4 { - let batch: Vec<_> = (0..25).map(|_| Idle.start()).collect(); + let batch: Vec<_> = (0..25).map(|_| Idle.spawn()).collect(); for actor in &batch { assert!(ids.insert(actor.id()), "M4: id reused"); actor.child_handle().stop(); @@ -1089,11 +1089,11 @@ mod tasks_contract { } fn start(&self, actor: A, place: Place) -> ActorRef { match place { - Place::Async => actor.start_with_backend(Backend::Async), - Place::Blocking => actor.start_with_backend(Backend::Blocking), - Place::Thread => actor.start_with_backend(Backend::Thread), - Place::AsyncGroup => actor.start_on(&self.async_group).unwrap(), - Place::ThreadGroup => actor.start_on(&self.thread_group).unwrap(), + Place::Async => actor.with_options(Backend::Async).spawn().unwrap(), + Place::Blocking => actor.with_options(Backend::Blocking).spawn().unwrap(), + Place::Thread => actor.with_options(Backend::Thread).spawn().unwrap(), + Place::AsyncGroup => actor.with_options(&self.async_group).spawn().unwrap(), + Place::ThreadGroup => actor.with_options(&self.thread_group).spawn().unwrap(), } } async fn finish(self) { @@ -1657,7 +1657,12 @@ mod tasks_contract { ); assert_eq!(handled.load(Ordering::SeqCst), 0, "L2 {place:?}"); release.send(()).unwrap(); - timeout(LIMIT, actor.ready()).await.unwrap().unwrap(); + timeout(LIMIT, actor.ready()) + .await + .unwrap() + .unwrap() + .started() + .expect("started"); for (i, reply) in replies.into_iter().enumerate() { assert_eq!(timeout(LIMIT, reply).await.unwrap().unwrap(), i); } @@ -1781,7 +1786,7 @@ mod tasks_contract { /// timer (and enqueues) only when first polled. #[tokio::test(start_paused = true)] async fn default_request_timeout_is_five_seconds_from_first_poll() { - let actor = Napper.start(); + let actor = Napper.spawn(); assert!(actor .request(Nap(Duration::from_millis(4_900))) .await @@ -1846,7 +1851,7 @@ mod tasks_contract { } let system = ActorSystem::new().unwrap(); let group = system.async_group("unowned", 1).unwrap(); - let actor = Spawner.start_on(&group).unwrap(); + let actor = Spawner.with_options(&group).spawn().unwrap(); let task_drops = Arc::new(AtomicUsize::new(0)); let thread_done = Arc::new(AtomicUsize::new(0)); let (release, thread_release) = std::sync::mpsc::channel(); @@ -2302,7 +2307,8 @@ mod supervision_contract { fired: self.fired.clone(), downs: self.downs.clone(), } - .start_on(&ctx.execution_group().unwrap()) + .with_options(&ctx.execution_group().unwrap()) + .spawn() .is_ok(); (registered, started) } @@ -2344,17 +2350,19 @@ mod supervision_contract { fired: fired.clone(), downs: downs.clone(), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - late.ready().await.unwrap(); + late.ready().await.unwrap().started().expect("started"); let result = Arc::new(Mutex::new(None)); let prober = Prober { late: late.clone(), result: result.clone(), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - prober.ready().await.unwrap(); + prober.ready().await.unwrap().started().expect("started"); assert!(system.shutdown(LIMIT).await.complete); let (registered, started) = result.lock().unwrap().expect("probe ran"); assert!(!registered, "S1: registration admitted during shutdown"); @@ -2405,18 +2413,20 @@ mod supervision_contract { stopping: None, release: None, } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - earlier.ready().await.unwrap(); + earlier.ready().await.unwrap().started().expect("started"); let (stopping, stopping_rx) = oneshot::channel(); let (release, release_rx) = oneshot::channel(); let later = Gated { stopping: Some(stopping), release: Some(release_rx), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - later.ready().await.unwrap(); + later.ready().await.unwrap().started().expect("started"); drop(group); let began = Instant::now(); drop(system); @@ -2623,8 +2633,8 @@ mod native_partial { for reason in linked_reasons() { for _ in 0..ROUNDS { let seen = Arc::new(Mutex::new(Vec::new())); - let bystander = Bystander(seen.clone()).start(); - let peer = Peer.start(); + let bystander = Bystander(seen.clone()).spawn(); + let peer = Peer.spawn(); bystander .request_with_timeout(LinkTo(peer.child_handle()), LIMIT) .unwrap(); @@ -2723,7 +2733,7 @@ mod native_partial { in_stopped: (cleanup == Cleanup::KillThenStoppedPanics) .then_some((began, resume_rx)), } - .start(); + .spawn(); let handle = actor.child_handle(); // Started first, as in the tasks version. actor @@ -2811,11 +2821,11 @@ mod tasks_partial { } fn start(&self, actor: A, place: Place) -> ActorRef { match place { - Place::Async => actor.start_with_backend(Backend::Async), - Place::Blocking => actor.start_with_backend(Backend::Blocking), - Place::Thread => actor.start_with_backend(Backend::Thread), - Place::AsyncGroup => actor.start_on(&self.async_group).unwrap(), - Place::ThreadGroup => actor.start_on(&self.thread_group).unwrap(), + Place::Async => actor.with_options(Backend::Async).spawn().unwrap(), + Place::Blocking => actor.with_options(Backend::Blocking).spawn().unwrap(), + Place::Thread => actor.with_options(Backend::Thread).spawn().unwrap(), + Place::AsyncGroup => actor.with_options(&self.async_group).spawn().unwrap(), + Place::ThreadGroup => actor.with_options(&self.thread_group).spawn().unwrap(), } } async fn finish(self) { diff --git a/concurrency/tests/shutdown_payload.rs b/concurrency/tests/shutdown_payload.rs index f80d442d6..77741b251 100644 --- a/concurrency/tests/shutdown_payload.rs +++ b/concurrency/tests/shutdown_payload.rs @@ -137,10 +137,10 @@ mod tasks_mode { Seen::default(), Seen::default(), ); - let target = Node(target_seen.clone(), false).start(); - let plain = Node(plain_seen.clone(), false).start(); - let trapper = Node(trap_seen.clone(), true).start(); - let watcher = Node(watch_seen.clone(), false).start(); + let target = Node(target_seen.clone(), false).spawn(); + let plain = Node(plain_seen.clone(), false).spawn(); + let trapper = Node(trap_seen.clone(), true).spawn(); + let watcher = Node(watch_seen.clone(), false).spawn(); settle(&trapper).await; let t = target.child_handle(); plain.context().link(&t); @@ -175,8 +175,8 @@ mod tasks_mode { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn exit_signal_with_payload_ends_or_is_trapped() { let (plain_seen, trap_seen) = (Seen::default(), Seen::default()); - let plain = Node(plain_seen.clone(), false).start(); - let trapper = Node(trap_seen.clone(), true).start(); + let plain = Node(plain_seen.clone(), false).spawn(); + let trapper = Node(trap_seen.clone(), true).spawn(); settle(&plain).await; settle(&trapper).await; plain.child_handle().exit(why()); @@ -269,7 +269,7 @@ mod tasks_mode { async fn metrics_count_payload_exit_as_shutdown() { use spawned_concurrency::metrics; let before = metrics::snapshot(); - let actor = Node(Seen::default(), false).start(); + let actor = Node(Seen::default(), false).spawn(); actor.request(StopWith(why())).await.unwrap(); assert_eq!(ended(&actor.child_handle()).await, why()); let after = metrics::snapshot(); @@ -359,10 +359,10 @@ mod threads_mode { Seen::default(), Seen::default(), ); - let target = Node(target_seen.clone(), false).start(); - let plain = Node(plain_seen.clone(), false).start(); - let trapper = Node(trap_seen.clone(), true).start(); - let watcher = Node(watch_seen.clone(), false).start(); + let target = Node(target_seen.clone(), false).spawn(); + let plain = Node(plain_seen.clone(), false).spawn(); + let trapper = Node(trap_seen.clone(), true).spawn(); + let watcher = Node(watch_seen.clone(), false).spawn(); settle(&trapper); let t = target.child_handle(); plain.context().link(&t); @@ -397,8 +397,8 @@ mod threads_mode { #[test] fn exit_signal_with_payload_ends_or_is_trapped() { let (plain_seen, trap_seen) = (Seen::default(), Seen::default()); - let plain = Node(plain_seen.clone(), false).start(); - let trapper = Node(trap_seen.clone(), true).start(); + let plain = Node(plain_seen.clone(), false).spawn(); + let trapper = Node(trap_seen.clone(), true).spawn(); settle(&plain); settle(&trapper); plain.child_handle().exit(why()); diff --git a/concurrency/tests/shutdown_publication.rs b/concurrency/tests/shutdown_publication.rs index 1aaf4c402..7a6da3dcb 100644 --- a/concurrency/tests/shutdown_publication.rs +++ b/concurrency/tests/shutdown_publication.rs @@ -101,7 +101,7 @@ fn admitted_request_racing_shutdown_drops_its_reply_before_join_completes() { .unwrap(); runtime.block_on(async { let (stopped, stop_rx) = tokio::sync::oneshot::channel(); - let actor = Worker(Some(stopped)).start(); + let actor = Worker(Some(stopped)).spawn(); let _cleanup = Cleanup(actor.child_handle()); let (entered, entered_rx) = tokio::sync::oneshot::channel(); let (release, gate) = tokio::sync::oneshot::channel(); diff --git a/concurrency/tests/sim.rs b/concurrency/tests/sim.rs index be60fab41..4f62a9099 100644 --- a/concurrency/tests/sim.rs +++ b/concurrency/tests/sim.rs @@ -140,7 +140,15 @@ async fn shutdown_order(log: Log) -> Vec { log: log.clone(), }; for id in 0..2 { - start(id).start_on(&group).unwrap().ready().await.unwrap(); + start(id) + .with_options(&group) + .spawn() + .unwrap() + .ready() + .await + .unwrap() + .started() + .expect("started"); } let mut tree = system .supervisor(config(SupervisorStrategy::OneForOne)) @@ -160,7 +168,15 @@ async fn shutdown_order(log: Log) -> Vec { .unwrap(); let _tree = tree.start().await.unwrap(); for id in 200..202 { - start(id).start_on(&group).unwrap().ready().await.unwrap(); + start(id) + .with_options(&group) + .spawn() + .unwrap() + .ready() + .await + .unwrap() + .started() + .expect("started"); } let report = system.shutdown(LIMIT).await; assert!(report.complete, "shutdown incomplete"); @@ -374,7 +390,9 @@ fn blocking_backend_actor_fails_the_run() { trap: false, log, } - .start_with_backend(spawned_concurrency::tasks::Backend::Blocking); + .with_options(spawned_concurrency::tasks::Backend::Blocking) + .spawn() + .unwrap(); spawned_rt::tasks::sleep(Duration::from_millis(10)).await; drop(node); }); @@ -1040,7 +1058,7 @@ async fn drain_pending(log: Log) -> Vec { let parker = Parker { pending: spawned_concurrency::tasks::PendingReplies::with_capacity(CALLERS as usize), } - .start(); + .spawn(); let callers: Vec<_> = (0..CALLERS) .map(|id| { let (parker, log) = (parker.clone(), log.clone()); diff --git a/concurrency/tests/sim_explore.rs b/concurrency/tests/sim_explore.rs index b1f9080d4..d5eafa6e9 100644 --- a/concurrency/tests/sim_explore.rs +++ b/concurrency/tests/sim_explore.rs @@ -128,7 +128,7 @@ async fn bump_all(noises: &[u32]) -> Vec { value: 0, log: Vec::new(), } - .start(); + .spawn(); let workers: Vec<_> = noises .iter() .map(|&noise| { @@ -136,7 +136,7 @@ async fn bump_all(noises: &[u32]) -> Vec { store: store.clone(), noise, } - .start() + .spawn() }) .collect(); let replies: Vec<_> = workers diff --git a/concurrency/tests/sim_ordering.rs b/concurrency/tests/sim_ordering.rs index 529cc135e..e4372b7cf 100644 --- a/concurrency/tests/sim_ordering.rs +++ b/concurrency/tests/sim_ordering.rs @@ -107,7 +107,7 @@ impl Handler for Sink { /// and exactly once, and every reply must carry its own request's key. async fn per_sender_fifo_body() { let seen = log(); - let sink = Sink(seen.clone()).start(); + let sink = Sink(seen.clone()).spawn(); let by_ref = sink.clone(); let by_recipient: Recipient = sink.recipient(); let mixed = (sink.clone(), sink.recipient::()); @@ -233,7 +233,7 @@ impl Handler for Turns { /// another task stops the actor, so handlers race both hooks. async fn turns_body() { let events = log(); - let actor = Turns(events.clone()).start(); + let actor = Turns(events.clone()).spawn(); let mut tasks = Vec::new(); for sender in 0..2 { let actor = actor.clone(); @@ -429,7 +429,7 @@ const PER_QUEUER: usize = 4; async fn stop_discards_body() { let seen = log(); let drops = Arc::new(AtomicUsize::new(0)); - let actor = Holder(seen.clone()).start(); + let actor = Holder(seen.clone()).spawn(); let (entered_tx, entered_rx) = oneshot::channel(); let (release_tx, release_rx) = oneshot::channel(); actor @@ -663,8 +663,8 @@ impl Handler for Source { async fn exit_after_last_message_body(end: End) { let notices = log(); let (notify, mut notified) = mpsc::channel(); - let observer: ActorRef = Observer(notices.clone(), notify).start(); - let source = Source.start(); + let observer: ActorRef = Observer(notices.clone(), notify).spawn(); + let source = Source.spawn(); observer .request_with_timeout(Watch(source.child_handle()), LIMIT) .await diff --git a/concurrency/tests/sim_turn_budget.rs b/concurrency/tests/sim_turn_budget.rs index 54a5a5ee5..c390f4a36 100644 --- a/concurrency/tests/sim_turn_budget.rs +++ b/concurrency/tests/sim_turn_budget.rs @@ -106,7 +106,7 @@ enum Work { fn run(budget: Option, work: Work) -> Option { let body = move |_seed| async move { sim::set_turn_budget(budget); - let worker = Worker.start(); + let worker = Worker.spawn(); match work { Work::Nap => worker.request(Nap).await.unwrap(), Work::Spin => { @@ -164,7 +164,7 @@ fn blocking_child() { .named(env!("CARGO_PKG_NAME"), module_path!(), "blocking_child") .seeds(0..1) .run(|_seed| async { - Worker.start().request(Nap).await.unwrap(); + Worker.spawn().request(Nap).await.unwrap(); }); } diff --git a/concurrency/tests/stack_budget.rs b/concurrency/tests/stack_budget.rs index 6dd65ac55..1232dc179 100644 --- a/concurrency/tests/stack_budget.rs +++ b/concurrency/tests/stack_budget.rs @@ -177,7 +177,7 @@ mod tasks_chain { system.async_group("stack", 2) } .unwrap(); - let hops = chain(|hop| group.start(hop).unwrap()); + let hops = chain(|hop| hop.with_options(&group).spawn().unwrap()); rt.block_on(async { tokio::time::timeout(LIMIT, drive(hops[0].clone())) .await @@ -197,7 +197,7 @@ mod tasks_chain { let rt = small_runtime(); let hops = { let _entered = rt.enter(); - chain(|hop| hop.start()) + chain(|hop| hop.spawn()) }; rt.block_on(async { tokio::time::timeout(LIMIT, drive(hops[0].clone())) @@ -336,7 +336,7 @@ mod threads_chain { let mut hops: Vec> = Vec::new(); for index in (0..depth()).rev() { let next = hops.last().cloned(); - hops.push(Hop { index, next }.start()); + hops.push(Hop { index, next }.spawn()); } hops.reverse(); for round in 0..ROUNDS { diff --git a/concurrency/tests/start_contract.rs b/concurrency/tests/start_contract.rs new file mode 100644 index 000000000..8e5c153e7 --- /dev/null +++ b/concurrency/tests/start_contract.rs @@ -0,0 +1,996 @@ +//! The unified start contract (docs/design/unified-start.md; semantics.md +//! L1, L2, L8-L10, X5, X8, S6, S7, S9, T5) in both modes. Each test fails +//! when the guarantee it names breaks: a refused start that ran a hook or +//! kept a name, a failed start that ran `stopped()` or answered before its +//! state was dropped, a consumed notice that still arrived, a lost one, a +//! continuation behind a message, a child outside its parent's group. + +#![cfg_attr(not(feature = "async"), allow(unused_imports))] +#![cfg_attr(spawned_sim, allow(dead_code, unused_imports))] + +use spawned_concurrency::{ + message::Message, ChildHandle, Down, Exit, ExitReason, PendingReply, StartError, StartOutcome, +}; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +const LIMIT: Duration = Duration::from_secs(10); + +#[derive(Default)] +struct Hooks { + started: AtomicUsize, + stopped: AtomicUsize, + dropped: AtomicUsize, +} + +struct Work; +impl Message for Work { + type Result = (); +} +struct Cont; +impl Message for Cont { + type Result = (); +} +struct Boom; +impl Message for Boom { + type Result = (); +} + +#[derive(Clone, Copy, Debug)] +enum Then { + Ready, + Continue, + Error, + Ignore, + Panic, + Stop, + CtxStop, + PanicLater, +} + +#[cfg(feature = "async")] +mod tasks_mode { + use super::*; + use spawned_concurrency::system::ActorSystem; + use spawned_concurrency::tasks::{ + Actor, ActorRef, ActorStart, Backend, Context, Handler, Init, Name, StartOptions, + }; + + struct Child { + then: Then, + hooks: Arc, + log: Arc>>, + gate: Option>, + group: Arc>>>, + } + impl Child { + fn new(then: Then) -> Self { + Self { + then, + hooks: Arc::default(), + log: Arc::default(), + gate: None, + group: Arc::default(), + } + } + } + impl Drop for Child { + fn drop(&mut self) { + self.hooks.dropped.fetch_add(1, Ordering::SeqCst); + } + } + impl Actor for Child { + async fn started(&mut self, ctx: &Context) -> Init { + self.hooks.started.fetch_add(1, Ordering::SeqCst); + *self.group.lock().unwrap() = Some(ctx.execution_group().and_then(|g| g.name())); + if let Some(gate) = self.gate.take() { + let _ = gate.await; + } + match self.then { + Then::Ready => Init::ready(), + Then::Continue => Init::continue_with(Cont), + Then::Error => Init::error("bad config"), + Then::Ignore => Init::ignore(), + Then::Panic => panic!("init panic"), + Then::Stop => Init::stop(ExitReason::Shutdown(None)), + Then::CtxStop => { + ctx.stop(); + Init::ready() + } + Then::PanicLater => Init::continue_with(Boom), + } + } + async fn stopped(&mut self, _: &Context) { + self.hooks.stopped.fetch_add(1, Ordering::SeqCst); + } + } + impl Handler for Child { + async fn handle(&mut self, _: Work, reply: PendingReply<'_, ()>, _: &Context) { + self.log.lock().unwrap().push("work"); + reply.reply(()); + } + } + impl Handler for Child { + async fn handle(&mut self, _: Cont, _: PendingReply<'_, ()>, _: &Context) { + self.log.lock().unwrap().push("continue"); + } + } + impl Handler for Child { + async fn handle(&mut self, _: Boom, _: PendingReply<'_, ()>, _: &Context) { + panic!("failure right after readiness"); + } + } + + #[derive(Default, Clone)] + struct Seen { + exits: Arc>>, + downs: Arc>>, + } + struct Parent { + trap: bool, + seen: Seen, + } + impl Actor for Parent { + async fn started(&mut self, ctx: &Context) { + ctx.trap_exit(self.trap); + } + async fn exit_received(&mut self, exit: Exit, _: &Context) { + self.seen.exits.lock().unwrap().push(exit); + } + } + impl Handler for Parent { + async fn handle(&mut self, down: Down, reply: PendingReply<'_, ()>, _: &Context) { + self.seen.downs.lock().unwrap().push(down); + reply.reply(()); + } + } + impl Handler for Parent { + async fn handle(&mut self, _: Work, reply: PendingReply<'_, ()>, _: &Context) { + reply.reply(()); + } + } + async fn parent(trap: bool) -> (ActorRef, Seen) { + let seen = Seen::default(); + let p = Parent { + trap, + seen: seen.clone(), + } + .start() + .await + .unwrap() + .started() + .unwrap(); + (p, seen) + } + + /// L9: every row of the 2.3 table, waiting and through spawn + ready(); + /// no failed start runs stopped(), and its state is dropped before the + /// error is answered (L8, failure after publication). + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn init_outcomes_answer_after_publication_without_stopped() { + for then in [ + Then::Ready, + Then::Continue, + Then::Error, + Then::Ignore, + Then::Panic, + Then::Stop, + Then::CtxStop, + ] { + for waiting in [true, false] { + let child = Child::new(then); + let hooks = child.hooks.clone(); + let result = if waiting { + child.start().await.map(|o| o.map(|_| ())) + } else { + child.spawn().ready().await + }; + let failed = !matches!(result, Ok(StartOutcome::Started(()))); + if failed { + // Answered after publication: the state is already gone. + assert_eq!(hooks.dropped.load(Ordering::SeqCst), 1, "{then:?}"); + } + assert_eq!(hooks.stopped.load(Ordering::SeqCst), 0, "{then:?}"); + match (then, result) { + (Then::Ready | Then::Continue, Ok(StartOutcome::Started(()))) => {} + ( + Then::Error, + Err(StartError::InitFailed { + cause, + pending: None, + }), + ) => { + assert_eq!(cause, "bad config") + } + (Then::Ignore, Ok(StartOutcome::Ignored { pending: None })) => {} + ( + Then::Panic, + Err(StartError::InitExited { + reason: ExitReason::Panic(_), + pending: None, + }), + ) => {} + ( + Then::Stop, + Err(StartError::InitExited { + reason: ExitReason::Shutdown(None), + pending: None, + }), + ) => {} + ( + Then::CtxStop, + Err(StartError::InitExited { + reason: ExitReason::Normal, + pending: None, + }), + ) => {} + (then, result) => panic!("{then:?} (waiting {waiting}): {result:?}"), + } + } + } + } + + /// L2/L8: readiness is final. A child that fails right after readiness, + /// with its waiter not yet polled, still answers Started, and the Down + /// of its later death reaches the monitoring parent. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn ready_then_immediate_crash_with_unpolled_waiter_returns_started_and_delivers_down() { + let (p, seen) = parent(false).await; + let starting = Child::new(Then::PanicLater).start_monitor(&p.context()); + let child = starting.actor_ref().unwrap(); + assert!(matches!(child.wait_exit().await, ExitReason::Panic(_))); + let (_, monitor) = starting.await.unwrap().started().unwrap(); + tokio::time::timeout(LIMIT, async { + while seen.downs.lock().unwrap().is_empty() { + tokio::time::sleep(Duration::from_millis(1)).await; + } + }) + .await + .unwrap(); + let down = seen.downs.lock().unwrap()[0].clone(); + assert_eq!(down.monitor_ref, monitor); + assert!(matches!(down.reason, ExitReason::Panic(_))); + } + + /// X8: a failed start's notice is consumed for a trapping parent (Exit) + /// and a monitor (Down); a non-trapping parent still exits on a + /// non-Normal failure, and a Normal one spares it. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn failed_start_notices_follow_the_disposition_table() { + let (p, seen) = parent(true).await; + assert!(Child::new(Then::Panic) + .start_link(&p.context()) + .await + .is_err()); + assert!(Child::new(Then::Error) + .start_monitor(&p.context()) + .await + .is_err()); + p.request(Work).await.unwrap(); + tokio::time::sleep(Duration::from_millis(20)).await; + assert!(seen.exits.lock().unwrap().is_empty(), "Exit not consumed"); + assert!(seen.downs.lock().unwrap().is_empty(), "Down not consumed"); + + let (spared, _) = parent(false).await; + assert!(Child::new(Then::Error) + .start_link(&spared.context()) + .await + .is_err()); + spared.request(Work).await.unwrap(); + assert!(spared.child_handle().is_alive()); + + let (ended, _) = parent(false).await; + assert!(Child::new(Then::Panic) + .start_link(&ended.context()) + .await + .is_err()); + assert!(matches!(ended.wait_exit().await, ExitReason::Panic(_))); + } + + /// X8 abandonment: a dropped Starting hands its diverted notice on. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn abandoned_start_monitor_still_delivers_down() { + let (p, seen) = parent(false).await; + let starting = Child::new(Then::Error).start_monitor(&p.context()); + let child = starting.actor_ref().unwrap(); + drop(starting); + child.wait_exit().await; + tokio::time::timeout(LIMIT, async { + while seen.downs.lock().unwrap().is_empty() { + tokio::time::sleep(Duration::from_millis(1)).await; + } + }) + .await + .unwrap(); + assert_eq!(seen.downs.lock().unwrap().len(), 1); + } + + /// T5: the init timeout answers InitTimeout and never reaches a waiting + /// parent (trapping or not). + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn init_timeout_never_reaches_the_parent() { + for trap in [true, false] { + let (p, seen) = parent(trap).await; + let (release, gate) = tokio::sync::oneshot::channel(); + let mut child = Child::new(Then::Ready); + child.gate = Some(gate); + let hooks = child.hooks.clone(); + let starting = child + .with_options(StartOptions::new().init_timeout(Duration::from_millis(20))) + .start_link(&p.context()); + tokio::time::sleep(Duration::from_millis(60)).await; + release.send(()).unwrap(); + assert!(matches!( + starting.await, + Err(StartError::InitTimeout { pending: None }) + )); + assert_eq!(hooks.stopped.load(Ordering::SeqCst), 0); + p.request(Work).await.unwrap(); + assert!(seen.exits.lock().unwrap().is_empty()); + assert!( + p.child_handle().is_alive(), + "the timeout's Killed reached P" + ); + } + } + + /// L10: the continuation runs before messages sent during started(). + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn continuation_runs_before_messages_sent_during_started() { + let (release, gate) = tokio::sync::oneshot::channel(); + let mut child = Child::new(Then::Continue); + child.gate = Some(gate); + let log = child.log.clone(); + let child = child.spawn(); + child.send(Work).unwrap(); + release.send(()).unwrap(); + child.request(Work).await.unwrap(); + assert_eq!(*log.lock().unwrap(), ["continue", "work", "work"]); + } + + /// X5/L8: a start through a parent whose exit began is refused; nothing + /// ran and nothing remains. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn start_link_through_dead_context_is_refused() { + let (p, _) = parent(false).await; + let ctx = p.context(); + p.child_handle().stop(); + p.wait_exit().await; + let child = Child::new(Then::Ready); + let hooks = child.hooks.clone(); + assert!(matches!( + child.start_link(&ctx).await, + Err(StartError::ParentExited) + )); + assert_eq!(hooks.started.load(Ordering::SeqCst), 0); + assert_eq!(hooks.dropped.load(Ordering::SeqCst), 1); + } + + /// S7: link and monitor children inherit the parent's group; plain + /// starts never inherit; placement_of opts in. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn link_and_monitor_inherit_the_parents_group() { + let system = ActorSystem::new().unwrap(); + for group in [ + system.async_group("inherit-async", 2).unwrap(), + system.thread_group("inherit-threads", 4).unwrap(), + ] { + let p = Parent { + trap: false, + seen: Seen::default(), + } + .with_options(&group) + .start() + .await + .unwrap() + .started() + .unwrap(); + let name = group.name(); + for verb in 0..4 { + let child = Child::new(Then::Ready); + let seen_group = child.group.clone(); + let started = match verb { + 0 => child.start_link(&p.context()).await.map(|o| o.map(|_| ())), + 1 => child + .start_monitor(&p.context()) + .await + .map(|o| o.map(|_| ())), + 2 => child.start().await.map(|o| o.map(|_| ())), + _ => child + .with_options(StartOptions::new().placement_of(&p.context())) + .start() + .await + .map(|o| o.map(|_| ())), + }; + assert!(matches!(started, Ok(StartOutcome::Started(())))); + let expected = if verb == 2 { None } else { name.clone() }; + assert_eq!(*seen_group.lock().unwrap(), Some(expected), "verb {verb}"); + } + p.child_handle().stop(); + p.wait_exit().await; + } + assert!(system.shutdown(LIMIT).await.complete); + } + + /// S7: an inherited full thread group refuses with Capacity (no + /// fallback), and nothing of the refused start is observable. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn inherited_start_into_full_thread_group_fails_capacity() { + let system = ActorSystem::new().unwrap(); + let group = system.thread_group("inherit-full", 1).unwrap(); + let p = Parent { + trap: false, + seen: Seen::default(), + } + .with_options(&group) + .start() + .await + .unwrap() + .started() + .unwrap(); + let child = Child::new(Then::Ready); + let hooks = child.hooks.clone(); + assert!(matches!( + child.start_link(&p.context()).await, + Err(StartError::Capacity) + )); + assert_eq!(hooks.started.load(Ordering::SeqCst), 0); + p.child_handle().stop(); + p.wait_exit().await; + assert!(system.shutdown(LIMIT).await.complete); + } + + /// S9: typed names: a clash reports the holder; a failed named start can + /// be retried at once (the name is free when the error is answered); a + /// non-actor holder reports no handle. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn names_are_committed_and_released_atomically() { + static NAME: Name = Name::new("start-contract-tasks"); + let holder = Child::new(Then::Ready) + .with_options(StartOptions::new().name(NAME.clone())) + .start() + .await + .unwrap() + .started() + .unwrap(); + assert_eq!(NAME.whereis().map(|a| a.id()), Some(holder.id())); + let loser = Child::new(Then::Ready); + let hooks = loser.hooks.clone(); + match loser + .with_options(StartOptions::new().name(NAME.clone())) + .start() + .await + { + Err(StartError::AlreadyStarted { existing, name }) => { + assert_eq!(name, NAME.key()); + assert_eq!(existing.map(|h: ChildHandle| h.id()), Some(holder.id())); + } + other => panic!("{other:?}"), + } + assert_eq!(hooks.started.load(Ordering::SeqCst), 0); + holder.child_handle().stop(); + holder.wait_exit().await; + // A failed start answers after the name's release. + for _ in 0..3 { + assert!(Child::new(Then::Error) + .with_options(StartOptions::new().name(NAME.clone())) + .start() + .await + .is_err()); + } + assert!(NAME.whereis().is_none()); + spawned_concurrency::registry::register("start-contract-plain", 7u8).unwrap(); + match Child::new(Then::Ready) + .with_options(StartOptions::new().name("start-contract-plain")) + .spawn() + { + Err(StartError::AlreadyStarted { existing: None, .. }) => {} + other => panic!("{other:?}"), + } + spawned_concurrency::registry::unregister("start-contract-plain"); + } + + /// Review #291 item 6: a name held through `ctx.register_name` reports + /// its holder in `AlreadyStarted { existing }`. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn already_started_reports_a_ctx_registered_holder() { + struct Registrar; + impl Actor for Registrar { + async fn started(&mut self, ctx: &Context) { + ctx.register_name("start-contract-ctx").unwrap(); + } + } + let system = spawned_concurrency::system::ActorSystem::new().unwrap(); + let group = system.async_group("start-contract-ctx", 2).unwrap(); + let holder = Registrar + .with_options(&group) + .start() + .await + .unwrap() + .started() + .unwrap(); + match Child::new(Then::Ready) + .with_options(StartOptions::new().name("start-contract-ctx")) + .start() + .await + { + Err(StartError::AlreadyStarted { existing, .. }) => { + assert_eq!(existing.map(|h: ChildHandle| h.id()), Some(holder.id())); + } + other => panic!("{other:?}"), + } + holder.child_handle().stop(); + holder.wait_exit().await; + assert!(system.shutdown(Duration::from_secs(5)).await.complete); + } + + /// Options are validated before anything exists. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn invalid_options_are_refused() { + for options in [ + StartOptions::new().init_timeout(Duration::ZERO), + StartOptions::new().max_outstanding(0), + StartOptions::new().max_native_jobs(0), + StartOptions::new().backend(Backend::Thread).stack_size(0), + StartOptions::new().stack_size(1 << 20), + ] { + let child = Child::new(Then::Ready); + let hooks = child.hooks.clone(); + assert!(matches!( + child.with_options(options).spawn(), + Err(StartError::InvalidOptions(_)) + )); + assert_eq!(hooks.started.load(Ordering::SeqCst), 0); + } + let ok = Child::new(Then::Ready) + .with_options( + StartOptions::new() + .backend(Backend::Thread) + .stack_size(1 << 20), + ) + .start() + .await; + assert!(matches!(ok, Ok(StartOutcome::Started(_)))); + } + + /// L9: a trapping child whose parent exits while it initialises commits + /// ready, runs its continuation, then ends from the queued parent exit. + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn trapping_child_with_parent_exit_in_started_commits_ready_then_ends() { + struct Trapping( + Arc>>, + Option>, + ); + impl Actor for Trapping { + async fn started(&mut self, ctx: &Context) -> Init { + ctx.trap_exit(true); + let _ = self.1.take().unwrap().await; + Init::continue_with(Cont) + } + async fn stopped(&mut self, _: &Context) { + self.0.lock().unwrap().push("stopped"); + } + } + impl Handler for Trapping { + async fn handle(&mut self, _: Cont, _: PendingReply<'_, ()>, _: &Context) { + self.0.lock().unwrap().push("continue"); + } + } + let (p, _) = parent(false).await; + let (release, gate) = tokio::sync::oneshot::channel(); + let log: Arc>> = Arc::default(); + let starting = Trapping(log.clone(), Some(gate)).start_link(&p.context()); + let child = starting.actor_ref().unwrap(); + p.child_handle().stop(); + p.wait_exit().await; + release.send(()).unwrap(); + assert!(matches!(starting.await, Ok(StartOutcome::Started(_)))); + assert_eq!(child.wait_exit().await, ExitReason::Normal); + assert_eq!(*log.lock().unwrap(), ["continue", "stopped"]); + } + + /// A runtime dropped before the child ran: the start is decided at its + /// death point (`Killed`) and answered, never left pending forever. + #[cfg(not(spawned_sim))] // real Tokio runtimes + #[test] + fn runtime_drop_before_started_answers_init_exited_killed() { + let rt = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let child = Child::new(Then::Ready); + let hooks = child.hooks.clone(); + let starting = { + let _entered = rt.enter(); + child.start() + }; + drop(rt); + let other = tokio::runtime::Runtime::new().unwrap(); + let result = other.block_on(async { tokio::time::timeout(LIMIT, starting).await }); + assert!( + matches!( + result, + Ok(Err(StartError::InitExited { + reason: ExitReason::Killed, + pending: None + })) + ), + "{result:?}" + ); + assert_eq!(hooks.started.load(Ordering::SeqCst), 0); + assert_eq!(hooks.dropped.load(Ordering::SeqCst), 1); + } + + /// The init timer's pending answer: a started() that never returns + /// keeps the instance, answered with its handle at the cleanup + /// deadline (stuck policy). + #[cfg(not(spawned_sim))] // real Tokio + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn stuck_started_reports_pending_after_the_timeout() { + stuck_started_reports_pending().await; + } + + /// Review #291 item 3: the tasks-mode init timer runs on the runtime's + /// clock. Under the simulator an hour-long `init_timeout` expires in + /// virtual time, before a two-hour `started()` returns (a real-time + /// timer thread would let the start succeed). + #[cfg(spawned_sim)] + #[spawned_macros::sim_test] + async fn init_timer_uses_the_runtime_clock() { + struct Slow; + impl Actor for Slow { + async fn started(&mut self, _: &Context) { + spawned_rt::tasks::sleep(Duration::from_secs(7200)).await; + } + } + let began = spawned_rt::time::Instant::now(); + let result = Slow + .with_options(StartOptions::new().init_timeout(Duration::from_secs(3600))) + .start() + .await; + assert!( + matches!(result, Err(StartError::InitTimeout { .. })), + "{result:?}" + ); + // At the timeout plus the cleanup deadline, not at started()'s end. + assert!( + began.elapsed() < Duration::from_secs(3700), + "{:?}", + began.elapsed() + ); + } + + async fn stuck_started_reports_pending() { + static STUCK: AtomicBool = AtomicBool::new(true); + struct Stuck; + impl Actor for Stuck { + async fn started(&mut self, _: &Context) { + while STUCK.load(Ordering::SeqCst) { + tokio::time::sleep(Duration::from_millis(5)).await; + } + } + } + let result = Stuck + .with_options(StartOptions::new().init_timeout(Duration::from_millis(10))) + .start() + .await; + let pending = match result { + Err(StartError::InitTimeout { + pending: Some(pending), + }) => pending, + other => panic!("{other:?}"), + }; + assert!(pending.is_alive()); + STUCK.store(false, Ordering::SeqCst); + assert_eq!(pending.wait_exit_async().await, ExitReason::Killed); + } +} + +mod threads_mode { + use super::*; + use spawned_concurrency::threads::{ + Actor, ActorRef, ActorStart, Context, Handler, Init, Name, StartOptions, + }; + + struct Child { + then: Then, + hooks: Arc, + log: Arc>>, + } + impl Child { + fn new(then: Then) -> Self { + Self { + then, + hooks: Arc::default(), + log: Arc::default(), + } + } + } + impl Drop for Child { + fn drop(&mut self) { + self.hooks.dropped.fetch_add(1, Ordering::SeqCst); + } + } + impl Actor for Child { + fn started(&mut self, ctx: &Context) -> Init { + self.hooks.started.fetch_add(1, Ordering::SeqCst); + match self.then { + Then::Ready => Init::ready(), + Then::Continue => Init::continue_with(Cont), + Then::Error => Init::error("bad config"), + Then::Ignore => Init::ignore(), + Then::Panic => panic!("init panic"), + Then::Stop => Init::stop(ExitReason::Shutdown(None)), + Then::CtxStop => { + ctx.stop(); + Init::ready() + } + Then::PanicLater => Init::continue_with(Boom), + } + } + fn stopped(&mut self, _: &Context) { + self.hooks.stopped.fetch_add(1, Ordering::SeqCst); + } + } + impl Handler for Child { + fn handle(&mut self, _: Work, reply: PendingReply<'_, ()>, _: &Context) { + self.log.lock().unwrap().push("work"); + reply.reply(()); + } + } + impl Handler for Child { + fn handle(&mut self, _: Cont, _: PendingReply<'_, ()>, _: &Context) { + self.log.lock().unwrap().push("continue"); + } + } + impl Handler for Child { + fn handle(&mut self, _: Boom, _: PendingReply<'_, ()>, _: &Context) { + panic!("failure right after readiness"); + } + } + + struct Parent { + exits: Arc>>, + downs: Arc>>, + } + impl Actor for Parent { + fn exit_received(&mut self, exit: Exit, _: &Context) { + self.exits.lock().unwrap().push(exit); + } + } + impl Handler for Parent { + fn handle(&mut self, down: Down, reply: PendingReply<'_, ()>, _: &Context) { + self.downs.lock().unwrap().push(down); + reply.reply(()); + } + } + impl Handler for Parent { + fn handle(&mut self, _: Work, reply: PendingReply<'_, ()>, _: &Context) { + reply.reply(()); + } + } + type Log = Arc>>; + fn parent(trap: bool) -> (ActorRef, Log, Log) { + let (exits, downs): (Log, Log) = Default::default(); + let p = Parent { + exits: exits.clone(), + downs: downs.clone(), + } + .spawn(); + p.context().trap_exit(trap); + p.request(Work).unwrap(); + (p, exits, downs) + } + + #[test] + fn init_outcomes_answer_after_publication_without_stopped() { + for then in [ + Then::Ready, + Then::Continue, + Then::Error, + Then::Ignore, + Then::Panic, + Then::Stop, + Then::CtxStop, + ] { + for waiting in [true, false] { + let child = Child::new(then); + let hooks = child.hooks.clone(); + let result = if waiting { + child.start().map(|o| o.map(|_| ())) + } else { + child.spawn().ready() + }; + if !matches!(result, Ok(StartOutcome::Started(()))) { + assert_eq!(hooks.dropped.load(Ordering::SeqCst), 1, "{then:?}"); + } + assert_eq!(hooks.stopped.load(Ordering::SeqCst), 0, "{then:?}"); + let ok = matches!( + (then, &result), + (Then::Ready | Then::Continue, Ok(StartOutcome::Started(()))) + | ( + Then::Error, + Err(StartError::InitFailed { pending: None, .. }) + ) + | (Then::Ignore, Ok(StartOutcome::Ignored { pending: None })) + | ( + Then::Panic, + Err(StartError::InitExited { + reason: ExitReason::Panic(_), + pending: None, + }), + ) + | ( + Then::Stop, + Err(StartError::InitExited { + reason: ExitReason::Shutdown(None), + pending: None, + }), + ) + | ( + Then::CtxStop, + Err(StartError::InitExited { + reason: ExitReason::Normal, + pending: None, + }), + ) + ); + assert!(ok, "{then:?} (waiting {waiting}): {result:?}"); + } + } + } + + #[test] + fn failed_start_notices_follow_the_disposition_table() { + let (p, exits, downs) = parent(true); + assert!(Child::new(Then::Panic).start_link(&p.context()).is_err()); + assert!(Child::new(Then::Error).start_monitor(&p.context()).is_err()); + p.request(Work).unwrap(); + std::thread::sleep(Duration::from_millis(20)); + assert!(exits.lock().unwrap().is_empty(), "Exit not consumed"); + assert!(downs.lock().unwrap().is_empty(), "Down not consumed"); + let (ended, _, _) = parent(false); + assert!(Child::new(Then::Panic) + .start_link(&ended.context()) + .is_err()); + assert!(matches!(ended.wait_exit(), ExitReason::Panic(_))); + } + + #[test] + fn ready_then_immediate_crash_returns_started_and_delivers_down() { + let (p, _, downs) = parent(false); + let (child, monitor) = Child::new(Then::PanicLater) + .start_monitor(&p.context()) + .unwrap() + .started() + .unwrap(); + assert!(matches!(child.wait_exit(), ExitReason::Panic(_))); + let deadline = std::time::Instant::now() + LIMIT; + while downs.lock().unwrap().is_empty() { + assert!(std::time::Instant::now() < deadline); + std::thread::sleep(Duration::from_millis(1)); + } + assert_eq!(downs.lock().unwrap()[0].monitor_ref, monitor); + } + + #[test] + fn continuation_runs_first() { + let child = Child::new(Then::Continue); + let log = child.log.clone(); + let child = child.start().unwrap().started().unwrap(); + child.request(Work).unwrap(); + assert_eq!(*log.lock().unwrap(), ["continue", "work"]); + } + + #[test] + fn names_and_refusals() { + static NAME: Name = Name::new("start-contract-threads"); + let holder = Child::new(Then::Ready) + .with_options(StartOptions::new().name(NAME.clone())) + .start() + .unwrap() + .started() + .unwrap(); + assert_eq!(NAME.whereis().map(|a| a.id()), Some(holder.id())); + let loser = Child::new(Then::Ready); + let hooks = loser.hooks.clone(); + assert!(matches!( + loser + .with_options(StartOptions::new().name(NAME.clone())) + .start(), + Err(StartError::AlreadyStarted { + existing: Some(_), + .. + }) + )); + assert_eq!(hooks.started.load(Ordering::SeqCst), 0); + holder.context().stop(); + holder.wait_exit(); + assert!(NAME.whereis().is_none()); + assert!(matches!( + Child::new(Then::Ready) + .with_options(StartOptions::new().init_timeout(Duration::ZERO)) + .spawn(), + Err(StartError::InvalidOptions(_)) + )); + let (p, _, _) = parent(false); + let ctx = p.context(); + p.context().stop(); + p.wait_exit(); + assert!(matches!( + Child::new(Then::Ready).start_link(&ctx), + Err(StartError::ParentExited) + )); + } + + /// Review #291 item 7 (threads): a parent killed while it blocks in + /// `start_link` returns `ParentExited` at once and dies `Killed`; it does + /// not wait for the child's `started()` (OTP: the caller dies at once). + #[test] + fn parent_killed_while_awaiting_start_link_abandons_the_wait() { + type Gate = std::sync::mpsc::Receiver<()>; + struct Gated(Mutex>, std::sync::mpsc::Sender<()>); + impl Actor for Gated { + fn started(&mut self, _: &Context) { + let _ = self.1.send(()); + if let Some(gate) = self.0.lock().unwrap().take() { + let _ = gate.recv_timeout(Duration::from_secs(10)); + } + } + } + struct Waiter; + impl Actor for Waiter {} + type Done = std::sync::mpsc::Sender>; + struct LinkGated(Mutex>); + impl Message for LinkGated { + type Result = (); + } + impl Handler for Waiter { + fn handle(&mut self, msg: LinkGated, reply: PendingReply<'_, ()>, ctx: &Context) { + reply.reply(()); + let (child, done) = msg.0.lock().unwrap().take().unwrap(); + let _ = done.send(child.start_link(ctx).map(|_| ())); + } + } + let (release, gate) = std::sync::mpsc::channel(); + let (done_tx, done) = std::sync::mpsc::channel(); + let (entered_tx, entered) = std::sync::mpsc::channel(); + let p = Waiter.start().unwrap().started().unwrap(); + p.request(LinkGated(Mutex::new(Some(( + Gated(Mutex::new(Some(gate)), entered_tx), + done_tx, + ))))) + .unwrap(); + // The child is in started(): the parent waits in start_link. + entered.recv_timeout(Duration::from_secs(5)).unwrap(); + p.child_handle().kill(); + let result = done + .recv_timeout(Duration::from_secs(5)) + .expect("the wait was abandoned"); + assert!( + matches!(result, Err(StartError::ParentExited)), + "{result:?}" + ); + let until = std::time::Instant::now() + Duration::from_secs(5); + while p.child_handle().exit_reason().is_none() { + assert!(std::time::Instant::now() < until, "the parent did not die"); + std::thread::sleep(Duration::from_millis(1)); + } + assert_eq!(p.child_handle().exit_reason(), Some(ExitReason::Killed)); + release.send(()).unwrap(); + } +} diff --git a/concurrency/tests/start_trace.rs b/concurrency/tests/start_trace.rs new file mode 100644 index 000000000..185676601 --- /dev/null +++ b/concurrency/tests/start_trace.rs @@ -0,0 +1,1196 @@ +//! Records unified-start traces (stage 1, the startup decision, the cell, +//! notices, names, the init timer, continuations) from real starts for TLA+ +//! trace validation against `verification/tla/Start.tla` (mapping in +//! `TraceStart.tla`). Each scenario writes `start_.json` to +//! `$LIFECYCLE_TRACE_DIR` (default `target/lifecycle-traces`); +//! `verification/tla/trace_check.py` checks them (schema `start-v1`). +//! +//! Recording is process-wide, so the scenarios run one at a time. Every +//! scenario waits for the events it needs explicitly (the children's +//! publication and owned-work join, the parent's notices) before taking +//! the trace: nothing races the end of the recording. +#![cfg(all(feature = "lifecycle-trace", not(spawned_sim)))] + +use spawned_concurrency::{ + lifecycle_trace, + message::Message, + tasks::{Actor, ActorStart, Context, Handler, Init, Name, StartOptions}, + ChildHandle, Down, Exit, ExitReason, PendingReply, StartError, StartOutcome, +}; +use std::{ + path::PathBuf, + sync::{Arc, Mutex}, + time::Duration, +}; + +const LIMIT: Duration = Duration::from_secs(10); + +/// Held for a whole scenario (also across awaits, so an async mutex). +static SERIAL: tokio::sync::Mutex<()> = tokio::sync::Mutex::const_new(()); + +fn trace_dir() -> PathBuf { + std::env::var_os("LIFECYCLE_TRACE_DIR") + .map(PathBuf::from) + .unwrap_or_else(|| { + PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../target/lifecycle-traces") + }) +} + +/// Wait until `cond` holds (the trace has the events a scenario needs). +async fn until(what: &str, cond: impl Fn() -> bool) { + let deadline = std::time::Instant::now() + LIMIT; + while !cond() { + assert!( + std::time::Instant::now() < deadline, + "timed out waiting for {what}" + ); + tokio::time::sleep(Duration::from_millis(1)).await; + } +} + +fn count(action: &str) -> usize { + lifecycle_trace::start_count(action, "") +} + +/// Stop recording and write the trace; returns `action[:x]` per event. +fn write(name: &str, pplace: &str, ptrap: bool, cap: usize) -> Vec { + let events = lifecycle_trace::take_start(); + let json = format!( + "{{\"name\":\"start_{name}\",\"schema\":\"start-v1\",\"pplace\":\"{pplace}\",\"ptrap\":{ptrap},\"cap\":{cap},\"events\":[\n{}\n]}}\n", + events + .iter() + .map(|e| e.to_json()) + .collect::>() + .join(",\n") + ); + let dir = trace_dir(); + std::fs::create_dir_all(&dir).unwrap(); + std::fs::write(dir.join(format!("start_{name}.json")), json).unwrap(); + events + .iter() + .map(|e| { + if e.x.is_empty() { + e.action.to_string() + } else { + format!("{}:{}", e.action, e.x) + } + }) + .collect() +} + +fn has(events: &[String], wanted: &[&str]) { + // `wanted` must appear in this order (other events may interleave). + let mut at = 0; + for w in wanted { + match events[at..].iter().position(|e| e == w) { + Some(i) => at += i + 1, + None => panic!("{w} missing (in order) from {events:#?}"), + } + } +} + +// --------------------------------------------------------------------------- +// Actors +// --------------------------------------------------------------------------- + +#[derive(Default, Clone)] +struct Seen { + exits: Arc>>, + downs: Arc>>, +} + +/// The model's P. +struct Parent { + trap: bool, + seen: Seen, +} + +impl Actor for Parent { + async fn started(&mut self, ctx: &Context) { + if self.trap { + ctx.trap_exit(true); + } + } + async fn exit_received(&mut self, exit: Exit, _: &Context) { + self.seen.exits.lock().unwrap().push(exit); + } +} + +impl Handler for Parent { + async fn handle(&mut self, down: Down, reply: PendingReply<'_, ()>, _: &Context) { + self.seen.downs.lock().unwrap().push(down); + reply.reply(()); + } +} + +struct Panic; +impl Message for Panic { + type Result = (); +} +impl Handler for Parent { + async fn handle(&mut self, _: Panic, _: PendingReply<'_, ()>, _: &Context) { + panic!("parent fails"); + } +} + +struct Stop; +impl Message for Stop { + type Result = (); +} +impl Handler for Parent { + async fn handle(&mut self, _: Stop, reply: PendingReply<'_, ()>, ctx: &Context) { + ctx.stop(); + reply.reply(()); + } +} + +async fn parent(trap: bool) -> (spawned_concurrency::tasks::ActorRef, Seen) { + let seen = Seen::default(); + let p = Parent { + trap, + seen: seen.clone(), + } + .start() + .await + .unwrap() + .started() + .unwrap(); + (p, seen) +} + +#[derive(Clone, Copy)] +enum Outcome { + Ready, + Continue, + Error, + Ignore, + Panic, + StopShutdown, + StopPayload, +} + +/// A child whose `started()` may wait on a gate, then returns `outcome`. +struct Child { + outcome: Outcome, + gate: Option>, + log: Arc>>, +} + +impl Child { + fn new(outcome: Outcome) -> Self { + Self { + outcome, + gate: None, + log: Arc::default(), + } + } + fn gated(outcome: Outcome) -> (Self, tokio::sync::oneshot::Sender<()>) { + let (release, gate) = tokio::sync::oneshot::channel(); + let mut child = Self::new(outcome); + child.gate = Some(gate); + (child, release) + } +} + +impl Actor for Child { + async fn started(&mut self, _: &Context) -> Init { + if let Some(gate) = self.gate.take() { + let _ = gate.await; + } + match self.outcome { + Outcome::Ready => Init::ready(), + Outcome::Continue => Init::continue_with(Cont), + Outcome::Error => Init::error("no config"), + Outcome::Ignore => Init::ignore(), + Outcome::Panic => panic!("child fails in started()"), + Outcome::StopShutdown => Init::stop(ExitReason::Shutdown(None)), + Outcome::StopPayload => Init::stop(ExitReason::shutdown_with("no backend")), + } + } +} + +struct Cont; +impl Message for Cont { + type Result = (); +} +impl Handler for Child { + async fn handle(&mut self, _: Cont, _: PendingReply<'_, ()>, _: &Context) { + self.log.lock().unwrap().push("continue"); + } +} + +struct Work; +impl Message for Work { + type Result = (); +} +impl Handler for Child { + async fn handle(&mut self, _: Work, reply: PendingReply<'_, ()>, _: &Context) { + self.log.lock().unwrap().push("work"); + reply.reply(()); + } +} + +impl Handler for Child { + async fn handle(&mut self, _: Stop, reply: PendingReply<'_, ()>, ctx: &Context) { + ctx.stop(); + reply.reply(()); + } +} + +struct Unlink(ChildHandle); +impl Message for Unlink { + type Result = (); +} +impl Handler for Child { + async fn handle(&mut self, msg: Unlink, reply: PendingReply<'_, ()>, ctx: &Context) { + ctx.unlink(&msg.0); + reply.reply(()); + } +} + +/// Wait until `n` children have finished their owned work (published, jobs +/// joined): the last events of a child. +async fn children_done(n: usize) { + until("children's owned work", || count("SWorkDone") >= n).await; +} + +// --------------------------------------------------------------------------- +// Scenarios +// --------------------------------------------------------------------------- + +/// start_link of a ready child; a message; an administrative stop (the +/// child's ordinary exit); its Normal notice reaches the non-trapping P. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn link_ready_then_stop() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let child = Child::new(Outcome::Ready) + .start_link(&p.context()) + .await + .unwrap() + .started() + .unwrap(); + child.request(Work).await.unwrap(); + child.request(Stop).await.unwrap(); + assert_eq!(child.wait_exit().await, ExitReason::Normal); + children_done(1).await; + until("P's notice", || count("PNotice") >= 1).await; + let events = write("link_ready_then_stop", "U", false, 1000); + has( + &events, + &[ + "SBegin:link", + "SCommit:U", + "SStartedBegin", + "SResolve:ready", + "SAnswer:ok", + "SSend", + "SHandle", + "SExit:Normal", + "SDie:Normal", + "SPublish:deliver", + "PNotice:link", + "SWorkDone", + ], + ); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// A failed start_monitor (Init::error) consumes its Down; ready() +/// reports the same result. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn monitor_error_consumes_down() { + let _serial = SERIAL.lock().await; + let (p, seen) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let starting = Child::new(Outcome::Error).start_monitor(&p.context()); + let child = starting.actor_ref().unwrap(); + let result = starting.await; + assert!( + matches!(&result, Err(StartError::InitFailed { cause, pending: None }) if cause == "no config"), + "{result:?}" + ); + assert!(matches!( + child.ready().await, + Err(StartError::InitFailed { pending: None, .. }) + )); + children_done(1).await; + let events = write("monitor_error_consumes_down", "U", false, 1000); + has( + &events, + &[ + "SBegin:monitor", + "SCommit:U", + "SStartedBegin", + "SResolve:error", + "SDie:Normal", + "SPublish:divert", + "SAnswer:failed", + "SReady:failed", + ], + ); + assert!(seen.downs.lock().unwrap().is_empty()); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// A trapping parent: a child panicking in started() fails start_link with +/// the final reason, and the Exit is consumed (OTP's flush). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn link_panic_in_started_trapping_parent() { + let _serial = SERIAL.lock().await; + let (p, seen) = parent(true).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let result = Child::new(Outcome::Panic).start_link(&p.context()).await; + assert!(matches!( + result, + Err(StartError::InitExited { + reason: ExitReason::Panic(_), + pending: None + }) + )); + children_done(1).await; + let events = write("link_panic_in_started_trapping_parent", "U", true, 1000); + has( + &events, + &[ + "SBegin:link", + "SResolve:panic", + "SDie:Panic", + "SPublish:divert", + "SAnswer:failed", + ], + ); + assert!(seen.exits.lock().unwrap().is_empty()); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// A non-trapping parent: a child stopping with Shutdown in started() +/// fails start_link, and the notice still ends the parent (OTP: the link +/// kills the caller). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn link_stop_in_started_ends_nontrapping_parent() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let result = Child::new(Outcome::StopShutdown) + .start_link(&p.context()) + .await; + assert!( + matches!( + result, + Err(StartError::InitExited { + reason: ExitReason::Shutdown(None), + pending: None + }) + ), + "{result:?}" + ); + assert_eq!(p.wait_exit().await, ExitReason::Shutdown(None)); + children_done(1).await; + until("P's publication", || count("PPublish") >= 1).await; + let events = write( + "link_stop_in_started_ends_nontrapping_parent", + "U", + false, + 1000, + ); + has( + &events, + &[ + "SResolve:stopS", + "SDie:Shutdown", + "SPublish:deliver", + "PNotice:link", + "PDie", + "PPublish", + ], + ); +} + +/// Names: committed with the start, found by a typed lookup, refused to a +/// competitor, released before publication, free again after. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn named_start_clash_and_release() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + static NAME: Name = Name::new("start-trace-name"); + let first = Child::new(Outcome::Ready) + .with_options(StartOptions::new().name(NAME.clone())) + .start() + .await + .unwrap() + .started() + .unwrap(); + assert_eq!(NAME.whereis().map(|a| a.id()), Some(first.id())); + let clash = Child::new(Outcome::Ready) + .with_options(StartOptions::new().name(NAME.clone())) + .start() + .await; + match clash { + Err(StartError::AlreadyStarted { existing, .. }) => { + assert_eq!(existing.map(|h| h.id()), Some(first.id())) + } + other => panic!("{other:?}"), + } + first.request(Stop).await.unwrap(); + first.wait_exit().await; + assert!(NAME.whereis().is_none()); + let again = Child::new(Outcome::Ready) + .with_options(StartOptions::new().name(NAME.clone())) + .start() + .await + .unwrap() + .started() + .unwrap(); + again.request(Stop).await.unwrap(); + again.wait_exit().await; + children_done(2).await; + let events = write("named_start_clash_and_release", "U", false, 1000); + has( + &events, + &[ + "SCommit:U", + "SLookup:hit", + "SRefuse:already", + "SReleaseName", + "SPublish:none", + "SLookup:miss", + "SCommit:U", + ], + ); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// spawn + ready() for an ignoring child, and a waiting start that is +/// ignored. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn ignore_answers_ignored() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let spawned = Child::new(Outcome::Ignore).spawn(); + assert!(matches!( + spawned.ready().await, + Ok(StartOutcome::Ignored { pending: None }) + )); + assert_eq!(spawned.wait_exit().await, ExitReason::Normal); + let waited = Child::new(Outcome::Ignore).start().await.unwrap(); + assert!(waited.is_ignored()); + // Invalid options: refused before anything exists. + assert!(matches!( + Child::new(Outcome::Ready) + .with_options(StartOptions::new().init_timeout(Duration::ZERO)) + .spawn(), + Err(StartError::InvalidOptions(_)) + )); + children_done(2).await; + let events = write("ignore_answers_ignored", "U", false, 1000); + has( + &events, + &[ + "SBegin:spawn", + "SResolve:ignore", + "SReady:ignored", + "SBegin:start", + "SResolve:ignore", + "SAnswer:ignored", + ], + ); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// The continuation runs before the messages sent while started() ran. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn continuation_precedes_queued_messages() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let (child, release) = Child::gated(Outcome::Continue); + let log = child.log.clone(); + let child = child.spawn(); + until("started() began", || count("SStartedBegin") >= 1).await; + child.send(Work).unwrap(); + child.send(Work).unwrap(); + release.send(()).unwrap(); + child.request(Stop).await.unwrap(); + child.wait_exit().await; + children_done(1).await; + assert_eq!(*log.lock().unwrap(), ["continue", "work", "work"]); + let events = write("continuation_precedes_queued_messages", "U", false, 1000); + has( + &events, + &[ + "SStartedBegin", + "SSend", + "SSend", + "SResolve:continue", + "SContinue", + "SHandle", + "SHandle", + ], + ); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// The init timer wins over a stuck started(): the waiting start answers +/// InitTimeout after publication; the trapping parent's Exit is consumed. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn init_timeout_waiting() { + let _serial = SERIAL.lock().await; + let (p, seen) = parent(true).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let (child, release) = Child::gated(Outcome::Ready); + let starting = child + .with_options(StartOptions::new().init_timeout(Duration::from_millis(20))) + .start_link(&p.context()); + until("the timer", || count("STimer") >= 1).await; + release.send(()).unwrap(); + let result = starting.await; + assert!( + matches!(result, Err(StartError::InitTimeout { pending: None })), + "{result:?}" + ); + children_done(1).await; + let events = write("init_timeout_waiting", "U", true, 1000); + has( + &events, + &[ + "SStartedBegin", + "STimer:won", + "SResolve:ready", + "SDie:Killed", + "SPublish:divert", + "SAnswer:timeout", + ], + ); + assert!(seen.exits.lock().unwrap().is_empty()); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// An abandoned start_link whose child then times out: the diverted +/// notice is handed on, and the trapping parent gets the Exit (Killed). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn abandoned_timeout_delivers_killed() { + let _serial = SERIAL.lock().await; + let (p, seen) = parent(true).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let (child, release) = Child::gated(Outcome::Ready); + let starting = child + .with_options(StartOptions::new().init_timeout(Duration::from_millis(20))) + .start_link(&p.context()); + let child = starting.actor_ref().unwrap(); + drop(starting); + until("the timer", || count("STimer") >= 1).await; + release.send(()).unwrap(); + assert_eq!(child.wait_exit().await, ExitReason::Killed); + children_done(1).await; + until("P's notice", || count("PNotice") >= 1).await; + until("the Exit", || !seen.exits.lock().unwrap().is_empty()).await; + let events = write("abandoned_timeout_delivers_killed", "U", true, 1000); + has( + &events, + &[ + "SAbandon", + "STimer:won", + "SResolve:ready", + "SPublish:deliver", + "PNotice:link", + ], + ); + assert_eq!(seen.exits.lock().unwrap()[0].reason, ExitReason::Killed); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// A kill while started() runs fails the start (row 2, no stopped()); an +/// administrative stop waits for started(): the start succeeds, then the +/// actor exits through stopped() (OTP gen_server:stop). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kill_and_admin_stop_during_started() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let (child, release) = Child::gated(Outcome::Ready); + let starting = child.start(); + let handle = starting.actor_ref().unwrap().child_handle(); + until("started() began", || count("SStartedBegin") >= 1).await; + handle.kill(); + release.send(()).unwrap(); + assert!(matches!( + starting.await, + Err(StartError::InitExited { + reason: ExitReason::Killed, + pending: None + }) + )); + let (child, release) = Child::gated(Outcome::Ready); + let starting = child.start(); + let handle = starting.actor_ref().unwrap().child_handle(); + until("started() began", || count("SStartedBegin") >= 2).await; + handle.stop_with_reason(ExitReason::Shutdown(None)); + release.send(()).unwrap(); + // OTP gen_server:stop: the stop waits for started(); the start succeeds, + // then the actor exits with the stop's reason through stopped(). + let child = match starting.await { + Ok(StartOutcome::Started(child)) => child, + other => panic!("{other:?}"), + }; + assert_eq!(child.wait_exit().await, ExitReason::Shutdown(None)); + children_done(2).await; + let events = write("kill_and_admin_stop_during_started", "U", false, 1000); + has( + &events, + &[ + "SStartedBegin", + "SKill", + "SResolve:ready", + "SDie:Killed", + "SAnswer:failed", + "SStartedBegin", + "SAdminStop", + "SResolve:ready", + "SExit:Shutdown", + "SDie:Shutdown", + ], + ); + // The ready answer races the stop's death; it follows the ready resolve. + has(&events, &["SAdminStop", "SResolve:ready", "SAnswer:ok"]); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// The parent's abnormal exit reaches a linked child still in started(): +/// held, it fails the start (row 3). The waiter here is outside P. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn parent_exit_during_child_started() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let (child, release) = Child::gated(Outcome::Ready); + let starting = child.start_link(&p.context()); + until("started() began", || count("SStartedBegin") >= 1).await; + let _ = p.send(Panic); + assert!(matches!(p.wait_exit().await, ExitReason::Panic(_))); + until("P's publication", || count("PPublish") >= 1).await; + // A start through the dead parent's context is refused. + assert!(matches!( + Child::new(Outcome::Ready).start_link(&p.context()).await, + Err(StartError::ParentExited) + )); + release.send(()).unwrap(); + assert!(matches!( + starting.await, + Err(StartError::InitExited { + reason: ExitReason::Panic(_), + pending: None + }) + )); + children_done(1).await; + let events = write("parent_exit_during_child_started", "U", false, 1000); + has( + &events, + &[ + "SStartedBegin", + "PExitBegin:Panic", + "PDie", + "PPublish", + "SResolve:ready", + "SDie:Panic", + "SAnswer:failed", + ], + ); +} + +/// A ready child unlinks from P, then dies: no notice reaches P. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn unlinked_child_sends_no_notice() { + let _serial = SERIAL.lock().await; + let (p, seen) = parent(true).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let child = Child::new(Outcome::Ready) + .start_link(&p.context()) + .await + .unwrap() + .started() + .unwrap(); + child.request(Unlink(p.child_handle())).await.unwrap(); + child.request(Stop).await.unwrap(); + child.wait_exit().await; + children_done(1).await; + let events = write("unlinked_child_sends_no_notice", "U", true, 1000); + has( + &events, + &["SAnswer:ok", "SUnlink", "SExit:Normal", "SPublish:deliver"], + ); + assert!(!events.iter().any(|e| e.starts_with("PNotice"))); + assert!(seen.exits.lock().unwrap().is_empty()); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// A managed parent in a thread group: link and monitor children inherit +/// its group; a full group refuses (Capacity); the slot is released at the +/// scope's completion, after publication. +#[cfg(not(spawned_sim))] +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn inherited_thread_group_capacity() { + let _serial = SERIAL.lock().await; + let system = spawned_concurrency::system::ActorSystem::new().unwrap(); + let group = system.thread_group("start-trace", 2).unwrap(); + let seen = Seen::default(); + let p = Parent { + trap: false, + seen: seen.clone(), + } + .with_options(&group) + .start() + .await + .unwrap() + .started() + .unwrap(); + lifecycle_trace::record_start_parent(&p.child_handle()); + let first = Child::new(Outcome::Ready) + .start_link(&p.context()) + .await + .unwrap() + .started() + .unwrap(); + let full = Child::new(Outcome::Ready).start_monitor(&p.context()).await; + assert!(matches!(full, Err(StartError::Capacity)), "{full:?}"); + first.request(Stop).await.unwrap(); + first.wait_exit().await; + until("the slot's release", || count("SReleaseSlot") >= 1).await; + let (second, _monitor) = Child::new(Outcome::Ready) + .start_monitor(&p.context()) + .await + .unwrap() + .started() + .unwrap(); + second.request(Stop).await.unwrap(); + second.wait_exit().await; + until("both slots", || count("SReleaseSlot") >= 2).await; + until("the Down", || !seen.downs.lock().unwrap().is_empty()).await; + let events = write("inherited_thread_group_capacity", "G", false, 2); + has( + &events, + &[ + "SBegin:link", + "SCommit:G", + "SAnswer:ok", + "SBegin:monitor", + "SRefuse:capacity", + "SPublish:deliver", + "SWorkDone", + "SReleaseSlot", + "SBegin:monitor", + "SCommit:G", + "SPublish:deliver", + "PNotice:monitor", + ], + ); + p.request(Stop).await.unwrap(); + p.wait_exit().await; + assert!(system.shutdown(LIMIT).await.complete); +} + +/// A failed start_link whose wait is abandoned after publication: the Exit +/// diverted to the waiter is handed on to the trapping parent, exactly once. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn abandon_after_publication_hands_on_exit() { + let _serial = SERIAL.lock().await; + let (p, seen) = parent(true).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let starting = Child::new(Outcome::Panic).start_link(&p.context()); + let child = starting.actor_ref().unwrap(); + assert!(matches!(child.wait_exit().await, ExitReason::Panic(_))); + until("the diversion", || count("SPublish") >= 1).await; + drop(starting); + until("the Exit", || !seen.exits.lock().unwrap().is_empty()).await; + children_done(1).await; + let events = write("abandon_after_publication_hands_on_exit", "U", true, 1000); + has( + &events, + &[ + "SResolve:panic", + "SPublish:divert", + "SAbandon", + "PNotice:link", + ], + ); + assert_eq!(seen.exits.lock().unwrap().len(), 1); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// The ignoring child's destructor hangs: the waiter answers `Ignored` +/// with `pending` at the cleanup deadline (5 s), before publication. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn stuck_ignore_answers_pending() { + struct HangsInDrop(std::sync::mpsc::Receiver<()>); + impl Drop for HangsInDrop { + fn drop(&mut self) { + let _ = self.0.recv(); + } + } + struct Stuck(#[allow(dead_code)] Option); + impl Actor for Stuck { + async fn started(&mut self, _: &Context) -> Init { + Init::ignore() + } + } + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let (release, hang) = std::sync::mpsc::channel(); + let outcome = Stuck(Some(HangsInDrop(hang))).start().await.unwrap(); + let pending = match outcome { + StartOutcome::Ignored { + pending: Some(pending), + } => pending, + other => panic!("{other:?}"), + }; + release.send(()).unwrap(); + assert_eq!(pending.wait_exit_async().await, ExitReason::Normal); + children_done(1).await; + let events = write("stuck_ignore_answers_pending", "U", false, 1000); + has( + &events, + &[ + "SResolve:ignore", + "SDie:Normal", + "SDeadline", + "SAnswer:ignored", + "SPublish:none", + ], + ); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// Threads mode: a blocking start_link that fails, and a ready child with a +/// continuation. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn threads_mode_link_and_continuation() { + use spawned_concurrency::threads; + let _serial = SERIAL.lock().await; + struct TParent; + impl threads::Actor for TParent {} + struct TChild(bool, Arc>>); + impl threads::Actor for TChild { + fn started(&mut self, _: &threads::Context) -> threads::Init { + if self.0 { + threads::Init::error("bad") + } else { + threads::Init::continue_with(Cont) + } + } + } + impl threads::Handler for TChild { + fn handle(&mut self, _: Cont, _: PendingReply<'_, ()>, _: &threads::Context) { + self.1.lock().unwrap().push("continue"); + } + } + impl threads::Handler for TChild { + fn handle(&mut self, _: Stop, reply: PendingReply<'_, ()>, ctx: &threads::Context) { + ctx.stop(); + reply.reply(()); + } + } + let p = threads::ActorStart::spawn(TParent); + lifecycle_trace::record_start_parent(&p.child_handle()); + let ctx = p.context(); + let log: Arc>> = Arc::default(); + let (failed, ready) = tokio::task::spawn_blocking({ + let log = log.clone(); + move || { + let failed = threads::ActorStart::start_link(TChild(true, log.clone()), &ctx); + let ready = threads::ActorStart::start_link(TChild(false, log), &ctx) + .unwrap() + .started() + .unwrap(); + ready.request(Stop).unwrap(); + ready.wait_exit(); + (failed, ready) + } + }) + .await + .unwrap(); + assert!(matches!(failed, Err(StartError::InitFailed { .. }))); + assert_eq!(*log.lock().unwrap(), ["continue"]); + drop(ready); + children_done(2).await; + let events = write("threads_mode_link_and_continuation", "U", false, 1000); + has( + &events, + &[ + "SResolve:error", + "SPublish:drop", + "SAnswer:failed", + "SResolve:continue", + "SContinue", + "SExit:Normal", + ], + ); + p.context().stop(); + p.wait_exit(); +} + +/// Review #291 item 1: a committed managed child whose dispatch fails (its +/// group dropped mid-shutdown; failpoint `system.dispatch_actor`) is killed +/// before started() (`KilledEarly`): the start_link answers +/// `InitExited { Killed }` (not a stage-1 refusal with the edge applied), +/// the waiting trapping parent's Exit is consumed by the answer, and the +/// start_monitor's Down likewise. No waiter stays `Waiting`. +#[cfg(all(feature = "failpoints", not(spawned_sim)))] +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn dispatch_failure_after_commit_kills_child() { + use spawned_concurrency::failpoints::{self, Action, FailPoint, FailScenario}; + let _serial = SERIAL.lock().await; + let _scenario = FailScenario::setup(); + let system = spawned_concurrency::system::ActorSystem::new().unwrap(); + let group = system.thread_group("start-dispatch", 3).unwrap(); + let seen = Seen::default(); + let p = Parent { + trap: true, + seen: seen.clone(), + } + .with_options(&group) + .start() + .await + .unwrap() + .started() + .unwrap(); + lifecycle_trace::record_start_parent(&p.child_handle()); + failpoints::configure("system.dispatch_actor", FailPoint::new(Action::Error)).unwrap(); + let linked = tokio::time::timeout(LIMIT, Child::new(Outcome::Ready).start_link(&p.context())) + .await + .expect("the start_link answers"); + assert!( + matches!( + linked, + Err(StartError::InitExited { + reason: ExitReason::Killed, + pending: None + }) + ), + "{linked:?}" + ); + let monitored = tokio::time::timeout( + LIMIT, + Child::new(Outcome::Ready).start_monitor(&p.context()), + ) + .await + .expect("the start_monitor answers"); + assert!( + matches!( + monitored, + Err(StartError::InitExited { + reason: ExitReason::Killed, + pending: None + }) + ), + "{monitored:?}" + ); + failpoints::remove("system.dispatch_actor"); + until("both slots", || count("SReleaseSlot") >= 2).await; + let events = write("dispatch_failure_after_commit_kills_child", "G", true, 2); + has( + &events, + &[ + "SBegin:link", + "SCommit:G", + "SKill", + "SResolve:none", + "SDie:Killed", + "SPublish:divert", + "SAnswer:failed", + "SBegin:monitor", + "SCommit:G", + "SKill", + "SResolve:none", + "SPublish:divert", + "SAnswer:failed", + ], + ); + assert!( + seen.exits.lock().unwrap().is_empty(), + "Exit consumed by the answer" + ); + assert!( + seen.downs.lock().unwrap().is_empty(), + "Down consumed by the answer" + ); + assert!(p.child_handle().is_alive(), "a trapping parent survives"); + p.request(Stop).await.unwrap(); + p.wait_exit().await; + assert!(system.shutdown(LIMIT).await.complete); +} + +/// Review #291 item 1: a committed `Backend::Thread` child whose thread +/// cannot be spawned (failpoint `actor.thread_spawn`) is killed before +/// started(), and its start_link answers `InitExited { Killed }` instead of +/// panicking in the parent. +#[cfg(feature = "failpoints")] +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn thread_spawn_failure_after_commit_kills_child() { + use spawned_concurrency::failpoints::{self, Action, FailPoint, FailScenario}; + let _serial = SERIAL.lock().await; + let _scenario = FailScenario::setup(); + let (p, seen) = parent(true).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + failpoints::configure("actor.thread_spawn", FailPoint::new(Action::Error)).unwrap(); + let linked = Child::new(Outcome::Ready) + .with_options(StartOptions::new().backend(spawned_concurrency::tasks::Backend::Thread)) + .start_link(&p.context()) + .await; + failpoints::remove("actor.thread_spawn"); + assert!( + matches!( + linked, + Err(StartError::InitExited { + reason: ExitReason::Killed, + pending: None + }) + ), + "{linked:?}" + ); + children_done(1).await; + let events = write( + "thread_spawn_failure_after_commit_kills_child", + "U", + true, + 1000, + ); + has( + &events, + &[ + "SBegin:link", + "SCommit:U", + "SKill", + "SResolve:none", + "SDie:Killed", + "SPublish:divert", + "SAnswer:failed", + ], + ); + assert!(seen.exits.lock().unwrap().is_empty()); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} + +/// Starts a gated child linked to the parent from the parent's own turn and +/// reports the start's result. +type Done = tokio::sync::oneshot::Sender>; +struct LinkGated(Mutex>, Mutex>); +impl Message for LinkGated { + type Result = (); +} +impl Handler for Parent { + async fn handle(&mut self, msg: LinkGated, reply: PendingReply<'_, ()>, ctx: &Context) { + reply.reply(()); + let child = msg.0.lock().unwrap().take().unwrap(); + let result = child.start_link(ctx).await.map(|_| ()); + let done = msg.1.lock().unwrap().take(); + if let Some(done) = done { + let _ = done.send(result); + } + } +} + +/// Review #291 item 7: a non-trapping parent killed while it awaits a +/// `start_link` does not wait for the child's `started()` (OTP: the caller +/// dies at once): the wait is abandoned (`ParentExited`), the parent's turn +/// ends and it dies `Killed`; the child, still in `started()`, holds the +/// parent's exit and fails its start when `started()` returns. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn parent_killed_while_awaiting_start_link_abandons_the_wait() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(false).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let (child, release) = Child::gated(Outcome::Ready); + let (done_tx, done) = tokio::sync::oneshot::channel(); + p.request(LinkGated( + Mutex::new(Some(child)), + Mutex::new(Some(done_tx)), + )) + .await + .unwrap(); + until("the child's started()", || count("SStartedBegin") >= 1).await; + p.child_handle().kill(); + let result = tokio::time::timeout(LIMIT, done).await.unwrap().unwrap(); + assert!( + matches!(result, Err(StartError::ParentExited)), + "{result:?}" + ); + assert_eq!( + tokio::time::timeout(LIMIT, p.wait_exit()).await.unwrap(), + ExitReason::Killed, + "the parent died without waiting for started()" + ); + release.send(()).unwrap(); + children_done(1).await; + let events = write( + "parent_killed_while_awaiting_start_link_abandons_the_wait", + "U", + false, + 1000, + ); + has( + &events, + &[ + "SBegin:link", + "SCommit:U", + "SStartedBegin", + "PExitBegin:Killed", + "SAbandon", + "PDie", + "SResolve:ready", + "SDie:Killed", + ], + ); +} + +/// `{shutdown, Term}` (G7): `Init::stop(shutdown_with(..))` fails the start +/// with the full reason, and the start trace classes it as `Shutdown` +/// (the model's `stopS`). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn init_stop_with_shutdown_payload() { + let _serial = SERIAL.lock().await; + let (p, _) = parent(true).await; + lifecycle_trace::record_start_parent(&p.child_handle()); + let result = Child::new(Outcome::StopPayload) + .start_link(&p.context()) + .await; + match &result { + Err(StartError::InitExited { + reason, + pending: None, + }) => { + assert!(reason.is_shutdown()); + assert_eq!(reason.shutdown_payload(), Some("no backend")); + } + other => panic!("{other:?}"), + } + children_done(1).await; + let events = write("init_stop_with_shutdown_payload", "U", true, 1000); + has( + &events, + &[ + "SResolve:stopS", + "SDie:Shutdown", + "SPublish:divert", + "SAnswer:failed", + ], + ); + p.request(Stop).await.unwrap(); + p.wait_exit().await; +} diff --git a/concurrency/tests/startup_admission.rs b/concurrency/tests/startup_admission.rs index 7ba0a712b..6b93c398a 100644 --- a/concurrency/tests/startup_admission.rs +++ b/concurrency/tests/startup_admission.rs @@ -61,14 +61,19 @@ impl Drop for Probe { // Not simulated: blocks the group's OS worker on purpose. #[cfg(not(spawned_sim))] #[spawned_macros::sim_test(flavor = "current_thread")] -async fn managed_actor_stopped_before_first_poll_runs_no_hooks_but_drops_state() { +/// A managed actor stopped (`ChildHandle::stop`) before its first poll: the +/// stop waits for `started()` (OTP `gen_server:stop`), the start succeeds, +/// then the actor exits through `stopped()` without handling its queue. +async fn managed_actor_stopped_before_first_poll_starts_then_stops() { let system = ActorSystem::new().unwrap(); let group = system.async_group("startup-gate", 1).unwrap(); - let owner = Probe(Arc::default()).start_on(&group).unwrap(); + let owner = Probe(Arc::default()).with_options(&group).spawn().unwrap(); spawned_rt::tasks::timeout(LIMIT, owner.ready()) .await .unwrap() - .unwrap(); + .unwrap() + .started() + .expect("started"); let (entered, arrival) = tokio::sync::oneshot::channel(); let (release, gate) = std::sync::mpsc::channel(); // Intentionally occupy the sole worker to force the next root to remain @@ -86,7 +91,7 @@ async fn managed_actor_stopped_before_first_poll_runs_no_hooks_but_drops_state() .unwrap() .unwrap(); let counts = Arc::new(Counts::default()); - let actor = Probe(counts.clone()).start_on(&group).unwrap(); + let actor = Probe(counts.clone()).with_options(&group).spawn().unwrap(); let dropped = Arc::new(AtomicUsize::new(0)); let replies: Vec<_> = (0..2) .map(|_| { @@ -112,12 +117,14 @@ async fn managed_actor_stopped_before_first_poll_runs_no_hooks_but_drops_state() .unwrap(), ExitReason::Shutdown(None) ); - assert!(spawned_rt::tasks::timeout(LIMIT, actor.ready()) - .await - .unwrap() - .is_err()); - assert_eq!(counts.started.load(Ordering::SeqCst), 0); - assert_eq!(counts.stopped.load(Ordering::SeqCst), 0); + assert!(matches!( + spawned_rt::tasks::timeout(LIMIT, actor.ready()) + .await + .unwrap(), + Ok(spawned_concurrency::StartOutcome::Started(())) + )); + assert_eq!(counts.started.load(Ordering::SeqCst), 1); + assert_eq!(counts.stopped.load(Ordering::SeqCst), 1); assert_eq!(counts.dropped.load(Ordering::SeqCst), 1); assert_eq!(dropped.load(Ordering::SeqCst), 2); assert_eq!( @@ -134,9 +141,11 @@ async fn managed_actor_stopped_before_first_poll_runs_no_hooks_but_drops_state() } #[spawned_macros::sim_test(flavor = "current_thread")] -async fn legacy_actor_stopped_before_first_poll_keeps_both_hooks() { +/// An actor stopped before its first poll runs both hooks: the stop waits +/// for `started()` (OTP `gen_server:stop`; unified start 2.3). +async fn actor_stopped_before_first_poll_keeps_both_hooks() { let counts = Arc::new(Counts::default()); - let actor = Probe(counts.clone()).start(); // this runtime has not yielded + let actor = Probe(counts.clone()).spawn(); // this runtime has not yielded actor .child_handle() .stop_with_reason(spawned_concurrency::ExitReason::Shutdown(None)); diff --git a/concurrency/tests/supervision.rs b/concurrency/tests/supervision.rs index a8b6ad043..e10930c24 100644 --- a/concurrency/tests/supervision.rs +++ b/concurrency/tests/supervision.rs @@ -734,9 +734,14 @@ async fn system_shutdown_orders_standalone_actors_and_supervisor_trees_together( name: "a", log: log.clone(), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap(); - within!(a.ready()).await.unwrap(); + within!(a.ready()) + .await + .unwrap() + .started() + .expect("started"); let mut trees = Vec::new(); for name in ["b", "d"] { let mut tree = system.supervisor(config(5)).unwrap(); @@ -752,11 +757,14 @@ async fn system_shutdown_orders_standalone_actors_and_supervisor_trees_together( name: "c", log: log.clone(), } - .start_on(&group) + .with_options(&group) + .spawn() .unwrap() .ready() .await - .unwrap(); + .unwrap() + .started() + .expect("started"); } } assert!(system.shutdown(LIMIT).await.complete); @@ -813,8 +821,12 @@ async fn stuck_supervisor_does_not_prevent_earlier_standalone_shutdown_after_sys use spawned_concurrency::tasks::ActorStart; let system = ActorSystem::new().unwrap(); let group = system.async_group("stuck-tree-order", 1).unwrap(); - let earlier = Counter(0).start_on(&group).unwrap(); - within!(earlier.ready()).await.unwrap(); + let earlier = Counter(0).with_options(&group).spawn().unwrap(); + within!(earlier.ready()) + .await + .unwrap() + .started() + .expect("started"); let (entered_tx, entered) = tokio::sync::oneshot::channel(); let (release, release_rx) = tokio::sync::oneshot::channel(); let args = Arc::new(std::sync::Mutex::new(Some((entered_tx, release_rx)))); diff --git a/concurrency/tests/support/deadline.rs b/concurrency/tests/support/deadline.rs index 7c14ab612..e67f9aff9 100644 --- a/concurrency/tests/support/deadline.rs +++ b/concurrency/tests/support/deadline.rs @@ -9,7 +9,7 @@ //! #[macro_use] //! #[path = "support/deadline.rs"] //! mod deadline; -//! let tree = within!(tree.start()).await.unwrap(); +//! let tree = within!(tree.spawn()).await.unwrap(); //! ``` #![allow(dead_code, unused_macros)] use std::time::Duration; diff --git a/concurrency/tests/support/failure_actor.rs b/concurrency/tests/support/failure_actor.rs index 6a6469458..a3eb5b56c 100644 --- a/concurrency/tests/support/failure_actor.rs +++ b/concurrency/tests/support/failure_actor.rs @@ -311,18 +311,23 @@ impl Response { } impl Handle { pub fn start(backend: Backend, worker: Worker) -> Self { - #[cfg(feature = "async")] - use spawned_concurrency::tasks::ActorStart as _; use spawned_concurrency::threads::ActorStart as _; match backend { - Backend::Native => Self::Native(NativeWorker(worker).start()), + Backend::Native => Self::Native(NativeWorker(worker).spawn()), #[cfg(feature = "async")] - other => Self::Async(AsyncWorker(worker).start_with_backend(match other { - Backend::Async => spawned_concurrency::tasks::Backend::Async, - Backend::Blocking => spawned_concurrency::tasks::Backend::Blocking, - Backend::Thread => spawned_concurrency::tasks::Backend::Thread, - Backend::Native => unreachable!(), - })), + other => Self::Async( + spawned_concurrency::tasks::ActorStart::with_options( + AsyncWorker(worker), + match other { + Backend::Async => spawned_concurrency::tasks::Backend::Async, + Backend::Blocking => spawned_concurrency::tasks::Backend::Blocking, + Backend::Thread => spawned_concurrency::tasks::Backend::Thread, + Backend::Native => unreachable!(), + }, + ) + .spawn() + .expect("spawn"), + ), } } pub fn child(&self) -> ChildHandle { diff --git a/concurrency/tests/ui-pass/actor_lint_allow_shared_state.rs b/concurrency/tests/ui-pass/actor_lint_allow_shared_state.rs index 5bd447f35..d026e95fa 100644 --- a/concurrency/tests/ui-pass/actor_lint_allow_shared_state.rs +++ b/concurrency/tests/ui-pass/actor_lint_allow_shared_state.rs @@ -37,7 +37,7 @@ fn main() { let rt = tokio::runtime::Runtime::new().unwrap(); rt.block_on(async { let seen = Arc::new(AtomicU64::new(0)); - let actor = Shared::new(seen.clone()).start(); + let actor = Shared::new(seen.clone()).spawn(); assert_eq!(actor.request(Tick).await.unwrap(), 0); assert_eq!(seen.load(Ordering::SeqCst), 1); }); diff --git a/concurrency/tests/ui-pass/actor_lint_ordinary_fields.rs b/concurrency/tests/ui-pass/actor_lint_ordinary_fields.rs index 92d06459f..23ac2ca9e 100644 --- a/concurrency/tests/ui-pass/actor_lint_ordinary_fields.rs +++ b/concurrency/tests/ui-pass/actor_lint_ordinary_fields.rs @@ -48,7 +48,7 @@ impl Owned { fn main() { let rt = tokio::runtime::Runtime::new().unwrap(); rt.block_on(async { - let actor = Owned::new(Arc::new("cfg".into()), Arc::new(vec![1, 2])).start(); + let actor = Owned::new(Arc::new("cfg".into()), Arc::new(vec![1, 2])).spawn(); assert_eq!(actor.request(Get).await.unwrap(), 10); }); } diff --git a/concurrency/tests/ui-pass/actor_tasks_lifecycle.rs b/concurrency/tests/ui-pass/actor_tasks_lifecycle.rs index 3870b3e95..3bda4b276 100644 --- a/concurrency/tests/ui-pass/actor_tasks_lifecycle.rs +++ b/concurrency/tests/ui-pass/actor_tasks_lifecycle.rs @@ -52,13 +52,13 @@ fn main() { use spawned_concurrency::tasks::ActorStart as _; use spawned_concurrency::threads::ActorStart as _; spawned_rt::tasks::run(async { - let echo = asynchronous::Echo.start(); + let echo = asynchronous::Echo.spawn(); assert_eq!(echo.request(Ping(1)).await.unwrap(), 2); assert_eq!(echo.request(Pong(4)).await.unwrap(), 8); echo.child_handle().stop(); echo.join().await; }); - let idle = native::Idle.start(); + let idle = native::Idle.spawn(); idle.context().stop(); idle.join(); } diff --git a/concurrency/tests/ui-pass/actor_threads_lifecycle.rs b/concurrency/tests/ui-pass/actor_threads_lifecycle.rs index 320de79d8..4d1b278f3 100644 --- a/concurrency/tests/ui-pass/actor_threads_lifecycle.rs +++ b/concurrency/tests/ui-pass/actor_threads_lifecycle.rs @@ -46,7 +46,7 @@ impl Holder { fn main() { let hooks = Arc::new(AtomicUsize::new(0)); - let actor = Holder::new(7u32, hooks.clone()).start(); + let actor = Holder::new(7u32, hooks.clone()).spawn(); assert_eq!(actor.get().unwrap(), "7"); actor.context().stop(); actor.join(); diff --git a/concurrency/tests/ui-pass/protocol_actor.rs b/concurrency/tests/ui-pass/protocol_actor.rs index fc031bae5..e7a462276 100644 --- a/concurrency/tests/ui-pass/protocol_actor.rs +++ b/concurrency/tests/ui-pass/protocol_actor.rs @@ -14,7 +14,7 @@ impl Counter { } fn main() { spawned_rt::tasks::run(async { - let actor = Counter.start(); + let actor = Counter.spawn(); let interface: CounterRef = actor.to_counter_ref(); assert_eq!(interface.read().await.unwrap(), 42); actor.child_handle().stop(); diff --git a/concurrency/tests/ui-pass/protocol_cfg_methods.rs b/concurrency/tests/ui-pass/protocol_cfg_methods.rs index c4dd5a964..de96aa342 100644 --- a/concurrency/tests/ui-pass/protocol_cfg_methods.rs +++ b/concurrency/tests/ui-pass/protocol_cfg_methods.rs @@ -22,7 +22,7 @@ impl Fixed { } fn main() { - let actor = Fixed.start(); + let actor = Fixed.spawn(); assert_eq!(Variant::get(&actor).unwrap(), 3); actor.context().stop(); actor.join(); diff --git a/concurrency/tests/ui-pass/protocol_edge_cases.rs b/concurrency/tests/ui-pass/protocol_edge_cases.rs index b3f3a7342..16c01f2f0 100644 --- a/concurrency/tests/ui-pass/protocol_edge_cases.rs +++ b/concurrency/tests/ui-pass/protocol_edge_cases.rs @@ -63,7 +63,7 @@ impl Store { } fn main() { - let store = Store::default().start(); + let store = Store::default().spawn(); store.event(Event(1)); store.record(2).unwrap(); assert_eq!(store.events().unwrap(), vec![Event(1)]); diff --git a/concurrency/tests/ui-pass/protocol_supertraits.rs b/concurrency/tests/ui-pass/protocol_supertraits.rs index 56ff1c724..60005131d 100644 --- a/concurrency/tests/ui-pass/protocol_supertraits.rs +++ b/concurrency/tests/ui-pass/protocol_supertraits.rs @@ -51,14 +51,14 @@ fn assert_debug(_: &T) {} async fn main() { use spawned_concurrency::tasks::ActorStart as _; use spawned_concurrency::threads::ActorStart as _; - let native = native::Native.start(); + let native = native::Native.spawn(); let configured = native.with_request_timeout(Duration::from_secs(5)); assert_debug(&configured); assert_eq!(configured.to_probe_ref().value().unwrap(), 42); assert_eq!(native.kind(), "native"); assert_eq!(native.to_custom_ref().other().unwrap(), 7); native.context().stop(); native.join(); - let actor = asynchronous::Async.start(); + let actor = asynchronous::Async.spawn(); assert_eq!(actor.with_request_timeout(Duration::from_secs(5)).to_probe_ref().value().await.unwrap(),42); assert_eq!(actor.kind(), "async"); assert_eq!(actor.to_custom_ref().other().await.unwrap(),7); diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 21a85a540..0a6a57fcb 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -37,7 +37,7 @@ or model that checks each one, are collected in the Every actor goes through three phases: -1. **`started()`** — called once before the actor begins processing messages. Use this for initialization: starting timers, registering with the registry, etc. +1. **`started()`** — called once before the actor begins processing messages. Use this for initialization: starting timers, opening resources, etc. It may fail or opt out (`Result`, `Init`; see [Starting an actor](#starting-an-actor-v09)); a failed start never runs `stopped()`. 2. **Message loop** — the actor processes messages one at a time via `Handler` implementations. 3. **`stopped()`** — called once after the message loop exits, unless the exit was a kill or an exit signal (see [When `stopped()` does not run](#when-stopped-does-not-run)). Use this for cleanup. @@ -161,7 +161,7 @@ As in OTP (`terminate/2` runs only for exits the process takes itself), `stopped - a kill (`ChildHandle::kill()`, `exit(Kill)`, a supervisor's kill escalation at `shutdown_timeout`), unless `stopped()` already began; - an untrapped exit signal or link notice (`ChildHandle::exit(reason)`, a linked peer's abnormal exit); -- a supervisor's or system's shutdown, or a `start_linked` parent's exit, to an actor that does **not** trap exits: the parent's exit is an exit signal (design E3); +- a supervisor's or system's shutdown, or a `start_link` parent's exit, to an actor that does **not** trap exits: the parent's exit is an exit signal (design E3); - the runtime dropping the actor (`Killed`). Every skip is logged at `debug` with `event = "stopped_skipped"`, the actor id and type, `cause` (`kill`, `exit_signal` or `runtime_drop`) and `reason`. The log is emitted whether or not the actor overrides `stopped()` (there is no cheap way to tell), so filter on the actor type. The lifecycle trace (feature `lifecycle-trace`) records the same decision as `Hook` `skip`. @@ -248,20 +248,101 @@ async fn handle_tick(&mut self, _msg: Tick, ctx: &Context) { | `actor_ref.id()` | The actor's `ActorId` | | `actor_ref.child_handle()` | Type-erased `ChildHandle` for monitors, links and exit observation | | `actor_ref.exit_reason()` / `actor_ref.wait_exit()` | The `ExitReason` once stopped | -| `actor_ref.ready()` | Wait until a managed actor's `started()` completed (v0.7) | +| `actor_ref.ready()` | The instance's committed startup result: `Started`, `Ignored` or the start's error (every placement, v0.9) | | `actor_ref.mailbox_depth()` | Queued messages (with the `metrics` feature) | -### Starting an actor +### Starting an actor (v0.9) + +The verbs are OTP's ([design](design/unified-start.md)): + +| Verb | Waits for `started()` | Link/monitor | Placement | OTP | +|---|---|---|---|---| +| `a.spawn()` | no (`ready()` reports later) | no | unmanaged `Backend::Async` | `proc_lib:spawn` | +| `a.start().await` | yes | no | unmanaged `Backend::Async` | `gen_server:start` | +| `a.start_link(&ctx).await` | yes | linked to `ctx`'s actor before `started()` | the parent's | `gen_server:start_link` | +| `a.start_monitor(&ctx).await` | yes | monitored by `ctx`'s actor | the parent's | `gen_server:start_monitor` | +| `a.with_options(opts).` | as the verb | as the verb | `opts` (`group`, `backend`, `placement_of`), else as the verb | `spawn_opt`, `{local, Name}`, `{timeout, T}` | ```rust -use spawned_concurrency::tasks::ActorStart as _; +use spawned_concurrency::tasks::{ActorStart as _, Backend, StartOptions}; +use spawned_concurrency::{StartError, StartOutcome}; + +let actor = MyActor::new().spawn(); // no wait +let actor = MyActor::new().start().await?.started().unwrap(); // waits for started() +let actor = MyActor::new().with_options(Backend::Thread).spawn()?; +let actor = MyActor::new().with_options(&group).spawn()?; // an ExecutionGroup +let actor = MyActor::new() + .with_options(StartOptions::new().name("rpc").init_timeout(Duration::from_secs(5))) + .start() + .await?; +``` -let actor_ref = MyActor::new().start(); +A waiting start answers `Ok(StartOutcome::Started(..))` when `started()` +returned ready, `Ok(StartOutcome::Ignored { .. })` for `Init::ignore()`, or a +`StartError`. Stage 1 runs in the call (before any `.await`), so a dropped +`Starting` still started the actor; dropping it only abandons the wait. +`Starting::actor_ref()` gives the not-yet-ready instance. + +**Results** (`StartError`). Stage 1 refusals create nothing: `Closed`, +`Capacity`, `AlreadyStarted { name, existing }`, `ParentExited`, +`InvalidOptions`, `System`. After the child existed: `InitFailed { cause }` +(`Err(e)` or `Init::error(e)`, the child ended `Normal`), `InitExited { +reason }` (a kill, an untrapped exit signal, a panic, `ctx.stop*` or +`Init::stop` during `started()`; `reason` is the final one), +`InitTimeout`. An error is answered only after the child's publication: its +state is dropped, its notices are sent and its name is free, so a retry never +overlaps it. If its destructor hangs past the cleanup deadline (5 s +unmanaged), the answer carries `pending: Some(handle)`. + +**Fallible `started()`.** It may return `()`, `Result<(), E: Display>`, +`Result, E>` or `Init`: -// tasks mode: choose a backend -let actor_ref = MyActor::new().start_with_backend(Backend::Thread); +```rust +#[started] +async fn started(&mut self, ctx: &Context) -> Result, std::io::Error> { + let socket = bind(self.port).await?; // Err(e): InitFailed + if self.disabled { return Ok(Init::ignore()); } // Ok(Ignored) + self.socket = Some(socket); + Ok(Init::continue_with(Handshake)) // ready; Handshake runs first +} ``` +No failed start runs `stopped()` (OTP skips `terminate/2` when `init/1` +fails): release what `started()` acquired before returning an error, or hold +it in RAII guards. `ctx.stop()` inside `started()` is the same as returning +`Init::stop(Normal)`. An administrative stop (`ChildHandle::stop`) that +arrives while `started()` runs waits for it, as OTP's `gen_server:stop`: the +start succeeds, then the actor stops through `stopped()`. `Init::continue_with(msg)` (OTP `handle_continue`) handles +`msg` before any mailbox entry, including messages sent while `started()` +ran; readiness is committed before it, so it is not a barrier against +termination. + +**Links and monitors at start.** `start_link`/`start_monitor` commit the edge +atomically with the start, before `started()`; a parent whose exit began is +refused (`ParentExited`). A parent killed (or hit by an untrapped exit +signal) while it waits for the child's `started()` stops waiting at once: +the start answers `ParentExited` and the parent's turn ends, as in OTP the +caller dies without waiting for `init/1`; the child gets the parent's exit +through its link. A failed start's `Exit` (trapping parent) or `Down` +is consumed by the error; a non-trapping parent still exits on a +non-`Normal` failure, as the link kills the caller in OTP. An init timeout +never reaches a waiting parent. The waiting parent's turn is blocked: a +request from the child's `started()` back to its parent waits for its +timeout (as `gen_server:call` from `init/1`). + +**Placement.** `start_link` and `start_monitor` place the child in the +parent's placement (its group and shard, or its backend; no fallback: a +closed or full group fails the start). `spawn` and `start` never inherit; +`StartOptions::placement_of(&ctx)` opts in. + +**Names.** `StartOptions::name` takes a typed `Name` (`Name::new("x")` is +`const`): committed with the start (a clash is `AlreadyStarted` with the +holder's handle), found with `Name::whereis()`, released at publication +before any notice or answer. + +threads mode has the same verbs; the waiting ones block and return +`Result, StartError>`, and `StartOptions` has no placement. + --- ## Execution Modes and Backends @@ -292,9 +373,9 @@ requirement, then measure the application. ```rust use spawned_concurrency::tasks::{ActorStart as _, Backend}; -let a = MyActor::new().start(); // async on Tokio -let b = MyActor::new().start_with_backend(Backend::Blocking); // blocking I/O -let c = MyActor::new().start_with_backend(Backend::Thread); // dedicated thread +let a = MyActor::new().spawn(); // async on Tokio +let b = MyActor::new().with_options(Backend::Blocking).spawn()?; // blocking I/O +let c = MyActor::new().with_options(Backend::Thread).spawn()?; // dedicated thread ``` `Backend::Async` emits a tracing warning (debug builds only) if a poll takes @@ -1016,7 +1097,7 @@ produces `Killed`. Links are bidirectional. Use `ctx.link(&child_handle)` and `ctx.unlink(&child_handle)` for existing actors, or -`child.start_linked(&parent_ctx)`, which links before the child's `started()` +`child.start_link(&parent_ctx)`, which links before the child's `started()` and makes the caller its parent. Repeated links and links to oneself are no-ops. When a linked actor dies, each peer receives a link notice with its final reason, under the same rules as an exit signal except that a `Kill` @@ -1062,8 +1143,8 @@ example from a helper that outlives it) is inert: no record and no notice. ### Parent exits An actor's parent is fixed when it starts: the actor passed to -`start_linked`, the supervisor of a supervised child, or the `ActorSystem` of -a standalone `start_on` actor; a plain `.start()` actor has none. A signal +`start_link`, the supervisor of a supervised child, or the `ActorSystem` of +a standalone actor started in a group; a plain `.spawn()` actor has none. A signal from the parent to a **trapping** actor is queued as a parent exit: when dispatched, the actor exits with that reason (`Normal` included) and `stopped()` runs; `exit_received` never sees it. A non-trapping actor treats @@ -1117,7 +1198,7 @@ Callers see the same final reason: see [Error Handling](#error-handling) ## Execution groups and supervision Available in v0.7.0. These are opt-in capabilities for async actors; -ordinary `.start()`, message types, handlers and protocols retain their behavior. +ordinary `.spawn()`, message types, handlers and protocols retain their behavior. Parked async pools, persistent native actor-thread pools and parked Linux pinned shards share the same ownership API. Supervisors support nested child specs, OneForOne, OneForAll and RestForOne. Spinning remains experimental. @@ -1132,8 +1213,7 @@ use std::time::Duration; let system = ActorSystem::new()?; let critical = system.async_group("critical", 1)?; let background = system.async_group("background", 3)?; -let actor = MyActor::new().start_on(&critical)?; -actor.ready().await?; +let actor = MyActor::new().with_options(&critical).start().await?; let report = system.shutdown(Duration::from_secs(5)).await; assert!(report.complete); ``` @@ -1169,7 +1249,7 @@ or Flux broadcast/loss semantics are introduced. let roots = system.thread_group("blocking", 4)?; let cpus = spawned_rt::tasks::affinity::available_cpus()?; // Linux let pipeline = system.pinned_group("pipeline", cpus.into_iter().take(2).collect())?; -let stage = MyActor::new().start_on(&pipeline.shard(0)?)?; +let stage = MyActor::new().with_options(&pipeline.shard(0)?).spawn()?; ``` Actors started through `ctx.execution_group()` retain explicit pinned-shard placement. Native thread slots are exclusive, so a native actor’s context-derived group admits a new actor on any free slot instead of inheriting its occupied slot. Use `group.shard(i)` when reserving a particular native slot explicitly. @@ -1217,13 +1297,13 @@ threads/resources alive indefinitely; the host decides process-level escalation. To adopt ownership incrementally, use `system.async_group_on("client", &tokio::runtime::Handle::current())?` and -`actor.start_on(&group)`. Group shutdown closes admission and joins only managed +`actor.with_options(&group).spawn()`. Group shutdown closes admission and joins only managed actors and their registered helpers. It leaves the borrowed runtime and ambient -`.start()`/`tokio::spawn` tasks running. Keep the runtime alive and driven until +`.spawn()`/`tokio::spawn` tasks running. Keep the runtime alive and driven until cleanup completes; cloning a Tokio handle does not keep its runtime alive. On a current-thread runtime, keep calling `Runtime::block_on` while awaiting shutdown. Use an owned `async_group` for independent scheduling capacity and runtime -lifetime guarantees. Ambient `.start()` remains unmanaged, including inside a +lifetime guarantees. Ambient `.spawn()` remains unmanaged, including inside a managed handler; use `ctx.execution_group().start(...)` for owned child work. The restart limit counts restart actions across the supervisor, as OTP's @@ -1289,7 +1369,7 @@ If a managed actor's admission closes before `started()` begins, neither `started()` nor `stopped()` runs; the unstarted state and queued messages are dropped. This applies to standalone `start_on` actors as well as supervised children. Resources acquired by a constructor must therefore be released by -Drop, not only by `stopped()`. Legacy unmanaged `.start()` and synchronous +Drop, not only by `stopped()`. Unmanaged `.spawn()` and synchronous `threads` actors retain their behavior of running both hooks after an early cooperative stop, provided startup itself succeeds. Failed readiness waits for the actual startup outcome, so a non-returning hook @@ -1476,7 +1556,7 @@ automatic durable state recovery nor exactly-once effects. ### Owned helpers and registrations `Context::spawn_helper` and `spawn_blocking_helper` require an actor started with -`start_on(&group)` (or by a supervisor). An ambient `.start()` returns +`with_options(&group)` (or by a supervisor). An ambient `.spawn()` returns `SystemError::UnmanagedActor` from these methods, even inside a Tokio runtime. Use an ActorSystem to establish cleanup ownership; no detached fallback is used. @@ -1504,7 +1584,7 @@ registered helpers finish cleanup. Guard removal checks an ownership nonce so an old generation cannot delete a newer registration. Named registry instances and transparent reference rebinding are not implemented. -Arbitrary `tokio::spawn`, nested blocking/native work and ambient `.start()` do not +Arbitrary `tokio::spawn`, nested blocking/native work and ambient `.spawn()` do not automatically enroll in a scope. Track them explicitly or accept their unmanaged lifetime. An ambient actor created inside a group may be aborted by runtime teardown without running `stopped()`. Actor `join()` still waits for that actor; @@ -1744,9 +1824,9 @@ use spawned_test::tasks::{Mock, Probe}; let rates = Mock::new() .on::(|msg| if msg.currency == "EUR" { 110 } else { 100 }) - .start(); + .spawn(); let out = Probe::::new(); -let converter = Converter { rates: rates.to_rates_ref(), out: out.recipient() }.start(); +let converter = Converter { rates: rates.to_rates_ref(), out: out.recipient() }.spawn(); assert_eq!(converter.convert("EUR".into(), 3).await.unwrap(), Ok(330)); assert_eq!(out.expect(Duration::from_secs(1)).await.cents, 330); @@ -1932,10 +2012,10 @@ release's semver promise. - `spawned-rt`'s re-exports of Tokio and std types (beyond `run`, `block_on`, `ctrl_c`, `time::Instant` and the simulation entry points). -`start_linked` always starts an unmanaged `Backend::Async` actor, even from -a managed parent: the child is outside the parent's execution group and -ordered shutdown (the unified start API, planned for 0.9, will inherit the -parent's placement). +`start_link` and `start_monitor` admit the child to its managed parent's +execution group, but in 0.9 the child is not yet owned by the parent's scope +(design section 4.1, S8): the group's ordered shutdown stops it as any +member, and the parent's own exit reaches it only through the link. ## Further reading diff --git a/docs/MIGRATION.md b/docs/MIGRATION.md index 72945d7b8..0d4e60280 100644 --- a/docs/MIGRATION.md +++ b/docs/MIGRATION.md @@ -14,6 +14,77 @@ account for when upgrading. `async`, the default, if you need it explicitly). The research experiments live in the unpublished `benchmarks/experiments` crate. +#### Unified start: API call-site table + +0.9 replaces the start methods with OTP's verbs +([design](design/unified-start.md)): `spawn` does not wait (`proc_lib:spawn`); +`start`, `start_link` and `start_monitor` wait for `started()` +(`gen_server:start*`). Every break, old call to new call: + +| Old call (0.8) | New call (0.9) | Behaviour | +|---|---|---| +| `a.start()` | `a.spawn()` | same (unmanaged `Backend::Async`, does not wait) | +| | or `a.start().await?` | waits for `started()`; reports init failures | +| `a.start_with_backend(b)` | `a.with_options(b).spawn()?` | same; returns `Result` (invalid options or executor failure) | +| `a.start_on(&g)?` | `a.with_options(&g).spawn()?` | same; error type `StartError` (was `SystemError`) | +| `g.start(a)?` (`ExecutionGroup::start`) | `a.with_options(&g).spawn()?` | same | +| `a.start_linked(&ctx)` | `a.start_link(&ctx).await?` | **changes**: inherits the parent's placement, waits for `started()`, refuses an exiting parent (`ParentExited`); a failed start answers the error and consumes a trapping parent's `Exit` | +| | `a.start_link(&ctx).actor_ref()?` (then drop the `Starting`) | the 0.8 behaviour (no wait; notices delivered ordinarily), but with inherited placement | +| `ctx.monitor(&a.start().child_handle())` | `a.start_monitor(&ctx).await?` → `(ActorRef, MonitorRef)` | atomic; the parent needs `Handler` | +| `ctx.register_name(n)` in `started()` | `a.with_options(StartOptions::new().name(n))` | atomic with the start; released at publication; `Name` is typed | +| `actor.ready().await -> Result<(), SystemError>` (managed only) | `actor.ready().await -> Result, StartError>` | every placement; `Ok(Ignored)` for `Init::ignore()` | +| `SystemError::StartupFailed(r)` | `StartError::InitExited { reason: r, .. }` (or `InitFailed`, `InitTimeout`) | removed | +| `SystemError::UnmanagedActor` from `ready()` | `ready()` answers for unmanaged actors too | | +| threads `a.start()` | `a.spawn()`, or `a.start()?` (blocks for `started()`) | same / waits | +| threads `a.start_linked(&ctx)` | `a.start_link(&ctx)?` (blocks) | as tasks | +| `Mock::new()..start()` | `Mock::new()..spawn()` | same | +| `Mock::new()..start_on(&g)` | `Mock::new()..spawn_with(&g)` | same; `StartError` | +| `TestSystem::start(a)` | `TestSystem::spawn(a)`, or `TestSystem::start(a).await` (waits; panics on failure) | same / waits | +| `TestSystem::start_mock(m)` | `TestSystem::spawn_mock(m)` | same | +| `async fn started(..) {}` | unchanged | `#[started]` methods are unchanged | +| manual `impl Actor` with `async fn started` | unchanged, but add `#[allow(refining_impl_trait)]` (or allow the lint crate-wide) | `started()` now returns `impl Into>`; returning `()` refines it, which rustc's warn-by-default `refining_impl_trait` reports. `#[actor]`/`#[started]` add the allow | +| `stopped()` ran after a `ctx.stop*()` called in `started()` (and after a held exit signal resolved there) | it no longer runs after any failed start; `ctx.stop*()` in `started()` is `Init::stop` | **changes**: as OTP skips `terminate/2` when `init/1` fails. Do the cleanup in `started()` and return `Init::stop(r)`/`Init::error(e)`, or hold resources in RAII guards. An administrative stop (`ChildHandle::stop*`) during `started()` still waits for it and runs `stopped()` | + +New, no migration needed: `started()` may return `Result<(), E>`, +`Result, E>` or `Init` (`Init::ready`, `continue_with(msg)`, +`error(e)`, `stop(reason)`, `ignore()`); `StartOptions` (`name`, `label`, +`init_timeout`, `stack_size`, `max_outstanding`, `max_native_jobs`, and in +tasks mode `group`, `backend`, `placement_of`); `Starting::actor_ref()`; +`StartOutcome`, `StartError`, `Timeout`. + +Behaviour changes to check at call sites: + +- **Error type.** Functions that propagated `start_on`'s `SystemError` with + `?` now get a `StartError` (`From` exists for the other + direction's callers: `StartError::from(system_error)`). +- **Waiting starts block the parent's turn.** A `start_link` awaited inside a + handler waits for the child's `started()`; a request from that `started()` + back to its parent waits for its timeout (as OTP's `gen_server:call` from + `init/1`). Move slow initialisation into a continuation. +- **An administrative stop during `started()`** (`ChildHandle::stop*`) + waits for it, as OTP's `gen_server:stop`: the start answers `Started`, + then the actor exits with the stop's reason through `stopped()` (0.8 did + the same). A supervisor's or system's shutdown is an exit signal to its + child: a non-trapping child in `started()` fails its start without + `stopped()`, as before. +- **`ctx.stop()` inside `started()`** is the same as returning + `Init::stop(Normal)` (OTP's `{stop, R}`; a process cannot otherwise stop + itself during `init`): the start answers `InitExited { Normal }`, the + actor ends `Normal`, and **`stopped()` does not run** (0.8 ran it). Code + that called `ctx.stop()` after a failed initialisation and relied on + `stopped()` for cleanup must do that cleanup in `started()` and return + `Init::stop(reason)` or `Init::error(e)` (or `Err(e)`) instead; RAII guards + are dropped with the state either way. +- **`#[must_use]`.** `Starting` and `StartOutcome` are `#[must_use]`: + `a.start();` without `.await` still starts the actor (stage 1 is eager) and + warns. `actor.ready().await.unwrap()` now yields a `StartOutcome<()>`: + check it (`.started().expect(..)`) or ignore it explicitly. +- **Not in this release** (still 0.8 behaviour): scope ownership of linked + children (S8: a linked child of a managed parent is admitted to the + parent's group but not owned by its scope), `ChildStart::Ignore` and + `ChildState::Ignored` for supervisors (a supervised child's + `Init::ignore()` is a failed start), `ChildSpec::options`. + - `ExitReason::Shutdown` carries an optional payload, OTP's `{shutdown, Term}`: it is now `Shutdown(Option>)`. Every policy treats `Shutdown(Some(why))` exactly like the bare form (no restart of a diff --git a/docs/TESTING.md b/docs/TESTING.md index 6848b1728..76c7049c1 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -336,18 +336,22 @@ giving collateral children their own backoff fails | R3 | `protocol_timeouts` precedence; `tasks_contract::default_request_timeout_*` (4.9 s Ok, 5.1 s timeout, no timer before first poll) | A N, Sim | partial → covered | send-only methods: by type (no `Response`) | | R4 | `cancellation_safety`; `request_histories` pending ops | A B T G TG, Sim | covered | | | R5 | `semantics_gaps::*::panicking_request_handler_*`; `cross_backend` | A B T N G TG P | covered | | -| L1 | `sim_ordering::turns_*`; `*_contract::hooks_and_turns_*`; Fz | A B T N G TG, Sim (PCT, DFS) | covered (A only) → covered | | -| L2 | `startup_admission`; `execution_groups::failed_startup_*`; `tasks_contract::managed_readiness_*` (historical `ready()`) | G TG, Sim (G) | partial → covered | | +| L1 | `sim_ordering::turns_*`; `*_contract::hooks_and_turns_*`; Fz; v0.9: `start_contract::*::init_outcomes_*` (no `stopped()` after any failed start), `early_stop_regression::stop_in_started_*` | A B T N G TG, Sim (PCT, DFS) | covered (A only) → covered | | +| L2 | `startup_admission`; `execution_groups::failed_startup_*`; `tasks_contract::managed_readiness_*` (historical `ready()`); v0.9 every placement: `start_contract::*::ready_then_immediate_crash_*`, `Start!ReadyIsFinal`, `Start!ReadyOnlyWhileOpen` | A N G TG, Sim (G), Model | partial → covered | | | L3 | `context_exit` (first reason wins); `sim_ordering::turns_*` (stop never splits a turn); `*_partial::stop_reason_precedence_through_cleanup` (every requested reason x second request, `stopped()` panic, kill then `stopped()` panic, destructor panic after D) | A B T N G TG, Sim | partial → covered | | | L4 | `exit_signal_matrix`, `context_exit`, `panic_safety`; `shutdown_payload` (`{shutdown, Term}`: `join`, `Down`, trapped `Exit`, untrapped linked peer, exit signals, metrics class) | A B T N G TG | covered | owned by #240 | | L5 | `foreign_panics` (strict); `tasks_contract::panics_in_hooks_*` | A B T N G TG | covered (B, T unchecked) → covered | | | L6 | `mailbox_drop_panic` (5 placements x 1/2/256), `native_mailbox_drop`, lifecycle traces | A B T N G TG | covered | not simulated (subprocesses) | | L7 | `shutdown_publication`; `*_contract::stop_window_*`; `sim_ordering::stop_discards_*`; `src/tasks/actor/monitor_cleanup_regressions.rs::observer_cancels_its_monitors_before_publishing_its_exit` (wake order on a current-thread runtime) | A B T N G TG, Sim | partial → covered | monitor ordering only on A (the cleanup is shared with threads mode) | +| L8 (v0.9) | `Start.tla` (8 configurations, 40 controls); start traces (16 scenarios, 15 recorded-trace controls); `start_contract::*::{init_outcomes_*, names_*}`, `tasks_mode::{start_link_through_dead_context_is_refused, stuck_started_reports_pending_*}`; `link_fence_loom::start_edge_commit_races_parent_exit_begin` | A T N G TG, Model, Loom | new: covered | the sim and fuzz layers of design 16.3-16.4 | +| L9 (v0.9) | `Start!KillBeforeDIsKilled`, `Start!IgnoredExitsNormal`, `Start!ReadyReturnCommitsReady` (control `AdminStopFailsInit`: a held administrative stop waits for the start); traces `kill_and_admin_stop_during_started`, `parent_exit_during_child_started`; `start_contract::*::init_outcomes_*` (every row but 1-3 per mode), `trapping_child_with_parent_exit_*` | A N, Model | new: covered | rows 2-3 not in threads mode tests | +| L10 (v0.9) | `Start!ContinueFirst`; trace `continuation_precedes_queued_messages` and its control; `start_contract::*::continuation_*` | A N, Model | new: covered | | | X1 | `actor_scenarios` monitor races; `exit_signal_matrix`; Fz | A B T N, Sim | covered | owned by #240 | | X2 | unit `demonitor_*`, `closed_monitor_table_*`; `leaks` | A N | covered | demonitor checks use a bounded window | | X3 | unit link tests; `exit_signal_matrix`; Fz | A N, Sim | covered | owned by #240; threads lacks the Normal and chain variants (after #240) | | X4 | `kill_links` | A N | covered | owned by #240 | -| X5 | unit link tests; `link_regression_tests` | A N | covered | owned by #240 | +| X5 | unit link tests; `link_regression_tests`; v0.9: `Start!LinkBeforeStarted`, `Start!EdgeOnlyToRunningParent`, the trace control "edge committed after started() began", `link_fence_loom::start_edge_commit_races_parent_exit_begin`, `start_contract::*::start_link_through_dead_context_is_refused`/`names_and_refusals` | A N, Model, Loom | covered | owned by #240 | +| X8 (v0.9) | `Start!{FailureFlushed, NoticeOnce, NoLostNotice, TimeoutContained, AbandonedNoticeDelivered}`; traces with diverted, dropped and handed-on notices and their controls; `start_contract::*::failed_start_notices_*`, `tasks_mode::{abandoned_start_monitor_*, init_timeout_never_reaches_the_parent}` | A N, Model | new: covered | | | X6 | `link_regression_tests` | A | covered | owned by #240 | | X7 | `semantics_gaps::*::exit_and_down_follow_*`; `sim_ordering::exit_and_down_follow_the_last_message_*`; `*_partial::linked_stop_waits_for_the_running_handler_*`; Fz | A B T N G TG, Sim (PCT, DFS) | partial → covered | | | V1 | `supervisor_policies` oracle; `supervision::restart_modes_*` (real `Normal`, `Shutdown`, `Shutdown` with a payload and panic exits); `shutdown_payload::*_restart_table_*` (payload on G and TG); `Supervision!RestartFollowsPolicy`, `TemporaryStartsOnce` (`MCSupervisionTemporary`) | G, Sim, Model | covered | | @@ -365,9 +369,12 @@ giving collateral children their own backoff fails | S4 | `src/system.rs::coordinator_fault_*`; failpoints | G, Model | covered | not simulated (failpoints) | | S5 | `execution_groups`, `supervision`, `src/registry.rs` tests; Fz | G, Sim | covered | | | S6 | `execution_groups::ambient_actor_*`; `tasks_contract::unmanaged_tasks_and_threads_*` | G | partial → covered | | +| S7 (v0.9) | `Start!InheritedPlacement`, `CapacityRespected`; trace `inherited_thread_group_capacity` and its control; `start_contract::tasks_mode::{link_and_monitor_inherit_*, inherited_start_into_full_thread_group_*}` | G TG, Model | new: covered | pinned shards and the Blocking/Thread backends are not asserted (no backend query) | +| S9 (v0.9) | `Start!{NameUnique, NameKeptWhileLive, NoOverlapOnReuse, NameFreeBeforeObservation}`; trace `named_start_clash_and_release` and its controls; `start_contract::*::names_*` | A N, Model | new: covered | | | T1 | `semantics_gaps::tasks::send_after_fires_once_*` (paused time); `resource_cleanup`; `cross_backend::timers_match` | A B T N G TG P | covered | | | T2 | `semantics_gaps::tasks::send_interval_is_fixed_delay_*` (paused time) | A | covered | threads by construction | | T3 | `semantics_gaps::tasks::timers_are_not_carried_*`; `tasks_contract::managed_timers_run_on_the_group_*` | G | partial → covered | `send_message_on` in threads | +| T5 (v0.9) | `Start!TimeoutNeverKillsReady`; traces `init_timeout_waiting`, `abandoned_timeout_delivers_killed` and the control "init timer wins after a ready commitment"; `start_contract::tasks_mode::{init_timeout_never_reaches_the_parent, stuck_started_reports_pending_*}` | A, Model | new: covered | threads mode; the timer under the simulator's virtual time | | T4 | paused-time tests and the simulator clock; `src/threads/timer_tests.rs::threads_send_*`; `supervision::native_restart_window_expires_on_the_monotonic_clock` | A N TG, Sim | partial → covered | | | P1 | `panic_safety`; `foreign_panics`; `mailbox_drop_panic`; failpoints | A B T N G TG | covered | not simulated | | P2 | `panic-abort` CI job; `foreign_panics::panicking_panic_hook_*` | A N | covered | no observer for `Down`/`Exit` | @@ -915,7 +922,7 @@ replays from the input alone. | Part | Contents | |---|---| | Schedule | `Random` (seeded walk), `Pct { d = 1..4, k = 1..4000 }` (one PCT run) or `Canonical` (smallest ready task), all from `sim::explore` | -| Actors | spawn unmanaged (`start`), managed (`start_on` an async group of up to three systems) or `start_linked`; hooks that sleep (virtual time) or panic in `started()`, `stopped()` or the state destructor | +| Actors | spawn unmanaged (`spawn`), managed (`with_options(&group)` an async group of up to three systems) or `start_link`; hooks that sleep (virtual time) or panic in `started()`, `stopped()` or the state destructor | | Messages | `send` (by `ActorRef` or `Recipient`), requests inline or in their own task with no or any timeout; handler commands: no-op, panic, sleep, stop self with any `ExitReason`, send a `Note`, link, unlink, monitor, trap exits, register a name, stop another actor | | Signals | link, unlink, monitor, demonitor and `trap_exit` from outside; `stop`, `shutdown`, `stop_with_reason(Kill/Panic)` | | Timers and time | `send_after`, `send_interval`, cancel; advance virtual time; yield | @@ -1262,9 +1269,10 @@ restart generations and factory calls, supervisor and system shutdown order. (mapped to logical indices), cross-sender interleaving at one actor, and arrival order between distinct timer deadlines. One lifecycle difference is the intended contract, asserted by -`stop_before_first_poll_follows_managed_and_legacy_contract`: a managed actor -canceled before startup skips both hooks and drops its state; legacy unmanaged -tasks and threads actors retain their existing behavior of running both hooks. +`stop_before_first_poll_follows_managed_and_legacy_contract`: since 0.9 every +placement runs both hooks when stopped (`ChildHandle::stop`) before startup +(the stop waits for `started()`, OTP); a managed actor whose admission closes +before startup skips both and drops its state (`startup_admission`). **Cancellation contract observed:** `request`-family futures are `async fn` and enqueue nothing until first polled; after that, dropping the caller never diff --git a/docs/design/deferred-replies.md b/docs/design/deferred-replies.md index 66625905c..173ba768d 100644 --- a/docs/design/deferred-replies.md +++ b/docs/design/deferred-replies.md @@ -494,7 +494,7 @@ current turn's slot) and a job set. The order is: - `pipe*` now returns `OwnerStopping`. The run loop arms the cleanup deadline for every actor, standalone - `group.start` and `.start()` actors included: + group-placed and unmanaged (`.spawn()`) actors included: - the supervisor's `shutdown_timeout`; - the system grace; - otherwise `DEFAULT_STOP_DEADLINE` (5 s). diff --git a/docs/design/execution-groups-supervision.md b/docs/design/execution-groups-supervision.md index 44dc9ac44..9c39e8a2f 100644 --- a/docs/design/execution-groups-supervision.md +++ b/docs/design/execution-groups-supervision.md @@ -3,7 +3,7 @@ Status: **owned async/native/pinned groups and nested supervision strategies available in v0.7.0**. Updated 2026-11-09. See the [usage guide](../GUIDE.md#execution-groups-and-supervision) for actual signatures and supported limits. This document records the design contract; illustrative API shapes below are not copyable code. -v0.7.0 also includes the dedicated-runtime worker-budget correction. Ordinary handlers, protocols and `.start()` remain usable. +v0.7.0 also includes the dedicated-runtime worker-budget correction. Ordinary handlers, protocols and `.start()` (0.9: `.spawn()`) remain usable. The combined goal is straightforward: an application chooses where its actors run, and the framework can reconstruct a failed child in that same execution @@ -64,7 +64,7 @@ the method is called from another runtime or a plain native thread. Group closure rejects new registration/placement, with the helper API's documented error or cancelled/inert-handle outcome rather than an ambient-runtime fallback. -Existing unsupervised `.start()` remains ambient. Calling it inside a managed +Existing unsupervised `.start()` (0.9: `.spawn()`/`.start()`) remains ambient. Calling it inside a managed actor does not enroll the new actor in its supervision tree or ordered cleanup, even if it happens to run on the group's runtime. Treat it like unregistered application `tokio::spawn`: group teardown may abort it without an async diff --git a/docs/design/introspection.md b/docs/design/introspection.md index a9cabd16c..ee6a5e290 100644 --- a/docs/design/introspection.md +++ b/docs/design/introspection.md @@ -663,7 +663,7 @@ G23). | | Default | Override | |---|---|---| | Compiled | `console` feature, in `default` | omit it: no thread, socket, table or symbol | -| Starts | on `ActorSystem::new` (automatic) or `console::start(cfg) -> ConsoleGuard` (explicit, for bare-actor programs); never on a bare `.start()` | `SPAWNED_CONSOLE=off` or `ConsoleConfig { enabled: false }` | +| Starts | on `ActorSystem::new` (automatic) or `console::start(cfg) -> ConsoleGuard` (explicit, for bare-actor programs); never on a bare `.spawn()` | `SPAWNED_CONSOLE=off` or `ConsoleConfig { enabled: false }` | | Mode | read-only | `SPAWNED_CONSOLE_ALLOW=kill,terminate,trace` (each separately) or `ConsoleConfig::allow(..)` | | Directory | `$XDG_RUNTIME_DIR/spawned`, else `/tmp/spawned-` | `SPAWNED_CONSOLE_DIR=/dir` or `ConsoleConfig::dir(..)` | | Failure | one `WARN spawned::console` (`ENOENT`, `EROFS`, `EACCES`, `EPERM`/seccomp, untrusted directory) and the program continues | — | @@ -949,7 +949,7 @@ Frozen before release: | Flight recorder in v1 | Deferred to 0.10 | Same ring as full `sys:trace`; design once. | | Per-turn `tracing` span | Deferred to rank 28 | Not free even when filtered. | | Console on a system's control runtime | Rejected (revision 2) | A system may end while others live; console work must not delay supervision. | -| Console started by bare `.start()` | Rejected (revision 2) | Hidden threads and sockets in library calls, seccomp failures; bare programs opt in with `console::start`. | +| Console started by bare `.spawn()` | Rejected (revision 2) | Hidden threads and sockets in library calls, seccomp failures; bare programs opt in with `console::start`. | | Changing the umask to create the socket | Rejected (revision 2) | Process-global, races other threads, needs `unsafe`. | | Pid-only socket names, pid-based stale cleanup | Rejected (revision 2) | Pid namespaces and reuse; nonce plus lock instead. | | `atexit` cleanup | Rejected | Needs `unsafe`; the lock makes leftovers identifiable. | @@ -1155,7 +1155,7 @@ Placements: A `Backend::Async`, B `Blocking`, T `Thread`, N threads mode, G | Test | Fails if | |---|---| -| `starts_with_actor_system_not_with_bare_start` | a socket appears for bare `.start()` | +| `starts_with_actor_system_not_with_bare_start` | a socket appears for bare `.spawn()` | | `console_start_guard_for_bare_programs` | no socket with the guard; left after its drop | | `refcount_binds_and_unbinds_and_binds_again` (two systems, then a third later) | socket missing while a system lives; not removed at 1→0; no rebind | | `forked_child_never_unlinks_parent_socket` (`drop_subprocess`-style fork) | the parent's files removed | @@ -1269,7 +1269,7 @@ an exhaustive `match` on `Lifecycle` or `Unavailable` without a wildcard. | Status via `InTurnEnvelope` (panic ends actor) | `callback_panic_*` | | Abandoned status callback still runs | `status_skipped_when_caller_gave_up` | | `tree()` holds the parent lock while expanding | `tree_holds_no_lock_across_children` | -| Console binds on bare `.start()` / unlinks without the pid check | `starts_with_actor_system_*`, `forked_child_*` | +| Console binds on bare `.spawn()` / unlinks without the pid check | `starts_with_actor_system_*`, `forked_child_*` | | Stale cleanup without the lock | `stale_pair_removed_only_*` | | Blocking write without a deadline | `slow_reader_*` | | Unordered shard iteration | `reads_do_not_change_the_tape` | @@ -1445,7 +1445,7 @@ Each finding was treated as a claim and checked against the design and the | Codex 11 | A kill-capable `check_now()` cannot be tape-neutral | Accepted | Neutrality only for reads and report-only checks; kill-capable checks promise identical replay (`kill_check_replays`) | | Codex 12, agy P1-4 | Changing the umask is unsafe and racy; leaf-only checks | Accepted | Umask never touched; private `0700` directory, bind inside, `0600` as defence in depth, effective-uid peer check; ancestor and symlink checks; root refused unless the process is root | | Codex 13, agy P1-5 | Pid-based stale cleanup and system-owned lifetime are unsafe | Accepted | Instance nonce, a held `File::lock` as liveness, removal only with the lock and a `(dev, ino)` match, the server removes only its own files; refcounted lifetime independent of any one system | -| agy P0-2 | Console on bare `.start()`, `atexit`, multiple systems, `fork()` | Accepted (coordinator decision 1) | Automatic only on `ActorSystem::new`; `console::start` guard for bare programs; refcount 0→1 bind, 1→0 unlink, rebind later; `bound_pid` check; no `atexit`, no `unsafe`; failures warn once | +| agy P0-2 | Console on bare `.spawn()`, `atexit`, multiple systems, `fork()` | Accepted (coordinator decision 1) | Automatic only on `ActorSystem::new`; `console::start` guard for bare programs; refcount 0→1 bind, 1→0 unlink, rebind later; `bound_pid` check; no `atexit`, no `unsafe`; failures warn once | | Codex 14, agy P1-6 | Console and watchdog can block the control plane | Accepted | Own console thread; per-frame 2 s write deadline; limits table (connections, requests, rows, bytes, depth, strings, rate); `state` awaited, not blocking; threads-only builds use a thread per connection; watchdog sampling separate from delivery; one outstanding probe per target | | Codex 15 | `restart` is misleading and unfenced | Accepted (coordinator decision 3) | Renamed `kill`; consequences per supervisor policy documented; targets carry system id and an expected `id@generation`; `stale_target`; `kill`, `terminate`, `trace` allowed separately | | Codex 16 | Wire and data contracts not frozen | Accepted | [Wire and data contract](#wire-and-data-contract): framing, envelope, error codes, encodings, `Unavailable` values, enums, evolution, cursors; `SystemId` in snapshots; `#[non_exhaustive]` on every public type; `serde_json` with dedicated wire structs | diff --git a/docs/design/otp-exit-signals.md b/docs/design/otp-exit-signals.md index e9c22e2ae..cc55bcc9c 100644 --- a/docs/design/otp-exit-signals.md +++ b/docs/design/otp-exit-signals.md @@ -358,12 +358,12 @@ that traps exits receives its parent's death in `exit_received`. - **Parent identity.** Each actor instance has an immutable parent, fixed at spawn before the link is registered and before `started()`: - - `start_linked(parent_ctx)`: the actor whose `Context` was passed - (normally the caller); + - `start_link(parent_ctx)` (0.8: `start_linked`): the actor whose + `Context` was passed (normally the caller); - a supervised child: its supervisor (each replacement generation gets the same supervisor parent); - a standalone actor started on an `ActorSystem` or group: the system; - - an unmanaged `.start()`: none. + - an unmanaged `.spawn()`/`.start()` (0.9): none. Supervisors and systems are not actors; their signals use an internal entry point on the actor's scope that marks them as parent signals. @@ -378,9 +378,10 @@ that traps exits receives its parent's death in `exit_received`. This includes `Normal`: a trapping `start_linked` child ends when its parent ends normally, as a gen_server does. Users who want to observe the parent's exit instead use `ctx.link`, not `start_linked`. -- **A parent already past D.** `start_linked` with such a parent registers the - link against a gone actor (E4): the child receives a parent exit with - `NoProc` (trapping: queued; non-trapping: begins exit with `NoProc`). +- **A parent already past D.** In 0.8, `start_linked` with such a parent + registered the link against a gone actor (E4): the child received a parent + exit with `NoProc`. Since 0.9 `start_link` refuses it (`ParentExited`, + unified start X5): the child is never created. - **No early commitment.** Queuing a parent exit commits no reason and cancels neither the child's token nor its helpers. Supervisor or system shutdown still closes admission for the whole managed tree at once (semantics S1), so @@ -397,7 +398,7 @@ that traps exits receives its parent's death in `exit_received`. | rollback of a failed start (S1, S2) | same | `kill` | same | | a child ends on its own, by an administrative stop or by a kill | `shutdown_timeout` from exit begin | cleanup stuck (V3, unchanged) | none: it is already exiting, and a kill would relabel its reason | | `ActorSystem::shutdown(grace)` / group shutdown, standalone actor | until its turn ends or `grace` expires (S2, reverse order, parent Shutdown) | `kill` to every remaining actor at `grace` | report incomplete; pending work stays owned by the system (S3, unchanged) | - | unmanaged `.start()` actor | none | none | none | + | unmanaged `.spawn()`/`.start()` actor | none | none | none | A supervisor `shutdown(deadline)` keeps `deadline` as the report deadline only: at it, siblings not yet signalled get their Shutdown and the report is diff --git a/docs/design/semantics.md b/docs/design/semantics.md index 6968d3171..6e467e35b 100644 --- a/docs/design/semantics.md +++ b/docs/design/semantics.md @@ -241,18 +241,28 @@ Checked by: [`deferred_jobs`][deferred_jobs]. ## 4. Lifecycle **L1. Hook order.** `started()` runs once before any handler; `stopped()` runs -at most once, after the last handler. No handler runs after `stopped()` begins. +at most once, after the last handler, and only after a ready startup +commitment: never after a failed or ignored start, including a `ctx.stop*` +called in `started()` (OTP skips `terminate/2` when `init/1` fails; v0.9). +No handler runs after `stopped()` begins. Checked by: [`sim_ordering::turns_and_hooks_never_overlap_{pct,dfs}`][sim_ordering], [`semantics_gaps`][semantics_gaps]`::{native,tasks}_contract::hooks_and_turns_never_overlap_under_concurrent_requests_and_stop` (every placement), [`cross_backend::stop_before_first_poll_follows_managed_and_legacy_contract`][cross_backend], -the fuzz M1/L1/L5 oracle. - -**L2. Managed readiness.** For actors admitted with `start_on` or by a -supervisor, readiness commits after `started()` returns while admission is -open; ordinary handlers run only after it. `ready()` reports that historical -barrier, not current liveness. If admission closes before `started()` begins, -neither hook runs and the state is dropped (its `Drop` runs). Unmanaged -`.start()` and threads actors stopped that early run both hooks. +the fuzz M1/L1/L5 oracle; +[`start_contract`][start_contract]`::{tasks,threads}_mode::init_outcomes_answer_after_publication_without_stopped`, +`src/{tasks,threads}/actor/early_stop_regression.rs::stop_in_started_fails_the_start_without_stopped`. + +**L2. Readiness.** Every instance, on every placement and verb, has a +startup result: the startup cell's `Ready`, committed by the child at the end +of `started()` while its admission is open, before any continuation or +handler. It is final: a later death is an ordinary death. `ready()` and the +waiting starts read the same committed value; it is a historical barrier, +not current liveness. If a managed actor's admission closes before +`started()` begins (group or system shutdown), neither hook runs and the +state is dropped. An administrative stop that arrives before or during +`started()` waits for it, on every placement (OTP `gen_server:stop`, a +system message handled after `init/1`): the start commits ready, then the +actor exits with the stop's reason through `stopped()` (v0.9, L9). Checked by: [`startup_admission`][startup_admission], [`cross_backend::stop_before_first_poll_follows_managed_and_legacy_contract`][cross_backend], [`execution_groups::failed_startup_discards_queued_payloads_and_reply_channels_without_dispatch`][execution_groups], @@ -320,7 +330,7 @@ after a kill, an untrapped exit signal or link notice, or a runtime drop (see X3, X4); one that already began finishes. Checked by: `src/{tasks,threads}/actor/tests.rs::panic_in_started_stops_actor`, `panic_in_handler_stops_actor`, `panic_in_stopped_still_completes`, -`panic_during_early_stop_is_reported`; [`panic_safety`][panic_safety], +`stop_in_started_fails_the_start_without_stopped`; [`panic_safety`][panic_safety], [`foreign_panics::panicking_subscriber_keeps_handler_reason_and_runs_stopped`][foreign_panics], [`semantics_gaps`][semantics_gaps]`::tasks_contract::panics_in_hooks_follow_the_contract_on_every_placement` (reason prefix, handled count and `stopped()` calls); @@ -354,6 +364,52 @@ outgoing monitors cancelled before publication: (the cleanup is shared with threads mode); the fuzz L7 probe also checks that an exited observer's waiters are released while the target lives on. +**L8. Start (v0.9).** Stage 1 runs in the start call: the options are +validated and the instance prepared privately, then one commit, under the lock +order group admission, the parent's link or monitor table, the parent's exit +request, the registry, applies the group slot, the parent's edge and the name, +all or nothing. A refusal (`Closed`, `Capacity`, `AlreadyStarted`, +`ParentExited`, `InvalidOptions`) is never observable: no hook runs, no edge, +name or slot remains. `spawn` does not wait; `start`, `start_link` and +`start_monitor` answer `Started` at once on a ready commitment, and an +`Ignored` or error answer only after the instance's publication (its state +dropped, its notices disposed of, its name free) with the final reason, or +after its cleanup deadline with `pending: Some(handle)`. Dropping a `Starting` +abandons the wait, nothing else. +Checked by: `Start!AbortInvisible`, `Start!ResultMatchesCommit`, +`Start!ReadyIsFinal`, `Start!FailureAfterPublication`, +`Start!PendingOnlyAfterDeadline`, `Start!CommitImmutable` (TLC, 8 +configurations, 40 negative controls); start traces +([`start_trace`][start_trace], `TraceStart.tla`, 15 recorded-trace negative +controls); [`start_contract`][start_contract]`::{tasks,threads}_mode::{init_outcomes_answer_after_publication_without_stopped, names_and_refusals}`, +`tasks_mode::{start_link_through_dead_context_is_refused, ready_then_immediate_crash_with_unpolled_waiter_returns_started_and_delivers_down, stuck_started_reports_pending_after_the_timeout}`; +[`link_fence_loom::start_edge_commit_races_parent_exit_begin`][link_fence_loom]. + +**L9. Init resolution (v0.9).** When `started()` returns, the first +matching row decides the start and the requested reason, under the child's +exit-request and cell locks: (1) the init timer committed: `Killed`, +`InitTimeout`; (2) a kill arrived: `Killed`; (3) a +held non-`Normal` signal to a non-trapping child, the first in arrival order; +(4) `started()` panicked; (5) `ctx.stop*` in `started()`; (6) `Init::stop(r)`; +(7) `Err(e)`/`Init::error(e)`: `Normal`, `InitFailed`; (8) `Init::ignore()`: +`Normal`, `Ignored`; (9) ready, but admission closed: `Shutdown`; (10) ready. +A non-ready decision fixes the reason for good: a later kill does nothing. +Held signals of a ready child resolve as in phase 0 (E2, E3). No failed row +runs `stopped()`. +Checked by: `Start!KillBeforeDIsKilled`, `Start!IgnoredExitsNormal`, `Start!ReadyReturnCommitsReady` (control `AdminStopFailsInit`; TLC); +start traces `kill_and_admin_stop_during_started`, +`parent_exit_during_child_started`, `link_stop_in_started_ends_nontrapping_parent`; +[`start_contract`][start_contract]`::{tasks,threads}_mode::init_outcomes_answer_after_publication_without_stopped`, +`tasks_mode::trapping_child_with_parent_exit_in_started_commits_ready_then_ends`. + +**L10. Continuation (v0.9).** `Init::continue_with(m)` handles `m` as the +first turn after the readiness commit, before every mailbox entry (messages +sent during `started()`, signals queued for a trapping child). It is not a +barrier against termination; a stop observed before it begins discards it. +Checked by: `Start!ContinueFirst` (TLC); start trace +`continuation_precedes_queued_messages` and its negative control; +[`start_contract`][start_contract]`::{tasks_mode::continuation_runs_before_messages_sent_during_started, threads_mode::continuation_runs_first}`. + ## 5. Links, monitors and trapped exits **X1. Down exactly once.** `ctx.monitor(&handle)` (requires `Handler`) @@ -408,7 +464,7 @@ read when it arrives: | `Kill` signal (`kill()`, `exit(Kill)`) | ends `Killed` | ends `Killed` | | `Normal` | nothing | `exit_received(Exit { from, reason: Normal })` | | any other reason (a `Kill` notice included) | ends with that reason after its current handler, without `stopped()`, queue discarded; propagates further | `exit_received(Exit { from, reason })` | -| from the receiver's parent (`start_linked` caller, supervisor, system) | as above | a parent exit is queued: when dispatched the actor exits with that reason through `stopped()`; `exit_received` never sees it | +| from the receiver's parent (`start_link` caller, supervisor, system) | as above | a parent exit is queued: when dispatched the actor exits with that reason through `stopped()`; `exit_received` never sees it | Propagation is transitive through non-trapping actors. Links work across modes. Signals arriving during `started()` take effect when it returns, with @@ -450,8 +506,14 @@ gets exactly one notice, the real reason or `NoProc`. Repeating a link `unlink` and `demonitor` are fences: once they return, no notice of that edge takes effect; an `Exit` already queued stays, a `Down` already queued is flushed (X2). Each edge has a generation, so a stale -notice never applies to a later link. `start_linked` registers the link -before the child's `started()`. +notice never applies to a later link. `start_link` and `start_monitor` +commit their edge before the child's `started()`, on every placement, in the +start's commit; a start through a parent whose exit began is refused +(`ParentExited`, v0.9). A waiting parent whose exit a kill or an untrapped +signal begins abandons its pending start waits (`ParentExited`), so it does +not wait for the child's `started()` (OTP: the caller dies at once; +`parent_killed_while_awaiting_start_link_abandons_the_wait`, both modes, +recorded and validated as a `Start` trace). Checked by: `Lifecycle!NoProcIffLate`, `Lifecycle!NoEdgeToDead`, `Lifecycle!CancelFenced` (TLC, `MCLifecycleFence`); [`link_fence_loom`][link_fence_loom] (the production link table and monitor fence under Loom: `unlink_fences_a_death_notice`, `link_racing_death_gets_exactly_one_notice`, @@ -463,9 +525,31 @@ fence under Loom: `unlink_fences_a_death_notice`, `link_racing_death_gets_exactl [`otp_exit_signals::tasks_mode::registrations_after_death_report_noproc`][otp_exit_signals]; `src/tasks/actor/tests.rs::duplicate_link_is_idempotent`, `unlink_prevents_signal_delivery`; `src/native_link_regression_tests.rs::start_linked_registers_before_threads_started`; +`Start!LinkBeforeStarted`, `Start!EdgeOnlyToRunningParent` (TLC) and the start +trace control "edge committed after started() began"; +[`link_fence_loom::start_edge_commit_races_parent_exit_begin`][link_fence_loom]; lifecycle traces `late_registrations_get_noproc`, `unlink_fences_a_notice_taken_at_death`, `demonitor_fences_a_down`. +**X8. Startup notices (v0.9).** A start edge's notice is disposed of at the +child's publication by its startup commitment and its waiter: after a ready +commitment, or for an abandoned or absent waiter, it is ordinary; while the +waiter waits on a failed or ignored start, a trapping parent's `Exit` and a +monitor's `Down` are diverted to it (its error consumes them), a +non-trapping parent is still ended by a non-`Normal` reason (a `Normal` one +is dropped); a timed-out start's notice is diverted for both trap settings, +so an init timeout never reaches a waiting parent. An abandoned wait hands a +diverted notice on exactly once. No notice is lost or delivered twice. +Checked by: `Start!FailureFlushed`, `Start!NoticeOnce`, `Start!NoLostNotice`, +`Start!TimeoutContained`, `Start!AbandonedNoticeDelivered`, +`Start!FailureEndsNonTrappingParent`, `Start!NormalFailureSparesParent` +(TLC); start traces `monitor_error_consumes_down`, +`link_panic_in_started_trapping_parent`, +`abandon_after_publication_hands_on_exit`, `abandoned_timeout_delivers_killed`, +`init_timeout_waiting` and their negative controls; +[`start_contract`][start_contract]`::{tasks,threads}_mode::failed_start_notices_follow_the_disposition_table`, +`tasks_mode::{abandoned_start_monitor_still_delivers_down, init_timeout_never_reaches_the_parent}`. + **X6. Runtime teardown still notifies.** An actor whose future or thread is dropped by its runtime, polled or never polled, ends `Killed` in both modes (unless a non-`Normal` reason was requested), without `stopped()`, and still @@ -692,12 +776,36 @@ Checked by: [`execution_groups::old_scope_cleanup_cannot_remove_a_replacement_re [`supervision::owned_registration_is_released_before_the_replacement_registers`][supervision], `src/registry.rs::old_guard_preserves_new_registration_even_with_same_actor_identity`. -**S6. Unmanaged work is not owned.** `.start()`, `tokio::spawn` and raw -threads, even inside a managed handler, are not joined by shutdown and can be -aborted at group teardown without `stopped()`. +**S6. Unmanaged work is not owned.** `spawn`/`start` without a placement, +`tokio::spawn` and raw threads, even inside a managed handler, are not joined +by shutdown and can be aborted at group teardown without `stopped()`. Checked by: [`execution_groups::ambient_actor_started_inside_group_is_unmanaged_and_aborted_at_teardown`][execution_groups], [`semantics_gaps`][semantics_gaps]`::tasks_contract::unmanaged_tasks_and_threads_are_not_joined_by_shutdown`. +**S7. Placement inheritance (v0.9).** A `start_link` or `start_monitor` +child is placed in its parent's placement (its group and pinned shard, or its +backend) unless the start names one; `spawn` and `start` never inherit +(`placement_of(&ctx)` opts in). A closed or full inherited group fails the +start (`Closed`, `Capacity`), with no fallback. Scope ownership of linked +children (design S8) is not implemented yet. +Checked by: `Start!InheritedPlacement`, `Start!AdmittedOnlyOpen`, +`Start!CapacityRespected` (TLC); start trace +`inherited_thread_group_capacity` and its control "linked child not in its +parent's group"; [`start_contract`][start_contract]`::tasks_mode::{link_and_monitor_inherit_the_parents_group, inherited_start_into_full_thread_group_fails_capacity}`. + +**S9. Start-time names (v0.9).** `StartOptions::name(Name)` registers the +name in the start's commit (a clash is `AlreadyStarted` with the holder's +handle, `None` for a non-actor value; nothing is created), holds it until the +instance's publication and releases it before any notice, `wait_exit` or +answer, so a retry after a failed start never overlaps the old instance. A +group slot is released when the instance's scope completes (after +publication and the owned-work join). +Checked by: `Start!NameUnique`, `Start!NameKeptWhileLive`, +`Start!NoOverlapOnReuse`, `Start!NameFreeBeforeObservation`, +`Start!SlotHeldUntilReleased` (TLC); start trace +`named_start_clash_and_release` and its negative controls; +[`start_contract`][start_contract]`::{tasks_mode::names_are_committed_and_released_atomically, threads_mode::names_and_refusals}`. + ## 8. Timers and time **T1. `send_after`.** Sends its message at most once, no earlier than the delay, @@ -744,6 +852,18 @@ and the simulator's virtual clock for tasks; `src/threads/timer_tests.rs::thread `threads_send_interval_is_fixed_delay_while_the_actor_stalls` and [`supervision::native_restart_window_expires_on_the_monotonic_clock`][supervision] (`thread_group`) for threads. +**T5. Init timeout (v0.9).** `StartOptions::init_timeout(d)` arms a +framework-owned timer at the start's commit. At expiry it commits `TimedOut` +only if nothing is committed (it loses to a ready commitment) and kills the +instance; the kill is applied when `started()` returns (nothing is +interrupted): a `started()` that never returns keeps its name and slot, and +the waiter answers `InitTimeout { pending: Some(handle) }` at the cleanup +deadline. Default `Infinity`, as OTP. +Checked by: `Start!TimeoutNeverKillsReady`, `Start!CommitImmutable` (TLC); +start traces `init_timeout_waiting`, `abandoned_timeout_delivers_killed` and +the control "init timer wins after a ready commitment"; +[`start_contract`][start_contract]`::tasks_mode::{init_timeout_never_reaches_the_parent, stuck_started_reports_pending_after_the_timeout}`. + ## 9. Panics **P1. Where panics are caught.** With unwinding, these are contained and turned @@ -878,6 +998,8 @@ remaining differences of the exit-signal design are listed in See the [ranked backlog](../../ROADMAP.md#ranked-backlog). +[start_contract]: ../../concurrency/tests/start_contract.rs +[start_trace]: ../../concurrency/tests/start_trace.rs [deferred_replies]: ../../concurrency/tests/deferred_replies.rs [deadline_service]: ../../concurrency/tests/deadline_service.rs [deferred_jobs]: ../../concurrency/tests/deferred_jobs.rs diff --git a/docs/design/unified-start-client-mapping.md b/docs/design/unified-start-client-mapping.md index 9f8011d46..527a057e2 100644 --- a/docs/design/unified-start-client-mapping.md +++ b/docs/design/unified-start-client-mapping.md @@ -121,6 +121,84 @@ Unchanged: - `#[actor(max_native_jobs = ..)]` (`crates/net/p2p/src/lib.rs:1572`): an `Actor` constant, unchanged. +## Implementation revision (0.9, re-checked against the canary sources) + +Re-run against the sources the 0.8.0 canary built: ethrex `bc13c9d28` +(`spawned-v0.8`, the canary's `ethrex_candidate`) and ethlambda `0300c22f`. +The mechanical rewrite above still holds; line numbers moved, and three +findings change the mapping: + +1. **One behaviour change in ethrex (was "none").** + `crates/networking/p2p/rlpx/connection/server.rs:553`, `PeerConnectionServer::started`: + when `initialize_connection` fails, it sets + `self.state = ConnectionState::Established(..)` and calls `ctx.stop()`, + relying on `stopped()` (`:644`) to remove the peer from the peer table and + the tx broadcaster and to run `established_state.teardown()`. In 0.9 a + `ctx.stop()` in `started()` is a failed start and `stopped()` does not run + (as OTP's `terminate/2` after a failed `init/1`): `ctx.stop()` inside + `started()` is now the same as returning `Init::stop(Normal)`, so that + cleanup would be skipped. **Edit:** move the body of `stopped()` into a + helper (say `async fn release(&mut self)`), call it in that failure + branch, and return the failure from `started()` instead of calling + `ctx.stop()`: `-> Result<(), PeerConnectionError>` (or + `Init::error(reason)` / `Init::stop(..)`). The two other `ctx.stop()` + calls in `started()` leave `HandshakeFailed`, for which `stopped()` does + nothing: unaffected, but they read better as `return Init::stop(..)`. + (An administrative `ChildHandle::stop` that arrives during the handshake + still waits for `started()` and runs `stopped()`, as in 0.8 and OTP.) +2. **A site the mapping missed (ethrex).** `crates/l2/sequencer/proof_coordinator.rs:98`: + `AssignmentTracker::new(..).start()` inside `ProofCoordinator::new` → + `.spawn()`. +3. **ethlambda `bin/ethlambda/src/http_server.rs:74`** (`#[started] + start_serving`) calls `ctx.stop()` when `spawn_helper` fails; its + `stopped()` only cancels a token whose server was never started, and the + supervisor already treated it as a failed start in 0.8: no change needed + (optional: return `Err(err)` to report `InitFailed`). + +Exact edits (ethrex 38, ethlambda 14): + +| Client | File:line | 0.8 | 0.9 | +|---|---|---|---| +| ethrex | `crates/l2/sequencer/native_rollup/l1_watcher.rs:334` | `watcher.start()` | `watcher.spawn()` | +| ethrex | `crates/l2/sequencer/native_rollup/l1_advancer.rs:488` | `advancer.start()` | `advancer.spawn()` | +| ethrex | `crates/l2/sequencer/native_rollup/block_producer.rs:504` | `producer.start()` | `producer.spawn()` | +| ethrex | `crates/l2/sequencer/l1_watcher.rs:194` | `Ok(state.start())` | `Ok(state.spawn())` | +| ethrex | `crates/l2/sequencer/state_updater.rs:110` | `state_updater.start()` | `state_updater.spawn()` | +| ethrex | `crates/l2/sequencer/proof_coordinator.rs:98` | `AssignmentTracker::new(..).start()` | `.spawn()` (new) | +| ethrex | `crates/l2/sequencer/proof_coordinator.rs:111` | `state.start()` | `state.spawn()` | +| ethrex | `crates/l2/sequencer/proof_coordinator.rs:438` | `connection_handler.start()` | `connection_handler.spawn()` | +| ethrex | `crates/l2/sequencer/l1_committer.rs:253` | `state.start_with_backend(Backend::Blocking)` | `state.with_options(Backend::Blocking).spawn()?` | +| ethrex | `crates/l2/sequencer/block_producer.rs:350` | `block_producer.start_with_backend(Backend::Blocking)` | `block_producer.with_options(Backend::Blocking).spawn()?` | +| ethrex | `crates/l2/sequencer/metrics.rs:69` | `Ok(metrics.start())` | `Ok(metrics.spawn())` | +| ethrex | `crates/l2/sequencer/l1_proof_sender.rs:176` | `state.start()` | `state.spawn()` | +| ethrex | `crates/l2/based/block_fetcher.rs:337` | `state.start()` | `state.spawn()` | +| ethrex | `crates/networking/rpc/subscription_manager.rs:173` | `Self::new(store).start()` | `.spawn()` | +| ethrex | `crates/networking/rpc/subscription_manager.rs:193` (+ `:40`, `:192` signature) | `Self::new(store).start_on(group)` returning `SystemError` | `Self::new(store).with_options(group).spawn()` returning `StartError` | +| ethrex | `crates/networking/p2p/tx_broadcaster.rs:206-207` (+ `:17`, `:496` `#[from] SystemError`) | `state.start_on(group)?` / `state.start()` | `state.with_options(group).spawn()?` / `state.spawn()`; `TxBroadcasterError::ActorSystem(#[from] StartError)` | +| ethrex | `crates/networking/p2p/rlpx/connection/server.rs:213, 228, 292, 4030` | `connection.start()` / `server.start()` | `.spawn()` | +| ethrex | `crates/networking/p2p/rlpx/connection/server.rs:553-640` | `ctx.stop()` after `initialize_connection` fails, cleanup in `stopped()` | finding 1 above | +| ethrex | `crates/networking/p2p/rlpx/initiator.rs:50` (+ `:9`, `:48`) | `.start_on(group)?` returning `SystemError` | `.with_options(group).spawn()?` returning `StartError` | +| ethrex | `crates/networking/p2p/rlpx/initiator.rs:66-67` | `state.start_with_backend(b)` / `state.start()` | `state.with_options(b).spawn().expect("executor construction failed")` / `state.spawn()` | +| ethrex | `crates/networking/p2p/peer_table.rs:616` | `.start()` | `.spawn()` | +| ethrex | `crates/networking/p2p/peer_table.rs:635` (+ `:34`, `:629`) | `.start_on(group)` returning `SystemError` | `.with_options(group).spawn()` returning `StartError` | +| ethrex | `crates/networking/p2p/discovery/server.rs:156` | `.await?.start();` | `.await?.spawn();` | +| ethrex | `crates/prover/src/prover.rs:251, 260, 268, 276` | `prover.start_with_backend(Backend::Blocking)` | `prover.with_options(Backend::Blocking).spawn()?` | +| ethrex | `tooling/monitor/src/app.rs:102` | `monitor.start();` | `monitor.spawn();` | +| ethrex | `test/tests/p2p/peer_table_timeout_tests.rs:33, 47, 59`; `shutdown_order_tests.rs:141, 169, 196` | `Mock::new()..start()` | `Mock::new()..spawn()` | +| ethrex | `cmd/ethrex/initializers.rs:1050, 1081, 1088` | callers of the E sites | error type follows (`StartError`) | +| ethlambda | `crates/net/p2p/src/lib.rs:1277` (+ `:88`, `:1193`) | `server.start_on(group)?`, `Result` | `server.with_options(group).spawn()?`, `Result` | +| ethlambda | `crates/blockchain/src/lib.rs:454` (+ `:43`, `:250`, `:358`, `:412`) | `.start_on(group)?`, `Result` | `.with_options(group).spawn()?`, `Result` | +| ethlambda | `crates/blockchain/src/lib.rs:6386, 6421, 6443` | `beacon_server(..).start()` | `.spawn()` | +| ethlambda | `crates/net/p2p/src/beacon/verdict.rs:767, 774, 1835` | `Mock::new()..start()` / `.await.start()` | `.spawn()` | +| ethlambda | `crates/net/p2p/src/beacon/column_checks.rs:133, 164`; `envelope_checks.rs:191` | `Mock::new().allow_sends().start()` | `.spawn()` | +| ethlambda | `crates/net/p2p/src/req_resp/envelope_client.rs:615, 702`; `gossipsub/handler.rs:1736` | `Mock::new()..start()` | `.spawn()` | + +Unchanged: the supervisor `start().await` calls (ethrex +`discovery/supervised.rs:151`, ethlambda `main.rs:989`, +`http_server.rs:219`), `ChildSpec::new`, and every `#[started]` method (the +macro allows the `refining_impl_trait` lint for them; neither client has a +manual `impl Actor` with `started`). + ## Related work - **ethrex.** Site #18 is the per-connection actor that the dynamic diff --git a/docs/design/unified-start.md b/docs/design/unified-start.md index 5a7947466..eb08d0a9a 100644 --- a/docs/design/unified-start.md +++ b/docs/design/unified-start.md @@ -1,9 +1,88 @@ # Unified start API (0.9): design contract -Status: design in review, revision 3, not implemented. Target release 0.9. -The implementation follows after two independent reviews. It is designed -against the 0.8 API as #240 ships it: exit signals, `try_link`, `NoProc`, and -the 0.8 API-review cleanups (`send_request`, `#[non_exhaustive]` builders). +Status: revision 3, approved; **implemented in part for 0.9** (see +[Implementation status](#implementation-status-09)). It is designed against +the 0.8 API as #240 ships it: exit signals, `try_link`, `NoProc`, and the 0.8 +API-review cleanups (`send_request`, `#[non_exhaustive]` builders). + +## Implementation status (0.9) + +**Implemented** (both modes unless noted): sections 2.1-2.5 (the verbs, +`Starter`, `Starting`, `StartOutcome`, `Init` and the 2.3 table, `StartOptions`, +`Name`, `StartError`), 3 (prepare and one commit), 4 (inheritance; not 4.1), +5.1-5.5 (the startup cell, disposition at publication, abandonment, answering +with `pending`), 6 (the init timer and stuck policy), 7 (start-time names), 8 +(continuations), 10 partly (`Mock::spawn`/`spawn_with`, `TestSystem::spawn`/ +`start`/`spawn_mock`/`options`), 12 (migration), 16.2 (trace validation: +`TraceStart.tla`, all 28 event kinds, 16 recorded scenarios, 15 +recorded-trace controls). + +**Not yet implemented** (follow-up PRs): + +- 4.1, S8: scope ownership of linked children (the owner gate, the polite + `Shutdown` at the parent's publication, the scope deadline, the + generation's end). A linked child of a managed parent is admitted to the + parent's group but is not owned by its scope. +- 9: `ChildStart::Ignore`, `ChildState::Ignored`, `ChildStartError`, + `ChildSpec::options` and the absolute attempt deadline; the + `Supervision.tla`/`Cohorts.tla` changes. A supervised child's + `Init::ignore()` is a failed start, as before. +- 10: `Mock::on_started`, `start_delay`, `drop_flag`. +- 16.3-16.5: the simulator races, the fuzzer's start generator and oracles, + the planted-bug patches of 16.5. The Loom row has one of five tests + (`start_edge_commit_races_parent_exit_begin`). +- `label` sets a managed actor's diagnostic name (threads mode: the thread + name); introspection's `set_label` (#256) does not exist yet. + +**Deviations, each forced by the code or by Rust:** + +- **Slot release** (5.2 step 4, `SReleaseSlot`). A group slot is released at + the instance's scope completion, after publication and the owned-work join, + not before publication: a thread group's slot is the native worker that + still runs the job join, so releasing it earlier admits a new actor onto a + busy worker. `Start.tla` was changed to match (`ReleaseSlot` after + `Publish` and `WorkDoneC`; P's slot after `WorkDoneP`); every configuration + and control was rerun. A retry into a full thread group may therefore see + `Capacity` until the old instance's jobs are joined; the name is free at + publication as designed. +- **Lock order of the commit** (section 3, ranks 3-4). The parent's link or + monitor table is taken before its exit-request lock, the order every other + 0.8 path uses (`link/table.rs`: tables, then exit request); the design's + reverse order would deadlock against notice delivery. The commit stays + atomic: the exit check, the edge and the name are applied under all of + them. +- **Non-ready resolution and D** (2.3). The decision fixes the reason under + the exit-request lock and marks it final (a later kill or signal is a + no-op, as past D); the link table is closed by the ordinary D right after, + outside that lock, as in 0.8. Observably the same as one critical section; + the trace records D as a folded step (`SDie` with `f`). +- **`refining_impl_trait`.** `started()` returns `impl Into>`; an + impl returning `()` refines it, which rustc's warn-by-default lint reports. + `#[actor]`/`#[started]` add the allow; manual impls add it themselves + (MIGRATION.md). There is no stable alternative (no associated-type + defaults). +- **Bare `Backend::Thread` spawn.** The thread is spawned after the commit, + not parked in prepare; a thread-spawn failure panics, as 0.8's did. +- **A `Normal` failed-start notice to a waiting non-trapping parent** is + dropped at publication (the model's legitimate drop) rather than delivered + as a no-op; found by trace validation, which rejected the first recording. + +**Revision 4 (maintainer decision, after implementation): an administrative +stop during `started()` waits for it, as OTP.** Revision 3's row 3 made a +`ChildHandle::stop*` that arrives while `started()` runs a failed start +without `stopped()`. In OTP, `gen_server:stop/1` is a system message: it is +handled after `init/1` returns, so the start succeeds and `terminate/2` runs. +No Rust constraint forces a difference, so (project rule) Spawned now does +the same: the stop is held, the start commits ready (`Started`), then the +actor exits with the stop's reason through `stopped()`; a start that fails +for another reason discards it. Exit signals during `started()` (a kill, a +held non-trapping parent exit or link notice) keep their rows: they fail the +start without `stopped()`, as in OTP they kill the starting process. A +`ctx.stop*` inside `started()` stays row 5: a process cannot stop itself +mid-`init` in OTP except by returning `{stop, R}`, so it is the same as +`Init::stop(r)` (no `stopped()`). Model: `AdminStopApply`, invariant +`ReadyReturnCommitsReady` (control `AdminStopFailsInit`, revision 3's row 3); +trace control "an administrative stop during started() fails the start". Revision 2 answered both round-one reviews. Revision 3 answers both round-two reviews (Codex and agy, both "approve after the listed changes", no P0). What @@ -217,7 +296,7 @@ which reopens the window) and `KillBeforeDIsKilled` (control |---|---|---|---|---| | 1 | the init timer committed `timedout` (section 6) | `Killed` | no | `timedout` → `InitTimeout` | | 2 | a kill arrived (`kill()`, `exit(Kill)`, the S8 scope deadline) | `Killed` | no | failed → `InitExited` | -| 3 | an administrative stop (`ChildHandle::stop*`, `shutdown`) arrived, or the child does not trap and a held non-`Normal` signal (parent exit, link notice, exit signal) arrived; the first of these in arrival order wins | its reason | no | failed → `InitExited` | +| 3 | the child does not trap and a held non-`Normal` signal (parent exit, link notice, exit signal) arrived; the first in arrival order wins | its reason | no | failed → `InitExited` | | 4 | `started()` panicked | `Panic("panic in started(): ..")` | no | failed → `InitExited` | | 5 | `ctx.stop*(r)` was called in `started()` | `r` | **no** (as `{stop, r}`) | failed → `InitExited` | | 6 | `Init::stop(r)` | `r` | no | failed → `InitExited` | @@ -232,15 +311,22 @@ which reopens the window) and `KillBeforeDIsKilled` (control even a `Normal` one (E3: it ends the child when handled). The child commits ready, runs its continuation first, then handles the queued entries. This is OTP's behaviour for an `init/1` that sets `trap_exit` and - returns `{continue, C}`. Administrative stops and kills are untrappable - (rows 2-3). + returns `{continue, C}`. Kills are untrappable (row 2). +- **An administrative stop** (`ChildHandle::stop*`, a scope's stop) that + arrives during `started()` decides nothing (revision 4): it is held, and + applied once the start commits ready (row 10), which answers `Started`; + the actor then exits with the stop's reason through `stopped()`, without + dispatching its continuation or messages. A failed start discards it. This + is OTP's `gen_server:stop/1`, a system message handled after `init/1`. - **A superseded diagnostic.** When a higher row wins over a returned `Init::error(e)`, `e` is logged at debug level with the actor id. - **Cooperative deviation.** OTP applies a signal when it arrives; Spawned reads the trap flag when `started()` returns. This is 0.8's documented phase-0 deviation (R: no preemption). - **Behaviour change from 0.8.** A `ctx.stop()` inside `started()` no longer - runs `stopped()`. + runs `stopped()`: it is the same as returning `Init::stop(r)` (OTP's + `{stop, R}`; a process cannot otherwise stop itself mid-`init`). Clean up + in `started()` before returning, or hold resources in RAII guards. ### 2.4 `StartOptions` diff --git a/examples/README.md b/examples/README.md index 5cf2e8d8b..70b567074 100644 --- a/examples/README.md +++ b/examples/README.md @@ -18,6 +18,7 @@ Usage of each feature is documented in the [guide](../docs/GUIDE.md). | Stream listeners and per-message recipients | [streams_and_recipients](capabilities/src/bin/streams_and_recipients.rs) | Same example | | Shared protocol contracts and downstream handwritten fakes | [contract library](protocol_contract/src/lib.rs), [consumer test](capabilities/tests/cross_crate_protocol.rs) | Same test covers native actors | | Lifecycle hooks (`started`/`stopped`) and `ctx.stop()` | [bank](bank), [updater](updater), [signal_test](signal_test) | [bank_threads](bank_threads), [updater_threads](updater_threads), [signal_test_threads](signal_test_threads) | +| Unified start (`spawn`, `start`, `start_link`, `start_monitor`, fallible `started()`, `Init::continue_with`), v0.9 | [start_contract tests](../concurrency/tests/start_contract.rs) | [native_lifecycle](capabilities/src/bin/native_lifecycle.rs) | | Timers (`send_after`, `send_interval`) | [updater](updater), [signal_test](signal_test), [blocking_genserver](blocking_genserver) | [updater_threads](updater_threads), [signal_test_threads](signal_test_threads) | | `send_message_on` and Ctrl-C handling | [signal_test](signal_test) | [signal_test_threads](signal_test_threads) | | Backend selection (`Async`, `Blocking`, `Thread`) | [blocking_genserver](blocking_genserver), [signal_test](signal_test) | n/a | @@ -26,7 +27,7 @@ Usage of each feature is documented in the [guide](../docs/GUIDE.md). | Global registry (`registry::register`/`whereis`) | [service_discovery](service_discovery), [registrations and fakes](capabilities/src/bin/registrations_and_fakes.rs) | [registrations and fakes](capabilities/src/bin/registrations_and_fakes.rs), [thread_only](thread_only/README.md) | | Exit reasons, `join`, `wait_exit`, monitors, links, trapped exits | [exit_reason](exit_reason) | [native_lifecycle](capabilities/src/bin/native_lifecycle.rs) | | Exit signals as in OTP (`exit`/`kill`, trapping, `NoProc`, `try_link`, parent exits, `exit_pending`), v0.8 | [exit_signals](capabilities/src/bin/exit_signals.rs) | gap (threads mode is tested in `otp_exit_signals::threads_mode`) | -| Execution groups (`ActorSystem`, `start_on`, owned helpers), v0.7 | [execution_groups](capabilities/src/bin/execution_groups.rs) | n/a | +| Execution groups (`ActorSystem`, `with_options(&group)`, owned helpers), v0.7 | [execution_groups](capabilities/src/bin/execution_groups.rs) | n/a | | Supervision, v0.7 | [supervision_recovery](capabilities/src/bin/supervision_recovery.rs) | n/a | | Fixed native actor pools / parked Linux pinned shards, v0.7 | [execution_profiles](capabilities/src/bin/execution_profiles.rs) | n/a | | Nested supervisors, OneForAll / RestForOne, v0.7 | [supervision_tree](capabilities/src/bin/supervision_tree.rs) | n/a | diff --git a/examples/bank/src/main.rs b/examples/bank/src/main.rs index 486ced605..24c74118e 100644 --- a/examples/bank/src/main.rs +++ b/examples/bank/src/main.rs @@ -8,7 +8,7 @@ use spawned_rt::tasks as rt; fn main() { rt::run(async { - let bank = Bank::new().start(); + let bank = Bank::new().spawn(); // Testing initial balance for "main" account let result = bank.withdraw("main".into(), 15).await.unwrap(); diff --git a/examples/bank_threads/src/main.rs b/examples/bank_threads/src/main.rs index b580ee591..4b1cee8f2 100644 --- a/examples/bank_threads/src/main.rs +++ b/examples/bank_threads/src/main.rs @@ -8,7 +8,7 @@ use spawned_rt::threads as rt; fn main() { rt::run(|| { - let bank = Bank::new().start(); + let bank = Bank::new().spawn(); // Testing initial balance for "main" account let result = bank.withdraw("main".into(), 15).unwrap(); diff --git a/examples/blocking_genserver/main.rs b/examples/blocking_genserver/main.rs index 6b54f990e..60a164e68 100644 --- a/examples/blocking_genserver/main.rs +++ b/examples/blocking_genserver/main.rs @@ -67,9 +67,12 @@ impl WellBehavedTask { pub fn main() { rt::run(async move { // If we change BadlyBehavedTask to Backend::Async instead, it can stop the entire program - let badboy = BadlyBehavedTask.start_with_backend(Backend::Thread); + let badboy = BadlyBehavedTask + .with_options(Backend::Thread) + .spawn() + .unwrap(); let _ = badboy.do_block(); - let goodboy = WellBehavedTask { count: 0 }.start(); + let goodboy = WellBehavedTask { count: 0 }.spawn(); let _ = goodboy.tick(); rt::sleep(Duration::from_secs(1)).await; let count = goodboy.get_count().await.unwrap(); diff --git a/examples/busy_genserver_warning/main.rs b/examples/busy_genserver_warning/main.rs index d930724b4..3e3b05bd1 100644 --- a/examples/busy_genserver_warning/main.rs +++ b/examples/busy_genserver_warning/main.rs @@ -39,7 +39,7 @@ impl BusyWorker { pub fn main() { rt::run(async move { // If we change BusyWorker to Backend::Blocking instead, it won't print the warning - let badboy = BusyWorker.start(); + let badboy = BusyWorker.spawn(); let _ = badboy.do_work(); rt::sleep(Duration::from_secs(5)).await; diff --git a/examples/capabilities/Cargo.toml b/examples/capabilities/Cargo.toml index 34053477d..843172e54 100644 --- a/examples/capabilities/Cargo.toml +++ b/examples/capabilities/Cargo.toml @@ -29,3 +29,6 @@ required-features = ["diagnostics"] # tests/deterministic_simulation.rs gates its virtual-time test on the simulator. [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(spawned_sim)'] } +# Manual `impl Actor` `started()` methods return `()` or an `Init`, refining +# the trait's `impl Into>` (`#[started]` adds this allow itself). +refining_impl_trait = { level = "allow", priority = -1 } diff --git a/examples/capabilities/src/bin/byte_budget_pipeline.rs b/examples/capabilities/src/bin/byte_budget_pipeline.rs index d53c6cf2e..b0e2633d8 100644 --- a/examples/capabilities/src/bin/byte_budget_pipeline.rs +++ b/examples/capabilities/src/bin/byte_budget_pipeline.rs @@ -65,8 +65,8 @@ impl Handler for Forwarder { #[tokio::main(flavor = "multi_thread", worker_threads = 2)] async fn main() { let budget = Arc::new(Semaphore::new(BUDGET)); - let sink = Sink(0).start(); - let forwarder = Forwarder(sink.clone()).start(); + let sink = Sink(0).spawn(); + let forwarder = Forwarder(sink.clone()).spawn(); let mut producers = Vec::new(); for _ in 0..8 { let budget = budget.clone(); diff --git a/examples/capabilities/src/bin/dedicated_thread.rs b/examples/capabilities/src/bin/dedicated_thread.rs index 80c9990aa..1f7befab9 100644 --- a/examples/capabilities/src/bin/dedicated_thread.rs +++ b/examples/capabilities/src/bin/dedicated_thread.rs @@ -40,7 +40,7 @@ impl Handler for Worker { #[tokio::main(flavor = "multi_thread", worker_threads = 2)] async fn main() { - let actor = Worker.start_with_backend(Backend::Thread); + let actor = Worker.with_options(Backend::Thread).spawn().unwrap(); let (workers, answer) = actor.request(Check).await.unwrap(); assert_eq!(workers, 1); assert_eq!(answer, 42); diff --git a/examples/capabilities/src/bin/deferred_replies.rs b/examples/capabilities/src/bin/deferred_replies.rs index 0134524a4..ae80c4444 100644 --- a/examples/capabilities/src/bin/deferred_replies.rs +++ b/examples/capabilities/src/bin/deferred_replies.rs @@ -323,11 +323,11 @@ impl Parker { #[tokio::main(flavor = "multi_thread", worker_threads = 2)] async fn main() { // 1. Delegation: the slow request does not block the router. - let backend = Backend.start(); + let backend = Backend.spawn(); let router = Router { backend: backend.clone(), } - .start(); + .spawn(); let slow = router.route(300); assert_eq!(router.ping().await.unwrap(), "pong"); assert_eq!(router.route(1).await.unwrap(), 1); @@ -338,7 +338,7 @@ async fn main() { let verifier = Verifier { verdicts: Vec::new(), } - .start(); + .spawn(); verifier .send(Verify { id: 1, @@ -364,7 +364,7 @@ async fn main() { let conn = Connection { current_requests: PendingReplies::with_capacity(2), } - .start(); + .spawn(); let a = conn .send_request( Outgoing { id: 1 }, @@ -414,7 +414,7 @@ async fn main() { free: 0, waiters: VecDeque::new(), } - .start(); + .spawn(); let first = table .send_request(WaitForPeer, spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -434,7 +434,7 @@ async fn main() { parked: Vec::new(), answer_first: 0, } - .start(); + .spawn(); let one = parker.send_request(Park, Deadline::Never).unwrap(); let two = parker.send_request(Park, Deadline::Never).unwrap(); parker.request(StopAnswering(1)).await.unwrap(); diff --git a/examples/capabilities/src/bin/execution_groups.rs b/examples/capabilities/src/bin/execution_groups.rs index 34aa0305e..3cc180322 100644 --- a/examples/capabilities/src/bin/execution_groups.rs +++ b/examples/capabilities/src/bin/execution_groups.rs @@ -38,9 +38,9 @@ async fn main() { let system = ActorSystem::new().unwrap(); let critical = system.async_group("critical", 1).unwrap(); let general = system.async_group("general", 2).unwrap(); - let worker = Worker.start_on(&general).unwrap(); - let router = Router(worker).start_on(&critical).unwrap(); - router.ready().await.unwrap(); + let worker = Worker.with_options(&general).spawn().unwrap(); + let router = Router(worker).with_options(&critical).spawn().unwrap(); + router.ready().await.unwrap().started().expect("started"); assert_eq!(router.request(Double(21)).await.unwrap(), 42); let helper = router .context() diff --git a/examples/capabilities/src/bin/execution_profiles.rs b/examples/capabilities/src/bin/execution_profiles.rs index 267671d8f..cbfc311b5 100644 --- a/examples/capabilities/src/bin/execution_profiles.rs +++ b/examples/capabilities/src/bin/execution_profiles.rs @@ -28,12 +28,12 @@ impl Handler for Worker { async fn main() { let system = ActorSystem::new().unwrap(); let native = system.thread_group("blocking-roots", 2).unwrap(); - let a = Worker.start_on(&native).unwrap(); - let b = Worker.start_on(&native).unwrap(); - a.ready().await.unwrap(); - b.ready().await.unwrap(); + let a = Worker.with_options(&native).spawn().unwrap(); + let b = Worker.with_options(&native).spawn().unwrap(); + a.ready().await.unwrap().started().expect("started"); + b.ready().await.unwrap().started().expect("started"); assert!( - Worker.start_on(&native).is_err(), + Worker.with_options(&native).spawn().is_err(), "capacity is two live scopes" ); println!("native actor: {:?}", a.request(Location).await.unwrap()); @@ -55,8 +55,11 @@ async fn main() { .collect(); let pipeline = system.pinned_group("pipeline", cpus.clone()).unwrap(); for (index, cpu) in cpus.iter().enumerate() { - let stage = Worker.start_on(&pipeline.shard(index).unwrap()).unwrap(); - stage.ready().await.unwrap(); + let stage = Worker + .with_options(&pipeline.shard(index).unwrap()) + .spawn() + .unwrap(); + stage.ready().await.unwrap().started().expect("started"); let location = stage.request(Location).await.unwrap(); assert_eq!(location.1, Some(*cpu)); println!("pinned stage {index}: {location:?}"); diff --git a/examples/capabilities/src/bin/exit_signals.rs b/examples/capabilities/src/bin/exit_signals.rs index 3b93af217..105f06cbb 100644 --- a/examples/capabilities/src/bin/exit_signals.rs +++ b/examples/capabilities/src/bin/exit_signals.rs @@ -144,7 +144,7 @@ async fn main() { // 1. exit(reason) follows OTP's exit_signal/2: Normal is ignored by a // non-trapping actor, any other reason ends it without stopped(). let seen = Seen::default(); - let plain = Worker::new(false, &seen).start(); + let plain = Worker::new(false, &seen).spawn(); plain.request(Sync).await.unwrap(); plain.child_handle().exit(ExitReason::Normal); plain.request(Sync).await.unwrap(); @@ -164,8 +164,8 @@ async fn main() { // kill is untrappable and ends it `Killed`, also without stopped(). // An administrative stop (`stop`, `shutdown`) always runs stopped(). let (observer_seen, target_seen) = (Seen::default(), Seen::default()); - let observer = Worker::new(true, &observer_seen).start(); - let target = Worker::new(true, &target_seen).start(); + let observer = Worker::new(true, &observer_seen).spawn(); + let target = Worker::new(true, &target_seen).spawn(); observer .request(Watch(target.child_handle())) .await @@ -212,7 +212,7 @@ async fn main() { Some(ExitReason::Killed) ); // A non-trapping actor that links to a dead one exits NoProc itself. - let linker = Worker::new(false, &Seen::default()).start(); + let linker = Worker::new(false, &Seen::default()).spawn(); linker.request(Watch(target.child_handle())).await.unwrap(); assert_eq!(ended(&linker.child_handle()).await, ExitReason::NoProc); observer.child_handle().stop(); diff --git a/examples/capabilities/src/bin/framework_metrics.rs b/examples/capabilities/src/bin/framework_metrics.rs index 2e1d663b5..eb75aaab2 100644 --- a/examples/capabilities/src/bin/framework_metrics.rs +++ b/examples/capabilities/src/bin/framework_metrics.rs @@ -23,7 +23,7 @@ impl Handler for Worker { #[tokio::main(flavor = "multi_thread", worker_threads = 2)] async fn main() { metrics::configure(Config::default().timing_sample_every(16)).unwrap(); - let actor = Worker.start(); + let actor = Worker.spawn(); for _ in 0..32 { actor.request(Ping).await.unwrap(); } diff --git a/examples/capabilities/src/bin/graceful_drain.rs b/examples/capabilities/src/bin/graceful_drain.rs index 139f93cc1..f8536601c 100644 --- a/examples/capabilities/src/bin/graceful_drain.rs +++ b/examples/capabilities/src/bin/graceful_drain.rs @@ -36,7 +36,7 @@ impl Handler for Worker { } #[tokio::main(flavor = "multi_thread", worker_threads = 2)] async fn main() { - let worker = Worker(0).start(); + let worker = Worker(0).spawn(); let mut producers = Vec::new(); for _ in 0..4 { let actor = worker.clone(); diff --git a/examples/capabilities/src/bin/native_lifecycle.rs b/examples/capabilities/src/bin/native_lifecycle.rs index b371eae83..60d4cf9d7 100644 --- a/examples/capabilities/src/bin/native_lifecycle.rs +++ b/examples/capabilities/src/bin/native_lifecycle.rs @@ -2,8 +2,8 @@ use spawned_concurrency::{ actor, message::Message, - threads::{Actor, ActorStart, Context, Handler}, - Down, Exit, ExitReason, + threads::{Actor, ActorStart, Context, Handler, Init}, + Down, Exit, ExitReason, StartError, }; use std::{sync::mpsc, time::Duration}; const LIMIT: Duration = Duration::from_secs(2); @@ -52,6 +52,23 @@ impl Failing { panic!("intentional linked startup failure"); } } +/// Ready, then fails in its continuation (before any message). +struct FailsAfterStart; +struct Boom; +impl Message for Boom { + type Result = (); +} +impl Actor for FailsAfterStart { + #[allow(refining_impl_trait)] + fn started(&mut self, _: &Context) -> Init { + Init::continue_with(Boom) + } +} +impl Handler for FailsAfterStart { + fn handle(&mut self, _: Boom, _: spawned_concurrency::PendingReply<'_, ()>, _: &Context) { + panic!("intentional failure after start"); + } +} struct Idle; impl Actor for Idle {} fn main() { @@ -62,19 +79,38 @@ fn main() { downs, counts: (0, 0), } - .start(); + .spawn(); observer.context().trap_exit(true); - let failed = Failing.start_linked(&observer.context()); - let monitor = observer.context().monitor(&failed.child_handle()); - let exit = exit_rx.recv_timeout(LIMIT).unwrap(); + // A failed start_link answers the error; the trapping parent's Exit is + // consumed by it (OTP's start_link flushes the 'EXIT'). + let failed = Failing.start_link(&observer.context()); + assert!(matches!( + failed, + Err(StartError::InitExited { + reason: ExitReason::Panic(_), + pending: None + }) + )); + assert_eq!(observer.request(Counts).unwrap(), (0, 0)); + // A ready child's later death is an ordinary Exit, and Down to a monitor. + let (started, monitor) = FailsAfterStart + .start_monitor(&observer.context()) + .unwrap() + .started() + .unwrap(); + observer.context().link(&started.child_handle()); let down = down_rx.recv_timeout(LIMIT).unwrap(); - assert_eq!(exit.from, failed.id()); assert_eq!(down.monitor_ref, monitor); - assert!(matches!(exit.reason, ExitReason::Panic(_))); assert!(matches!(down.reason, ExitReason::Panic(_))); + let exit = exit_rx.recv_timeout(LIMIT).unwrap(); + assert_eq!(exit.from, started.id()); + assert!(matches!( + exit.reason, + ExitReason::Panic(_) | ExitReason::NoProc + )); assert_eq!(observer.request(Counts).unwrap(), (1, 1)); - failed.join(); - let idle = Idle.start(); + started.join(); + let idle = Idle.spawn(); let watch = observer.context().monitor(&idle.child_handle()); observer.context().demonitor(watch); idle.context().stop(); @@ -82,11 +118,11 @@ fn main() { assert_eq!(observer.request(Counts).unwrap(), (1, 1)); observer.context().stop(); observer.join(); - // Without trap_exit, a linked abnormal exit stops the peer as well. - let peer = Idle.start(); - let failed = Failing.start_linked(&peer.context()); + // Without trap_exit, a failed start_link still stops the parent (the + // link kills the caller, as in OTP). + let peer = Idle.spawn(); + assert!(Failing.start_link(&peer.context()).is_err()); assert!(matches!(peer.wait_exit(), ExitReason::Panic(_))); - failed.join(); peer.join(); println!("Native monitor/link notifications preserve reasons; trapped peers keep running."); } diff --git a/examples/capabilities/src/bin/protocol_timeouts.rs b/examples/capabilities/src/bin/protocol_timeouts.rs index da788a389..b5df0581f 100644 --- a/examples/capabilities/src/bin/protocol_timeouts.rs +++ b/examples/capabilities/src/bin/protocol_timeouts.rs @@ -53,7 +53,7 @@ mod asynchronous { pub async fn run() { let (release, gate) = tokio::sync::oneshot::channel(); let calls = Arc::new(AtomicUsize::new(0)); - let actor = Worker(gate, calls.clone()).start(); + let actor = Worker(gate, calls.clone()).spawn(); // Startup is held, so neither timed-out request can complete early. assert!(actor.value().await.is_err()); assert!(actor.quick().await.is_err()); @@ -109,7 +109,7 @@ mod synchronous { pub fn run() { let (release, gate) = std::sync::mpsc::channel(); let calls = Arc::new(AtomicUsize::new(0)); - let actor = Worker(gate, calls.clone()).start(); + let actor = Worker(gate, calls.clone()).spawn(); assert!(actor.value().is_err()); assert!(actor.quick().is_err()); release.send(()).unwrap(); diff --git a/examples/capabilities/src/bin/registrations_and_fakes.rs b/examples/capabilities/src/bin/registrations_and_fakes.rs index b0ae7fd6e..814e388c6 100644 --- a/examples/capabilities/src/bin/registrations_and_fakes.rs +++ b/examples/capabilities/src/bin/registrations_and_fakes.rs @@ -99,10 +99,22 @@ async fn main() { use spawned_concurrency::threads::ActorStart as _; struct Native; impl spawned_concurrency::threads::Actor for Native {} - let native = Native.start(); - let old = registry::register_owned("example-manual", native.id(), 1u64).unwrap(); + let native = Native.spawn(); + let old = registry::register_owned( + "example-manual", + native.id(), + 1u64, + Some(native.child_handle()), + ) + .unwrap(); registry::unregister("example-manual"); - let new = registry::register_owned("example-manual", native.id(), 2u64).unwrap(); + let new = registry::register_owned( + "example-manual", + native.id(), + 2u64, + Some(native.child_handle()), + ) + .unwrap(); drop(old); assert_eq!(registry::whereis::("example-manual"), Some(2)); native.context().stop(); diff --git a/examples/capabilities/src/bin/runtime_isolation.rs b/examples/capabilities/src/bin/runtime_isolation.rs index 3dad79d33..06fad0b02 100644 --- a/examples/capabilities/src/bin/runtime_isolation.rs +++ b/examples/capabilities/src/bin/runtime_isolation.rs @@ -52,11 +52,11 @@ fn main() { .unwrap(); let worker = { let _entered = critical.enter(); - Critical.start() + Critical.spawn() }; let router = { let _entered = background.enter(); - Router(worker.clone()).start() + Router(worker.clone()).spawn() }; critical.block_on(async { let thread = router.request(Ping).await.unwrap(); diff --git a/examples/capabilities/src/bin/streams_and_recipients.rs b/examples/capabilities/src/bin/streams_and_recipients.rs index 919dde467..b05405963 100644 --- a/examples/capabilities/src/bin/streams_and_recipients.rs +++ b/examples/capabilities/src/bin/streams_and_recipients.rs @@ -40,8 +40,8 @@ mod asynchronous { } } pub async fn run() { - let first = Counter::<1>(0).start(); - let second = Counter::<2>(0).start(); + let first = Counter::<1>(0).spawn(); + let second = Counter::<2>(0).spawn(); let recipients: Vec> = vec![first.recipient(), second.recipient()]; for member in &recipients { member.send(Item(1)).unwrap(); @@ -124,8 +124,8 @@ mod synchronous { } } pub async fn run() { - let first = Counter::<1>(0).start(); - let second = Counter::<2>(0).start(); + let first = Counter::<1>(0).spawn(); + let second = Counter::<2>(0).spawn(); let recipients: Vec> = vec![first.recipient(), second.recipient()]; for member in &recipients { member.send(Item(1)).unwrap(); diff --git a/examples/capabilities/src/bin/test_kit.rs b/examples/capabilities/src/bin/test_kit.rs index cd5d85bae..d24e5632f 100644 --- a/examples/capabilities/src/bin/test_kit.rs +++ b/examples/capabilities/src/bin/test_kit.rs @@ -62,7 +62,7 @@ fn converter(rates: RatesRef, out: &kit::Probe) -> ConverterRef { rates, out: out.recipient(), } - .start() + .spawn() .to_converter_ref() } @@ -71,7 +71,7 @@ async fn main() { // 1. Mock the dependency and probe the output. let rates = kit::Mock::new() .on::(|msg| if msg.currency == "EUR" { 110 } else { 100 }) - .start(); + .spawn(); let out = kit::Probe::::new(); let under_test = converter(rates.to_rates_ref(), &out); assert_eq!(under_test.convert("EUR".into(), 3).await.unwrap(), Ok(330)); @@ -84,7 +84,7 @@ async fn main() { let slow = kit::Mock::new() .on::(|_| 1) .delay::(Duration::from_millis(300)) - .start(); + .spawn(); let under_test = converter(slow.to_rates_ref(), &out); let error = under_test .convert("USD".into(), 1) @@ -97,7 +97,7 @@ async fn main() { // 3. Fault: the dependency crashes; unmocked messages would crash it too. let crashing = kit::Mock::new() .crash_on::("rates feed lost") - .start(); + .spawn(); let under_test = converter(crashing.to_rates_ref(), &out); assert!(under_test.convert("USD".into(), 1).await.unwrap().is_err()); assert_eq!( @@ -142,7 +142,7 @@ async fn main() { // 5. The same mock in threads mode, with an injected stop reason. let native = native_kit::Mock::new() .on::(|_| 7) - .start(); + .spawn(); assert_eq!(native.rate("X".into()).unwrap(), 7); faults::exit(&native.child_handle(), ExitReason::Shutdown(None)); assert_eq!( diff --git a/examples/capabilities/src/bin/timeout_outcomes.rs b/examples/capabilities/src/bin/timeout_outcomes.rs index fcf745a96..b86b2c18e 100644 --- a/examples/capabilities/src/bin/timeout_outcomes.rs +++ b/examples/capabilities/src/bin/timeout_outcomes.rs @@ -48,7 +48,7 @@ impl Handler for Counter { #[tokio::main(flavor = "multi_thread", worker_threads = 2)] async fn main() { - let counter = Counter(0).start(); + let counter = Counter(0).spawn(); let (entered_tx, entered_rx) = oneshot::channel(); let (release_tx, release_rx) = oneshot::channel(); // The deadline is fixed at send (OTP's call/3): already due here. An diff --git a/examples/capabilities/tests/cross_crate_protocol.rs b/examples/capabilities/tests/cross_crate_protocol.rs index dbf549d3f..ea445aa4f 100644 --- a/examples/capabilities/tests/cross_crate_protocol.rs +++ b/examples/capabilities/tests/cross_crate_protocol.rs @@ -67,14 +67,14 @@ async fn exported_protocols_support_actors_configured_refs_and_downstream_fakes( let fake: Arc = Arc::new(Fake); assert_eq!(fake.value().unwrap(), 7); assert_eq!(GenericFake("state").value().unwrap(), 8); - let native = native::Worker.start(); + let native = native::Worker.spawn(); assert_eq!(native.value().unwrap(), 42); let configured = native.with_request_timeout(Duration::from_secs(5)); assert_eq!(configured.to_shared_ref().value().unwrap(), 42); assert_eq!(configured.to_where_ref().version().unwrap(), 1); native.context().stop(); native.join(); - let actor = asynchronous::Worker.start(); + let actor = asynchronous::Worker.spawn(); assert_eq!(actor.value().await.unwrap(), 42); let configured = actor.with_request_timeout(Duration::from_secs(5)); assert_eq!(configured.to_shared_ref().value().await.unwrap(), 42); diff --git a/examples/capabilities/tests/deterministic_simulation.rs b/examples/capabilities/tests/deterministic_simulation.rs index 46b160a4d..e6efdbe3b 100644 --- a/examples/capabilities/tests/deterministic_simulation.rs +++ b/examples/capabilities/tests/deterministic_simulation.rs @@ -52,7 +52,7 @@ impl Log { /// arrive in the order it sent them: this must hold for every seed. #[spawned_concurrency::sim_test] async fn per_sender_order_holds_under_every_schedule() { - let log = Log::default().start(); + let log = Log::default().spawn(); let senders: Vec<_> = (0..2) .map(|sender| { let log = log.clone(); @@ -124,7 +124,7 @@ mod virtual_time { // virtual time too (determinism D1). let wall = spawned_rt::tasks::sim::wall_now(); let virtual_start = spawned_rt::time::Instant::now(); - let reminder = Reminder::default().start(); + let reminder = Reminder::default().spawn(); rt::sleep(HOUR - Duration::from_secs(1)).await; assert!(!reminder.rung().await.unwrap()); diff --git a/examples/chat_room/src/main.rs b/examples/chat_room/src/main.rs index 7b2c7d4f5..ff313fc5b 100644 --- a/examples/chat_room/src/main.rs +++ b/examples/chat_room/src/main.rs @@ -11,9 +11,9 @@ use user::User; fn main() { rt::run(async { - let room = ChatRoom::new().start(); - let alice = User::new("Alice".into()).start(); - let bob = User::new("Bob".into()).start(); + let room = ChatRoom::new().spawn(); + let alice = User::new("Alice".into()).spawn(); + let bob = User::new("Bob".into()).spawn(); alice.join_room(room.to_room_ref()).unwrap(); bob.join_room(room.to_room_ref()).unwrap(); diff --git a/examples/chat_room_threads/src/main.rs b/examples/chat_room_threads/src/main.rs index 753950c6d..c6018255d 100644 --- a/examples/chat_room_threads/src/main.rs +++ b/examples/chat_room_threads/src/main.rs @@ -13,9 +13,9 @@ use user::User; fn main() { rt::run(|| { - let room = ChatRoom::new().start(); - let alice = User::new("Alice".into()).start(); - let bob = User::new("Bob".into()).start(); + let room = ChatRoom::new().spawn(); + let alice = User::new("Alice".into()).spawn(); + let bob = User::new("Bob".into()).spawn(); alice.join_room(room.to_room_ref()).unwrap(); bob.join_room(room.to_room_ref()).unwrap(); diff --git a/examples/exit_reason/Cargo.toml b/examples/exit_reason/Cargo.toml index 4b85a7b5b..1b8dd7812 100644 --- a/examples/exit_reason/Cargo.toml +++ b/examples/exit_reason/Cargo.toml @@ -13,3 +13,8 @@ spawned-concurrency = { workspace = true } spawned-rt = { workspace = true, features = ["async"] } tracing = { workspace = true } tracing-subscriber = { workspace = true } + +# Manual `impl Actor` `started()` methods return `()`, refining the trait's +# `impl Into>` (`#[started]` adds this allow itself). +[lints.rust] +refining_impl_trait = { level = "allow", priority = -1 } diff --git a/examples/exit_reason/src/main.rs b/examples/exit_reason/src/main.rs index 865c55b79..dbaa38777 100644 --- a/examples/exit_reason/src/main.rs +++ b/examples/exit_reason/src/main.rs @@ -65,7 +65,7 @@ fn main() { // 1. Clean stop via ctx.stop() println!("--- Scenario 1: Clean stop ---"); - let worker1 = Worker::new("worker-1").start(); + let worker1 = Worker::new("worker-1").spawn(); let pong = worker1.ping().await.unwrap(); println!(" {pong}"); worker1.stop().await.unwrap(); @@ -75,7 +75,7 @@ fn main() { // 2. Panic in handler println!("--- Scenario 2: Panic in handler ---"); - let worker2 = Worker::new("worker-2").start(); + let worker2 = Worker::new("worker-2").spawn(); let _ = worker2.panic_now().await; // will fail, actor panics let reason = worker2.wait_exit().await; println!(" Exit reason: {reason}"); @@ -103,14 +103,14 @@ fn main() { reply.reply(()); } } - let worker3 = PanicOnStart.start(); + let worker3 = PanicOnStart.spawn(); let reason = worker3.wait_exit().await; println!(" Exit reason: {reason}"); println!(" is_abnormal: {}\n", reason.is_abnormal()); // 4. Polling exit_reason() while running println!("--- Scenario 4: Polling exit_reason() ---"); - let worker4 = Worker::new("worker-4").start(); + let worker4 = Worker::new("worker-4").spawn(); println!(" While running: {:?}", worker4.exit_reason()); worker4.stop().await.unwrap(); worker4.join().await; @@ -118,7 +118,7 @@ fn main() { // 5. ChildHandle — type-erased supervision handle println!("\n--- Scenario 5: ChildHandle ---"); - let worker5 = Worker::new("worker-5").start(); + let worker5 = Worker::new("worker-5").spawn(); let handle: ChildHandle = worker5.child_handle(); println!(" ActorId: {}", handle.id()); println!(" Same id as ActorRef: {}", handle.id() == worker5.id()); @@ -133,7 +133,7 @@ fn main() { // 6. ChildHandle from a panicking actor println!("\n--- Scenario 6: ChildHandle observes panic ---"); - let worker6 = Worker::new("worker-6").start(); + let worker6 = Worker::new("worker-6").spawn(); let handle6 = worker6.child_handle(); let _ = worker6.panic_now().await; let reason = handle6.wait_exit_async().await; @@ -145,8 +145,8 @@ fn main() { struct Idler; impl Actor for Idler {} - let w = Worker::new("worker-7").start(); - let i = Idler.start(); + let w = Worker::new("worker-7").spawn(); + let i = Idler.spawn(); let handles: Vec = vec![w.child_handle(), i.child_handle()]; println!(" {} actors managed via Vec", handles.len()); for h in &handles { @@ -215,11 +215,11 @@ fn main() { } let log = Arc::new(Mutex::new(Vec::new())); - let observer = Observer { log: log.clone() }.start(); + let observer = Observer { log: log.clone() }.spawn(); - let worker_a = Worker::new("worker-a").start(); - let worker_b = Worker::new("worker-b").start(); - let worker_c = Worker::new("worker-c").start(); + let worker_a = Worker::new("worker-a").spawn(); + let worker_b = Worker::new("worker-b").spawn(); + let worker_c = Worker::new("worker-c").spawn(); let ref_a = observer .request(StartMonitor(worker_a.child_handle())) @@ -298,10 +298,10 @@ fn main() { let supervisor = Supervisor { log: sup_log.clone(), } - .start(); + .spawn(); - let worker_x = Worker::new("worker-x").start(); - let worker_y = Worker::new("worker-y").start(); + let worker_x = Worker::new("worker-x").spawn(); + let worker_y = Worker::new("worker-y").spawn(); supervisor .request(LinkChild(worker_x.child_handle())) .await diff --git a/examples/name_server/src/main.rs b/examples/name_server/src/main.rs index 6d16638af..5c67e51bc 100644 --- a/examples/name_server/src/main.rs +++ b/examples/name_server/src/main.rs @@ -8,7 +8,7 @@ use spawned_rt::tasks as rt; fn main() { rt::run(async { - let ns = NameServer::new().start(); + let ns = NameServer::new().spawn(); ns.add("Joe".into(), "At Home".into()).await.unwrap(); diff --git a/examples/ping_pong/src/main.rs b/examples/ping_pong/src/main.rs index 2eb4de9e2..10e1e8fa8 100644 --- a/examples/ping_pong/src/main.rs +++ b/examples/ping_pong/src/main.rs @@ -11,12 +11,12 @@ use std::time::Duration; fn main() { rt::run(async { - let producer = Producer { consumer: None }.start(); + let producer = Producer { consumer: None }.spawn(); let consumer = Consumer { producer: producer.to_pong_receiver_ref(), } - .start(); + .spawn(); producer .send(SetConsumer(consumer.to_ping_receiver_ref())) diff --git a/examples/ping_pong_threads/src/main.rs b/examples/ping_pong_threads/src/main.rs index 860f8f186..b27561d9c 100644 --- a/examples/ping_pong_threads/src/main.rs +++ b/examples/ping_pong_threads/src/main.rs @@ -12,12 +12,12 @@ use spawned_rt::threads as rt; fn main() { rt::run(|| { - let producer = Producer { consumer: None }.start(); + let producer = Producer { consumer: None }.spawn(); let consumer = Consumer { producer: producer.to_pong_receiver_ref(), } - .start(); + .spawn(); producer .send(SetConsumer(consumer.to_ping_receiver_ref())) diff --git a/examples/service_discovery/src/main.rs b/examples/service_discovery/src/main.rs index f3548aeaa..25c7ce75c 100644 --- a/examples/service_discovery/src/main.rs +++ b/examples/service_discovery/src/main.rs @@ -10,7 +10,7 @@ use spawned_rt::tasks as rt; fn main() { rt::run(async { // Start the service registry actor - let svc = ServiceRegistry::new().start(); + let svc = ServiceRegistry::new().spawn(); // Register the protocol ref by name — other actors can discover it registry::register("service_registry", svc.to_service_registry_ref()).unwrap(); diff --git a/examples/signal_test/src/main.rs b/examples/signal_test/src/main.rs index f553959be..0b4dc002d 100644 --- a/examples/signal_test/src/main.rs +++ b/examples/signal_test/src/main.rs @@ -73,22 +73,40 @@ fn main() { "async" => { tracing::info!("Using Backend::Async"); ( - TickingActor::new("async-1").start_with_backend(Backend::Async), - TickingActor::new("async-2").start_with_backend(Backend::Async), + TickingActor::new("async-1") + .with_options(Backend::Async) + .spawn() + .unwrap(), + TickingActor::new("async-2") + .with_options(Backend::Async) + .spawn() + .unwrap(), ) } "blocking" => { tracing::info!("Using Backend::Blocking"); ( - TickingActor::new("blocking-1").start_with_backend(Backend::Blocking), - TickingActor::new("blocking-2").start_with_backend(Backend::Blocking), + TickingActor::new("blocking-1") + .with_options(Backend::Blocking) + .spawn() + .unwrap(), + TickingActor::new("blocking-2") + .with_options(Backend::Blocking) + .spawn() + .unwrap(), ) } "thread" => { tracing::info!("Using Backend::Thread"); ( - TickingActor::new("thread-1").start_with_backend(Backend::Thread), - TickingActor::new("thread-2").start_with_backend(Backend::Thread), + TickingActor::new("thread-1") + .with_options(Backend::Thread) + .spawn() + .unwrap(), + TickingActor::new("thread-2") + .with_options(Backend::Thread) + .spawn() + .unwrap(), ) } _ => { diff --git a/examples/signal_test_threads/src/main.rs b/examples/signal_test_threads/src/main.rs index e6d9606f4..57775ba3f 100644 --- a/examples/signal_test_threads/src/main.rs +++ b/examples/signal_test_threads/src/main.rs @@ -66,8 +66,8 @@ fn main() { tracing::info!("Starting signal test for threads Actor"); tracing::info!("Press Ctrl+C to test signal handling..."); - let actor1 = TickingActor::new("actor-1").start(); - let actor2 = TickingActor::new("actor-2").start(); + let actor1 = TickingActor::new("actor-1").spawn(); + let actor2 = TickingActor::new("actor-2").spawn(); send_message_on(actor1.context(), rt::ctrl_c(), Shutdown); send_message_on(actor2.context(), rt::ctrl_c(), Shutdown); diff --git a/examples/thread_only/src/main.rs b/examples/thread_only/src/main.rs index 878e96a59..22ce92e8d 100644 --- a/examples/thread_only/src/main.rs +++ b/examples/thread_only/src/main.rs @@ -54,7 +54,7 @@ impl Handler for Observer { } } fn main() { - let counter = Counter(0).start(); + let counter = Counter(0).spawn(); assert_eq!(counter.next().unwrap(), 1); assert_eq!( counter @@ -72,7 +72,7 @@ fn main() { .unwrap(); assert_eq!(named.next().unwrap(), 4); let events = Arc::new(AtomicUsize::new(0)); - let observer = Observer(events.clone()).start(); + let observer = Observer(events.clone()).spawn(); observer.context().monitor(&counter.child_handle()); spawned_concurrency::threads::send_after(Duration::from_millis(5), observer.context(), Tick); counter @@ -95,8 +95,8 @@ fn main() { assert_eq!(observer.wait_exit(), ExitReason::Normal); spawned_concurrency::registry::unregister("thread-only-counter"); // Links retain the same exit semantics without an async runtime. - let first = Counter(0).start(); - let second = Counter(0).start(); + let first = Counter(0).spawn(); + let second = Counter(0).spawn(); first.context().link(&second.child_handle()); second .child_handle() diff --git a/examples/updater/src/main.rs b/examples/updater/src/main.rs index 3b72684ed..9c7414068 100644 --- a/examples/updater/src/main.rs +++ b/examples/updater/src/main.rs @@ -19,7 +19,7 @@ fn main() { "https://httpbin.org/ip".to_string(), Duration::from_millis(1000), ) - .start(); + .spawn(); // giving it some time before ending rt::sleep(Duration::from_secs(10)).await; diff --git a/examples/updater_threads/src/main.rs b/examples/updater_threads/src/main.rs index 5ccf1b88f..0036b6d73 100644 --- a/examples/updater_threads/src/main.rs +++ b/examples/updater_threads/src/main.rs @@ -18,7 +18,7 @@ fn main() { "https://httpbin.org/ip".to_string(), Duration::from_millis(1000), ) - .start(); + .spawn(); // giving it some time before ending thread::sleep(Duration::from_secs(10)); diff --git a/fuzz/Cargo.toml b/fuzz/Cargo.toml index 872f853d6..0cfe55af3 100644 --- a/fuzz/Cargo.toml +++ b/fuzz/Cargo.toml @@ -39,3 +39,5 @@ bench = false [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(spawned_sim)', 'cfg(tokio_unstable)', 'cfg(fuzzing)'] } +# Manual `started()` impls return a refined `impl Future` (0.9 unified start). +refining_impl_trait = { level = "allow", priority = -1 } diff --git a/fuzz/regressions/held_signal_outranks_init_panic.input b/fuzz/regressions/held_signal_outranks_init_panic.input new file mode 100644 index 000000000..1babbda1e Binary files /dev/null and b/fuzz/regressions/held_signal_outranks_init_panic.input differ diff --git a/fuzz/regressions/linked_child_joins_parent_system.input b/fuzz/regressions/linked_child_joins_parent_system.input new file mode 100644 index 000000000..fb28a697d Binary files /dev/null and b/fuzz/regressions/linked_child_joins_parent_system.input differ diff --git a/fuzz/regressions/linked_child_joins_parent_system.tape b/fuzz/regressions/linked_child_joins_parent_system.tape new file mode 100644 index 000000000..21ec4f4b5 --- /dev/null +++ b/fuzz/regressions/linked_child_joins_parent_system.tape @@ -0,0 +1,230 @@ +# spawned-sim tape v2 +seed 3675721402518676785 +spin 10000 +c 0 +p 2 +p 1 +c 0 +t 1000000 +c 0 +c 2 +c 0 +t 2000000 +c 0 +c 2 +c 0 +c 2 +c 0 +c 3 +c 0 +t 3000000 +c 0 +c 2 +c 0 +c 3 +c 0 +t 4000000 +c 0 +c 2 +c 0 +c 3 +c 0 +c 2 +c 0 +c 3 +c 0 +t 5000000 +c 0 +c 2 +c 0 +c 3 +c 0 +t 6000000 +c 0 +c 2 +c 0 +c 3 +c 0 +p 5 +p 4 +c 2 +c 3 +c 5 +c 1 +c 0 +t 56000000 +c 4 +c 0 +t 57000000 +c 0 +c 4 +c 0 +t 58000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 59000000 +c 0 +c 4 +c 0 +t 60000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 61000000 +c 0 +c 4 +c 0 +t 62000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 63000000 +c 0 +c 4 +c 0 +t 64000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 65000000 +c 0 +c 4 +c 0 +t 66000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 67000000 +c 0 +c 4 +c 0 +t 68000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 69000000 +c 0 +c 4 +c 0 +t 70000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 71000000 +c 0 +c 4 +c 0 +t 72000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 73000000 +c 0 +c 4 +c 0 +t 74000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 75000000 +c 0 +c 4 +c 0 +t 76000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 77000000 +c 0 +c 4 +c 0 +t 78000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 79000000 +c 0 +c 4 +c 0 +t 80000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 81000000 +c 0 +c 4 +c 0 +t 82000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 83000000 +c 0 +c 4 +c 0 +t 84000000 +c 0 +c 4 +c 0 +c 4 +c 0 +t 85000000 +c 0 +c 4 +c 0 +t 86000000 +c 0 +c 4 +c 0 +t 276000000 +c 0 +c 4 +c 0 +t 277000000 +c 0 +c 4 +c 0 +t 278000000 +c 0 +c 4 +c 0 +c 4 +t 1228000000 +c 4 +c 0 +c 6 +t 1238000000 +c 0 +c 7 +c 0 +p 8 +c 0 +c 6 +c 0 diff --git a/fuzz/src/driver.rs b/fuzz/src/driver.rs index b56470b95..425cb9c41 100644 --- a/fuzz/src/driver.rs +++ b/fuzz/src/driver.rs @@ -293,10 +293,10 @@ async fn monitor_release_probe(world: &World) { if !world.lock().violations.is_empty() { return; } - let target = Probe.start(); + let target = Probe.spawn(); rt::sleep(Duration::from_millis(10)).await; let alone = sim::stats().tasks; - let observer = Probe.start(); + let observer = Probe.spawn(); observer.context().monitor(&target.child_handle()); observer.child_handle().stop(); if rt::timeout(LONG, observer.wait_exit()).await.is_err() { @@ -917,14 +917,14 @@ impl Driver { let worker = Worker::new(&self.world, None, hooks); let instance = worker.instance(); let (aref, sys) = match how { - SpawnHow::Unmanaged => (worker.start(), None), + SpawnHow::Unmanaged => (worker.spawn(), None), SpawnHow::Managed { group } => { let groups = self.all_groups(); if groups.is_empty() { return; } let (sys, group) = &groups[group as usize % groups.len()]; - match worker.start_on(group) { + match worker.with_options(group).spawn() { Ok(aref) => { if self.systems[*sys].shut { self.violate( @@ -939,12 +939,21 @@ impl Driver { } SpawnHow::Linked { parent } => { if self.actors.is_empty() { - (worker.start(), None) + (worker.spawn(), None) } else { - let parent = self.actors[parent as usize % self.actors.len()] - .aref - .clone(); - let aref = worker.start_linked(&parent.context()); + let parent = &self.actors[parent as usize % self.actors.len()]; + // 0.9: start_link inherits the parent's placement, so a + // managed parent's child joins its system (and its + // shutdown is a stop cause). + let parent_sys = parent.sys; + let parent = parent.aref.clone(); + // 0.9 start_link, its wait abandoned at once: the 0.8 + // non-waiting start_linked. A parent whose exit began + // refuses it (ParentExited): nothing is created. + let aref = match worker.start_link(&parent.context()).actor_ref() { + Ok(aref) => aref, + Err(_) => return, + }; self.may_stop(parent.id()); self.may_stop(aref.id()); self.links.push(LinkRec { @@ -955,7 +964,7 @@ impl Driver { at: self.op, unlinked: false, }); - (aref, None) + (aref, parent_sys) } } }; @@ -1866,9 +1875,13 @@ impl Driver { .map(|a| (a.aref.id(), a.instance, a.aref.child_handle().exit_reason())) .collect(); for (id, instance, exit) in standalone { - let (may_stop, panicked) = { + let (may_stop, panicked, started_panicked) = { let s = self.world.lock(); - (s.may_stop.contains(&id), s.insts[instance].panicked) + ( + s.may_stop.contains(&id), + s.insts[instance].panicked, + s.insts[instance].started_panicked, + ) }; let Some(exit) = exit else { continue }; if !may_stop { @@ -1884,7 +1897,13 @@ impl Driver { // non-abnormal reason a signal can carry is `Shutdown`. let trapping = self.traps.get(&id).is_some_and(|t| t.0); let late_shutdown = exit == ExitReason::Shutdown(None) && !trapping; - if panicked && !exit.is_abnormal() && !late_shutdown { + // 0.9 (design 2.3): an untrapped exit signal held while started() + // ran outranks its panic (row 3 over row 4: in OTP the signal + // kills the initialising process first). The trap flag that + // counts is the one read when started() returned, which a later + // TrapExit op may have changed, so any shutdown is accepted here. + let held_signal_wins = started_panicked && exit.is_shutdown(); + if panicked && !exit.is_abnormal() && !late_shutdown && !held_signal_wins { self.violate( "L4: an actor that panicked exited with a non-abnormal reason", format!("w{instance} {exit:?}"), diff --git a/fuzz/src/scenario.rs b/fuzz/src/scenario.rs index ac35e9fa7..0bd1a78d1 100644 --- a/fuzz/src/scenario.rs +++ b/fuzz/src/scenario.rs @@ -100,11 +100,11 @@ pub enum Cmd { #[derive(Arbitrary, Debug, Clone, Copy, PartialEq, Eq)] pub enum SpawnHow { - /// `.start()`: unmanaged async actor. + /// `.spawn()`: unmanaged async actor. Unmanaged, - /// `.start_on(group)`: managed by an `ActorSystem` async group. + /// `.with_options(group).spawn()`: managed by an `ActorSystem` async group. Managed { group: u8 }, - /// `.start_linked(parent)`. + /// `.start_link(parent)`, its wait abandoned (the 0.8 `start_linked`). Linked { parent: u8 }, } diff --git a/fuzz/tests/held_signal_outranks_init_panic.rs b/fuzz/tests/held_signal_outranks_init_panic.rs new file mode 100644 index 000000000..2dbee1c24 --- /dev/null +++ b/fuzz/tests/held_signal_outranks_init_panic.rs @@ -0,0 +1,120 @@ +//! Generated by `cargo run --example repro -- --emit held_signal_outranks_init_panic`. +//! +//! Failure: none (passing input) +//! Strategy dfs:bound=0, seed 18437982626083144225. +//! +//! ```text +//! Scenario { +//! seed: 18437982626083144225, +//! sched: Canonical, +//! ops: [ +//! Spawn { +//! how: Linked { +//! parent: 198, +//! }, +//! hooks: Hooks { +//! flags: 202, +//! started_sleep: 51, +//! stopped_sleep: 157, +//! }, +//! }, +//! Stop { +//! who: 93, +//! reason: Shutdown, +//! }, +//! Stop { +//! who: 93, +//! reason: Shutdown, +//! }, +//! Stop { +//! who: 93, +//! reason: Normal, +//! }, +//! Stop { +//! who: 93, +//! reason: Shutdown, +//! }, +//! Link { +//! a: 157, +//! b: 11, +//! }, +//! Spawn { +//! how: Linked { +//! parent: 243, +//! }, +//! hooks: Hooks { +//! flags: 255, +//! started_sleep: 119, +//! stopped_sleep: 255, +//! }, +//! }, +//! Supervisor( +//! SupSpec { +//! system: 11, +//! policy: Policy { +//! strategy: 238, +//! max_restarts: 225, +//! window: 68, +//! backoff: 99, +//! jitter: true, +//! stable: 248, +//! }, +//! children: [], +//! }, +//! ), +//! ShutdownSystem { +//! sys: 93, +//! mode: 93, +//! ms: 23901, +//! }, +//! Stop { +//! who: 93, +//! reason: Normal, +//! }, +//! TrapExit { +//! who: 93, +//! on: true, +//! }, +//! Stop { +//! who: 93, +//! reason: Shutdown, +//! }, +//! Stop { +//! who: 83, +//! reason: Shutdown, +//! }, +//! Stop { +//! who: 93, +//! reason: Normal, +//! }, +//! Stop { +//! who: 93, +//! reason: Shutdown, +//! }, +//! Spawn { +//! how: Unmanaged, +//! hooks: Hooks { +//! flags: 0, +//! started_sleep: 0, +//! stopped_sleep: 0, +//! }, +//! }, +//! ], +//! } +//! ``` +#![cfg(spawned_sim)] + +const INPUT: &[u8] = include_bytes!("../regressions/held_signal_outranks_init_panic.input"); + +/// The input under its own schedule and under its recorded tape. +#[test] +fn held_signal_outranks_init_panic_tape() { + spawned_fuzz::assert_passes(INPUT, None); +} + +/// The same operations under other simulator schedules. +#[spawned_macros::sim_test(seeds = 32)] +async fn held_signal_outranks_init_panic_seeds(seed: u64) { + spawned_fuzz::init(); + spawned_fuzz::run_body(spawned_fuzz::decode(INPUT).unwrap(), seed).await; +} diff --git a/fuzz/tests/linked_child_joins_parent_system.rs b/fuzz/tests/linked_child_joins_parent_system.rs new file mode 100644 index 000000000..97ae64eb1 --- /dev/null +++ b/fuzz/tests/linked_child_joins_parent_system.rs @@ -0,0 +1,96 @@ +//! Generated by `cargo run --example repro -- --emit linked_child_joins_parent_system`. +//! +//! Failure: none (passing input) +//! Strategy random, seed 3675721402518676785. +//! +//! ```text +//! Scenario { +//! seed: 3675721402518676785, +//! sched: Random, +//! ops: [ +//! Spawn { +//! how: Managed { +//! group: 8, +//! }, +//! hooks: Hooks { +//! flags: 0, +//! started_sleep: 50, +//! stopped_sleep: 14, +//! }, +//! }, +//! Fence, +//! Spawn { +//! how: Linked { +//! parent: 154, +//! }, +//! hooks: Hooks { +//! flags: 126, +//! started_sleep: 59, +//! stopped_sleep: 255, +//! }, +//! }, +//! Fence, +//! Fence, +//! Spawn { +//! how: Unmanaged, +//! hooks: Hooks { +//! flags: 0, +//! started_sleep: 201, +//! stopped_sleep: 219, +//! }, +//! }, +//! TrapExit { +//! who: 102, +//! on: true, +//! }, +//! Stop { +//! who: 48, +//! reason: Normal, +//! }, +//! Link { +//! a: 255, +//! b: 255, +//! }, +//! ShutdownSystem { +//! sys: 76, +//! mode: 144, +//! ms: 19105, +//! }, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Fence, +//! Advance { +//! ms: 190, +//! }, +//! ], +//! } +//! ``` +#![cfg(spawned_sim)] + +const INPUT: &[u8] = include_bytes!("../regressions/linked_child_joins_parent_system.input"); +const TAPE: &str = include_str!("../regressions/linked_child_joins_parent_system.tape"); + +/// The input under its own schedule and under its recorded tape. +#[test] +fn linked_child_joins_parent_system_tape() { + spawned_fuzz::assert_passes(INPUT, Some(TAPE)); +} + +/// The same operations under other simulator schedules. +#[spawned_macros::sim_test(seeds = 32)] +async fn linked_child_joins_parent_system_seeds(seed: u64) { + spawned_fuzz::init(); + spawned_fuzz::run_body(spawned_fuzz::decode(INPUT).unwrap(), seed).await; +} diff --git a/macros/src/actor.rs b/macros/src/actor.rs index dd7aac5e1..343316344 100644 --- a/macros/src/actor.rs +++ b/macros/src/actor.rs @@ -304,6 +304,11 @@ pub(crate) fn expand(attr: TokenStream, item: TokenStream) -> TokenStream { } let mut m = method.clone(); m.attrs.retain(|a| !a.path().is_ident("started")); + // `started()` returns `()`, a `Result` or an `Init`: the impl's + // return type refines the trait's `impl Into>` on + // purpose (unified start, 0.9). + m.attrs + .push(syn::parse_quote!(#[allow(refining_impl_trait)])); m.vis = syn::Visibility::Inherited; m.sig.ident = format_ident!("started"); if m.sig.asyncness.is_some() { diff --git a/scripts/tests/test_model_sync.py b/scripts/tests/test_model_sync.py index d1228014c..462928cb7 100644 --- a/scripts/tests/test_model_sync.py +++ b/scripts/tests/test_model_sync.py @@ -168,11 +168,11 @@ def test_allowlist_needs_tracking(self): def test_design_only_model_with_code_fails(self): tmp = self.copy() - self.edit_map(tmp, "code = []\n[models.Start.tlc_only]", - 'code = ["concurrency/src/system.rs"]\n[models.Start.tlc_only]') + self.edit_map(tmp, "code = []\n[models.Introspection.tlc_only]", + 'code = ["concurrency/src/system.rs"]\n[models.Introspection.tlc_only]') code, out = run("--root", str(tmp), "models") self.assertEqual(code, 1, out) - self.assertIn("Start is a design-only model", out) + self.assertIn("Introspection is a design-only model", out) def test_unrun_trace_spec_is_not_validation(self): tmp = self.copy() @@ -249,7 +249,7 @@ def setUp(self): shutil.copy(ROOT / "verification" / "model-map.toml", self.repo / "verification") self.write("verification/tla/Lifecycle.tla", "---- MODULE Lifecycle ----\n====\n") self.write("concurrency/src/actor_cleanup.rs", ACTOR_CLEANUP) - self.write("concurrency/src/registry.rs", "pub fn lookup() {}\n") + self.write("concurrency/src/response.rs", "pub fn lookup() {}\n") # Loom-checked cores: fence.rs has only a Loom harness, core.rs also # a TLA+ model (Lifecycle). self.write("concurrency/src/monitor/fence.rs", "pub fn close() -> u32 {\n 1\n}\n") @@ -348,7 +348,7 @@ def test_code_and_test_change_fails(self): self.assertEqual(code, 1, out) def test_unmapped_code_passes(self): - self.edit("concurrency/src/registry.rs", "{}", "{ let _ = 1; }") + self.edit("concurrency/src/response.rs", "{}", "{ let _ = 1; }") code, out = self.check() self.assertEqual(code, 0, out) diff --git a/testing/README.md b/testing/README.md index 4c0e2cc4e..0df9297c3 100644 --- a/testing/README.md +++ b/testing/README.md @@ -29,7 +29,7 @@ use std::time::Duration; struct Lookup { key: u64 } impl Message for Lookup { type Result = u64; } -let server = Mock::new().on::(|msg| msg.key + 1).start(); +let server = Mock::new().on::(|msg| msg.key + 1).spawn(); assert_eq!(server.request(Lookup { key: 41 }).unwrap(), 42); let received: Lookup = server.expect(Duration::from_secs(1)); assert_eq!(received.key, 41); diff --git a/testing/src/tasks.rs b/testing/src/tasks.rs index 854bdceef..0850c0a56 100644 --- a/testing/src/tasks.rs +++ b/testing/src/tasks.rs @@ -16,12 +16,13 @@ use crate::shared::SETTLE_TIMEOUT; use spawned_concurrency::error::ActorError; use spawned_concurrency::message::Message; use spawned_concurrency::supervision::{ChildState, SupervisedChild, Supervisor, SupervisorReport}; -use spawned_concurrency::system::{ActorSystem, ExecutionGroup, ShutdownReport, SystemError}; +use spawned_concurrency::system::{ActorSystem, ExecutionGroup, ShutdownReport}; use spawned_concurrency::tasks::{ - Actor, ActorRef, ActorStart, Context, Handler, Receiver, Recipient, + Actor, ActorRef, ActorStart, Context, Handler, Receiver, Recipient, StartOptions, }; use spawned_concurrency::{ChildHandle, ExitReason}; use spawned_concurrency::{Deadline, PendingReply, ReplyReceiver}; +use spawned_concurrency::{StartError, StartOutcome}; use spawned_rt::tasks::{sleep, timeout}; use std::future::Future; use std::ops::Deref; @@ -46,7 +47,7 @@ use std::time::Duration; /// [`MockRef::assert_no_unexpected`] / [`MockRef::received`] say why. /// /// ```ignore -/// let prices = Mock::new().on::(|_| 7).start(); +/// let prices = Mock::new().on::(|_| 7).spawn(); /// assert_eq!(invoice(&prices.to_price_ref(), 3).await, 21); /// let _asked: price_protocol::Price = prices.expect(Duration::from_secs(1)).await; /// ``` @@ -72,20 +73,20 @@ impl Mock { self } - /// Start on the ambient Tokio runtime. - pub fn start(self) -> MockRef { + /// Spawn on the ambient Tokio runtime (unmanaged, `Backend::Async`). + pub fn spawn(self) -> MockRef { let inbox = self.inbox.clone(); MockRef { - actor: ActorStart::start(self), + actor: ActorStart::spawn(self), inbox, } } - /// Start on an [`ExecutionGroup`]. - pub fn start_on(self, group: &ExecutionGroup) -> Result { + /// Spawn with options: an [`ExecutionGroup`], a backend, a name. + pub fn spawn_with(self, options: impl Into>) -> Result { let inbox = self.inbox.clone(); Ok(MockRef { - actor: ActorStart::start_on(self, group)?, + actor: ActorStart::with_options(self, options).spawn()?, inbox, }) } @@ -503,15 +504,33 @@ impl TestSystem { &self.group } - /// Start `actor` on the test group. Panics if the system is closed. - pub fn start(&self, actor: A) -> ActorRef { - ActorStart::start_on(actor, &self.group).expect("start on test group") + /// Spawn `actor` on the test group without waiting for `started()`. + /// Panics if the system is closed. + pub fn spawn(&self, actor: A) -> ActorRef { + ActorStart::with_options(actor, &self.group) + .spawn() + .expect("spawn on test group") } - /// Start a mock on the test group. - pub fn start_mock(&self, mock: Mock) -> MockRef { - mock.start_on(&self.group) - .expect("start mock on test group") + /// Start `actor` on the test group and wait for `started()`. Panics if + /// the start fails or is ignored. + pub async fn start(&self, actor: A) -> ActorRef { + match ActorStart::with_options(actor, &self.group).start().await { + Ok(StartOutcome::Started(actor)) => actor, + Ok(StartOutcome::Ignored { .. }) => panic!("start on test group: ignored"), + Err(error) => panic!("start on test group: {error}"), + } + } + + /// Start options placing an actor on the test group. + pub fn options(&self) -> StartOptions { + StartOptions::new().group(&self.group) + } + + /// Spawn a mock on the test group. + pub fn spawn_mock(&self, mock: Mock) -> MockRef { + mock.spawn_with(&self.group) + .expect("spawn mock on test group") } /// [`shutdown_system`]: panics with a report if not complete by `grace`. diff --git a/testing/src/threads.rs b/testing/src/threads.rs index 9bd312451..6e87bd977 100644 --- a/testing/src/threads.rs +++ b/testing/src/threads.rs @@ -39,11 +39,11 @@ impl Mock { self } - /// Start on a dedicated OS thread. - pub fn start(self) -> MockRef { + /// Spawn on a dedicated OS thread. + pub fn spawn(self) -> MockRef { let inbox = self.inbox.clone(); MockRef { - actor: ActorStart::start(self), + actor: ActorStart::spawn(self), inbox, } } diff --git a/testing/tests/ethlambda_pattern.rs b/testing/tests/ethlambda_pattern.rs index f35994bbd..9b3325147 100644 --- a/testing/tests/ethlambda_pattern.rs +++ b/testing/tests/ethlambda_pattern.rs @@ -60,7 +60,7 @@ fn check_and_forward(server: &Server, sidecars: Vec) { async fn only_what_passes_is_forwarded_as_checked() { // Before: `struct RecordingChain(mpsc::UnboundedSender)` plus a // full `impl P2PToBlockChain for RecordingChain`. After: one line. - let chain = Mock::new().allow_sends().start(); + let chain = Mock::new().allow_sends().spawn(); let server = Server { blockchain: Some(chain.to_p2p_to_block_chain_ref()), }; diff --git a/testing/tests/tasks_kit.rs b/testing/tests/tasks_kit.rs index c63296a6a..6d12b54d0 100644 --- a/testing/tests/tasks_kit.rs +++ b/testing/tests/tasks_kit.rs @@ -64,13 +64,13 @@ async fn mock_answers_protocol_calls_and_probe_records_sends() { let inventory = Mock::new() .on::(|msg| msg.item == "apple") .on::(|_| ()) - .start(); + .spawn(); let sales = Probe::::new(); let checkout = Checkout { inventory: inventory.to_inventory_ref(), sales: sales.recipient(), } - .start(); + .spawn(); assert!(checkout.buy("apple".into()).await.unwrap()); assert!(!checkout.buy("pear".into()).await.unwrap()); @@ -99,7 +99,7 @@ async fn protocol_methods_run_directly_on_the_mock() { let mock = Mock::new() .on::(|_| true) .allow_sends() - .start(); + .spawn(); assert!(mock.reserve("x".into()).await.unwrap()); mock.restock("x".into(), 1).unwrap(); mock.audit().unwrap(); @@ -110,7 +110,7 @@ async fn protocol_methods_run_directly_on_the_mock() { #[tokio::test] async fn unconfigured_request_fails_loudly() { - let mock = Mock::new().allow_sends().start(); + let mock = Mock::new().allow_sends().spawn(); let error = mock.reserve("x".into()).await.unwrap_err(); assert!(matches!( error, @@ -135,7 +135,7 @@ async fn unconfigured_request_fails_loudly() { #[tokio::test] async fn unconfigured_send_names_allow_sends() { - let mock = Mock::new().start(); + let mock = Mock::new().spawn(); mock.audit().unwrap(); let ExitReason::Panic(reason) = join_within(&mock, WAIT).await else { panic!("strict mock should panic on an unconfigured send"); @@ -148,7 +148,7 @@ async fn delay_forces_a_request_timeout() { let mock = Mock::new() .on::(|_| true) .delay::(Duration::from_millis(300)) - .start(); + .spawn(); let error = mock .with_request_timeout(Duration::from_millis(20)) .reserve("x".into()) @@ -164,7 +164,7 @@ async fn discard_and_crash_on_inject_message_faults() { let mock = Mock::new() .discard::() .crash_on::("audit backend down") - .start(); + .spawn(); mock.restock("x".into(), 1).unwrap(); mock.audit().unwrap(); assert_eq!( @@ -256,7 +256,7 @@ impl Handler for Counter { #[tokio::test] async fn settle_waits_for_earlier_sends() { let seen = Arc::new(AtomicUsize::new(0)); - let counter = Counter(seen.clone()).start(); + let counter = Counter(seen.clone()).spawn(); for _ in 0..100 { counter.send(Bump).unwrap(); } @@ -279,12 +279,12 @@ async fn exit_injects_every_reason() { ), (ExitReason::Kill, ExitReason::Killed), ] { - let actor = Counter(Arc::default()).start(); + let actor = Counter(Arc::default()).spawn(); faults::exit(&actor.child_handle(), reason); assert_eq!(join_within(&actor, WAIT).await, expected); } // Normal is ignored by a non-trapping target. - let actor = Counter(Arc::default()).start(); + let actor = Counter(Arc::default()).spawn(); faults::exit(&actor.child_handle(), ExitReason::Normal); assert_eq!(settle(&actor, Get, WAIT).await, 0); assert_eq!(actor.child_handle().exit_reason(), None); @@ -295,7 +295,7 @@ async fn exit_injects_every_reason() { #[tokio::test] #[should_panic(expected = "did not exit within")] async fn exit_within_fails_instead_of_hanging() { - let actor = Counter(Arc::default()).start(); + let actor = Counter(Arc::default()).spawn(); exit_within(&actor.child_handle(), Duration::from_millis(20)).await; } @@ -337,7 +337,7 @@ async fn supervised_child_restarts_after_injected_crash() { #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn mock_starts_on_a_test_group() { let test = TestSystem::new(); - let mock = test.start_mock(Mock::new().on::(|_| true)); + let mock = test.spawn_mock(Mock::new().on::(|_| true)); assert!(mock.reserve("x".into()).await.unwrap()); test.shutdown(WAIT).await; } @@ -367,7 +367,7 @@ impl Handler for Stuck { async fn bounded_shutdown_reports_unfinished_actors() { let test = TestSystem::new(); let entered = Arc::new(AtomicBool::new(false)); - let stuck = test.start(Stuck(entered.clone())); + let stuck = test.spawn(Stuck(entered.clone())); stuck.send(Hang).unwrap(); // Shut down only once the handler is inside its blocking sleep, so the // report is deterministic. Dropping the system during the unwind starts a @@ -389,13 +389,13 @@ pub trait LedgerProtocol: Send + Sync { async fn allow_sends_also_answers_unit_requests_but_allow_send_is_per_type() { // A handler cannot tell a send from a request, so `allow_sends` also // acknowledges a request whose reply is `()`. - let lenient = Mock::new().allow_sends().start(); + let lenient = Mock::new().allow_sends().spawn(); lenient.note(1).unwrap(); lenient.flush().await.unwrap(); lenient.assert_no_unexpected(); // `allow_send::()` acknowledges only the chosen type. - let strict = Mock::new().allow_send::().start(); + let strict = Mock::new().allow_send::().spawn(); strict.note(1).unwrap(); assert!(strict.flush().await.is_err()); let ExitReason::Panic(reason) = join_within(&strict, WAIT).await else { @@ -416,7 +416,7 @@ async fn settle_within_uses_the_given_bound() { let mock = Mock::new() .on::(|_| true) .delay::(Duration::from_millis(500)) - .start(); + .spawn(); // Occupy the mock; the fence queues behind the slow reply. let _pending = mock.actor_ref().send_request( inventory_protocol::Reserve { item: "x".into() }, @@ -452,7 +452,7 @@ async fn helpers_follow_paused_tokio_time() { .await; // exit_within: an actor stopped after a virtual delay. - let mock = Mock::new().start(); + let mock = Mock::new().spawn(); let handle = mock.child_handle(); tokio::spawn(async move { tokio::time::sleep(Duration::from_secs(600)).await; @@ -475,7 +475,7 @@ async fn delayed_reply_is_handled_only_after_the_delay() { let mock = Mock::new() .on::(|_| true) .delay::(delay) - .start(); + .spawn(); let caller = { let mock = mock.clone(); tokio::spawn(async move { @@ -546,7 +546,7 @@ async fn restart_child_waits_out_a_restart_in_progress() { #[tokio::test] async fn exit_injection_never_blocks_a_current_thread_runtime() { for reason in [ExitReason::Panic("injected".into()), ExitReason::Kill] { - let actor = Counter(Arc::default()).start(); + let actor = Counter(Arc::default()).spawn(); faults::exit(&actor.child_handle(), reason.clone()); let observed = join_within(&actor, WAIT).await; assert!(matches!( @@ -604,8 +604,8 @@ async fn async_expect_names_its_caller() { async fn kill_stops_a_trapping_target_and_other_signals_are_trapped() { // A kill is untrappable: the trapping target ends Killed and its trapping // peer receives the (trappable) Killed notice and survives. - let target = Counter(Arc::default()).start(); - let observer = Counter(Arc::default()).start(); + let target = Counter(Arc::default()).spawn(); + let observer = Counter(Arc::default()).spawn(); target.context().trap_exit(true); observer.context().trap_exit(true); observer.context().link(&target.child_handle()); @@ -616,7 +616,7 @@ async fn kill_stops_a_trapping_target_and_other_signals_are_trapped() { observer.context().stop(); assert_eq!(join_within(&observer, WAIT).await, ExitReason::Normal); // Any other signal to a trapping target is queued as an Exit. - let target = Counter(Arc::default()).start(); + let target = Counter(Arc::default()).spawn(); target.context().trap_exit(true); faults::exit(&target.child_handle(), ExitReason::Shutdown(None)); assert_eq!(settle(&target, Get, WAIT).await, 0); @@ -643,7 +643,7 @@ async fn in_actor_runs_a_closure_in_a_turn() { impl Message for AddOne { type Result = (); } - let tally = Tally(0).start(); + let tally = Tally(0).spawn(); tally.send(AddOne(2)).unwrap(); let (seen, own) = spawned_test::tasks::in_actor(&tally, |state, ctx| { state.0 += 1; diff --git a/testing/tests/threads_kit.rs b/testing/tests/threads_kit.rs index b6686c130..d10e64b5e 100644 --- a/testing/tests/threads_kit.rs +++ b/testing/tests/threads_kit.rs @@ -54,13 +54,13 @@ fn native_mock_and_probe() { let ledger = Mock::new() .on::(|msg| if msg.account == 1 { 100 } else { 0 }) .on::(|_| ()) - .start(); + .spawn(); let alerts = Probe::::new(); let teller = Teller { ledger: ledger.to_ledger_ref(), alerts: alerts.recipient(), } - .start(); + .spawn(); assert!(teller.withdraw(1, 40).unwrap()); assert!(!teller.withdraw(2, 40).unwrap()); @@ -78,7 +78,7 @@ fn native_delay_forces_timeout_and_unconfigured_fails_loudly() { let slow = Mock::new() .on::(|_| 1) .delay::(Duration::from_millis(300)) - .start(); + .spawn(); let via_protocol = slow .with_request_timeout(Duration::from_millis(20)) .balance(1); @@ -91,7 +91,7 @@ fn native_delay_forces_timeout_and_unconfigured_fails_loudly() { .unwrap_err(); assert!(matches!(error, ActorError::RequestTimeout)); - let strict = Mock::new().start(); + let strict = Mock::new().spawn(); assert!(strict.balance(1).is_err()); let ExitReason::Panic(reason) = join_within(&strict, WAIT) else { panic!("strict mock should panic"); @@ -105,7 +105,7 @@ fn native_message_faults_and_settle() { let mock = Mock::new() .discard::() .on::(|_| 5) - .start(); + .spawn(); mock.record(1, 1).unwrap(); mock.settle(); assert_eq!(mock.received()[0].outcome, Outcome::Discarded); @@ -119,7 +119,7 @@ fn native_message_faults_and_settle() { let crashing = Mock::new() .crash_on::("disk full") - .start(); + .spawn(); crashing.record(1, 1).unwrap(); assert_eq!( join_within(&crashing, WAIT), @@ -166,13 +166,13 @@ fn native_exit_injection() { ExitReason::Shutdown(None), ExitReason::Panic("injected".into()), ] { - let actor = Idle.start(); + let actor = Idle.spawn(); settle(&actor, Ping, WAIT); faults::exit(&actor.child_handle(), reason.clone()); assert_eq!(join_within(&actor, WAIT), reason); } // Normal is ignored by a non-trapping target. - let actor = Idle.start(); + let actor = Idle.spawn(); faults::exit(&actor.child_handle(), ExitReason::Normal); settle(&actor, Ping, WAIT); assert_eq!(actor.child_handle().exit_reason(), None); @@ -182,7 +182,7 @@ fn native_exit_injection() { #[test] fn native_kill_injection() { - let actor = Idle.start(); + let actor = Idle.spawn(); faults::exit(&actor.child_handle(), ExitReason::Kill); assert_eq!(join_within(&actor, WAIT), ExitReason::Killed); } @@ -190,7 +190,7 @@ fn native_kill_injection() { #[test] #[should_panic(expected = "did not exit within")] fn native_exit_within_fails_instead_of_hanging() { - let actor = Idle.start(); + let actor = Idle.spawn(); exit_within(&actor.child_handle(), Duration::from_millis(20)); } @@ -211,7 +211,7 @@ fn native_expect_waits_for_the_reply_closure() { gate.recv().unwrap(); q.0 * 10 }) - .start(); + .spawn(); let caller = { let mock = mock.clone(); std::thread::spawn(move || mock.request_with_timeout(Quote(4), WAIT)) @@ -237,7 +237,7 @@ fn native_expect_no_message_reports_in_flight_messages() { let _ = gate.recv(); q.0 }) - .start(); + .spawn(); let _reply = mock .send_request(Quote(1), spawned_concurrency::DEFAULT_REQUEST_TIMEOUT) .unwrap(); @@ -249,7 +249,7 @@ fn native_expect_no_message_reports_in_flight_messages() { fn native_panicking_reply_closure_still_logs_the_message() { let mock = Mock::new() .on::(|_| panic!("reply exploded")) - .start(); + .spawn(); assert!(mock.request_with_timeout(Quote(7), WAIT).is_err()); assert_eq!( join_within(&mock, WAIT), @@ -292,7 +292,7 @@ fn native_nothing_is_logged_after_close_returns() { #[test] fn native_panic_injection_into_an_exited_target_is_a_no_op() { - let actor = Idle.start(); + let actor = Idle.spawn(); actor .child_handle() .stop_with_reason(spawned_concurrency::ExitReason::Shutdown(None)); @@ -305,7 +305,7 @@ fn native_panic_injection_into_an_exited_target_is_a_no_op() { #[test] fn native_kill_injection_is_reliable() { for _ in 0..200 { - let actor = Idle.start(); + let actor = Idle.spawn(); faults::exit(&actor.child_handle(), ExitReason::Kill); assert_eq!(join_within(&actor, WAIT), ExitReason::Killed); } @@ -415,8 +415,8 @@ fn native_helpers_name_their_caller() { #[test] fn kill_stops_a_trapping_target_and_other_signals_are_trapped() { - let target = Idle.start(); - let observer = Idle.start(); + let target = Idle.spawn(); + let observer = Idle.spawn(); target.context().trap_exit(true); observer.context().trap_exit(true); observer.context().link(&target.child_handle()); @@ -426,7 +426,7 @@ fn kill_stops_a_trapping_target_and_other_signals_are_trapped() { assert_eq!(observer.child_handle().exit_reason(), None); observer.context().stop(); assert_eq!(join_within(&observer, WAIT), ExitReason::Normal); - let target = Idle.start(); + let target = Idle.spawn(); target.context().trap_exit(true); faults::exit(&target.child_handle(), ExitReason::Shutdown(None)); settle(&target, Ping, WAIT); @@ -453,7 +453,7 @@ fn native_in_actor_runs_a_closure_in_a_turn() { impl Message for AddOne { type Result = (); } - let tally = Tally(0).start(); + let tally = Tally(0).spawn(); tally.send(AddOne(2)).unwrap(); let (seen, own) = spawned_test::threads::in_actor(&tally, |state, ctx| { state.0 += 1; diff --git a/verification/model-map.toml b/verification/model-map.toml index a485265c0..d975fea8d 100644 --- a/verification/model-map.toml +++ b/verification/model-map.toml @@ -196,17 +196,33 @@ reason = "rank 12a design model; the dynamic supervisor is not implemented" tracking = "trace validation is part of implementing rank 12a (design: PR #245)" [models.Start] -# Unified start API design model (0.9, rank 5h, PR #259); no code yet. +# Unified start (0.9, rank 5h; design docs/design/unified-start.md): stage 1 +# (prepare, the atomic commit: group admission, the parent's edge, the +# name), the startup decision and readiness commit, the startup cell and +# its waiter, notice disposition at publication, abandonment, the init +# timer, continuations, typed names, placement inheritance. Scope ownership +# (S8) is not implemented yet: its actions have no events. files = [ "verification/tla/Start.tla", "verification/tla/MCStart*", "verification/tla/TraceStart.tla", ] -code = [] -[models.Start.tlc_only] -design_only = true -reason = "0.9 unified start design model; the unified start API is not implemented" -tracking = "trace validation is part of implementing the unified start API, design PR #259" +code = [ + "concurrency/src/start.rs", + "concurrency/src/tasks/actor.rs", + "concurrency/src/tasks/actor/**", + "concurrency/src/threads/actor.rs", + "concurrency/src/threads/actor/**", + "concurrency/src/link.rs", + "concurrency/src/link/table.rs", + "concurrency/src/child_handle.rs", + "concurrency/src/actor_cleanup.rs", + "concurrency/src/registry.rs", + "concurrency/src/system.rs", + "concurrency/src/system/group.rs", + "concurrency/src/lifecycle_trace.rs", +] +coverage_floor = 28 # of 28 trace-spec arms (trace_coverage.py STATIC) [models.Introspection] # Introspection v1 design model (PR #256, 0.9); no code yet. @@ -335,6 +351,23 @@ EarlyFire = "fault device only (Fault = EarlyNotify: links and monitors fire whe Spawn = { into = [], why = "trace_check.py numbers actors from Spawn events" } Send = { into = ["Request", "Enqueue", "Send"], why = "trace_check.py splits a Send from outside code into Request and Enqueue, and keeps one from a recorded actor as the model's Send" } +[[trace]] +struct = "StartEvent" +emitters = [ + { call = '\bStartEvent::new\(', arg = 0 }, +] +specs = ["verification/tla/TraceStart.tla"] +[trace.unlogged] +OwnerClose = "scope ownership (S8) is not implemented: no owner gate closes" +ScopeDeadline = "scope ownership (S8) is not implemented: no scope deadline" +GenDone = "scope ownership (S8) is not implemented: no generation end" +GClose = "group closure is outside the start traces (a closed group's refusal is a stage-1 Closed)" +WorkDoneP = "P's owned work has no event; TraceStart.tla takes it silently" +RegisterFirst = "fault device (SplitCommit) only" +CommitRest = "fault device (SplitCommit) only" +LateLink = "fault device (LinkAfterStarted) only" +LateRelease = "fault device (LateUnregister) only" + [[trace]] struct = "JobEvent" emitters = [ diff --git a/verification/planted/patches/13-exit-linked-cause-dropped.patch b/verification/planted/patches/13-exit-linked-cause-dropped.patch index 20d7a8c9d..e4f46ab54 100644 --- a/verification/planted/patches/13-exit-linked-cause-dropped.patch +++ b/verification/planted/patches/13-exit-linked-cause-dropped.patch @@ -1,10 +1,10 @@ Planted bug 13 (exit-linked-cause-dropped): An actor killed through a link exits Normal, so the crash stops propagating diff --git a/concurrency/src/link.rs b/concurrency/src/link.rs -index ff5b35ef..cf77cea9 100644 +index 542fceec..aea638b1 100644 --- a/concurrency/src/link.rs +++ b/concurrency/src/link.rs -@@ -310,7 +310,13 @@ fn apply_signal( +@@ -492,7 +492,13 @@ fn apply_signal( if exit.reason == ExitReason::Normal { return; } @@ -16,6 +16,6 @@ index ff5b35ef..cf77cea9 100644 + exit.reason + }; + if request.untrapped(reason) { + let waits = request.take_waiting_starts(); drop(request); (target.wake)(); - } diff --git a/verification/planted/patches/14-exit-shutdown-reports-normal.patch b/verification/planted/patches/14-exit-shutdown-reports-normal.patch index 9ef571ad8..4382711a2 100644 --- a/verification/planted/patches/14-exit-shutdown-reports-normal.patch +++ b/verification/planted/patches/14-exit-shutdown-reports-normal.patch @@ -13,6 +13,6 @@ index 3b94d213..1ac963ad 100644 + } else { + reason + }; - crate::link::stop_with_reason(self.target.id, &self.target.request, reason, || { + crate::link::admin_stop_with_reason(self.target.id, &self.target.request, reason, || { (self.target.wake)() }); diff --git a/verification/planted/patches/16-mon-dead-target-no-down.patch b/verification/planted/patches/16-mon-dead-target-no-down.patch index 503cdc169..ca6058e68 100644 --- a/verification/planted/patches/16-mon-dead-target-no-down.patch +++ b/verification/planted/patches/16-mon-dead-target-no-down.patch @@ -1,12 +1,12 @@ Planted bug 16 (mon-dead-target-no-down): Monitoring an already-dead actor returns an inert ref and no Down diff --git a/concurrency/src/monitor/fence.rs b/concurrency/src/monitor/fence.rs -index f7bbaa27..d40d7c7e 100644 +index 1b7006f0..d82513c4 100644 --- a/concurrency/src/monitor/fence.rs +++ b/concurrency/src/monitor/fence.rs -@@ -70,10 +70,8 @@ pub(crate) fn register( +@@ -72,10 +72,8 @@ pub(crate) fn register( return Registration::Inert; - }; + } if target_dead() { - late(); - entries.insert(key, Entry::Queued); diff --git a/verification/planted/patches/18-reg-not-removed-on-death.patch b/verification/planted/patches/18-reg-not-removed-on-death.patch index 21df9e455..736c5e799 100644 --- a/verification/planted/patches/18-reg-not-removed-on-death.patch +++ b/verification/planted/patches/18-reg-not-removed-on-death.patch @@ -1,15 +1,15 @@ Planted bug 18 (reg-not-removed-on-death): A managed actor's registered name is never removed when its scope ends diff --git a/concurrency/src/system.rs b/concurrency/src/system.rs -index 0de9b0f5..c6457430 100644 +index 5e3a8933..a82b5c5c 100644 --- a/concurrency/src/system.rs +++ b/concurrency/src/system.rs -@@ -1233,7 +1233,7 @@ impl Drop for Lease { +@@ -1418,7 +1418,7 @@ impl Drop for Lease { } } if last { - drop(registrations); + std::mem::forget(registrations); - if let Some(g) = self.scope.group.upgrade() { - g.remove_scope(self.scope.id); - } + // The scope's owned work is done (`Start.tla` WorkDoneC), then its + // slot is released under the group lock (ReleaseSlot). + #[cfg(feature = "lifecycle-trace")] diff --git a/verification/planted/patches/19-mb-handle-after-stop.patch b/verification/planted/patches/19-mb-handle-after-stop.patch index a9af41409..eb3fe4f8a 100644 --- a/verification/planted/patches/19-mb-handle-after-stop.patch +++ b/verification/planted/patches/19-mb-handle-after-stop.patch @@ -1,17 +1,17 @@ Planted bug 19 (mb-handle-after-stop): The run loop keeps handling queued messages after stop (D4) diff --git a/concurrency/src/tasks/actor/lifecycle.rs b/concurrency/src/tasks/actor/lifecycle.rs -index a88489ac..443f67ff 100644 +index c05182ba..a4cafd84 100644 --- a/concurrency/src/tasks/actor/lifecycle.rs +++ b/concurrency/src/tasks/actor/lifecycle.rs -@@ -443,9 +443,7 @@ pub(super) async fn run_actor_lifecycle( - #[cfg(all(test, not(feature = "lifecycle-trace"), not(spawned_sim)))] - stop_receive_probe::before_receive(); - let received = rx.poll_recv(cx); -- let result = cancellation_token.after_receive(received, |item| { -- rx.hold_for_shutdown(item); -- }); -+ let result = received; - #[cfg(feature = "lifecycle-trace")] - if let std::task::Poll::Ready(Some(envelope)) = &result { - crate::lifecycle_trace::life_emit(trace_id, || { +@@ -901,9 +901,7 @@ pub(super) async fn run_actor_lifecycle( + #[cfg(all(test, not(feature = "lifecycle-trace"), not(spawned_sim)))] + stop_receive_probe::before_receive(); + let received = rx.poll_recv(cx); +- let result = cancellation_token.after_receive(received, |item| { +- rx.hold_for_shutdown(item); +- }); ++ let result = received; + #[cfg(feature = "lifecycle-trace")] + if let std::task::Poll::Ready(Some(envelope)) = &result { + crate::lifecycle_trace::life_emit(trace_id, || { diff --git a/verification/tla/MCStart.cfg b/verification/tla/MCStart.cfg index 7cb84c1dd..2b6854ce7 100644 --- a/verification/tla/MCStart.cfg +++ b/verification/tla/MCStart.cfg @@ -10,6 +10,7 @@ CONSTANTS KillEnabled = FALSE AdminStopEnabled = FALSE ParentCanExit = TRUE + ParentKillEnabled = FALSE CloseEnabled = FALSE OwnerCloseEnabled = FALSE MaxMsgs = 0 @@ -17,4 +18,4 @@ CONSTANTS UnlinkEnabled = FALSE ScopeEnabled = FALSE Fault = "None" -INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned +INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal ReadyReturnCommitsReady TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned diff --git a/verification/tla/MCStartCap.cfg b/verification/tla/MCStartCap.cfg index 2895f0173..0bf67c36a 100644 --- a/verification/tla/MCStartCap.cfg +++ b/verification/tla/MCStartCap.cfg @@ -10,6 +10,7 @@ CONSTANTS KillEnabled = FALSE AdminStopEnabled = FALSE ParentCanExit = FALSE + ParentKillEnabled = FALSE CloseEnabled = TRUE OwnerCloseEnabled = FALSE MaxMsgs = 0 @@ -17,4 +18,4 @@ CONSTANTS UnlinkEnabled = FALSE ScopeEnabled = FALSE Fault = "None" -INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned +INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal ReadyReturnCommitsReady TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned diff --git a/verification/tla/MCStartContinue.cfg b/verification/tla/MCStartContinue.cfg index b64be8b8c..550d7b6c5 100644 --- a/verification/tla/MCStartContinue.cfg +++ b/verification/tla/MCStartContinue.cfg @@ -10,6 +10,7 @@ CONSTANTS KillEnabled = FALSE AdminStopEnabled = FALSE ParentCanExit = TRUE + ParentKillEnabled = FALSE CloseEnabled = TRUE OwnerCloseEnabled = FALSE MaxMsgs = 2 @@ -17,4 +18,4 @@ CONSTANTS UnlinkEnabled = FALSE ScopeEnabled = FALSE Fault = "None" -INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned +INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal ReadyReturnCommitsReady TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned diff --git a/verification/tla/MCStartDeep.cfg b/verification/tla/MCStartDeep.cfg index 321826326..4e12b3cd2 100644 --- a/verification/tla/MCStartDeep.cfg +++ b/verification/tla/MCStartDeep.cfg @@ -10,6 +10,7 @@ CONSTANTS KillEnabled = FALSE AdminStopEnabled = FALSE ParentCanExit = TRUE + ParentKillEnabled = FALSE CloseEnabled = FALSE OwnerCloseEnabled = FALSE MaxMsgs = 0 @@ -17,4 +18,4 @@ CONSTANTS UnlinkEnabled = FALSE ScopeEnabled = FALSE Fault = "None" -INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned +INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal ReadyReturnCommitsReady TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned diff --git a/verification/tla/MCStartLive.cfg b/verification/tla/MCStartLive.cfg index f3d31d93e..56a507b2a 100644 --- a/verification/tla/MCStartLive.cfg +++ b/verification/tla/MCStartLive.cfg @@ -10,6 +10,7 @@ CONSTANTS KillEnabled = FALSE AdminStopEnabled = FALSE ParentCanExit = TRUE + ParentKillEnabled = FALSE CloseEnabled = FALSE OwnerCloseEnabled = FALSE MaxMsgs = 0 @@ -17,6 +18,6 @@ CONSTANTS UnlinkEnabled = FALSE ScopeEnabled = FALSE Fault = "None" -INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned +INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal ReadyReturnCommitsReady TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned PROPERTY ParentExitReachesChild PROPERTY StartAnswered diff --git a/verification/tla/MCStartNames.cfg b/verification/tla/MCStartNames.cfg index f1083abc1..e6b8efcd2 100644 --- a/verification/tla/MCStartNames.cfg +++ b/verification/tla/MCStartNames.cfg @@ -10,6 +10,7 @@ CONSTANTS KillEnabled = FALSE AdminStopEnabled = FALSE ParentCanExit = FALSE + ParentKillEnabled = FALSE CloseEnabled = TRUE OwnerCloseEnabled = FALSE MaxMsgs = 0 @@ -17,4 +18,4 @@ CONSTANTS UnlinkEnabled = FALSE ScopeEnabled = FALSE Fault = "None" -INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned +INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal ReadyReturnCommitsReady TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned diff --git a/verification/tla/MCStartScope.cfg b/verification/tla/MCStartScope.cfg index 330a894ce..9925f25a1 100644 --- a/verification/tla/MCStartScope.cfg +++ b/verification/tla/MCStartScope.cfg @@ -10,6 +10,7 @@ CONSTANTS KillEnabled = FALSE AdminStopEnabled = FALSE ParentCanExit = TRUE + ParentKillEnabled = FALSE CloseEnabled = FALSE OwnerCloseEnabled = TRUE MaxMsgs = 0 @@ -17,5 +18,5 @@ CONSTANTS UnlinkEnabled = TRUE ScopeEnabled = TRUE Fault = "None" -INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned +INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal ReadyReturnCommitsReady TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned PROPERTY OwnedScopeFinishes diff --git a/verification/tla/MCStartTimeout.cfg b/verification/tla/MCStartTimeout.cfg index bd21be393..c557b139f 100644 --- a/verification/tla/MCStartTimeout.cfg +++ b/verification/tla/MCStartTimeout.cfg @@ -10,6 +10,7 @@ CONSTANTS KillEnabled = TRUE AdminStopEnabled = TRUE ParentCanExit = TRUE + ParentKillEnabled = TRUE CloseEnabled = TRUE OwnerCloseEnabled = FALSE MaxMsgs = 0 @@ -17,4 +18,4 @@ CONSTANTS UnlinkEnabled = FALSE ScopeEnabled = FALSE Fault = "None" -INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned +INVARIANT TypeOK InheritedPlacement LinkBeforeStarted EdgeOnlyToRunningParent NoEdgeToDeadParent NameUnique NameHeldUntilReleased NameKeptWhileLive NoOverlapOnReuse NameFreeBeforeObservation AbortInvisible CommitImmutable ResultMatchesCommit ReadyIsFinal ReadyReturnCommitsReady TimeoutNeverKillsReady ReadyOnlyWhileOpen FailureAfterPublication ReportedReasonFinal KillBeforeDIsKilled IgnoredExitsNormal PendingOnlyAfterDeadline FailureFlushed FailureEndsNonTrappingParent NormalFailureSparesParent TimeoutContained AdmittedOnlyOpen OwnedOnlyWhileOwnerOpen CapacityRespected ParentSlotHeld SlotHeldUntilReleased NoticeOnce NoLostNotice AbandonedNoticeDelivered ContinueFirst GenDoneMeansOwnedDone OwnerRefusalOnlyOnShutdown DeadlineKillsAllOwned diff --git a/verification/tla/README.md b/verification/tla/README.md index 6b0afb50d..b6d4c5647 100644 --- a/verification/tla/README.md +++ b/verification/tla/README.md @@ -86,7 +86,7 @@ runs real actors through the scenarios below and writes each trace as JSON; TLC then checks that each trace is a behavior of `Supervision.tla`. ```bash -cargo test -p spawned-concurrency --features lifecycle-trace,failpoints --test supervision_trace --test cohort_trace --test tree_trace --test nested_supervision --test coverage_trace --test job_trace --test reply_trace --test lifecycle_trace --test coordinator_trace +cargo test -p spawned-concurrency --features lifecycle-trace,failpoints --test supervision_trace --test cohort_trace --test tree_trace --test nested_supervision --test coverage_trace --test job_trace --test reply_trace --test lifecycle_trace --test start_trace --test coordinator_trace nix run .#check-tla -- --traces-only target/lifecycle-traces --negative-controls # or --traces DIR to also model-check nix develop -c python3 verification/tla/trace_check.py target/lifecycle-traces ``` @@ -189,7 +189,7 @@ Model actions without an arm are listed in the map's `[trace.unlogged]` with the reason. ```bash -cargo test -p spawned-concurrency --features lifecycle-trace,failpoints --test supervision_trace --test cohort_trace --test tree_trace --test nested_supervision --test coverage_trace --test job_trace --test reply_trace --test disposal_trace --test lifecycle_trace +cargo test -p spawned-concurrency --features lifecycle-trace,failpoints --test supervision_trace --test cohort_trace --test tree_trace --test nested_supervision --test coverage_trace --test job_trace --test reply_trace --test disposal_trace --test lifecycle_trace --test start_trace nix develop -c python3 verification/tla/trace_coverage.py --traces dedicated=target/lifecycle-traces \ --model-keys verification/tla/coverage-model.json.gz --baseline verification/tla/coverage-baseline.json # every test binary: each finished supervisor dumps its traces @@ -560,7 +560,7 @@ admission check with the push, D with the exit-table close. A recorded actor's sections take the process-wide trace section first, so the log is a linearization. `TraceLifecycle.tla` replays it with no unlogged steps; its header lists the mapping (actors `a1`, `a2` in spawn order, -`start_linked` parents as `LinkParent`, every send a request from `Ext`, +`start_link` parents as `LinkParent`, every send a request from `Ext`, folded request panics, the late `NoProc` notice of a non-trapping caller, and `Hook` "skip" accepted after a kill that arrived before `stopped()` began, which the model's `Kill` leaves out). Phase 0 (signals held during @@ -571,7 +571,7 @@ request answered `Killed` and a later `NoProc`, kill during `stopped()`, untrapped and `Normal` signals, trapped exits (also from an actor, then a kill), late link/try_link/monitor (`NoProc`), a late link ending a non-trapping caller, the unlink and demonitor fences between D and -publication, a monitor's `Down`, a `start_linked` parent exit to a trapping +publication, a monitor's `Down`, a `start_link` parent exit to a trapping child, a link notice ending a non-trapping peer, runtime drop (`Killed`), and one threads-mode run. Controls (`lifecycle_trace_controls`): `stopped()` after a kill or after an untrapped signal, `Kill` instead of `Killed`, @@ -837,21 +837,48 @@ five edited traces (a wake-up before its expiry, a payload run before the hand-off, a handed-off payload dropped while held, a required worker start not taken, and a worker running an item it never took). -## Unified start (0.9 design) +## Unified start (0.9) `Start.tla` backs the [unified start design](../../docs/design/unified-start.md) -(approved after three review rounds). It was written before the implementation, as the +(approved after three review rounds), written before the implementation, as the deferred-replies models were. Design section 15 lists what it covers, its -invariants and what it leaves out. It is TLC-only in -`verification/model-map.toml` until the implementation adds trace -validation (design section 16.2). +invariants and what it leaves out. + +**Change with the implementation.** A group slot is released when the +instance's scope completes, after publication and its owned work (the job +join, helpers): `ReleaseSlot` now follows `Publish` and `WorkDoneC`, and P's +slot is held until P is published and its work joined. The code holds a +thread group's slot until then because the slot is the native worker that +still runs the join; the design's release before publication would admit a +new actor onto a busy worker. Every per-PR configuration and all 40 controls +were rerun after the change. + +**Trace validation** (`TraceStart.tla`, schema `start-v1`). With +`lifecycle-trace`, `lifecycle_trace::record_start_parent(&p)` records, for +every start made from then on and for the parent P, one `StartEvent` per +model action at its linearization point (the table in `TraceStart.tla`'s +header and design 16.2): `SBegin`, `SCommit`/`SRefuse` under the commit's +locks, `SResolve` under the exit-request and cell locks, `SPublish` with the +notice's disposition under the cell lock, `SAnswer`/`SReady`/`SAbandon`, +`STimer`, `SDeadline`, `SExit`/`SDie`, `SReleaseName` under the registry +lock, `SWorkDone`/`SReleaseSlot` at the scope's completion, `PExitBegin`, +`PDie`, `PPublish` and `PNotice` for the parent. `concurrency/tests/start_trace.rs` +records 16 scenarios (link, monitor, failed and ignored starts, names, +continuations, the init timer, kills and stops during `started()`, the +parent's exit during a child's `started()`, abandonment, unlink, an +inherited thread group's capacity, threads mode); each waits for the events +it needs before taking the trace. `negative_checks.py` +(`start_trace_controls`) requires 15 edits of recorded traces to be rejected +(design 16.2's list). Coverage: all 28 arms (`coverage-baseline.json`, floor +28 in `model-map.toml`). Not mapped: scope ownership (S8: `OwnerClose`, +`ScopeDeadline`, `GenDone`), group closure, a second group `G2`. | Configuration | Run | Instance | |---|---|---| | `MCStart` | per PR | one child (`start_link`, `start_monitor`) of a managed parent that may trap and exit: every init outcome, every notice disposition, abandonment, the cleanup deadline | | `MCStartNames` | per PR | two named `start` children in a group with room for both and the parent: lookups, refusals, release, retry | | `MCStartCap` | per PR | three children spawned into P's group of three: capacity binds, slots are held until release, a retry gets the slot | -| `MCStartTimeout` | per PR | one child of each waiting variant, managed or unmanaged parent; the init timer, outside kills (also before D), administrative stops, group closure | +| `MCStartTimeout` | per PR | one child of each waiting variant, managed or unmanaged parent; the init timer, outside kills (also before D), administrative stops, group closure, a killed parent (its waits abandoned) | | `MCStartContinue` | per PR | one child of every variant with every explicit placement, two messages, trapping | | `MCStartScope` | per PR | a managed parent and one linked child (owned in `G` or `G2`, or unmanaged), trapping, unlink, owner closure; P's D and publication, the polite Shutdown, the scope deadline; liveness `OwnedScopeFinishes` | | `MCStartLive` | per PR | one child of each waiting variant, trapping; liveness `ParentExitReachesChild`, `StartAnswered` | @@ -888,6 +915,7 @@ validation (design section 16.2). | `MCStartTimeout` | `ReadyAfterClose` | `ReadyOnlyWhileOpen` | | `MCStartTimeout` | `SkipEscalation` | `KillBeforeDIsKilled` | | `MCStartTimeout` | `SplitResolutionD` | `IgnoredExitsNormal` | +| `MCStartTimeout` | `AdminStopFailsInit` | `ReadyReturnCommitsReady` | | `MCStartCap` | `AdmitClosed` | `AdmittedOnlyOpen` | | `MCStartCap` | `IgnoreCap` | `CapacityRespected` | | `MCStartCap` | `EarlySlotRelease` | `SlotHeldUntilReleased` | diff --git a/verification/tla/Start.tla b/verification/tla/Start.tla index 1b07b73f6..03708c6d2 100644 --- a/verification/tla/Start.tla +++ b/verification/tla/Start.tla @@ -40,6 +40,7 @@ CONSTANTS KillEnabled, \* an outside kill (ChildHandle::kill) may hit a child AdminStopEnabled, \* an administrative stop may hit a child in started() ParentCanExit, \* P may exit (Normal or Panic) + ParentKillEnabled,\* P's exit may be a kill (its start waits are abandoned) CloseEnabled, \* G may close (system shutdown) OwnerCloseEnabled,\* P's scope may close (its supervisor's shutdown began) MaxMsgs, \* messages others send to each child @@ -69,6 +70,7 @@ VARIABLES phase, place, admOpen, owned, ownOpenAt, readyOpen, edgeWhenRun, edge, mon, linkEver, monEver, began, killed, skill, xkill, astop, pex, qpex, pexAbn, ctrap, + rdyRet, \* [Kids -> started() returned ready with nothing that fails the start] commit, commit0, initR, why, wait, res, resR, pend, dl, oref, closedBy, oref, \* [Kids -> readiness refused because the owner gate was closed] closedBy, \* who closed the owner gate: "none", "ancestor" (or "parent" under a fault) @@ -81,7 +83,7 @@ pV == <> sV == <> eV == <> -xV == <> +xV == <> cV == <> nV == <> mV == <> @@ -91,7 +93,10 @@ vars == <> Waiting(k) == k \in {"start", "link", "monitor"} LiveP == {"starting", "ready", "exiting", "dead"} \* holds name and slot -PSlot == IF pplace = "G" /\ pst # "published" /\ ~(Fault = "ParentSlotAtD" /\ pst = "dead") +\* P's slot is held until its scope completes: P published and its owned +\* work joined (WorkDoneP), as a child's (ReleaseSlot). +PSlot == IF pplace = "G" /\ ~(pst = "published" /\ pwork) + /\ ~(Fault = "ParentSlotAtD" /\ pst \in {"dead", "published"}) THEN 1 ELSE 0 Used == Cardinality(slots) + PSlot @@ -122,7 +127,7 @@ TypeOK == /\ linkEver \in [Kids -> BOOLEAN] /\ monEver \in [Kids -> BOOLEAN] /\ began \in [Kids -> BOOLEAN] /\ killed \in [Kids -> BOOLEAN] /\ skill \in [Kids -> BOOLEAN] /\ xkill \in [Kids -> BOOLEAN] - /\ astop \in [Kids -> BOOLEAN] + /\ astop \in [Kids -> BOOLEAN] /\ rdyRet \in [Kids -> BOOLEAN] /\ pex \in [Kids -> Reasons \cup {None}] /\ qpex \in [Kids -> Reasons \cup {None}] /\ pexAbn \in [Kids -> BOOLEAN] /\ ctrap \in [Kids -> BOOLEAN] /\ commit \in [Kids -> Commits] /\ commit0 \in [Kids -> Commits] @@ -155,7 +160,7 @@ Init == /\ edgeWhenRun = F(FALSE) /\ edge = F(FALSE) /\ mon = F(FALSE) /\ linkEver = F(FALSE) /\ monEver = F(FALSE) /\ began = F(FALSE) /\ killed = F(FALSE) /\ skill = F(FALSE) /\ xkill = F(FALSE) - /\ astop = F(FALSE) /\ pex = F(None) /\ qpex = F(None) /\ pexAbn = F(FALSE) + /\ astop = F(FALSE) /\ rdyRet = F(FALSE) /\ pex = F(None) /\ qpex = F(None) /\ pexAbn = F(FALSE) /\ ctrap = F(FALSE) /\ commit = F(None) /\ commit0 = F(None) /\ initR = F(None) /\ why = F(None) /\ wait = F("none") /\ res = F("none") /\ resR = F(None) /\ pend = F(FALSE) @@ -320,7 +325,7 @@ ResolveD(c, r) == StartedBegin(c) == /\ phase[c] = "starting" /\ ~began[c] /\ ~killed[c] /\ ~skill[c] /\ ~xkill[c] /\ began' = [began EXCEPT ![c] = TRUE] - /\ UNCHANGED <> LateLink(c) == @@ -338,12 +343,12 @@ OutsideKill(c) == \* Before D a kill makes the reason Killed at once (L3). /\ why' = IF phase[c] = "exiting" /\ Fault # "SkipEscalation" THEN [why EXCEPT ![c] = "Killed"] ELSE why - /\ UNCHANGED <> AdminStop(c) == /\ AdminStopEnabled /\ ~astop[c] /\ phase[c] = "starting" /\ astop' = [astop EXCEPT ![c] = TRUE] - /\ UNCHANGED <> AnyKill(c) == killed[c] \/ skill[c] \/ xkill[c] @@ -358,13 +363,18 @@ KilledEarly(c) == pubAlive, div, divEver, handed, lost, mV, seen, blamed, dV>> \* started() returns; design 2.3 precedence. No failed row runs stopped(). -StartedEnd(c) == +\* started() returns one of `os` (trace validation pins the recorded one). +StartedEndOf(c, os) == /\ phase[c] = "starting" /\ began[c] - /\ \E o \in Outcomes, t \in (IF ChildCanTrap THEN BOOLEAN ELSE {FALSE}), - first \in {"admin", "signal"}: + /\ \E o \in os, t \in (IF ChildCanTrap THEN BOOLEAN ELSE {FALSE}): + \* Row 3 is a held untrapped non-Normal signal. An administrative stop + \* (astop) does not decide the start: as OTP's gen_server:stop it is + \* applied after a ready commitment (AdminStopApply) and stopped() + \* runs. AdminStopFailsInit (negative control) restores revision 3's + \* row 3, which failed the start. LET sig == pex[c] \in NonNormal /\ ~t /\ Fault # "DropHeld" - row3 == astop[c] \/ sig - r3 == IF astop[c] /\ (~sig \/ first = "admin") THEN "Shutdown" ELSE pex[c] + row3 == sig \/ (astop[c] /\ Fault = "AdminStopFailsInit") + r3 == IF sig THEN pex[c] ELSE "Shutdown" closed == (place[c] = "G" /\ ~gopen) \/ (owned[c] /\ ~ownerOpen) failWith(r) == /\ ResolveD(c, r) /\ initR' = [initR EXCEPT ![c] = r] @@ -372,6 +382,8 @@ StartedEnd(c) == ELSE UNCHANGED <> /\ UNCHANGED <> IN /\ ctrap' = [ctrap EXCEPT ![c] = t] + /\ rdyRet' = [rdyRet EXCEPT ![c] = o \in {"ready", "continue"} /\ ~AnyKill(c) + /\ ~sig /\ ~closed] /\ pex' = [pex EXCEPT ![c] = None] /\ IF AnyKill(c) THEN failWith("Killed") ELSE IF row3 THEN failWith(r3) @@ -393,7 +405,9 @@ StartedEnd(c) == ELSE /\ phase' = [phase EXCEPT ![c] = "ready"] /\ SetCommit(c, "ready") /\ readyOpen' = [readyOpen EXCEPT ![c] = ~closed] - /\ cont' = [cont EXCEPT ![c] = o = "continue"] + \* A held administrative stop is observed before the + \* continuation begins, which discards it. + /\ cont' = [cont EXCEPT ![c] = o = "continue" /\ ~astop[c]] /\ qpex' = [qpex EXCEPT ![c] = IF t THEN pex[c] ELSE None] /\ UNCHANGED <> /\ UNCHANGED <> +StartedEnd(c) == StartedEndOf(c, Outcomes) + Send(c) == /\ phase[c] \in {"starting", "ready"} /\ msgs[c] < MaxMsgs /\ msgs' = [msgs EXCEPT ![c] = msgs[c] + 1] /\ UNCHANGED <> RunContinue(c) == - /\ phase[c] = "ready" /\ cont[c] + /\ phase[c] = "ready" /\ cont[c] /\ ~astop[c] /\ cont' = [cont EXCEPT ![c] = FALSE] /\ UNCHANGED <> Handle(c) == - /\ phase[c] = "ready" /\ msgs[c] > 0 + /\ phase[c] = "ready" /\ msgs[c] > 0 /\ ~astop[c] /\ ~cont[c] \/ Fault = "ContinueQueued" /\ msgs' = [msgs EXCEPT ![c] = msgs[c] - 1] /\ early' = [early EXCEPT ![c] = early[c] \/ cont[c]] /\ UNCHANGED <> HandleQueuedExit(c) == - /\ phase[c] = "ready" /\ qpex[c] # None + /\ phase[c] = "ready" /\ qpex[c] # None /\ ~astop[c] /\ ~cont[c] \/ Fault = "ContinueQueued" /\ early' = [early EXCEPT ![c] = early[c] \/ cont[c]] /\ qpex' = [qpex EXCEPT ![c] = None] /\ ExitK(c, qpex[c]) /\ UNCHANGED <> ApplyParentExit(c) == @@ -433,7 +449,7 @@ ApplyParentExit(c) == /\ pex' = [pex EXCEPT ![c] = None] /\ IF pex[c] \in NonNormal THEN ExitK(c, pex[c]) ELSE UNCHANGED <> /\ UNCHANGED <> ReadyExit(c) == @@ -448,6 +464,16 @@ KillReady(c) == /\ UNCHANGED <> +\* An administrative stop held during started() begins the ready child's +\* exit with the stop's reason; stopped() runs (OTP gen_server:stop). +AdminStopApply(c) == + /\ phase[c] = "ready" /\ astop[c] + /\ astop' = [astop EXCEPT ![c] = FALSE] + /\ ExitK(c, "Shutdown") + /\ UNCHANGED <> + ChildUnlink(c) == /\ UnlinkEnabled /\ phase[c] = "ready" /\ edge[c] /\ edge' = [edge EXCEPT ![c] = FALSE] @@ -462,8 +488,12 @@ Die(c) == linkEver, monEver, xV, cV, pubAlive, div, divEver, handed, lost, mV, seen, blamed, dV>> -\* After destruction: the name is released (registry lock), then the slot -\* (group lock); two atomic points, two trace events (design 5.2, 16.2). +\* After destruction the name is released (registry lock), before +\* publication. The slot (group lock) is released when the instance's scope +\* completes: after publication and its owned work (the job join, helpers), +\* because a thread group's slot is the native worker that still runs that +\* join. Two atomic points, two trace events (design 5.2, 16.2; the slot's +\* point moved after publication with the implementation). ReleaseName(c) == /\ phase[c] = "dead" /\ phase' = [phase EXCEPT ![c] = "namerel"] @@ -471,16 +501,14 @@ ReleaseName(c) == /\ UNCHANGED <> ReleaseSlot(c) == - /\ phase[c] = "namerel" - /\ phase' = [phase EXCEPT ![c] = "released"] + /\ c \in slots /\ phase[c] = "published" /\ cwork[c] /\ slots' = slots \ {c} - /\ UNCHANGED <> + /\ UNCHANGED <> \* Publication: each taken notice is disposed of by the commitment and the \* waiter (design 5.2); answers become final. Publish(c) == - /\ phase[c] = "released" + /\ phase[c] = "namerel" /\ phase' = [phase EXCEPT ![c] = "published"] /\ pubAlive' = [pubAlive EXCEPT ![c] = Alive(pst)] /\ LET r == why[c] @@ -549,7 +577,7 @@ Timeout(c) == /\ IF commit[c] = None \/ Fault = "RewriteCommit" THEN SetCommit(c, "timedout") ELSE UNCHANGED <> /\ killed' = [killed EXCEPT ![c] = TRUE] - /\ UNCHANGED <> \* The bounded wait for publication expires (measured from the commitment). @@ -620,7 +648,11 @@ Abandon(c) == \* ParentExitClosesOwner (negative control) closes the gate here. PExit == /\ ParentCanExit /\ pst = "run" - /\ \E r \in {"Normal", "Panic"} : pst' = "exiting" /\ pwhy' = r + \* A kill (or an untrapped signal) also abandons P's pending start waits + \* in the code (review #291 item 7; OTP: the caller dies at once): the + \* model's Abandon, enabled while P exits. + /\ \E r \in {"Normal", "Panic"} \cup (IF ParentKillEnabled THEN {"Killed"} ELSE {}) : + pst' = "exiting" /\ pwhy' = r /\ IF Fault = "ParentExitClosesOwner" /\ ownerOpen THEN ownerOpen' = FALSE /\ closedBy' = "parent" ELSE UNCHANGED <> @@ -665,7 +697,7 @@ PPublish == /\ phase[c] \in {"starting", "ready"})] /\ pn' = F(None) /\ UNCHANGED <> + oV, sV, eV, began, killed, skill, xkill, astop, rdyRet, ctrap, cV, nV, mV, rV, dV>> \* S8: from P's exit begin, the scope deadline (a timer, not assumed fair) \* kills every live owned child directly, exiting ones included. @@ -678,7 +710,7 @@ ScopeDeadline == /\ why' = [c \in Kids |-> IF owned[c] /\ phase[c] = "exiting" /\ Fault \notin {"NoScopeKill", "SkipEscalation"} THEN "Killed" ELSE why[c]] /\ UNCHANGED <> \* Owned work (helpers, jobs) is joined after publication (0.8 scope @@ -718,7 +750,7 @@ ChildStep(c) == \/ StartedBegin(c) \/ LateLink(c) \/ OutsideKill(c) \/ AdminStop(c) \/ KilledEarly(c) \/ StartedEnd(c) \/ Send(c) \/ RunContinue(c) \/ Handle(c) \/ HandleQueuedExit(c) - \/ ApplyParentExit(c) \/ ReadyExit(c) \/ KillReady(c) + \/ ApplyParentExit(c) \/ ReadyExit(c) \/ KillReady(c) \/ AdminStopApply(c) \/ ChildUnlink(c) \/ Die(c) \/ ReleaseName(c) \/ ReleaseSlot(c) \/ Publish(c) \/ Deliver(c) \/ LateRelease(c) \/ WorkDoneC(c) \/ Timeout(c) \/ CleanupDeadline(c) \/ AckOk(c) \/ AckFail(c) \/ Abandon(c) @@ -731,7 +763,7 @@ Next == (\E c \in Kids : ChildStep(c)) \/ Lookup \/ PExit \/ OwnerClose \/ PDie \* and the cleanup deadline are not assumed to fire. Progress(c) == \/ StartedBegin(c) \/ KilledEarly(c) \/ StartedEnd(c) \/ RunContinue(c) - \/ HandleQueuedExit(c) \/ ApplyParentExit(c) \/ KillReady(c) + \/ HandleQueuedExit(c) \/ ApplyParentExit(c) \/ KillReady(c) \/ AdminStopApply(c) \/ Die(c) \/ ReleaseName(c) \/ ReleaseSlot(c) \/ Publish(c) \/ Deliver(c) \/ AckOk(c) \/ WorkDoneC(c) Fairness == /\ \A c \in Kids : /\ WF_vars(Progress(c)) /\ WF_vars(Commit(c)) @@ -789,6 +821,10 @@ ResultMatchesCommit == /\ res[c] = "ignored" => commit0[c] = "ignored" /\ res[c] = "timeout" => commit0[c] = "timedout" +\* started() returned ready and no kill, held signal or closure failed it: +\* the start commits ready (an administrative stop waits for it; OTP). +ReadyReturnCommitsReady == \A c \in Kids : rdyRet[c] => commit0[c] = "ready" + ReadyIsFinal == \A c \in Kids : commit0[c] = "ready" => res[c] \notin {"failed", "ignored", "timeout"} TimeoutNeverKillsReady == \A c \in Kids : commit0[c] = "ready" => ~killed[c] diff --git a/verification/tla/TraceStart.tla b/verification/tla/TraceStart.tla new file mode 100644 index 000000000..d657e2777 --- /dev/null +++ b/verification/tla/TraceStart.tla @@ -0,0 +1,229 @@ +------------------------------ MODULE TraceStart ------------------------------ +(***************************************************************************) +(* Trace validation for Start.tla (unified start, 0.9; design *) +(* docs/design/unified-start.md, 16.2). *) +(* *) +(* With the `lifecycle-trace` feature, spawned-concurrency records one *) +(* event per model action for the starts made while `record_start` is on *) +(* (concurrency/src/lifecycle_trace.rs, StartEvent), at the code's *) +(* linearization point of the state change it describes: *) +(* SBegin entry of a start verb (kind, explicit placement, named) *) +(* SPrepareFail invalid options, before anything exists (PrepareFail) *) +(* SCommit the stage-1 commit, all its locks held (Commit) *) +(* SRefuse a stage-1 refusal decided under the same locks (Commit) *) +(* SLookup Name::whereis under the registry lock (Lookup; a miss is *) +(* a stuttering step that requires the name free) *) +(* SStartedBegin before started() is first polled (StartedBegin); also *) +(* for a managed child whose admission closed before it: *) +(* started() is skipped and the start resolves as row 9 *) +(* SAdminStop, SKill an administrative stop or kill arriving (exit- *) +(* request lock: AdminStop, OutsideKill) *) +(* SResolve the startup decision under the exit-request and cell *) +(* locks (StartedEnd; KilledEarly when started() never ran) *) +(* STimer the init timer under the cell lock (Timeout; "lost" is *) +(* a stuttering step that requires a commitment) *) +(* SDeadline the waiter's cleanup deadline (CleanupDeadline) *) +(* SSend, SHandle, SContinue admission of a message, its dispatch, the *) +(* continuation's turn (Send, Handle, RunContinue) *) +(* SExit a ready child's exit begins (ReadyExit, KillReady, *) +(* ApplyParentExit, HandleQueuedExit; or AdminStopApply: *) +(* an administrative stop held during started(), recorded *) +(* right after the ready SResolve) *) +(* SUnlink the child unlinks from P (ChildUnlink) *) +(* SDie D (Die); f = TRUE: the D of a non-ready decision, which *) +(* StartedEnd already took (a stuttering step whose reason *) +(* must match) *) +(* SReleaseName the name's release under the registry lock *) +(* SPublish the notice disposition under the cell lock (Publish) *) +(* SAnswer the waiter takes its answer (AckOk, AckFail) *) +(* SReady ready() returns (a stuttering step that requires the *) +(* matching commitment) *) +(* SAbandon the Starting is dropped unanswered (Abandon) *) +(* SWorkDone, SReleaseSlot the scope's completion: owned work joined, *) +(* then the slot released under the group lock *) +(* PExitBegin, PDie, PPublish the parent's exit begin (not one begun *) +(* by a child's notice), D and publication *) +(* PNotice a child's notice enqueued to or applied on P (Deliver) *) +(* *) +(* Mapping notes: *) +(* - ReleaseName of an unnamed child changes nothing observable and has *) +(* no event: it is taken silently. So is WorkDoneP. *) +(* - The code decides a notice's disposition at publication whether or *) +(* not P is alive; the model only for a live P. SPublish's "divert" is *) +(* checked against the model where P's mailbox admits. *) +(* - Not in the mapping: scope ownership (S8: OwnerClose, the scope *) +(* deadline, GenDone; ScopeEnabled = FALSE), group closure, and a *) +(* second group G2. *) +(* *) +(* The trace is accepted iff some behavior consumes the whole log *) +(* (TraceIncomplete violated). verification/tla/trace_check.py (schema *) +(* start-v1) generates the constants and runs TLC. *) +(***************************************************************************) +EXTENDS Start, Sequences, TLC + +CONSTANTS Trace, TracePlace, TraceTrap + +VARIABLE i + +traceVars == <> + +TraceInit == Init /\ pplace = TracePlace /\ ptrap = TraceTrap /\ i = 0 + +Kind(e) == e.x +ExplicitOf(e) == IF e.y = "none" THEN None ELSE e.y + +BeginAs(e) == + /\ Begin(e.c) + /\ kind'[e.c] = e.x /\ expl'[e.c] = ExplicitOf(e) /\ wname'[e.c] = e.f + +CommitAs(e) == + /\ Commit(e.c) + /\ IF e.a = "SCommit" + THEN phase'[e.c] = "starting" /\ place'[e.c] = e.x + ELSE phase'[e.c] = "aborted" /\ res'[e.c] = e.x + +LookupAs(e) == + IF e.x = "hit" THEN Lookup /\ seen'[e.c] /\ e.c \in reg + ELSE reg = {} /\ UNCHANGED vars + +\* The outcome code maps to the model's: a closed admission that skipped +\* started() resolves as "ready". +Outcome(e) == IF e.x = "none" THEN "ready" ELSE e.x + +\* What started() returned, as the model's outcomes: the recorded code +\* (`e.x`) pins it. A `ctx.stop*` called inside a started() that returned +\* ready is the model's stop outcome (row 5), whose reason `e.z` pins. +Returns(e) == + IF e.y = "stop" /\ e.x \in {"ready", "continue"} + THEN {"stopN", "stopS"} + ELSE {Outcome(e)} + +ResolveAs(e) == + IF e.x = "none" /\ e.y \in {"kill", "timer"} + THEN KilledEarly(e.c) + ELSE /\ StartedEndOf(e.c, Returns(e)) + /\ ctrap'[e.c] = e.f + /\ IF e.y = "ready" + THEN /\ phase'[e.c] = "ready" + /\ cont'[e.c] = (Outcome(e) = "continue" /\ ~astop[e.c]) + ELSE /\ phase'[e.c] = "dead" + /\ (e.z # "" => why'[e.c] = e.z) + /\ commit0'[e.c] = CASE e.y = "ignore" -> "ignored" + [] e.y = "timer" -> "timedout" + [] OTHER -> commit0'[e.c] + +TimerAs(e) == + IF e.x = "won" THEN Timeout(e.c) + ELSE commit[e.c] # None /\ UNCHANGED vars + +ExitAs(e) == + /\ \/ ReadyExit(e.c) \/ KillReady(e.c) \/ ApplyParentExit(e.c) \/ HandleQueuedExit(e.c) + \/ AdminStopApply(e.c) + /\ phase'[e.c] = "exiting" /\ why'[e.c] = e.x + +DieAs(e) == + IF e.f THEN phase[e.c] = "dead" /\ why[e.c] = e.x /\ UNCHANGED vars + ELSE Die(e.c) /\ why[e.c] = e.x + +\* The recorded disposition must be the model's: a diverted notice is +\* "divert", a notice the model hands to P is "deliver", and "drop" neither +\* diverts nor delivers. (For a dead P the code still records its decision; +\* the model has nothing to deliver, so only the implications below apply.) +PublishAs(e) == + /\ Publish(e.c) + /\ (div'[e.c] # None) => e.x = "divert" + /\ (e.x = "divert" /\ Alive(pst) /\ (lnote[e.c] \/ mnote[e.c])) => div'[e.c] # None + /\ (obN'[e.c] > obN[e.c]) => e.x = "deliver" + /\ e.x = "drop" => (obN'[e.c] = obN[e.c] /\ div'[e.c] = None) + +AnswerAs(e) == + IF e.x = "ok" THEN AckOk(e.c) + ELSE /\ AckFail(e.c) + /\ res'[e.c] = CASE e.x = "ignored" -> "ignored" + [] e.x = "timeout" -> "timeout" + [] OTHER -> "failed" + /\ pend'[e.c] = e.f + /\ (e.x = "failed" /\ e.z # "") => resR'[e.c] = e.z + +ReadyAs(e) == + /\ CASE e.x = "ok" -> commit0[e.c] = "ready" + [] e.x = "ignored" -> commit0[e.c] = "ignored" + [] e.x = "timeout" -> commit0[e.c] = "timedout" + [] OTHER -> commit0[e.c] = "failed" + /\ e.x # "ok" => (phase[e.c] = "published" \/ dl[e.c]) + /\ UNCHANGED vars + +AbandonAs(e) == Abandon(e.c) /\ (handed'[e.c] => e.f) + +NoticeAs(e) == + /\ Deliver(e.c) + /\ obR[e.c] = e.z + /\ (e.x = "monitor") = (kind[e.c] = "monitor") + +PExitAs(e) == PExit /\ pwhy' = e.x + +\* The model action named by logged event e, constrained by its parameters. +Step(e) == + CASE e.a = "SBegin" -> BeginAs(e) + [] e.a = "SPrepareFail" -> PrepareFail(e.c) + [] e.a = "SCommit" -> CommitAs(e) + [] e.a = "SRefuse" -> CommitAs(e) + [] e.a = "SLookup" -> LookupAs(e) + [] e.a = "SStartedBegin" -> StartedBegin(e.c) + [] e.a = "SAdminStop" -> AdminStop(e.c) + [] e.a = "SKill" -> OutsideKill(e.c) + [] e.a = "SResolve" -> ResolveAs(e) + [] e.a = "STimer" -> TimerAs(e) + [] e.a = "SDeadline" -> CleanupDeadline(e.c) + [] e.a = "SSend" -> Send(e.c) + [] e.a = "SHandle" -> Handle(e.c) + [] e.a = "SContinue" -> RunContinue(e.c) + [] e.a = "SExit" -> ExitAs(e) + [] e.a = "SUnlink" -> ChildUnlink(e.c) + [] e.a = "SDie" -> DieAs(e) + [] e.a = "SReleaseName" -> ReleaseName(e.c) + [] e.a = "SPublish" -> PublishAs(e) + [] e.a = "SAnswer" -> AnswerAs(e) + [] e.a = "SReady" -> ReadyAs(e) + [] e.a = "SAbandon" -> AbandonAs(e) + [] e.a = "SWorkDone" -> WorkDoneC(e.c) + [] e.a = "SReleaseSlot" -> ReleaseSlot(e.c) + [] e.a = "PExitBegin" -> PExitAs(e) + [] e.a = "PDie" -> PDie + [] e.a = "PPublish" -> PPublish + [] e.a = "PNotice" -> NoticeAs(e) + [] OTHER -> FALSE + +\* Unlogged steps: an unnamed child's name release, P's owned work. +Silent == + \/ \E c \in Kids : ~wname[c] /\ ReleaseName(c) + \/ WorkDoneP + +TraceNext == + /\ i < Len(Trace) + /\ \/ Step(Trace[i + 1]) /\ i' = i + 1 + \/ Silent /\ UNCHANGED i + +TraceSpec == TraceInit /\ [][TraceNext]_traceVars + +\* All the model's invariants hold along the trace. +Safety == + /\ TypeOK /\ InheritedPlacement /\ LinkBeforeStarted /\ EdgeOnlyToRunningParent + /\ NoEdgeToDeadParent /\ NameUnique /\ NameHeldUntilReleased /\ NameKeptWhileLive + /\ NoOverlapOnReuse /\ NameFreeBeforeObservation /\ AbortInvisible /\ CommitImmutable + /\ ResultMatchesCommit /\ ReadyIsFinal /\ ReadyReturnCommitsReady /\ TimeoutNeverKillsReady /\ ReadyOnlyWhileOpen + /\ FailureAfterPublication /\ ReportedReasonFinal /\ PendingOnlyAfterDeadline + /\ KillBeforeDIsKilled /\ IgnoredExitsNormal /\ FailureFlushed + /\ FailureEndsNonTrappingParent /\ NormalFailureSparesParent /\ TimeoutContained + /\ AdmittedOnlyOpen /\ OwnedOnlyWhileOwnerOpen /\ CapacityRespected /\ ParentSlotHeld + /\ SlotHeldUntilReleased /\ NoticeOnce /\ NoLostNotice /\ AbandonedNoticeDelivered + /\ ContinueFirst /\ OwnerRefusalOnlyOnShutdown + +\* Violated exactly when the whole log has been matched and nothing the +\* model owes a live P is still undelivered: TLC reporting this invariant +\* as violated means the trace is ACCEPTED. (A recording ends after the +\* notices it waits for, so a disposed notice never delivered is a lost one.) +TraceIncomplete == + \/ i < Len(Trace) + \/ \E c \in Kids : obN[c] > 0 /\ Alive(pst) +============================================================================= diff --git a/verification/tla/coverage-baseline.json b/verification/tla/coverage-baseline.json index 8409ffb20..fdb0739fb 100644 --- a/verification/tla/coverage-baseline.json +++ b/verification/tla/coverage-baseline.json @@ -139,6 +139,36 @@ "A|Shorten", "A|Stuck" ], + "Start": [ + "A|PDie", + "A|PExitBegin", + "A|PNotice", + "A|PPublish", + "A|SAbandon", + "A|SAdminStop", + "A|SAnswer", + "A|SBegin", + "A|SCommit", + "A|SContinue", + "A|SDeadline", + "A|SDie", + "A|SExit", + "A|SHandle", + "A|SKill", + "A|SLookup", + "A|SPrepareFail", + "A|SPublish", + "A|SReady", + "A|SRefuse", + "A|SReleaseName", + "A|SReleaseSlot", + "A|SResolve", + "A|SSend", + "A|SStartedBegin", + "A|STimer", + "A|SUnlink", + "A|SWorkDone" + ], "Coordinator": [ "A|Advance", "A|Begin", diff --git a/verification/tla/negative_checks.py b/verification/tla/negative_checks.py index f4e6ee91b..6c99d29ff 100644 --- a/verification/tla/negative_checks.py +++ b/verification/tla/negative_checks.py @@ -333,6 +333,7 @@ def named_temporal(module, fault, prop, tlc, constants=None): ("MCStartTimeout", "ReadyAfterClose", "ReadyOnlyWhileOpen"), ("MCStartTimeout", "SkipEscalation", "KillBeforeDIsKilled"), ("MCStartTimeout", "SplitResolutionD", "IgnoredExitsNormal"), + ("MCStartTimeout", "AdminStopFailsInit", "ReadyReturnCommitsReady"), ("MCStartCap", "AdmitClosed", "AdmittedOnlyOpen"), ("MCStartCap", "IgnoreCap", "CapacityRespected"), ("MCStartCap", "EarlySlotRelease", "SlotHeldUntilReleased"), @@ -358,6 +359,152 @@ def start_controls(tlc): for module, fault, prop in START_TEMPORAL_CONTROLS: named_temporal(module, fault, prop, tlc) + +def start_trace_controls(traces, tlc): + """Edits of recorded unified-start traces that Start.tla must reject + (design 16.2): each reintroduces a bug the start protocol excludes.""" + by_name = {t["name"]: t for t in traces if t.get("schema") == "start-v1"} + + def at(events, action, x=None, after=0, nth=1): + hits = [i for i, e in enumerate(events) + if i >= after and e["a"] == action and (x is None or e["x"] == x)] + return hits[nth - 1] + + def move(events, src, dst): + events.insert(dst, events.pop(src)) + + def commit_after_started(events): + move(events, at(events, "SCommit"), at(events, "SStartedBegin")) + require_rejected(by_name["start_link_ready_then_stop"], commit_after_started, + "edge committed after started() began", tlc) + + def overlapping_names(events): + refused = events[at(events, "SRefuse", "already")] + refused["a"], refused["x"] = "SCommit", "U" + require_rejected(by_name["start_named_start_clash_and_release"], overlapping_names, + "two children hold one name", tlc) + + def release_after_competitor(events): + release = at(events, "SReleaseName") + competitor = at(events, "SCommit", after=release, nth=1) + move(events, release, competitor) + require_rejected(by_name["start_named_start_clash_and_release"], release_after_competitor, + "name released after a competing commit of it", tlc) + + def slot_before_publish(events): + move(events, at(events, "SReleaseSlot"), at(events, "SPublish")) + require_rejected(by_name["start_inherited_thread_group_capacity"], slot_before_publish, + "slot released before publication", tlc) + + def notice_without_delivery(events): + c = events[at(events, "SAnswer")]["c"] + events.append({"a": "PNotice", "c": c, "x": "monitor", "y": "", "z": "Normal", "f": False}) + require_rejected(by_name["start_monitor_error_consumes_down"], notice_without_delivery, + "Down of a failed start_monitor delivered", tlc) + + def second_notice(events): + notice = at(events, "PNotice") + events.insert(notice + 1, dict(events[notice])) + require_rejected(by_name["start_link_ready_then_stop"], second_notice, + "one notice delivered twice", tlc) + + def managed_as_unmanaged(events): + events[at(events, "SCommit", "G")]["x"] = "U" + require_rejected(by_name["start_inherited_thread_group_capacity"], managed_as_unmanaged, + "linked child not in its parent's group", tlc) + + def exit_after_error(events): + answer = events[at(events, "SAnswer", "failed")] + events.append({"a": "PNotice", "c": answer["c"], "x": "link", "y": "", + "z": "Panic", "f": False}) + require_rejected(by_name["start_link_panic_in_started_trapping_parent"], exit_after_error, + "Exit to a trapping parent after the start's error", tlc) + + def answer_before_publication(events): + move(events, at(events, "SAnswer", "failed"), at(events, "SPublish")) + require_rejected(by_name["start_monitor_error_consumes_down"], answer_before_publication, + "error answered before publication without a deadline", tlc) + + def timer_after_ready(events): + resolved = at(events, "SResolve", "ready") + c = events[resolved]["c"] + events.insert(resolved + 1, {"a": "STimer", "c": c, "x": "won", "y": "", "z": "", + "f": False}) + require_rejected(by_name["start_link_ready_then_stop"], timer_after_ready, + "init timer wins after a ready commitment", tlc) + + def requested_reason_reported(events): + events[at(events, "SAnswer", "failed")]["z"] = "Normal" + require_rejected(by_name["start_kill_and_admin_stop_during_started"], + requested_reason_reported, + "answer reports a reason other than the final Killed", tlc) + + def admin_stop_fails_start(events): + # Revision 3's row 3: a stop during started() failed the start. + stop = at(events, "SAdminStop") + answer = events[at(events, "SAnswer", "ok", after=stop)] + answer["x"], answer["z"] = "failed", "Shutdown" + require_rejected(by_name["start_kill_and_admin_stop_during_started"], + admin_stop_fails_start, + "an administrative stop during started() fails the start", tlc) + + def resolve_ignores_return(events): + # Review #291 item 2a: started() returned ready, yet the start + # resolved as an error (the decision must follow the return). + events[at(events, "SResolve", "error")]["x"] = "ready" + require_rejected(by_name["start_monitor_error_consumes_down"], + resolve_ignores_return, + "a ready return resolved as an error", tlc) + + def deliver_as_drop(events): + # Review #291 item 2b/2c: the Normal notice owed to P is dropped. + events[at(events, "SPublish", "deliver")]["x"] = "drop" + del events[at(events, "PNotice")] + require_rejected(by_name["start_link_ready_then_stop"], deliver_as_drop, + "a delivered notice recorded as dropped (and never delivered)", tlc) + + def notice_never_delivered(events): + # Item 2c: the log ends with the notice disposed but not delivered. + del events[at(events, "PNotice")] + require_rejected(by_name["start_link_ready_then_stop"], notice_never_delivered, + "a disposed notice never delivered to the live P", tlc) + + def dispatch_failure_answered_ready(events): + # Review #291 item 1: the committed child the dispatch lost must + # resolve killed; a waiter answered `ok` with no ready child is wrong. + events[at(events, "SAnswer", "failed")]["x"] = "ok" + require_rejected(by_name["start_dispatch_failure_after_commit_kills_child"], + dispatch_failure_answered_ready, + "a lost dispatch answers the start ok", tlc) + + def dispatch_failure_unkilled(events): + del events[at(events, "SKill")] + require_rejected(by_name["start_dispatch_failure_after_commit_kills_child"], + dispatch_failure_unkilled, + "a lost dispatch resolves killed without a kill", tlc) + + def abandon_drops_diverted(events): + events[at(events, "SAbandon")]["f"] = False + require_rejected(by_name["start_abandon_after_publication_hands_on_exit"], + abandon_drops_diverted, "abandoned wait drops the diverted Exit", tlc) + + def continuation_after_message(events): + move(events, at(events, "SContinue"), at(events, "SHandle") + 1) + require_rejected(by_name["start_continuation_precedes_queued_messages"], + continuation_after_message, "a message handled before the continuation", tlc) + + def stopped_after_failed_init(events): + # v0.8: a ctx.stop in started() was a ready start that then exited. + resolved = events[at(events, "SResolve", "stopS")] + resolved["x"], resolved["y"], resolved["z"] = "ready", "ready", "" + require_rejected(by_name["start_link_stop_in_started_ends_nontrapping_parent"], + stopped_after_failed_init, "Init::stop answered as a ready start", tlc) + + def pending_without_deadline(events): + events.pop(at(events, "SDeadline")) + require_rejected(by_name["start_stuck_ignore_answers_pending"], pending_without_deadline, + "pending answer before the cleanup deadline", tlc) + INTROSPECTION_CONTROLS = [ ("MCIntrospection", "CheckThenAct", "NoChargeFinishedTurn"), ("MCIntrospection", "FailedCasKills", "NoChargeFinishedTurn"), @@ -853,6 +1000,7 @@ def decide_at_once(events): reply_trace_controls(traces, args.tlc) disposal_trace_controls(traces, args.tlc, args.traces) lifecycle_trace_controls(traces, args.tlc) + start_trace_controls(traces, args.tlc) for module, fault, invariant in LIFECYCLE_CONTROLS: named_fault(module, fault, invariant, args.tlc) for module, fault, invariant in INTROSPECTION_CONTROLS: diff --git a/verification/tla/trace_check.py b/verification/tla/trace_check.py index 3d14e4f42..119678f8e 100644 --- a/verification/tla/trace_check.py +++ b/verification/tla/trace_check.py @@ -50,8 +50,10 @@ def tla_event(event, cohort=False): def generate(trace, workdir): schema = trace.get("schema", "one-for-one-v1") if schema not in {"one-for-one-v1", "cohort-v1", "tree-v1", "jobs-v1", "replies-v1", - "lifecycle-v1", "coordinator-v1", "disposal-v1"}: + "lifecycle-v1", "start-v1", "coordinator-v1", "disposal-v1"}: raise ValueError(f"unsupported trace schema: {schema}") + if schema == "start-v1": + return generate_start(trace, workdir) if schema == "tree-v1": return generate_tree(trace, workdir) if schema == "jobs-v1": @@ -356,6 +358,64 @@ def tla_set(values): "INVARIANT Safety\nINVARIANT TraceIncomplete\n") +def start_records(trace): + """Normalize a start-v1 trace: children k1, k2, ... in SBegin order + (events of the parent, a group or a lookup miss carry c = 0, "none"). + Returns (records, kids, max_sends).""" + kids, sends = {}, {} + for e in trace["events"]: + if e["a"] == "SBegin": + if int(e["c"]) in kids: + raise ValueError(f"child {e['c']} begun twice") + kids[int(e["c"])] = f"k{len(kids) + 1}" + if not kids: + raise ValueError("a start trace needs at least one start") + records = [] + for e in trace["events"]: + raw = int(e["c"]) + if raw and raw not in kids: + raise ValueError(f"event for an unrecorded child {raw}: {e}") + c = kids.get(raw, "none") + if e["a"] == "SSend": + sends[c] = sends.get(c, 0) + 1 + records.append("[a |-> {}, c |-> {}, x |-> {}, y |-> {}, z |-> {}, f |-> {}]".format( + tla_string(e["a"]), tla_string(c), tla_string(e["x"]), tla_string(e["y"]), + tla_string(e["z"]), "TRUE" if e["f"] else "FALSE")) + return records, sorted(kids.values()), max(sends.values(), default=0) + + +def generate_start(trace, workdir): + """A parent's starts against Start.tla (unified start, 0.9).""" + for model in ("Start.tla", "TraceStart.tla"): + shutil.copy(os.path.join(HERE, model), workdir) + records, kids, sends = start_records(trace) + place = trace.get("pplace", "U") + if place not in {"G", "U"}: + raise ValueError(f"parent placement must be G or U: {place}") + kid_set = "{" + ", ".join(tla_string(k) for k in kids) + "}" + with open(os.path.join(workdir, "TraceRun.tla"), "w") as out: + out.write("---- MODULE TraceRun ----\nEXTENDS TraceStart\n" + f"TraceKids == {kid_set}\n" + "TraceKinds == {\"spawn\", \"start\", \"link\", \"monitor\"}\n" + "TraceExplicit == {\"none\", \"G\", \"G2\", \"U\", \"same\"}\n" + f"TraceParentPlaces == {{{tla_string(place)}}}\n" + "TraceLog == <<\n " + ",\n ".join(records) + "\n>>\n====\n") + with open(os.path.join(workdir, "TraceRun.cfg"), "w") as out: + out.write("INIT TraceInit\nNEXT TraceNext\nCONSTANTS\n" + " Kids <- TraceKids\n Kinds <- TraceKinds\n" + " Explicit <- TraceExplicit\n ParentPlaces <- TraceParentPlaces\n" + f" Cap = {int(trace.get('cap', 1000))}\n" + " Named = TRUE\n TimeoutEnabled = TRUE\n KillEnabled = TRUE\n" + " AdminStopEnabled = TRUE\n ParentCanExit = TRUE\n ParentKillEnabled = TRUE\n" + " CloseEnabled = FALSE\n OwnerCloseEnabled = FALSE\n" + f" MaxMsgs = {sends}\n ChildCanTrap = TRUE\n" + " UnlinkEnabled = TRUE\n ScopeEnabled = FALSE\n" + " Fault = \"None\"\n Trace <- TraceLog\n" + f" TracePlace = {tla_string(place)}\n" + f" TraceTrap = {'TRUE' if trace.get('ptrap') else 'FALSE'}\n" + "INVARIANT Safety\nINVARIANT TraceIncomplete\n") + + def generate_coordinator(trace, workdir): """One ActorSystem shutdown against Coordinator.tla.""" for model in ("Coordinator.tla", "TraceCoordinator.tla"): diff --git a/verification/tla/trace_coverage.py b/verification/tla/trace_coverage.py index bec611705..c9e1339ba 100644 --- a/verification/tla/trace_coverage.py +++ b/verification/tla/trace_coverage.py @@ -98,6 +98,7 @@ "Lifecycle": ("TraceLifecycle.tla", {"lifecycle-v1"}), "DeferredJobs": ("TraceJobs.tla", {"jobs-v1"}), "DeferredReplies": ("TraceReplies.tla", {"replies-v1"}), + "Start": ("TraceStart.tla", {"start-v1"}), "Coordinator": ("TraceCoordinator.tla", {"coordinator-v1"}), "Disposal": ("TraceDisposal.tla", {"disposal-v1"}), }