Repository navigation
feat!: unified start API (0.9): spawn/start/start_link/start_monitor, StartOptions, fallible started(), continuation - #291
unbalancedparentheses wants to merge 33 commits into
Conversation
… StartOptions, fallible started(), continuation Implements docs/design/unified-start.md (G1, G2, G8) in both modes: - verbs: spawn (no wait), start/start_link/start_monitor (wait for started(); answer Started, Ignored or the error after publication), with_options(StartOptions<A>) -> Starter; Starting::actor_ref(); ActorRef::ready() for every placement - started() returns (), Result<(), E>, Result<Init<Self>, E> or Init<Self>: ready, continue_with, error, stop, ignore; the 2.3 precedence table; no failed start runs stopped() - stage 1: prepare privately, then one commit (group admission, the parent's edge, the name) under the lock order; refusals are invisible - the startup cell: immutable commitment, waiter, notice disposition at publication, abandonment, cleanup-deadline pending answers, init timer - placement inheritance for start_link/start_monitor; typed Name<A> - removed: start_on, start_with_backend, start_linked, ExecutionGroup::start, SystemError::StartupFailed; test kit Mock::spawn/spawn_with, TestSystem::spawn/start/spawn_mock/options Trace validation for Start.tla: StartEvent recording (lifecycle-trace), TraceStart.tla, start-v1 in trace_check.py, 16 recorded scenarios (tests/start_trace.rs), 15 recorded-trace negative controls, coverage 28/28 arms. Model fix: a group slot is released at scope completion (after publication and the owned-work join), not before publication.
…ics, TESTING; contract tests - MIGRATION.md: 0.8 to 0.9 old-call -> new-call table for every start break - GUIDE.md: starting actors (verbs, results, fallible started(), Init, continuations, links and monitors at start, placement, names) - semantics.md: L1, L2, X5, S6 changed; L8, L9, L10, X8, S7, S9, T5 new, each with its checks; TESTING.md coverage rows - unified-start.md: implementation status, deviations and their reasons - tests/start_contract.rs: the new guarantees in both modes - Loom: start_edge_commit_races_parent_exit_begin (LinkFence harness)
…ied start (same faults)
e098a79 to
99b014c
Compare
…ary sources (one behaviour change in ethrex)
…tes; fuzz L4 oracle follows L9 precedence
… sections (SSend before SHandle)
… the death point instead of staying pending
…p during started() fails the start)
… gen_server:stop
The start commits ready (Started), then the actor exits with the stop's
reason through stopped(); a failed start discards the stop. Exit signals
during started() (kill, a held non-trapping parent exit or link notice) and
Init::stop/error/ignore keep failing the start without stopped();
ctx.stop() inside started() is Init::stop (OTP {stop, R}).
Design revision 4 (2.3 row 3), Start.tla (AdminStopApply,
ReadyReturnCommitsReady, control AdminStopFailsInit), TraceStart mapping,
a recorded-trace control, semantics L2/L9, MIGRATION, GUIDE, client
mapping. Reverts the panic_safety and failure_windows workarounds; the
early-stop tests expect both hooks on every placement again.
…ild (InitExited Killed), not a stage-1 refusal Review #291 item 1. The edge, name and scope were already applied, so the caller got Closed/System while its waiter stayed Waiting forever. The child is now killed before started() (KilledEarly) and its lifecycle is driven inline; the Starting answers InitExited { Killed }. Failpoints system.dispatch_actor and actor.thread_spawn; two recorded traces (managed link+monitor, Backend::Thread) and two trace controls.
…tion, monitor fence order, holder handle, frozen runtime drop)
- TraceStart: SResolve binds started()'s recorded return (StartedEndOf);
SPublish deliver/drop constrained; acceptance also requires no notice
owed to a live P undelivered. Three new trace controls.
- SCommit/SRefuse recorded at the decision point under the innermost lock
(registry, exit request or group), ordered with SReleaseName, SLookup and
SReleaseSlot (commit_begin/commit_point/commit_end).
- fence::register checks the table open before the observer callback: a
start_monitor never registers a name for a refused start; the
debug_assert that could fire in the parent's turn is gone.
- ctx.register_name stores the holder: AlreadyStarted { existing } is Some.
- ExitRequest::runtime_drop respects a frozen (non-ready) decision.
- SAbandon: a start_monitor waiter that ends without a Down clears
down_pending (DownWatch), so f records only a real hand-on.
- Docs: StartOutcome::started on Ignored, StartOptions::label (now the
Backend::Thread thread name), abandoned start_monitor's Down.
- failpoints test: stop the running actor (ctx.stop in started is Init::stop).
…ead (threads); spawn allocates no watch channel Review #291 items 3 and 4. A tasks-mode init timer is a runtime sleep task (virtual time under spawned_sim; sim test init_timer_uses_the_runtime_clock fails with the old per-start thread as a determinism leak); threads mode uses one process-wide timer thread whose entries are removed at the commitment. The startup cell's watch channel is created by the first async observer, so a spawn nobody awaits allocates none. New hotpath benchmarks async_ct_spawn (gated) and threads_spawn (wall clock). Also: start_contract and start_trace tokio tests are gated off the simulator (the CI sim suite timed out on them); exit_signals_sim's late signal oracle accepts the OTP outcome of an exit signal during started(); the thread-only build (StartError From<SystemError> cfg, test imports).
# Conflicts: # .github/workflows/verification.yml # verification/model-map.toml # verification/tla/README.md # verification/tla/coverage-baseline.json # verification/tla/trace_check.py # verification/tla/trace_coverage.py
…tart now has code and trace validation)
…on is the loop's first turn); coordinator_trace on the 0.9 start API
… out of each message's poll)
…r of a decided start is kept)
The 2/25 fede flake of link_stop_in_started_ends_nontrapping_parent: the
failed child's Shutdown notice is delivered to its non-trapping parent at
publication, before the answer is published; that begins the parent's exit
by a signal, which (item 7) abandoned the wait, so start_link answered
ParentExited instead of InitExited { Shutdown }. Item 7 is about not
waiting for the child's started(): a decided start is answered at its
publication (bounded by the cleanup deadline). Unit tests fail first.
…arness allows refined started() impls
|
Trace-test flake (2/25 fede runs): root cause and fix ( The failing scenario was It is a real bug, introduced by review item 7, not a test race:
Fix: item 7's purpose is "don't wait for the child's Failing-first unit tests:
The scenario now prints the result on failure. Stability on fede at CI fixes in
Blocker: main just merged #287 (G7: Renaming the "wip: item 7" commit needs a force push to |
# Conflicts: # concurrency/src/link.rs # concurrency/src/tasks/actor/lifecycle.rs # concurrency/src/threads/actor/context.rs # concurrency/src/threads/actor/idle_stop_regression.rs # concurrency/src/threads/actor/lifecycle.rs # concurrency/src/threads/actor/stop_wakeup_regression.rs # concurrency/tests/exit_signals_sim.rs # concurrency/tests/otp_exit_signals.rs # docs/MIGRATION.md # docs/TESTING.md # fuzz/src/driver.rs # testing/tests/threads_kit.rs # verification/planted/patches/14-exit-shutdown-reports-normal.patch
…8 L4 exception's unused binding
# Conflicts: # .github/workflows/verification.yml # benchmarks/hotpath.py # concurrency/src/lifecycle_trace.rs # concurrency/src/registry.rs # verification/planted/patches/16-mon-dead-target-no-down.patch # verification/tla/README.md # verification/tla/trace_check.py
…n; metrics churn stops actors administratively; fuzz L4 oracle accepts a held signal outranking an init panic (regression added); untrack examples/thread_only/target
….9 placement inheritance), so the system's shutdown is a stop cause for X3 (regression added)
Implements the approved unified start design (rank 5h; gaps G1 fallible startup, G2 continuation, G8
start_monitorand link/monitor on every placement with oneStartOptions) for 0.9, in both modes. Draft: the first PR of a stack (scope ownership S8 and supervisorignorefollow; see "Not in this PR").API (breaking; every break in
docs/MIGRATION.md, 0.8 to 0.9)a.start()a.spawn()(no wait), ora.start().await?(waits forstarted())a.start_with_backend(b)/a.start_on(&g)?/g.start(a)?a.with_options(b).spawn()?/a.with_options(&g).spawn()?a.start_linked(&ctx)a.start_link(&ctx).await?(inherits the parent's placement, waits, refuses an exiting parent)ctx.monitor(&a.start().child_handle())a.start_monitor(&ctx).await?(atomic)ready() -> Result<(), SystemError>(managed only)ready() -> Result<StartOutcome<()>, StartError>(every placement)stopped()afterctx.stop()instarted()ctx.stop()instarted()isInit::stop: nostopped()(OTP); an administrative stop duringstarted()waits for it and runsstopped()started()returns(),Result<(), E>,Result<Init<Self>, E>orInit<Self>(ready,continue_with,error,stop,ignore), resolved by the design's 2.3 precedence table.StartOptions<A>:group,backend,placement_of, typedname(Name<A>),label,init_timeout,stack_size,max_outstanding,max_native_jobs. Errors answer after the child's publication (state dropped, name free) with the final reason, or withpendingafter the cleanup deadline.Trace validation for Start.tla (no longer TLC-only)
StartEvent(28 kinds) recorded underlifecycle-traceat each linearization point (zero cost off: every hook is#[inline(always)]no-op orcfg'd).verification/tla/TraceStart.tlamaps each event to its Start.tla action;trace_check.pyschemastart-v1.concurrency/tests/start_trace.rs: 16 deterministic scenarios (each waits for the events it needs), added to the "Record supervisor lifecycle traces" CI step. All accepted.negative_checks.py::start_trace_controls: 15 recorded-trace mutations (design 16.2's list), all rejected.coverage_floor = 28;model-map.tomllists the code,tlc_onlyremoved.ReleaseSlotafterPublish/WorkDoneC); all 7 per-PR configurations and 40 controls rerun; (2) trace validation rejected the first threads-mode recording: the code delivered a no-opNormalfailed-start notice to a waiting non-trapping parent where the model (and design table) drops it; the code was wrong and now drops it.Further findings while validating (fixed in this PR):
SSendwas recorded after the message became receivable, so the child'sSHandlecould precede it (a recording linearization gap, not a model or runtime bug). Admission and dispatch of start-recorded actors are now trace sections, as for Lifecycle-recorded ones.InitExited { Killed }), with a test.started()always wins); per the 2.3 table an administrative stop that arrived first outranks it. Oracle updated; Linux-only, simulator-only, fuzz and experiments call sites migrated.PeerConnectionServer::started(rlpx/connection/server.rs:553) callsctx.stop()after a failedinitialize_connectionand relies onstopped()for peer-table/broadcaster cleanup; under 0.9 that cleanup must move intostarted(). The client mapping (now re-checked against the canary sources) said no site changes behaviour; corrected.Deviations from the design (each in the design's new "Implementation status" section)
Slot release point (above); the commit takes the parent's table before its exit-request lock (the existing 0.8 order; the design's order would deadlock against notice delivery); a non-ready decision freezes the reason under the exit-request lock and the table closes right after (observably one step);
refining_impl_traitlint on manualstarted()impls (#[started]adds the allow); aBackend::Threadthread is spawned after the commit.Not in this PR
Scope ownership (4.1/S8), supervisor
ignore/ChildStart/ChildStartError/ChildSpec::options(section 9, with Supervision/Cohorts model changes), the simulator races and fuzzer start generator (16.3-16.4), four of the five Loom tests.Mock::on_started/start_delay/drop_flagand the ten start planted bugs are in the stacked #293.Revision after maintainer decisions and the independent review
OTP stop during
started()(design revision 4).ChildHandle::stop*arriving whilestarted()runs is now held, as OTP'sgen_server:stopsystem message. The start answersStarted, then the actor exits with the stop's reason throughstopped(). A start that fails for another reason discards the held stop.started()keep failing the start withoutstopped(): a kill, or a held non-trapping parent exit or link notice. So doInit::stop,Init::errorandInit::ignore.ctx.stop()insidestarted()isInit::stop(OTP{stop, R}). It does not runstopped().PeerConnectionServerguidance (MIGRATION, client mapping) is to do the cleanup instarted()and returnInit::stop/Init::error.AdminStopApply, invariantReadyReturnCommitsReady, and controlAdminStopFailsInit(revision 3's row 3). There is also a recorded-trace control.Review items (each code bug got a test that fails without the fix):
started()(KilledEarly) and drives its lifecycle inline. TheStartinganswersInitExited { Killed }, and no waiter staysWaiting.system.dispatch_actor,actor.thread_spawn.Backend::Thread. Two trace controls.Err(Closed).SResolvebindsstarted()'s recorded return (StartedEndOf).SPublishdeliver and drop are constrained.SCommit/SRefuseare recorded at the decision point under the innermost lock (registry, exit request or group), so they are ordered withSReleaseName,SLookupandSReleaseSlot.init_timer_uses_the_runtime_clock: the old per-start thread fails it as a determinism leak.available_parallelismwas read on every spawn (about a third of a spawn's instructions, already in 0.8.0). It is now read once per process.async_ct_spawn, andthreads_spawn(wall clock).fence::registerchecks the table is open before the observer callback, so no name is registered for a refused start. Thedebug_assertis gone. This is defensive: no reachable failing test.AlreadyStarted.existing. It is now the holder for names held throughctx.register_name.register_ownedtakes the handle. Test fails before the fix.start_link/start_monitorwait (ParentExited). The parent's turn ends and it dies without waiting for the child'sstarted(); the child gets the parent's exit through the link.Starttrace.ParentKillEnabled, set on inMCStartTimeoutand in trace validation.runtime_droprespects a frozen decision. Unit test fails before the fix (Killed instead of Normal).SAbandonrecords only a real hand-on:DownWatchclearsdown_pendingwhen the monitor waiter ends without aDown.StartOutcome::startedonIgnored.StartOptions::labelis now also theBackend::Threadthread name and is documented as ignored for unmanagedAsync/Blocking.start_monitorstill delivers itsDown(documented).Also fixed:
exit_signals_simoracle now accepts the OTP outcome.--no-default-features) and its clippy job.coordinator_trace(from main) is ported to the 0.9 start API.Validation (this head)
MCStart*configs pass, and all 41 Start model controls are rejected.MCStartDeep(weekly, model with the OTP stop): 268,520,014 distinct states, depth 31, 3,332 s, no violation.-D warnings(all features and thread-only) pass;--cfg spawned_sim, passes;async_ct_burstasync_ct_request_seqasync_ct_request_conc16async_ct_send_p1/p2async_ct_spawn(new)threads_request_seqModel: no change because
supervision/run.rsonly reads the scope's readiness barrier directly instead of throughActorRef::ready(same barrier, same states),supervision/policy_tests.rsonly migrates call sites, andtasks/jobs.rs/reply/jobs.rsonly switch to themanaged()accessor, addBounds::orand cache the default native-job bound; the Supervision, Cohorts, Nested, DeferredJobs and DeferredReplies behaviour is unchanged. Start.tla and the LinkFence Loom harness change in this PR for the start code.