Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
195 changes: 101 additions & 94 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,77 @@

## 0.2.0 (unreleased)

The theme of the release is explanation: a failing seed now diffs itself
against the last passing one, shrinks its schedule to the steps that
mattered, and draws itself as a timeline — and the simulation underneath
grew TLS, real backpressure, disks a crash can tear, restartable hosts and
clocks that lie. Upgrading costs one thing, so it leads the list: every
recorded trace hash moves, once, for every workload.

- Traces now say *where*, and that changes every hash. Each scheduling event
carries the host it belongs to — the machine that asked for a callback on
`schedule`, the machine that owns it on `run` and `cancel`, and nothing at
all for the simulation's own work, like a clock advance or the network's
delivery step — and every packet that reaches a machine records a `deliver`
event pairing with its `send` by uid. Both are format changes, so **every
recorded trace hash differs from the one 0.1.0 produced**, whatever the
workload does. That subsumes the narrower disclosure this list used to
carry, when recording crashes was the only thing that moved a hash: there
is now no workload whose trace is byte-identical to 0.1.0's. What a hash
means inside a version is untouched — same seed, same code, same
interpreter, same hash, checked across processes and hash-randomization
seeds as before.
- `TraceEvent` is a `NamedTuple` rather than a frozen dataclass: one is built
for every callback the simulation schedules and runs, and a frozen
dataclass's per-field `object.__setattr__` cost more than twice as much on
the hottest path in the package. Fields, attribute access and immutability
are unchanged, and events now unpack and compare as plain tuples — but
`dataclasses.replace()` and `dataclasses.fields()` on an event raise
`TypeError` where they used to work.
- Failure reports now diff the failing seed's schedule against the last
passing seed's: the report shows how long the two runs agreed, what each
did at the first disagreement, and a window of context from both traces.
On by default, costs one retained trace.
- Schedule shrinking, experimental: `--simloop-shrink` (or
`explore(shrink=True)`) minimizes a failing schedule toward FIFO order,
keeping only the decisions that reproduce the failure and naming what ran
at each kept step. Bounded by `--simloop-shrink-budget` (default 500
extra runs).
- A failing seed can draw itself: `--simloop-timeline[=DIR]` writes
`simloop-timeline-seed<N>.html` for each failing seed and names the file in
the report, and `simloop.timeline_html(events)` renders any trace the same
way. One lane per machine and one for the simulation, virtual time left to
right, a dot per scheduling decision, an arrow for every packet that
crossed and a stub for every one that did not, with crashes and restarts
marking the lane they struck. The page is self-contained — inline CSS,
script and SVG, nothing fetched — and draws the last 5,000 events by
default, saying so when it dropped any.
- Schedules can be searched by priority instead of by luck:
`--simloop-policy=pct` (also `explore(policy="pct")` and
`@sim_test(policy="pct")`) runs Burckhardt et al.'s PCT (ASPLOS 2010) —
random priorities per chain of work, highest ready one runs, a few randomly
placed demotions — which buys a stated per-run probability of hitting a bug
that needs `--simloop-pct-depth` ordering constraints (default 3). The
horizon those demotions spread over is measured rather than guessed: seed 0
runs the seeded schedule and its step count sizes it, which costs one extra
run of the workload and is what lets a found seed replay the schedule that
found it. PCT explores sequentially, so asking for it alongside
`--simloop-jobs` is refused rather than quietly ignored. It is a floor, not
a speed-up — on a shallow race uniform draws reach the bug some 46 times
sooner, and `docs/supported-api.md` publishes that measurement next to the
guarantee.
- Seed exploration can use every core: `explore(fn, seeds, jobs=N)` and
`--simloop-jobs=N` fan seed batches out over worker processes and report
exactly what a sequential run would have — the earliest failing seed, with
its report rebuilt by re-running that seed in the parent, which also
proves the replay held across processes. Workloads must be picklable to
cross that boundary, so lambdas, closures and fixture-taking tests stay
sequential and say so.
- Every scheduling decision flows through a policy seam: seeded draws by
default, making the same draws the loop made when it owned the PRNG itself,
with recorded choice lists that can replay a schedule independently of its
seed (internal, powers shrinking). Policies are shown who is ready, not
just how many, which is what a priority policy needs.
- TLS runs inside the simulation. `create_connection(ssl=...)`,
`create_server(ssl=...)` and `start_tls` on an already-established
connection all work, and so do `asyncio.open_connection` and `start_server`
Expand Down Expand Up @@ -51,49 +122,22 @@
real thing — deliberately, since that is what makes a slow consumer
visibly slow. With the switch off `get_write_buffer_size()` reports `0`,
which is the truth: nothing is charged and writes leave immediately.
- Traces now say *where*, and that changes every hash. Each scheduling event
carries the host it belongs to — the machine that asked for a callback on
`schedule`, the machine that owns it on `run` and `cancel`, and nothing at
all for the simulation's own work, like a clock advance or the network's
delivery step — and every packet that reaches a machine records a `deliver`
event pairing with its `send` by uid. Both are format changes, so **every
recorded trace hash differs from the one 0.1.0 produced**, whatever the
workload does. That subsumes the narrower disclosure this list used to
carry, when recording crashes was the only thing that moved a hash: there
is now no workload whose trace is byte-identical to 0.1.0's. What a hash
means inside a version is untouched — same seed, same code, same
interpreter, same hash, checked across processes and hash-randomization
seeds as before.
- `TraceEvent` is a `NamedTuple` rather than a frozen dataclass: one is built
for every callback the simulation schedules and runs, and a frozen
dataclass's per-field `object.__setattr__` cost more than twice as much on
the hottest path in the package. Fields, attribute access and immutability
are unchanged, and events now unpack and compare as plain tuples — but
`dataclasses.replace()` and `dataclasses.fields()` on an event raise
`TypeError` where they used to work.
- Schedules can be searched by priority instead of by luck:
`--simloop-policy=pct` (also `explore(policy="pct")` and
`@sim_test(policy="pct")`) runs Burckhardt et al.'s PCT (ASPLOS 2010) —
random priorities per chain of work, highest ready one runs, a few randomly
placed demotions — which buys a stated per-run probability of hitting a bug
that needs `--simloop-pct-depth` ordering constraints (default 3). The
horizon those demotions spread over is measured rather than guessed: seed 0
runs the seeded schedule and its step count sizes it, which costs one extra
run of the workload and is what lets a found seed replay the schedule that
found it. PCT explores sequentially, so asking for it alongside
`--simloop-jobs` is refused rather than quietly ignored. It is a floor, not
a speed-up — on a shallow race uniform draws reach the bug some 46 times
sooner, and `docs/supported-api.md` publishes that measurement next to the
guarantee.
- A failing seed can draw itself: `--simloop-timeline[=DIR]` writes
`simloop-timeline-seed<N>.html` for each failing seed and names the file in
the report, and `simloop.timeline_html(events)` renders any trace the same
way. One lane per machine and one for the simulation, virtual time left to
right, a dot per scheduling decision, an arrow for every packet that
crossed and a stub for every one that did not, with crashes and restarts
marking the lane they struck. The page is self-contained — inline CSS,
script and SVG, nothing fetched — and draws the last 5,000 events by
default, saying so when it dropped any.
- A host can connect to its own listener. The stream registry used to key
each connection end by host alone, so the two ends of a self-connection
collapsed onto one entry and the run deadlocked; the key now carries the
end's own port, the connect handshake answers to the connector's port
rather than the listener's, and a loopback connect to a closed port is
refused instead of hanging. Packets between distinct hosts are keyed,
ordered and traced exactly as before, so existing hashes do not move.
- `server.sockets` on a simulated server answers with an empty tuple
instead of not existing, which is all aiohttp's `web.TCPSite` and
websockets' `serve()` need to start; both now run their documented
startup paths under simulation.
- Name resolution now stays inside the simulation: every sim host gets a
stable synthetic IPv4 address (`10.7.0.0/16`, registration order), the
loop's `getaddrinfo`/`getnameinfo` resolve names and addresses against
the topology (no real DNS), connections accept either form, and unknown
names raise `socket.gaierror` deterministically.
- Crashed hosts can come back: `loop.net.restart(name)` (or
`host.restart()`) revives a machine as a fresh incarnation. It restores
liveness and nothing else — the old tasks stay cancelled and the
Expand All @@ -120,6 +164,20 @@
wrong wall clock does to a real machine. Traces stay on the true clock, so
skew never perturbs scheduling and a run that configures no offset makes
exactly the decisions it made without the feature.
- Executor submissions run inline instead of fencing: `loop.run_in_executor`
executes the function at an ordinary scheduled step — ordered by the
seeded draw, labelled `executor:<function>` in the trace, costing no
virtual time — and its result or exception lands on the returned future
the way a worker would land it, so `asyncio.to_thread` works under
simulation. The executor argument is never used (there is no pool and
nothing runs concurrently), and `set_default_executor` still fences: a
pool that would never run anything is refused rather than accepted.
`call_soon_threadsafe` from the loop's own thread is now `call_soon`,
which is all it ever was without a second thread; from any other thread
it still fences. None of this reaches `anyio.to_thread`, whose worker
threads are real ones spawned through no loop API — a real thread racing
a virtual clock ends in the cross-thread fence, a hang, or the caller's
own timeout, whichever the race picks.
- A second flagship demo: `examples/raft/` is a teaching-sized Raft (leader
election + log replication, plain asyncio on streams) tested only under
simulation — four safety invariants checked over 50,000 chaos seeds, six
Expand Down Expand Up @@ -148,57 +206,6 @@
package, seeds are deliberately not a strategy (a seed has no size to shrink
toward, and two shrinkers aimed at one failure fight), and Hypothesis is a
dev dependency of this repository rather than something simloop imports.
- Executor submissions run inline instead of fencing: `loop.run_in_executor`
executes the function at an ordinary scheduled step — ordered by the
seeded draw, labelled `executor:<function>` in the trace, costing no
virtual time — and its result or exception lands on the returned future
the way a worker would land it, so `asyncio.to_thread` works under
simulation. The executor argument is never used (there is no pool and
nothing runs concurrently), and `set_default_executor` still fences: a
pool that would never run anything is refused rather than accepted.
`call_soon_threadsafe` from the loop's own thread is now `call_soon`,
which is all it ever was without a second thread; from any other thread
it still fences. None of this reaches `anyio.to_thread`, whose worker
threads are real ones spawned through no loop API — a real thread racing
a virtual clock ends in the cross-thread fence, a hang, or the caller's
own timeout, whichever the race picks.
- A host can connect to its own listener. The stream registry used to key
each connection end by host alone, so the two ends of a self-connection
collapsed onto one entry and the run deadlocked; the key now carries the
end's own port, the connect handshake answers to the connector's port
rather than the listener's, and a loopback connect to a closed port is
refused instead of hanging. Packets between distinct hosts are keyed,
ordered and traced exactly as before, so existing hashes do not move.
- `server.sockets` on a simulated server answers with an empty tuple
instead of not existing, which is all aiohttp's `web.TCPSite` and
websockets' `serve()` need to start; both now run their documented
startup paths under simulation.
- Name resolution now stays inside the simulation: every sim host gets a
stable synthetic IPv4 address (`10.7.0.0/16`, registration order), the
loop's `getaddrinfo`/`getnameinfo` resolve names and addresses against
the topology (no real DNS), connections accept either form, and unknown
names raise `socket.gaierror` deterministically.
- Failure reports now diff the failing seed's schedule against the last
passing seed's: the report shows how long the two runs agreed, what each
did at the first disagreement, and a window of context from both traces.
On by default, costs one retained trace.
- Every scheduling decision flows through a policy seam: seeded draws by
default, making the same draws the loop made when it owned the PRNG itself,
with recorded choice lists that can replay a schedule independently of its
seed (internal, powers shrinking). Policies are shown who is ready, not
just how many, which is what a priority policy needs.
- Seed exploration can use every core: `explore(fn, seeds, jobs=N)` and
`--simloop-jobs=N` fan seed batches out over worker processes and report
exactly what a sequential run would have — the earliest failing seed, with
its report rebuilt by re-running that seed in the parent, which also
proves the replay held across processes. Workloads must be picklable to
cross that boundary, so lambdas, closures and fixture-taking tests stay
sequential and say so.
- Schedule shrinking, experimental: `--simloop-shrink` (or
`explore(shrink=True)`) minimizes a failing schedule toward FIFO order,
keeping only the decisions that reproduce the failure and naming what ran
at each kept step. Bounded by `--simloop-shrink-budget` (default 500
extra runs).

## 0.1.0 (2026-07-18)

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "simloop"
version = "0.1.0"
version = "0.2.0"
description = "Deterministic simulation testing for Python asyncio: seeded scheduling, virtual time, replayable failures"
readme = "README.md"
requires-python = ">=3.12"
Expand Down
2 changes: 1 addition & 1 deletion src/simloop/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from simloop._timeline import timeline_html
from simloop._trace import TraceEvent

__version__ = "0.1.0"
__version__ = "0.2.0"

__all__ = [
"Host",
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading