Skip to content

feat(mirror): drive the mirror pass from disk events, timer as backstop - #470

Draft
MichaelTaylor3d wants to merge 11 commits into
mainfrom
loop/mc-event
Draft

feat(mirror): drive the mirror pass from disk events, timer as backstop#470
MichaelTaylor3d wants to merge 11 commits into
mainfrom
loop/mc-event

Conversation

@MichaelTaylor3d

@MichaelTaylor3d MichaelTaylor3d commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

DRAFT — do not merge. The gate round has not run.

Drives the §25 mirror pass from disk events, with the round timer kept unchanged as the backstop.

Closes #465

What already existed (measured before building, §2.0)

  • Disk: no event source. lifecycle::observe_diskNode::cache_list_cached() is a directory scan. There is no capsule-cached broadcast/watch channel anywhere in dig-node-core or dig-node-service, and no filesystem-watcher crate was in Cargo.lock. The debounced presence deltas in runner.rs:243-245 are computed from consecutive scans by PresenceTracker — a settling filter over polled state, not an event source. So the watcher is genuinely new and is not a rival implementation.
  • Chain: nothing subscribed, and chia-block-listener is not a dependency of this repo at all (zero hits for chia_block_listener under crates/). Peak is read from the wallet replica's sync_state, polled.

The invariant

No behaviour depends on an event arriving. A wake decides only when the next pass runs, never whether one runs and never what it concludes. The timer is neither deleted nor lengthened. silencing_every_event_source_leaves_the_timer_untouched asserts both halves, and pins MIRROR_ROUND_LENGTH_MS at 10 minutes so a later edit cannot lengthen the backstop to compensate for having events.

The chain fork, decided

A chain event does not trigger a pass. Three reasons, in order of weight: a new peak arrives roughly every 18.75s and each pass reads chain and may spend money, so peaks are the largest amplification source available here; chain activity cannot make a create newly correct, because creates are decided from disk presence; and the epoch rollover a reclaim waits on is wall-clock, already derived locally by current_epoch_now(). A peak subscription would be new integration bought to obtain the one signal most worth suppressing. Chain stays observed inside the pass, on the timer, where one round already bounds reclaim latency far inside an epoch.

The coalescing bound

bound value why that figure
QUIET_PERIOD_MS 5_000 debounce, so copying a large .dig in causes one pass at the end rather than one per write event
MIN_EVENT_PASS_INTERVAL_MS SETTLING_WINDOW_MS (30_000) derived: a pass cannot act on a change the tracker has not seen hold for a window, so anything closer is amplification on a money path
  • Two wakes per burst, and it terminates. PresenceTracker stamps since_ms at the first observation, not at file creation, so a single wake could only ever RECORD an appearance. One burst yields an observing wake and one settling wake a window later, which does not re-arm. N events produce exactly one of each, never N.
  • Events during a pass collapse into the single owed wake — the coalescer holds one instant, not a queue — so a wedged pass cannot be followed by a burst of drained passes.
  • Passes still cannot overlap. The pass loop is one sequential task. wait_for_next_pass computes the round deadline once on entry and every return happens at or before it, so an event can only lower the next wake instant, never postpone it.
  • Latency: a hand-copied capsule goes from up to two round timers (~20 min, because the first round only stamps it) to ~35s, with no figure in the settling contract changed.

Verification

  • cargo test -p dig-node-service --lib mirror::113 passed, 0 failed (7 of them new in mirror::events). Count checked, not just the exit status.
  • cargo clippy -p dig-node-service --lib --all-features -- -D warnings — clean.
  • Revert-proof, each guard disabled ALONE, on a committed tree. Every one fails exactly its own test and nothing else:
    • floor → 0: no_two_event_driven_passes_are_closer_than_the_floor FAILED (6 passed, 1 failed)
    • quiet period → 0: a_burst_of_events_in_one_window_produces_exactly_one_observing_pass FAILED (6 passed, 1 failed)
    • settling wake re-arms forever: a_burst_produces_a_settling_pass_and_then_stops FAILED (6 passed, 1 failed)
  • The watcher test measures the delay against the round timer rather than asserting a signal eventually arrived, and deliberately does not use tokio::time::pause() — a paused clock auto-advances past the wall-clock interval under test.

Blast radius checked

impact could not be trusted here: the registered gitnexus indexes point at the primary checkout and are stale by hundreds of commits (dig-node ~301 behind), and a stale index returns a false-safe impactedCount: 0. Radius was established by direct read plus ripgrep instead, and stated rather than assumed.

  • No existing symbol was edited. mirror/events.rs is new. The only change to existing code is inside spawn_mirror_passes (server.rs), plus one new private free function wait_for_next_pass with no other callers.
  • runner.rs, resolve.rs and spend_audit.rs are untouched — the live The mirror lifecycle must self-heal: every store gets a coin, every orphaned coin is spent, even after a failed attempt #464 lane's surface. PassRunner, MirrorEffects, observe_disk and observe_chain keep their exact signatures and semantics; the event source sits entirely outside the pass and changes only when the loop waits.
  • One new dependency, notify = "8", and the lock refreshed in the same commit (cargo update -w --offline).
  • Risk: LOW. Nothing on a spend path changed; the worst case of a watcher misbehaving is an earlier pass, bounded by the floor above.

Version

0.202.0 (minor — new capability, no behaviour removed), Cargo.lock refreshed in the same commit.

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor Author

Greened the red Clippy lints on loop/mc-event. Two manual modulo checks replaced with .is_multiple_of() method as per clippy's guidance:

  • crates/dig-node-service/src/mirror/events.rs:257: (now - T0) % step == 0(now - T0).is_multiple_of(step)
  • crates/dig-node-service/src/mirror/events.rs:337: (now - T0) % spacing == 0(now - T0).is_multiple_of(spacing)

Verified with local clippy (exit 0, no errors). Pushed to origin/loop/mc-event. New PR head: 0b6163c (includes 4 commits including the release bump and fixes). CI re-running.

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor Author

Lane update — conflicts resolved, rebased onto main via merge

Outcome 1: still needed and distinct. §2.0 already-shipped check against origin/main (12a3f49): crates/dig-node-service/src/mirror/mod.rs on main declares advertise, funding, lifecycle, observe, pass, plan, pointers, presence, resolve, runner, signer, spends, states — there is no events module, and nothing on main drives the mirror pass from disk events. The delta here is genuinely absent from main.

State

  • branch: loop/mc-event
  • head: 41387ee
  • version: 0.223.0 (read from Cargo.toml on disk after the merge, not from the log)
  • merged origin/main with git merge (never rebase — a rebase whose patch matches what already merged prints dropping <sha> and silently discards the version bump)
  • conflicts were Cargo.toml (version) and Cargo.lock only; resolved by taking origin/main's lock then cargo update -w --offline in the same commit
  • the previously-reported half-finished merge was not live: the worktree tree was clean and no MERGE_HEAD existed, so the prior attempt had been committed at dc146b1

Next action: wait for CI on 41387ee; then gate. Not undrafted, not merged.

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor Author

Local verification on the merged head 41387ee:

cargo test -p dig-node-service --lib mirror::
test result: ok. 131 passed; 0 failed; 0 ignored; 0 measured; 591 filtered out

131 tests actually executed -- count checked rather than exit status alone.

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor Author

Head moved — origin/main advanced to a375f46 (#477) mid-pass, which put this back to DIRTY. Re-merged (git merge, never rebase); this time zero conflicts, and the version was re-read from Cargo.toml on disk afterwards rather than taken from the commit log.

  • head: b172413
  • version: 0.223.0 (still above main's 0.217.0)
  • mergeStateStatus: BLOCKED (draft + checks pending), no longer DIRTY

Next action: watch CI on b172413, then gate. Still draft, not merged.

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor Author

IN PROGRESS — not the verdict. Gate on head b172413, read via git show (primary checkout untouched). gitnexus NOT used: the dig-node index is ~301 commits stale and returns a false-safe impactedCount: 0; blast radius done by grep + direct read instead.

Confirmed so far, with evidence:

  1. The watched path is the path the pass actually observes. events::capsule_cache_dir joins modules (mirror/events.rs), the watcher is attached to <cache>/modules recursively (server.rs:2765-2772), and lifecycle::observe_disk reads cache_list_cached, whose backing scan is read_dir(self.cache_dir.join("modules")) (dig-node-core/src/lib.rs:2276, scan_cached_modules). So the event source and the observation are over the same set — the accelerant is not watching a directory nothing writes to.

  2. Reentrancy: passes cannot overlap. spawn_mirror_passes is one sequential task; wait_for_next_pass is awaited at the tail of the loop body (server.rs, replacing the former tokio::time::sleep(MIRROR_PASS_INTERVAL)). An event moves the next pass earlier; it never starts a second one, and a wedged pass blocks only its own successor. This was the highest-consequence question (the pass spends money) and it is answered structurally rather than by a lock.

  3. The timer is a genuine ceiling, not a fallback. deadline is computed once on entry to wait_for_next_pass and every return path is at or before it (next = ... .min(deadline), plus the >= deadline return inside the select arm). Events can only lower the wake instant. MIRROR_ROUND_LENGTH_MS is unchanged and a test asserts it is still 10*60*1000.

  4. Coalescing is a real bound, not a hope. WakeCoalescer holds one instant, not a queue; MIN_EVENT_PASS_INTERVAL_MS = SETTLING_WINDOW_MS floors the event-driven pass rate; the settling wake does not re-arm, so a burst of any size yields at most two passes.

Still open: whether the SPEC/instruction scope (observe_chain was named by the user and this PR deliberately declines chain events), a revert proof on at least one test, and the suite count.

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor Author

IN PROGRESS — not the verdict (2/2). Head b172413.

  1. The observe_chain omission is the ticket's own settled fork, not a scope miss. Drive observe_disk and observe_chain from events, keeping the timer as the backstop #465 says verbatim: "Should a chain event trigger a pass at all, or only refresh the observation? ... Decide, and say why (§1.10)." The PR decides it — no chain trigger — and records the reasoning in three places (mirror/events.rs module doc; SPEC.md "Chain events do NOT trigger a pass"). The reasons hold up: a peak every ~18.75s is the largest amplification source on a path that spends money; creates are decided from disk presence; and the epoch rollover a reclaim waits on is derived from collateral::current_epoch_now, i.e. wall-clock, not a chain event. So the signal a peak subscription would buy is one the node already has. Not gating. Flagged for the user only because their verbatim named observe_chain, and this PR is the record of that being answered "timer, on purpose".

  2. Self-heal (The mirror lifecycle must self-heal: every store gets a coin, every orphaned coin is spent, even after a failed attempt #464) is NOT narrowed. The pass body is untouched — the only change to the loop is that the tail tokio::time::sleep(MIRROR_PASS_INTERVAL) became wait_for_next_pass(...). Every pass still runs the full observe_diskobserve_chainpass::decide, so a prior attempt that failed is retried on the next pass exactly as before. Events change when, never what is considered — this is the failure mode the brief warned about (an event-triggered pass that only looks at what changed) and it is not present.

  3. GATING (see the inline thread on server.rs): wait_for_next_pass has no test. Every property the PR is about — the deadline as a ceiling, the None-events path, and the select turning a watcher signal into an early return — lives in that function, and all six events.rs tests plus the watcher test would pass unchanged if it were wrong. They test WakeCoalescer (a pure type) and notify (a third-party crate); nothing tests the seam between them and the pass loop. Detail and the suggested three cases are on the inline thread.

@MichaelTaylor3d MichaelTaylor3d left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate on b172413. One gating finding, inline. Everything else in this PR checks out and is recorded in the two interim comments above.

Comment thread crates/dig-node-service/src/server.rs

@MichaelTaylor3d MichaelTaylor3d left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHANGES-REQUIRED

Head read: b172413664f099883c3c051ef668d42c07d16bb7, resolved from the remote at review time. Read via git show <ref>:<path> and an isolated worktree; the primary checkout was not touched.

One gating finding, on the inline thread at crates/dig-node-service/src/server.rs:2950wait_for_next_pass is the entire seam this PR exists to create, and no test exercises it. All six WakeCoalescer tests and the watcher test pass unchanged if it is wrong. Three cases requested there, all reachable with #[tokio::test(start_paused = true)] and without touching either constant.

Everything else clears, and I want that on the record because the rest of this diff is unusually careful:

  • Reentrancy (the money question) is answered structurally. One sequential task; an event moves the next pass earlier and never starts a second. A wedged pass blocks only its own successor, and the events arriving meanwhile collapse into one owed wake rather than a queue that drains as a burst.
  • The timer is a real ceiling. deadline is computed on entry and every return is at or before it; MIRROR_ROUND_LENGTH_MS is unchanged, and a test asserts it is still 10*60*1000 — the "do not lengthen the timer to compensate" clause of #465, made mechanical.
  • The watched path is the observed path. <cache>/modules, recursive (server.rs:2765-2772) versus scan_cached_modules's read_dir(self.cache_dir.join("modules")) (dig-node-core/src/lib.rs:2276). The accelerant is not watching a directory nothing writes to.
  • Self-heal (#464) is not narrowed. The pass body is untouched; only the tail sleep changed. A prior failed attempt is retried on the next pass exactly as before.
  • observe_chain staying on the timer is #465's own settled fork, decided with reasons and recorded in SPEC.md. Not a scope miss. Flagged for the user because their verbatim named observe_chain.

Evidence. cargo test -p dig-node-service --lib mirror::131 passed; 0 failed; 591 filtered out — 131 tests genuinely executed, not a filter that matched nothing. Revert proof, in my own worktree, by file copy: deleting the MIN_EVENT_PASS_INTERVAL_MS clause from due_at_ms turns no_two_event_driven_passes_are_closer_than_the_floor RED with a real assertion (a pass at 1700000011000 followed one at 1700000005000, closer than the 30000ms floor), not a compile error; the other six stay green. So the floor test is discriminating.

What I could not reach: I did not see an event drive a pass on a real running node — only the watcher signalling in a test, and the coalescer owing a wake in tests. That gap is precisely the gating finding. gitnexus was not used: the dig-node index is ~301 commits stale and returns a false-safe impactedCount: 0, so blast radius was done by grep and direct read.

MichaelTaylor3d and others added 2 commits September 1, 2026 08:44
…_chain is timer-only

The gate found the seam this change creates untested: every property the PR is
about lives in `wait_for_next_pass`, and both halves it composes were already
covered while their composition was not.

Three cases on a paused clock: the round timer with no events (both the
no-watcher path and the silent-watcher path), a single event waking a quiet
period later, and a storm that OUTLASTS the round still returning by its
deadline. The third is the one that matters -- it is the branch where the
`events.changed()` arm keeps winning the `select!` and the deadline check in
the sibling arm is re-entered rather than reached.

Testing it at all required one clock rather than two: the deadline and every
sleep are on tokio's monotonic clock while the coalescer's windows are unix
milliseconds, and reading the wall clock per turn compared two quantities free
to disagree -- under a wall-clock step (NTP, suspend/resume) as much as under a
paused test clock. The unix figure is now anchored to elapsed monotonic time at
entry.

SPEC.md §25.5 also stated the chain half as a preference. It is a limitation:
`ChainSource` is request/response with no subscription surface, §14.2
chain-watch is itself a poll loop, and the one real push path publishes a
FIELDLESS `SyncEvent::CoinState` that names no coin. Recorded with the
mechanism that would have to exist, as #482.

Refs #465
Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Drive observe_disk and observe_chain from events, keeping the timer as the backstop

1 participant