Skip to content

fix(dash-spv): harden Windows dashd_sync startup wallet load#904

Draft
thepastaclaw wants to merge 7 commits into
dashpay:devfrom
thepastaclaw:fix-windows-dashd-sync-startup-flake
Draft

fix(dash-spv): harden Windows dashd_sync startup wallet load#904
thepastaclaw wants to merge 7 commits into
dashpay:devfrom
thepastaclaw:fix-windows-dashd-sync-startup-flake

Conversation

@thepastaclaw

@thepastaclaw thepastaclaw commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes intermittent Windows dashd_sync failures tracked in #903:

  • failed to create wallet 'default' ... Database already exists
  • dashd failed to start within timeout

Root cause

The regtest fixtures already ship both wallet and default under regtest/. DashdTestContext starts dashd with -wallet=wallet, then needs the mining default wallet loaded.

The old ensure_wallet treated any loadwallet error as permission to call createwallet. When the fixture wallet was present but load failed for a non-missing reason, create raced into Core's -4 Database already exists panic — observed on Windows under parallel suite load.

Startup failures were also hard to diagnose: readiness only returned a boolean, retained no datadir on mid-setup panics, and did not surface process exit / cookie / last RPC error context.

Fix

  • Add load_wallet for fixture wallets that must never be recreated; context uses it for mining default.
  • Keep ensure_wallet for create-if-missing cases, but only create when Core reports a wallet-related missing error and no on-disk DB exists.
  • Classify wallet RPC errors (not found, already loaded, database already exists) with message+code guards that reject unrelated "not found" responses.
  • Clear stale fixture .lock / .walletlock files before start (including legacy regtest/.walletlock).
  • Readiness: poll process liveness, report p2p_ready/cookie_seen/last RPC error on timeout, dump debug.log tail, retain datadir once for CI artifacts.
  • Windows default readiness timeout 90s (Unix remains 30s), overridable via DASHD_STARTUP_TIMEOUT_SECS. Capacity allowance for parallel dashd starts, not a blind sleep.

Validation

  • Unit tests: cargo test -p dash-spv --features test-utils --lib test_utils::node::tests (6 passed) covering wallet error classification (including Windows CI "Database already exists" message and "Method not found" false-positive rejection), fixture lock cleanup, log tailing, and database-path detection.
  • cargo fmt -p dash-spv
  • cargo clippy -p dash-spv --features test-utils --all-targets -- -D warnings
  • Local live dashd smoke could not be executed on this agent host: the downloaded arm64 dashd binary is killed immediately with SIGKILL even for -version (unsigned binary / host policy). Windows validation is deferred to CI on this PR.

Test plan

Fixes #903

Summary by CodeRabbit

  • Bug Fixes

    • Improved node startup reliability by detecting stale runtime locks and waiting for both network and RPC readiness.
    • Improved wallet loading and management to avoid unnecessary wallet creation or conflicts.
    • Startup failures now provide clearer diagnostics, including relevant log output.
  • Tests

    • Added coverage for wallet error handling, startup diagnostics, and log processing.
    • Temporary test data is retained when startup fails unexpectedly, simplifying troubleshooting.

Stop treating every loadwallet failure as permission to createwallet.
The regtest fixtures already ship a `default` wallet, so the old
fallback raced into "Database already exists" under parallel Windows
CI. Clear stale fixture lock files, classify wallet RPC errors, surface
readiness failures with debug.log context, and retain datadirs on
startup panics so CI artifacts are useful.

Fixes: dashpay#903
Remove needless returns in wallet availability confirmation and apply
rustfmt to the readiness timeout helper.
Share cookie-based Client construction across wallet and base RPC paths,
and clear stale fixture locks only in DashCoreNode::start so all callers
get the same policy without a double walk from DashdTestContext.
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6fdfe88d-83d9-4a33-84fe-1e7fd4a95a12

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Dashd test utilities now clear stale runtime locks, retain failed startup datadirs conditionally, provide configurable readiness diagnostics, distinguish wallet RPC states, and reuse wallet-scoped transaction helpers.

Changes

Dashd test startup reliability

Layer / File(s) Summary
Fixture cleanup and panic retention
dash-spv/src/test_utils/context.rs, dash-spv/src/test_utils/fs_helpers.rs
Startup clears stale regtest locks, loads the mining wallet without creating it, and retains the datadir only when post-start setup panics.
Node readiness and startup diagnostics
dash-spv/src/test_utils/node.rs
Startup uses configurable timeouts, detects early process exits, probes P2P and RPC readiness, and reports failures with retained datadirs and debug-log tails.
Wallet RPC lifecycle and transaction helpers
dash-spv/src/test_utils/node.rs
Wallet operations use base RPC endpoints with structured error classification, while address sends and raw transaction sends reuse existing wallet-scoped helpers and added utility tests validate parsing and filesystem behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DashdTestContext
  participant DashCoreNode
  participant dashd
  participant RPC
  DashdTestContext->>DashCoreNode: start node
  DashCoreNode->>dashd: clear locks and spawn
  DashCoreNode->>RPC: probe P2P, cookie, and RPC readiness
  RPC-->>DashCoreNode: readiness result or diagnostic error
  DashCoreNode-->>DashdTestContext: ready or startup failure
  DashdTestContext->>RPC: load default mining wallet
Loading

Suggested labels: ready-for-review

Suggested reviewers: xdustinface

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the Windows dashd_sync startup and wallet-loading hardening in this PR.
Linked Issues check ✅ Passed The changes address the reported Windows startup timeout, wallet database collisions, and diagnostics requested in #903.
Out of Scope Changes check ✅ Passed The refactors and tests stay within startup reliability and wallet-loading hardening; no unrelated feature work is evident.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Install RetainOnPanic only after start succeeds so startup failures retain
once, share cookie-based soft/hard RPC client construction, and thin-wrap
primary wallet send helpers. Fixture mining wallet now uses load_wallet.
Require a wallet-related message before treating a loadwallet failure as
permission to createwallet, and cover the "Method not found" false
positive in unit tests.
Stop the node before reading/copying debug.log so Windows retain does
not hit sharing violations on open handles, and only load the last 64KiB
of debug.log when dumping a readiness failure.
@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.18%. Comparing base (19690d3) to head (f45b18f).

Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #904      +/-   ##
==========================================
- Coverage   74.54%   74.18%   -0.37%     
==========================================
  Files         327      327              
  Lines       75032    75032              
==========================================
- Hits        55936    55659     -277     
- Misses      19096    19373     +277     
Flag Coverage Δ
core 77.29% <ø> (ø)
ffi 48.55% <ø> (-2.40%) ⬇️
rpc 20.00% <ø> (ø)
spv 91.02% <ø> (-0.19%) ⬇️
wallet 74.41% <ø> (ø)
see 22 files with indirect coverage changes

@thepastaclaw

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
dash-spv/src/test_utils/node.rs (2)

887-915: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover the newly added filesystem and size-limit branches.

Move the lock-cleanup test into fs_helpers.rs and cover regtest/wallets/<name>/.walletlock. Also add a log larger than 64 KiB to verify the capped-tail path and partial-line removal.

As per coding guidelines, individual components require comprehensive in-module unit tests.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dash-spv/src/test_utils/node.rs` around lines 887 - 915, Expand the in-module
tests for the filesystem helpers: move
clear_stale_runtime_locks_removes_fixture_locks into fs_helpers.rs and include a
regtest/wallets/<name>/.walletlock fixture. Update
read_log_tail_returns_last_lines to create a log exceeding 64 KiB, assert the
returned tail respects the cap, and verify any incomplete leading line is
removed.

Source: Coding guidelines


269-270: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift

Move loopback addresses into node configuration.

The changed readiness and RPC paths hardcode 127.0.0.1. Derive both socket addresses and URLs from configured network parameters instead.

As per coding guidelines, network parameters and addresses must never be hardcoded.

Also applies to: 339-347, 354-360

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dash-spv/src/test_utils/node.rs` around lines 269 - 270, The readiness and
RPC paths currently hardcode the loopback address; update the node configuration
used by the relevant connection checks and URL construction around the shown
TcpStream readiness logic and the additional RPC paths to provide the configured
bind/host address. Derive both SocketAddr values and RPC URLs from those
configuration parameters, preserving the existing ports and behavior without any
literal loopback addresses.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@dash-spv/src/test_utils/fs_helpers.rs`:
- Around line 27-53: Update clear_stale_runtime_locks and remove_if_exists to
return io::Result<()> and propagate filesystem and directory-read errors instead
of logging and continuing. Adjust DashCoreNode::start to handle the cleanup
result and fail immediately when stale-lock removal is incomplete, while
preserving successful cleanup behavior for missing files.

In `@dash-spv/src/test_utils/node.rs`:
- Around line 746-755: Update wallet_does_not_exist to require code ==
RPC_WALLET_NOT_FOUND alongside the existing wallet-related and missing-message
checks; remove the broad nonzero-code condition. Add a test covering the
wallet-not-found RPC code and ensure unrelated nonzero errors such as -32601 do
not authorize createwallet.
- Around line 233-246: Ensure startup-failure retention waits for dashd to fully
exit before reading or copying the datadir: update fail_startup in
dash-spv/src/test_utils/node.rs (lines 233-246) to use a blocking child wait
after initiating termination. In dash-spv/src/test_utils/context.rs (lines
67-73), ensure node shutdown occurs before RetainOnPanic is dropped and performs
retention. dash-spv/src/test_utils/fs_helpers.rs (lines 98-120) requires no
direct change; its synchronous retention path must be invoked only after the
process has exited.
- Around line 262-298: Update the startup readiness loop around
try_rpc_client_base and get_blockchain_info so the synchronous RPC probe runs
via spawn_blocking or an async RPC client and cannot stall the timeout future.
Remove the early P2P readiness continue, ensuring every loop independently
samples P2P connectivity, RPC/cookie state, and blockchain readiness while
preserving the existing diagnostic state updates.

---

Nitpick comments:
In `@dash-spv/src/test_utils/node.rs`:
- Around line 887-915: Expand the in-module tests for the filesystem helpers:
move clear_stale_runtime_locks_removes_fixture_locks into fs_helpers.rs and
include a regtest/wallets/<name>/.walletlock fixture. Update
read_log_tail_returns_last_lines to create a log exceeding 64 KiB, assert the
returned tail respects the cap, and verify any incomplete leading line is
removed.
- Around line 269-270: The readiness and RPC paths currently hardcode the
loopback address; update the node configuration used by the relevant connection
checks and URL construction around the shown TcpStream readiness logic and the
additional RPC paths to provide the configured bind/host address. Derive both
SocketAddr values and RPC URLs from those configuration parameters, preserving
the existing ports and behavior without any literal loopback addresses.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7242bcd0-e56b-4107-951f-147b9c728a6c

📥 Commits

Reviewing files that changed from the base of the PR and between 19690d3 and bc012fc.

📒 Files selected for processing (3)
  • dash-spv/src/test_utils/context.rs
  • dash-spv/src/test_utils/fs_helpers.rs
  • dash-spv/src/test_utils/node.rs

Comment thread dash-spv/src/test_utils/fs_helpers.rs Outdated
Comment thread dash-spv/src/test_utils/node.rs Outdated
Comment thread dash-spv/src/test_utils/node.rs
Comment thread dash-spv/src/test_utils/node.rs
@thepastaclaw

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Reviews resumed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: Windows dashd_sync tests intermittently fail during node startup

1 participant