Skip to content

Parallel Horizon range chunking for cold scans #175

Description

@truthixify

Tier: S (1-2 days) | Type: perf

Context. createHorizonClient (PR #155) added retry and backoff, and PR #154 pipelined fetch with decrypt, but a cold scan still walks Horizon page-by-page sequentially. For a fresh wallet with a distant first-seen ledger, splitting the range into N chunks fetched in parallel is a straight wall-clock win.

Scope.

  • Add parallelism?: number to the stellar scan config (default 1 for backwards compat).
  • Split the total ledger range into N contiguous chunks, fetch each concurrently, merge back into the async iterable in (ledger, seq) order.
  • Respect the existing retry / circuit breaker per chunk.
  • Bench in test/chains/stellar/bench/scan.bench.ts at parallelism = 1 / 4 / 8.

Acceptance.

  • Bench shows measurable improvement at parallelism >= 4 on a 50k-ledger canned range
  • Ordering guarantee holds under out-of-order chunk completion (property test)
  • Parallel path shares one retry policy so total request budget is bounded
  • Default behaviour is identical to today (verified by snapshot)

Files. src/chains/stellar/scan.ts, src/chains/stellar/announcements.ts, src/chains/stellar/rpc.ts, test/chains/stellar/bench/scan.bench.ts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Stellar WaveIssues in the Stellar wave programdripsFunded via Drips Networkhelp wantedExtra attention is neededperfPerformance / optimizationstellarTouches Stellar / Soroban codestellar-waveAuto-created for Wave 8wave-8Auto-created for Wave 8

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions