Add a multinomial fast path to the charge-diffusion sampler - #196
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #196 +/- ##
=======================================
Coverage 99.57% 99.57%
=======================================
Files 118 118
Lines 6829 6857 +28
=======================================
+ Hits 6800 6828 +28
Misses 29 29
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The inner loop of `_electrons_measured_numba` relocated each of a
photon's electrons individually, which costs O(m) per photon and
dominates the Monte Carlo at short wavelengths, where a single photon
can liberate thousands of pairs.
The relocations are i.i.d. on integer offsets, so the counts landing at
each offset are multinomial. `_diffuse_electrons` samples that
multinomial directly for large m: electrons are partitioned across a
bounded window of x offsets via conditional binomials, then each
occupied column across y offsets, with per-bin probabilities given by
the Gaussian integrated over each pixel (an erf difference). This costs
O(window) rather than O(m). Probability mass beyond the +/- 6 sigma
window (~1e-9) folds into the edge bins, so charge is still conserved
exactly.
The per-electron path is retained as the small-m branch of the same
function, selected by comparing m against `_factor_multinomial` times
the window size; setting that module constant to `inf` or `0` forces
either path, and a new test does so to check both against the analytic
charge-diffusion width.
Benchmarks (32x32 grid, e2v-CCD97-like parameters, wrap=True), where
`auto` is the default crossover:
wavelength photons/px brute multinomial auto
1.5 A 200 34.9 s 11.0 s 11.0 s (3.2x)
5.0 A 200 11.6 s 7.1 s 7.1 s (1.6x)
20.0 A 1000 14.1 s 18.9 s 14.1 s (1.0x)
100.0 A 5000 14.5 s 56.1 s 14.5 s (1.0x)
1000.0 A 20000 8.4 s 75.7 s 8.4 s (1.0x)
The crossover tracks the faster path at every wavelength, and the
distributions agree: totals, spatial means, and spatial variances of
the two forced paths match within Monte Carlo noise over ~5e7
electrons.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011q4461XE8hCcZCKsViMQC1
roytsmart
force-pushed
the
perf/diffuse-electrons-multinomial
branch
from
August 17, 2026 14:13
74a8f8b to
6fe0216
Compare
roytsmart
added a commit
that referenced
this pull request
Aug 17, 2026
The multinomial fast path (#196) and PMF caching (#197) make the Monte Carlo cheaper, and since the VMR is independent of the illumination level, fewer photons per pixel buy more wavelength samples at no statistical cost. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192abyvBf2Zw5rq5CQ62JFb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The inner loop of
_electrons_measured_numbarelocates each of a photon's electrons individually, which costs O(m) per photon. At short wavelengths a single photon liberates hundreds to thousands of pairs, and this loop dominates the Monte Carlo — it is a large part of why thesignal()comparisons in #195 are so slow to iterate on.Since the relocations are i.i.d. on integer pixel offsets, the counts landing at each offset are multinomial. The new
_diffuse_electronshelper samples that multinomial directly when m is large: electrons are partitioned across a bounded window of x offsets via conditional binomials, then each occupied column across y offsets, with per-bin probabilities given by the Gaussian integrated over each pixel (an erf difference). This costs O(window) rather than O(m). Probability mass beyond the ±6σ window (~1e-9) folds into the edge bins, so charge is conserved exactly, and off-sensor columns/bins are dropped whenwrap=False, matching the per-electron behavior.The brute-force path is retained, as the small-m branch of the same function: per-photon, the sampler compares m against
_factor_multinomialtimes the window area and takes whichever path is cheaper. Setting the module constant_factor_multinomialtoinfforces per-electron sampling everywhere (the exact previous behavior), and0forces the multinomial path, so either can be pinned for testing.No public API changes, no new files.
Benchmarks
32×32 grid, e2v-CCD97-like parameters (
thickness_depletion=2 μm,thickness_substrate=14 μm,width_pixel=5 μm,wrap=True), wall time per image;autois the default crossover (_factor_multinomial = 0.25, tuned from these measurements):The crossover tracks the faster path at every wavelength (
auto/best = 1.00throughout), so long-wavelength workloads are unaffected.Validation
test_electrons_measured_factor_multinomialforces each path viamonkeypatchand checks the measured spread against the analyticoptika.sensors.charge_diffusionwidth, at a wavelength short enough (~680 pairs/photon) to exercise the large-m regime.electrons_measuredtests and the 40 downstream_materials_test.pytests touchingsignal/electrons_measured/uncertaintypass locally.Follow-up worth knowing about
While profiling I found that
probability_of_n_pairstakes ~1.6 s per call (for a 20-element PMF) and is recomputed on everyelectrons_measuredcall — for single-wavelength calls this fixed cost exceeds the entire electron loop. Caching it would speed up every Monte-Carlo-heavy workflow, including the #195 test loop, and is probably a bigger win than this PR for long-wavelength work. Left out of scope here since it touches evaluation caching rather than the sampler.🤖 Generated with Claude Code
https://claude.ai/code/session_011q4461XE8hCcZCKsViMQC1