Cache the Ramanathan PMF and vectorize its temperature interpolation - #197
Merged
Conversation
`probability_of_n_pairs` took ~1.6 s per call, a fixed overhead paid by every `electrons_measured` invocation regardless of photon count. For photon-light calls this cost exceeded the entire Monte Carlo. Profiling split the 1.6 s into: - 0.25 s parsing the three tabulated `.dat` files, repeated on every call to `_probability_of_n_pairs_ramanathan()`. The function takes no arguments and returns constant data, so it is now wrapped in `functools.cache`, which also benefits `quantum_yield_ideal` and `fano_factor`. - 1.35 s in `na.interp` over the temperature axis, which loops over the ~20,000 elements of the non-interpolated (wavelength, num_electron) axes in Python. Replaced with a vectorized gather-and-blend: locate the bracketing temperature samples by a broadcast comparison, gather them by named-array indexing, and blend linearly. The tabulated axis is renamed internally so a `temperature` argument carrying an axis of the same name still works, as it did with `na.interp`. `probability_of_n_pairs` now takes ~3 ms warm (~600x). A zero-photon `electrons_measured` call drops from 1.71 s to 0.11 s. The new implementation matches the previous one to 6e-17 over scalar and array wavelengths and temperatures, including values outside the tabulated range (clamped, as `np.interp` does) and an argument axis named `temperature`. A new `test_probability_of_n_pairs` pins the result to an `na.interp` reference and checks normalization. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011q4461XE8hCcZCKsViMQC1
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #197 +/- ##
=======================================
Coverage 99.57% 99.57%
=======================================
Files 118 118
Lines 6829 6852 +23
=======================================
+ Hits 6800 6823 +23
Misses 29 29
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
roytsmart
added a commit
that referenced
this pull request
Aug 17, 2026
The multinomial fast path (#196) and PMF caching (#197) make the Monte Carlo cheaper, and since the VMR is independent of the illumination level, fewer photons per pixel buy more wavelength samples at no statistical cost. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192abyvBf2Zw5rq5CQ62JFb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to the profiling in #196:
probability_of_n_pairstook ~1.6 s per call, a fixed overhead paid by everyelectrons_measuredinvocation regardless of photon count. For photon-light calls (unit tests, small ROIs, single-wavelength VMR checks) this intercept exceeded the entire Monte Carlo — a zero-photonelectrons_measuredcall took 1.71 s. Together with #196 (which attacks the per-electron slope), this removes the intercept, and directly speeds up the #195 test loop.Where the 1.6 s went
.dattables, every callna.interpover the temperature axisna.interpover the energy axisThe temperature interpolation is slow because
na.interploops over the ~20,000 elements of the non-interpolated(wavelength, num_electron)axes in Python — to interpolate at a single temperature.Changes
_probability_of_n_pairs_ramanathan()is nowfunctools.cached. It takes no arguments and returns constant tabulated data, so the parse happens once per process.quantum_yield_idealandfano_factorcall it too and benefit equally. The docstring notes the returned data is shared and read-only.np.interpdoes. The tabulated axis is renamed internally (_temperature_interp) so atemperatureargument carrying an axis literally namedtemperaturestill works, as it did withna.interp.na.interp.No public API changes, no new files.
Results
probability_of_n_pairs: 1.60 s → 2.6 ms warm (~600×)electrons_measured: 1.71 s → 0.11 squantum_yield_ideal+fano_factor(cold interp, warm cache): 90 msValidation
na.interpchain to 6×10⁻¹⁷ (one ulp) across scalar and array wavelengths and temperatures, including values outside the tabulated range and an argument axis namedtemperature.test_probability_of_n_pairs(8 parametrized cases) pins the result to anna.interpreference and checks the PMF sums to 1._ramanathan_2020_test.pyand the 48 downstream_materials_test.pytests touchingsignal/electrons_measured/uncertainty/quantum_yield/fanopass locally.Merges independently of #196 (non-overlapping regions of the same files); whichever lands second rebases trivially.
The residual 0.11 s intercept is
absorption_effective/multilayer_efficiencyon the_materials.pyside — left alone here, but worth a look if the fixed cost still matters after this.🤖 Generated with Claude Code
https://claude.ai/code/session_011q4461XE8hCcZCKsViMQC1