Skip to content

Cache the Ramanathan PMF and vectorize its temperature interpolation - #197

Merged
roytsmart merged 2 commits into
mainfrom
perf/cache-ramanathan-pmf
Aug 17, 2026
Merged

Cache the Ramanathan PMF and vectorize its temperature interpolation#197
roytsmart merged 2 commits into
mainfrom
perf/cache-ramanathan-pmf

Conversation

@roytsmart

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to the profiling in #196: probability_of_n_pairs took ~1.6 s per call, a fixed overhead paid by every electrons_measured invocation regardless of photon count. For photon-light calls (unit tests, small ROIs, single-wavelength VMR checks) this intercept exceeded the entire Monte Carlo — a zero-photon electrons_measured call took 1.71 s. Together with #196 (which attacks the per-electron slope), this removes the intercept, and directly speeds up the #195 test loop.

Where the 1.6 s went

step cost
parsing the three .dat tables, every call 0.25 s
na.interp over the temperature axis 1.35 s
na.interp over the energy axis 0.002 s

The temperature interpolation is slow because na.interp loops over the ~20,000 elements of the non-interpolated (wavelength, num_electron) axes in Python — to interpolate at a single temperature.

Changes

  • _probability_of_n_pairs_ramanathan() is now functools.cached. It takes no arguments and returns constant tabulated data, so the parse happens once per process. quantum_yield_ideal and fano_factor call it too and benefit equally. The docstring notes the returned data is shared and read-only.
  • The temperature interpolation is a vectorized gather-and-blend: locate the bracketing samples with a broadcast comparison, gather both by named-array indexing, blend linearly, clamping outside the tabulated range exactly as np.interp does. The tabulated axis is renamed internally (_temperature_interp) so a temperature argument carrying an axis literally named temperature still works, as it did with na.interp.
  • The (cheap) energy interpolation is left as na.interp.

No public API changes, no new files.

Results

  • probability_of_n_pairs: 1.60 s → 2.6 ms warm (~600×)
  • zero-photon electrons_measured: 1.71 s → 0.11 s
  • quantum_yield_ideal + fano_factor (cold interp, warm cache): 90 ms

Validation

  • New implementation matches the previous na.interp chain to 6×10⁻¹⁷ (one ulp) across scalar and array wavelengths and temperatures, including values outside the tabulated range and an argument axis named temperature.
  • New test_probability_of_n_pairs (8 parametrized cases) pins the result to an na.interp reference and checks the PMF sums to 1.
  • All 46 tests in _ramanathan_2020_test.py and the 48 downstream _materials_test.py tests touching signal/electrons_measured/uncertainty/quantum_yield/fano pass locally.

Merges independently of #196 (non-overlapping regions of the same files); whichever lands second rebases trivially.

The residual 0.11 s intercept is absorption_effective/multilayer_efficiency on the _materials.py side — left alone here, but worth a look if the fixed cost still matters after this.

🤖 Generated with Claude Code

https://claude.ai/code/session_011q4461XE8hCcZCKsViMQC1

`probability_of_n_pairs` took ~1.6 s per call, a fixed overhead paid by
every `electrons_measured` invocation regardless of photon count. For
photon-light calls this cost exceeded the entire Monte Carlo.

Profiling split the 1.6 s into:

- 0.25 s parsing the three tabulated `.dat` files, repeated on every
  call to `_probability_of_n_pairs_ramanathan()`. The function takes no
  arguments and returns constant data, so it is now wrapped in
  `functools.cache`, which also benefits `quantum_yield_ideal` and
  `fano_factor`.
- 1.35 s in `na.interp` over the temperature axis, which loops over the
  ~20,000 elements of the non-interpolated (wavelength, num_electron)
  axes in Python. Replaced with a vectorized gather-and-blend: locate
  the bracketing temperature samples by a broadcast comparison, gather
  them by named-array indexing, and blend linearly. The tabulated axis
  is renamed internally so a `temperature` argument carrying an axis of
  the same name still works, as it did with `na.interp`.

`probability_of_n_pairs` now takes ~3 ms warm (~600x). A zero-photon
`electrons_measured` call drops from 1.71 s to 0.11 s.

The new implementation matches the previous one to 6e-17 over scalar
and array wavelengths and temperatures, including values outside the
tabulated range (clamped, as `np.interp` does) and an argument axis
named `temperature`. A new `test_probability_of_n_pairs` pins the
result to an `na.interp` reference and checks normalization.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011q4461XE8hCcZCKsViMQC1
@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.57%. Comparing base (64e05e6) to head (4fa15fa).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #197   +/-   ##
=======================================
  Coverage   99.57%   99.57%           
=======================================
  Files         118      118           
  Lines        6829     6852   +23     
=======================================
+ Hits         6800     6823   +23     
  Misses         29       29           
Flag Coverage Δ
unittests 99.57% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@roytsmart
roytsmart merged commit 43f7e49 into main Aug 17, 2026
12 checks passed
@roytsmart
roytsmart deleted the perf/cache-ramanathan-pmf branch August 17, 2026 18:53
roytsmart added a commit that referenced this pull request Aug 17, 2026
The multinomial fast path (#196) and PMF caching (#197) make the
Monte Carlo cheaper, and since the VMR is independent of the
illumination level, fewer photons per pixel buy more wavelength
samples at no statistical cost.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0192abyvBf2Zw5rq5CQ62JFb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant