Repository navigation
feat: YAML config front-end, CLI, Randles 2013 benchmarks, and analysis viz suite - #5
Closed
VANvonZHANG wants to merge 26 commits into
Closed
VANvonZHANG wants to merge 26 commits into
VANvonZHANG wants to merge 26 commits into
Conversation
…r run_satellite MPS data (A1/A2)
…ns, Legendre decay (T2-inputs)
…ocks_on_grid with ssa,g (T2-blocks)
…so HEATING_RATE_COLUMN resolves (T1)
… (column energy identity, albedo-aware) (B2)
…ared with regression test) + build_scene_no_aerosol for DRF
… + spectral attribution (T1/T2/T3/B2)
…ression (B3); CHANGELOG
… demo two-run (flux+heating) -> rt_heating_rate.png (23/23 plots)
argmin(|wavelength - 0.55|) on an nm grid always picks index 0 (401nm), so every '550nm' scalar (energy/heating logs, baseline fixture) was a 401nm value; the regression test had the same two literals and passed consistently-wrong against the equally-wrong baseline. Also drop the ×1000 on the DRF wl axis (same unit confusion) that made interp(550) clamp to the 401nm endpoint. Baseline regenerated at true 550nm: edir_surf 613.45->798.75, eup_toa 361.41->194.79, F_abs_atm 293.85->336.94 W/m2.
config_version: 1 document: scene (builder-kwargs pass-through), aerosol (LEGO blocks discriminated on kind: mie/bulk/opac_preset/explicit_layer, with od_inversion/mass/exponential/tabulated placements), analysis (orchestrator intents, consumed in the next commit). load_config() maps a validated config onto the same builder chain as examples/multicomponent_viz/canonical.py, so YAML and API-built scenes are interchangeable — enforced by test_config_roundtrip.py: identical uvspec text (modulo explicit-file path) AND byte-identical .master/ .LAYER files via the layer writer's content-hash naming. pyyaml>=6.0 added to core dependencies.
…tents run_config(source, *, uvspec_exe, data_path, plot_dir) executes the loaded scene then applies the analysis section: energy_conservation (assert with albedo falling back to the scene surface), heating (second uvspec invocation in heating-rate mode, merged into rt), drf (no-aerosol baseline; spectral TOA/surface/atmosphere forcing), attribution (leave-one-out N+1 runs), plots (name-based registry over the viz layer), save_netcdf. Multi-run flows mirror examples/multicomponent_viz/run_demo.py; attribution runs sequentially pending the Runner.execute_many fix (T7). Unit tests on synthetic data (no uvspec needed): heating-section variant, albedo fallback errors, hand-computed DRF, plot-registry guards.
Configs are the user-facing surface now, so the package needs a CLI: - run: load + run_config, then a compact summary (rt sizes, budget/DRF @550 nm, merged heating, attribution pieces, figures, netcdf) - validate: load only (no uvspec), one-line ok/scene/blocks/intents - export-config: canonical YAML re-emission via loader.export_config (loader change staged from the previous task, verified against spec) Errors users routinely hit (ValidationError/ValueError/TypeError/ FileNotFoundError) print one concise stderr line and exit 1; anything else propagates with a traceback for development. argparse stdlib only. Also registers the `pyradtran` console script and adds `python -m pyradtran` (__main__.py) for installs without the script.
T5 characterization tests for pyradtran.config (all offline, no uvspec): - scene-section loader errors: solar source without sza, wavelength without min_nm, unknown source kind 'flashlight' (ValueError with the key name) - placement discrimination: od_inversion/mass/exponential/tabulated each build the matching profile class (od_inversion -> MassProfile) - constraint violations raise ValidationError: tau_ref <= 0, one-element altitude_grid_km, phase_function 'delta', opac preset 'not_a_preset' - explicit_layer block builds a bare DirectLayerOpticsBlock piece without reading the (fake) master file - export_config round trip: dict -> YAML -> load_config builds a scene whose build_input() equals the dict-loaded scene's 14 new tests; suite: 518 passed, 1 skipped.
make_yaml.py regenerates canonical.yaml from canonical.py constants via export_config (deterministic dump, comment header prepended by hand since safe_dump emits no comments). aerosol.output_dir is the relative 'output' so pyradtran run writes explicit files next to the config when run from the example dir (validate loads objects only — works from any cwd). Round-trip test: chdir into the example dir, load_config, uvspec text == canonical.build_scene(build_composite()) modulo the masked explicit-file path.
…cess pool
Two bugs in Runner.execute_many (surfaced by the multicomponent example):
(a) Swallowed failures: the worker path appended (idx, exception) tuples to
the results, so failing scenes were silently returned to callers instead
of raising. Now futures are consumed in submission order and the first
failure raises RuntimeError(f"scene {i} failed") from the original
exception, after cancelling outstanding futures.
(b) Pickling crashes: ProcessPoolExecutor had to pickle scenes, and scenes
carrying CompositeAerosol can carry unpicklable payloads (aerosol3D
BulkAerosolOpticsData size distributions store local closures). uvspec
runs as a subprocess and releases the GIL while each worker waits, so a
ThreadPoolExecutor parallelizes the uvspec invocations just as well and
nothing needs pickling.
Public signature and return contract (list[xr.Dataset], input order) are
unchanged. Side effect: the multi-threaded fork() DeprecationWarnings from
the execute_many tests are gone.
Tests: tests/test_runner_parallel.py (monkeypatched Runner.execute) —
fail-fast on first/later scene failure, ordered results, CompositeAerosol
scene accepted, and an unpicklable-payload regression test.
Add pyradtran.benchmarks with the Randles et al. (2013) shortwave benchmark (ACP 13, 2347): Rayleigh-only case1 and AOD550-0.2/Å1 aerosol cases 2a (SSA 1.0) / 2b (SSA 0.8), built-in LBL reference values (Tables 3/5/A3/A4) bundled as package data, and a 24-invocation runner (3 cases x 2 AFGL atmospheres x 2 SZA x 2 bands). RandlesAerosol implements the duck-typed Piece protocol: column tau(lambda) = 0.2*(lambda/0.55um)^-1 on the linear-taper layer weights (0.25/0.75 over 0-2 km, zero above), constant SSA per case, g=0.7 via PMOM moments beta_l = g^l. run_randles2013() executes the matrix via the thread-based Runner.execute_many, normalizes every flux by the own-run TOA incident flux times the LBL median constants (removing the solar-constant choice), derives absorptance/NIR partition and case2-minus-case1 RF, and saves everything to JSON. Aerosol explicit files are pre-written serially so worker threads only hit the on-disk content-hash cache (no concurrent writers on shared .LAYER files). Offline unit tests cover the taper weights, column tau, Ångström law, SSA/moments, scene keywords, the normalization math (uvspec stubbed), and the reference-file structure (64 LBL values, locked anchors). Live smoke (saw30): all tabulated fluxes within 0.6% of LBL, RF within 2.2%, absorptance -4.0% (reptran-coarse band shift).
…ay plots compare_benchmark flattens run_randles2013 results against the bundled LBL reference into one row per (case, quantity, atm, sza) with flux thresholds (8%/12%) and the RF rule (PASS |rel|<=15% or |abs|<=1.5 W/m2, WARN <=25% or <=3 W/m2 by analogy, else FAIL); case-2a atmospheric RF is report-only (n/a) per the paper. Stray results_path keys, string SZA keys from JSON round trips, and broadband-only RF handling are covered by tests. format_report renders per-case Markdown tables with status emojis and pass/warn/fail counts, write_report adds a plain-word CSV with the same stem, and plot_benchmark_overlay draws lazy-matplotlib grouped LBL-vs-pyRadtran bars (SFC down / TOA up / RF panels).
plot_benchmark_overlay built its bar positions from all four CONFIGS but looked values up in by_key, which only contains configs present in the comparison rows — so a subset run missing a single config (e.g. trop30 gone while saw30/saw75/trop75 are present) crashed with a bare KeyError. Mirror format_report's tolerance by filtering positions to (case, quantity, config) keys that exist in by_key; x positions stay contiguous because the remaining pairs are re-enumerated over range(len(positions)). Add a regression test that deletes exactly one config of one quantity (case1 nir_sfc_down @ trop30) and asserts a non-empty PNG is still written.
Full 24-invocation matrix once per session via memoized fixture (module-scoped impossible: conftest gate fixtures are function-scoped, ScopeMismatch). Asserts vs load_reference() at test time: - case1 SFC-down/TOA-up fluxes (direct/diffuse bb, uvvis, nir) within +/-8% (FLUX_PASS_PCT) for all 4 configs (20 checks, superset of the brief's 8 minimum) - case2b TOA RF positive @ SZA30 both atmospheres (paper's sign result) - case2a SFC RF negative in all 4 configs - compare_benchmark: zero FAIL rows (WARN tolerated; summary printed with -s); absorptance/nir-trop75 documented WARN-tolerant (reptran) - report md/csv + overlay PNG written to fixture tmp dir and asserted Real run: 28 passed in ~23 s; comparison 64 pass / 0 warn / 0 fail (+4 report-only n/a). Skips cleanly without libRadtran (28 skipped).
…are/report driver
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two weeks of work (2026-08-10 → 08-16), 26 commits previously sitting unpushed on local
main; moved to this branch for review + CI.What's included
YAML config front-end + CLI
run_configdrives run + analysis intents)pyradtran run / validate / export-configconsole entry pointscanonical.yaml— YAML twin of the canonical sceneRandles 2013 benchmarks
compare_benchmarkwith Markdown/CSV report + LBL overlay plots (tolerates partial-config rows)Analysis & visualization
compute_energy_budget+assert_energy_conservation(column energy identity, albedo-aware)parse_heating_asciiwide format + Runner auto-detect + two-run (flux+heating) demo →rt_heating_rate.pngFixes
execute_many: fail-fast errors + threads instead of process poolheating_rate(wascol_7)_trapzguard for normalization tests; skip gate for run_satellite MPS dataVerification