Repository navigation
v0.13.0: hold back the unvalidated features on develop - #181
Merged
Merged
Conversation
kriboe90
added a commit
that referenced
this pull request
Nov 9, 2026
Brings the fixes of the step 3 flags and the restart into develop: #180 (-s3_fast with -r, -g or -b no longer also runs the full step 3), #182 (a colour flag picks the channels of -re instead of running step 3) and #183 (-re resumes in the fit mode and in all channels of the interrupted run). They are merged before the v0.13.0 cleanup (#181) lands on master, so that develop takes the fixes without the removals. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A config file keeps the options of the version that wrote it, and the parser reads every option a file sets whether or not LEDSA uses it. An option a release removes would therefore stop taking effect without a word, and the result of an analysis would change unnoticed: a config_analysis.ini of v0.12.0 sets snr_full_weight = 10.0, which v0.13.0 no longer has. report_removed_options names every such option once per process, with what became of it. config.ini and config_analysis.ini each get a REMOVED_OPTIONS registry, empty here; the commits that remove options add them, so that reverting one of those commits takes its entries with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
num_moving_avg_images smoothed the reference property of every LED with a centred moving average over neighbouring images. It came with the SNR weighting and has not been validated on the data of an experiment, so v0.13.0 leaves it out; develop keeps it. average_images with n_summarize, which summarizes consecutive images into one, stays. A config_analysis.ini that still sets num_moving_avg_images is told that the option is ignored. Reverting this commit brings the option back as it was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The linear solver weighted every LED with min(1, SNR / snr_full_weight) and set a layer to NaN when the path weighted mean weight of its LEDs stayed below min_layer_weight. The SNR came from noise_sum_col_val, the read noise of the sum, which step 3 estimated from the standard deviation of the optical black pixels. Neither has been validated beyond the Warsaw tests V008 and V009, so v0.13.0 leaves them out; develop keeps them. What stays from the same change is the unclipped data: the black level of every Bayer color is still taken from the optical black, and the linear solver still leaves out an LED whose relative intensity is at or below zero. Without the weighting that rule alone decides, as it already did in the stacked analysis. Its known limitation is documented with the solver: where the transmission reaches the read noise, the LEDs that are left are those whose noise happened to be positive, and the coefficients come out low (14 to 31 % at 1.24 and 1.64 1/m in V008 and V009, where the weighting gave NaN). The read noise goes from the RAW reader as well: _optical_black_mean and read_channel_data_and_mask_from_img return to the names and the results they had before the noise was added, and ImageChannels builds the data and the mask of a channel. Step 3 writes the layouts of v0.11.0 again (10 names, 24 with the fit), so the reader names their columns as before; the v0.12.0 layouts with noise_sum_col_val are still read. config_analysis.ini files that set snr_full_weight or min_layer_weight are told that the options are ignored. test_led_weighting.py goes; the rule for LEDs at or below zero moves to test_extinction_coefficients_linear.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The linear solver marks an LED whose relative intensity is not finite as not measured, but it zeroed the row of such an LED by multiplying it with its weight of 0, and the optical depth of a NaN intensity is NaN, which the product keeps. nnls then stopped with "array must not contain infs or NaNs" instead of leaving the LED out. An LED whose reference intensity is 0 gives such a value. The optical depth of a left out LED is now set to 0 with np.where. For finite intensities the system is the same as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 3 wrote three background-corrected sums besides the plain sum: bgsub_sum_col_val (median of a border ring of the search area), bgsub2_sum_col_val (mean of that ring) and sidebg_sum_col_val (mean of side bands next to the LED strip, with sidebg_pedestal). They have not been validated on the data of an experiment, so v0.13.0 leaves them out; develop keeps them. Step 3 writes sum_col_val, mean_col_val and max_col_val again, plus the fit columns with -s3. The fast layout is the one of the versions before v0.11.0; the fitted one has 20 names and is new, because those versions glued max_col_val to led_center_x. The reader still knows every earlier layout, so result files of v0.11.0 and v0.12.0 can be analysed, including their background-corrected columns. The search areas keep being stacked by their shape and calibrated once for all channels; only the side band pixels leave the selection. The options sideband_distances, background_floor and sideband_exclusion_margin leave config.ini, and a config.ini that still sets them is told that they are ignored. The tests of the corrections go: test_background_subtraction.py, test_ring_background_mean.py, test_sideband_background.py and the acceptance suite 10. What they held beyond the corrections moves: - the column layouts to test_file_handling.py, now with the layouts of v0.13.0 and v0.12.0, - the Bayer channel mask to test_image_channels.py, - the NaN check of the analysed quantity to test_analysis_quantities.py, - the acceptance cases for several quantities per run (now sum_col_val and mean_col_val) and for fitted step 3 results to suite 08, renamed to 08_test_analysis_quantities.robot. The README section and the documentation of the corrections go as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v0.13.0 is v0.12.0 with the clock offset, the faster step 3 and the removal of the features that are not validated yet: the background corrections, the SNR weighting with the read noise and the moving average. They stay on develop. The README says where they went, and it describes what remains in dense smoke without the weighting: the linear solver leaves out an LED at or below zero, and the coefficients come out low where the transmission reaches the read noise. Its Python requirement names the versions pyproject.toml allows (3.9 to 3.12) instead of 3.8. The commented-out study of lambda_reg goes from ExtinctionCoefficientsLinear.py; it wrote to a directory of one machine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kriboe90
force-pushed
the
claude/version-0-13-0-cleanup-cea6d8
branch
from
November 9, 2026 13:03
db74f13 to
4f83091
Compare
kriboe90
marked this pull request as ready for review
November 9, 2026 13:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v0.13.0 is master without the recent features that are not validated yet. They are kept on the new
developbranch: master before this PR, including #180 and #183, plus version0.14.0.dev0and CI on every push.Please merge with "Create a merge commit", not squash or rebase, which would change the commit IDs below. Each removal is one commit, so a feature validated on
developcomes back to master withgit revert <commit>:num_moving_avg_imagessnr_full_weight,min_layer_weight) and the read noise of the sums (noise_sum_col_val)bgsub_sum_col_val,bgsub2_sum_col_val,sidebg_sum_col_val/sidebg_pedestaland theconfig.inioptionssideband_distances,background_floor,sideband_exclusion_marginThe removals go in this order so that step 3 always writes a layout the reader already knows by its number of columns. Removing the background first would have written 7 columns, which the reader takes for the old bgsub layout.
Kept: the black level from the optical black, the unclipped data,
average_imageswithn_summarize, the temperature correction, the nonlinear solver, the LED fit, the colour correction, several quantities per run, and the rest of v0.12.0 and of the PRs merged since.Also in here
config.ini/config_analysis.inithat still sets a removed option gets a note that the option is ignored. Without it, a v0.12.0config_analysis.ini(snr_full_weight = 10.0) would lose its weighting silently.nnlsthen stopped with "array must not contain infs or NaNs".lambda_regstudy inExtinctionCoefficientsLinear.py, which had a hard-coded local path, is gone.Rebased onto master after #180 and #183. Two places conflicted: the signature of
_generate_led_analysis_data, next to the newfind_fit_leds_for_restartof #183, and the-s3_fastrow of the CLI table. The restart of #183 tells the fit mode from the header of the result files, which keepsfit_successin the 0.13.0 layout, and its tests pass here.Changes for users of v0.12.0
sum_col_val,mean_col_valandmax_col_val: 5 columns, 19 with the fit, where v0.12.0 wrote 10 and 24. Result files of v0.11.0 and v0.12.0 are still read, including their background columns.Test plan
PYTHONPATHset to the branch, as the local editable install points elsewhere).sum/mean/maxand the fit columns (exceptfit_time) of 16 result files, and the extinction coefficients of 9 runs: linear forsum_col_val,mean_col_valandmax_col_valwith-s3_fastand with-s3,average_images, nonlinear and the stacked analysis. Master ran with its defaultsnr_full_weight = 10, which switches itself off for JPG images (no read noise), so this also shows that nothing else changed.sphinx_copybutton, which isn't installed here).After merging
developand revert cae109c, bfc5c49 and fae2ed7 there, keeping its version0.14.0.dev0and dropping the README section on the features in validation, sodevelopkeeps the features and later merges carry only fixes.🤖 Generated with Claude Code