Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 20 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ LEDSA (LEDSmokeAnalysis) is a Python-based software package for the computation

### Requirements

- Python 3.8
- Python 3.9 to 3.12
- pip (Python package installer)

### Installation from PyPI
Expand Down Expand Up @@ -99,7 +99,7 @@ another, each solver and quantity writing into a directory of its own:

```ini
[DEFAULT]
reference_property = sum_col_val bgsub_sum_col_val
reference_property = sum_col_val mean_col_val
```

```
Expand Down Expand Up @@ -135,21 +135,6 @@ individual runs. Since image IDs are counted per run, `num_ref_images` and `ref_
be set per `[simulation_N]` section; pick the reference images of all cameras from the same time
window before the fire.

### Background correction (optional)

Besides the plain sum `sum_col_val`, step 3 writes background-corrected sums that can be selected
as `reference_property`:

| `reference_property` | Background estimate |
|---|---|
| `sum_col_val` (default) | none |
| `bgsub_sum_col_val` | median of a 2 px border ring of the search area, subtracted pixel-wise |
| `bgsub2_sum_col_val` | mean of the same border ring, subtracted per pixel of the search area |
| `sidebg_sum_col_val` | mean of side bands next to the LED strip, subtracted per pixel of the search area |

The side bands lie `sideband_distances` (default 25 to 60 px) across the strip in `config.ini`;
raise `sideband_exclusion_margin` if light of a neighbouring strip reaches them.

### Temperature correction (optional)

LEDs lose radiant power as they heat up, which would otherwise be read as extinction. Reduction
Expand Down Expand Up @@ -177,6 +162,24 @@ The time counts from `start_time` of `config.ini`, like `Experiment_Time[s]` in
factors before I0 is computed; images and LEDs outside the table stay uncorrected, with a
warning. The header of each result file states the correction applied.

### Dense smoke and the read noise of the sensor

The intensities are not clipped at zero after the black level of the sensor is subtracted, so an
LED that has vanished in dense smoke can give a relative intensity at or below zero. The linear
solver leaves such LEDs out. Where the transmission reaches the read noise of the sensor, the LEDs
that remain are those whose noise happened to be positive, and the extinction coefficients come
out too low: in fog tests by 14 to 31 % at 1.24 and 1.64 1/m, where the transmission of the red
LEDs was 0.1 to 0.2 % of I0.

### Features in validation

The background-corrected sums of step 3 (`bgsub_sum_col_val`, `bgsub2_sum_col_val`,
`sidebg_sum_col_val`), the weighting of the LEDs by their signal-to-noise ratio and the moving
average of the reference property, which v0.12.0 had, are left out of the releases until they are
validated. They are kept on the
[develop](https://github.com/FireDynamics/LEDSmokeAnalysis/tree/develop) branch. A config file that
still sets one of their options gets a note that the option is ignored.

## Demo

LEDSA includes a demo that demonstrates its functionality using sample data.
Expand Down
2 changes: 1 addition & 1 deletion doc/cli_documentation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Data Extraction
- Step 3: Analyse all images to extract pixel values inside the search areas and fit them to an LED model (SLOW!).
- ``-r`` (Use the red channel, default), ``-g`` (Use the green channel), ``-b`` (Use the blue channel), ``-rgb`` (Run for each channel)
* - ``-s3_fast``, ``--step_3_fast`, ``--analyse_photo_fast``
- Step 3: Analyse all images to extract pixel values and calculate the accumulated values inside the search areas (FAST!). Besides the plain sum, the background-corrected sums ``bgsub_sum_col_val``, ``bgsub2_sum_col_val`` and ``sidebg_sum_col_val`` are written (see ``reference_property`` in the analysis configuration).
- Step 3: Analyse all images to extract pixel values and calculate the accumulated values inside the search areas (FAST!).
- Channel flags as for ``-s3``
* - ``-re``, ``--restart``
- Restart step 3 if it was previously interrupted. Only the images that lack the result file of one of the selected channels are analysed, in all of these channels. The restart resumes in the mode of the interrupted run, with the LED fits after ``-s3`` and without them after ``-s3_fast``, as the headers of its result files tell. If there are no result files yet, it runs without the fits. If the result files were written in both modes, it stops, because it cannot tell which mode to resume.
Expand Down
8 changes: 0 additions & 8 deletions doc/data_extraction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,6 @@ Data Extraction
:show-inheritance:

.. automodule:: ledsa.data_extraction.LEDAnalysisData
:members:
:undoc-members:
:show-inheritance:

Side Band Pedestal
------------------

.. automodule:: ledsa.data_extraction.sideband_background
:members:
:undoc-members:
:show-inheritance:
52 changes: 20 additions & 32 deletions ledsa/analysis/ConfigDataAnalysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,24 @@
from typing import Iterable, List, Optional, Union

from ledsa.analysis.temperature_correction import TEMPERATURE_CORRECTION_CHANNELS
from ledsa.core.config_file import save_config
from ledsa.analysis.ExtinctionCoefficients import DEFAULT_SNR_FULL_WEIGHT
from ledsa.core.config_file import report_removed_options, save_config

DEFAULT_REFERENCE_PROPERTY = 'sum_col_val'

# Options of config_analysis.ini that a release of LEDSA has removed, with what became of them. A
# config written by an older version still sets them, and loading it names them.
REMOVED_OPTIONS = {
'snr_full_weight': 'LEDSA 0.13.0 removed the weighting of the LEDs by their signal-to-noise ratio '
'until it is validated; it is kept on the develop branch of the repository. The '
'linear solver weights all LEDs equally and leaves out those at or below zero.',
'min_layer_weight': 'It belonged to the weighting of the LEDs by their signal-to-noise ratio, which '
'LEDSA 0.13.0 removed until it is validated; it is kept on the develop branch of '
'the repository.',
'num_moving_avg_images': 'LEDSA 0.13.0 removed the moving average of the reference property until '
'it is validated; it is kept on the develop branch of the repository. '
'average_images with n_summarize averages consecutive images instead.',
}


def parse_reference_properties(value: str) -> List[str]:
"""
Expand All @@ -31,7 +44,7 @@ def parse_reference_properties(value: str) -> List[str]:
properties.append(name)
if not properties:
raise ValueError(f"The reference_property option names no quantity ({value!r}). Give one or "
f"more quantities separated by spaces, e.g. 'sum_col_val bgsub_sum_col_val'.")
f"more quantities separated by spaces, e.g. 'sum_col_val mean_col_val'.")
return properties


Expand Down Expand Up @@ -59,9 +72,7 @@ def __init__(self, load_config_file=True, camera_position=None, num_layers=20, d
reference_property='sum_col_val',
average_images=False, n_summarize=2, solver='linear', weighting_preference=-6e-3,
weighting_curvature=1e-2,
num_iterations=1000, lambda_reg=1e-3, snr_full_weight=DEFAULT_SNR_FULL_WEIGHT,
min_layer_weight=0.5,
num_moving_avg_images=1, reduction_files=None):
num_iterations=1000, lambda_reg=1e-3, reduction_files=None):
"""
:param load_config_file: Determines whether to load the config file on initialization. Defaults to True.
:type load_config_file: bool
Expand All @@ -82,10 +93,8 @@ def __init__(self, load_config_file=True, camera_position=None, num_layers=20, d
:param num_cores: Number of CPU cores for (multicore) processing. If greater than 1, multicore processing is applied. Defaults to 1.
:type num_cores: int
:param reference_property: Quantity or quantities the extinction coefficients are computed
from. Defaults to 'sum_col_val'. Use 'bgsub_sum_col_val' to base the analysis on
intensities with the local background subtracted, estimated from the border of each LED
search area. Several quantities, given as a list or separated by spaces, are analysed one
after another, each into a directory of its own.
from. Defaults to 'sum_col_val'. Several quantities, given as a list or separated by
spaces, are analysed one after another, each into a directory of its own.
:type reference_property: Union[str, Iterable[str]]
:param average_images: Determines if intensities are computed as an average from two consecutive images. Defaults to False.
:type average_images: bool
Expand All @@ -102,19 +111,6 @@ def __init__(self, load_config_file=True, camera_position=None, num_layers=20, d
:type num_iterations: int
:param lambda_reg: Regularization parameter for the linear solver. Defaults to 1e-3.
:type lambda_reg: float
:param snr_full_weight: Linear solver only: signal-to-noise ratio from which on an LED gets the
full weight. Below it the weight falls linearly to zero at a signal of zero. None weights all
LEDs equally. The weighting needs a reference property with a noise column, which only
sum_col_val has, and noise values in the data; where either is missing it is switched off with
a note.
:type snr_full_weight: float or None
:param min_layer_weight: Linear solver with snr_full_weight only: a layer whose path length
weighted mean LED weight stays below this value is set to NaN. Defaults to 0.5.
:type min_layer_weight: float
:param num_moving_avg_images: Linear solver only: number of consecutive images over which the
reference property is smoothed by a centred moving average, every image keeping its own time.
1 leaves it as it is and is the default. Cannot be combined with average_images.
:type num_moving_avg_images: int
:param reduction_files: Path of the reduction factor file of the temperature correction per
camera channel. Channels without an entry are analysed without the correction, which is
the default for every channel.
Expand Down Expand Up @@ -149,15 +145,6 @@ def __init__(self, load_config_file=True, camera_position=None, num_layers=20, d
self['DEFAULT'][' num_iterations'] = str(num_iterations)
self.set('DEFAULT', ' # Options for linear solver (ignored when nonlinear solver is used) ')
self['DEFAULT'][' lambda_reg'] = str(lambda_reg)
self.set('DEFAULT', ' # SNR from which on an LED gets the full weight, None for equal weights.')
self.set('DEFAULT', ' # Needs a reference property with a noise column (sum_col_val) and')
self.set('DEFAULT', ' # noise values in the data, otherwise the LEDs are weighted equally.')
self['DEFAULT'][' snr_full_weight'] = str(snr_full_weight)
self.set('DEFAULT', ' # Layers whose mean LED weight stays below this value become NaN')
self['DEFAULT'][' min_layer_weight'] = str(min_layer_weight)
self.set('DEFAULT', ' # Images for a centred moving average of the reference property, 1 is off.')
self.set('DEFAULT', ' # Every image keeps its own time, unlike average_images with n_summarize.')
self['DEFAULT'][' num_moving_avg_images'] = str(num_moving_avg_images)

self['experiment_geometry'] = {}
self.set('experiment_geometry', '# Global X Y Z position of the camera ')
Expand Down Expand Up @@ -205,6 +192,7 @@ def load(self) -> None:
'config_analysis.ini not found in working directory! Please create it with argument "--config_analysis".')
exit(1)
print('config_analysis.ini loaded!')
report_removed_options(self, 'config_analysis.ini', REMOVED_OPTIONS)

def save(self) -> None:
"""
Expand Down
2 changes: 1 addition & 1 deletion ledsa/analysis/ConfigDataStacked.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ class ConfigDataStacked(cp.ConfigParser):
camera_channels = 0 1 2
num_layers = 20
domain_bounds = 0.0 3.0
reference_property = sum_col_val bgsub_sum_col_val
reference_property = sum_col_val mean_col_val
num_ref_images = 10
ref_img_indices = None
average_images = False
Expand Down
22 changes: 0 additions & 22 deletions ledsa/analysis/ExperimentData.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from dataclasses import dataclass

from ledsa.analysis.ConfigDataAnalysis import ConfigDataAnalysis
from ledsa.analysis.ExtinctionCoefficients import DEFAULT_SNR_FULL_WEIGHT
from ledsa.analysis.Experiment import Camera, Layers
# Todo: Import init_function for request of missing data
from ledsa.core.ConfigData import ConfigData
Expand Down Expand Up @@ -38,16 +37,6 @@ class ExperimentData:
:type num_ref_images: int
:ivar ref_img_indices: Indices of reference images to use. If None, use num_ref_imgs.
:type ref_img_indices: list[int] or None
:ivar snr_full_weight: Signal-to-noise ratio from which on an LED gets the full weight in the linear
solver. None weights all LEDs equally. Configurations written before the option existed do not
hold it and therefore get the default.
:type snr_full_weight: float or None
:ivar min_layer_weight: Only with snr_full_weight: a layer whose path length weighted mean LED weight
stays below this value is set to NaN.
:type min_layer_weight: float
:ivar num_moving_avg_images: Number of consecutive images for a centred moving average of the reference
property. 1 leaves it as it is.
:type num_moving_avg_images: int
:ivar average_images: Intensities are computed as an average from consecutive images.
:type average_images: bool
:ivar n_summarize: Number of consecutive images summarized into one when averaging.
Expand Down Expand Up @@ -78,9 +67,6 @@ def __init__(self, load_config_file=True):
self.average_images = False
self.n_summarize = 2
self.lambda_reg = None
self.snr_full_weight = DEFAULT_SNR_FULL_WEIGHT
self.min_layer_weight = 0.5
self.num_moving_avg_images = 1
self.reference_properties = None
self.reduction_files = {}
self.merge_led_arrays = None
Expand Down Expand Up @@ -127,14 +113,6 @@ def load_config_parameters(self) -> None:
self.num_iterations = int(config_analysis['DEFAULT']['num_iterations'])
elif self.solver == 'linear':
self.lambda_reg = float(config_analysis['DEFAULT']['lambda_reg'])
if config_analysis.has_option('DEFAULT', 'snr_full_weight'):
# an explicit None switches the weighting off, which the default no longer does
entry = config_analysis['DEFAULT']['snr_full_weight'].strip()
self.snr_full_weight = None if entry.lower() == 'none' else float(entry)
if config_analysis.has_option('DEFAULT', 'min_layer_weight'):
self.min_layer_weight = float(config_analysis['DEFAULT']['min_layer_weight'])
if config_analysis.has_option('DEFAULT', 'num_moving_avg_images'):
self.num_moving_avg_images = int(config_analysis['DEFAULT']['num_moving_avg_images'])
self.reference_properties = config_analysis.get_reference_properties()
self.reduction_files = {channel: config_analysis.get_reduction_file(channel)
for channel in (self.channels or [])}
Expand Down
Loading
Loading