Skip to content

Estimate the noise floor by minimum statistics, sampled at a fixed rate - #7

Open
mmmorks wants to merge 1 commit into
staging/meshcore-devfrom
pr/08-noise-floor
Open

Estimate the noise floor by minimum statistics, sampled at a fixed rate#7
mmmorks wants to merge 1 commit into
staging/meshcore-devfrom
pr/08-noise-floor

Conversation

@mmmorks

@mmmorks mmmorks commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Staged on the fork for review. Final destination: meshcore-dev/MeshCore dev (or the fork; to decide). Base here is staging/meshcore-dev so the diff shows only this change.

Summary

Replaces the batch noise-floor estimator in RadioLibWrapper with a robust, continuously running one (NoiseFloorTracker), sampled on a wall-clock schedule instead of once per loop() call. The public mesh::Radio interface is unchanged: getNoiseFloor() still reports the estimated noise mean in whole dBm, triggerNoiseFloorCalibrate() still conveys the operator's int.thresh, and Dispatcher is untouched.

Why

The batch estimator (64 samples, accepted only when !isReceivingPacket() and below _noise_floor + 14) had four coupled problems, all platform-independent, found by running a repeater for 20 h and logging the reported floor:

  • Its window was emergent, not configured. 64 qualifying samples with no time bound means the window is 64 / (loop_rate * qualifying_fraction). Measured: mean 2.5 s, but 1.2 % of windows took 10–64 s.
  • It was driven by loop() rate, so the same radio in the same RF environment characterised the channel differently on a fast MCU (64 heavily correlated GET_RSSI_INST reads a few ms apart) than on a slower or sleeping board.
  • Gating on !isReceivingPacket() biased the sample population toward quiet moments, under-reporting the floor exactly when the band is busy. Since 79ef74e isReceivingPacket() is also not a pure read (it drives a timeout state machine and calls clearIrqFlags()), so polling it from the sampler had the noise floor participating in packet detection.
  • Accepting only samples below _noise_floor + 14 made the estimate depend on its own previous value. resetAGC() re-seeding the estimate existed partly to escape the resulting stuck-at-(−120) attractor.

A first replacement (a quantile tracker) turned out to be robust to the wrong thing: quantile estimators resist contamination but have essentially zero breakdown point against runs. Over 20 h on a live repeater it logged 316 excursions, 11.9 % of wall time above −108 dBm against a −115 dBm median, each climbing at exactly the tracker's own slew limit — i.e. for 12 % of the time the reported floor was neither the noise floor nor the interference level, only how far the tracker had crawled.

The algorithm

src/helpers/NoiseFloorTracker.h is Martin's minimum statistics (IEEE Trans. Speech & Audio Processing, 2001), which selects rather than chases:

  • Signal only ever adds power, so the lowest readings in a window are noise-only by construction. No activity gate, no outlier rejection, no assumption that interference is a minority of samples — a burst shorter than the window cannot move the estimate at all. Breakdown against a run is (U−1)/U of the window.
  • The statistic kept per sub-window is the second smallest raw reading, not the minimum: an isolated low outlier becomes the first smallest and is never the stored value, so a single spurious −135 dBm read has no effect, while the second smallest is no more reachable by signal than the first. (An EMA in front of the minimiser was tried first and cost 4.7 dB of upward bias under 20 % dispersed interference; the second-smallest reads within 0.08 dB.)
  • Window: 12 sub-windows × 150 samples at 100 ms = 180 s, sized against the 67 s worst excursion measured on hardware (~2.7× margin). The first sub-window is 30 samples so a fresh node has a usable floor in ~3 s. Cost: 12 floats plus a handful of scalars.
  • The window statistic sits below the distribution mean by biasK · sigma. biasK is a 12-entry table indexed by ring occupancy, determined by simulating this exact algorithm (the asymptotic extreme-value formula is ~20 % off at these window sizes, and one constant over-corrects by up to 0.9 dB during warm-up).
  • Sigma is estimated from the mean excess of samples over the window statistic, censored at a fixed 8.5 dB above it. Both the anchor and the gate are deliberately sigma-independent: measuring the spread about the floor estimate, or gating relative to sigma, closes a positive feedback loop under sustained near-floor traffic (a packet a few dB above the floor gets admitted, widens sigma, the wider gate admits stronger packets; simulated at 80 % occupancy this pinned sigma at the 3 dB cap and silently widened the CSMA margin from 2.8 dB to 10.5 dB). The fixed gate removes the loop, not the spread itself — the header documents the measured sweep and why no gate can do better.
  • The estimate falls instantly when the floor genuinely falls and rises only as contaminated sub-windows age out. Reading low is the safe direction for a noise floor: it fires the busy check slightly early (absorbed by CSMA backoff) rather than late.

Sampling in RadioLibWrapper::loop() is rate-limited to NOISE_SAMPLE_INTERVAL_MS (100 ms) of wall clock and resyncs rather than catches up after a gap, so returning from a long transmit does not feed a burst of correlated samples. It is opportunistic ("sample when called, at most every 100 ms"), never a Dispatcher timer, so it demands no wakeup and cannot cap how long a battery board sleeps. Samples are taken while state == STATE_RX only, including mid-packet — the estimator discards those by construction.

Behaviour changes to be aware of

  • int.thresh gets a floor. isChannelActive() still treats interference_threshold as dB, so existing settings behave as before, but it enforces margin = max(int.thresh, NOISE_THRESHOLD_SIGMA_K · sigma) with NOISE_THRESHOLD_SIGMA_K = 3.5 and sigma clamped to [0.5, 3.0] dB. A margin narrower than the noise itself produced continuous false-busy, which protects nothing: it delays every packet until getCADFailMaxDuration() expires and the node transmits regardless. The worst case is a margin of ~10.5 dB when sigma is pinned at 3 dB, so a very tight int.thresh may have less effect than its number suggests; measured sigma on an SX1262 at 250 kHz is 0.6–0.9 dB.
  • resetAGC() no longer re-seeds the estimate. Measured on a live repeater, that was strictly harmful: the reset goes through sleep()startReceive(), which clears the IRQ flags, so a packet already in the air is joined mid-symbol and isReceivingPacket() reports idle for the rest of it. 20 of 257 re-seeds over 20 h landed on signal that way, throwing the reported floor to −85…−95 dBm against a true −114. An AGC reset does not move the floor by anything like the estimator's tracking range, and a genuine change is followed on its own.
  • The estimate is discarded when the receiver changes. updatePreamble() — which every setParams() override routes through — and a successful setRxBoostedGainMode() call resetNoiseFloor(). A bandwidth change moves the thermal floor ~6 dB, and with the estimator's slow-rise asymmetry a stale floor would otherwise read busy for the full 180 s window. For the ~3 s until the first sub-window closes after a reset, _nf.ready() is false, the RSSI branch of isChannelActive() is skipped, and getNoiseFloor() reports 0 — accepted deliberately, and documented at the reset site.
  • The MESH_DEBUG_NOISE_FLOOR line is rate limited to one per 20 samples (2 s), matching the previous log volume, and now also prints sigma in tenths of a dB.

Tunables

All NOISE_TRACKER_* and NOISE_* values are macros overridable per environment via build_flags (macros rather than in-class static constexpr float so an ODR-use cannot produce a link error on the C++11/14 toolchains some targets build with). A static_assert ties the bias table to the ring depth so the two cannot be overridden inconsistently.

Tests

test/test_noise_floor_tracker (googletest, host): 24 cases, including the ones that caught the failures above — a 67 s sustained interferer, interference outlasting the window, the fast-fall/slow-rise asymmetry, the low-outlier weakness, the sigma ratchet under 80 % occupancy at 8 dB and 5 dB, and the warm-up direction at every ring occupancy. Clean noise recovers the mean within 0.04 dB and sigma within 3 %.

Files

  • src/helpers/NoiseFloorTracker.h (new)
  • src/helpers/radiolib/RadioLibWrappers.{h,cpp}
  • src/helpers/radiolib/Custom{SX1262,SX1268,LLCC68,LR1110}Wrapper.h (one-line resetNoiseFloor() on a successful gain-mode change; the LLCC68 wrapper is not compiled by any environment in the tree, see the companion fixes PR)
  • test/test_noise_floor_tracker/

Dependencies

Independent. Applies to dev.

Replace RadioLibWrapper's batch noise-floor estimator with a continuously
running one (NoiseFloorTracker), sampled on a 100 ms wall-clock schedule
inside loop() rather than once per iteration. mesh::Radio's interface is
unchanged: getNoiseFloor() still reports the estimated noise mean in whole
dBm, triggerNoiseFloorCalibrate() still conveys int.thresh, and Dispatcher
is untouched.

The batch estimator (64 samples, accepted only while !isReceivingPacket()
and below _noise_floor + 14) had four coupled, platform-independent
problems, measured over 20 h on a live repeater:

  * Its window was emergent: 64 qualifying samples with no time bound,
    so 64 / (loop_rate * qualifying_fraction). Mean 2.5 s, but 1.2% of
    windows took 10-64 s.
  * It was driven by loop() rate, so the same radio in the same RF
    environment characterised the channel differently on a fast MCU (64
    correlated GET_RSSI_INST reads a few ms apart) than on a slower or
    sleeping board.
  * Gating on !isReceivingPacket() biased the samples toward quiet
    moments -- under-reporting the floor exactly when the band is busy --
    and since 79ef74e that call is not a pure read (it drives a timeout
    state machine and clears IRQ flags), so the sampler was participating
    in packet detection.
  * Accepting only samples below _noise_floor + 14 made the estimate
    depend on its own previous value; resetAGC() re-seeding it existed
    partly to escape the resulting stuck-at-(-120) attractor.

A quantile tracker was tried first and turned out robust to the wrong
thing: quantile estimators resist contamination but have essentially zero
breakdown point against runs. Over 20 h it logged 316 excursions, 11.9% of
wall time above -108 dBm against a -115 dBm median, each climbing at
exactly its own slew limit -- for 12% of the time the reported floor was
neither the noise floor nor the interference level, only how far the
tracker had crawled.

Minimum statistics (Martin, IEEE Trans. Speech & Audio Processing, 2001)
selects rather than chases. Signal only ever adds power, so the lowest
readings in a window are noise-only by construction; a burst shorter than
the window cannot move the estimate at all, and breakdown against a run is
(U-1)/U of the window. The statistic kept per sub-window is the SECOND
smallest raw reading, so an isolated low outlier is never the stored value
(an EMA ahead of the minimiser was tried and cost 4.7 dB of upward bias
under 20% dispersed interference; the second smallest reads within
0.08 dB). Window: 12 sub-windows x 150 samples = 180 s, sized against the
67 s worst excursion measured on hardware; the first sub-window is 30
samples so a fresh node has a floor in ~3 s. Cost: 12 floats plus a few
scalars.

The window statistic sits biasK * sigma below the mean; biasK is a table
indexed by ring occupancy, determined by simulating this exact algorithm
(the asymptotic extreme-value formula is ~20% off at these sizes, and one
constant over-corrects by up to 0.9 dB during warm-up). Sigma is the mean
excess of samples over the window statistic, censored at a fixed 8.5 dB
above it. Anchor and gate are both deliberately sigma-independent:
measuring the spread about the floor estimate, or gating relative to
sigma, closes a positive feedback loop under sustained near-floor traffic
-- simulated at 80% occupancy it pinned sigma at the 3 dB cap and widened
the CSMA margin from 2.8 dB to 10.5 dB. The fixed gate removes the loop;
the header documents the measured sweep and why no gate can do better.

Behaviour changes:

  * isChannelActive() still treats interference_threshold as dB, but
    floors the margin at NOISE_THRESHOLD_SIGMA_K (3.5) * sigma, with
    sigma clamped to [0.5, 3.0] dB. A margin narrower than the noise
    itself produced continuous false-busy, which protects nothing: every
    packet waits out getCADFailMaxDuration() and goes anyway.
  * resetAGC() no longer re-seeds the estimate. Measured, that was
    strictly harmful: the reset goes through sleep() -> startReceive(),
    which clears the IRQ flags, so a packet already in the air is joined
    mid-symbol and isReceivingPacket() reports idle for the rest of it.
    20 of 257 re-seeds over 20 h landed on signal that way, throwing the
    floor to -85..-95 dBm against a true -114.
  * The estimate is discarded when the receiver changes: updatePreamble()
    (which every setParams() override routes through) and a successful
    setRxBoostedGainMode() call the new resetNoiseFloor(). A bandwidth
    change moves the thermal floor ~6 dB, and the estimator's slow-rise
    asymmetry would otherwise read busy for the full 180 s window.
  * The MESH_DEBUG_NOISE_FLOOR line is rate limited to one per 2 s and
    also prints sigma in tenths of a dB.

All tunables are build_flags-overridable macros (rather than in-class
static constexpr float, so an ODR-use cannot produce a link error on the
C++11/14 toolchains some targets build with); a static_assert ties the
bias table to the ring depth.

test/test_noise_floor_tracker covers the failures above on the host: a
67 s sustained interferer, interference outlasting the window, the
fast-fall/slow-rise asymmetry, the low-outlier weakness, the sigma ratchet
under 80% occupancy, and the warm-up direction at every ring occupancy.
Clean noise recovers the mean within 0.04 dB and sigma within 3%.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXSCjgNEbJfHwLjD2WSHW4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant