Estimate the noise floor by minimum statistics, sampled at a fixed rate - #7
Open
mmmorks wants to merge 1 commit into
Open
Estimate the noise floor by minimum statistics, sampled at a fixed rate#7mmmorks wants to merge 1 commit into
mmmorks wants to merge 1 commit into
Conversation
Replace RadioLibWrapper's batch noise-floor estimator with a continuously
running one (NoiseFloorTracker), sampled on a 100 ms wall-clock schedule
inside loop() rather than once per iteration. mesh::Radio's interface is
unchanged: getNoiseFloor() still reports the estimated noise mean in whole
dBm, triggerNoiseFloorCalibrate() still conveys int.thresh, and Dispatcher
is untouched.
The batch estimator (64 samples, accepted only while !isReceivingPacket()
and below _noise_floor + 14) had four coupled, platform-independent
problems, measured over 20 h on a live repeater:
* Its window was emergent: 64 qualifying samples with no time bound,
so 64 / (loop_rate * qualifying_fraction). Mean 2.5 s, but 1.2% of
windows took 10-64 s.
* It was driven by loop() rate, so the same radio in the same RF
environment characterised the channel differently on a fast MCU (64
correlated GET_RSSI_INST reads a few ms apart) than on a slower or
sleeping board.
* Gating on !isReceivingPacket() biased the samples toward quiet
moments -- under-reporting the floor exactly when the band is busy --
and since 79ef74e that call is not a pure read (it drives a timeout
state machine and clears IRQ flags), so the sampler was participating
in packet detection.
* Accepting only samples below _noise_floor + 14 made the estimate
depend on its own previous value; resetAGC() re-seeding it existed
partly to escape the resulting stuck-at-(-120) attractor.
A quantile tracker was tried first and turned out robust to the wrong
thing: quantile estimators resist contamination but have essentially zero
breakdown point against runs. Over 20 h it logged 316 excursions, 11.9% of
wall time above -108 dBm against a -115 dBm median, each climbing at
exactly its own slew limit -- for 12% of the time the reported floor was
neither the noise floor nor the interference level, only how far the
tracker had crawled.
Minimum statistics (Martin, IEEE Trans. Speech & Audio Processing, 2001)
selects rather than chases. Signal only ever adds power, so the lowest
readings in a window are noise-only by construction; a burst shorter than
the window cannot move the estimate at all, and breakdown against a run is
(U-1)/U of the window. The statistic kept per sub-window is the SECOND
smallest raw reading, so an isolated low outlier is never the stored value
(an EMA ahead of the minimiser was tried and cost 4.7 dB of upward bias
under 20% dispersed interference; the second smallest reads within
0.08 dB). Window: 12 sub-windows x 150 samples = 180 s, sized against the
67 s worst excursion measured on hardware; the first sub-window is 30
samples so a fresh node has a floor in ~3 s. Cost: 12 floats plus a few
scalars.
The window statistic sits biasK * sigma below the mean; biasK is a table
indexed by ring occupancy, determined by simulating this exact algorithm
(the asymptotic extreme-value formula is ~20% off at these sizes, and one
constant over-corrects by up to 0.9 dB during warm-up). Sigma is the mean
excess of samples over the window statistic, censored at a fixed 8.5 dB
above it. Anchor and gate are both deliberately sigma-independent:
measuring the spread about the floor estimate, or gating relative to
sigma, closes a positive feedback loop under sustained near-floor traffic
-- simulated at 80% occupancy it pinned sigma at the 3 dB cap and widened
the CSMA margin from 2.8 dB to 10.5 dB. The fixed gate removes the loop;
the header documents the measured sweep and why no gate can do better.
Behaviour changes:
* isChannelActive() still treats interference_threshold as dB, but
floors the margin at NOISE_THRESHOLD_SIGMA_K (3.5) * sigma, with
sigma clamped to [0.5, 3.0] dB. A margin narrower than the noise
itself produced continuous false-busy, which protects nothing: every
packet waits out getCADFailMaxDuration() and goes anyway.
* resetAGC() no longer re-seeds the estimate. Measured, that was
strictly harmful: the reset goes through sleep() -> startReceive(),
which clears the IRQ flags, so a packet already in the air is joined
mid-symbol and isReceivingPacket() reports idle for the rest of it.
20 of 257 re-seeds over 20 h landed on signal that way, throwing the
floor to -85..-95 dBm against a true -114.
* The estimate is discarded when the receiver changes: updatePreamble()
(which every setParams() override routes through) and a successful
setRxBoostedGainMode() call the new resetNoiseFloor(). A bandwidth
change moves the thermal floor ~6 dB, and the estimator's slow-rise
asymmetry would otherwise read busy for the full 180 s window.
* The MESH_DEBUG_NOISE_FLOOR line is rate limited to one per 2 s and
also prints sigma in tenths of a dB.
All tunables are build_flags-overridable macros (rather than in-class
static constexpr float, so an ODR-use cannot produce a link error on the
C++11/14 toolchains some targets build with); a static_assert ties the
bias table to the ring depth.
test/test_noise_floor_tracker covers the failures above on the host: a
67 s sustained interferer, interference outlasting the window, the
fast-fall/slow-rise asymmetry, the low-outlier weakness, the sigma ratchet
under 80% occupancy, and the warm-up direction at every ring occupancy.
Clean noise recovers the mean within 0.04 dB and sigma within 3%.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXSCjgNEbJfHwLjD2WSHW4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the batch noise-floor estimator in
RadioLibWrapperwith a robust, continuously running one (NoiseFloorTracker), sampled on a wall-clock schedule instead of once perloop()call. The publicmesh::Radiointerface is unchanged:getNoiseFloor()still reports the estimated noise mean in whole dBm,triggerNoiseFloorCalibrate()still conveys the operator'sint.thresh, and Dispatcher is untouched.Why
The batch estimator (64 samples, accepted only when
!isReceivingPacket()and below_noise_floor + 14) had four coupled problems, all platform-independent, found by running a repeater for 20 h and logging the reported floor:64 / (loop_rate * qualifying_fraction). Measured: mean 2.5 s, but 1.2 % of windows took 10–64 s.loop()rate, so the same radio in the same RF environment characterised the channel differently on a fast MCU (64 heavily correlatedGET_RSSI_INSTreads a few ms apart) than on a slower or sleeping board.!isReceivingPacket()biased the sample population toward quiet moments, under-reporting the floor exactly when the band is busy. Since 79ef74eisReceivingPacket()is also not a pure read (it drives a timeout state machine and callsclearIrqFlags()), so polling it from the sampler had the noise floor participating in packet detection._noise_floor + 14made the estimate depend on its own previous value.resetAGC()re-seeding the estimate existed partly to escape the resulting stuck-at-(−120) attractor.A first replacement (a quantile tracker) turned out to be robust to the wrong thing: quantile estimators resist contamination but have essentially zero breakdown point against runs. Over 20 h on a live repeater it logged 316 excursions, 11.9 % of wall time above −108 dBm against a −115 dBm median, each climbing at exactly the tracker's own slew limit — i.e. for 12 % of the time the reported floor was neither the noise floor nor the interference level, only how far the tracker had crawled.
The algorithm
src/helpers/NoiseFloorTracker.his Martin's minimum statistics (IEEE Trans. Speech & Audio Processing, 2001), which selects rather than chases:(U−1)/Uof the window.biasK · sigma.biasKis a 12-entry table indexed by ring occupancy, determined by simulating this exact algorithm (the asymptotic extreme-value formula is ~20 % off at these window sizes, and one constant over-corrects by up to 0.9 dB during warm-up).Sampling in
RadioLibWrapper::loop()is rate-limited toNOISE_SAMPLE_INTERVAL_MS(100 ms) of wall clock and resyncs rather than catches up after a gap, so returning from a long transmit does not feed a burst of correlated samples. It is opportunistic ("sample when called, at most every 100 ms"), never a Dispatcher timer, so it demands no wakeup and cannot cap how long a battery board sleeps. Samples are taken whilestate == STATE_RXonly, including mid-packet — the estimator discards those by construction.Behaviour changes to be aware of
int.threshgets a floor.isChannelActive()still treatsinterference_thresholdas dB, so existing settings behave as before, but it enforcesmargin = max(int.thresh, NOISE_THRESHOLD_SIGMA_K · sigma)withNOISE_THRESHOLD_SIGMA_K = 3.5and sigma clamped to[0.5, 3.0]dB. A margin narrower than the noise itself produced continuous false-busy, which protects nothing: it delays every packet untilgetCADFailMaxDuration()expires and the node transmits regardless. The worst case is a margin of ~10.5 dB when sigma is pinned at 3 dB, so a very tightint.threshmay have less effect than its number suggests; measured sigma on an SX1262 at 250 kHz is 0.6–0.9 dB.resetAGC()no longer re-seeds the estimate. Measured on a live repeater, that was strictly harmful: the reset goes throughsleep()→startReceive(), which clears the IRQ flags, so a packet already in the air is joined mid-symbol andisReceivingPacket()reports idle for the rest of it. 20 of 257 re-seeds over 20 h landed on signal that way, throwing the reported floor to −85…−95 dBm against a true −114. An AGC reset does not move the floor by anything like the estimator's tracking range, and a genuine change is followed on its own.updatePreamble()— which everysetParams()override routes through — and a successfulsetRxBoostedGainMode()callresetNoiseFloor(). A bandwidth change moves the thermal floor ~6 dB, and with the estimator's slow-rise asymmetry a stale floor would otherwise read busy for the full 180 s window. For the ~3 s until the first sub-window closes after a reset,_nf.ready()is false, the RSSI branch ofisChannelActive()is skipped, andgetNoiseFloor()reports 0 — accepted deliberately, and documented at the reset site.MESH_DEBUG_NOISE_FLOORline is rate limited to one per 20 samples (2 s), matching the previous log volume, and now also prints sigma in tenths of a dB.Tunables
All
NOISE_TRACKER_*andNOISE_*values are macros overridable per environment viabuild_flags(macros rather than in-classstatic constexpr floatso an ODR-use cannot produce a link error on the C++11/14 toolchains some targets build with). Astatic_assertties the bias table to the ring depth so the two cannot be overridden inconsistently.Tests
test/test_noise_floor_tracker(googletest, host): 24 cases, including the ones that caught the failures above — a 67 s sustained interferer, interference outlasting the window, the fast-fall/slow-rise asymmetry, the low-outlier weakness, the sigma ratchet under 80 % occupancy at 8 dB and 5 dB, and the warm-up direction at every ring occupancy. Clean noise recovers the mean within 0.04 dB and sigma within 3 %.Files
src/helpers/NoiseFloorTracker.h(new)src/helpers/radiolib/RadioLibWrappers.{h,cpp}src/helpers/radiolib/Custom{SX1262,SX1268,LLCC68,LR1110}Wrapper.h(one-lineresetNoiseFloor()on a successful gain-mode change; the LLCC68 wrapper is not compiled by any environment in the tree, see the companion fixes PR)test/test_noise_floor_tracker/Dependencies
Independent. Applies to
dev.