From fcac280dbe411d27bd4b6d83064b166a59d126c3 Mon Sep 17 00:00:00 2001 From: mmmorks Date: Mon, 7 Sep 2026 14:21:52 -0700 Subject: [PATCH] Estimate the noise floor by minimum statistics, sampled at a fixed rate Replace RadioLibWrapper's batch noise-floor estimator with a continuously running one (NoiseFloorTracker), sampled on a 100 ms wall-clock schedule inside loop() rather than once per iteration. mesh::Radio's interface is unchanged: getNoiseFloor() still reports the estimated noise mean in whole dBm, triggerNoiseFloorCalibrate() still conveys int.thresh, and Dispatcher is untouched. The batch estimator (64 samples, accepted only while !isReceivingPacket() and below _noise_floor + 14) had four coupled, platform-independent problems, measured over 20 h on a live repeater: * Its window was emergent: 64 qualifying samples with no time bound, so 64 / (loop_rate * qualifying_fraction). Mean 2.5 s, but 1.2% of windows took 10-64 s. * It was driven by loop() rate, so the same radio in the same RF environment characterised the channel differently on a fast MCU (64 correlated GET_RSSI_INST reads a few ms apart) than on a slower or sleeping board. * Gating on !isReceivingPacket() biased the samples toward quiet moments -- under-reporting the floor exactly when the band is busy -- and since 79ef74ea that call is not a pure read (it drives a timeout state machine and clears IRQ flags), so the sampler was participating in packet detection. * Accepting only samples below _noise_floor + 14 made the estimate depend on its own previous value; resetAGC() re-seeding it existed partly to escape the resulting stuck-at-(-120) attractor. A quantile tracker was tried first and turned out robust to the wrong thing: quantile estimators resist contamination but have essentially zero breakdown point against runs. Over 20 h it logged 316 excursions, 11.9% of wall time above -108 dBm against a -115 dBm median, each climbing at exactly its own slew limit -- for 12% of the time the reported floor was neither the noise floor nor the interference level, only how far the tracker had crawled. Minimum statistics (Martin, IEEE Trans. Speech & Audio Processing, 2001) selects rather than chases. Signal only ever adds power, so the lowest readings in a window are noise-only by construction; a burst shorter than the window cannot move the estimate at all, and breakdown against a run is (U-1)/U of the window. The statistic kept per sub-window is the SECOND smallest raw reading, so an isolated low outlier is never the stored value (an EMA ahead of the minimiser was tried and cost 4.7 dB of upward bias under 20% dispersed interference; the second smallest reads within 0.08 dB). Window: 12 sub-windows x 150 samples = 180 s, sized against the 67 s worst excursion measured on hardware; the first sub-window is 30 samples so a fresh node has a floor in ~3 s. Cost: 12 floats plus a few scalars. The window statistic sits biasK * sigma below the mean; biasK is a table indexed by ring occupancy, determined by simulating this exact algorithm (the asymptotic extreme-value formula is ~20% off at these sizes, and one constant over-corrects by up to 0.9 dB during warm-up). Sigma is the mean excess of samples over the window statistic, censored at a fixed 8.5 dB above it. Anchor and gate are both deliberately sigma-independent: measuring the spread about the floor estimate, or gating relative to sigma, closes a positive feedback loop under sustained near-floor traffic -- simulated at 80% occupancy it pinned sigma at the 3 dB cap and widened the CSMA margin from 2.8 dB to 10.5 dB. The fixed gate removes the loop; the header documents the measured sweep and why no gate can do better. Behaviour changes: * isChannelActive() still treats interference_threshold as dB, but floors the margin at NOISE_THRESHOLD_SIGMA_K (3.5) * sigma, with sigma clamped to [0.5, 3.0] dB. A margin narrower than the noise itself produced continuous false-busy, which protects nothing: every packet waits out getCADFailMaxDuration() and goes anyway. * resetAGC() no longer re-seeds the estimate. Measured, that was strictly harmful: the reset goes through sleep() -> startReceive(), which clears the IRQ flags, so a packet already in the air is joined mid-symbol and isReceivingPacket() reports idle for the rest of it. 20 of 257 re-seeds over 20 h landed on signal that way, throwing the floor to -85..-95 dBm against a true -114. * The estimate is discarded when the receiver changes: updatePreamble() (which every setParams() override routes through) and a successful setRxBoostedGainMode() call the new resetNoiseFloor(). A bandwidth change moves the thermal floor ~6 dB, and the estimator's slow-rise asymmetry would otherwise read busy for the full 180 s window. * The MESH_DEBUG_NOISE_FLOOR line is rate limited to one per 2 s and also prints sigma in tenths of a dB. All tunables are build_flags-overridable macros (rather than in-class static constexpr float, so an ODR-use cannot produce a link error on the C++11/14 toolchains some targets build with); a static_assert ties the bias table to the ring depth. test/test_noise_floor_tracker covers the failures above on the host: a 67 s sustained interferer, interference outlasting the window, the fast-fall/slow-rise asymmetry, the low-outlier weakness, the sigma ratchet under 80% occupancy, and the warm-up direction at every ring occupancy. Clean noise recovers the mean within 0.04 dB and sigma within 3%. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01EXSCjgNEbJfHwLjD2WSHW4 --- src/helpers/NoiseFloorTracker.h | 361 ++++++++++++ src/helpers/radiolib/CustomLLCC68Wrapper.h | 7 +- src/helpers/radiolib/CustomLR1110Wrapper.h | 7 +- src/helpers/radiolib/CustomSX1262Wrapper.h | 7 +- src/helpers/radiolib/CustomSX1268Wrapper.h | 7 +- src/helpers/radiolib/RadioLibWrappers.cpp | 153 ++++-- src/helpers/radiolib/RadioLibWrappers.h | 49 +- .../test_noise_floor_tracker.cpp | 513 ++++++++++++++++++ 8 files changed, 1060 insertions(+), 44 deletions(-) create mode 100644 src/helpers/NoiseFloorTracker.h create mode 100644 test/test_noise_floor_tracker/test_noise_floor_tracker.cpp diff --git a/src/helpers/NoiseFloorTracker.h b/src/helpers/NoiseFloorTracker.h new file mode 100644 index 0000000000..939cf956b0 --- /dev/null +++ b/src/helpers/NoiseFloorTracker.h @@ -0,0 +1,361 @@ +#pragma once + +#include + +// Tunables are macros rather than in-class `static constexpr float` so that +// ODR-using one (binding a reference, passing it to a function) cannot produce +// a link error on the C++11/14 toolchains some MeshCore targets build with. +// Each is overridable per-platform via build_flags. + +#ifndef NOISE_TRACKER_SUB_SAMPLES + // Samples per sub-window (15 s at a 100 ms sample interval). This is the + // resolution at which the estimate can rise when the floor genuinely rises. + #define NOISE_TRACKER_SUB_SAMPLES 150 +#endif + +#ifndef NOISE_TRACKER_FIRST_SUB_SAMPLES + // The first sub-window is short so a fresh node has a usable floor in ~3 s + // rather than 15 s. It lands in the ring alongside full-length sub-windows; + // a minimum over fewer samples is biased high, and since the window estimate + // takes the minimum across the ring, a high entry is simply never selected. + #define NOISE_TRACKER_FIRST_SUB_SAMPLES 30 +#endif + +#ifndef NOISE_TRACKER_SUB_WINDOWS + // Ring depth. Total window = SUB_WINDOWS * SUB_SAMPLES = 1800 samples = 180 s. + // This is the estimator's robustness budget: any interference burst shorter + // than the window leaves clean samples in it. Sized against measurement -- + // the longest excursion observed on a live repeater over 20 h was 67 s, so + // 180 s carries ~2.7x margin. Costs one float each. + #define NOISE_TRACKER_SUB_WINDOWS 12 +#endif + +#ifndef NOISE_TRACKER_MIN_SIGMA_DB + // Scale floor. Stops a freshly seeded tracker from reporting a zero-width + // noise distribution, which would collapse any threshold derived from sigma(). + #define NOISE_TRACKER_MIN_SIGMA_DB 0.5f +#endif + +#ifndef NOISE_TRACKER_MAX_SIGMA_DB + // Scale ceiling. Receiver noise spread is set by bandwidth, temperature and + // LNA gain state; measured on a live SX1262 at 250 kHz the median sits at + // 0.6-0.9 dB, so 3 dB is generous headroom and anything beyond it is not a + // noise spread. NOISE_TRACKER_SCALE_GATE_DB is derived from this value; + // raising one without the other leaves the gate as the binding constraint. + #define NOISE_TRACKER_MAX_SIGMA_DB 3.0f +#endif + +#ifndef NOISE_TRACKER_SIGMA_LAMBDA + // EMA rate for the scale estimate (~50-sample time constant, 5 s at 100 ms + // sampling). Slow on purpose: the spread physically changes far more slowly + // than the mean. + #define NOISE_TRACKER_SIGMA_LAMBDA 0.02f +#endif + +#ifndef NOISE_TRACKER_SCALE_GATE_DB + // Censoring gate for the scale estimate: samples more than this far above the + // window statistic are treated as signal and take no part in it. Signal + // energy must not widen the estimated noise spread. + // + // Measured from windowValue(), deliberately, and NOT from the floor estimate + // or from any multiple of sigma. An earlier revision censored above + // windowValue() + biasK*sigma + max(6 dB, 4*sigma): both of those terms grow + // with the quantity the gate exists to protect, which closes a positive + // feedback loop under sustained near-floor traffic. Packets a few dB above + // the floor are admitted, they raise _dev, the wider gate then admits + // stronger packets, and sigma ratchets up until MAX_SIGMA stops it. Simulated + // against this exact code -- 80% channel occupancy, packets 8 dB above a + // 0.8 dB floor -- sigma reached the 3.0 dB cap, which silently widens the + // CSMA margin RadioLibWrapper derives from it (NOISE_THRESHOLD_SIGMA_K * + // sigma) from 2.8 dB to 10.5 dB, overriding the operator's + // interference_threshold in exactly the busy mesh where it was set. The same + // run settles at 1.5 dB with the gate fixed -- but see the sweep below before + // taking that one figure for the whole story. + // + // What a fixed gate buys is the removal of that loop, and nothing more. It is + // NOT a tighter numeric bound: GATE/biasK is 8.5/2.835 = 3.0, the same number + // MAX_SIGMA already imposes. The difference is structural -- signal above the + // gate is excluded however much of it there is, instead of being kept out + // only until enough of it gets in to move the gate. + // + // Traffic *inside* the gate still inflates sigma, and needs no feedback loop + // to do it. At a 0.8 dB spread the gate sits about 6.2 dB above the mean + // (8.5 - biasK*0.8), so sustained traffic parked just under that is admitted + // wholesale. Swept at 80% occupancy against a 0.8 dB floor, worst sigma by + // packet level above the mean: + // + // dB above mean: +2 +3 +4 +5 +6 +7 +8 +9 + // this gate: 1.45 1.74 2.04 2.27 2.35 2.24 1.51 0.90 + // old gate: 1.45 1.74 2.04 2.33 2.63 2.92 3.00 0.87 + // + // So the fix is worth nothing below +5, everything above +6, and the residual + // peak is 2.35 dB at +6 -- a CSMA margin of 8.2 dB where 2.8 would be right. + // Do not read the gate as a guarantee about sigma; read it as a guarantee + // that the *set* of samples reaching sigma cannot grow. + // + // No gate can do better here. Censoring a packet 5 dB above the floor means + // sitting below mean + 5, which censors genuine noise at any spread past + // ~1.2 dB. Signal that weak and noise that wide are not separable by an RSSI + // threshold, and pretending otherwise just moves the damage. + // + // The value is biasK * MAX_SIGMA (2.835 * 3.0): never look further above the + // window statistic than the largest mean offset the estimator is allowed to + // report. Measured from the mean rather than from windowValue() the gate is + // still >=4 sigma for spreads up to 1.25 dB and >=3 sigma up to 1.45 dB, + // against the 0.6-0.9 dB these receivers actually produce, so censoring bias + // on genuine noise stays negligible across the whole plausible range. Above + // ~2 dB the noise tail starts to overflow the gate and sigma reads low + // (simulated: 1.98 dB at a true 2.5, 2.12 at a true 3.0). That is the + // deliberate trade, and it is the safe direction: a low sigma reads the floor + // low, which fires the busy check early -- absorbed by CSMA backoff -- rather + // than losing a detection. + #define NOISE_TRACKER_SCALE_GATE_DB 8.5f +#endif + +#ifndef NOISE_TRACKER_MIN_DBM + // Same clamp the original estimator applied: below this is not physically + // plausible for these receivers. + #define NOISE_TRACKER_MIN_DBM (-120) +#endif + +#ifndef NOISE_TRACKER_MIN_VALID_DBM + // Readings below this are discarded as bad reads rather than clamped. The + // output clamp alone is not enough protection for a minimum-based estimator: + // one spurious -200 dBm read would pin the window minimum for a full 180 s, + // and clamping the *output* to MIN_DBM would hide that as a plausible-looking + // stuck floor. + #define NOISE_TRACKER_MIN_VALID_DBM (-135.0f) +#endif + +// Bias correction, indexed by how many sub-windows have completed (1..N). +// +// The k-th smallest of a window sits below the distribution mean, so it must be +// corrected back up: mean ~= window_value + k * sigma. The coefficient depends +// only on how many samples were minimised over -- it is independent of sigma, +// as it must be, since the bias is proportional to it. +// +// Determined by simulating this exact algorithm rather than from the asymptotic +// extreme-value formula, which is not accurate at these window sizes. Indexing +// by occupancy rather than using one constant matters during warm-up: a partly +// filled ring has been minimised over fewer samples, so its value is closer to +// the mean, and the full-window coefficient would over-correct there. +// +// What that costs is sigma(), not the reported floor. The coefficient cancels +// out of the floor: updateScale() divides the measured mean excess by it and +// floorEstimate() multiplies it straight back, so as long as neither sigma +// clamp binds the floor is windowValue() + _dev whatever the table says. +// Getting it wrong therefore misreports the *spread*: the full-window value +// used throughout reads 16% low at the first sub-window and 12% at the second +// (measured, at sigma = 2) while leaving the floor within 0.03 dB of correct at +// both. That still matters, because sigma sets the CSMA margin under a +// too-narrow interference_threshold (NOISE_THRESHOLD_SIGMA_K * sigma, in +// RadioLibWrapper::isChannelActive) and decides where MIN_SIGMA/MAX_SIGMA start +// binding -- and those clamps are the only route by which a wrong coefficient +// can move the reported floor at all. +#ifndef NOISE_TRACKER_BIAS_TABLE + #define NOISE_TRACKER_BIAS_TABLE { \ + 2.375f, 2.502f, 2.579f, 2.632f, 2.674f, 2.708f, \ + 2.737f, 2.761f, 2.783f, 2.802f, 2.819f, 2.835f } +#endif + +/** + * \brief Robust noise-floor estimator for a stream of RSSI readings. + * + * Estimates the noise floor from a low order statistic of the RSSI readings + * over a sliding window, bias-corrected back to the distribution mean (Martin's + * minimum statistics, IEEE Trans. Speech & Audio Processing, 2001). + * + * The method rests on one asymmetry that holds for any radio: signal only ever + * ADDS power. The lowest readings in a window are therefore noise-only samples + * by construction -- no activity gate, no outlier rejection, and no assumption + * that interference is a minority of the samples. + * + * The statistic is the SECOND smallest reading of each sub-window, not the + * smallest. That single step is what makes the method safe here: the minimum is + * by construction the most outlier-sensitive statistic there is, so one spurious + * low reading would pin the floor for a whole window. The second smallest is + * indifferent to any isolated low sample while being no more reachable by signal + * than the first. It also lets the estimator work on raw readings -- an earlier + * revision pre-smoothed with an EMA to blunt low outliers, which cost 4.7 dB of + * upward bias under 20% dispersed interference because the smoothed sequence + * never settled to the true floor between bursts. + * + * That last point is why this replaced a quantile tracker. Quantile estimators + * are robust to *contamination* (a minority of samples drawn from another + * distribution) but have essentially zero breakdown point against *runs*: a + * sustained interferer simply drags the estimate along at its maximum slew + * rate. On a live repeater this produced hour-long stretches reporting a value + * that was neither the noise floor nor the interference level, only how far the + * tracker had crawled. Minimum statistics does not chase -- it selects -- so + * its breakdown point against a run is (U-1)/U of the window length. + * + * The estimate falls instantly when the floor genuinely falls, and rises only + * as contaminated sub-windows age out. That asymmetry is deliberate and is the + * correct one for a noise floor: real increases are rare and slow, spurious + * ones are common and fast. Reading low is also the safe direction -- it makes + * a busy-channel check fire slightly early (absorbed by CSMA backoff) rather + * than late (a lost detection, the expensive error). + * + * Deliberately free of any clock, radio, or Arduino dependency: the caller + * decides when to sample, which keeps the estimator unit-testable on the host + * and its behaviour independent of how often the main loop happens to run. + */ +class NoiseFloorTracker { + float _lo0, _lo1; // two smallest of current sub-window + uint16_t _sub_count; // samples into current sub-window + uint16_t _sub_target; // samples needed to close it + float _mins[NOISE_TRACKER_SUB_WINDOWS]; // ring of completed sub-window statistics + uint8_t _head; // next ring slot to write + uint8_t _valid; // completed sub-windows, saturating at ring size + float _dev; // mean excess of noise samples over windowValue() + float _sigma; + + // Sentinel for "no sample yet". Any real dBm reading is far below it, so + // comparisons need no special case. + static float sentinel() { return 1.0e30f; } + + /** Bias coefficient for the current ring occupancy (_valid >= 1). */ + float biasK() const { + // Unsized on purpose, so sizeof measures the table rather than the constant + // it is supposed to match. Both tunables are documented as independently + // overridable via build_flags, and a table shorter than the ring is the + // dangerous mismatch: an explicit bound would zero-fill the tail in silence, + // so biasK() would return 0 at exactly the occupancies that matter most and + // the floor would read ~2.8 dB low -- the direction that loses detections. + static const float k[] = NOISE_TRACKER_BIAS_TABLE; + static_assert(sizeof(k) / sizeof(k[0]) == NOISE_TRACKER_SUB_WINDOWS, + "NOISE_TRACKER_BIAS_TABLE must have exactly " + "NOISE_TRACKER_SUB_WINDOWS entries: override both or neither"); + uint8_t i = _valid > 0 ? (uint8_t)(_valid - 1) : 0; + if (i >= NOISE_TRACKER_SUB_WINDOWS) i = NOISE_TRACKER_SUB_WINDOWS - 1; + return k[i]; + } + + /** Lowest sub-window statistic across the window, including the in-progress + sub-window so a genuine drop in the floor starts showing immediately + rather than waiting for that sub-window to close. */ + float windowValue() const { + float m = sentinel(); + for (uint8_t i = 0; i < _valid; i++) { + if (_mins[i] < m) m = _mins[i]; + } + if (_lo1 < m) m = _lo1; // sentinel until the in-progress window has 2 samples + return m; + } + + /** Bias-corrected floor as a float. Undefined before the first sub-window. */ + float floorEstimate() const { + return windowValue() + biasK() * _sigma; + } + +public: + NoiseFloorTracker() { reset(); } + + /** Discard all state; the estimator re-warms from the next sample. */ + void reset() { + _lo0 = _lo1 = sentinel(); + _sub_count = 0; + _sub_target = NOISE_TRACKER_FIRST_SUB_SAMPLES; + _head = 0; + _valid = 0; + _sigma = NOISE_TRACKER_MIN_SIGMA_DB; + _dev = NOISE_TRACKER_MIN_SIGMA_DB * biasK(); // consistent with _sigma; needs _valid set + for (uint8_t i = 0; i < NOISE_TRACKER_SUB_WINDOWS; i++) _mins[i] = sentinel(); + } + + /** True once at least one sub-window has closed and floorDbm() is meaningful. */ + bool ready() const { return _valid > 0; } + + /** + * Feed one RSSI reading, in dBm. Call at a steady rate -- the window length + * is expressed in samples, so a varying rate varies the time it spans. + * + * Safe to call during packet reception. Samples that contain signal only + * raise the sequence, and a low order statistic ignores them; that is what + * removed the activity gate this estimator used to need, and with it the bias + * toward quiet moments the gate imposed. + */ + void addSample(float rssi_dbm) { + if (rssi_dbm < NOISE_TRACKER_MIN_VALID_DBM) return; // bad read, not a quiet channel + + // Keep the two smallest readings of this sub-window, _lo0 <= _lo1. + if (rssi_dbm < _lo0) { _lo1 = _lo0; _lo0 = rssi_dbm; } + else if (rssi_dbm < _lo1) { _lo1 = rssi_dbm; } + + if (++_sub_count >= _sub_target) { + _mins[_head] = _lo1; + _head = (uint8_t)((_head + 1) % NOISE_TRACKER_SUB_WINDOWS); + if (_valid < NOISE_TRACKER_SUB_WINDOWS) _valid++; + _lo0 = _lo1 = sentinel(); + _sub_count = 0; + _sub_target = NOISE_TRACKER_SUB_SAMPLES; // only the first one is short + } + + updateScale(rssi_dbm); + } + + /** + * \returns estimated standard deviation (dB) of the noise-only RSSI, always + * within [NOISE_TRACKER_MIN_SIGMA_DB, NOISE_TRACKER_MAX_SIGMA_DB]. + * + * Measured as the mean amount by which noise samples exceed the window + * statistic. That gap is exactly what the bias table predicts -- biasK() * + * sigma -- so dividing by biasK() inverts it. Samples far enough above the + * window statistic are censored out, which stops channel occupancy from + * widening this without limit. It does not stop it widening this at all: + * traffic inside the gate is indistinguishable from a wide noise floor, and + * sustained traffic 4-6 dB above the floor still reads as ~2.3 dB of spread + * against a true 0.8. NOISE_TRACKER_SCALE_GATE_DB has the measured sweep and + * the reason no gate can do better. + * + * Anchoring to windowValue() rather than to the floor estimate is deliberate, + * and it applies to the censoring gate as much as to the deviation itself. + * The floor is derived from sigma, so anything measured about it closes a + * positive feedback loop: a floor reading Delta too high makes the mean + * deviation ~Delta, which inflates sigma, which raises the floor further; a + * gate placed relative to it widens as sigma grows and admits the very signal + * that grew it. windowValue() is sigma-independent, so neither loop exists -- + * which fixes the set of samples that can reach this estimate, not the value + * it can reach. + */ + float sigma() const { return _sigma; } + + /** + * \returns estimated *mean* noise floor in whole dBm, clamped at + * NOISE_TRACKER_MIN_DBM, or 0 before the first sub-window closes. + * + * The mean rather than the raw window statistic, so the reported value stays + * comparable with the estimators this replaced (and therefore with other + * MeshCore nodes and historical telemetry). + */ + int16_t floorDbm() const { + if (_valid == 0) return 0; // "not calibrated yet", as previous estimators reported + float f = floorEstimate(); + // Round half away from zero without pulling in libm. + int v = (int) (f < 0.0f ? f - 0.5f : f + 0.5f); + return v < NOISE_TRACKER_MIN_DBM ? (int16_t) NOISE_TRACKER_MIN_DBM : (int16_t) v; + } + +private: + /** Update the scale estimate from samples that are plausibly noise-only. */ + void updateScale(float rssi_dbm) { + if (_valid == 0) return; // no window statistic to measure against yet + + // One ring scan serves both uses below: the censoring gate and the + // deviation are both anchored on the window statistic, which is the only + // quantity here that neither signal nor the scale estimate can move. + // Reading it once also makes the two impossible to drift apart. + const float wv = windowValue(); + + if (rssi_dbm >= wv + NOISE_TRACKER_SCALE_GATE_DB) return; // signal or interferer, not noise + + float d = rssi_dbm - wv; + if (d < 0.0f) d = 0.0f; // below the window statistic only by sampling noise + _dev += NOISE_TRACKER_SIGMA_LAMBDA * (d - _dev); + + float s = _dev / biasK(); + if (s > NOISE_TRACKER_MAX_SIGMA_DB) s = NOISE_TRACKER_MAX_SIGMA_DB; + if (s < NOISE_TRACKER_MIN_SIGMA_DB) s = NOISE_TRACKER_MIN_SIGMA_DB; + _sigma = s; + } +}; diff --git a/src/helpers/radiolib/CustomLLCC68Wrapper.h b/src/helpers/radiolib/CustomLLCC68Wrapper.h index ae0fe0a253..26f2b66a60 100644 --- a/src/helpers/radiolib/CustomLLCC68Wrapper.h +++ b/src/helpers/radiolib/CustomLLCC68Wrapper.h @@ -37,8 +37,13 @@ class CustomLLCC68Wrapper : public RadioLibWrapper { void doResetAGC() override { sx126xResetAGC((SX126x *)_radio); } + // Changing the LNA gain state moves the noise floor, and this path does not + // go through setParams(). See RadioLibWrapper::resetNoiseFloor(); reset only + // on a successful change, since a rejected one leaves the frontend alone. bool setRxBoostedGainMode(bool en) override { - return ((CustomLLCC68 *)_radio)->setRxBoostedGainMode(en) == RADIOLIB_ERR_NONE; + if (((CustomLLCC68 *)_radio)->setRxBoostedGainMode(en) != RADIOLIB_ERR_NONE) return false; + resetNoiseFloor(); + return true; } bool getRxBoostedGainMode() const override { return ((CustomLLCC68 *)_radio)->getRxBoostedGainMode(); diff --git a/src/helpers/radiolib/CustomLR1110Wrapper.h b/src/helpers/radiolib/CustomLR1110Wrapper.h index e7aaeb937a..97160b203f 100644 --- a/src/helpers/radiolib/CustomLR1110Wrapper.h +++ b/src/helpers/radiolib/CustomLR1110Wrapper.h @@ -43,8 +43,13 @@ class CustomLR1110Wrapper : public RadioLibWrapper { uint8_t getSpreadingFactor() const override { return ((CustomLR1110 *)_radio)->getSpreadingFactor(); } + // Changing the LNA gain state moves the noise floor, and this path does not + // go through setParams(). See RadioLibWrapper::resetNoiseFloor(); reset only + // on a successful change, since a rejected one leaves the frontend alone. bool setRxBoostedGainMode(bool en) override { - return ((CustomLR1110 *)_radio)->setRxBoostedGainMode(en) == RADIOLIB_ERR_NONE; + if (((CustomLR1110 *)_radio)->setRxBoostedGainMode(en) != RADIOLIB_ERR_NONE) return false; + resetNoiseFloor(); + return true; } bool getRxBoostedGainMode() const override { return ((CustomLR1110 *)_radio)->getRxBoostedGainMode(); diff --git a/src/helpers/radiolib/CustomSX1262Wrapper.h b/src/helpers/radiolib/CustomSX1262Wrapper.h index be3144716b..5f59d1de12 100644 --- a/src/helpers/radiolib/CustomSX1262Wrapper.h +++ b/src/helpers/radiolib/CustomSX1262Wrapper.h @@ -41,8 +41,13 @@ class CustomSX1262Wrapper : public RadioLibWrapper { ((CustomSX1262 *)_radio)->sleep(false); } + // Changing the LNA gain state moves the noise floor, and this path does not + // go through setParams(). See RadioLibWrapper::resetNoiseFloor(); reset only + // on a successful change, since a rejected one leaves the frontend alone. bool setRxBoostedGainMode(bool en) override { - return ((CustomSX1262 *)_radio)->setRxBoostedGainMode(en) == RADIOLIB_ERR_NONE; + if (((CustomSX1262 *)_radio)->setRxBoostedGainMode(en) != RADIOLIB_ERR_NONE) return false; + resetNoiseFloor(); + return true; } bool getRxBoostedGainMode() const override { return ((CustomSX1262 *)_radio)->getRxBoostedGainMode(); diff --git a/src/helpers/radiolib/CustomSX1268Wrapper.h b/src/helpers/radiolib/CustomSX1268Wrapper.h index 70f5dabdc6..19b921e6cb 100644 --- a/src/helpers/radiolib/CustomSX1268Wrapper.h +++ b/src/helpers/radiolib/CustomSX1268Wrapper.h @@ -38,8 +38,13 @@ class CustomSX1268Wrapper : public RadioLibWrapper { } uint8_t getSpreadingFactor() const override { return ((CustomSX1268 *)_radio)->spreadingFactor; } + // Changing the LNA gain state moves the noise floor, and this path does not + // go through setParams(). See RadioLibWrapper::resetNoiseFloor(); reset only + // on a successful change, since a rejected one leaves the frontend alone. bool setRxBoostedGainMode(bool en) override { - return ((CustomSX1268 *)_radio)->setRxBoostedGainMode(en) == RADIOLIB_ERR_NONE; + if (((CustomSX1268 *)_radio)->setRxBoostedGainMode(en) != RADIOLIB_ERR_NONE) return false; + resetNoiseFloor(); + return true; } bool getRxBoostedGainMode() const override { return ((CustomSX1268 *)_radio)->getRxBoostedGainMode(); diff --git a/src/helpers/radiolib/RadioLibWrappers.cpp b/src/helpers/radiolib/RadioLibWrappers.cpp index e4d2ba1c27..bc40896a9d 100644 --- a/src/helpers/radiolib/RadioLibWrappers.cpp +++ b/src/helpers/radiolib/RadioLibWrappers.cpp @@ -8,8 +8,36 @@ #define STATE_TX_DONE 4 #define STATE_INT_READY 16 -#define NUM_NOISE_FLOOR_SAMPLES 64 -#define SAMPLING_THRESHOLD 14 +// How often the noise floor is sampled, in wall-clock terms. Sampling is rate +// limited here rather than taken once per loop() call so the estimate does not +// depend on how fast the host happens to iterate: an idle Linux daemon blocking +// on poll() and an MCU spinning flat out must characterise the channel the same +// way. It also keeps consecutive samples far enough apart to be worth taking -- +// back-to-back GET_RSSI_INST reads are correlated, so they add little +// information for their SPI cost. +#ifndef NOISE_SAMPLE_INTERVAL_MS + #define NOISE_SAMPLE_INTERVAL_MS 100 +#endif + +// Minimum busy-channel margin, as a multiple of the estimated noise sigma. This +// is what sets the false-alarm rate of the interference check: for normally +// distributed noise, P(sample > mean + 3.5 sigma) = 2.3e-4 per isChannelActive() +// call, so a transmit attempt (a handful of calls) defers spuriously about once +// in a thousand -- comfortably absorbed by the CSMA backoff that follows. +// +// Applied as a floor under the operator's configured dB margin, not as a +// replacement for it: interference_threshold keeps meaning dB, but a margin +// narrower than the noise itself can no longer produce continuous false busy +// and trip ERR_EVENT_CAD_TIMEOUT. +#ifndef NOISE_THRESHOLD_SIGMA_K + #define NOISE_THRESHOLD_SIGMA_K 3.5f +#endif + +// One debug line per this many samples (20 * 100 ms = 2 s), matching the log +// volume of the batch estimator this replaced. +#ifndef NOISE_LOG_EVERY_N_SAMPLES + #define NOISE_LOG_EVERY_N_SAMPLES 20 +#endif static volatile uint8_t state = STATE_IDLE; @@ -34,13 +62,11 @@ void RadioLibWrapper::begin() { setFlag(); // LoRa packet is already received } - _noise_floor = 0; _threshold = 0; _cad_enabled = false; - // start average out some samples - _num_floor_samples = 0; - _floor_sample_sum = 0; + resetNoiseFloor(); // clears _nf, _noise_floor and the log rate limiter + _next_noise_sample = millis(); } uint32_t RadioLibWrapper::getRngSeed() { @@ -60,11 +86,11 @@ void RadioLibWrapper::idle() { } void RadioLibWrapper::triggerNoiseFloorCalibrate(int threshold) { + // The estimator now runs continuously, so there is no calibration batch to + // start and nothing here is periodic any more -- this only conveys the + // operator's interference threshold. Kept on the mesh::Radio interface, and + // still called on Dispatcher's 2 s timer, so that no caller has to change. _threshold = threshold; - if (_num_floor_samples >= NUM_NOISE_FLOOR_SAMPLES) { // ignore trigger if currently sampling - _num_floor_samples = 0; - _floor_sample_sum = 0; - } } void RadioLibWrapper::doResetAGC() { @@ -78,35 +104,75 @@ void RadioLibWrapper::resetAGC() { doResetAGC(); state = STATE_IDLE; // trigger a startReceive() - // Reset noise floor sampling so it reconverges from scratch. - // Without this, a stuck _noise_floor of -120 makes the sampling threshold - // too low (-106) to accept normal samples (~-105), self-reinforcing the - // stuck value even after the receiver has recovered. - _noise_floor = 0; - _num_floor_samples = 0; - _floor_sample_sum = 0; + // Deliberately does NOT reset the noise floor estimate. + // + // It used to, on the reasoning that the analog frontend had just changed so + // everything learned before it was about a different receiver. Measured on a + // live repeater that was strictly harmful: reset() re-seeds from a single + // sample, and the guard meant to keep that sample clean cannot do its job + // here. resetAGC() has just been through sleep() -> startReceive(), which + // clears the modem's IRQ flags, so a packet already in the air is joined + // mid-symbol -- its preamble and header are long past and neither will ever + // set again for that packet. isReceivingPacket() therefore reports "idle" + // precisely when it is most wrong, and stays wrong for the rest of the + // packet. 20 of 257 re-seeds over 20 h landed on signal that way, throwing + // the reported floor to -85..-95 dBm against a true floor of -114. + // + // Nothing is lost by keeping the estimate: an AGC reset does not move the + // noise floor by anything like the estimator's tracking range, and if it + // genuinely did, the estimator follows real floor changes on its own. } void RadioLibWrapper::loop() { - if (state == STATE_RX && _num_floor_samples < NUM_NOISE_FLOOR_SAMPLES) { - if (!isReceivingPacket()) { - int rssi = getCurrentRSSI(); - if (rssi < _noise_floor + SAMPLING_THRESHOLD) { // only consider samples below current floor + sampling THRESHOLD - _num_floor_samples++; - _floor_sample_sum += rssi; - } - } - } else if (_num_floor_samples >= NUM_NOISE_FLOOR_SAMPLES && _floor_sample_sum != 0) { - _noise_floor = _floor_sample_sum / NUM_NOISE_FLOOR_SAMPLES; - if (_noise_floor < -120) { - _noise_floor = -120; // clamp to lower bound of -120dBi - } - _floor_sample_sum = 0; + // Only sample while actually listening: during TX or standby the RSSI reading + // describes nothing about the channel. + if (state != STATE_RX) return; + + uint32_t now = millis(); + if ((int32_t)(now - _next_noise_sample) < 0) return; // not due yet + + _next_noise_sample += NOISE_SAMPLE_INTERVAL_MS; + if ((int32_t)(now - _next_noise_sample) >= 0) { + // More than one interval elapsed -- we were transmitting, or the caller + // stopped iterating for a while. Resync rather than catching up, otherwise + // the next few iterations would each fire immediately and feed a burst of + // correlated samples, which is exactly what the fixed rate exists to avoid. + _next_noise_sample = now + NOISE_SAMPLE_INTERVAL_MS; + } - #ifdef MESH_DEBUG_NOISE_FLOOR - MESH_DEBUG_PRINTLN("RadioLibWrapper: noise_floor = %d", (int)_noise_floor); - #endif + // Sampled unconditionally, including mid-packet. NoiseFloorTracker estimates + // the floor as a window minimum, and signal only ever adds power, so readings + // taken during reception are discarded by construction rather than needing to + // be gated out. Two things go away with the gate: + // + // - the selection bias it imposed, by restricting the sample population to + // moments the modem considered quiet; + // - a dependency on isReceivingPacket(), which is not a pure read. It drives + // a timeout state machine and calls clearIrqFlags(), so polling it at + // 10 Hz from the noise sampler would have this path participating in + // packet detection. Sampling the noise floor must not perturb reception. + // + // The gate was also a liability in its own right: before the IRQ-timeout fix + // in CustomSX1262::isReceiving(), a latched PREAMBLE_DETECTED that never + // completed into a packet held it true until the next startReceive(), + // silently suspending noise sampling for seconds at a time. + _nf.addSample(getCurrentRSSI()); + _noise_floor = _nf.floorDbm(); + + // Off unless asked for: upstream made this line opt-in behind + // MESH_DEBUG_NOISE_FLOOR because it drowns the rest of the debug output. + // When it is on, rate limit it rather than printing on every change -- the + // estimate now updates continuously and jitters by ~1 dB, so "print when it + // changes" would emit several lines a second. One line every + // NOISE_LOG_EVERY_N_SAMPLES keeps it at roughly the volume the 2 s batch + // estimator produced. +#ifdef MESH_DEBUG_NOISE_FLOOR + if (++_noise_log_ctr >= NOISE_LOG_EVERY_N_SAMPLES) { + _noise_log_ctr = 0; + MESH_DEBUG_PRINTLN("RadioLibWrapper: noise_floor = %d (sigma %d/10)", + (int)_noise_floor, (int)(_nf.sigma() * 10.0f)); } +#endif } void RadioLibWrapper::startRecv() { @@ -196,8 +262,23 @@ int16_t RadioLibWrapper::performChannelScan() { } bool RadioLibWrapper::isChannelActive() { - // int.thresh: RSSI-based interference detection (relative to noise floor) - if (_threshold != 0 && getCurrentRSSI() > _noise_floor + _threshold) return true; + // int.thresh: RSSI-based interference detection (relative to noise floor). + // Skipped when the check is disabled (_threshold == 0), and while the + // estimator has no floor yet -- neither is a reason to skip the CAD check + // below, so these are a guard rather than an early return. + if (_threshold != 0 && _nf.ready()) { + // The operator's configured dB margin still means dB, so existing + // interference_threshold settings behave as before. What is new is the floor + // under it: a margin narrower than NOISE_THRESHOLD_SIGMA_K sigma would fire on + // noise alone, and a channel that reads busy continuously does not protect + // anything -- it just delays every packet until getCADFailMaxDuration() + // expires and the node transmits regardless. + float margin = (float) _threshold; + float min_margin = NOISE_THRESHOLD_SIGMA_K * _nf.sigma(); + if (margin < min_margin) margin = min_margin; + + if (getCurrentRSSI() > (float)_noise_floor + margin) return true; + } // cad: hardware channel activity detection if (_cad_enabled) { diff --git a/src/helpers/radiolib/RadioLibWrappers.h b/src/helpers/radiolib/RadioLibWrappers.h index 77dd93116b..7300014e00 100644 --- a/src/helpers/radiolib/RadioLibWrappers.h +++ b/src/helpers/radiolib/RadioLibWrappers.h @@ -2,6 +2,7 @@ #include #include +#include #ifdef USE_CC310_HW_CRYPTO #include @@ -18,8 +19,11 @@ class RadioLibWrapper : public mesh::Radio { uint32_t n_recv, n_sent, n_recv_errors; int16_t _noise_floor, _threshold; bool _cad_enabled; - uint16_t _num_floor_samples; - int32_t _floor_sample_sum; + // _num_floor_samples/_floor_sample_sum (the upstream batch estimator) are gone: + // NoiseFloorTracker replaces them with a continuous, fixed-rate estimate. + NoiseFloorTracker _nf; + uint32_t _next_noise_sample; // millis() deadline for the next RSSI read + uint8_t _noise_log_ctr; // rate limiter for the noise-floor debug line uint8_t _preamble_sf; void idle(); @@ -29,7 +33,9 @@ class RadioLibWrapper : public mesh::Radio { virtual void doResetAGC(); public: - RadioLibWrapper(PhysicalLayer& radio, mesh::MainBoard& board) : _radio(&radio), _board(&board), _preamble_sf(0) { n_recv = n_sent = 0; } + RadioLibWrapper(PhysicalLayer& radio, mesh::MainBoard& board) + : _radio(&radio), _board(&board), _next_noise_sample(0), _noise_log_ctr(0), _preamble_sf(0) + { n_recv = n_sent = 0; } void begin() override; virtual void powerOff() { _radio->sleep(); } @@ -54,7 +60,42 @@ class RadioLibWrapper : public mesh::Radio { virtual float getCurrentRSSI() =0; virtual uint8_t getSpreadingFactor() const { return LORA_SF; } static uint16_t preambleLengthForSF(uint8_t sf) { return sf <= 8 ? 32 : 16; } - void updatePreamble(uint8_t sf) { _preamble_sf = sf; _radio->setPreambleLength(preambleLengthForSF(sf)); } + // Discard the noise-floor estimate because the receiver it characterises has + // changed. Bandwidth is the big one -- the thermal floor moves ~6 dB going + // from 62.5 to 250 kHz -- but frequency and LNA gain move it too. + // + // Without this the estimate can only *rise* as contaminated sub-windows age + // out of the ring, so a floor that has genuinely jumped up takes the full + // NOISE_TRACKER_SUB_WINDOWS * NOISE_TRACKER_SUB_SAMPLES window (180 s at the + // default sampling rate) to be reported. That asymmetry is right for a floor + // that drifts and wrong for one the operator has just moved: with + // interference_threshold set, isChannelActive() would read busy against a + // stale floor and defer every transmit to getCADFailMaxDuration() for three + // minutes. The batch estimator this replaced re-converged in ~2 s, so the + // reset is what keeps a runtime `set bw` as cheap as it used to be. + // + // _noise_floor goes with it: getNoiseFloor() reports the cached value, and + // leaving it behind would keep serving the old floor to isChannelActive() and + // to telemetry until the first sub-window of the new configuration closes. + // + // The cost, which is the whole reason this is a judgement call rather than an + // obvious win: for the ~3 s until the first sub-window closes, _nf.ready() is + // false, so isChannelActive() skips the interference-threshold branch + // entirely and getNoiseFloor() reports 0. The node transmits over the top of + // anything that check would have caught, and telemetry shows an uncalibrated + // floor. That is accepted deliberately -- 3 s of no RSSI check beats 180 s of + // a wrong one, and the CAD check is unaffected throughout -- but a caller + // adding a new reset site should know it is spending that, not nothing. + void resetNoiseFloor() { _nf.reset(); _noise_floor = 0; _noise_log_ctr = 0; } + + // Called by every setParams() override, which is why the reset above lives + // here: it is the one hook every radio family already routes a parameter + // change through. + void updatePreamble(uint8_t sf) { + _preamble_sf = sf; + _radio->setPreambleLength(preambleLengthForSF(sf)); + resetNoiseFloor(); + } PacketMillis calcMaxPacketMillis(uint8_t sf, float bw, uint8_t cr, uint8_t preambleSymbols); virtual int16_t performChannelScan(); diff --git a/test/test_noise_floor_tracker/test_noise_floor_tracker.cpp b/test/test_noise_floor_tracker/test_noise_floor_tracker.cpp new file mode 100644 index 0000000000..04b7e0ae58 --- /dev/null +++ b/test/test_noise_floor_tracker/test_noise_floor_tracker.cpp @@ -0,0 +1,513 @@ +#include + +#include +#include + +#include "helpers/NoiseFloorTracker.h" + +namespace { + +// Deterministic Gaussian source. A fixed LCG plus Box-Muller, so every run of +// these tests sees the identical sample stream -- a statistical estimator tested +// against a seeded-at-random stream produces flaky assertions. +class Rng { + uint32_t _s; +public: + explicit Rng(uint32_t seed) : _s(seed) { } + + float uniform() { // in (0,1), never exactly 0 (logf would blow up) + _s = _s * 1664525u + 1013904223u; + return (float)((_s >> 8) + 1u) / (float)((1u << 24) + 2u); + } + + float normal() { + float u1 = uniform(); + float u2 = uniform(); + return sqrtf(-2.0f * logf(u1)) * cosf(6.28318531f * u2); + } +}; + +const float NOISE_MEAN = -110.0f; +const float NOISE_SIGMA = 2.0f; + +// The spread a receiver actually shows: 0.6-0.9 dB measured on a live SX1262 at +// 250 kHz. NOISE_SIGMA above is deliberately wider so the bias correction gets +// exercised, but the traffic tests want the real number -- the distance from a +// true 0.8 dB up to the 3 dB MAX_SIGMA cap is the whole span the censoring gate +// used to be able to travel, and starting at 2.0 hides most of it. +const float MEASURED_SIGMA = 0.8f; + +// Samples needed to fill the ring completely: the short first sub-window plus a +// full-length one for every remaining slot. +const int FULL_WINDOW = NOISE_TRACKER_FIRST_SUB_SAMPLES + + (NOISE_TRACKER_SUB_WINDOWS - 1) * NOISE_TRACKER_SUB_SAMPLES; + +void feedNoise(NoiseFloorTracker& nf, Rng& rng, int n, + float mean = NOISE_MEAN, float sd = NOISE_SIGMA) { + for (int i = 0; i < n; i++) nf.addSample(mean + sd * rng.normal()); +} + +// A duty-cycled interferer: `cycles` repetitions of `on` samples at +// mean + delta (or, when delta_hi > delta, at a level drawn per burst from +// [delta, delta_hi]) followed by `off` samples of plain noise. Noise of the +// same spread rides on both, since a packet does not replace the noise. +// +// Bursty rather than i.i.d. on purpose: it is the quiet gaps that keep +// windowValue() on the true floor, so this is the shape of traffic that can +// inflate the scale estimate while leaving the floor itself correct. +// Returns the widest sigma seen at any point, not the final one: the failure +// this guards against is transient by nature -- the estimate climbs while the +// traffic runs and relaxes once it stops -- so sampling only at the end would +// miss exactly the excursion that matters. +float feedBurstyTraffic(NoiseFloorTracker& nf, Rng& rng, int cycles, int on, int off, + float delta, float delta_hi = -1.0f, float sd = NOISE_SIGMA) { + float worst = nf.sigma(); + for (int c = 0; c < cycles; c++) { + float lvl = delta_hi > delta ? delta + (delta_hi - delta) * rng.uniform() : delta; + for (int i = 0; i < on; i++) { + nf.addSample(NOISE_MEAN + lvl + sd * rng.normal()); + if (nf.sigma() > worst) worst = nf.sigma(); + } + for (int i = 0; i < off; i++) { + nf.addSample(NOISE_MEAN + sd * rng.normal()); + if (nf.sigma() > worst) worst = nf.sigma(); + } + } + return worst; +} + +// --------------------------------------------------------------------------- +// Basic contract +// --------------------------------------------------------------------------- + +TEST(NoiseFloorTracker, NotReadyUntilFirstSubWindowCloses) { + NoiseFloorTracker nf; + Rng rng(12345); + EXPECT_FALSE(nf.ready()); + EXPECT_EQ(0, nf.floorDbm()); // "not calibrated yet" + + feedNoise(nf, rng, NOISE_TRACKER_FIRST_SUB_SAMPLES - 1); + EXPECT_FALSE(nf.ready()) << "must not report a floor from a partial sub-window"; + EXPECT_EQ(0, nf.floorDbm()); + + nf.addSample(NOISE_MEAN); + EXPECT_TRUE(nf.ready()); + EXPECT_NE(0, nf.floorDbm()); +} + +TEST(NoiseFloorTracker, ConvergesToNoiseMean) { + NoiseFloorTracker nf; + Rng rng(12345); + feedNoise(nf, rng, 5000); + + // floorDbm() reports the estimated *mean* of the noise distribution, so the + // window minimum must be bias-corrected back up by BIAS_K * sigma. + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f); +} + +TEST(NoiseFloorTracker, BiasCorrectionHoldsAcrossScales) { + // The bias is proportional to sigma, so one coefficient must work for every + // plausible noise spread. If BIAS_K were tuned to a single sigma this fails. + // + // "Plausible" is bounded above by the censoring gate: the gate is a fixed + // width above the window statistic, so once the noise tail itself reaches + // past it -- somewhere above 2 dB, against the 0.6-0.9 dB these receivers + // measure -- part of the distribution is censored and the estimate reads low. + // That is the documented trade for a gate signal cannot widen + // (NOISE_TRACKER_SCALE_GATE_DB), and the next test pins the direction. + const float sigmas[] = { 0.5f, 1.0f, 2.0f }; + for (int i = 0; i < 3; i++) { + NoiseFloorTracker nf; + Rng rng(900 + i); + feedNoise(nf, rng, 6000, NOISE_MEAN, sigmas[i]); + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f + sigmas[i] * 0.5f) + << "bias correction wrong at sigma = " << sigmas[i]; + } +} + +// Beyond the gate's range the estimate must degrade in the safe direction. A +// floor that reads low fires the busy-channel check early and CSMA backoff +// absorbs it; a floor that reads high loses the detection outright, and that +// packet is not coming back. +TEST(NoiseFloorTracker, ImplausiblyWideNoiseReadsLowNotHigh) { + const float sigmas[] = { 2.5f, 3.0f }; + for (int i = 0; i < 2; i++) { + NoiseFloorTracker nf; + Rng rng(910 + i); + feedNoise(nf, rng, 6000, NOISE_MEAN, sigmas[i]); + EXPECT_LE((float)nf.floorDbm(), NOISE_MEAN + 1.0f) + << "floor read high at sigma = " << sigmas[i]; + EXPECT_GE((float)nf.floorDbm(), NOISE_MEAN - 2.0f * sigmas[i]) + << "floor read uselessly low at sigma = " << sigmas[i]; + } +} + +TEST(NoiseFloorTracker, EstimatesScale) { + NoiseFloorTracker nf; + Rng rng(999); + feedNoise(nf, rng, 5000); + EXPECT_NEAR(NOISE_SIGMA, nf.sigma(), 0.8f); +} + +// Warm-up has a direction, and only one of the two is affordable. A floor that +// reads high raises the busy-channel comparison with it, and the detection lost +// that way is lost for good; a floor that reads low only fires the check early, +// which CSMA backoff absorbs. The estimator is asymmetric by design for exactly +// this reason, and the asymmetry has to hold at every ring occupancy, not just +// at the full window -- a node that has just changed bandwidth walks through +// all twelve of them. +TEST(NoiseFloorTracker, WarmUpNeverReadsHighAtAnyRingOccupancy) { + for (uint32_t seed = 0; seed < 8; seed++) { + NoiseFloorTracker nf; + Rng rng(31000 + seed * 977); + for (int w = 1; w <= NOISE_TRACKER_SUB_WINDOWS; w++) { + feedNoise(nf, rng, w == 1 ? NOISE_TRACKER_FIRST_SUB_SAMPLES + : NOISE_TRACKER_SUB_SAMPLES); + ASSERT_TRUE(nf.ready()); + EXPECT_LE((float)nf.floorDbm(), NOISE_MEAN + 1.0f) + << "floor read high with " << w << " sub-window(s) in the ring (seed " + << seed << ")"; + // The other side of the same claim: low is safe, but not arbitrarily so. + EXPECT_GE((float)nf.floorDbm(), NOISE_MEAN - 5.0f) + << "floor read uselessly low with " << w << " sub-window(s) in the ring"; + } + } +} + +TEST(NoiseFloorTracker, ConvergesWithinAFewSecondsOfSamples) { + NoiseFloorTracker nf; + Rng rng(4242); + // The short first sub-window exists so a fresh node is usable quickly: + // 30 samples == 3 s at a 100 ms sampling interval. + feedNoise(nf, rng, NOISE_TRACKER_FIRST_SUB_SAMPLES); + ASSERT_TRUE(nf.ready()); + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 3.0f); +} + +TEST(NoiseFloorTracker, ClampsAtMinDbm) { + NoiseFloorTracker nf; + for (int i = 0; i < 500; i++) nf.addSample(-125.0f); + EXPECT_EQ((int16_t)NOISE_TRACKER_MIN_DBM, nf.floorDbm()); +} + +TEST(NoiseFloorTracker, ResetDiscardsEverything) { + NoiseFloorTracker nf; + Rng rng(60606); + feedNoise(nf, rng, 5000); + ASSERT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f); + + nf.reset(); + EXPECT_FALSE(nf.ready()); + EXPECT_EQ(0, nf.floorDbm()); + + // Re-warms at the new level rather than crawling there from the old one. + feedNoise(nf, rng, NOISE_TRACKER_FIRST_SUB_SAMPLES, -80.0f, NOISE_SIGMA); + EXPECT_NEAR(-80.0f, (float)nf.floorDbm(), 3.0f); +} + +// --------------------------------------------------------------------------- +// Robustness to signal. These are the tests the previous suite was too weak to +// fail, and each one corresponds to something observed on live hardware. +// --------------------------------------------------------------------------- + +TEST(NoiseFloorTracker, SingleStrongBurstDoesNotMoveEstimateAtAll) { + NoiseFloorTracker nf; + Rng rng(777); + feedNoise(nf, rng, 5000); + int16_t before = nf.floorDbm(); + + nf.addSample(-50.0f); // 60 dB above the floor + + // A minimum is not merely *resistant* to a high outlier, it is indifferent to + // it. The previous quantile estimator allowed 1 dB of movement here. + EXPECT_EQ(before, nf.floorDbm()); +} + +TEST(NoiseFloorTracker, SurvivesHeavyDispersedInterference) { + NoiseFloorTracker nf; + Rng rng(2026); + feedNoise(nf, rng, 5000); + + // 20% of samples are a loud interferer. The arithmetic mean of this mixture + // is 0.8*(-110) + 0.2*(-60) = -100 dBm, a 10 dB error. + for (int i = 0; i < 6000; i++) { + if (i % 5 == 0) nf.addSample(-60.0f); + else nf.addSample(NOISE_MEAN + NOISE_SIGMA * rng.normal()); + } + + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.5f); +} + +// Regression: dispersed contamination and *runs* of contamination are different +// problems, and only the second one broke on hardware. A quantile tracker has +// essentially zero breakdown point against a run -- it just gets dragged along +// at its maximum slew rate. A minimum ignores the run entirely, as long as the +// window still holds one clean sub-window. +TEST(NoiseFloorTracker, SurvivesRunsOfInterference) { + NoiseFloorTracker nf; + Rng rng(8080); + feedNoise(nf, rng, 5000); + ASSERT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f); + + int16_t worst = nf.floorDbm(); + for (int burst = 0; burst < 30; burst++) { + for (int i = 0; i < 25; i++) { // ~2.5 s of packet at 100 ms/sample + nf.addSample(-70.0f); + if (nf.floorDbm() > worst) worst = nf.floorDbm(); + } + for (int i = 0; i < 25; i++) { // quiet gap between packets + nf.addSample(NOISE_MEAN + NOISE_SIGMA * rng.normal()); + if (nf.floorDbm() > worst) worst = nf.floorDbm(); + } + } + + EXPECT_LT((float)worst, NOISE_MEAN + 1.5f) + << "floor moved to " << worst << " dBm during runs of -70 dBm samples"; +} + +// The specific failure measured on a live repeater: excursions with a median +// duration of 30 s and a maximum of 67 s, during which the previous estimator +// crawled upward at exactly its own slew limit and reported a meaningless +// intermediate value. The window is sized so that an excursion of this length +// cannot contaminate every sub-window. +TEST(NoiseFloorTracker, SurvivesSustainedInterferenceShorterThanTheWindow) { + NoiseFloorTracker nf; + Rng rng(31337); + feedNoise(nf, rng, 5000); + ASSERT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f); + + int16_t worst = nf.floorDbm(); + for (int i = 0; i < 670; i++) { // 67 s at 100 ms/sample + nf.addSample(-70.0f); + if (nf.floorDbm() > worst) worst = nf.floorDbm(); + } + + EXPECT_LT((float)worst, NOISE_MEAN + 1.5f) + << "a 67 s interferer moved the floor to " << worst << " dBm"; + + // ...and it is still correct once the interferer stops. + feedNoise(nf, rng, 200); + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f); +} + +// The other side of the same boundary: interference that outlasts the entire +// window IS the floor, and must be reported as such. Robustness must not mean +// blindness -- a node parked next to a permanent emitter needs to know. +TEST(NoiseFloorTracker, TracksInterferenceThatOutlastsTheWindow) { + NoiseFloorTracker nf; + Rng rng(5150); + feedNoise(nf, rng, 5000); + ASSERT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f); + + // Every sub-window in the ring must be replaced before the estimate can rise, + // and the scale estimate then has to re-converge on the new population, so + // this deliberately takes longer than one window. + for (int i = 0; i < FULL_WINDOW + 1500; i++) { + nf.addSample(-70.0f + 0.5f * rng.normal()); + } + EXPECT_NEAR(-70.0f, (float)nf.floorDbm(), 2.0f); +} + +TEST(NoiseFloorTracker, TracksARealFloorChange) { + NoiseFloorTracker nf; + Rng rng(5150); + feedNoise(nf, rng, 5000); + ASSERT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f); + + // Ambient genuinely rises by 15 dB and stays there. + feedNoise(nf, rng, FULL_WINDOW + 500, -95.0f, NOISE_SIGMA); + EXPECT_NEAR(-95.0f, (float)nf.floorDbm(), 1.5f); +} + +// The estimator's asymmetry is deliberate: it must rise slowly (a high reading +// might be signal) but fall fast (a low reading can only be noise). Rising takes +// a full window as contaminated sub-windows age out; falling takes one +// sub-window, a 12x difference. +TEST(NoiseFloorTracker, FallsFastWhenTheFloorGenuinelyDrops) { + NoiseFloorTracker nf; + Rng rng(2468); + feedNoise(nf, rng, 5000, -95.0f, NOISE_SIGMA); + ASSERT_NEAR(-95.0f, (float)nf.floorDbm(), 1.5f); + + // 30 samples == 3 s. Most of the 15 dB drop must already be reflected, which + // is why the in-progress sub-window counts toward the window statistic rather + // than only being read once it closes. It is not yet exact: the estimate is + // drawn from few samples of the new level, so the bias correction -- sized for + // a full window -- still over-corrects slightly. + feedNoise(nf, rng, 30, NOISE_MEAN, NOISE_SIGMA); + EXPECT_LT((float)nf.floorDbm(), -105.0f); + + // One full sub-window in, it is converged. + feedNoise(nf, rng, NOISE_TRACKER_SUB_SAMPLES, NOISE_MEAN, NOISE_SIGMA); + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 2.0f); +} + +// --------------------------------------------------------------------------- +// Weaknesses specific to a minimum-based estimator. These bound them rather +// than pretend they are absent. +// --------------------------------------------------------------------------- + +TEST(NoiseFloorTracker, ImplausiblyLowReadingIsDiscarded) { + NoiseFloorTracker nf; + Rng rng(1357); + feedNoise(nf, rng, 5000); + int16_t before = nf.floorDbm(); + + nf.addSample(-200.0f); // a bad SPI read, not a quiet channel + + // Without the input guard this would pin the window minimum for a full 180 s, + // and clamping the output at MIN_DBM would disguise it as a plausible floor. + EXPECT_EQ(before, nf.floorDbm()); +} + +TEST(NoiseFloorTracker, WorstCaseValidLowReadingHasNoEffect) { + NoiseFloorTracker nf; + Rng rng(2469); + feedNoise(nf, rng, 5000); + int16_t before = nf.floorDbm(); + + // The lowest reading the input guard still accepts, 25 dB below the floor. + // Tracking the *second* smallest of each sub-window rather than the smallest + // makes an isolated low sample irrelevant: it becomes the first smallest and + // is never the value stored. This is the whole reason for the order + // statistic -- a plain minimum would have taken the full 25 dB and held it + // for a 180 s window. + nf.addSample(NOISE_TRACKER_MIN_VALID_DBM); + EXPECT_EQ(before, nf.floorDbm()); + + // Two in the same sub-window can move it, but it still ages out. + nf.addSample(NOISE_TRACKER_MIN_VALID_DBM); + feedNoise(nf, rng, FULL_WINDOW + 400); + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 1.0f); +} + +// --------------------------------------------------------------------------- +// Scale estimate +// --------------------------------------------------------------------------- + +TEST(NoiseFloorTracker, SigmaHoldsAtFloorBeforeAnyData) { + NoiseFloorTracker nf; + EXPECT_FLOAT_EQ(NOISE_TRACKER_MIN_SIGMA_DB, nf.sigma()); +} + +// Regression: sigma used to be taken from the gap between two quantile +// trackers, which under a sustained run measured how far the two had diverged +// rather than the noise spread. Unbounded it reached 31.6 dB in test and drove +// a live repeater's reported floor from -114 dBm to -70. It is now measured +// only from samples near the floor, so signal cannot enter it at all. +TEST(NoiseFloorTracker, SigmaIsUnaffectedByASustainedInterferer) { + NoiseFloorTracker nf; + Rng rng(1234); + feedNoise(nf, rng, 5000); + float quiet_sigma = nf.sigma(); + ASSERT_NEAR(NOISE_SIGMA, quiet_sigma, 0.8f); + + float worst = quiet_sigma; + for (int i = 0; i < 1500; i++) { + nf.addSample(-70.0f); + if (nf.sigma() > worst) worst = nf.sigma(); + } + EXPECT_NEAR(quiet_sigma, worst, 0.3f) + << "interference widened the estimated noise spread to " << worst << " dB"; +} + +// Regression: the censoring gate used to be placed at +// windowValue() + biasK*sigma + max(6 dB, 4*sigma). Both of those terms grow +// with sigma, so admitting a weak packet widened the gate that had admitted it +// -- positive feedback, bounded only by MAX_SIGMA. Under sustained traffic a +// few dB above the floor it ratcheted all the way there, which silently takes +// the CSMA margin RadioLibWrapper derives from sigma (NOISE_THRESHOLD_SIGMA_K * +// sigma) from ~2.8 dB to 10.5 dB, overriding the operator's +// interference_threshold in exactly the busy mesh where they set it. +// +// The gate is a fixed width above the window statistic now, so an admitted +// sample cannot widen the set of samples admitted next. +TEST(NoiseFloorTracker, SigmaIsBoundedUnderSustainedNearFloorTraffic) { + NoiseFloorTracker nf; + Rng rng(20260802); + feedNoise(nf, rng, 5000, NOISE_MEAN, MEASURED_SIGMA); + ASSERT_NEAR(MEASURED_SIGMA, nf.sigma(), 0.3f); + + // 80% occupancy, every packet 8 dB above the floor: 2 s of packet, 0.5 s of + // gap, for ~85 minutes at a 100 ms sampling interval. This is the exact shape + // that used to pin sigma at MAX_SIGMA. + float worst = feedBurstyTraffic(nf, rng, 2000, 20, 5, 8.0f, -1.0f, MEASURED_SIGMA); + EXPECT_LT(worst, 2.2f) + << "sustained traffic 8 dB above the floor widened sigma to " << worst << " dB" + << " (CSMA margin " << 3.5f * worst << " dB)"; + + // ...and the floor itself is still the floor, which is what makes the sigma + // reading above wrong rather than merely large. Not exact: at this occupancy + // each sub-window holds far fewer clean samples than the bias table assumes, + // so the window statistic sits a little high. A couple of dB of that is + // inherent and is not what this test is about -- following the traffic to + // -102 would be. + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 3.0f); +} + +// The worst point on the curve, not the best one. The test above picks the +// packet level where the fix helps most; this one picks where it helps least. +// +// The gate sits about 6.2 dB above the mean at this spread, so traffic parked +// just under it is admitted wholesale -- no feedback needed, and none of the +// fix's benefit available. Sweeping level at 80% occupancy, worst sigma runs +// 1.45 / 1.74 / 2.04 / 2.27 / 2.35 / 2.24 / 1.51 at +2..+8 dB, against +// 1.45 / 1.74 / 2.04 / 2.33 / 2.63 / 2.92 / 3.00 for the pre-fix gate: nothing +// gained below +5, everything above +6. This pins the peak of that curve so the +// suite describes the whole bound and not just its good end. +// +// Read it as a ceiling on a known residual rather than as a regression guard. +// The pre-fix gate scores 2.63 here against this gate's 2.35, so it does trip +// the bound -- by 0.03 dB, which is seed noise, not a signal. The two tests +// either side of this one are the ones with real separation (3.00 and 2.83 +// against 1.51 and 1.83); this one exists to record where the curve peaks. +TEST(NoiseFloorTracker, SigmaResidualAtTheWorstPacketLevelIsBounded) { + NoiseFloorTracker nf; + Rng rng(20260802); + feedNoise(nf, rng, 5000, NOISE_MEAN, MEASURED_SIGMA); + ASSERT_NEAR(MEASURED_SIGMA, nf.sigma(), 0.3f); + + // +6 dB at 80% occupancy: the peak. + float worst = feedBurstyTraffic(nf, rng, 2000, 20, 5, 6.0f, -1.0f, MEASURED_SIGMA); + EXPECT_LT(worst, 2.6f) + << "residual grew past the level this gate is known to permit: sigma " + << worst << " dB (CSMA margin " << 3.5f * worst << " dB)"; + + // And the floor has still not followed the traffic, which is what keeps this + // a scale-estimate problem rather than a floor problem. + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 5.0f); +} + +// The same bound under the messier version: a spread of link budgets rather +// than one repeated level, which is what gave the old gate its foothold -- the +// weakest packets got in, widened it, and let the rest follow. +TEST(NoiseFloorTracker, SigmaIsBoundedUnderMixedStrengthTraffic) { + NoiseFloorTracker nf; + Rng rng(4711); + feedNoise(nf, rng, 5000, NOISE_MEAN, MEASURED_SIGMA); + ASSERT_NEAR(MEASURED_SIGMA, nf.sigma(), 0.3f); + + float worst = feedBurstyTraffic(nf, rng, 2000, 20, 20, 3.0f, 10.0f, MEASURED_SIGMA); + EXPECT_LT(worst, 2.4f) + << "traffic 3-10 dB above the floor widened sigma to " << worst << " dB"; + EXPECT_NEAR((float)NOISE_MEAN, (float)nf.floorDbm(), 3.0f); +} + +TEST(NoiseFloorTracker, SigmaStaysWithinPhysicalBounds) { + NoiseFloorTracker nf; + Rng rng(4321); + // Wildly over-dispersed input: sigma must still report something a receiver + // could plausibly have, because the reported floor extrapolates from it. + for (int i = 0; i < 5000; i++) { + nf.addSample(NOISE_MEAN + 40.0f * rng.normal()); + } + EXPECT_LE(nf.sigma(), (float)NOISE_TRACKER_MAX_SIGMA_DB + 0.01f); + EXPECT_GE(nf.sigma(), (float)NOISE_TRACKER_MIN_SIGMA_DB - 0.01f); +} + +} // namespace + +int main(int argc, char **argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +}