Repository navigation
Conversation
f17cb86 to
bc714ae
Compare
There was a problem hiding this comment.
Superseded: original blockers withdrawn
My original REQUEST_CHANGES on bc714aebfae92f6b68f1bb8847f8328e052ea2df overstated the two blocking findings and claimed invalid panel participation. See the corrected, provisional APPROVE review for the reconciled verdict, verified tests, actual panel coverage and remaining human release/strategy decisions. This is a correction of my review, not a change to the code. The original receipt is preserved locally; this older review's historical GitHub state is not its current recommendation.
Corrected code-review verdict: APPROVE (provisional coverage)
This supersedes my earlier REQUEST_CHANGES review on the same code. The late source-review batch prompted a reconciliation. I overstated two findings as blockers. Baseline comparison, a component timing probe, and bounded adversarial reconciliation do not support retaining those blockers. No branch changes were needed for this correction.
Repository: WithAutonomi/ant-node. Reviewed head: bc714aebfae92f6b68f1bb8847f8328e052ea2df. True merge-base: a6e3388e79d0907ef678bcb6ec0fcaa12822e148, not current main.
Findings corrected
- Extra synchronous scan: performance follow-up, not a demonstrated blocking DoS.
src/replication/neighbor_sync.rs:93-100really addsprice_contested_keys()on the async executor. However, the baseline already synchronously scans the full paid-list viaall_keys()on the same sync-request path. A scratch-only probe linked the exact frozen library, preseeded LMDB, and verified both key counts. At 1,000,000 entries the additional scan's median was 12.156 ms with no marks / 71.591 ms with every key marked; the existing scan was 14.940 / 15.510 ms. These are three warm-cache runs per case with the debug-linked library, not release-build, fleet-load, starvation or attack results. The added cost is real and should be reduced or offloaded, but I did not establish the blocking severity claimed earlier. - Audit read-error telemetry: operational follow-up.
src/replication/audit.rs:103-112intentionally treats an unreadable mark as set and skips judgement. The missing local log/metric is real; the fail-safe direction is correct. No paid-list failure-injection experiment was run. Missing observability alone does not justify the blocking correctness claim I made. - First-record-wins is deliberate, not a missing upgrade. ADR-0006:587-597 records the front-running trade-off. Unconditionally marking an existing unmarked key on a later cheap payment would allow audit suppression for gas. Retain the disclosed residual; do not present that upgrade as a safe fix.
Participation, dissent and coverage limits
- Original full-production first pass: GLM 5.3 requested changes; Qwen 3.8 27B approved. DS4's first response truncated; its sole correctness retry was labelled with a malformed 39-character OID, so it is not a valid exact-revision correctness receipt. My original public body also contained that malformed OID, although GitHub attached the review to the correct commit shown above.
- Adversarial reconciliation of the disputed claims: GLM 5.3, Qwen 3.8 27B and DS4 Flash all returned substantive APPROVE responses. GLM withdrew its blocking interpretation after the baseline/measurement comparison. GLM's first reconciliation request timed out and DS4's first reconciliation response truncated; each received one narrowed same-model retry. These are clarifications/retries, not extra votes.
- Qwen's adversarial pass included the full production delta; the successful GLM and DS4 reconciliation retries were focused packets. There are five valid model-role responses: three full-delta responses and two focused reconciliations, not six complete full-scope roles. Full-production first-pass participation is 2/3; whole-PR coverage remains provisional. No 6/6 or full whole-PR consensus is claimed. DS4's missing full correctness coverage is not silently filled by its focused answer.
- The two ordinary delegated source-tracing reviews found no concrete implementation blocker. They are not additional named-panel votes, and neither independently ran the tests.
Verification actually executed at this head
cargo fmt --check: exit 0.cargo test --locked --lib --features test-utils -- --test-threads=4: 1,202 passed, no failures.cargo test --locked --test e2e --features test-utils -- --test-threads=1: 124 passed, 3 ignored, no failures.cargo test --locked --test poc_price_floor_live --features test-utils -- --test-threads=1: 2 passed, no failures. These use real local Anvil settlements, but injected routing/gossip; they are not mixed-version production tests.- The opt-in production-wired
price_floor_abworkload and the author's telemetry/mutation campaign were not independently reproduced. - CI at this exact head has a failed Security Audit for the inherited
rustls-webpkiadvisory;Cargo.lockis unchanged from the merge-base. Other reported checks succeeded, apart from the skippedclaudejob. I did not enumerate branch-protection requirements, so the earlier claim that “all required checks” passed was unsupported.
Human release/strategy decisions remain open
The 30% constant is an empirical policy, not proof that all honest traffic clears the floor. My earlier extrapolation from a different shadow sample was too broad: that sample does not establish admission safety across changed views, lone fresh-node quotes, or aged proofs.
ADR-0006:598-619 explicitly describes old binaries penalising upgraded refusers, including marks that survive upgrade; new binaries' contested filtering does not protect old auditors. The operator override is removed by design. Disclosure in a Proposed ADR is not human acceptance of this release strategy. That risk and the unmeasured one-quote liveness residual need the proper human sign-off. This provisional implementation approval does not authorise acceptance of the ADR, a strategy decision, merge, release or deployment.
dirvine
left a comment
There was a problem hiding this comment.
Corrected code-review verdict: APPROVE (provisional coverage)
This supersedes my earlier REQUEST_CHANGES review on the same code. The late source-review batch prompted a reconciliation. I overstated two findings as blockers. Baseline comparison, a component timing probe, and bounded adversarial reconciliation do not support retaining those blockers. No branch changes were needed for this correction.
Repository: WithAutonomi/ant-node. Reviewed head: bc714aebfae92f6b68f1bb8847f8328e052ea2df. True merge-base: a6e3388e79d0907ef678bcb6ec0fcaa12822e148, not current main.
Findings corrected
- Extra synchronous scan: performance follow-up, not a demonstrated blocking DoS.
src/replication/neighbor_sync.rs:93-100really addsprice_contested_keys()on the async executor. However, the baseline already synchronously scans the full paid-list viaall_keys()on the same sync-request path. A scratch-only probe linked the exact frozen library, preseeded LMDB, and verified both key counts. At 1,000,000 entries the additional scan's median was 12.156 ms with no marks / 71.591 ms with every key marked; the existing scan was 14.940 / 15.510 ms. These are three warm-cache runs per case with the debug-linked library, not release-build, fleet-load, starvation or attack results. The added cost is real and should be reduced or offloaded, but I did not establish the blocking severity claimed earlier. - Audit read-error telemetry: operational follow-up.
src/replication/audit.rs:103-112intentionally treats an unreadable mark as set and skips judgement. The missing local log/metric is real; the fail-safe direction is correct. No paid-list failure-injection experiment was run. Missing observability alone does not justify the blocking correctness claim I made. - First-record-wins is deliberate, not a missing upgrade. ADR-0006:587-597 records the front-running trade-off. Unconditionally marking an existing unmarked key on a later cheap payment would allow audit suppression for gas. Retain the disclosed residual; do not present that upgrade as a safe fix.
Participation, dissent and coverage limits
- Original full-production first pass: GLM 5.3 requested changes; Qwen 3.8 27B approved. DS4's first response truncated; its sole correctness retry was labelled with a malformed 39-character OID, so it is not a valid exact-revision correctness receipt. My original public body also contained that malformed OID, although GitHub attached the review to the correct commit shown above.
- Adversarial reconciliation of the disputed claims: GLM 5.3, Qwen 3.8 27B and DS4 Flash all returned substantive APPROVE responses. GLM withdrew its blocking interpretation after the baseline/measurement comparison. GLM's first reconciliation request timed out and DS4's first reconciliation response truncated; each received one narrowed same-model retry. These are clarifications/retries, not extra votes.
- Qwen's adversarial pass included the full production delta; the successful GLM and DS4 reconciliation retries were focused packets. There are five valid model-role responses: three full-delta responses and two focused reconciliations, not six complete full-scope roles. Full-production first-pass participation is 2/3; whole-PR coverage remains provisional. No 6/6 or full whole-PR consensus is claimed. DS4's missing full correctness coverage is not silently filled by its focused answer.
- The two ordinary delegated source-tracing reviews found no concrete implementation blocker. They are not additional named-panel votes, and neither independently ran the tests.
Verification actually executed at this head
cargo fmt --check: exit 0.cargo test --locked --lib --features test-utils -- --test-threads=4: 1,202 passed, no failures.cargo test --locked --test e2e --features test-utils -- --test-threads=1: 124 passed, 3 ignored, no failures.cargo test --locked --test poc_price_floor_live --features test-utils -- --test-threads=1: 2 passed, no failures. These use real local Anvil settlements, but injected routing/gossip; they are not mixed-version production tests.- The opt-in production-wired
price_floor_abworkload and the author's telemetry/mutation campaign were not independently reproduced. - CI at this exact head has a failed Security Audit for the inherited
rustls-webpkiadvisory;Cargo.lockis unchanged from the merge-base. Other reported checks succeeded, apart from the skippedclaudejob. I did not enumerate branch-protection requirements, so the earlier claim that “all required checks” passed was unsupported.
Human release/strategy decisions remain open
The 30% constant is an empirical policy, not proof that all honest traffic clears the floor. My earlier extrapolation from a different shadow sample was too broad: that sample does not establish admission safety across changed views, lone fresh-node quotes, or aged proofs.
ADR-0006:598-619 explicitly describes old binaries penalising upgraded refusers, including marks that survive upgrade; new binaries' contested filtering does not protect old auditors. The operator override is removed by design. Disclosure in a Proposed ADR is not human acceptance of this release strategy. That risk and the unmeasured one-quote liveness residual need the proper human sign-off. This provisional implementation approval does not authorise acceptance of the ADR, a strategy decision, merge, release or deployment.
297f587 to
9c2d279
Compare
MaidSafe-Hermes
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES — re-review of 9c2d2798c2c78d9af5c756efcc877a55b7a83629
Coverage: 1/1 seat reviewed (sol-1, gpt-6.1-sol, xhigh). This is a focused re-review of the delta since bc714aebfae92f6b68f1bb8847f8328e052ea2df; the previous review did not cover this commit. The seat approved but flagged cache-only marks after failed commits. Source tracing confirms that a later successful unmarked insert can make this a persistent cache/database inconsistency.
P2 — Reconcile cache-only marks after successful unmarked writes
src/replication/paid_list.rs:268-274 adds a key to the in-memory mark set before committing a marked insert, then deliberately retains it if the commit returns an error. If that commit did not persist, a later plain insert(key) sees no LMDB row and successfully writes an empty value, but this branch does not remove the cached mark. retain_unmarked then suppresses the now-unmarked key from replica hints, while is_price_contested and the audit paths read the empty LMDB value and treat it as unmarked. That breaks the first-record-wins/cache-consistency invariant and can persist until restart.
After a successful plain insert commit, remove that key from the cached set while still holding write_order; keep the conservative cached mark when the commit fails. Also reconcile cached-only keys on successful batch removal: remove_batch currently removes from the set only keys LMDB reports as deleted (src/replication/paid_list.rs:628-634). Add regression tests for a cache-only mark followed by a successful plain insert and by a successful batch removal of an absent key.
The hosted Security Audit failure is not attributed to this delta: Cargo.lock is unchanged from the merge-base. This code review does not accept the Proposed ADR or authorise merge, release or deployment.
MaidSafe-Hermes
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES — validation gates remain
Coverage: 1/1 seat reviewed (sol-1, gpt-6.1-sol, xhigh). The focused re-review covers head 0425e1e07f1a2b93228532df0dd1e5fa659f0fc1. The earlier seat response was uncertain; its single narrowed retry approved the code change. Source inspection confirms the reported stale-mark paths are addressed. I found no new code defect in this delta.
This request is for outstanding required validation, not a code finding:
- T3 dev-testnet evidence is still missing. The PR marks this as T3 and says the required dev-testnet run remains outstanding. The successful hosted “WebRTC Direct five-node devnet” check is not that run. Please attach the required dev-testnet result.
- Security Audit is failing on this head. The log reports
RUSTSEC-2026-0382(rustls-webpki 0.103.15, upgrade to at least0.103.16) andRUSTSEC-2026-0383(webpki-root-certs 1.0.9, upgrade to at least1.0.10). Please resolve through the dependency/lockfile update and rerun this check, or provide the repository-approved disposition for this inherited advisory gate.
The targeted local paid-list test did not finish: compilation exceeded the available run window and was terminated before test output. I do not claim it passed. The code opinion is based on the reviewed source and the seat receipt.
0425e1e to
668f573
Compare
The ADR-0006 floor has run in shadow mode since v0.17.1. A week of production telemetry (2026-10-14..20, 699 node instances on 72 hosts) showed that its 65% default would have refused 2.5% of admissions, and 2.1% of chunks at every receiver that saw them. Neighbours gossiping inflated commitments could refuse a chunk everywhere. One accepting node could still replicate a below-floor write through PaidNotify and paid-list repair. And a peer that refused a write on price would be penalised for it by possession checks, responsible audits and prune audits. This makes enforcement safe and turns it on for every node, with nothing to configure. - Cap the reference at the receiver's own price: min(group median, calculate_price(own committed count)). A node never demands more than the tolerance of what it would itself quote, so lying neighbours cannot make it refuse a payment that clears its own price. - Lower the tolerance from 65% to 30%, a deliberate starting point. No measured admission is refused at 30% or at 40%, but the worst clears 30% by 1.78x and 40% by only 1.34x, on a week of 48 upload bursts. A baseline quote is still refused by 91% of receivers (98% at 40%). - Floor paid-list admission too. PaidNotify only announces a client's write, with the proof the client presented, so this reprices no historical receipt. It stops one accepting node from giving a below-floor write the paid-list votes that repair needs. - Judge no peer on a contested write. A payment is contested when the node cannot price it, or its paid price is below 60% of the uncapped group median (twice the floor). The verifier remembers each contested chunk payment it admits, in a memory of its own, and every record path (PUT, fresh offer, PaidNotify) marks the key's paid-list entry from that decision, never from the unchecked proof a cache hit carries; the first paid-list record of a key decides. A marked key goes out in paid hints only, so no repair proof is recorded, no responsible audit samples it and no peer penalises this node as its sole hinter; the paid list keeps its marked keys in memory too, so building hints scans nothing on disk for them. A prune audit still needs its proofs but scores no absent peer; paid-list pruning keeps the marked entry while the record is held; and a mark that cannot be read counts as set, and is logged. A chunk fan-out schedules no possession check when the write is marked (by the verified decision or by the record that won, read after its own is written) or the proof it forwards is contested, and a pointer fan-out none when its proof is contested. Releases before this one only test paid-list presence, so they read a marked key as paid. The 2x margin covers the largest spread measured between receivers' views (1.91x); it is a margin, not a guarantee. - One floor for every node. PriceFloorConfig and PaymentVerifierConfig::price_floor are removed: the tolerance is the constant PRICE_FLOOR_TOLERANCE_PERCENT and the contest threshold a private one. ANT_PRICE_FLOOR_ENFORCE and ANT_PRICE_FLOOR_TOLERANCE_PERCENT are no longer read, and backing the floor out takes a release. - The telemetry line adds target, group_median_price, own_price, paid_quote_age_secs and contested, and keeps enforce=true constant. - A comment on the group sample says why it needs no self-filter: the routing scan includes this node, but the gossip cache never holds it, since ingest accepts only senders in the routing table, so its own commitment joins the sample exactly once. - Repair tests/poc_price_floor_live.rs, which no longer exercised the close-group floor and failed on main. Pin the chunk and pointer possession schedulers, the mark and hints, the prune audit and all three record paths with e2e tests, each with a control. Add an env-gated A/B workload driver, tests/e2e/price_floor_ab.rs. ADR-0006 gains Amendment 2, recording the measurement, these decisions and the residuals. ADR-0008's status row is updated. BREAKING CHANGE: PriceFloorConfig (with from_env, enforce and tolerance_percent), PaymentVerifierConfig::price_floor, PRICE_FLOOR_ENFORCE_ENV and PRICE_FLOOR_TOLERANCE_ENV are removed, and PRICE_FLOOR_TOLERANCE_PERCENT is added. The floor is enforced at 30% on every node. neighbor_sync::build_replica_hints_for_peer and audit::audit_tick_with_repair_proofs take the paid list, and ReplicationEngine::replicate_fresh is only built with test-utils.
…clears The paid list keeps its price-contested keys in memory, so building replica hints reads nothing on disk. A marked insert puts its key in that copy just before its commit and keeps it if the commit fails, since the commit may still have reached the disk. When it had not, the copy held a mark the database lacked, and nothing removed it: a later plain insert of the key wrote an unmarked entry, and a batch removal cleared only the keys the database reported deleted. Replica hints then left out a key that every point read, the audit included, treated as unmarked, until a restart. A plain insert that adds the key now drops any in-memory mark for it once its commit succeeds, still under the write order, and a removal, single or batched, drops the mark of every key it names, held or not. A failed commit still leaves the copy as it was, so the copy stays a superset of the committed marks, and equals them for every key a successful write touched. Tests mark a key in memory only, as such a failed commit leaves it, and check that a plain insert, a batch removal and a single removal of the key each clear it, while a key whose entry carries the mark keeps it.
The mutation run on the price-floor code now covers the in-memory mark reconciliation too: 54 mutations, of which 53 fail at least one unit or e2e test. The one survivor is unchanged: prune-audit scoring reading a mark that cannot be read as unset, which only the e2e prune test reaches, and that test cannot make a paid-list read fail.
ADR-0006's validation list said the contest decision follows the view when a write is recorded, and the unit test pinning it said the same. That holds for the proof a fan-out forwards, chunk or pointer, which is priced against the view when the write is fanned out. The paid-list mark a chunk is recorded with comes from the decision its verification made, which the verifier remembers, not from the view at record time. Both texts now say so. No code changes.
The paid-list helper that holds the write order spelled out tokio::sync::oneshot and tokio::task::JoinHandle inline, and the verifier test for a node with no floor wired built its gossip cache as tokio::sync::RwLock inline. The paid-list tests now import oneshot and JoinHandle, the helper waits for the order with an assertion instead of expect, and the verifier test uses CommitmentCache::default(). No behaviour changes.
668f573 to
16e1fc7
Compare
Brings the branch up to main f37429a. The conflicts came from pressure-gated retention (WithAutonomi#271, ADR-0028), which rewrote the paid list, pruning and paid hints that the price-contested mark hooks into: - paid_list.rs: the mark, its in-memory copy, the write order and the mark reads now sit on main's paid list, with its saved record clocks, its MapFull reporting (a marked insert reports a full store the same way) and its range reads. The paid out-of-range timers and the prune cursor stay removed. Eviction goes through remove_batch, which already drops the in-memory mark of every key it was asked to remove. - pruning.rs: paid pruning by rank is gone, so the rule that kept a price-contested entry through it while its record was held goes too: eviction keeps the entry of every record still stored, marked or not. Prune audits still score no absent peer for a price-contested key. - neighbor_sync.rs: main's paid-hint reads and the replica-hint filter for marked keys are both kept. - tests: the e2e test that a prune audit scores no one for a price-contested key now runs its passes under disk pressure, since without it no prune audit is sent. The price-floor PoC keeps this branch's rewrite, which attaches a commitment source, rather than main's ignore of the old test that had none.
…h its entry Paid-list eviction (ADR-0028) removes entries through remove_batch. The new test marks two far entries, evicts one and keeps the other, and checks that the evicted entry's mark leaves the in-memory copy while the kept entry stays marked on disk and in memory, so no mark is held in memory alone after eviction.
ADR-0028 replaced paid-list pruning by rank with eviction above a threshold, and sends prune audits only under disk pressure. Amendment 2 described the mark against the old pruning, so it now says: - paid-list eviction keeps the entry of every record still held, marked or not, and an evicted entry takes its mark with it; the narrow gap where a record admitted mid-pass loses its mark is restated for eviction; - prune audits, and the e2e test that a marked key's audit scores no one, run under disk pressure; - of the 54 mutations run before the merge, two targeted the paid-pruning rule that is now gone, and two more run on the merged code both fail; - opening the paid list reads every entry once to fill the in-memory marks, which a list at the 15 million entry eviction threshold makes about a third of a second with a warm page cache.
A key marked price-contested goes out as a paid hint only, never as a replica hint (ADR-0006). Since pressure-gated retention (ADR-0028), a node with more than 100,000 paid entries builds paid hints only from keys near itself, while replica hints have no distance limit. So a marked record it retained far from itself reached its close group in neither hint set, where an unmarked one still goes out as a replica hint. Paid-hint candidates now also take every marked key the node stores and still holds a paid entry for, however far it is. The marks and the store's index are in memory, so each far marked key costs one paid-list read. build_paid_hints_for_peer takes the chunk store for this, PaidList gains marked_keys and paid_list gains is_near. ADR-0006 says so, and lists the new unit test.
… are unknown When the price-contested marks cannot be read as the paid list opens, the node hints no key as a replica, since any of them may be marked. Before ADR-0028 its paid hints still read every paid entry, so its stored records reached their close groups that way. Above 100,000 entries paid hints now read only near keys, so in that state a record the node retained far from itself went out in neither hint set. While the marks are unknown, every key the node stores that has a paid entry is now a paid-hint candidate at any distance, as a marked key is when the marks are known. That moves the per-key work the replica hints no longer do onto the paid hints. PaidList::marked_keys says when the marks are unknown, and the unit test covers that state. ADR-0006 says so.
… for now Paid-list eviction (ADR-0028) kept an entry only while ChunkStore::exists held for its record, and exists turns false while a record is out of service after a failed read, although its file stays in the index. An eviction pass in that moment removed the entry and, with it, the record's price-contested mark (ADR-0006). Once the record read again it went out as an unmarked replica hint, so peers that had refused it under the floor could be judged on it. Eviction now keeps an entry while its record is in the store's index, readable or not, through the named keeps_paid_entry. It only keeps more entries than before. A unit test makes a record unreadable and checks that its entry stays while an entry with no record does not. ADR-0006 says so.
…ery mark The far price-contested keys a hint build adds to its paid hints were found by copying every mark and checking each against the store. Marks also sit on paid entries whose records this node never stored, and stay until eviction, so with millions of them every hint build copied and checked them all, however few records the node holds. A hint build now reads the stored keys once, as it already did for replica hints, and splits them against the in-memory marks: the unmarked ones become replica-hint candidates, the marked ones far paid-hint candidates. While the marks are unknown every stored key counts as marked. The inbound sync handler builds both hint sets from that one read. PaidList::split_marked replaces marked_keys.
A paid hint makes no possession claim, so a node holding a marked record is never asked for its bytes. If a key's current close group holds no copy and every copy outside it is marked, the group cannot repair the key from them. The amendment's consequences now say so, and that fixing it without exposing the holder to sole-hinter penalties needs a wire change.
…nt would go A far price-contested key this node stores went out as a paid hint to every member of its paid close group it knows (20 peers), where a replica hint of the same key reaches its close group (7). So turning such keys from replica hints into paid hints could grow a sync message about three times past what main sends, in the worst case past the 10 MiB message limit, and the whole exchange would be dropped. Paid-hint candidates now keep the near keys read from the paid list apart from the far marked keys: the first still go to the paid close group, the second only to the close group, so they add no more to a message than replica hints of the same keys would. build_paid_hints_for_peer takes the close group size for this. ADR-0006 says so.
… the queried groups Verification asks a key's close group and its paid close group, and takes every member that answers present as a fetch source, marked or not. Only copies outside both groups, known to repairers through paid hints alone, are never fetched from. The consequence now says so.
Five targeted mutations were run on the code merged with ADR-0028, one for each check the merge added or ported, and each fails its test. The validation list now names all five instead of the first two.
MaidSafe-Hermes
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES — eviction race and open release gates
Repository: WithAutonomi/ant-node. Reviewed head: 06eea7957f01c973f2c52554651e3eda7a91daab.
Coverage: 2/2 seats reviewed (sol-1, gpt-6.1-sol xhigh; glm-1, z-ai/glm-5.3). GLM approved. Sol-1 remained uncertain after its one narrowed retry and identified the eviction race below. I checked both concerns against source: the alleged mark-set deep copy is not real (price_contested stores an Arc<HashSet<_>>, so the snapshot clones the Arc); the race is real.
P2 — Coordinate paid-entry eviction with record admission
src/replication/paid_list.rs:619-654 scans candidates and calls the keep closure under a read transaction, then releases that scan and calls remove_batch in a separate write transaction. The eviction closure in src/replication/pruning.rs:699-706 checks storage.is_indexed(key) only during the scan.
A concurrent fresh-offer path in src/replication/mod.rs:6623-6635 records the paid entry (including its contested mark) and then stores the record. If eviction sees the key as not indexed, the fresh offer can then commit the marked entry and record before remove_batch runs. That later delete removes the entry and in-memory mark while the record remains stored. Hint construction can then treat the record as unmarked and send a replica hint, exposing peers that refused the contested write to possession/audit penalties.
The ADR discloses this scan-to-delete race, but its rarity argument does not prevent the execution path or establish human acceptance. Coordinate admission with eviction so a record and its mark cannot be orphaned by a stale eviction candidate; add a deterministic interleaving regression test. A second uncoordinated is_indexed check alone would leave another race window.
Required T3 and strategy gates
The required T3 dev-testnet result and human sign-off on the rollout-skew and one-quote residuals are still outstanding. Please attach the testnet evidence and record the authorised sign-off before approval. These are open release/strategy gates, separate from the code finding.
The Security Audit still fails on RUSTSEC-2026-0387, but the current CI Gate succeeds and the base/head Cargo.lock objects match; I am not treating that shared advisory as a blocker for this review. Hosted test checks on this head succeed. I did not run the local test suite.
The ADR-0006 close-group price floor has run in shadow mode since v0.17.1. This turns it on for every node at 30%, with nothing to configure. It also fixes what made enforcement unsafe: the tolerance, griefing by neighbours, a bypass through paid-list repair, and, within a measured margin, trust penalties for refusing a write under the floor.
A week of shadow telemetry (ant-prod-01, 2026-10-14 to 2026-10-20, 699 node instances on 72 hosts, node 0.22.0 and 0.23.0) and a review of the enforcement path found four problems:
PaidNotifyand paid-list repair.What changes:
The reference is capped at the receiver's own price:
min(group median, calculate_price(own committed count)). A node never demands more than the tolerance of what it would itself quote. Neighbours therefore cannot make it refuse a payment that clears its own price, however many of them lie. While gossip is honest, receivers above the median price exactly as before.Tolerance 65% → 30%, a deliberate starting point. No measured admission is refused at 30%, or at 40%, in either production window: 1,898 admissions in October (1,801 of them matched for the capped replay), and 1,103 evaluations in September. The worst October admission clears 30% by 1.78x but 40% by only 1.34x, on a light week of 48 upload bursts. So the floor starts at 30%. A follow-up raises it toward 40%, by a later ADR amendment, once enforced traffic shows that 40% refuses no honest payment. A baseline quote (a near-empty or sybil node) is still refused by 91% of receivers, against 98% at 40%.
Paid-list admission is floored too.
PaidNotifyannounces a client's write, with the proof the client presented. Repair and fetch never call the verifier. So this reprices no historical receipt, and one accepting node can no longer give a below-floor write the paid-list votes that repair needs.No peer is judged on a contested write. When a node's verifier first admits the payment for a write (the PUT, a fresh offer or a
PaidNotify), it prices the proof against its current view; a later admission of the same write from the payment cache reuses that decision. The write is contested when the node cannot price it (no own commitment, or too thin a view), or when its paid price is below 60% of the uncapped group median, twice the floor. An admitted contested write, one the node cannot price or one paid between the floor and that threshold, is still stored and offered, but:PaidNotifyor fresh offer and nobody checked, can neither set nor veto the mark. The first paid-list record of a key decides its mark, and a fan-out never schedules a possession check for a write that is marked, by the verified decision or by the record that won, which it reads after recording its own;A merkle proof, a proof that does not parse, or a node with no floor wired does not by itself make a write contested; a remembered contested admission or an existing mark still turns the checks off for that key. The 2x margin covers the largest spread measured between receivers' views of one chunk (1.91x). Replayed over all 5,536 pairs of measured receivers, at payments from 0.2x to 1x the median in 0.02x steps, no refusal would be judged at 2x, against 4 pairs at 1.5x. The cost is that these checks are given up on 1.4% of the 1,801 measured admissions, and separately on the writes the floor cannot price, 2.6% of the 2,117 floor evaluations. It is a margin, not a guarantee. A guarantee needs the refuser to tell the sender, and a node that can say "refused on price" can also dodge ADR-0003's full-node detection that way, so that is a separate decision. The threshold is its own constant, so a later raise takes two releases: the threshold first, then the tolerance once nearly every node runs it.
Enforced on every node, nothing to configure.
PriceFloorConfigis gone: the tolerance is one public constant,PRICE_FLOOR_TOLERANCE_PERCENT, and the contest threshold a private one,PRICE_FLOOR_CONTESTED_PERCENT(60).ANT_PRICE_FLOOR_ENFORCEandANT_PRICE_FLOOR_TOLERANCE_PERCENTare no longer read, without a warning. The floor is the same everywhere, because a node set stricter than its peers would be penalised by their checks. Backing it out takes a release. A node whose replication engine fails to start runs without one, as before, and its floor skips every evaluation (skip_reason=no_commitment_cache), as doesant-devnet's.Telemetry adds
target,group_median_price,own_price,paid_quote_age_secsandcontestedto the existingant_node::payment::price_floorline.enforceis now alwaystrue, which, liketarget, tells upgraded nodes apart from shadow-mode ones during the rollout. Each line carries what a replay at any tolerance needs, and the follow-up uses that to judge 40%.Tests.
tests/poc_price_floor_live.rshad stopped exercising the close-group floor: its rejection test failed onmain, which now ignores it as stale. It is repaired here, and CI runs it since ci: consolidate PR CI, run every test target, build the shipped binary #269.PaidNotify), each of which must mark a key exactly when its own verified decision was contested, never from the forged proof attached.tests/e2e/price_floor_ab.rs, returns at once unlessPRICE_FLOOR_AB=1, so CI only compiles it.Linear issue
Closes V2-870
Risk tier
Compatibility
PriceFloorConfig(with itsfrom_env,enforceandtolerance_percent),PaymentVerifierConfig::price_floor,PaymentVerifier::price_floor_config, the test-utilsPaymentVerifier::set_price_floor_for_tests,PRICE_FLOOR_ENFORCE_ENVandPRICE_FLOOR_TOLERANCE_ENV; addsPRICE_FLOOR_TOLERANCE_PERCENT, fivePaidListmethods for the mark (insert_price_contested,is_price_contested,counts_as_price_contested,retain_unmarkedandsplit_marked) andpaid_list::is_near.neighbor_sync::build_replica_hints_for_peernow takes the paid list, andneighbor_sync::build_paid_hints_for_peerthe chunk store and the close group size (audit::audit_tick_with_repair_proofstakes the paid list onmainalready, since feat(replication)!: erase out-of-range records only under disk pressure #271), andReplicationEngine::replicate_fresh, which nothing in production calls, is now test-only (test-utils). The twoANT_PRICE_FLOOR_*variables are no longer read. ant-client's test support importsPriceFloorConfigand setsprice_floor: PriceFloorConfig::default(); both go when it moves to this release.Semver impact
Test evidence
Production replay, October. The source is shadow telemetry from ant-prod-01, 2026-10-14 to 2026-10-20: 2,117 floor lines from 699 instances on 72 hosts. Replaying the logged formula reproduces all 53 of the nodes' own
below_floor=truelines (47 distinct admissions). That leaves 1,898 admissions (each a distinct chunk and receiver) over 536 chunks, of which 1,801, over 521 chunks, are matched to the receiver's own committed count. Refused admissions out of those 1,801:The lowest settled-to-reference ratio is 0.534 with the cap (0.473 without). That clears 30% by 1.78x and 40% by 1.34x.
With zero refusals observed, the 95% upper bounds are 0.17% per admission, 0.58% per chunk (521 chunks hold a matched admission) and 6.4% per upload burst (47 bursts hold one).
Share of receivers refusing a 3x payment to a quoter, with the cap:
Every observed receiver refuses a baseline quote on 84.5% of chunks at 30% (95.2% at 40%). Not caught at 30%: a quoter 40% as full as its group is admitted by 72% of receivers and one half as full by all of them, nor a client shopping for the cheapest member of the close group (no lower than about 0.7x the median).
If every neighbour gossiped the protocol maximum, all 1,801 admissions would still clear at 30%. Today's rule would refuse all of them.
Replaying the contest margin over 5,536 ordered pairs of receivers of the same chunk, at payments from 0.2x to 1x the median in 0.02x steps: 0 pairs would see a refusal judged at 2x, against 4 at 1.5x. Moving the tolerance to 40% without first lifting the threshold would leave 3.
Second production window, September. This is an independent window a month earlier, 2026-09-10 to 2026-09-16: 1,134 floor lines from 557 instances on 71 hosts, with 1,103 evaluated over 271 chunks. It was measured under today's median-only rule, which refuses at least everything the capped rule does. At 50% that rule refused nothing. At 55% it refused 0.93% of admissions, and at 65% 3.43%, with 1.48% of chunks refused by every observed receiver. September's worst admission therefore sat at 0.50x or above under the stricter rule. October's sat at 0.473x under that rule and 0.534x capped. Neither window has a refusal at 30% or 40%. Own commitment counts were not joined for that window, and its raw dump was not kept, so these figures are from the analysis done at the time.
Live A/B,
tests/e2e/price_floor_ab.rs. The same workload ran onmain(shadow mode, 2026-10-20) and on this change at 30% (on 375d62f, this PR's code before its rebase ontomain). Themainrun used an earlier revision of the driver, which asserted the shadow outcome and checked trust only on the peers that refused a direct PUT; it was not re-run with this driver. Setup: 24 nodes seeded to median committed counts of 3,949 for this change (range 3,809–5,785) and 4,536 formain(range 3,841–5,818), with real payments on Anvil and a 20–30 s possession delay.main(shadow)PaidNotifyalso refusedTests.
cargo test --lib --bins --features test-utils: 1,276 pass in the library, and 10 and 5 in the two binaries, on 8d12c17, whose code is the head's, andcargo test --lib --no-default-features1,219.possession_check_skips_a_write_the_floor_cannot_price(a PUT through the handler schedules no possession check when the storer's verified decision was contested, when only the proof it forwards is, or when a marked record of the key won first; a marked key gets a paid hint only),a_fresh_pointer_write_the_floor_cannot_price_is_not_used_to_judge_peers,prune_audit_scores_no_peer_for_a_price_contested_key(its passes now run under disk pressure, since feat(replication)!: erase out-of-range records only under disk pressure #271 sends prune audits only then), andevery_record_path_marks_a_key_from_its_own_verified_decision. They settle nothing on chain, so where a test needs a verified contested admission it stands one in; unit tests pin that a real verification records it.cargo test --test poc_price_floor_live --features test-utils: 2/2 against a real Anvil vault. Onmainit is 2 of 2 with the stale test ignored.cargo test --test e2e --features test-utils -- --test-threads=1: 126 passed, 3 ignored (main'slive_testnettests), on 8d12c17.The head, 06eea79, is the previous head 16e1fc7 (five commits on
mainat 02c8682, reviewed before and unchanged), a merge ofmainat f37429a, efa4a57, and ten commits after it. The merge brings in pressure-gated retention (#271, ADR-0028), which rewrote the paid list, pruning and paid hints the mark hooks into: the mark, its in-memory copy and its readers now sit on #271's paid list; the rule that kept a marked entry through paid pruning is gone with that pruning, since eviction keeps the entry of every record still stored; prune audits still score no one for a marked key, and the e2e test of that now runs under disk pressure; andtests/poc_price_floor_live.rskeeps this PR's repair, not main's ignore of the stale test. After it, ce05871 pins that eviction takes an evicted entry's mark, and 813b103 brings ADR-0006 in line with ADR-0028. Reviews of the merged code then found where the two designs meet. Three fixes each come with a test that fails without them: ada05dd makes a stored marked key a paid-hint candidate at any distance, since above 100,000 paid entries #271 reads only near keys for paid hints and such a record would otherwise go out in neither hint set; 7bbe0e1 does the same for every stored key while the marks are unknown; and 6051640 makes eviction keep the entry of any record still in the store's index, since a read fault turnsexistsfalse and could cost a retained record its mark. Two more bound the cost of the first two: 04513a8 finds far marked keys among the stored keys instead of copying every mark on each hint build, and 7b66a6c sends those far paid hints only to the close group a replica hint would reach, so a sync message grows by no more than replica hints of the same keys would make it. Their unit test checks which keys are read, not which peers they go to, since that needs a routing table. e9728f5, 8d12c17 and 06eea79 change only ADR-0006: they record the repair-source limit below and the final mutation count.Gates, on 8d12c17, whose code is the head's (06eea79 changes only ADR-0006's mutation count), with CI's
RUSTFLAGS="-D warnings", on Rust 1.99.0 on macOS: main's consolidated CI apart from its Linux-only steps, the run-directory tests under sudo,storage_scaleand the storage jobs on ext4, xfs and btrfs, which ran in hosted CI only:cargo fmt --all -- --check;cargo doc --all-features --no-depswith-D warnings;cargo check --lib --no-default-features --locked;cargo +1.95.0 check --all-targets --all-features --locked(MSRV);cargo build --release, and the shipped-binary failpoint check on it;cargo test --doc --all-features: 1 passes, 2 ignored;--features test-utils -- --test-threads=1:e2eas above;build_script_git_watch3;chunk_store_crash_safety2 (its 1 ignored test is the crash test's child process);devnet_lifecycle3;poc_audit_handler_live16;poc_bootstrap_stall3;poc_commitment_audit_attacks19;poc_d1_bounded_queues7;poc_price_floor_live2, against a real Anvil vault;pointer_convergence17;webrtc_direct_devnet6, and its ignored five-node devnet test 1;scripts/adr-governance.pyagainstmain, also on the head;cargo clippy --all-targets --all-featureslints every target and reports one warning,unnecessary_wrapsongroup_writersinsrc/devnet/run_dir.rs(main's fix(ant-devnet): stop cleanly on SIGTERM and use a per-run data dir by default #263; on Linux that function can return an error).-D warningsmakes it the one error ofcargo +1.99.0 clippy --all-targets --all-features -- -D warnings, andcargo clippy -- -D warningsalso flags main'sunused_selfonskips_closeness_without_p2pinsrc/payment/verifier.rs.src/devnet/run_dir.rsis byte-identical to main, and that function is identical to main's; this PR changes other parts ofverifier.rs.cargo auditreports one vulnerability, RUSTSEC-2026-0387 (crypto-bigint 0.5.5, reached through alloy's k256 and webrtc's dtls, with no fixed release of either yet), and the same 6 allowed warnings as before.mainat f37429a has the sameCargo.lock, which this PR leaves unchanged. So the hosted Security Audit check fails here on the same vulnerability, as on every PR; since #269 it sits outside the required CI Gate, until those dependencies move or the maintainers decide on an ignore, a decision that is open and not taken here.Hosted checks on 06eea79: 15 of 16 pass, the required CI Gate among them (CI run 41981942807, plus the ADR governance and PR-check runs): Quick checks, Rust 1.95 MSRV, Third-party notices, Test on Ubuntu, macOS and Windows, Storage on ext4, xfs and btrfs, Build release (x86_64-unknown-linux-musl), ADR validation,
linear-link,pr-templateand the PR-check self-test. Security Audit fails on RUSTSEC-2026-0387 alone, with the same six allowed warnings, ascargo auditreports locally; RUSTSEC-2026-0382 and 0383, which the last formal review named, are fixed onmainby #258.The dev-testnet run that T3 asks for has not happened yet.
Residual risks, also recorded in the ADR.
PaidNotifyrecords it. A PUT served from the cache with no proof attached stores the record without any paid-list entry, as before. A paid-list insert that fails before its commit leaves the key out of the paid list, so unmarked. When a marked insert's commit is what failed, the in-memory copy keeps the mark, since the commit may still have reached the disk, so this node does not hint the key as a replica while the mark stays. A later plain insert that adds the key, or a removal of it, drops the mark once that write commits, and a restart rebuilds the copy from the database. The verifier's memory of contested admissions loses a decision before its key is recorded only if 100,000 newer contested admissions arrive in between. A record admitted between paid-list eviction's scan and its delete would lose its entry and so its mark; eviction takes only entries far outside the node's range, where with evenly spread IDs neither store path admits a record.New dependency
none
ADR
https://github.com/grumbach/ant-node/blob/06eea7957f01c973f2c52554651e3eda7a91daab/docs/adr/ADR-0006-receiver-side-revenue-floor.md#amendment-2-2026-10-20-enforced-on-every-node (amends ADR-0006, which is still Proposed, and updates ADR-0008's status row)
Mitigation / rollback
There is no per-node switch: reverting this PR in a release restores shadow mode. Older releases read the paid-list marks as ordinary paid keys, so a rolled-back node judges peers on those keys again, as it did before.