Skip to content

HF14: Prediction Markets - #124

Open
On1x wants to merge 180 commits into
masterfrom
pm
Open

HF14: Prediction Markets#124
On1x wants to merge 180 commits into
masterfrom
pm

Conversation

@On1x

@On1x On1x commented Jul 2, 2026

Copy link
Copy Markdown
Member

Summary

Introduces Hardfork 14 (HF14): Prediction Markets — a full on-chain
prediction-market subsystem for the VIZ blockchain, plus the read-only APIs,
wallet bindings, snapshot support, tests and documentation around it.

The feature is hardfork-gated (CHAIN_HARDFORK_14) and preserves the zero-sum
token invariant: prediction markets never mint or burn tokens, current_supply
is untouched, and settlement conserves value across bets, LP and forfeit pools.

What's included

Consensus / chain

  • Core PM protocol operations, objects and evaluators (markets, oracles,
    bets, liquidity, resolution, disputes, leverage) gated behind HF14.
  • Oracle rebuttals on disputes (pm_dispute_oracle_respond), early ban lift
    (pm_unban) and automatic ban expiry (pm_ban_expired virtual op via the
    per-block cron).
  • On-chain resolution statements on markets (decision_url/decision_reason)
    and dispute oracle responses, readable without history scans.
  • v5 chain properties for PM tuning (witness-median), including insurance
    coverage floors for listing (2.5x) and betting (1.5x, advisory).

Plugin / wallet / snapshot

  • prediction_market_api plugin with extensive read APIs: market/oracle/
    dispute queries, category taxonomy + live counts, klines, one-call enriched
    market view, and non-consensus leverage previews (quote/close/convert) that
    reuse the in-node margin math.
  • Wallet remote_node_api bindings for the PM read APIs.
  • Snapshot import support for all HF14 PM objects.

Docs & tests

  • EN + ru + zh-CN documentation (chain properties, plugin API, specification,
    operations, virtual operations) and workflow/integration design docs.
  • Comprehensive test_pm_lifecycle consensus-sim suite covering the full
    market lifecycle, disputes, leverage, bans and settlement invariants.

Notes

  • Base branch: master. Feature branch: pm (16 commits).

On1x added 24 commits June 16, 2026 19:33
…ront-running protection

- Introduce two optional execution modes per bet: batch (mode 1) and commit-reveal (mode 2)
- Preserve instant per-bet execution (mode 0) as default for best UX
- Define global and per-market parameters for configuration and governance control
- Implement batch epochs with unified uniform-price settlement to prevent intra-batch front-running
- Add commit-reveal operations: bet commitment, reveal, penalty for no-reveal with transfer into winners' pool
- Design batch settlement math preserving liquidity provider invariant and fairness via canonical CPMM aggregation
- Add database schema changes for bets, commitments, and market flags supporting new modes and states
- Extend market logs to track batch-related actions: commit, reveal, batch settlement, and forfeits
- Specify layered defenses against front-running including epoch snapshot pricing and min_tokens enforcement
- Define phased rollout plan: batch mode first, then commit-reveal, followed by tiering and client policies
- Document open questions and future improvements like encrypted sealed bids (mode 3) and cancellation policy
…off strategy

- Add new chain_properties versions 4 (hf13) and 5 (hf14 PM) for governance params
- Introduce 40+ Prediction Market consensus parameters including fees, disputes, batch settings, cron budget, lazy pool, and leverage
- Define market fee structure and governance cap rules focusing on oracle fee cap
- Document versioning, median calculation, and validator publishing rules for new params
- Implement leverage fund as sub-allocation of lazy pool free balance with detailed accounting and protocol state transitions
- Present full mathematical leverage model: CPMM/LMSR calculations, cancel values, liquidation thresholds, safety margin, and leverage constraints
- Describe atomic liquidation mechanism ensuring pool protection before opposing bets execute, including cascade logic and cascade loop handling
- Analyze risk types and mitigation: price-movement risk elimination via atomic liquidation, safety margins, and dynamic caps; outcome risk remains inherent
- Provide detailed architecture overview, fund allocation parameters, frontend UI terminology (Boost vs Leverage), and risk analysis
- Outline protocol operations, API endpoints, database schema, pre-calculation and slider logic for leverage positions
- Address MEV considerations for liquidation rebalancing and sandwich attacks with mitigation plans for VIZ DLT implementation
…ion by role

- Add comprehensive README outlining canonical scenarios for normal and disputed resolves
- Document master ledger calculations for payout distributions and zero-sum properties
- Include role-specific subfolders with interaction diagrams and signed/virtual operations
- Provide detailed token flow tables for normal and disputed market outcomes
- Outline leverage mechanics including open, close, liquidation, and settlement steps
- Describe dispute scenarios: disputer wins, loses, and forced auto-close with penalties
- Verify all operations and virtual operations present in code with references
- Add instructions on how to observe states and events via API plugin methods
- Cover edge cases like time penalties, liquidations, and refund mechanics
- Provide an index of workflow document folders by participant role for easy navigation
- Updated fc submodule pointer from 99b5d133 to 5a9d84a1
- Ensured thirdparty dependencies are current and consistent
- Added mermaid package version 11.4.1
- Added vitepress-plugin-mermaid version 2.0.17 for Mermaid integration
- Added multiple new dependencies related to mermaid and diagram rendering
- Included types packages for d3 and related libraries for better type support
- Updated package-lock.json to reflect new dependencies and their versions
- Removed several optional and deprecated dependencies to clean up lock file
- Increase total number of hardforks from 13 to 14
- Introduce hardfork 14 with features for prediction markets (binary CPMM, multi LMSR)
- Add oracles, dispute mechanisms, batch/commit-reveal, and lazy liquidity pools
- Define placeholder activation times for mainnet and testnet
- Use version 4.0.0 for hardfork 14 release candidate
- Add chain_properties_pm median evaluator and integrate into median calculation
- Implement lazy-pool logic for DAO-committee voting weight in committee_processing
- Register PM evaluators for all PM operations in database initialization
- Add core indexes for PM-related objects for consensus and chain state tracking
- Implement PM liquidity settlement, market resolution, leverage liquidation, and recall mechanics
- Add HF14 hardfork initialization including lazy-liquidity pool singleton creation
- Update CMakeLists.txt to include PM source and header files with proper compiler flags
- Enhance database.cpp with PM processing hooks and vote weight adjustments for lazy pool stake
- Provide detailed internal helpers for PM operation including settle market, refund bets, and liquidity allocation
…ive APIs

- Implement prediction_market_api plugin with full lifecycle management
- Provide APIs to query markets, outcomes, bets, positions, and liquidity
- Support account leverage positions and creator ban status retrieval
- Offer oracle info and list oracles with reliability scoring
- Include dispute info and vote tallying with projected verdict calculation
- Record and prune metadata and kline time-series data on block application
- Add support for lazy pool and deposit queries along with chain properties
- Implement metadata parsing and filtered market listing by category and jurisdiction
- Integrate with chain plugin database and handle post-operation kline recording
- Configure plugin with pmm-ttl-days option for metadata retention days
- Setup CMake build configuration for prediction_market_api plugin library
- Introduce functions to import pm_oracle, pm_market, pm_outcome, and pm_dispute objects with
  shared_string member handling
- Add export and import logic for all HF14 PM related indices in snapshot processing
- Clear existing PM objects before importing new ones during database initialization
- Enhance snapshot deserialization to handle absent PM objects in pre-HF14 snapshots
- Log import counts for each PM object type during snapshot loading to aid diagnostics
- Include prediction_market_api plugin in wallet build dependencies
- Add remote_prediction_market_api binding with optional connection handling
- Implement pm_api() accessor for prediction_market_api proxy with assertion
- Introduce prediction market helper methods for oracle registration, update, market creation,
  bet placement, commitment hashing, bet commit/reveal/cancel, liquidity management, market resolution,
  dispute creation/voting/resolution, position transfer, lazy deposit/withdraw, and leverage operations
- Add read API passthrough methods for markets, oracles, bets, positions, liquidity, disputes, lazy pool,
  chain properties, market metadata, and market kline data
- Extend fc::api remote_node_api.hpp with prediction market API message signature class and FC_API definition
- Update wallet.hpp and wallet.cpp with full prediction market API support and method declarations
- Ensure wallet starts normally even if prediction_market_api plugin is unavailable on connected node
- Include prediction_market_api in CMakeLists.txt for vizd program
- Add prediction_market_api header inclusion in main.cpp
- Register prediction_market_api plugin in appbase application initialization code
…ests

- Add tests covering oracle registration after HF14 activation
- Implement full binary market lifecycle: creation, betting, resolution, payout
- Test committee dispute scenario with outcome overturning via voting
- Verify lazy-pool stake contributes to dispute voting weight and quorum
- Ensure external oracle rejection refunds seed liquidity exactly once
- Add bet cancellation scenario reversing CPMM reserves and bets sum
- Adjust consensus_sim harness to support new pm tests and sanitizer flags conditionally
- Expose direct database access in simulated_node for test assertions
- Fix simulated_node block witness field to validator for accuracy in tests
…operties support

- Introduce HF13 distribution epoch length and HF14 prediction markets features in governance docs
- Add detailed median-voted HF14 prediction-market parameters and kill-switch flags explanation
- Add new `prediction_market_api` plugin with extensive JSON-RPC read-only methods for markets, bets,
  oracles, disputes, lazy pool, and governance data access
- Include computed DTOs and charting support for prediction markets with offset-from-newest pagination
- Document prediction market concepts analysis comparing Onix protocol to theoretical models
- Update advanced hardfork and chain properties docs to cover new prediction market functionality
- Introduce readonly JSON-RPC plugin for HF14 prediction markets state access
- Document market-related API methods including markets, outcomes, bets, liquidity, and metadata
- Describe position, leverage, oracle, dispute, lazy pool, and governance methods
- Provide details on kline/time series for market weight history and pagination approach
- Explain computed DTOs representing bets, oracles, votes, and payout structures
- Include example usage and code snippets for API calls and data processing
- Link to relevant protocol operations and chain property documentation

docs(governance): update chain properties with HF13 and PM parameters

- Add chain_properties_hf13 with distribution_epoch_length parameter
- Introduce chain_properties_pm (v5) for ~30 prediction market parameters and kill-switch flags
- Detail all median-voted parameters for oracle, market, batch, dispute, time penalty, lazy pool, leverage, and fairness
- Clarify live kill-switch flags to disable commit-reveal, lazy pool, or leverage without hardfork

docs(advanced): extend hardfork management with HF13 and prediction markets

- Add entries for HF13 epoch length and HF14 prediction markets including CPMM/LMSR, oracles, disputes, commit-reveal, lazy-pool, and chain properties v5

docs(prediction-markets): add comprehensive analysis of conceptual mapping of Onix PM protocol

- Provide detailed table comparing 90 theoretical prediction market concepts against VIZ Onix on-chain implementation
- Categorize concepts as solved, inherent, not needed, partial/roadmap, client layer, or open risks
- Discuss information theory, mechanism design, liquidity and trading aspects in depth
- Highlight Onix innovations: risk-free LP, CPMM binary, LMSR multi, commit-reveal batch bets, optional leverage subsystem, lazy pool governance voting weight
- Explain architectural decisions omitting orderbooks, combinatorial markets, and peer prediction
- Updated chainbase submodule commit from 39ab2c2 to d429230
- Ensures third-party library is aligned with latest upstream changes
…cycle, coverage floors and thin-client APIs

Consensus (HF14 follow-up ops, appended so operation indices stay stable):
- pm_dispute_oracle_respond (op 22): the market oracle posts a public rebuttal
  onto an open dispute; stored on the dispute object (public-hearing model),
  allowed only while open and within oracle_response_deadline, re-post overwrites.
- pm_unban (op 23): the resolver that imposed an account-mode ban (banned_by)
  may lift it early; sets banned_until to epoch and clears banned_by.
- pm_ban_expired (virtual): the per-block cron sweeps temporary oracle/creator
  bans at banned_until, clears them and emits the lift for history/indexers.

On-chain state:
- pm_market gains decision_url/decision_reason — the oracle's resolution
  statement stored on-chain (set by pm_resolve_market / pm_no_contest reason),
  readable via get_market with no history scan.
- pm_resolve_market_operation gains decision_reason (reflected on the wire).
- pm_dispute gains oracle_response/oracle_response_time.
- pm_oracle and pm_creator_ban gain banned_by; pm_creator_ban gains a
  by_ban_expiry index so the cron sweeps expired bans oldest-first (cleared
  bans sort into the 0-bucket, permanent bans past now, both skipped).

Chain properties (witness-median tunables):
- pm_listing_min_coverage_percent (2.5x): hide under-insured markets from the
  default catalog (enforced by the API plugin, revealed via show_risky).
- pm_betting_min_coverage_percent (1.5x, advisory): client risk-confirm
  threshold; validated betting <= listing.

Thin-client read APIs (non-consensus, for the viz-js client):
- get_leverage_quote / get_leverage_close_preview / get_leverage_convert_preview
  reuse the frozen pm::leverage math to mirror the open/close/convert evaluators.
- get_market_categories (taxonomy + live counts), get_market_full (one-call
  enriched, account-scoped), get_lazy_allocations / get_market_lazy_allocation.
- Wallet remote_node_api bindings for all of the above.

Docs & tests:
- EN + ru + zh-CN docs updated (chain-properties, prediction-market-api,
  specification, operations overview/prediction-markets/validators,
  virtual-operations); library-integration spec + thin-client plan added.
- test_pm_lifecycle: cases #58-#63 cover oracle rebuttal + decision_reason,
  no-contest rationale, manual unban and its guards, and ban auto-expiry vop.
…throughs

The cli_wallet build failed because remote_prediction_market_api and the
wallet_api pm_get_*/pm_list_* methods returned the node's typed objects. Those
chainbase state objects (pm_market_object, pm_bet_object, ...) and the API DTOs
embedding them are not default-constructible (deleted default ctor / shared_string
members require a segment manager), so fc::api's client deserializer (T tmp;
var.as<T>()) could not instantiate them.

Return fc::variant instead: the node already emits fully-formed JSON and cli_wallet
prints the variant unchanged, so the read surface is identical.
…te_node_api

cli_wallet failed to compile because remote_node_api.hpp pulled in
<graphene/plugins/prediction_market_api/prediction_market_api.hpp> transitively,
but programs/cli_wallet has no include path to that plugin. After the read
pass-throughs switched to fc::variant, the header (and the pmapi alias) are no
longer referenced anywhere in the wallet, so remove them. graphene_wallet still
builds; the public wallet header no longer leaks a plugin-only dependency.
…LP fee

Add two HF14 median-voted consensus parameters and their enforcement:

- pm_oracle_accept_window_sec (default 1h): a pending market the named
  oracle never accepts nor rejects is voided by the per-block cron once
  now >= created_time + window. The creators seed liquidity is refunded
  (return_liquidity); the non-refundable creation fee stays with the DAO
  fund. Tracked via a new pm_market_object.accept_deadline field and a
  by_accept_deadline index; emits the new pm_market_expired virtual op
  (op-id 101, appended last in the operation variant to keep tags stable).

- pm_lazy_min_liquidity_fee_percent (default 2%): the lazy pool skips
  markets whose liquidity_fee_percent is below this reward floor, so it
  never subsidizes depth it is not paid enough to provide.

Wired into calc_median and chain_properties_pm::validate().
… fee

Cover the new pm_oracle_accept_window_sec / pm_market_expired lifecycle
and the pm_lazy_min_liquidity_fee_percent reward-floor gate across:

- EN docs (chain-properties, specification, operations, virtual-operations)
- RU and zh-CN localizations (@l10n) at full parity with the EN source
- library integration spec (delta section + property/vop tables, op-id 101)
  and thin-client plan
- Onix paper EN + RU (state machine, acceptance flow, lazy-pool gate);
  PDFs rebuilt via pandoc + xelatex (EN 30pp, RU 32pp, 0 missing glyphs).
- Added warning that the live protocol uses basis points (bp), not permille (‰)
- Explained the conversion from original PHP prototype’s permille to bp in on-chain code
- Specified that all fee fields (oracle_fee_percent, creator_fee_percent, liquidity_fee_percent, etc.) use bp (10000 = 100%)
- Highlighted the use of `fromBP` parser for fee fields and rejection of markets exceeding fee sum 10000
- Warned that using deprecated `fromPermille` leads to incorrect fee values, off by a factor of 10
…line

The ?: between time_point_sec() and (now + fc::seconds(...)) has no common
type — the latter yields fc::time_point, and each type converts to the other,
which GCC rejects as ambiguous. Wrap the second branch in an explicit
time_point_sec(), matching the copy-init conversion already used for the
reveal/dispute deadlines in this file.
…erations

The generic impacted-account visitor only collected signing authorities, so
prediction-market events were missing from the histories of accounts that did
not sign them:

- signed ops lost their counterparties (pm_create_market -> oracle,
  pm_transfer_position -> recipient, pm_unban -> target, oracle auto-accept
  whitelist);
- virtual ops carry no authority at all, so payouts, forfeits, liquidations,
  oracle penalties, market accept/expire and ban expiry were invisible to the
  affected users.

Add explicit get_impacted_account_visitor overloads for the PM user and
virtual operations, inserting every account field they carry. Market-only
virtual ops that reference a market by id but carry no account name
(pm_batch_settle / pm_dispute_finalize / pm_dispute_auto_close /
pm_lazy_recall) are intentionally left to the generic handler.
@On1x

On1x commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Addressed the account-history review blocker in 3eebfd6: added explicit get_impacted_account_visitor overloads for the PM user and virtual operations (counterparties on signed ops + all account fields on virtual ops, which have no signing authority). Market-only virtual ops that carry no account name (pm_batch_settle / pm_dispute_finalize / pm_dispute_auto_close / pm_lazy_recall) are intentionally left to the generic handler.

…, per-node)

The free-form `metadata` JSON was stored in the consensus `pm_market_object`
(shared_string) permanently — never pruned — even though consensus never reads
it (it is written once and only parsed off-chain by the prediction_market_api
plugin). That let a market permanently bloat every node's chainbase/shared
memory with unbounded, unvalidated data.

Move it out of consensus entirely:

- pm_market_object: drop the `metadata` field (member, ctor, FC_REFLECT). The
  operation `pm_create_market_operation.metadata` is unchanged — clients still
  send it and it lives in the block log, exactly like custom_operation.json.
- pm_create_market_evaluator: stop persisting metadata into state.
- prediction_market_api: ingest metadata off-chain from the create operation
  (post_apply_operation) into the existing prunable pm_market_meta_object,
  instead of reading it back from the consensus object in on_block.
- snapshot: drop the metadata import/export for pm_market (auto-excluded from
  the reflected dump; import of legacy snapshots ignores the field).

Because it is now non-consensus, each node prunes it on its own schedule via
--pmm-ttl-days (default lowered 7 -> 5; 0 keeps it forever for archival nodes).
No consensus length/UTF-8 cap is needed — the blob no longer touches state.
@On1x

On1x commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Addressed the unbounded on-chain metadata review blocker in 5479b9a (design agreed with maintainer): the free-form pm_create_market.metadata is no longer persisted in the consensus pm_market_object at all — consensus never reads it, so it moved off-chain into the prediction_market_api plugin, ingested from the create operation into the already-prunable pm_market_meta_object. Consequences: no permanent chainbase bloat, no consensus length/UTF-8 cap needed (the blob never touches state; it lives in the block log like custom_operation.json), and each node prunes it on its own schedule via --pmm-ttl-days (default lowered 7→5; 0 = keep forever for archival nodes). Note: plugin metadata index is now built from the operation stream, so enabling the plugin on an existing chain requires a replay to backfill (standard for plugin indexes). Not compiled locally (no build env here) — CI/build to confirm.

…xed retention

A resolved+settled market (status 3, payout_status 3) is immutable — no betting,
dispute, resolve or payout can touch it again; it only lingered in chainbase
"for history", growing shared-memory state without bound.

process_pm_markets() now GCs such markets and their whole object cluster
(outcomes, bets, liquidity, commits, dispute votes, leverage positions, the
dispute and lazy-allocation rows) once they have been closed for a FIXED protocol
constant PM_CLOSED_MARKET_RETENTION_SEC = 5 days (measured from
result_expiration + dispute grace). The retention is hardcoded and identical on
every node, so pruning is fully deterministic: every node deletes exactly the
same markets at the same block, keeping shared-memory state and snapshots in
lock-step network-wide (a node syncing from a snapshot ends up with the same
market set as everyone else). Work is bounded by the existing per-block cap.

Only status-3/payout-3 markets are collected; disputed (payout_status 2) and
never-settled markets are left untouched. Nothing holds an id-reference to a
settled market, so there are no dangling references after removal.
@On1x

On1x commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Follow-up (31e8aab): resolved markets are now garbage-collected from consensus state. A settled market (status 3 / payout_status 3) is immutable, so process_pm_markets() reclaims it and its whole object cluster (outcomes/bets/liquidity/commits/dispute-votes/leverage/dispute/lazy-allocation) after a FIXED PM_CLOSED_MARKET_RETENTION_SEC = 5 days (from result_expiration + grace). Retention is a hardcoded protocol constant — identical on every node — so deletion is fully deterministic: all nodes prune the same markets at the same block and snapshots stay in lock-step (a node syncing from a snapshot gets the same market set as everyone else). Bounded by the existing per-block cap; only status-3/payout-3 markets are touched (disputed = payout_status 2 are excluded). Not compiled locally (no build env) — needs CI + ideally a consensus_sim case.

…ones

Extend the market garbage collector to reclaim ANY dead market a fixed 5 days
after it becomes terminal — not only resolved+paid ones. A market is dead once
nothing can act on it: resolved and paid out, void/no-contest, oracle-rejected,
or the oracle never accepted and the accept window expired.

To anchor the retention on the actual moment of death (rather than the declared
result_expiration), add a `finalized_time` field to pm_market_object, set to the
head-block time at every terminal transition:
  - oracle rejects the market (status -1)
  - accept window expires, market voided (pm_market_expired)
  - oracle misses resolution, refund (pm_oracle_missed_penalty)
  - dispute auto-close refund
  - settlement / auto-payout (covers resolved, no-contest, post-dispute)

A new by_finalized index (finalized_time, id) lets process_pm_markets() sweep
terminal markets in time order, skipping the finalized_time==0 live bucket, and
delete each cluster PM_CLOSED_MARKET_RETENTION_SEC (5 days) later. Retention is a
fixed protocol constant identical on every node, so pruning stays deterministic
and snapshots identical network-wide. Snapshot import reads finalized_time when
present. Work stays bounded by the per-block cap.
On1x and others added 2 commits August 12, 2026 17:34
…teemit#536)

Require loan >= pm_min_liquidity in the pm_leverage_open evaluator. validate()
only checks loan>0, so without a floor a Sybil could open unbounded near-zero
loan positions on one market, each consuming a slice of the capped leverage
fund and a settlement force-close slot -> a liquidation backlog that throttles
PM cron throughput for ~N/cap blocks (see the leverage_sybil_settlement_is_cap
_throttled coverage added in PR #151). A minimum loan bounds the global open
position count to fund_total / pm_min_liquidity, so the backlog is bounded by
construction. Reuses the existing median-voted pm_min_liquidity (no new chain
property or serialization change); governance can still raise the floor.

Adds the leverage_min_loan_floor_enforced consensus_sim case: a sub-floor loan
is rejected, a loan exactly at the floor is accepted (so PR #151's Sybil case,
which borrows exactly pm_min_liquidity, still opens).
Adds tests/pm/replay/t17_leverage_math_invariants.cpp, which LINKS the real
libraries/chain/pm/leverage.cpp margin math (no chain build required) and asserts the
safety properties the leverage subsystem depends on, across adversarial + fuzzed inputs:

  I.   bounded k on buy: new_reserve_a × new_reserve_b ∈ (k − divisor, k] — the CPMM floor
       never lets the product EXCEED the stored k (no phantom constant-product for the pool)
       and loses at most one divisor-unit, so tokens_out is not unboundedly inflated. The
       stored k is not reassigned on a bet, so this bound holds bet-over-bet.
  II.  round-trip non-profit: buy `amount`, immediately cancel the tokens at the same
       reserves → cancel_value ≤ amount. A self-sandwich cannot mint against the curve.
  III. monotonicity: cancel_value_after_opposing is non-increasing in the opposing bet m —
       the property that makes worst_opposing_bet the actual worst case for the open-time
       solvency check.
  IV.  solvency: max_leverage_loan only returns a loan L for which the worst-case cancel
       value cvw ≥ threshold_safe (= liquidation_threshold(L) × (1 + s%)), i.e. it never
       hands out an under-margined loan; returns 0 when nothing qualifies.
  V.   400k-case randomized fuzz over the reachable reserve range: no negative outputs, no
       128-bit wrap, k bounded, round-trip-safe — 0 violations.

The k bound is the exact floor guarantee: new_out = floor(k / new_in) ⇒
new_in × new_out ∈ (k − new_in, k]. (My first draft asserted the wrong direction / wrong
divisor; the math is correct — characterizing the exact bound is what the exercise
surfaced.) Registered in tests/pm/replay/build.sh; built + run green with g++.
@chiliec

chiliec commented Aug 12, 2026

Copy link
Copy Markdown
Member

Exploit-class audit — HF14 prediction markets (money paths)

Systematic pass over the PM money code against the well-known DeFi / AMM / prediction-market exploit classes. Verdict: the classic attack surfaces are closed. No new exploitable vulnerability found. Each conclusion is now backed by an executable replay test (links the real consensus math, no chain build needed) where the property is mechanically checkable.

Checked and defended

1. First-depositor / ERC-4626 share inflationpm_lazy_deposit prices shares off pool equity (free + allocated + leverage_fund_used − pending_withdrawals, not bare free_balance) and reverts on new_shares == 0 ("Zero shares minted"). The rounding-to-zero donation sink is closed — a griefer can't make a later depositor mint 0 shares and lose funds.

2. Rounding / round-trip theft (self-sandwich) — buy floors new_reserve_out (bettor-favourable by ≤1 unit); cancel is the exact mirror, capped at stake (F1/#300), with the curve-priced F2 refund. A buy→cancel or add→withdraw round-trip cannot net a profit. Verified in t17 (round-trip cancel_value ≤ amount; k bounded to the exact floor guarantee new_in × new_out ∈ (k − new_in, k], so no phantom constant-product).

3. MasterChef reward_per_share drainreward_snapshot is set to current rps at deposit, so a fresh depositor can't claim pre-entry rewards; every rps bump is guarded by total_shares > 0. LP distribution conserves exactly (Σ shares == bonus) — verified in t16 §E.

4. k-invariant / reserve manipulationk is preserved on bets/leverage (stored k reused) and recomputed only on liquidity add/remove/alloc (price-neutral scaling; the B4 round-trip fix). No mutable-k vector. Verified in t17 §I.

5. Settlement zero-sum / mint — the parimutuel split is strictly zero-sum: Σ winner_payout + oracle_take + creator_take + lp_bonus == Σ winner.amount + losers_sum + forfeit_pool + uncovered. The negative-winners_pool wrap (B3) is clamped and reported as uncovered (F1), charged to LP principal, never emitted. Empty-winners (oracle resolves to a non-participated outcome) routes the whole pool to LP. Verified in t16 §A–D + a 300k-case fuzz (0 violations).

6. Margin solvency + liquidationmax_leverage_loan only admits a loan whose worst-case cancel value clears the safety threshold; cancel_value_after_opposing is monotone in the opposing bet, so worst_opposing_bet is the true worst case. Verified in t17 §III–IV.

7. Self-oracle abuse — bounded economically (5000 VIZ insurance floor, 1000 VIZ dispute fee, 5% slash on missed deadline, listing hidden below 2.5× coverage), and conservation holds regardless of the resolved outcome (empty-winners → LP). Not a code bug.

8. Sybil / DoS via many positions — no per-account/per-market position-count cap and no minimum loan, but forced settlement is throttled: process_pm_markets §2d closes positions once betting is over and shares the single per-block done < cap budget, so ≤ cap close per block. Not a single-block DoS. (Covered by the consensus_sim leverage_sybil_settlement_is_cap_throttled case in #151.)

Residual hardening suggestions (defense-in-depth, not blocking)

  • Per-account leverage-position cap and/or a minimum loan size — bounds the settlement-liquidation backlog a Sybil flood can create (N positions take ~N/cap blocks to drain; liveness, not a halt).
  • Per-market loan sub-budget on the single global lazy pool — localizes the blast radius of leverage bad debt (which is a real, by-design LP loss; it's a zero-sum transfer, never a mint, but every LP across all markets currently shares every position's counterparty risk).

New test coverage

All findings are from reading the current audited source plus the linked executable tests; the #1#5 / B/F series fixes already in pm close the sharpest edges, including several from this review round.

On1x and others added 8 commits August 12, 2026 18:21
The parameter was declared in chain_properties_pm and reflected (so validators
publish it), but was missing from the calc_median() list in
update_median_validator_props(). As a result median_props.pm_closed_market_retention_sec
was frozen at the 432000 default and could never be governed, contradicting the
"median-voted" contract (pm_evaluator.cpp settlement/GC path). HF14-gated; with all
validators publishing the default it computes to the same 432000, so no behavior
change on existing chains. Unblocks the five gc_*_after_retention lifecycle tests,
which set retention=30 and previously could never reach it.
Eight consensus_sim/pm_lifecycle cases carried expectations that predated the
PR #124 audit fixes; update them to the current (live-accepted) behavior so the
whole suite is green again (58 cases, 0 failures). No production code touched here.

- lazy_recall / lazy_active / lazy_yield: markets now must post
  liquidity_fee_percent >= pm_lazy_min_liquidity_fee_percent to receive a lazy-pool
  allocation (min-fee gate), so give the fixture markets that fee.
- oracle_missed_refunds_and_slashes: the missed-resolution void now fires at
  result_expiration + pm_dispute_grace_sec (reachability fix), so publish a short
  grace instead of expecting the void at result_expiration.
- leverage_resolve_vop_at_settlement: a position force-closes at resolution (not at
  finalization), so connect the pm_leverage_resolve listener before resolving.
- leverage_cancel_bet_cascade_bad_debt: depth-normalized cancel (#1-C) → engineered
  cancel_value is 272727 (matches leverage_bad_debt_is_transfer_not_mint).
- leverage_open_and_close: a profitable voluntary close pays the pool its obligation
  and defers the trader surplus to settlement (F1), so the immediate balance delta is
  -collateral.
- bet_cancellation_reverses_cpmm: curve-priced cancel (F2) capped at stake (F1); assert
  refund <= stake and that stake - refund is conserved into forfeit_pool.

The five gc_*_after_retention cases are fixed by the companion node commit that makes
pm_closed_market_retention_sec median-voted (they needed no test change).
… LP×2, leverage)

Add docs/prediction-markets/guides/ with a hub index and six role guides written
for participants (persona: 'you are X'): bettor, market creator, oracle, active LP,
passive LP (lazy pool), leverage trader. Link the hub from the PM overview and the
sidebar. Feature guides to follow.
Two participant feature guides in docs/prediction-markets/guides/: why prices come
from the pool curve (CPMM/LMSR) instead of fixed odds, and how outcome disputes work
(grace window, escrow fee, voting modes, oracle penalty). Wire both into the guides hub.
Two more participant feature guides: the resolution mechanism/timeline (betting close →
resolve → deadline/grace → no-contest/missed-resolution → early resolution) and
multi-outcome markets (LMSR vs binary CPMM, pm_max_outcomes cap). Wire into the hub.
Curve-priced bet cancellation (cap-at-stake, residual to forfeit_pool) and the deep
lazy-pool mechanics (equity-priced shares, yield sources, FIFO withdrawal queue with
the free_balance >= 0 invariant, reward-only emergency penalty). Wire into the hub.
…lete)

Final two feature guides: early-exit deferred outcome-contingent claim (principal
returned immediately, profit tail paid at settlement from a bounded slice of the
losing pool, LP protection rationale) and hidden bets (commit-reveal + batches,
reveal window, no-reveal forfeit). Completes the guides feature set.
… (validate bound)

Defensive hardening for the commit-reveal escrow path. A commit's reveal deadline can
fall up to (pm_batch_epoch_blocks + pm_reveal_window_blocks) blocks after the commit, and
its escrow is only refunded/forfeited by the reveal-forfeit cron at that deadline.
gc_market deletes pm_commit rows unconditionally once a market has been finalized for
pm_closed_market_retention_sec, with no status-0 refund.

Today this is safe purely by parameter magnitudes (5 d retention vs ~11 min worst-case
reveal deadline). Nothing in code enforces the ordering, and pm_closed_market_retention_sec
had no lower bound — a misconfigured median (retention below the reveal window) plus a
sustained cron cap-starvation could let a still-unrevealed commit be garbage-collected
before its escrow is returned, stranding a bettor's stake.

Add a validate() assert requiring
  pm_closed_market_retention_sec > (pm_batch_epoch_blocks + pm_reveal_window_blocks) * CHAIN_BLOCK_INTERVAL
so the "a commit is always cleared before its market is GC'd" invariant holds by
construction. Both operands are already validated positive just above. Defaults satisfy it
with wide margin (432000 > (20 + 200) * 3 = 660); no testnet override changes these.

Header syntax-checks clean against the chain build flags. No behavioural change for any
valid configuration.
@chiliec

chiliec commented Aug 12, 2026

Copy link
Copy Markdown
Member

Deep-dive re-audit — dispute economics, commit-reveal, lazy-pool allocation

Traced every money path in the three least-tested PM subsystems. All conservation holds; one latent hardening gap found (fixed in #155).

Confirmed sound

Dispute fee conservation — the fee is debited once at filing and returned/routed in all six terminal branches: auto-close → disputer; committee uphold → oracle; committee override pp<0 → oracle carve-out + disputer refund; pp>=0 → disputer (fee + bonus); account-mode oracle-wrong → disputer; account-mode uphold → oracle. Slash accounting balances (bonus <= slash, remainder → forfeit_pool → winners); when the oracle has no insurance, slash = 0 clamps the bonus to 0 with no phantom reward.

Commit-reveal — the commitment hash binds (market_id, account, side, outcome_index, amount, min_tokens, salt), so there is no cross-market replay and no commitment theft. Reveal enforces amount <= escrow, refunds the surplus, and freezes only the revealed stake. No-reveal forfeit conserves exactly: escrow = refund + penalty, penalty → forfeit_pool.

Lazy-pool allocationfree_balance -= alloc; allocated_balance += alloc, matched by the created LP position and the recall-tracking object; capital returns via route_pool_lp_return at settlement/recall. Per-oracle exposure penalties (active-market decay + fault stamps) bound how much idle capital a single oracle can attract.

Finding (LOW / latent) — commit escrow vs. market GC

gc_market drops pm_commit rows unconditionally, with no status-0 refund. It is money-safe today only because of parameter magnitudes: a commit's reveal deadline is ~11 min worst case (the reveal-forfeit cron refunds/forfeits every commit there), vs. the 5-day pm_closed_market_retention_sec GC retention. But nothing in code enforced that ordering, and the retention had no lower bound.

A misconfigured median (retention below the reveal window) plus a sustained pm_processing_cap_per_block starvation could, in principle, let a still-unrevealed commit be garbage-collected before its escrow is returned. Not attacker-triggerable, cannot happen under sane params — but an invariant resting on luck rather than construction.

#155 closes the root cause with one validate() assert:
pm_closed_market_retention_sec > (pm_batch_epoch_blocks + pm_reveal_window_blocks) * CHAIN_BLOCK_INTERVAL
(defaults satisfy it with wide margin: 432000 > 660). A belt-and-suspenders alternative — make gc_market refund lingering status-0 commits before dropping them — is available if defense-in-depth is preferred, but the bound already prevents the reachable case.

This complements the earlier exploit-class pass (settlement zero-sum, leverage math, margin abuse) — the classic surfaces remain closed; this was the one spot where a money invariant was enforced only by parameter choice, now enforced in code.

On1x added 14 commits August 13, 2026 06:40
…oupling

fix(pm): couple closed-market retention to the commit reveal deadline (validate bound)
…ariants

test(pm): replay test — leverage/margin math invariants (t17)
refactor(pm): split pm_evaluator.cpp — extract cron + helpers into pm_process_markets.cpp
# Conflicts:
#	tests/pm/replay/build.sh
…t-classes

test(pm): replay test — settlement zero-sum vs known exploit classes (t16)
…ed to market

The cancel-bet guide framed the full stake↔refund gap as routing to
forfeit_pool (loss case only). Per pm_evaluator cancel logic (F1/steemit#300),
when the curve moved in the bettor's favor the curve profit above stake
is retained as the bettor's outcome-contingent deferred claim, not given
to the market. Distinguish the two directions, note the deferred tail,
and cross-link the early-exit guide.
…lash), not no-contest

The 'if no outcome' bullet lumped oracle silence past the deadline into
no-contest. Per the resolution flow, an active pm_no_contest refunds
bettors with no penalty, whereas silence past result_expiration+grace is
missed-resolution: bettors are still refunded but the oracle is slashed.
Distinguish the two and cross-link the resolution guide (was inconsistent
with oracle.md / resolution.md).
bettor.md repeated two inaccuracies fixed in sibling guides: the cancel
summary said the stake↔refund gap wholly stays with the market (omits the
curve-profit deferred claim), and the 'no outcome' bullet mislabeled oracle
silence as no-contest (it's missed-resolution + slash). Bring both in line
and cross-link cancel-bet. Also fix a Latin-t typo in lazy-pool.md.
…etry

H1: snapshot import_pm_markets now restores accept_deadline (plus
decision_url/decision_reason) — without it pending markets import with
accept_deadline=epoch and the §2b cron voids them on the next block
(consensus divergence vs a replaying node).

H2: queued/revealed-pending batch bets (status 5/6) hold escrowed stake,
but the §6 executor only runs on status-1 markets. Every terminal path
now refunds them nominally (never touched the curve): a unified pre-pass
in settle_market covers win<0 AND win>=0 resolves, refund_all_bets (§2
missed-resolution, §3 dispute auto-close) covers 0/5/6.

H4: §2 and §3 crons route forfeit_pool like #5 (pro-rata to refunded
bettors, burn when none) — previously GC dropped the market row and the
tokens stayed in current_supply unowned, breaking the re-armed PM supply
invariant on the next snapshot import.

M1: snapshot clear-block now clears pm_lazy_withdraw_request,
pm_deferred_claim and (has_index-guarded) pm_market_meta — leftover rows
from a prior snapshot conflicted on by_id and aborted hot-reload imports.

M2: partial transfer_position and leverage_convert now inherit/set the
#1-C entry_liquidity anchor — defaulting to 0 marked the new bet "legacy"
and let its cancel-refund bypass the liquidity-growth cap.

M6: validate() floors pm_dispute_grace_sec at 3600 s — zero grace races
the cleanup crons against settlement (default 12 h is unchanged).

No object-layout changes (snapshot/replay compatible). Single-TU
-fsyntax-only clean on pm_evaluator.cpp, pm_process_markets.cpp and
snapshot/plugin.cpp.
H3: expected_payout no longer walks every market bet per position (O(N²)
from get_market_full / get_account_positions = one UI call is a read-DoS
on an attacker-built market). Per-market per-side winner aggregates are
cached per head block (thread_local, same pattern as the risk-floor
cache) — position loops are now O(N + positions). Display-only.

M3: dispute ballots are free; cap NEW rows per disputed market at
MAX_PM_DISPUTE_VOTES_PER_MARKET (10000, steemit#349-style). The counting walk
stops at cap+1, so enforcing is bounded; ballot revisions stay allowed.

M4: per-market open-commit backlog is now an O(1) counter
(pm_market_object.open_commits, reflected + snapshot-imported with a
contains guard) capped at MAX_PM_OPEN_COMMITS_PER_MARKET (10000) —
commit escrow costs only the 20% no-reveal penalty, so an uncapped
backlog could throttle the shared pm_processing_cap_per_block in the §1
forfeit cron. Decrements are clamped so pre-M4 snapshot drift can only
relax the cap. Also floor pm_min_batch_bet at 0.1 VIZ in validate()
(was only >0 — votable to 1 satoshi).

M7: leverage open Constraint-1 used free_balance - leverage_fund_used,
but free_balance is already net of active loans — the double deduction
understated pool lending capacity (conservative bug). Now free_balance
is the solvency floor. (fund_total free-vs-NAV base: open design note.)

L1: §6 batch settle is no longer gated on pm_commit_reveal_enabled — the
kill-switch stops NEW commits (evaluator assert) but must not freeze
funds already queued; they drain to execution, and batch-A H2 refunds
any residual at terminal settle as a backstop.

Single-TU -fsyntax-only clean on pm_evaluator.cpp, pm_process_markets.cpp,
prediction_market_api.cpp and snapshot/plugin.cpp. NOTE: adds a field to
pm_market_object — old shared_memory incompatible, redeploy from snapshot.
Governance parameters are median-voted, so validate() is the only
consensus gate against pathological values — this adds a boundary probe
(below/at/above) for every ratio cap, floor and cross-field invariant,
including the audit-added floors:
  M6: pm_dispute_grace_sec >= 3600
  M4: pm_min_batch_bet >= 0.1 VIZ
plus the constructive GC invariant (retention strictly exceeds the
worst-case commit reveal deadline).

Local run: 9/9 test cases pass; existing pm_lmsr_vectors / pm_parimutuel
/ pm_leverage / pm_meta_parse stay green after batches A+B.
CI proved compilation only; the PM unit tests (LMSR vectors, parimutuel
conservation, leverage math, meta parsing, validate() boundary sweep)
were run manually. Add a `pm-tests`-label-gated job (same pattern as
check-boost-range) that builds only the tests/pm targets and runs
ctest -R '^pm_' — the consensus_sim harness stays configured but unbuilt,
and the default PR build is untouched.
…o the M6 grace floor

New consensus_sim cases (audit batch C, L3 gaps):
- lazy_withdraw_fifo_queue: a leverage loan drains pool free_balance to 0,
  emergency withdrawals queue owed requests, and every capital return
  (deposit, leverage close) services the queue strictly oldest-first with
  free_balance never negative; partial returns pay the head request only.
- oracle_profile_update_and_insurance_gate: insurance top-up/withdraw 1:1
  accounting, fee-percent cap, withdrawal floor, the unsettled-market
  withdrawal gate, and wrong-signer rejection.

Suite adaptation: batch A M6 floors pm_dispute_grace_sec at 3600 s in
chain_properties_pm::validate(), which rejected the test-short grace values
(30/5 s) the suite used to accelerate crons. All grace settings raised to
the 3600 s floor, fixed 120-300 block settlement waits replaced with a
grace-derived bound (cron_grace_blocks), and publish_fast_pm_props now
shrinks the batch epoch/reveal windows so the 30 s GC retention still
exceeds the worst-case commit reveal deadline. Full pm_lifecycle_suite
(61 cases) passes.
@On1x On1x added the pm-tests label Aug 13, 2026
On1x added 2 commits August 13, 2026 23:05
… M5)

import_pm_disputes set 11 of the 13 reflected fields, dropping
oracle_response and oracle_response_time. An imported dispute the oracle
had already answered regressed to the awaiting_response stage: gauge
seeding and pm_oracle_dispute_left_open then decremented the wrong bucket
when it closed — a permanent drift in the oracle workload display (not
money, but exactly the snapshot-symmetry class H1/#136 fixed for markets
and oracles). Restored both fields, contains-guarded so snapshots exported
before this change still import with the unanswered defaults.

Export side was already complete (fc::to_variant over the full reflect).
Single-TU -fsyntax-only clean.
…08-14)

Webserver response cache only hits byte-identical requests; walking `from`
misses it, so the work itself must be bounded:
- MAX_PM_PAGE_FROM=1e6 clamp on `from` in all 18 paginated read APIs.
- by_category newest/oldest: walk the (category,id) range with on-the-fly
  filters and early stop instead of materialize+sort; volume/expiration
  sorts capped at MAX_PM_SORT_POOL=32768, degrade to newest walk above.
- by_oracle/by_oracle_status/by_creator id-order paging via shared
  page_markets_id_order helper; sort pools capped with same degradation.
- get_market_categories cached per head block (winner_agg pattern).
- make_weight_sums / oracle_awaiting cached per (head block, key), cap 512.
- get_market_full my_positions/my_leverage/my_liquidity capped at 1000.
Read-only plugin: no consensus or object-layout changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants