Skip to content

Guarantee a full channel transaction history: record funding txs from block scans and from the mempool #9697

Open

Description

@cdecker

Problem

The transactions table (and its new our_txs counterpart) is populated opportunistically
and inconsistently for channel funding transactions. Whether a funding tx appears in the
table — and whether its rawtx is ever stored at all — depends on which code path happened to
touch it:

  • Block path (confirmed): filter_block_txs() (lightningd/chaintopology.c:44) inserts
    txs that fire a txowatch, match watch_check_tx_outputs(), watching_txid() or
    we_broadcast(). So confirmed funding txs do get recorded — but only after confirmation,
    and via conditions that are incidental to "this is a channel funding tx".
  • Broadcast path: topology_broadcast_soon() (lightningd/chaintopology.c:355) records
    the tx at height 0 — but the dual-open funding path bypasses it:
    send_funding_tx() (lightningd/dual_open_control.c:1885) calls
    bitcoind_sendrawtx() directly, and the insert in handle_tx_broadcast()
    (lightningd/dual_open_control.c:1802) is gated on
    wallet_extract_owned_outputs() — i.e. it only runs if the funding tx happens to pay an
    output to our wallet. For a typical dual-open funding tx (all outputs are the shared
    funding output) nothing is inserted, even though we broadcast the tx ourselves.
  • Mempool path (observed, not broadcast by us): does not exist. If the peer broadcasts
    the funding tx (the common zero-conf case, and any lockin where the funder is the peer),
    we have no way at all to see it until it confirms. PR lightningd: track funding-tx broadcast visibility, gate zero-conf forwarding on it #9437 highlights this: its 5-second
    poll uses getutxout (which is why it can only prove visibility, not record the tx —
    gettxout never returns the raw transaction), and introduces a persisted
    funding_tx_status enum because nothing else in the DB distinguishes "never broadcast"
    from "in mempool" from "confirmed".

Net effect: for a meaningful fraction of channels, listtransactions has no knowledge of
the funding tx until it is confirmed (or forever, for the funding txid if the channel fails
pre-confirmation), and we cannot answer basic questions like "have we ever seen this funding
transaction?" from the wallet alone.

Invariant we want

Every channel-relevant transaction — starting with the funding tx — has a row in
transactions (with rawtx populated) from the moment we first learn it exists
, and the
channel's persisted visibility state is consistent with the table:
funding_tx_status >= MEMPOOL ⇔ a row exists for the funding txid.

Concretely, a funding tx must be recorded, at height 0, the first time any of these
happens:

  1. We broadcast it — unconditionally, regardless of whether the tx pays us anything.
  2. It appears in our bitcoind's mempool — even if we only ever observed it there (peer
    broadcast, zero-conf channels, and any *_AWAITING_LOCKIN channel whose funding we did
    not send ourselves).
  3. It is confirmed in a block we process — via the block scan, regardless of whether it
    matches a wallet script or a watch.

Subsequent confirmations/promotions then just update blockheight/txindex on the existing
row (this part wallet_transaction_add() already handles correctly).

Proposed changes

1. Recording in the block path

When processing blocks (today filter_block_txs(); tomorrow the bwatch block-dispatch path),
record any tx that spends to / creates a known channel's funding output into the
transactions table unconditionally — not only when a wallet filter matches. The channel
subsystem knows its funding outpoint; that alone should qualify a block tx for insertion.

2. Recording on our own broadcast

Fix the dual-open path so send_funding_tx() records the tx unconditionally (either call
wallet_transaction_add(wtx, 0, 0) directly or route through topology_broadcast_soon(),
which also gives us rebroadcast-until-confirmed for free). Drop the
wallet_extract_owned_outputs() condition for funding-tx recording — it is about change
outputs, not about tx relevance. Same audit for the splice broadcast
(lightningd/channel_control.c:539).

3. Mempool monitoring for pending funding txs

While a channel is in CHANNELD_AWAITING_LOCKIN / DUALOPEND_AWAITING_LOCKIN, and for any
zero-conf channel still at funding_tx_status == UNKNOWN, poll our bitcoind's mempool for
the funding txid. On a hit: fetch the raw tx, wallet_transaction_add(tx, 0, 0), set
funding_tx_status = MEMPOOL, save, and kick channeld_zeroconf_funding_seen(). This
subsumes PR #9437's getutxout poll (which can stay as a cheap "did it get evicted /
reorged out?" backstop, but can no longer be the primary mechanism, since it cannot yield
the rawtx our invariant requires).

Polling cadence can piggyback on the existing chaintopology/bwatch poll loop; the set of
pending funding txids is small (typically 0–2).

Relevant bitcoind RPCs

RPC Purpose for us Notes
getrawmempool [verbosity] enumerate/set-diff the mempool against our pending funding txids full-tx mode (verbosity=2) returns rawtxs in one roundtrip but is expensive; txid-only mode + targeted fetch is cheaper
getmempoolentry <txid> cheap existence probe + fees/depends/ancestor* for one known txid RPC errors with "txn not in memory pool" if absent — treat as negative answer, not failure; also lets us detect underfunded/evicted fundings
getrawtransaction <txid> fetch the rawtx of a txid we know is in the mempool works for mempool txs without -txindex (the mempool is checked first); confirmed-tx lookups would need txindex — out of scope here
getmempoolinfo mempool.minrelaytxfee, full flag detects "our tx was evicted because the mempool was full" → useful for fee-bumping a stuck funding tx
testmempoolaccept pre-broadcast sanity / RBF checks optional; not needed for the invariant

No ZMQ (hashtransaction/hashtx) support exists in this tree today, so RPC polling is the
mechanism; a future ZMQ subscription could replace the poll with push notifications, but the
fetch-by-txid interface below stays the same either way.

Abstract methods to expose via bcli

Follow the existing pattern (plugins/bcli.c commands + bitcoind_* wrappers in
lightningd/bitcoind.h, with the _ macro fallback for backends that don't implement an
RPC — see bitcoind_getutxout_()):

bcli_getmempooltxids  -> bitcoind_getmempooltxids(ctx, bc, cb, cbarg)
    /* getrawmempool (verbosity 0): array of txids. Callback gets the set;
       caller diffs against pending funding txids. */

bcli_getmempoolentry  -> bitcoind_getmempoolentry(ctx, bc, txid, cb, cbarg)
    /* getmempoolentry: cb(txid, bool present, fee, size, ancestors...).
       cb(NULL ...) on "not in mempool" — a definitive negative, since we
       poll after block-connect, see race note below. */

bcli_getmempooltx     -> bitcoind_getmempooltx(ctx, bc, txid, cb, cbarg)
    /* getrawtransaction <txid> (verbosity 0 / hex): cb(txid, wtx) or
       cb(txid, NULL) if not found. This is the call that lets us satisfy
       the "rawtx is stored" half of the invariant. */

bcli_getmempoolinfo   -> bitcoind_getmempoolinfo(ctx, bc, cb, cbarg)
    /* getmempoolinfo: size, full, minfeerate — eviction context. */

A cleaner long-term home may be bwatch rather than raw lightningd polling: add a
WATCH_MEMPOOL (or extend WATCH_OUTPOINT with a mempool tier) so that lightningd simply
gets watch_found with blockheight == 0 for mempool hits, and the poll lives in one plugin
next to the block scanner. The bcli methods above are identical either way; only the caller
moves.

Race note: between getmempoolentry saying "absent" and our next block fetch, the tx
may confirm without us having seen it in the mempool. The block-path recording from (§1) is
what closes that hole — mempool monitoring must be understood as best-effort early
detection, never the sole recording path.

Consistency with PR #9437

  • The new funding_tx_status column stays: it's the cheap, persisted answer to
    "never broadcast / seen / confirmed" and gates zero-conf forwarding. This issue makes it
    derivable from — rather than merely correlated with — the transactions table.
  • Where this PR's zeroconf_utxo_checked() poll uses getutxout, replace (or precede) it
    with bitcoind_getmempooltx() so the same callback records the tx before flipping the
    status.
  • Migration: for existing channels past lockin, funding_tx_status is backfilled to
    CONFIRMED already (PR lightningd: track funding-tx broadcast visibility, gate zero-conf forwarding on it #9437); consider a companion backfill that at least records funding
    txids of AWAITING_LOCKIN channels via getmempoolentry + getrawtransaction at startup.

Downgrade / tooling

  • No new tables; recording into both transactions and our_txs already goes through
    wallet_transaction_add(), so lightning-downgrade is unaffected.
  • bcli: new RPCs must fail gracefully (backend probe via check/getutxout-style
    detection) so external backends lacking mempool access just disable the poll, as
    getfilteredblock absence does today.

Testing

  • tests/test_opening.py: zero-conf open where only the peer broadcasts (dev-feerate /
    manual broadcast topology) → assert listtransactions shows the funding tx (height null)
    before it confirms, on the non-funding side too.
  • Dual-open test asserting the funding tx appears in listtransactions immediately after
    openchannel_signed, including the no-change-output funding case.
  • Stuck-funding test: funding tx underfeued and evicted → assert we notice via
    getmempoolinfo.full / getmempoolentry absence and mark it appropriately.

Checklist

  • Changelog entries in the relevant commits
  • Tests added/modified
  • Docs for the new bcli_* RPCs (doc/) and backend check negotiation
  • tools/lightning-downgrade still works (no new tables, but verify)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions