Repository navigation
Description
Problem
The transactions table (and its new our_txs counterpart) is populated opportunistically
and inconsistently for channel funding transactions. Whether a funding tx appears in the
table — and whether its rawtx is ever stored at all — depends on which code path happened to
touch it:
- Block path (confirmed):
filter_block_txs()(lightningd/chaintopology.c:44) inserts
txs that fire a txowatch, matchwatch_check_tx_outputs(),watching_txid()or
we_broadcast(). So confirmed funding txs do get recorded — but only after confirmation,
and via conditions that are incidental to "this is a channel funding tx". - Broadcast path:
topology_broadcast_soon()(lightningd/chaintopology.c:355) records
the tx at height 0 — but the dual-open funding path bypasses it:
send_funding_tx()(lightningd/dual_open_control.c:1885) calls
bitcoind_sendrawtx()directly, and the insert inhandle_tx_broadcast()
(lightningd/dual_open_control.c:1802) is gated on
wallet_extract_owned_outputs()— i.e. it only runs if the funding tx happens to pay an
output to our wallet. For a typical dual-open funding tx (all outputs are the shared
funding output) nothing is inserted, even though we broadcast the tx ourselves. - Mempool path (observed, not broadcast by us): does not exist. If the peer broadcasts
the funding tx (the common zero-conf case, and any lockin where the funder is the peer),
we have no way at all to see it until it confirms. PR lightningd: track funding-tx broadcast visibility, gate zero-conf forwarding on it #9437 highlights this: its 5-second
poll usesgetutxout(which is why it can only prove visibility, not record the tx —
gettxoutnever returns the raw transaction), and introduces a persisted
funding_tx_statusenum because nothing else in the DB distinguishes "never broadcast"
from "in mempool" from "confirmed".
Net effect: for a meaningful fraction of channels, listtransactions has no knowledge of
the funding tx until it is confirmed (or forever, for the funding txid if the channel fails
pre-confirmation), and we cannot answer basic questions like "have we ever seen this funding
transaction?" from the wallet alone.
Invariant we want
Every channel-relevant transaction — starting with the funding tx — has a row in
transactions(withrawtxpopulated) from the moment we first learn it exists, and the
channel's persisted visibility state is consistent with the table:
funding_tx_status >= MEMPOOL⇔ a row exists for the funding txid.
Concretely, a funding tx must be recorded, at height 0, the first time any of these
happens:
- We broadcast it — unconditionally, regardless of whether the tx pays us anything.
- It appears in our bitcoind's mempool — even if we only ever observed it there (peer
broadcast, zero-conf channels, and any*_AWAITING_LOCKINchannel whose funding we did
not send ourselves). - It is confirmed in a block we process — via the block scan, regardless of whether it
matches a wallet script or a watch.
Subsequent confirmations/promotions then just update blockheight/txindex on the existing
row (this part wallet_transaction_add() already handles correctly).
Proposed changes
1. Recording in the block path
When processing blocks (today filter_block_txs(); tomorrow the bwatch block-dispatch path),
record any tx that spends to / creates a known channel's funding output into the
transactions table unconditionally — not only when a wallet filter matches. The channel
subsystem knows its funding outpoint; that alone should qualify a block tx for insertion.
2. Recording on our own broadcast
Fix the dual-open path so send_funding_tx() records the tx unconditionally (either call
wallet_transaction_add(wtx, 0, 0) directly or route through topology_broadcast_soon(),
which also gives us rebroadcast-until-confirmed for free). Drop the
wallet_extract_owned_outputs() condition for funding-tx recording — it is about change
outputs, not about tx relevance. Same audit for the splice broadcast
(lightningd/channel_control.c:539).
3. Mempool monitoring for pending funding txs
While a channel is in CHANNELD_AWAITING_LOCKIN / DUALOPEND_AWAITING_LOCKIN, and for any
zero-conf channel still at funding_tx_status == UNKNOWN, poll our bitcoind's mempool for
the funding txid. On a hit: fetch the raw tx, wallet_transaction_add(tx, 0, 0), set
funding_tx_status = MEMPOOL, save, and kick channeld_zeroconf_funding_seen(). This
subsumes PR #9437's getutxout poll (which can stay as a cheap "did it get evicted /
reorged out?" backstop, but can no longer be the primary mechanism, since it cannot yield
the rawtx our invariant requires).
Polling cadence can piggyback on the existing chaintopology/bwatch poll loop; the set of
pending funding txids is small (typically 0–2).
Relevant bitcoind RPCs
| RPC | Purpose for us | Notes |
|---|---|---|
getrawmempool [verbosity] |
enumerate/set-diff the mempool against our pending funding txids | full-tx mode (verbosity=2) returns rawtxs in one roundtrip but is expensive; txid-only mode + targeted fetch is cheaper |
getmempoolentry <txid> |
cheap existence probe + fees/depends/ancestor* for one known txid |
RPC errors with "txn not in memory pool" if absent — treat as negative answer, not failure; also lets us detect underfunded/evicted fundings |
getrawtransaction <txid> |
fetch the rawtx of a txid we know is in the mempool | works for mempool txs without -txindex (the mempool is checked first); confirmed-tx lookups would need txindex — out of scope here |
getmempoolinfo |
mempool.minrelaytxfee, full flag |
detects "our tx was evicted because the mempool was full" → useful for fee-bumping a stuck funding tx |
testmempoolaccept |
pre-broadcast sanity / RBF checks | optional; not needed for the invariant |
No ZMQ (hashtransaction/hashtx) support exists in this tree today, so RPC polling is the
mechanism; a future ZMQ subscription could replace the poll with push notifications, but the
fetch-by-txid interface below stays the same either way.
Abstract methods to expose via bcli
Follow the existing pattern (plugins/bcli.c commands + bitcoind_* wrappers in
lightningd/bitcoind.h, with the _ macro fallback for backends that don't implement an
RPC — see bitcoind_getutxout_()):
bcli_getmempooltxids -> bitcoind_getmempooltxids(ctx, bc, cb, cbarg)
/* getrawmempool (verbosity 0): array of txids. Callback gets the set;
caller diffs against pending funding txids. */
bcli_getmempoolentry -> bitcoind_getmempoolentry(ctx, bc, txid, cb, cbarg)
/* getmempoolentry: cb(txid, bool present, fee, size, ancestors...).
cb(NULL ...) on "not in mempool" — a definitive negative, since we
poll after block-connect, see race note below. */
bcli_getmempooltx -> bitcoind_getmempooltx(ctx, bc, txid, cb, cbarg)
/* getrawtransaction <txid> (verbosity 0 / hex): cb(txid, wtx) or
cb(txid, NULL) if not found. This is the call that lets us satisfy
the "rawtx is stored" half of the invariant. */
bcli_getmempoolinfo -> bitcoind_getmempoolinfo(ctx, bc, cb, cbarg)
/* getmempoolinfo: size, full, minfeerate — eviction context. */
A cleaner long-term home may be bwatch rather than raw lightningd polling: add a
WATCH_MEMPOOL (or extend WATCH_OUTPOINT with a mempool tier) so that lightningd simply
gets watch_found with blockheight == 0 for mempool hits, and the poll lives in one plugin
next to the block scanner. The bcli methods above are identical either way; only the caller
moves.
Race note: between getmempoolentry saying "absent" and our next block fetch, the tx
may confirm without us having seen it in the mempool. The block-path recording from (§1) is
what closes that hole — mempool monitoring must be understood as best-effort early
detection, never the sole recording path.
Consistency with PR #9437
- The new
funding_tx_statuscolumn stays: it's the cheap, persisted answer to
"never broadcast / seen / confirmed" and gates zero-conf forwarding. This issue makes it
derivable from — rather than merely correlated with — the transactions table. - Where this PR's
zeroconf_utxo_checked()poll usesgetutxout, replace (or precede) it
withbitcoind_getmempooltx()so the same callback records the tx before flipping the
status. - Migration: for existing channels past lockin,
funding_tx_statusis backfilled to
CONFIRMED already (PR lightningd: track funding-tx broadcast visibility, gate zero-conf forwarding on it #9437); consider a companion backfill that at least records funding
txids of AWAITING_LOCKIN channels viagetmempoolentry+getrawtransactionat startup.
Downgrade / tooling
- No new tables; recording into both
transactionsandour_txsalready goes through
wallet_transaction_add(), solightning-downgradeis unaffected. - bcli: new RPCs must fail gracefully (backend probe via
check/getutxout-style
detection) so external backends lacking mempool access just disable the poll, as
getfilteredblockabsence does today.
Testing
tests/test_opening.py: zero-conf open where only the peer broadcasts (dev-feerate /
manual broadcast topology) → assertlisttransactionsshows the funding tx (height null)
before it confirms, on the non-funding side too.- Dual-open test asserting the funding tx appears in
listtransactionsimmediately after
openchannel_signed, including the no-change-output funding case. - Stuck-funding test: funding tx underfeued and evicted → assert we notice via
getmempoolinfo.full/getmempoolentryabsence and mark it appropriately.
Checklist
- Changelog entries in the relevant commits
- Tests added/modified
- Docs for the new
bcli_*RPCs (doc/) and backendchecknegotiation -
tools/lightning-downgradestill works (no new tables, but verify)
Activity