Merkle block

BIP37 let a wallet hand a node a filter and receive only what matched, wrapped in a proof that nothing was fabricated. The design failed on privacy and on load: Bitcoin Core switched the feature off by default in version 0.19.0, released in November 2019. If you run a modern light wallet, it is almost certainly using compact block filters instead.

How it works

Status: BIP37 bloom filtering, and the merkleblock message that serves it, was disabled by default in Bitcoin Core 0.19.0 in November 2019 on privacy and denial of service grounds. Compact block filters, specified in BIP157 and BIP158, replaced it, and Bitcoin Core has been able to serve those to peers since version 0.21.0 in January 2021.

A merkle block is a proof, not a summary. Its structure has four parts: the block header, the total number of transactions in the block, an ordered list of hashes, and a compact bit field of flags. The flags describe a depth first walk through the block's merkle tree, telling the receiving wallet whether to descend into a branch or to take the next hash from the list as a stand-in for everything underneath it. The wallet replays that walk, arrives at a root, and compares it with the root in the header it was handed.

That comparison is the entire point. A hostile peer can hide transactions from you by simply not mentioning them, and no filtered protocol can prevent that. What it cannot do is invent a transaction you never received, because the fabricated hash would not reconstruct to a root matching a header with real proof of work behind it.

Requesting one is a two step exchange. The wallet uploads a filter with a filterload message, then asks for a block using an inventory type of filtered block rather than the ordinary one. The merkle block comes back first, followed by the full matching transactions as separate messages. A block with 2,000 transactions and one match costs the wallet a header, about a dozen hashes, and one transaction, instead of a megabyte or more.

Where you see it

Merkle blocks belong to a generation of mobile wallets that has largely retired.

The BitcoinJ library popularised them, and the Android wallets built on it spent most of the 2010s syncing this way. Anyone running an old build of one of those apps may still be issuing filterload messages and quietly failing to find a peer that will answer, since serving requires an operator to have set peerbloomfilters back on by hand.

What replaced them inverts the direction of the request. With compact block filters the node publishes a small filter per block describing what that block contains, and the wallet downloads filters, decides locally which blocks are interesting, and fetches those whole. Nothing about the wallet's addresses leaves the device. That is the model behind lnd's Neutrino backend and the mobile Lightning wallets built on it.

The privacy failure is worth stating plainly, because it is the reason for the switch. Work published in 2014 by Arthur Gervais and co-authors showed that a node serving a bloom filtered client could recover the large majority of that client's addresses, and that requesting two blocks with the same filter leaked more still. The padding of false positives that was supposed to hide the wallet did not hide it.

Merkle block vs bloom filter

The bloom filter is what a wallet sends up; the merkle block is what comes back down. A filter is a probabilistic, lossy description of the addresses and outpoints somebody cares about, and it lives on the serving node. A merkle block is a deterministic proof about one specific block, and it lives in the reply. They were deprecated as a pair, but only the filter was the privacy problem. Merkle proofs themselves are still fine, which is why compact block filters kept the proof idea and threw away the part where you tell a stranger what you own.