Compact block filter

One filter covers one block, and it lists output scripts rather than addresses, amounts or transaction ids. BIP-158 fixes the basic type at 0x00 with P of 19 and M of 784,931, so an element that is not in the block still matches about once in 784,931 tries. Because matching runs on your device, the node serving the filter never learns which coins are yours.

How it works

A compact block filter is assembled by hashing the output scripts a block creates and spends into one numeric range, sorting the results, and writing down only the gaps between them.

The item list is fixed by the specification, so two honest nodes produce byte-identical filters for the same block. For each transaction, the filter takes the previous output script being spent by each input, skipping the coinbase transaction, and the scriptPubKey of each output. One category is left out on purpose: outputs beginning with OP_RETURN. BIP-158 excludes them so that filters can later be committed to inside a coinbase OP_RETURN output, the way the witness commitment already is, without the commitment having to cover itself.

Hashing uses SipHash with c set to 2 and d set to 4, keyed on the first 16 bytes of the block hash in its standard little-endian form. That key changes with every block, so the same script lands somewhere different in each one and nobody can build a lookup table that works chain-wide. Each 64-bit output is then mapped into the range zero to N times M with a multiply and shift rather than a division.

Compression is what the name points at: sorted values are replaced by their successive differences, and each difference becomes a quotient in unary followed by 19 remainder bits, which is Golomb-Rice coding. The output cannot be smaller than N times 20 bits, and it ships with the item count N in front as a CompactSize; a block whose filter has no elements is stored as a single byte of zeroes. A node willing to serve these filters advertises the service bit NODE_COMPACT_FILTERS, defined as 1 shifted left by 6.

Where you see it

Compact block filters are the sync method chosen by wallets that would rather spend bandwidth than hand an address list to somebody's server.

Wasabi Wallet is the example in our own reviews: desktop only, bitcoin only, every connection over Tor by default, and syncing from filters so that no server learns which addresses belong to you. We rate it 3.6 and mark it caution, for a reason that has nothing to do with syncing. Electrum, rated 4.4, takes the other road: it verifies by SPV against an indexing server anybody can run, and that server ends up holding the list of addresses you own.

Two limits are worth knowing before you lean on filters. A filter is keyed on scripts, so you cannot look a transaction id up in one: with only a txid you still need a node or an explorer. And OP_RETURN outputs are absent by construction, so software that watches for data-carrying outputs cannot be built on this mechanism at all.

Compact block filter vs compact block relay

A compact block filter tells a wallet whether a block is worth downloading; compact block relay, specified in BIP152, shrinks a block already on its way between two full nodes.

The audiences are different. Filters are per-block, deterministic, keyed on the first 16 bytes of the block hash, and pulled down by clients that hold no chain of their own. Relay replaces most of a block's transactions with 48-bit identifiers keyed from the block header hashed together with a nonce the sender chooses, and it saves nothing for a receiver whose mempool does not already hold those transactions. One saves a phone from downloading blocks it does not need; the other saves a node from receiving transactions twice. The shared word "compact" is the whole of the resemblance.