CompactSize

Bitcoin's binary formats carry no field names, so a parser has to be told how many inputs follow and how long each script is. CompactSize answers in as little as one byte: 0 to 252 is that byte alone, and larger values take a marker plus two, four or eight more. Every payment you send contains at least four of them.

How it works

CompactSize reads its own first byte to decide how much of itself is left.

A first byte below 0xfd is the value, so counts from 0 to 252 occupy one byte and nothing else. A first byte of 0xfd means the value sits in the next two bytes, 0xfe means the next four, and 0xff means the next eight. All of those follow-on numbers are little endian. The longest possible encoding is nine bytes and can express a value above 18 quintillion, which no real structure comes close to needing.

Encodings must be minimal. Writing the number 5 as 0xfd 0x05 0x00 is arithmetically identical to writing 0x05, and Bitcoin Core rejects it, raising a non-canonical error while deserializing. Without that rule the same transaction could be written several ways, each with a different identifier, which is a malleability problem rather than a tidiness one.

There is a practical ceiling far below the arithmetic one. Bitcoin Core refuses to allocate for any deserialized object claiming to be larger than 0x02000000 bytes, about 33.5 megabytes, and the peer protocol separately caps one message at 4,000,000 bytes. A hostile peer therefore cannot make your node reserve gigabytes of memory by announcing a transaction with a quintillion inputs.

One naming trap is worth flagging. The wire documentation calls this a var_int, wallet libraries usually call it a VarInt, and Bitcoin Core's own source uses the name VarInt for a completely different and unrelated encoding that appears only in its database and undo files. When reading code, check which of the two you are looking at.

Where you see it

CompactSize is the most frequently encountered structure in bitcoin that has no user facing name at all.

In a raw transaction it appears immediately after the four byte version, or after the segwit marker and flag when those are present, holding the input count. It shows up again before every script to give its length, and again before the output count. In a serialized block it carries the transaction count. In the peer protocol it opens the payload of inv, getdata, addr and headers messages.

The headers message shows the encoding at its strangest. Each 80 byte header is followed by a CompactSize transaction count of zero, so one entry occupies 81 bytes instead of 80, and a full batch of 2,000 headers comes to 162,000 bytes on the wire. The count is always zero because a headers message never carries transactions. The field exists so that the same parsing code can read a header whether or not a block body follows it.

If you decode a raw transaction by hand and the fields stop making sense partway through, a misread CompactSize is almost always the cause. Consume one byte too few and everything after it is shifted.