Entropy: measuring the unpredictable

To discuss chance seriously, we need to measure it. The tool for this is called entropy, and it is measured in bits.

The useful definition

The entropy of a value is the base-2 logarithm of the number of equally likely outcomes.

Trial	Possible outcomes	Entropy
A coin toss	2	1 bit
A six-sided die	6	2.58 bits
A random letter	26	4.7 bits

In other words: the number of yes-or-no questions needed to guess the value.

Equi-probability is essential

A 256-bit key only has 256 bits of entropy if all values are equally likely. A generator that produces only one million values offers barely 20 bits, regardless of the displayed length.

Remember the formula in this form: entropy measures what the attacker does not know, not the size of what you store.

The example of passwords

The calculation sheds light on a familiar debate:

Secret	Entropy
8 varied characters, chosen at random	approximately 52 bits
4 common words chosen at random	approximately 52 bits

The two are equivalent, even though the second is much easier to remember. This is the reasoning behind passphrases.

But be careful with the condition, which is systematically overlooked: chosen at random. Four words chosen by a human form a meaningful sentence, and the number of meaningful sentences is infinitely smaller than the number of possible combinations. The entropy plummets.

What the calculation doesn’t tell you

Entropy assumes an attacker who knows nothing about your method. If they know you’re selecting four words from a list of 7,776, they’ll calculate exactly the same number as you — and that’s perfectly fine.

The danger lies in the opposite scenario: believing you have 52 bits when the actual method only produces 20.