Entropy

Entropy is a measure of the disorder or unpredictability of a source: the higher it is, the harder the result produced is to predict. It does not describe an isolated event, but the amount of uncertainty a source holds before its result is observed.

A useful image is to compare two questions that should be answered yes or no. A question whose answer is already known brings no information: the uncertainty is zero. A question whose two answers are equally plausible keeps the uncertainty at its maximum. Entropy measures precisely this volume of the unknown, that is, how many questions would have to be asked to remove the doubt.

The principle holds on simple examples. A perfectly balanced coin has more entropy than a rigged coin that almost always lands on the same side, because its result is harder to anticipate. A twenty-sided die has more than a coin, because it offers far more equally possible outcomes. At the extreme, a totally predictable source, whose result is known in advance, has zero entropy.

A widespread confusion is to equate entropy with simply a large number of results. What matters is not only the number of outcomes, but their balance. A loaded twenty-sided die that almost always lands on the same number offers, despite its many faces, far less entropy than a plain honest coin, because its behavior is in reality much more predictable.

In computing, this notion is concrete and decisive. To produce quality chance, a generator needs to be initialized from a source of real entropy, unpredictable and hard to guess. It is this raw material of uncertainty that makes the difference between a solidly unpredictable draw and a sequence that could be reconstructed, and therefore what guarantees the fairness of the site's chance tools.