Expectation: the long-run average

The expected value is the average value a random variable takes when the experiment is repeated a great many times. It is the first number computed from a distribution.

The definition

$$E(X) = x_1 \cdot P(X = x_1) + x_2 \cdot P(X = x_2) + \dots + x_n \cdot P(X = x_n)$$

In other words: each value is weighted by its probability. It is an average, but one where the more likely values weigh more.

For a fair die:

$$E(X) = 1 \times \tfrac{1}{6} + 2 \times \tfrac{1}{6} + 3 \times \tfrac{1}{6} + 4 \times \tfrac{1}{6} + 5 \times \tfrac{1}{6} + 6 \times \tfrac{1}{6} = \frac{1 + 2 + 3 + 4 + 5 + 6}{6} = \frac{21}{6} = 3.5$$

What the expected value is not

$E(X) = 3.5$ although the die never lands on $3.5$. The expected value is not an expected result in the everyday sense: it is the long-run average.

Number of rolls	Observed average
$10$	$3.9$
$100$	$3.42$
$10\,000$	$3.504$
$1\,000\,000$	$3.4998$

The observed average tightens around $3.5$. The law of large numbers guarantees this convergence. The expected value is therefore the forecast of what is collected on average per round, over a great many rounds.

Fair game, losing game

Expected value	Reading
$E(X) = 0$	fair game: neither gain nor loss in the long run
$E(X) > 0$	game favourable to the player
$E(X) < 0$	game unfavourable to the player

Take the game from the previous lesson: €2 stake, €9 on a 6.

$$E(X) = (-2) \times \tfrac{5}{6} + 7 \times \tfrac{1}{6} = \frac{-10 + 7}{6} = -\frac{3}{6} = -0.50$$

That is 50 cents lost per round. The player loses 50 cents per round on average. Over 1000 rounds that is about €500 lost — and it is the business model of every gambling game: an expected value slightly negative for the player, hence positive for the operator.

To make this game fair, the payout $g$ would have to satisfy:

$$(-2) \times \tfrac{5}{6} + (g - 2) \times \tfrac{1}{6} = 0 \quad \Longrightarrow \quad g = 12$$

so a payout of €12 would be needed.

Linearity

This is the property that avoids recomputing everything:

$$E(aX + b) = a \cdot E(X) + b$$

$$E(X + Y) = E(X) + E(Y) \quad \text{(always true, even if } X \text{ and } Y \text{ are related)}$$

For instance, if $E(X) = 3.5$, then $E(10X - 4) = 10 \times 3.5 - 4 = 31$.

The second formula is especially powerful: the expected value of a sum is always the sum of the expected values, with no independence assumption. That is what makes the expected value of a binomial distribution immediate, as we shall see.

Summary

• $E(X) = \sum x_i \cdot P(X = x_i)$: each value weighted by its probability.
• It is a long-run average, not a value $X$ must reach.
• The law of large numbers justifies this reading.
• $E(X) = 0$ defines a fair game; $E(X) < 0$, an unfavourable one.
• Linearity: $E(aX + b) = aE(X) + b$ and $E(X + Y) = E(X) + E(Y)$, always.