Expectation, variance and sample size

Two formulas save long computations: those for the expectation and variance of a binomial distribution.

The formulas

X ~ B(n ; p)

E(X) = n p
V(X) = n p (1 - p)
σ(X) = √( n p (1-p) )

The expectation is reassuringly obvious: in 100 coin tosses one expects 100 × 0.5 = 50 heads.

Why it is true

The proof fits in one line, thanks to the linearity of expectation. Write X as a sum:

X = X1 + X2 + ... + Xn        where Xi is 1 if the i-th trial is a success

Each Xi is a Bernoulli variable with expectation p. Hence:

E(X) = E(X1) + ... + E(Xn) = p + p + ... + p = n p

No independence assumption is needed here — expectation always adds. For the variance, however, independence is essential:

V(X) = V(X1) + ... + V(Xn) = n · p(1-p)      (the trials being independent)

An example

A line produces 2 % defective parts. Fifty parts are inspected.

X ~ B(50 ; 0.02)

E(X) = 50 × 0.02 = 1 defective part on average
V(X) = 50 × 0.02 × 0.98 = 0.98
σ(X) = √0.98 ≈ 0.99

So one expects about 1 ± 1 defective part per batch of 50. Finding 5 defects would be 4σ from the mean: a serious signal that the machine has drifted.

The effect of sample size

This is the most important point for applications:

ABSOLUTE standard deviation : σ = √(np(1-p))        grows like √n
RELATIVE standard deviation : σ/n = √(p(1-p)/n)     shrinks like 1/√n

n = 100    ->  σ/n ≈ 5 %
n = 1,000  ->  σ/n ≈ 1.6 %
n = 10,000 ->  σ/n ≈ 0.5 %

In other words: the precision of a survey depends on the sample size, not on the size of the population. Polling 1000 people gives the same precision for a town of 50,000 as for a country of 60 million — a counter-intuitive result, but that is what the formula says.

And halving the margin of error requires quadrupling the sample. That is why surveys plateau in practice around 1000 people: beyond that, cost grows far faster than precision.

The normal approximation

When n is large, the binomial takes a bell shape that is advantageously replaced by a normal distribution with the same parameters:

B(n ; p)  ≈  N( np , np(1-p) )        for large n
                                       (in practice np ≥ 5 and n(1-p) ≥ 5)

   B(100 ; 0.5)                       the continuous bell fitting it
                                          ___
      ▁▂▃▅▇█▇▅▃▂▁              ≈       __/   \__
     40   50   60                     40   50   60

This is the subject of the course on the normal distribution, and the reason it appears everywhere.

Summary

• E(X) = np — obvious, and proved by the linearity of expectation.
• V(X) = np(1-p) — requires the trials to be independent.
• The absolute standard deviation grows as √n, the relative one shrinks as 1/√n.
• A survey's precision depends on n, not on the population size.
• Halving the margin of error requires quadrupling the sample.
• For large n, B(n ; p) is approximated by a normal distribution.