Estimating a Population from a Sample

The Confidence Interval

Giving a range rather than a single number

Announcing "the proportion in favor is 47%" is misleading: it is just an estimate, subject to uncertainty. We prefer to give a confidence interval: a range that has a good chance of containing the true value, for example "between 44% and 50%, at the 95% confidence level".

What "95%" really means

The confidence level (often 95%) describes the method, not a probability about a specific range already computed. It means: if we repeated the poll a large number of times, about 95% of the intervals constructed this way would contain the true value.

True value p (fixed, unknown):               |
                                             v
poll 1  :           [-----------|-----------]      (contains p)
poll 2  :                 [-----------|-----------] (contains p)
poll 3  :      [-----------]|                       (misses p! -> ~5% of cases)
poll 4  :             [--------|--------]            (contains p)

(over many polls, ~95% of the intervals catch the true value p)

A simple order of magnitude

For a proportion and a sample of size n, a widely used approximation gives an interval half-width of about 1 over the square root of n at the 95% level. For n = 1000, that is about 0.03, i.e. ±3 points: this is the famous "margin of error" of polls. For n = 100, the margin rises to ±10 points, much coarser.

Width and trade-off

A wider interval is more cautious (more likely to contain the truth) but less informative. A narrower interval is more precise but riskier. Increasing the sample size is the only way to tighten the interval without lowering the confidence level.

Classic pitfall