From discrete to continuous

The normal distribution and its parameters

Among all possible densities, one keeps coming back: the bell curve of the normal distribution, also called the Gaussian or Laplace-Gauss distribution.

The curve

   f(x)
     |                 __
     |               _/  \_                 symmetric about μ
     |             _/      \_
     |          __/          \__            decays very fast in the tails
     |     ____/                \____
     +----|-------|-------|-------|----> x
        μ-2σ     μ-σ  μ  μ+σ    μ+2σ

It is entirely described by two parameters:

μ  (mu)     the expectation      ->  the POSITION of the peak
σ  (sigma)  the standard deviation -> the WIDTH of the bell

We write X ~ N(μ , σ²).

small σ                     large σ
   |                          |
   |   /\                     |     ___
   |  /  \                    |   _/   \_
   | /    \                   | _/       \_
   +--------                  +--------------
 sharp and narrow           flat and spread out

Changing μ translates the curve without deforming it; changing σ narrows or widens it, the area always staying equal to 1.

Two useful visual landmarks: the peak is at x = μ, and the inflection points — where the curve stops hollowing and starts flaring — are exactly at μ - σ and μ + σ. That is the graphical reading of the standard deviation.

The three-sigma rule

This is the result to remember above all, and it holds for every normal distribution, whatever μ and σ:

    μ-3σ   μ-2σ   μ-σ    μ    μ+σ   μ+2σ   μ+3σ
      |      |     |     |     |      |      |
      |------|-----|-----|-----|------|------|
                   <----68 %---->
            <---------95 %--------->
      <-------------99.7 %--------------->

P(μ - σ  ≤ X ≤ μ + σ)  ≈ 68 %       (68.27 %)
P(μ - 2σ ≤ X ≤ μ + 2σ) ≈ 95 %       (95.45 %)
P(μ - 3σ ≤ X ≤ μ + 3σ) ≈ 99.7 %     (99.73 %)

In other words: a value beyond three standard deviations is exceptional — three cases in a thousand.

An example

IQ is calibrated to follow a normal distribution with mean 100 and standard deviation 15.

   between 85 and 115  (μ ± σ)   ->  about 68 % of the population
   between 70 and 130  (μ ± 2σ)  ->  about 95 %
   above 145           (μ + 3σ)  ->  about 0.13 %, one person in 740

This immediate reading is what makes the normal distribution so convenient: a deviation is interpreted in "number of sigmas", regardless of unit or context.

What symmetry gives for free

P(X ≤ μ) = P(X ≥ μ) = 0.5              the mean is also the median
P(X ≥ μ + a) = P(X ≤ μ - a)            the two tails are identical

So if 95 % of values lie in μ ± 2σ, the remaining 5 % split equally: 2.5 % above, 2.5 % below.

Summary