Type I and Type II Errors

You can be wrong in both directions

A hypothesis test makes a decision, but it can be wrong. Since the decision relies on a sample subject to chance, two symmetric errors are possible, and it is essential to distinguish them.

• A type I error (false positive): rejecting H0 when it was true. We believe we have found an effect that does not exist.
• A type II error (false negative): failing to reject H0 when it was false. We miss a real effect.

The table of four cases

Decision	REALITY: H0 true	REALITY: H0 false
we do NOT reject H0	correct (true neg.)	type II error (false neg.)
we reject H0	type I error (false pos.)	correct (true pos.)

(the two diagonal cells are the right decisions; the other two are the errors)

An unavoidable trade-off

The threshold alpha directly controls the risk of a type I error: setting it at 5% guarantees that we wrongly reject H0 in at most 5% of the cases where it is true. But lowering alpha (being more demanding before rejecting) mechanically increases the type II risk: we miss more real effects. We cannot reduce both errors at once without increasing the sample size.

Choosing according to the stakes

The right threshold depends on the cost of each error. For screening for a serious disease, a false negative (missing a sick person) is much worse than a false positive (a needless alarm cleared up by a follow-up exam): we then accept more type I errors to limit type II errors. In court, on the contrary, we prefer to limit false positives (convicting an innocent person), even if it means letting some guilty people go free.

Classic pitfall

Confusing the two errors is common. One way to remember: the type I error is the "crying wolf" error (raising the alarm for nothing), the type II error is the "missed wolf" error (the real danger goes unnoticed). The test threshold only moves the cursor between these two risks; it does not eliminate them.