Type I and type II errors

Definition

Consider a hypothesis test of $H_0 \colon \theta \in \Theta_0$ against $H_1 \colon \theta \in \Theta_1$ that rejects $H_0$ when the data $X$ are in a rejection region $R$. A type I error occurs when $H_0$ is true and the test rejects $H_0$. Its probability at $\theta \in \Theta_0$ is $P_\theta(X \in R)$. A type II error occurs when $H_0$ is false and the test does not reject $H_0$. Its probability at $\theta \in \Theta_1$ is $\beta(\theta) = P_\theta(X \notin R)$.

Remarks

The size of the test is $\alpha = \sup_{\theta \in \Theta_0} P_\theta(X \in R)$. The power of the test at $\theta \in \Theta_1$ is $1 - \beta(\theta)$. For a fixed sample size, a smaller rejection region lowers the probability of a type I error. It raises the probability of a type II error. In the frequentist practice, one fixes a bound $\alpha$ on the type I error, and then chooses a test with a small type II error.

Explanations

L1 human

A type I error is a false alarm: the test says "effect" when there is none. A type II error is a miss: the test says "no effect" when there is one. For a medical test, a positive result for a healthy person is a type I error. A negative result for an ill person is a type II error.