Method of moments

Summary

The method of moments is a way to build estimators of unknown parameters. Suppose that the law of an i.i.d. sample $X_1, \dots, X_n$ depends on $k$ real parameters $\theta = (\theta_1, \dots, \theta_k)$, and that the moments $\mu_j(\theta) = E_\theta[X_1^j]$ exist for $j \le k$. The method replaces each $\mu_j(\theta)$ with the sample moment $\hat{m}_j = \frac{1}{n} \sum_{i=1}^{n} X_i^j$, and solves the $k$ equations $\mu_j(\hat{\theta}) = \hat{m}_j$ for $\hat{\theta}$.

Remarks

Example: for $X_i \sim \operatorname{Exp}(\lambda)$, $E[X_1] = 1/\lambda$, so the method gives $\hat{\lambda} = 1/\bar{X}_n$. Example: for $X_i \sim \mathcal{N}(\mu, \sigma^2)$, it gives $\hat{\mu} = \bar{X}_n$ and $\hat{\sigma}^2 = \frac{1}{n}\sum_i (X_i - \bar{X}_n)^2$. If the map $\theta \mapsto (\mu_1(\theta), \dots, \mu_k(\theta))$ has a continuous inverse, the law of large numbers makes the estimators consistent. The estimators are often simple, but they are usually less efficient than maximum likelihood estimators.

Explanations

L1 human

The method of moments chooses the parameters so that the model has the same averages as the data. For example, if the data have mean $2$, an exponential model with mean $1/\lambda$ gets the rate $\lambda = 1/2$.