Engel's theorem

Remarks

Part 1 is proved by induction on $\dim \mathfrak g$. A maximal proper subalgebra $\mathfrak h$ of $\mathfrak g$ turns out to be an ideal of codimension $1$, so $\mathfrak g = \mathfrak h + kx$. By induction the vectors killed by $\mathfrak h$ form a nonzero subspace $W$, and $x$ maps $W$ to itself; since $x$ is nilpotent, it kills some nonzero vector of $W$. Part 2 follows by applying part 1 to the image of the adjoint representation in $\mathfrak{gl}(\mathfrak g)$ and inducting on the dimension through $\mathfrak g / Z(\mathfrak g)$.

It is not enough that a set of matrices consists of nilpotent ones. In the special linear Lie algebra $\mathfrak{sl}_2$, the matrices $e = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$ and $f = \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}$ are nilpotent, but no nonzero vector is killed by both. The subalgebra they generate contains $[e, f] = \operatorname{diag}(1, -1)$, which is not nilpotent, so part 1 does not apply. Also, in part 1 it is not enough that $\mathfrak g$ is a nilpotent Lie algebra. The one-dimensional algebra spanned by the identity of $V$ is abelian, yet it kills no nonzero vector.

Going deeper

For a student

The theorem is one of the first structural results in the theory of Lie algebras, and it holds over every field. It gives the practical test for nilpotence: check that each $\operatorname{ad} x$ is nilpotent. It is used in the proof of Lie's theorem and in the theory of Cartan subalgebras, which are nilpotent by definition. It also underlies the Fitting decomposition of a module over a nilpotent Lie algebra. For a finite-dimensional solvable Lie algebra $\mathfrak g$, it shows that $[\mathfrak g, \mathfrak g]$ is nilpotent in characteristic $0$, by combining with Lie's theorem.