Harmonic analysis on homogeneous spaces

Remarks

An invariant measure exists, for example, when $G$ and $H$ are both unimodular, or when $H$ is compact; see invariant measure on a homogeneous space.

Example: for $G = \R$ and $H = 2\pi\Z$, the space $X$ is the circle. Then $\lambda_X$ is the direct sum of the characters $x \mapsto e^{i n x}$, $n \in \Z$, each with multiplicity one, and the expansion is the classical Fourier series.

Example: for $G = \mathrm{SO}(3)$ and $H = \mathrm{SO}(2)$, $L^2(S^2)$ is the direct sum of the spaces of spherical harmonics of degree $\ell \ge 0$, each irreducible and occurring once.

Non-example: let $G$ be the affine group $\set{x \mapsto a x + b : a > 0, \ b \in \R}$ and $H$ the stabilizer $\set{x \mapsto a x : a > 0}$ of $0$. Then $G / H \cong \R$, but there is no nonzero $G$-invariant Radon measure on $\R$: invariance under translations forces a multiple of Lebesgue measure, and dilations change Lebesgue measure. So the first condition fails, and the formula above does not give a unitary representation.