{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Statement\n\nLet $a \u003c b$ and let $(X_t)_{t \\in [a, b]}$ be a real [[stochastic process|0QH5ENKZM1]] such that the map $(t, \\omega) \\mapsto X_t(\\omega)$ is jointly measurable, $E[X_t] = 0$ and $E[X_t^2] \u003c \\infty$ for every $t$.\nAssume that the [[covariance|0KBFAWYRTE]] function $K(s, t) = E[X_s X_t]$ is continuous on $[a, b]^2$.\nLet $T_K$ be the integral operator on $L^2[a, b]$ given by $(T_K f)(s) = \\int_a^b K(s, t) f(t) \\, dt$.\nLet $(\\lambda_k)_{k \\ge 1}$ be its non-zero [[eigenvalues|0BQAA2BK3Q]], listed with multiplicity, and let $(e_k)$ be corresponding [[orthonormal|0S9GGTR0EX]] eigenfunctions.\nThen:\n\n1. each $\\lambda_k$ is positive and each $e_k$ is continuous;\n2. the random variables $Z_k = \\int_a^b X_t \\, e_k(t) \\, dt$ satisfy $E[Z_k] = 0$ and $E[Z_j Z_k] = \\lambda_k$ if $j = k$ and $0$ otherwise;\n3. $X_t = \\sum_{k} Z_k e_k(t)$, where the series converges in mean square uniformly in $t$:\n$$\\lim_{n \\to \\infty} \\sup_{t \\in [a, b]} E\\left[\\left(X_t - \\sum_{k=1}^{n} Z_k e_k(t)\\right)^2\\right] = 0.$$\n\nIf $X$ is moreover a Gaussian process, the $Z_k$ are independent normal random variables.\n\n## Remarks\n\nThe proof rests on Mercer's theorem: $K$ is continuous, symmetric and positive semidefinite, so $K(s, t) = \\sum_k \\lambda_k e_k(s) e_k(t)$ with uniform convergence on $[a, b]^2$.\nFubini's theorem gives the moments of the $Z_k$ and $E[X_t Z_k] = \\lambda_k e_k(t)$.\nExpanding the square then gives\n$$E\\left[\\left(X_t - \\sum_{k \\le n} Z_k e_k(t)\\right)^2\\right] = K(t, t) - \\sum_{k \\le n} \\lambda_k e_k(t)^2,$$\nwhich tends to $0$ uniformly in $t$ by Mercer's theorem.\n\nExample: for the standard [[Wiener process|22735FGJV6]] on $[0, 1]$, $K(s, t) = \\min(s, t)$.\nSolving the eigenvalue equation gives $e_k(t) = \\sqrt{2} \\sin\\left((k - \\tfrac{1}{2})\\pi t\\right)$ and $\\lambda_k = \\dfrac{1}{(k - \\frac{1}{2})^2 \\pi^2}$ for $k \\ge 1$.\nSo $W_t = \\sum_{k \\ge 1} \\sqrt{2} \\, \\xi_k \\dfrac{\\sin\\left((k - \\frac{1}{2})\\pi t\\right)}{(k - \\frac{1}{2})\\pi}$ with i.i.d. standard normal $\\xi_k$.\n\n## Explanations\n\n### L1 human\n\nA random curve can be written as a sum of fixed shapes multiplied by random coefficients.\nThe Karhunen–Loève theorem gives the best such shapes for a given process: the eigenfunctions of its covariance.\nWith this choice, the random coefficients are uncorrelated, and the first few terms capture as much of the variance as possible.\n\n### L2 human\n\nThe expansion is the continuous analogue of principal component analysis, which diagonalises a covariance matrix.\nTruncating it after $n$ terms gives the best approximation by $n$ terms for the mean square error integrated over $[a, b]$, among all expansions in an orthonormal basis of $L^2[a, b]$.\nIt is used to simulate Gaussian processes, to reduce dimension in signal processing and uncertainty quantification, and to compute laws of quadratic functionals such as $\\int_0^1 W_t^2 \\, dt = \\sum_k \\lambda_k \\xi_k^2$.\n",
  "commit": "547b6eddb8f2feb8d579f2d633d22a27b7e132fc",
  "front": {
    "about": [
      "0QH5ENKZM1",
      "0KBFAWYRTE"
    ],
    "kind": "statement",
    "msc": [
      "60G12",
      "60G15"
    ],
    "name": "Karhunen–Loève theorem",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q2046647",
      "wikipedia": "Kosambi–Karhunen–Loève theorem"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Kosambi%E2%80%93Karhunen%E2%80%93Lo%C3%A8ve_theorem"
      }
    ],
    "tag": "5CN7KY8MWP",
    "uses": [
      "0QH5ENKZM1",
      "0KBFAWYRTE",
      "0BQAA2BK3Q",
      "0S9GGTR0EX",
      "0WQNRCNY3M"
    ]
  },
  "kind": "statement",
  "link": "https://math2.tamnd.com/node/5CN7KY8MWP/",
  "links": [
    {
      "field": "about",
      "target": "0QH5ENKZM1"
    },
    {
      "field": "about",
      "target": "0KBFAWYRTE"
    },
    {
      "field": "uses",
      "target": "0QH5ENKZM1"
    },
    {
      "field": "uses",
      "target": "0KBFAWYRTE"
    },
    {
      "field": "uses",
      "target": "0BQAA2BK3Q"
    },
    {
      "field": "uses",
      "target": "0S9GGTR0EX"
    },
    {
      "field": "uses",
      "target": "0WQNRCNY3M"
    }
  ],
  "markdown": "https://math2.tamnd.com/node/5CN7KY8MWP.md",
  "msc": [
    "60G12",
    "60G15"
  ],
  "name": "Karhunen–Loève theorem",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/5CN/5CN7KY8MWP.md",
  "status": null,
  "tag": "5CN7KY8MWP",
  "uri": "m2://5CN7KY8MWP",
  "url": "https://math2.tamnd.com/wiki/karhunen-loeve-theorem/",
  "writing": {
    "tag": "5CN7KY8MWP",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}