{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Definition\n\nLet $(X, \\mu)$ be a measure space and let $\\Theta \\subseteq \\R^n$ be open.\nA *statistical model* is a family $\\set{p(\\cdot\\,; \\theta) : \\theta \\in \\Theta}$ of positive [[probability densities|04YG690N5V]] on $X$ with respect to $\\mu$ such that:\n\n1. $\\theta \\mapsto p(x; \\theta)$ is smooth for every $x \\in X$, and derivatives in $\\theta$ may be taken under the integral sign;\n2. with $\\ell(x; \\theta) = \\log p(x; \\theta)$ and $\\partial_i = \\partial/\\partial\\theta^i$, the functions $\\partial_1 \\ell, \\dots, \\partial_n \\ell$ of $x$ are linearly independent for each $\\theta$.\n\nThe *Fisher information metric* is the [[Riemannian metric|Q61VFHCH2J]] on $\\Theta$ given by\n$$g_{ij}(\\theta) = \\mathbb{E}_\\theta\\left[\\partial_i \\ell \\; \\partial_j \\ell\\right],$$\nwhere $\\mathbb{E}_\\theta$ is the [[expected value|0DCT0QZVQ1]] for the density $p(\\cdot\\,; \\theta)$.\nFor a real number $\\alpha$, the *$\\alpha$-connection* is the [[affine connection|JX65KSVHSJ]] on $\\Theta$ with\n$$\\Gamma^{(\\alpha)}_{ij,k}(\\theta) = \\mathbb{E}_\\theta\\left[\\left(\\partial_i \\partial_j \\ell + \\frac{1 - \\alpha}{2}\\, \\partial_i \\ell \\; \\partial_j \\ell\\right) \\partial_k \\ell\\right],$$\nwhere $\\Gamma^{(\\alpha)}_{ij,k} = g\\left(\\nabla^{(\\alpha)}_{\\partial_i} \\partial_j, \\partial_k\\right)$.\n*Information geometry* is the study of a statistical model through its Fisher metric and its $\\alpha$-connections.\n\n## Remarks\n\nExample: the normal densities $p(x; \\mu, \\sigma) = (2\\pi\\sigma^2)^{-1/2} e^{-(x - \\mu)^2/(2\\sigma^2)}$ on $\\R$, with $\\theta = (\\mu, \\sigma)$ and $\\sigma \u003e 0$.\nHere $\\partial_\\mu \\ell = (x - \\mu)/\\sigma^2$ and $\\partial_\\sigma \\ell = (Z^2 - 1)/\\sigma$ with $Z = (x - \\mu)/\\sigma$ standard normal.\nSince $\\mathbb{E}[Z^2] = 1$, $\\mathbb{E}[Z^4] = 3$ and odd moments vanish, the metric is $(d\\mu^2 + 2\\, d\\sigma^2)/\\sigma^2$.\nThis is a hyperbolic plane of constant curvature $-1/2$.\nNon-example: the family $p(x; \\theta_1, \\theta_2)$ of normal densities with mean $\\theta_1 + \\theta_2$ and variance $1$ fails clause 2, since $\\partial_1 \\ell = \\partial_2 \\ell$; its Fisher matrix has all entries $1$ and is singular.\nThe $0$-connection is the [[Levi-Civita connection|CAZYGE1T4E]] of $g$.\nThe $1$-connection and the $(-1)$-connection are called the exponential and the mixture connection, and they are dual to each other with respect to $g$.\nThe Fisher metric is the Hessian at $\\theta' = \\theta$ of the [[Kullback–Leibler divergence|PQBW43X5RE]] $D(p_\\theta \\,\\|\\, p_{\\theta'})$.\n",
  "commit": "547b6eddb8f2feb8d579f2d633d22a27b7e132fc",
  "front": {
    "defines": "F64Z9541DV",
    "kind": "definition",
    "msc": [
      "53B12",
      "62B11"
    ],
    "name": "Information geometry",
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Information_geometry"
      }
    ],
    "tag": "FF2KNFNTB7",
    "uses": [
      "04YG690N5V",
      "0DCT0QZVQ1",
      "Q61VFHCH2J",
      "JX65KSVHSJ",
      "CAZYGE1T4E"
    ]
  },
  "kind": "definition",
  "link": "https://math2.tamnd.com/node/FF2KNFNTB7/",
  "links": [
    {
      "field": "defines",
      "target": "F64Z9541DV"
    },
    {
      "field": "uses",
      "target": "04YG690N5V"
    },
    {
      "field": "uses",
      "target": "0DCT0QZVQ1"
    },
    {
      "field": "uses",
      "target": "Q61VFHCH2J"
    },
    {
      "field": "uses",
      "target": "JX65KSVHSJ"
    },
    {
      "field": "uses",
      "target": "CAZYGE1T4E"
    }
  ],
  "markdown": "https://math2.tamnd.com/node/FF2KNFNTB7.md",
  "msc": [
    "53B12",
    "62B11"
  ],
  "name": "Information geometry",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/FF2/FF2KNFNTB7.md",
  "status": null,
  "tag": "FF2KNFNTB7",
  "uri": "m2://FF2KNFNTB7",
  "url": "https://math2.tamnd.com/wiki/information-geometry-definition/",
  "writing": {
    "tag": "FF2KNFNTB7",
    "kind": "definition",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}