{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Statement\n\nLet $X_1, \\dots, X_n$ be [[independent|0GQ938GDE0]] real random variables with $E[X_i] = 0$ for each $i$.\nAssume that there is a constant $M \u003e 0$ with $\\abs{X_i} \\le M$ almost surely for each $i$.\nLet $\\sigma^2 = \\sum_{i=1}^n E[X_i^2]$.\nThen for every $t \\ge 0$,\n$$P\\left(\\sum_{i=1}^n X_i \\ge t\\right) \\le \\exp\\left(-\\frac{t^2 / 2}{\\sigma^2 + M t / 3}\\right).$$\n\n## Remarks\n\nProof idea: for $\\lambda \u003e 0$, the bound $\\abs{X_i} \\le M$ gives $E[e^{\\lambda X_i}] \\le \\exp\\left(E[X_i^2] \\, \\frac{e^{\\lambda M} - 1 - \\lambda M}{M^2}\\right)$.\nThe [[Chernoff bound|014PEBGDYF]] with the best $\\lambda$ then gives Bennett's inequality, and an elementary estimate of the resulting function gives the bound above.\nApplying the bound to $-X_i$ gives the same bound for the lower tail, so $P(\\abs{\\sum_i X_i} \\ge t)$ is at most twice the right-hand side.\nFor small $t$, the exponent is about $-t^2 / (2 \\sigma^2)$, as for a normal variable with the true variance $\\sigma^2$.\nFor large $t$, it is about $-3t / (2M)$, an exponential tail.\nExample: take $n$ independent centred variables with $\\abs{X_i} \\le 1$ and $E[X_i^2] = p$ small.\n[[Hoeffding's inequality|0WPDNXGSSN]] gives the exponent $-t^2 / (2n)$.\nBernstein gives about $-t^2 / (2np)$ when $t$ is much smaller than $np$, which is much better.\nThe name covers a family of related bounds; other versions replace the bound $\\abs{X_i} \\le M$ by moment conditions $E[\\abs{X_i}^k] \\le \\frac{k!}{2} E[X_i^2] M^{k-2}$ for all $k \\ge 2$.\n\n## Explanations\n\n### L1 human\n\nA sum of many small independent random pieces is rarely far from its mean.\nBernstein's inequality says how rare, and it uses the actual variance of the pieces, not only their range.\nWhen the pieces have small variance, it gives much stronger bounds than inequalities that see only the range.\n\n### L2 human\n\nBernstein's inequality sits between [[Hoeffding's inequality|0WPDNXGSSN]], which uses only bounds on the range, and the [[central limit theorem|0EPZC4Z1FC]], which describes the normal approximation but gives no bound for finite $n$.\nIts tail is Gaussian for moderate deviations and exponential for large ones, which matches the behaviour of sub-exponential variables.\nIt is a standard tool in statistics, randomized algorithms and the theory of empirical processes.\n",
  "commit": "5fc425f97f1f6b11da5892c2948ef8c5d8805665",
  "front": {
    "about": [
      "0GQ938GDE0"
    ],
    "kind": "statement",
    "msc": [
      "60E15",
      "60F10"
    ],
    "name": "Bernstein inequalities",
    "role": "theorem",
    "same_as": {
      "wikidata": "Q826791",
      "wikipedia": "Bernstein inequalities (probability theory)"
    },
    "sources": [
      {
        "url": "https://en.wikipedia.org/wiki/Bernstein_inequalities_(probability_theory)"
      }
    ],
    "tag": "MX6157RAVA",
    "uses": [
      "0GQ938GDE0",
      "0WQNRCNY3M",
      "0477R9PYAE",
      "014PEBGDYF"
    ]
  },
  "kind": "statement",
  "link": "https://math2.tamnd.com/node/MX6157RAVA/",
  "links": [
    {
      "field": "about",
      "target": "0GQ938GDE0"
    },
    {
      "field": "uses",
      "target": "0GQ938GDE0"
    },
    {
      "field": "uses",
      "target": "0WQNRCNY3M"
    },
    {
      "field": "uses",
      "target": "0477R9PYAE"
    },
    {
      "field": "uses",
      "target": "014PEBGDYF"
    }
  ],
  "markdown": "https://math2.tamnd.com/node/MX6157RAVA.md",
  "msc": [
    "60E15",
    "60F10"
  ],
  "name": "Bernstein inequalities",
  "prov": null,
  "role": "theorem",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/MX6/MX6157RAVA.md",
  "status": null,
  "tag": "MX6157RAVA",
  "uri": "m2://MX6157RAVA",
  "url": "https://math2.tamnd.com/wiki/bernstein-inequalities/",
  "writing": {
    "tag": "MX6157RAVA",
    "kind": "statement",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      },
      {
        "what": "a source with a locator",
        "have": 0,
        "want": 1
      }
    ]
  }
}