Probability theory and stochastic processes Proof

Proof of the Chernoff bound

Builds on Markov's inequality Moment-generating function The exponential function is strictly increasing The exponential function is positive

Proof

We apply Markov's inequality to $e^{tX}$ for each $t > 0$ .

Let $X$ be a real random variable with moment-generating function $M_X(t) = E[e^{tX}] \in (0, \infty]$ . Let $a \in \R$ .
Fix $t > 0$ . The map $x \mapsto tx$ is increasing, and the exponential function is strictly increasing. So $x \ge a$ holds if and only if $e^{tx} \ge e^{ta}$ .
Hence $\set{X \ge a} = \set{e^{tX} \ge e^{ta}}$ as events.
The exponential function is positive, so $e^{tX}$ takes values in $[0, \infty)$ and $e^{ta} > 0$ .
By Markov's inequality for $e^{tX}$ and the level $e^{ta}$ , $P(e^{tX} \ge e^{ta}) \le E[e^{tX}]/e^{ta}$ . This holds in $[0, \infty]$ , also when $M_X(t) = \infty$ .
By steps 3 and 5, $P(X \ge a) \le e^{-ta} M_X(t)$ for each $t > 0$ .
So $P(X \ge a)$ is a lower bound of the set $\set{e^{-ta} M_X(t) : t > 0}$ . It is therefore at most the infimum of this set.
Thus $P(X \ge a) \le \inf_{t > 0} e^{-ta} M_X(t)$ .

Idea

For $t > 0$ the map $x \mapsto e^{tx}$ is increasing, so $X \ge a$ exactly when $e^{tX} \ge e^{ta}$ . Markov's inequality bounds this event for every $t$ . Then one picks the best $t$ .

Reading path

To understand this page from the ground up, read these 274 pages in order. The foundations come first. The path takes the proofs Proof of the Cauchy–Hadamard theorem, Proof of the Weierstrass M-test, Proof of the limit of a sum of sequences, Proof of Rolle's theorem, Proof of the approximation property of the supremum, Proof that every sequence has a monotone subsequence, by peaks, Proof of the formula for the derivative of a polynomial, Proof of the commutativity of addition by induction, Proof of term-by-term differentiation, Proof of the epsilon-delta form of continuity, Proof of the power rule by induction, Proof that absolutely convergent series converge, Proof of the sum of a finite geometric series, Proof that convergent sequences are Cauchy, Proof that there is a natural number n with 1/n below any positive epsilon, Proof of the chain rule with an auxiliary difference quotient, Proof of the probability of the complement, Proof of the Cauchy criterion for series, Proof of the Cauchy criterion for sequences by the Bolzano–Weierstrass theorem, Proof that the exponential series has infinite radius of convergence, Proof that the reals form an ordered field, with the Cauchy sequence construction, Proof of the root test, Proof of the double negation law by a truth table, Proof that power series converge uniformly on smaller intervals inside the radius of convergence, Proof of the extreme value theorem by the Bolzano–Weierstrass theorem, Proof that the exponential function is positive, Proof of the uniqueness of limits, Proof of strong induction from ordinary induction, Proof that power series are differentiable inside the radius of convergence, Proof of the limit of a product of sequences, Proof that uniform convergence of derivatives allows term-by-term differentiation, Proof that the absolute value is multiplicative, Proof of the squeeze theorem for sequences, Proof of Pascal's rule from the recursive definition, Proof of the linearity of differentiation, Proof of the monotonicity of probability, Proof of the sequential characterization of limits, Proof that the exponential function is strictly increasing, Proof that Cauchy sequences are bounded, Proof of the triangle inequality for real numbers, Proof of the epsilon-N definition of the limit of a sequence, Proof of the comparison test, Proof of Fermat's theorem on stationary points, Proof that limits preserve non-strict inequalities, Proof of the factorial formula for binomial coefficients by induction, Proof that a function with positive derivative is strictly increasing, Proof of the binomial theorem by induction, Proof that exponentials dominate powers in sequences, by the ratio of consecutive terms, Proof that differentiable functions are continuous, Proof of the epsilon-delta form of the limit of a function, Proof of the well-ordering principle from strong induction, Proof of the Bolzano–Weierstrass theorem by a monotone subsequence, Proof of the Archimedean property in an Archimedean ordered field, Proof that a composition of continuous functions is continuous, Proof of the exponential of a sum by the Cauchy product, Proof that a decreasing sequence bounded below converges, Proof of the term test, Proof that the exponential function is its own derivative, by term-by-term differentiation, Proof that the reals are Archimedean from the completeness axiom, Proof of the ratio test, Proof that convergent sequences are bounded, Proof of the product rule, Proof of the monotone convergence theorem for sequences, Proof that continuity is equivalent to sequential continuity, Proof of the boundedness theorem by Bolzano–Weierstrass, Proof of Markov's inequality, Proof of the ratio formula for the radius of convergence, Proof that a series of nonnegative terms converges iff its partial sums are bounded, Proof of the mean value theorem from Rolle's theorem, Proof that the Cauchy product of absolutely convergent series converges absolutely to the product, Proof of the constant multiple rule.

Binomial coefficient definition
Construction of the reals from Cauchy sequences concept
Definition by recursion concept
Element definition
Factorial definition
Group concept
Integers object
Limit point of a set definition
Local maximum definition
Mathematical induction concept
Natural numbers object
Ordered field definition
Peano axioms concept
Proposition concept
Random experiment concept
Rational numbers object
Real numbers object
Sequence definition
Set bounded above definition
Strictly monotone function definition
Proof of Pascal's rule from the recursive definition proof
Set definition
Ring definition
Ring of integers statement
Limit of a function definition
Local minimum definition
Archimedean ordered field definition
Successor function definition
Conjunction definition
Disjunction definition
Implication definition
Negation definition
Predicate definition
The reals form a field statement
Real sequence definition
Subsequence definition
Infimum definition
Supremum definition
Pascal's rule statement
Ordered pair definition
Commutative ring definition
Continuity at a point definition
Derivative definition
One-sided limit definition
Union definition
Addition of natural numbers definition
Sample space definition
Proof of the Archimedean property in an Archimedean ordered field proof
Biconditional definition
The rationals form a field statement
The successor function is injective statement
Truth table concept
Zero is not a successor statement
Modus ponens concept
Existential quantifier definition
Universal quantifier definition
Bounded sequence definition
Cauchy sequence definition
Geometric sequence definition
Limit of a sequence definition
Monotone sequence definition
Sequence tending to infinity definition
Series definition
Completeness axiom statement
Cartesian product definition
Proof of the factorial formula for binomial coefficients by induction proof
Continuous function definition
Differentiable function definition
One-sided derivative definition
Archimedean property statement
Proof of the commutativity of addition by induction proof
Subset definition
Order on the natural numbers definition
Propositional logic concept
The rationals form an ordered field statement
Mathematical proof concept
Uniqueness quantifier definition
Limit superior definition
Proof of the sum of a finite geometric series proof
Convergent sequence definition
Pointwise convergence definition
Sequential continuity definition
The sequence r^n tends to infinity for r > 1 example
Cauchy product definition
Partial sum definition
Power series definition
Factorial formula for binomial coefficients statement
Proof that a composition of continuous functions is continuous proof
Binary relation definition
Universal set definition
Order on the reals definition
Power set definition
Commutativity of addition statement
Proof of strong induction from ordinary induction proof
Tautology definition
Uniform convergence definition
Sum of a finite geometric series statement
Radius of convergence definition
Convergent series definition
Proof of the binomial theorem by induction proof
Antisymmetric relation definition
Reflexive relation definition
Transitive relation definition
Interval definition
Complement definition
Function definition
Strong induction statement
Uniform convergence on a set definition
Logical equivalence definition
Sum of a series definition
Series of functions definition
Absolute convergence definition
Binomial theorem statement
Preorder definition
Proof of the epsilon-delta form of continuity proof
Proof of the epsilon-delta form of the limit of a function proof
Sigma-algebra definition
Injective function definition
Indicator function definition
Preimage of a set definition
Proof of the well-ordering principle from strong induction proof
Proof of the double negation law by a truth table proof
Partial order definition
Epsilon-delta definition of continuity statement
Epsilon-delta definition of a limit statement
Countable set definition
Event definition
Borel sigma-algebra definition
Well-ordering principle statement
Double negation law statement
Total order definition
Composition of continuous functions is continuous statement
Probability axioms concept
Proof by contradiction concept
Proof that every sequence has a monotone subsequence, by peaks proof
Probability measure definition
Proof that the reals form an ordered field, with the Cauchy sequence construction proof
Every sequence has a monotone subsequence statement
Probability distribution definition
Probability space definition
The reals form an ordered field statement
Absolute value definition
Random variable definition
Proof of the approximation property of the supremum proof
Cumulative distribution function definition
Law of a random variable definition
Proof of the triangle inequality for real numbers proof
Proof that the absolute value is multiplicative proof
Discrete random variable definition
Random variable with a density definition
Proof of the epsilon-N definition of the limit of a sequence proof
Approximation property of the supremum statement
Triangle inequality statement
Absolute value is multiplicative statement
Probability density function definition
Probability mass function definition
Expected value definition
Proof that the reals are Archimedean from the completeness axiom proof
Epsilon-N definition of the limit of a sequence statement
Proof that differentiable functions are continuous proof
Law of the unconscious statistician statement
Proof of the limit of a sum of sequences proof
Proof of the squeeze theorem for sequences proof
Proof of the uniqueness of limits proof
Proof that limits preserve non-strict inequalities proof
Linearity of expectation statement
The reals are Archimedean statement
Proof that convergent sequences are bounded proof
Proof that convergent sequences are Cauchy proof
Proof that Cauchy sequences are bounded proof
Proof of the term test proof
The sequence 1/n tends to zero example
Proof of the monotone convergence theorem for sequences proof
Differentiable functions are continuous statement
Moment definition
Proof of the probability of the complement proof
Proof of Markov's inequality proof
Proof that there is a natural number n with 1/n below any positive epsilon proof
Limit of a sum of sequences statement
Proof of the ratio formula for the radius of convergence proof
Uniqueness of limits statement
Limits preserve non-strict inequalities statement
Convergent sequences are bounded statement
Squeeze theorem statement
Convergent sequences are Cauchy statement
Cauchy sequences are bounded statement
Monotone convergence theorem for sequences statement
Moment-generating function definition
Probability of the complement statement
Proof of the limit of a product of sequences proof
There is a natural number n with 1/n below any positive epsilon statement
Term test statement
Proof that a series of nonnegative terms converges iff its partial sums are bounded proof
Proof that a decreasing sequence bounded below converges proof
The sequence r^n tends to zero for |r| < 1 example
Proof of the monotonicity of probability proof
Proof of the sequential characterization of limits proof
Proof that continuity is equivalent to sequential continuity proof
Decreasing sequence bounded below converges statement
Limit of a product of sequences statement
Geometric series example
A series of nonnegative terms converges iff its partial sums are bounded statement
Monotonicity of probability statement
Continuity is equivalent to sequential continuity statement
Sequential characterization of limits statement
Proof that the Cauchy product of absolutely convergent series converges absolutely to the product proof
Proof of the root test proof
Proof of the Cauchy–Hadamard theorem proof
Proof that exponentials dominate powers in sequences, by the ratio of consecutive terms proof
Proof of the ratio test proof
Proof of the Bolzano–Weierstrass theorem by a monotone subsequence proof
Proof of the comparison test proof
Markov's inequality statement
Proof of the chain rule with an auxiliary difference quotient proof
Proof of the product rule proof
Proof of the linearity of differentiation proof
Comparison test statement
Cauchy product of absolutely convergent series statement
Bolzano–Weierstrass theorem statement
Exponentials dominate powers in sequences statement
Chain rule statement
Product rule statement
Linearity of differentiation statement
Ratio test statement
Root test statement
Proof of the boundedness theorem by Bolzano–Weierstrass proof
Proof of the Cauchy criterion for sequences by the Bolzano–Weierstrass theorem proof
Proof of the constant multiple rule proof
Proof of the power rule by induction proof
Boundedness theorem statement
Ratio formula for the radius of convergence statement
Cauchy–Hadamard theorem statement
Cauchy criterion for sequences statement
Constant multiple rule statement
Power rule statement
Proof of the extreme value theorem by the Bolzano–Weierstrass theorem proof
Uniformly Cauchy sequence of functions definition
Proof of the Cauchy criterion for series proof
Extreme value theorem statement
Proof of the formula for the derivative of a polynomial proof
Proof of Fermat's theorem on stationary points proof
Cauchy criterion for series statement
Derivative of a polynomial statement
Fermat's theorem on stationary points statement
Proof that absolutely convergent series converge proof
Proof of Rolle's theorem proof
Absolutely convergent series converge statement
Rolle's theorem statement
Proof of the Weierstrass M-test proof
Proof of the mean value theorem from Rolle's theorem proof
Weierstrass M-test statement
Mean value theorem statement
Proof that power series converge uniformly on smaller intervals inside the radius of convergence proof
Proof that uniform convergence of derivatives allows term-by-term differentiation proof
Proof that a function with positive derivative is strictly increasing proof
Power series converge uniformly on smaller discs inside the radius of convergence statement
Uniform convergence of derivatives allows term-by-term differentiation statement
A function with positive derivative is strictly increasing statement
Proof that the exponential series has infinite radius of convergence proof
The exponential series has infinite radius of convergence statement
Proof of term-by-term differentiation proof
Term-by-term differentiation statement
Exponential function definition
Proof of the exponential of a sum by the Cauchy product proof
Proof that power series are differentiable inside the radius of convergence proof
Exponential of a sum statement
Power series are differentiable inside the radius of convergence statement
Proof that the exponential function is positive proof
Proof that the exponential function is its own derivative, by term-by-term differentiation proof
The exponential function is positive statement
The exponential function is its own derivative statement
Proof that the exponential function is strictly increasing proof
The exponential function is strictly increasing statement
Proof of the Chernoff bound proof

Where this leads

6 pages build on this one.

Tag	`DN38VXHCF2`
URI	`m2://DN38VXHCF2`
Kind	proof
Writing	W0 stub
MSC	60E15
Proves	Chernoff bound
Uses	Markov's inequality, Moment-generating function, The exponential function is strictly increasing, The exponential function is positive
Changed	4 November 2026

Proof of the Chernoff bound

Proof

Idea

Reading path

Where this leads

Further reading

What this page needs for W1

What links here