TOPICS
Search

Chernoff Bound


The Chernoff bound is an exponential upper bound on the tail probability of a real random variable X. Let D_+={t>0:E[e^(tX)]<infty} be nonempty. For any real number a, Markov's inequality applied to e^(tX) gives

 P(X>=a)<=inf_(t in D_+){e^(-ta)E[e^(tX)]},

where E[Y] denotes the expectation value of Y (Durrett 2019, p. 106). Taking the infimum gives the best bound within this family, even if no minimizing parameter exists. Only a finite positive exponential moment is needed. Finiteness on a two-sided neighborhood of zero, used for existence of a moment-generating function, is a stronger requirement.

Applying the same argument to -X gives a lower tail probability bound when the corresponding negative exponential moment is finite. For a sum of independent random variables with the required exponential moments finite, the exponential expectation value factors into a product. If the variables have finite expectation values, the same argument applied to their centered versions bounds deviations from the expectation value of the sum.


See also

Expectation Value, Independent Statistics, Markov's Inequality, Moment-Generating Function, Random Variable, Tail Probability

Explore with Wolfram|Alpha

References

Chernoff, H. "A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the Sum of Observations." Ann. Math. Stat. 23, 493-507, 1952. https://doi.org/10.1214/aoms/1177729330.Durrett, R. Probability: Theory and Examples. Version 5, Jan. 11, 2019. Author manuscript. https://services.math.duke.edu/~rtd/PTE/PTE5_011119.pdf.

Cite this as:

Weisstein, Eric W. "Chernoff Bound." From MathWorld--A Wolfram Resource. https://mathworld.wolfram.com/ChernoffBound.html

Subject classifications