TOPICS
Search

Rényi Entropy

The Rényi entropy of a discrete probability distribution (p_1,p_2,...,p_n) is

H_alpha(p_1,p_2,...,p_n)=1/(1-alpha)ln(sum_(i=1)^np_i^alpha),
(1)

where alpha>0, alpha!=1.

As alpha->1, Rényi entropy converges to Shannon entropy,

lim_(alpha->1)H_alpha(p_1,p_2,...,p_n)=H(p_1,p_2,...,p_n).
(2)

The limits as alpha->0 and alpha->infty give max-entropy and min-entropy, respectively.

Rényi's measure satisfies

H_alpha(p_1,p_2,...,p_n)>=H_(alpha^')(p_1,p_2,...,p_n)
(3)

for alpha<=alpha^'.

See also

Entropy, Max-Entropy, Min-Entropy, Shannon Entropy, Tsallis Entropy

Portions of this entry contributed by Narayan L. Bhamidipati

Explore with Wolfram|Alpha

References

Karmeshu, J. (Ed.). Entropy Measures, Maximum Entropy Principle and Emerging Applications. New York: Springer-Verlag, 2003.Rényi, A. "On Measures of Entropy and Information." Proc. Fourth Berkeley Symp. Math. Stat. and Probability, Vol. 1. Berkeley, CA: University of California Press, pp. 547-561, 1961.

Referenced on Wolfram|Alpha

Rényi Entropy

Cite this as:

Weisstein, Eric W., with contributions by Narayan L. Bhamidipati. "Rényi Entropy." From MathWorld--A Wolfram Resource. https://mathworld.wolfram.com/RenyiEntropy.html

Subject classifications