TOPICS
Search

Rényi Entropy

Rényi entropy is defined as:

H_alpha(p_1,p_2,...,p_n)=1/(1-alpha)ln(sum_(i=1)^np_i^alpha),

where alpha>0, alpha!=1.

As alpha->1, H_alpha(p_1,p_2,...,p_n) converges to H(p_1,p_2,...,p_n), which is Shannon's measure of entropy.

Rényi's measure satisfies

H_alpha(p_1,p_2,...,p_n)<=H_(alpha^')(p_1,p_2,...,p_n)

for alpha<=alpha^'.

See also

Entropy

This entry contributed by Narayan L. Bhamidipati

Explore with Wolfram|Alpha

References

Karmeshu, J. (Ed.). Entropy Measures, Maximum Entropy Principle and Emerging Applications. New York: Springer-Verlag, 2003.Rényi, A. "On Measures of Entropy and Information." Proc. Fourth Berkeley Symp. Math. Stat. and Probability, Vol. 1. Berkeley, CA: University of California Press, pp. 547-561, 1961.

Referenced on Wolfram|Alpha

Rényi Entropy

Cite this as:

Bhamidipati, Narayan L. "Rényi Entropy." From MathWorld--A Wolfram Web Resource, created by Eric W. Weisstein. https://mathworld.wolfram.com/RenyiEntropy.html

Subject classifications