Entropy is a term for several numerical measures of uncertainty, information, disorder, or dynamical complexity whose precise definition depends on context. In information
theory, the entropy of a discrete random variable
is its Shannon entropy. Related quantities include
conditional entropy, cross
entropy, and relative entropy, while Rényi
entropy, Tsallis entropy, min-entropy,
and max-entropy give generalizations or extremal
variants. The continuous analogue of Shannon entropy is differential
entropy.
Dynamical notions include Kolmogorov entropy and topological entropy (Ott 1993), while
von Neumann entropy is a matrix entropy used
in quantum information.
See also
Conditional Entropy,
Cross Entropy,
Differential Entropy,
Information
Theory,
Joint Entropy,
Kolmogorov
Entropy,
Max-Entropy,
Maximum
Entropy Method,
Metric Entropy,
Min-Entropy,
Mutual Information,
Nat,
Ornstein's Theorem,
Redundancy,
Relative Entropy,
Rényi
Entropy,
Shannon Entropy,
Topological
Entropy,
Tsallis Entropy,
von
Neumann Entropy
Explore with Wolfram|Alpha
References
Lasota, A. and Mackey, M. C. Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, 2nd ed. New York: Springer-Verlag,
1994.Ott, E. "Entropies." §4.5 in Chaos
in Dynamical Systems. New York: Cambridge University Press, pp. 138-144,
1993.Referenced on Wolfram|Alpha
Entropy
Cite this as:
Weisstein, Eric W. "Entropy." From MathWorld--A Wolfram Resource. https://mathworld.wolfram.com/Entropy.html
Subject classifications
