TOPICS
Search

Conditional Mutual Information

Conditional mutual information measures the dependence between random variables X and Y after another variable Z is revealed. For discrete variables, using natural logarithms, it is

I(X;Y|Z)=sum_(x,y,z)p(x,y,z)ln(p(x,y|z))/(p(x|z)p(y|z)).

Terms with zero joint probability contribute zero. For general variables on standard Borel spaces, the definition is the average relative entropy between the conditional joint law and the product of its conditional marginals. When the conditional entropies are finite, the discrete identity is I(X;Y|Z)=H(X|Z)-H(X|Y,Z), with the same logarithm base.

See also

Conditional Entropy, Mutual Information, Relative Entropy, Shannon Entropy

Explore with Wolfram|Alpha

References

Gray, R. M. Entropy and Information Theory, 1st ed., corrected. June 26, 2023. https://ee.stanford.edu/~gray/it.pdf.

Cite this as:

Weisstein, Eric W. "Conditional Mutual Information." From MathWorld--A Wolfram Resource. https://mathworld.wolfram.com/ConditionalMutualInformation.html

Subject classifications