The joint entropy of discrete random variables 
, for 
, is the Shannon entropy
of their joint probability distribution,
 |
For two random variables, this gives
 |
In particular, 
.
When 
and 
are finite, equality holds precisely
when 
and 
are independent.
