The cross entropy of discrete probability distributions 
and 
on the same countable set 
, for 
, is
 |
where a term is infinite when 
and 
. In terms of the Shannon
entropy 
and relative entropy 
,
 |
When 
is finite, it therefore equals 
exactly when 
.
