Covariance provides a measure of the strength of the correlation between two or more sets of random variates. The covariance for two random variates and , each with sample size , is defined by the expectation value
(1)
 
(2)

where and are the respective means, which can be written out explicitly as
(3)

For uncorrelated variates,
(4)

so the covariance is zero. However, if the variables are correlated in some way, then their covariance will be nonzero. In fact, if , then tends to increase as increases, and if , then tends to decrease as increases. Note that while statistically independent variables are always uncorrelated, the converse is not necessarily true.
In the special case of ,
(5)
 
(6)

so the covariance reduces to the usual variance . This motivates the use of the symbol , which then provides a consistent way of denoting the variance as , where is the standard deviation.
The derived quantity
(7)
 
(8)

is called statistical correlation of and .
The covariance is especially useful when looking at the variance of the sum of two random variates, since
(9)

The covariance is symmetric by definition since
(10)

Given random variates denoted , ..., , the covariance of and is defined by
(11)
 
(12)

where and are the means of and , respectively. The matrix of the quantities is called the covariance matrix.
The covariance obeys the identities
(13)
 
(14)
 
(15)
 
(16)

By induction, it therefore follows that
(17)
 
(18)
 
(19)
 
(20)
 
(21)
