Ordinary least squares is a method for estimating the coefficient vector in a linear regression model by minimizing the unweighted
sum of squared residuals. For a design matrix 
and response vector 
, the estimate 
minimizes
 |
It therefore satisfies the normal equations
 |
When the columns of 
are linearly independent,
the estimate is unique. In contrast, weighted and generalized least squares modify
the residual norm to account for unequal variances or correlated errors.
