Matrix calculus extends differentiation to functions whose arguments or values are vectors
or matrices. For a scalar-valued
function 
,
its first derivative is represented by the gradient 
. For a vector-valued
function 
,
the first derivative is the Jacobian
 |
The second derivative of a scalar-valued function is its Hessian.
For a function of a matrix 
, the derivative
can similarly be collected from the partial derivatives
with respect to the entries 
. Authors use both numerator-layout and denominator-layout
conventions, which transpose some vector and matrix derivatives. A matrix-calculus
formula therefore requires its derivative convention to be stated explicitly.
