Matrix calculus extends differentiation to functions whose arguments or values are vectors
or matrices. For a scalar-valued function , its first derivative
is represented by the gradient
. For a vector-valued function
, the first derivative is the Jacobian
The second derivative of a scalar-valued function is its Hessian.
For a function of a matrix , the derivative
can similarly be collected from the partial derivatives
with respect to the entries
. Authors use both numerator-layout
and denominator-layout conventions, which transpose some vector and matrix derivatives.
A matrix-calculus formula therefore requires its derivative convention to be stated
explicitly.