TOPICS
Search

Levenberg-Marquardt Method

Levenberg-Marquardt is a popular alternative to the Gauss-Newton method of finding the minimum of a function F(x) that is a sum of squares of nonlinear functions,

F(x)=1/2sum_(i=1)^m[f_i(x)]^2.

Let the Jacobian of f_i(x) be denoted J_i(x), then the Levenberg-Marquardt method searches in the direction given by the solution p to the equations

(J_k^(T)J_k+lambda_kI)p_k=-J_k^(T)f_k,

where lambda_k are nonnegative scalars and I is the identity matrix. The method has the nice property that, for some scalar Delta related to lambda_k, the vector p_k is the solution of the constrained subproblem of minimizing ||J_kp+f_k||_2^2/2 subject to ||p||_2<=Delta (Gill et al. 1981, p. 136).

The method is used by the command FindMinimum[f, {x, x0}] when given the Method -> LevenbergMarquardt option.

See also

Minimum, Optimization

Explore with Wolfram|Alpha

References

Bates, D. M. and Watts, D. G. Nonlinear Regression and Its Applications. New York: Wiley, 1988.Gill, P. R.; Murray, W.; and Wright, M. H. "The Levenberg-Marquardt Method." §4.7.3 in Practical Optimization. London: Academic Press, pp. 136-137, 1981.Levenberg, K. "A Method for the Solution of Certain Problems in Least Squares." Quart. Appl. Math. 2, 164-168, 1944.Marquardt, D. "An Algorithm for Least-Squares Estimation of Nonlinear Parameters." SIAM J. Appl. Math. 11, 431-441, 1963.

Referenced on Wolfram|Alpha

Levenberg-Marquardt Method

Cite this as:

Weisstein, Eric W. "Levenberg-Marquardt Method." From MathWorld--A Wolfram Web Resource. https://mathworld.wolfram.com/Levenberg-MarquardtMethod.html

Subject classifications