An algorithm for finding the nearest local minimum of a function which presupposes that the gradient
of the function can be computed. The method of steepest descent, also called the
gradient descent method, starts at a point 
and, as many times as needed, moves from 
to 
by minimizing along the line extending from 
in the direction of 
, the local downhill gradient.
When applied to a 1-dimensional function 
, the method takes the form of iterating
 |
from a starting point 
for some small 
until a fixed point is reached. The results are
illustrated above for the function 
with 
and starting points 
and 0.01, respectively.
This method has the severe drawback of requiring a great many iterations for functions which have long, narrow valley structures. In such cases, a conjugate gradient method is preferable.
