The method of steepest descent is an algorithm for finding the nearest local minimum of a function
whose gradient can be computed. Also called the gradient
descent method, it starts at a point 
and, as many times as needed, moves from 
to 
by minimizing along the line extending from 
in the direction of 
, the local downhill gradient.
When applied to a 1-dimensional function 
, the method takes the form of iterating
 |
from a starting point 
for some small 
until a fixed point is reached. The results are
illustrated above for the function 
with 
and starting points 
and 0.01, respectively.
This method has the severe drawback of requiring a great many iterations for functions which have long, narrow valley structures. In such cases, a conjugate gradient method is preferable.
