Differential evolution is a stochastic parallel direct search evolution strategyoptimization method that is fairly
fast and reasonably robust. Differential evolution is implemented in the Wolfram
Language as NMinimize[f,
vars, Method -> "DifferentialEvolution"] and NMaximize[f,
vars, Method -> "DifferentialEvolution"].

Differential evolution is capable of handling nondifferentiable, nonlinear and multimodal objective functions. It has been used to train neural networks having real and constrained integer weights.

In a population of potential solutions within an -dimensional search space, a fixed number of vectors are randomly
initialized, then evolved over time to explore the search space and to locate the
minima of the objective function.

At each iteration, called a generation, new vectors are generated by the combination of vectors randomly chosen from the current population (mutation). The outcoming vectors are then mixed with a predetermined target vector. This operation is called recombination and produces the trial vector. Finally, the trial vector is accepted for the next generation if and only if it yields a reduction in the value of the objective function. This last operator is referred to as a selection.

Ilonen, J.; Kamarainen, J. K.; and Lampinen, J. "Differential Evolution Training Algorithm for Feed Forward Neural Networks." Neurol. Proc.
Lett.17, 93-105, 2003.Plagianakos, V. P. and Vrahatis,
M. N. "Parallel Evolutionary Training Algorithms for Hardware-Friendly
Neural Networks." Natural Comp.1, 307-322, 2002.Price,
K. and Storn, R. "Differential Evolution." Dr. Dobb's J., Issue
264, 18-24 and 78, Apr. 1997.Storn, R. "System Design by Constraint
Adaptation and Differential Evolution." IEEE Trans. Evol. Comput.3,
22-34, 1999.Storn, R. and Price, K. "Differential Evolution: A
Simple and Efficient Adaptive Scheme for Global Optimization over Continuous Spaces."
J. Global Optimization11, 341-359, 1997.