Optimization algorithms
Below you can find a number of optimization algorithms implemented in MATLAB. All implementation are inspired by things found on the internet. I do have to admit that I'm not an expert in the field of optimization. I am an engineer, and I use these algorithms to the problems I'm facing. I'm sure that the implementations can be improved, and I would be very happy to learn from your suggestions. So please contact me if you have any.
UPDATE: To illustrate how (well) the algorithms work, I have worked out an example in which the parameters of a dynamic model are estimated from experimental data. A zip file containing a the .m files is found here!
Simplex-Simulated annealing (SIMPSA)
Algorithms based on simulated annealing employ a stochastic generation of solution vectors and employ similarities between the physical process of annealing (i.e. melting a solid by heating it, followed by slow cooling and crystallization into a minimum free energy state) and a minimization problem. During the cooling process, transitions are accepted to occur from a low to a high energy level through a Boltzmann probability distribution. For an optimization problem, this corresponds to "wrong-way" movements. This implementation contains an algorithm that was described in Cardoso et al. (1996) and is based on the combination of the non-linear smplex and simulated annealing algorithms (the SIMPSA algorithm).
Download here.
Particle swarm optimization
Particle swarm optimization (PSO) is a population based stochastic optimization technique developed by Dr. Eberhart and Dr. Kennedy in 1995, inspired by social behavior of bird flocking or fish schooling. In PSO, each particle moves in the search space and updates its velocity according to best previous positions already found by himself and its neighbors, trying to find an even better position.
Download here.
Shuffled complex evolution
The SCE method is based on a synthesis of four concepts that have proved successful for global optimization: (a) combination of probabilistic and deterministic approaches; (b) clustering; (c) systematic evolution of a complex of points spanning the space, in the direction of global improvement; and (d) competitive evolution.
Download here.