Evolutionary and Metaheuristic Optimization
Introduction
What Question Does It Ask?
Evolutionary and metaheuristic optimization searches for a good solution to problems without derivatives or with huge search spaces through a loop of trying, selecting and varying. Inspired by processes such as natural selection, annealing of metals or ant colonies, these methods do not guarantee the optimum but find a good solution quickly.
At a glance
population-based heuristic search
objective, solution encoding
good, not proven optimal, solution
quality, evaluation count
low-medium
none; evaluations suffice
Introduction
A Short History
Holland publishes "Adaptation in Natural and Artificial Systems", laying the basis of genetic algorithms.
Kirkpatrick, Gelatt and Vecchi introduce simulated annealing.
Dorigo proposes ant colony optimization in his PhD thesis.
Kennedy and Eberhart publish particle swarm optimization (PSO).
Wolpert and Macready publish the "no free lunch" theorem: no method wins on all problems.
Deb and colleagues publish NSGA-II for multi-objective search.
The Course
Core Concepts
How a solution is encoded: bit string, permutation, real vector.
The objective value that says how good a solution is.
Giving good solutions more offspring: tournament, roulette.
Combining parts of two parents into a new solution.
Changing a solution by small random steps to add diversity.
Balance between searching new regions and refining a good one.
Hill climbing, simulated annealing, tabu search.
Solutions in a multi-objective problem that do not dominate each other.
The Course
A Typical Workflow
The Course
What Does the Course Cover?
- Optimization problems, search spaces, representations
- Local search: hill climbing, tabu search
- Simulated annealing and cooling schedules
- Genetic algorithms: selection, crossover, mutation
- Evolution strategies and CMA-ES
- Differential evolution
- Genetic programming
- Ant colony optimization
- Particle swarm optimization
- Constraint handling: penalties and repair
- Multi-objective search: Pareto, NSGA-II
- Experimental design and statistical comparison
Skills you will gain
A typical semester; the order and coverage vary between universities.
Try It
Evolve the Delivery Van's Route
The delivery van leaves the depot (D), visits 11 addresses and returns. A genetic algorithm improves a population of 40 tours by crossover and mutation. Advance generations, change the mutation rate and compare.
Context
Prerequisites
Courses worth knowing before this one, and why they are needed.
| Course | Why it is needed |
|---|---|
| Artificial Intelligence | heuristic search |
| Algorithms | complexity, NP-hard problems and greedy methods |
| Programming | arrays, random number generation and loops |
Context
Are You Ready for This Course?
Ask yourself; for every "no", the right-hand column shows where to start.
| Ask yourself | If not, start with |
|---|---|
| Can you state the time complexity of an algorithm in big-O notation? | Complexity and NP-hardness from an algorithms course |
| Can you find the minimum of a function by taking its derivative? | Optimization |
| Can you explain mean, standard deviation and a hypothesis test? | Probability and Statistics |
| Can you write a small Python program with arrays, loops and random numbers? | An intro programming course, then a short NumPy tutorial |
Context
Neighbouring Courses
Overlap
Overlapping Courses
| Relation | Course | Shared ground |
|---|---|---|
| subfield | This course is a subfield of Artificial Intelligence | heuristic search |
| shares methods | Shares methods with Artificial Neural Networks | neuroevolution, hyperparameter search |
| shares methods | Shares methods with Optimization | derivative-free and global optimization |
Differences
Where It Differs (1/2)
| Criterion | Evolutionary Optimization | Optimization | Artificial Intelligence |
|---|---|---|---|
| Core question | How do we find a good solution when there is no gradient or the search space is huge? | How do we find the values that make an objective function best? | How does a machine act intelligently by searching, reasoning and planning? |
| Approach | population-based heuristic search | mathematical model of objective and constraints | search, logic, probability and learning combined |
| Input | objective, solution encoding | objective function and constraints | problem definition: states, actions, goal |
| Output | good, not proven optimal, solution | optimal or near-optimal solution | action sequence, plan or decision |
| Success measure | quality, evaluation count | objective value, convergence rate | reaching the goal, cost, rationality |
| Math load | low-medium | the theory itself | medium: logic, probability, graphs |
| Data needs | none; evaluations suffice | none | little; often a model and knowledge suffice |
- Optimization: Optimization reaches a provable optimum using derivatives and convexity; this course teaches derivative-free search with no guarantee but wide applicability.
- Artificial Intelligence: Searches such as A* in AI find a path to a goal in a state space; this course searches directly for a good complete solution.
Differences
Where It Differs (2/2)
| Criterion | Evolutionary Optimization | Reinforcement Learning | Neural Networks |
|---|---|---|---|
| Core question | How do we find a good solution when there is no gradient or the search space is huge? | How can good decisions be learned by trial and error from a reward signal alone? | How do networks of simple units learn by adjusting their weights? |
| Approach | population-based heuristic search | trial and error with rewards | network of simple weighted units |
| Input | objective, solution encoding | environment: state, reward | numeric feature vector |
| Output | good, not proven optimal, solution | policy: state → action | class or numeric prediction |
| Success measure | quality, evaluation count | total expected reward | loss and accuracy |
| Math load | low-medium | high: probability, dynamic programming | medium-high: derivatives, matrices |
| Data needs | none; evaluations suffice | generated through interaction | moderate |
- Reinforcement Learning: RL learns a policy from states to actions by interacting with an environment; this course seeks a good solution for a single objective function.
- Neural Networks: Neural networks train weights with gradients; this course searches without gradients, for example evolving network structure or hyperparameters.
Practice
Applications
Shortening courier and delivery tours.
Timetables, production and shift plans.
Designing antenna, wing and structural shapes.
Finding settings for learning models.
Circuit component and facility placement.
Multi-objective trade-off between return and risk.
Practice
Risks and Ethics
Search finds and exploits flaws in the objective: the score rises but the intended behaviour never appears; evolution runs often show this.
A single run, unequal evaluation budgets or tuning only your own method make a new algorithm look better than it is.
An antenna or structure that looks great in simulation can fail in reality because of effects the model leaves out.
Running expensive simulations over thousands of generations can burn days of processor time and a lot of energy.
In depth: Explainable AI and AI Ethics.
Practice
Tools and Datasets
Tools and libraries
Classic benchmark problems
Practice
Common Misconceptions
| Common belief | In fact |
|---|---|
| A genetic algorithm finds the optimal solution. | Metaheuristics do not guarantee the optimum; they aim for a good solution in reasonable time. |
| Nature-inspired methods are more powerful. | The inspiration does not determine performance; many newly named methods are minor variants of older ones. |
| One method is best for every problem. | By the "no free lunch" theorem, no method is better on average over all problems. |
| Use evolutionary methods even when derivatives exist. | When derivatives are available, gradient methods usually reach the result with far fewer evaluations. |
Next Steps
After This Course
Courses that build on or extend this one
Related careers
Next Steps
Further Reading
- Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, 1989.
- Eiben and Smith, Introduction to Evolutionary Computing, 2nd ed., Springer, 2015.
- Talbi, Metaheuristics: From Design to Implementation, Wiley, 2009.
- Luke, Essentials of Metaheuristics, 2nd ed., Lulu, 2013.