Lecture 16 - Genetic Algorithms

Question 1

What are Genetic Algorithms?

Answer 1

• Inspired by Darwinian evolution and modern genetics.
• Core operators:
  ○ Mutation
  ○ Crossover (recombination/mating)
  ○ Selection

Question 2

What are the Key Differences from Evolution Strategies (ES)?

Answer 2

• GA focuses on individual vectors as chromosomes/genomes, traditionally in binary encoding.
• Crossover is essential, not just mutation.
• Various methods for parent selection, often probabilistic.

Question 3

What are the Similarities to (µ,λ) ES?

Answer 3

They are similar to (µ,λ) ES, but
- focus on individual vector as chromosome/genome
○ traditionally binary encoding not real/floating point (but of course… )
- crossover is a fundamental operation (not only mutation)
- many approaches to parent selection (sometimes probabilistic)

Question 4

How do Genetic Algorithms Work?

Answer 4

• Start with a bunch of random solutions.
• Keep the best one seen so far.
• Select good parents.
• Mix and mutate them to create new ones.
• Repeat — improving each generation.

Question 5

Genetic Algorithm Code

Answer 5

REFER TO SLIDES

Question 6

What are the Key Differences from (μ, λ) Evolutionary Strategy?

Answer 6

• Uses crossover as a core operator (vs. just mutation)
• Parents are selected probabilistically, not just top μ
• Creates a new population from parent pairs rather than selecting best individuals directly
• Follows a “generate and replace” model using selection → crossover → mutation

Question 7

What is Initiasation?

Answer 7

• “Before we can evolve anything, we need a starting population — and that starts with initialising individuals.
• Explains how to generate random binary vectors, which are the most common encoding in basic GAs.
• But it also reminds us that binary is not the only option — GAs can use other representations (e.g., trees, real-valued vectors, or even chemical compounds!).

Question 8

Initialisation Code - why use binary?

Answer 8

REFER TO SLIDES
- Using binary vectors (remember there are other ways to do this)
- Why use binary?
○ Easy to mutate (flip bits)
○ Simple crossover logic
○ Common in theoretical work

Question 9

Mutation Code

Answer 9

REFER TO SLIDES

Question 10

What is One-Point Crossover?

Answer 10

One-point crossover is a method of combining two parent chromosomes to produce offspring by swapping parts of their genetic information after a randomly selected crossover point.
- Imagine two binary strings (chromosomes). We pick a random position in the middle, and we swap everything after that point between the two parents. This creates two new children that are combinations of both parents.

Question 11

One-Point Crossover Code

Answer 11

REFER TO SLIDES

Question 12

What are the Disadvantages of One-Point Crossover?

Answer 12

REFER TO SLIDES - NEED FORMULAS TO EXPLAIN

Question 13

What is Bias Recombination?

Answer 13

“The algorithm is biased against combining genes that are far apart. This can slow down convergence or stop the GA from finding optimal combinations.”
That’s why this slide notes:
- “We’re biasing against solutions where distant genes work well together.”

Question 14

What is Two-Point Crossover?

Answer 14

• Two-point crossover is a genetic operator that takes two parent chromosomes and swaps the segment between two randomly chosen points. It’s more balanced than one-point crossover and can better preserve building blocks of genes — even if they’re far apart

Question 15

Two-Point Crossover Code

Answer 15

REFER TO SLIDES

Question 16

Two-Point Crossover - Further Details Through Visualisation

Answer 16

REFER TO SLIDES

Question 17

What is Uniform Crossover?

Answer 17

• Uniform crossover decides whether to swap genes at each position individually, instead of cutting the chromosome at fixed points. It gives every gene position an equal, independent chance to be swapped.”
  ○ This removes the positional bias seen in one-point and two-point crossover, and provides maximum mixing potential.

Question 18

Uniform Crossover Code

Answer 18

REFER TO SLIDES

Question 19

Why is Crossover is Not Mutation?

Answer 19

• Crossover recombines existing values — it does not invent new values like mutation can.
  ○ Core issue: If the initial population lacks diversity, crossover alone can’t fix it. You may never explore some parts of the solution space.
• Expands the issue to geometry:
  ○ Binary chromosomes of length l sit on the corners of an l-dimensional hypercube. Crossover only moves you around that existing space — it cannot escape it.
  ○ Dangers:
  * If all individuals have the same bit in one position, crossover can’t restore variation there.
  * That dimension is collapsed, and permanently lost.
  * Eventually, everyone converges to a single solution — even if it’s not optimal.
  ○ This leads to:
  * Premature convergence
  * Loss of diversity
  * Reliance on mutation to restore variability

Question 20

So why do we use Crossover?

Answer 20

• Just because its in “biology” doesn’t mean it makes for the best computer algorithm, so more justification is needed
  
  • Argument for Crossover:
    ○ It spreads good building blocks (short gene patterns that work well).
    ○ This idea is supported by:
    * Empirical evidence (some success in practice)
    * Schema Theory (a theoretical model explaining how gene patterns evolve)
    ○ But…
    * Schema theory is complex and hard to generalize
    * It’s not fully satisfying in explaining why crossover works reliably

Question 21

Why does Crossover rely on the assumption of Epistasis?

Answer 21

Linkage is key:
* One-point crossover: depends on position
* Two-point crossover: depends on distance
* Uniform crossover: depends on probability p

Question 22

What is Selection?

Answer 22

• “Selection is the process of choosing which individuals in the population will become parents and produce offspring for the next generation.”
  ○ Selection influences how fit individuals are rewarded and how diversity is maintained.
• Recall truncation selection in ES’s

REFER TO SLIDES ABOUT THIS
- SelectWithReplacement is open to a wide range of options

Question 23

What is Fitness-Proportionate Selection?

Answer 23

• Original and common approach
  ○ Also known as roulette selection
  * “Roulette selection is a method where individuals are chosen to reproduce with a probability proportional to their fitness.
  * It’s like spinning a weighted roulette wheel — the more fit an individual is, the bigger their slice of the wheel, and the more likely they are to be selected.”
  ====
• Select individuals in proportion to their fitness
  ○ “If an individual has twice the fitness of another, it should be twice as likely to be chosen.”

Question 24

Fitness-Proportionate Selection Code

Answer 24

REFER TO SLIDES

Question 25

What is the time for Fitness-Proportionate Selection?

Answer 25

○ “This algorithm runs in O(n) time for each selection — but with binary search on the CDF, it can be improved to O(log n).”

Question 26

What is Stochastic Universal Sampling?

Answer 26

- “Stochastic Universal Sampling is a low-variance selection method that still chooses individuals based on fitness, but does so in a more evenly distributed way. ○ It helps reduce randomness and makes the selection process fairer and more consistent.”

Question 27

How are pointers spaced in Stochastic Universal Sampling?

Answer 27

How Pointers Are Spaced * Total range = s (sum of all fitness) * Number of individuals to choose = n * Each pointer is spaced s/n apart * First pointer starts randomly in the range [0, s/n] “This ensures coverage of the whole fitness space, not just the parts with the highest fitness.”

Question 28

What are the Advantages of Stochastic Universal Sampling

Answer 28

* Low variance resampling * Advantage: Reduces randomness in the selection process compared to methods like roulette wheel selection. * Leads to more stable and consistent selection of individuals over generations. * Guarantees selection of individuals with fi≥snf_i \geq \frac{s}{n}fi​≥ns​ * Advantage: Ensures that individuals with high fitness are always selected, which promotes survival of the fittest. * Leans toward exploitation * Advantage or trade-off: Since it focuses more on selecting fitter individuals, it preserves good solutions. * Potential downside only if you're in a landscape where exploration is critical — in such cases, diversity may be reduced, leading to premature convergence.

Question 29

What is Tournament Selection?

Answer 29

- “Tournament selection is a non-parametric selection method — it doesn’t care about the actual fitness values, only who’s better. - We run a mini-competition (‘tournament’) among a few individuals and select the best one from that group.” ○ It's easy to implement. ○ Works well even if fitness values are noisy, close together, or poorly scaled.

Question 30

Tournament Selection Code

Answer 30

REFER TO SLIDES

Question 31

Why is Tournament Selection used so commonly?

Answer 31

“Not sensitive to size of fitness values” “Unlike roulette or SUS, tournament selection only cares about who’s better, not how much better. That makes it immune to weird scaling or small differences in fitness.” ==== “Simple (and easily parallelised)” “Tournament selection is very easy to implement and fast. You can also run many tournaments at the same time — which makes it ideal for parallel processing in large-scale applications.” ==== “Tunable by setting tournament size t” The beauty of tournament selection is that you can control how aggressive it is by changing t, the tournament size. 🔸 When t = 1 “You just pick one random individual — no competition. This is like pure random selection, with no selection pressure at all.”

Question 32

What is Elitism?

Answer 32

Elitism is the idea that if you already have a really good solution, you should keep it! ○ So, you automatically carry over the best individual(s) to the next generation, without any modification or competition.” - These individuals are called the ‘elite group’ * They’re protected from mutation, crossover, or being replaced. * This guarantees that the best solution so far is never lost.

Question 33

Why use Elitism?

Answer 33

- Why Use Elitism? * “It’s a form of exploitation — making sure we don't throw away a good thing. * It’s like saying: ‘We already found gold — don’t forget where we buried it!’” * Similar in spirit to (μ + λ) Evolution Strategies, where parents can survive into the next generation. * Helps improve convergence speed because the population keeps getting better (or at least not worse).

Question 34

What is the catch with Elitism?

Answer 34

“Too much elitism can be dangerous!” Problem: Premature Convergence “If we only focus on elites, the population may lose diversity — and all individuals may become nearly identical. That can lead to getting stuck in a local optimum.”

Question 35

How can you keep Elitism in check?

Answer 35

How to Keep Elitism in Check The slide suggests three strategies: 1. Increase mutation “Add more randomness to help escape local optima and reintroduce variety.” 2. Breed in diversity from outside the elite group “Allow less-fit individuals to contribute genes — they might introduce useful traits.” 3. Reduce how many elites are stored “Only preserve the top 1 or 2 — not the entire top half of the population.”

Question 36

Elitism Code

Answer 36

REFER TOP SLIDES