Bias and Variance

Question 1

What is variance?

Answer 1

This is a measure in how spread out the values of a data set are

Question 2

What is the bias?

Answer 2

This is how accurate the data points are to the target value

Question 3

What happens with overfitting with regards to the bias and variance?

Answer 3

When we start overfitting we get a better representation of the function, lowering the bias, but we become more dependent on the data set, so the variance becomes higher.

Question 4

What is the equation for bias-variance decomposition?

Answer 4

E_D[(y - f)²] = E_D [(y - E_D [y])²] + E_D[(E_D[y] - f)²]

Question 5

Which part of the decomposition is the variance?

Answer 5

Var = E_D [(y - E_D [y])²] This is the difference between the model and the mean of the model

Question 6

Which part of the decomposition is the bias²?

Answer 6

Bias = E_D[(E_D[y] - f)²] This is the distance between the mean and the actual value.

Question 7

What happens during training (and overfitting) to the bias and variance?

Answer 7

We are shifting the error from the bias to the variance

Question 8

What happens to the expected error of a model during training?

Answer 8

The error of the model does not change but it is shared and shifts between bias and variance

Question 9

How can the bias and variance be shown for this graph? [Picture 8]

Answer 9

> The distance between the average of all the possible models that can be learned and the original function is the bias

> If I take all possible models, the difference between the models and the average is the variance