bias-variance in classification? Flashcards
(28 cards)
what is the goal of bias/variance
Construct an estimator
θn for the unknown parameter θ∗ given
observed data S. The index n is to indicate that we obtained this
estimate from n examples
what does n mean in θn
index n indicates we obtained this from n examples
how does error change for different training sets
different between expected value and group value
bias as an estimator definition
Bias of θn is Bias(θn) = E[θn]−θ∗
.
estimator is unbiased if
E[θn] = θ∗ for all θ∗
also if bias = 0
variance of an estimator
Variance of θn is Var(θn) = Cov [θn].
Unlike bias, the variance does not directly depend on the true
parameter θ∗
.
if θn is scalar then variance =
E[(θn - θ∗)^2]
if θn is vector then variance =
Cov [θn]
worded bias definition
the amount of assumptions a bias has
which model has a high bias
linear regression:
- assumes data has linear distribution
-
what does high bias lead to in a model
underfitting
more assumptions =
high bias
worded variance definition
sensitivity of model to traning
more sensitivity =
higher variance
can not preform as well =
hgher variance
what algorithm has high variance
decision trees
what is the bias and variance of a decision tree
low bias and high variance
what do is the most ideal for bias and variance
a abalance between bias and variance
what s the bias and variance trade-off
balance between bias and variance
what does high variance lead to
overfitting
solutions for high bias
train more increase model complexity
use different model architecture
try increasing complexity of model
solutions for high variance
introduce more data
use regularisation
try a different model
what do we want estimators to have (the most ideal estimator )
low bias and low variance
what is the bias- vaiance of mle
low bias if n is sufficiently large but has hhigh variance
