Model Improvement

Question 1

Forward end backward selection

Answer 1

Similarties:
Perform variable selection
Greedy algorithm: looks at best or worse, immediate best doesnt guarantee best result

Differences:
-Forward selection works best with high dimension (to many predictors, but cause overfitting and hard to find pattern)
-backwards works well with complementing predictors

Question 2

Regularization

Answer 2

Shrinkage method
-Reduce overfitting by shrinking he size of the coefficient estimates
- lower the flexibility of a model by bringing estimate close to zero. Varience reduce at expense of bias
- uses binarize dummy variables
-useful in high dimension

Serve to strike a balance btw goodness of fit and model complexity

Question 3

When selecting the best model

Answer 3

Prediction performance: should perform well in test data w.r.t.
Interpretability: the predictions should be easily explained
Ease of implementation: computationally, financially, logistically.