model selection Flashcards
(15 cards)
What is the goal of model selection?
To choose the simplest model that explains the data well (parsimony), balancing underfitting (too few variables) and overfitting (too many variables).
What is collinearity, and why is it a problem?
Collinearity occurs when predictors are correlated. It causes unstable coefficient estimates and inflated standard errors, making it hard to isolate individual effects.
How do you detect collinearity?
Use Variance Inflation Factor (VIF).
Rule of thumb: VIF > 5 (or 10) indicates high collinearity.
R code: car::vif(model)
How do you fix collinearity?
Remove one of the correlated predictors.
Combine them (e.g., average left/right leg length).
Use regularization (e.g., ridge regression).
What is the difference between forward selection and backward elimination?
Forward: Start with no predictors; add one at a time based on significance.
Backward: Start with all predictors; remove the least significant one at a time.
What is AIC, and how do you interpret it?
Akaike Information Criterion balances model fit and complexity:
AIC=−2log(likelihood)+2P
Lower AIC = better model.
R code: AIC(model1, model2)
When should you use AICc instead of AIC?
Use AICc (corrected for small samples) when the sample size is close to the number of parameters (e.g., n/P<40).
What is the difference between AIC and BIC?
AIC: Favors better-fitting models (+2P penalty).
BIC: Favors simpler models (+Plog(n) penalty, harsher for large n).
How do you compare non-nested models (e.g., Y ~ X1 + X2 vs. Y ~ X1 + X3)?
Use AIC/BIC (F-tests only work for nested models).
What does dredge() (from MuMIn) do?
It ranks all possible models by a fit criterion (e.g., AICc).
Key output: Models sorted by AICc; “delta” shows difference from the best model.
R code: options(na.action = “na.fail”); MuMIn::dredge(model)
How do you interpret AIC weights?
Weights sum to 1 across models; higher weights indicate stronger evidence for that model.
Why might automated selection (e.g., stepwise) be risky?
It can overfit noise in the data and produce unstable models. Always validate with theory/hypotheses.
What’s the difference between Type I and Type II SS in ANOVA?
Type I: Sequential (order matters).
Type II: Tests each predictor after accounting for others (order doesn’t matter).
R code: car::Anova(model, type = “II”)
When should you use p-values vs. AIC for selection?
p-values: For hypothesis testing (e.g., “Is SST significant?”).
AIC: For prediction-focused model comparison.
What is Occam’s Razor in model selection?
Among models with similar explanatory power, the simplest (fewest parameters) is best.”
