Model 2: Linear Regression Models

Question 1

Model Building Principles

Answer 1

1. Consider all predictor variables that may be important in describing the outcome.
2. Check for multicollinearity
3. Consider possible interactions between variables.
4. Think about expected value of predictors.

Question 2

What model selection approaches are covered in STAT210?

Answer 2

Gelman and Hill
Backward/Forward/Sidewise-Step Selection
Akiake’s Information Criterion (AIC)

Question 3

Describe the Gelmen & Hill Approach

Answer 3

Look at the statistical significance and sign of estimated effect.
- significant & expected = keep
- non-significant & expected = find to keep unless interaction
- significant & unexpected = possible lurking variables
- non-significant & unexpected = remove.

Question 4

Describe Backwards Selection

Answer 4

1. Start with all possible predictors
2. Remove predictor with largest p-value
3. Refit model
4. Repeat unit all predictors included are significant.

remember if the interaction is keep, the interactions main effect must be keep

Question 5

Describe Forward Selection

Answer 5

1. Start with the basic model (no predictors in model)
2. Consider all possible predictor variables at one time. Include the predictor with the smallest p-value
3. Consider all possible predictors, minus the previous. Include the predictor with the smallest p-value.
4. Repeat until no significant predictors are left.

Question 6

Describe Sidewise-Step Selection

Answer 6

A combination of backward and forward selection, ensure the model is refit with each change.

Question 7

Describe AIC Selection

Answer 7

1) Narrow down selection to smallest number of possible models.
2) Specify the basic and full model, include models between the two.
3) Choose the model with the smallest AIC score.

Question 8

What does a high/low/similar score in AIC mean?

Answer 8

High score means the model has a lot of parameters.
Low score means the model is a good fit.
AIC with similar scores mean the models are similarly a good fit. Discussing which model to choose may depend on whether you want to prioritise simplicity or covariates.

Question 9

Cons and Benefits of Gelmen & Hill Approach

Answer 9

This approach is a helpful guideline for model building and invites discussion about variables. However, this method can be easily abused.

Question 10

Cons and Benefits of Backward/Forward/Sidewise-Step Approach

Answer 10

These approaches are easy and fast, however, each method can decide different models are best.

Question 11

Cons and Benefits of AIC Approach

Answer 11

This approach thinks about the model upfront, but may lead to multiple models being a possible good fit.

Question 12

What are additive models?

Answer 12

When explanatory/predictor variables are summed, which means the individual independent effects are summed. Visually there should only be one slope on the plot.