The Linear Model

Question 1

What is the residual (e_i) of a model?

Answer 1

The (vertical) difference between the estimation and real value. e_i = y_i -ŷ_i

Question 2

What does it mean when the residual (e_i) is lower than 0?

Answer 2

The model overestimates the outcome for observation i.

Question 3

What does it mean when the residual (e_i) is larger than 0?

Answer 3

The model underestimates the outcome for observation i.

Question 4

What is RSS?

Answer 4

The RSS is the Residual Sum of Squares. Σ e_i²

Question 5

How is the regression called if we estimate the parameters based on RSS?

Answer 5

Ordinary Least Squares regression (OLS)

Question 6

What is the expected relationship between the residuals and the fitted values of a linear model?

Answer 6

There should be no relationship, if there would be it would mean that the true model is not a linear model at all.

Question 7

What behaviour do we expect in a histogram of the residuals for a linear model?

Answer 7

The residuals should be symmetric around 0. (high residuals are less likely to occur than low residuals)

Question 8

What is RMSE? and how does the formula look?

Answer 8

Root mean squared error.

Question 9

What is RSE?

Answer 9

Residual standard error.

Question 10

What is the formula for the Coefficient of Determination or R²?

Answer 10

Question 11

What does the Coefficient of Determination or R² tell us?

Answer 11

It explains how much of the total observed variability is acounted for/explained by the model.

Question 12

Is the coefficient of determination (R²)the square of pearson’s correlated coefficient r ?

Answer 12

Only if we have one IV in our model.

Question 13

Does R² = 1 imply we have the true model? i.e. all estimated parameters of the model (ß_j) are correct to the truth.

Answer 13

No, when R² = 1 it means that the model accounts for all variability with the data set, however this does not mean we found the model that created the data. f.i. R² = 1 can always be reached if the model is overfitted.

Question 14

What is meant with two caveats?

Answer 14

In linear modelling, the residuals, and hence RSS and RMSE, are calculated vertically. If doen horizontally, the new estimated model will differ from the vertical one.

Question 15

What are the 3 main reasons to add more predictors to a model?

Answer 15

1. It reduces the RSS, and hence is more accurate.
2. It accounts for factors other than the one of interest, and thus adding these other factors eliminates their effect on outcome Y.
3. If the effect of X on Y is dependent on a third variable, we need to model it explicitly.

Question 16

When using the linear model with multiple predictors, the R² should be updated to account for the model complexity. What is the new relation of R² for the multiple linear model?

Answer 16

Where n is the amount of training data, and p is the amount of predictors used.

Question 17

Are linear models based on numerical or categorial variables or both or niether?

Answer 17

Typically numerical, but categorial variables can be included using dummy variables.

Question 18

What are dummy variables (z)?

Answer 18

It is a way to map categorial predictors into numerical ones for a multi linear model. (f.i. no = 1, yes = 0)

Question 19

How does a dummy variable (z) influence the parameter estimates of a linear model?

Answer 19

As it is an ‘binary predictor’ it only influences the intercept and not the slope in the estimated parameters. (ß₀ ⇒ ß₀ + ß₁)

Question 20

What does the estimated value (ß₀) of a dummy variable (z) represent?

Answer 20

It represents the relative change in intercept, compared to the original intercept. (ß₀)

Question 21

How many dummy levels are expected for an categorial variable with 5 levels?

Answer 21

4, there is always one level incorperated into the baseline or reference level (the intercept ß₀)

Question 22

Is it possible for a linear model to have no intercept level?

Answer 22

Yes, when the intercept level is devided into a set of dummy variables of which only one can be active simultaneously. Then this dummy ‘parameter’ now acts as the intercept.

Question 23

Answer this: What effects are displayed and which linear model explains the behavious of the coloured slopes.

Answer 23

One intercepts and two slopes: we study the interaction effect and only the main effect of the numerical variable.

Y = ß₀ + (ß₁ + ß₂ x_{insurence = no}) x₁

Question 24

Answer this: What effects are displayed and which linear model explains the behavious of the coloured slopes.

Answer 24

Two intercepts and two slopes: we study the main- and interaction effects of the numerical and nominal variable. Y = ß₀ + ß₁x₁ + (ß₂ + ß₃x₁) x_{insurence = no}

Question 25

Answer this: What effects are displayed and which linear model explains the behavious of the coloured slopes.

Answer 25

Two intercepts but only one common slope: we study only the main effects of the numerical and nominal variable. Y = ß₀ + ß₁x₁ + ß₂x_{insurence = no}

Question 26

What is the definition of a linear model?

Answer 26

When the relationship between the predictors and response is linear, even tho the predictors itself are not.

Question 27

Is this a linear model?

Answer 27

No, relationships between predictors and response is non-linear.

Question 28

Is this a linear model?

Answer 28

Yes, relationships between predictors and response is linear.