3. MLR

Question 1

Purpose of Multiple Linear Regression

Answer 1

Explain the variation of Y.

( Each coefficient is interpreted as the estimated change in y corresponding to a unit change of its related variable, with all other variables held constant. )

Question 2

Assumptions for error terms

Answer 2

1. Normally distributed with mean = 0
   ( else not efficient estimators )
2. Constant Variance
   ( fix heteroscedacity with weighted least squares )
3. Error terms are independent of each other

Question 3

Common issues of MLR

Answer 3

1. Overfitting ( non-contributing variables )
2. Multicollinearity ( redundant variables )

Question 4

Testing for Overfitting

Answer 4

Single linear regression for each variable. Weak correlations signal non-contributing variable.

Question 5

Testing for Multicollinearity

Answer 5

( After checking significance of each variable )

• High correlation between explanatory variables
• Sensitivities when including explanatory variables
• Increase in Standard error, sharp decrease of adjusted R^2
• Variance Inflation Factor (VIF) > 10
  ( VIF = 1/(1-R^2) )

Question 6

Testing significance

Answer 6

ANOVA - Analysis of Variance / F-Test ( MODEL )

p < alpha means variable explains some variation of y with stat. significance

• t-Test ( INDIVIDUAL VARIABLES )

Question 7

Goodness of fit test measures

Answer 7

• R^2
• Adjusted R^2
• MSE
• F-TEST (VANOVA)

Question 8

Formula for F-Test

Answer 8

MSR / MSE or (SSR/k) / (SSE/n-k-1)

Question 9

Formula for R^2

Answer 9

R^2 = 1 - (SSE/SST)