Linear regression

Question 1

How to quantify the quality of a model and its predictions?

Answer 1

By calculating Sum of Squared Residuals.

Question 2

How do you calculate the sum of squared residuals?

Answer 2

1. First calculate the residuals by finding the differences between observed and predicted values.
2. Then square the residuals and sum up the squared residuals.

Question 3

Sum of squared residuals formula

Answer 3

SSR = Sigma(observed - predicted) ** 2

Question 4

What kind of models can we apply SSR

Answer 4

All kinds of models - linear or curve

Question 5

How do we calculate the residuals - vertical or perpendicular distance to the model

Answer 5

By calculating the vertical distance

Question 6

Perpendicular distance to the model is also called as

Answer 6

Shortest distance.

Question 7

Why do we use vertical distance instead of the shortest distance

Answer 7

Since the perpendicular or the shortest distance doesn’t give the correct values on x

Question 8

What is the problem of SSR?

Answer 8

SSR is not easy to interpret since it depends on the amount of data we have. For example - For three data points, SSR is 14. For 5 data points, SSR is 22. It doesn’t imply that the second model is worst than first. Higher the data, worse the result. It only tells us that the model with more data has more residuals.

Question 9

Should SSR be low or high

Answer 9

The smaller the value of SSR, the better the model fits the data. If SSR is zero, the model fits perfectly to the data.

Question 10

How to compare two models that may fit to different sized datasets is to calculate

Answer 10

Mean Squared Error

Question 11

Formula for Mean squared error

Answer 11

SSR/number of observations
sigma(observed - predicted) ** 2/ n

Question 12

What does MSE calculate intutively

Answer 12

Average of residual, so MSE is present than SSR which increases when we add more data.

Question 13

Why are MSEs difficult to interpret

Answer 13

When comparing two models, the values depend on the scale that is used in the models. One model using mm has MSE 4.7 while the other model using meters has MSE 0.0000047

Question 14

How to overcome the disadvantage of MSE

Answer 14

Using R squared

Question 15

How R squared overcomes the issue with MSE

Answer 15

R squared is independent of both size of the dataset and scale

Question 16

How is R squared calcualted?

Answer 16

1. R squared is calculated by comparing the SSR/MSE around the mean y-axis value. Compare this to SSR/MSE around the model we are interested in. Therefore R squared gives the percentage of how much the predictions improved by using the model instead of just mean.

Question 17

What is the range of R squared values

Answer 17

0 to 1

Question 18

When R squared is closer to one it means

Answer 18

The model fits the data better than using the mean y-axis value.

Question 19

R squared formula

Answer 19

SSR(mean) - SSR(fitted_line)/SSR(mean)

Question 20

SSR(mean) - SSR(fitted_line) - what does it mean

Answer 20

Tells us the percentage the residuals around the mean shrank when we used the fitted line.

Question 21

Rsquare = 1 means

Answer 21

Fitted line fits data perfectly

Question 22

Rsquare = 0 means

Answer 22

SSR(mean) = SSR(fitted_line) - they are both equally good or bad

Question 23

SSR(fiited_line) = 0 mean

Answer 23

Fitted line fits data perfectly

Question 24

In what scenarios does Rsquared results have low confidence

Answer 24

Small amount of data can have high (close to 1) R squared. Anytime we see trend in a small dataset, it is difficult to have confidence that a high R squared value is not due to random chance.

Question 25

When does R squared result have high confidence

Answer 25

When there is large amount of data.

Question 26

Is intuition only way to have confidence in R squared results?

Answer 26

No, having large data intuition is not enough. So, statisticians developed p-values.

Question 27

R squared formula using MSE

Answer 27

MSE(mean) - MSE(fitted_line)/MSE(mean)

Question 28

Does R squared always compare the mean to a straight fitted line?

Answer 28

The most common way to calculate R squared is to compare mean to a fitted line. We can calculate R squared to compare square wave to sine wave.