Week 2: finding the quantitative relationship between 2 variables

Question 1

What principle do we use when we estimate b0 and b1 (using their formulas) ?

Answer 1

the Least Square principle

Question 2

What does the least square principle guarantee?

Answer 2

that the regression line is the best fit of data

Question 3

What are the b0 and b1 equations derived from?

Answer 3

minimising the sum of the squares of the vertical distances between the observed Yi and predicted Ŷi values of the Dependent Variable:

min∑(Yi−Ŷ)^2 = min∑(Yi−(b0+b1Xi))^2

Question 4

What does the least square principles guarantee that?

Answer 4

• that the regression line obtained has the smallest sum of squared residuals
• a regression line is the best approximation to the quantitative relationship existing between the variable Y

Question 5

What assumptions under-lie linear regression? (4)

Answer 5

Linearity

Independence of Errors

Normality of Error

Equal Variance (AKA homoscedasticity)

Question 6

What is the linearity assumption?

Answer 6

the relationship between X and Y is linear

Question 7

What is the ‘independence of errors’ assumption?

Answer 7

error values are statistically independent

this is particularly important when data is collected over a period of time

Question 8

What is the ‘normality of error’ assumption?

Answer 8

error values are normally distributed

Question 9

What is the ‘Equal Variance’ assumption?

Answer 9

the probability distribution of the errors has constant variance

Question 10

What is the residual for the observation i, ei??

Answer 10

the difference between its observed and predicted value

ei = Yi - Ŷi

Question 11

How do you check the assumptions of regression?

Answer 11

by examining the residuals:

-examine for linearity assumption
-evaluate independence assumption
-evaluate normality assumption
-examine for constant variance for all levels of X (homoscedasticity)

Question 12

How would you do a graphical analysis of residuals to investigate the assumptions?

Answer 12

plot residuals vs X

Question 13

What happens to the histogram of the residuals when the assumption of Normality is satisfied?

Answer 13

the histogram of the residuals approximate the bell shape of a normal distribution

Question 14

Why do we need to compare two or more different regression models?

Answer 14

different estimation methods (different formulas to calculate the slope and intercept)

different populations, different samples, different variables

Question 15

What statistical instruments can be used to make a comparison?

Answer 15

total sum of squares

R^2

standard error

Question 16

What equation do you use to work out total variation?

Answer 16

SST = SSR + SSE

Total Sum of Squares = Regression Sum of Squares + Error Sum of Squares

Question 17

What does SST stand for?

Answer 17

Total Sum of Squares

Question 18

What does SSR stand for?

Answer 18

Regression Sum of Squares

Question 19

What does SSE stand for?

Answer 19

Error Sum of Squares

Question 20

How do you work out SST (Total Sum of Squares)?

Answer 20

SST = ∑(Yi - ȳ)^2

Question 21

How do you work out SSR (Regression Sum of Squares)?

Answer 21

SSR = ∑(Ŷi - ȳ)^2

Question 22

How do you work out SSE (Error Sum of Squares)?

Answer 22

SSE = ∑(Yi - Ŷi)^2

Question 23

What type of variation is SST (Total Sum of Squares)?

Answer 23

Total Variation

Measures the variation of the Yi values around their mean ȳ

Question 24

What type of variation is SSR (Regression Sum of Squares)?

Answer 24

Explained Variation

Variation attributable to the relationship between X and Y

Question 25

What type of variation is SSE (Error Sum of Squares)?

Answer 25

Unexplained Variation Variation in Y attributable to factors other than X

Question 26

What is the coefficient of determination?

Answer 26

the portion of the total variation in the dependent variable that is explained by variation in the independent variable

Question 27

What is the coefficient of determination also known as?

Answer 27

R-square, denoted as R^2

Question 28

What is the equation for R^2 (the Coefficient of Determination)?

Answer 28

R^2 = SSR / SST = regression sum of squares / total sum of squares --> R^2 = ∑(Ŷi - ȳ)^2 / ∑(Yi - ȳ)^2

Question 29

What does R^2 have to be between?

Answer 29

0 ≤ R^2 ≤ 1

Question 30

If R^2 = 1 describe the relationship between X and Y and the variation.

Answer 30

there is a perfect linear relationship between X and Y: 100% of the variation in Y is explained by variation in X

Question 31

If R^2 = 0 describe the relationship between X and Y and the variation

Answer 31

no linear relationship between X and Y: none of the variation in Y is explained by variation in X

Question 32

If R^2 = 0.6 describe the relationship between X and Y and the variation.

Answer 32

Strong linear relationships between X and Y: Most of the variation in Y is explained by variation in X

Question 33

If R^2 = 0.4 describe the relationship between X and Y and the variation

Answer 33

Weaker linear relationships between X and Y: Some but not all of the variation in Y is explained by variation in X

Question 34

What is another way to work out R^2?

Answer 34

by working out the correlation coefficient (R) and then squaring it

Question 35

If R^2 = 0.576, how could this be expressed as a proportion or percent?

Answer 35

57.6 percent of the variation in the Y variable is explained by the variation in the X variable

Question 36

What is the equation for the Standard deviation of the variation of observations around the regression line? (What does Syx = ?)

Answer 36

Syx = √(SSE / n-2) = √(∑(Yi - Ŷi)^2 / n-2) where SSE = error sum of squares n = sample size

Question 37

What are the steps for working out regression in excel?

Answer 37

1) select DATA from the Title bar Menu 2) click on DATA ANALYSIS button 3) select REGRESSION from the contextual menu 4) enter Y range and X range desired options 5) get coefficient values, intercept coefficient goes before X variable 1 coefficient Ŷi (eg sells) = Coefficient Intercept + Coefficient X Variable 1 ( X variable) eg calls

Question 38

How do you get R^2 in excel?

Answer 38

1) select DATA from the Title bar Menu 2) click on DATA ANALYSIS button 3) select REGRESSION from the contextual menu 4) enter Y range and X range desired options 5) Look at the R square value in the Regression Statistics table 6) OR Look at ANOVA table, SS regression value = SSR and SS Total = SST 7) put values into equation R^2 = SSR / SST

Question 39

How do you get the value for Syx (standard error) in excel?

Answer 39

1) select DATA from the Title bar Menu 2) click on DATA ANALYSIS button 3) select REGRESSION from the contextual menu 4) enter Y range and X range desired options 5) Look at regression statistics table, standard error value = Syx

Question 40

How do you add the prediction line to the Plot of Fitted and observed data?

Answer 40

1) click on one of the observed values (Blue dots) 1) right click the mouse and select "Add Trendline" from the contextual menu