Simple Linear Regression

Question 1

Regression analysis

Answer 1

Regression analysis is used to:
predict the value of a dependent variable (Y) based on the value of at least one independent variable (X)
explain the impact of changes in an independent variable on the dependent variable

Question 2

Dependent variable (y)

Answer 2

Dependent variable (Y): the variable we wish to predict or explain (response variable)

Question 3

Independent variable (x)

Answer 3

Independent variable (X): the variable used to explain the dependent variable (explanatory variable)

Question 4

Simple linear regression

Answer 4

Only one independent variable, X
Relationship between X and Y is described by a linear function
Changes in Y are assumed to be caused by changes in X

Question 5

b0 and b1

Answer 5

b0 and b1 are obtained by finding the values of b0 and b1 that minimise the sum of the squared differences between actual values (Y) and predicted values ( )

Question 6

b0

Answer 6

b0 is the estimated average value of Y when the value of X is zero

Question 7

b1

Answer 7

b1 is the estimated change in the average value of Y as a result of a one-unit change in X

Question 8

SST

Answer 8

Total Sum of Squares

Measures the variation of the Yi values around their mean Y

Question 9

SSR

Answer 9

Regression Sum of Squares

Explained variation attributable to the relationship between X and Y

Question 10

SSE

Answer 10

Error Sum of Squares
Variation attributable to factors other than
the relationship between X and Y

Question 11

Coefficient of Determination, r2

Answer 11

The coefficient of determination is the portion of the total variation in the dependent variable that is explained by variation in the independent variable
The coefficient of determination is also called r-squared and is denoted as r2

Question 12

ASSUMPTIONS OF REGRESSION

Answer 12

Linearity
Independence of errors
Normality of errors
Equal variance

Question 13

Linearity

Answer 13

The underlying relationship between X and Y is linear

Question 14

Independence of errors

Answer 14

Error values are statistically independent

Question 15

Normality of errors

Answer 15

Error values (ε) are normally distributed for any given value of X

Question 16

Equal variance

Answer 16

The probability distribution of the errors has constant variance

Question 17

residual for observation

Answer 17

The residual for observation i, ei, is the difference between its observed and predicted value

Question 18

Check the assumptions of regression by examining the residuals:

Answer 18

Examine for linearity assumption
Evaluate independence assumption
Evaluate normal distribution assumption
Examine for constant variance for all levels of X (homoscedasticity)

Graphical Analysis of Residuals
Can plot residuals vs. X

Question 19

Pitfalls of regression analysis

Answer 19

Lacking an awareness of the assumptions underlying least-squares regression

Not knowing how to evaluate the assumptions

Not knowing the alternatives to least-squares regression if a particular assumption is violated

Using a regression model without knowledge of the subject matter

Extrapolating outside the relevant range

Concluding that a significant relationship in observational study is due to a cause and effect relationship

Question 20

Types of relationships

Answer 20

1-2

Question 21

Equation

Answer 21

3

Question 22

Equation explained

Answer 22

4

Question 23

Sample equation and least squares method

Answer 23

5-6

Question 24

Example 1

Answer 24

7-12

Question 25

Interpolation v extrapolation

Answer 25

13

Question 26

Measures of variation

Answer 26

14-15

Question 27

rsquared

Answer 27

16-19

Question 28

Standard error

Answer 28

20-22

Question 29

Residual analysis

Answer 29

23-28

Question 30

Slope inferences

Answer 30

29-31

Question 31

T test

Answer 31

32-34

Question 32

F test

Answer 32

35-37

Question 33

Confidence interval

Answer 33

38