Simple Regression

Question 1

What is the most common took of the applied economist?

Answer 1

Regression

Question 2

What is regression?

Answer 2

It is used to help understand the relationships between many variables

Question 3

What does regression do on an XY-plot?

Answer 3

It fits a line through the points in the XY-plot that best captures the relationship between X & Y

Question 4

What is the equation of a straight line (linear function)?

Answer 4

Y = 𝛼 + 𝛽X

Question 5

What is 𝛼 in the straight line equation?

Answer 5

The intercept

Question 6

What is 𝛽 in the straight line equation?

Answer 6

The slope

Question 7

Why would we never get all points on an XY-plot lying precisely on it?

Answer 7

Due to measurement error

Question 8

Is the straight line the true relationship in an XY-plot?

Answer 8

The true relationship is probably more complicated, a straight line may just be an approximation

Question 9

What happens to important variables which affect Y?

Answer 9

They may be omitted

Question 10

What is the simple regression model?

Answer 10

Y = 𝛼 + 𝛽X + 𝑒

Question 11

What is 𝑒 in the simple regression model?

Answer 11

The error term

Question 12

What does regression analysis use?

Answer 12

It uses data (X and Y) to make a guess or estimate of what 𝛼 and 𝛽 are

Question 13

What happens if there are more than two points on the XY-plot?

Answer 13

It won’t be possible to find a line that fits perfectly through all points

Question 14

Why do we need to fine the “best fitting” line?

Answer 14

Because it makes the residuals as small as possible

Question 15

What do we mean by “as small as possible”?

Answer 15

The one that minimises the sum of squared residuals

Question 16

What is the most common method used to fit a line to the data?

Answer 16

We obtain the “Ordinary Least Squares” or OLS estimator

Question 17

How do we choose 𝛼 and 𝛽?

Answer 17

So that the vertical distances from the data points to the fitted line are minimised

Question 18

What does OLS do?

Answer 18

It minimises the sum of the squared residuals

Question 19

What is Y?

Answer 19

Dependent variable

Question 20

What is X?

Answer 20

Explanatory (or independent) variable

Question 21

What are 𝛼 and 𝛽?

Answer 21

Coefficients

Question 22

What are 𝛼’ and 𝛽’ ?

Answer 22

OLS estimates of coefficients

Question 23

How do we decide which is the dependent variable?

Answer 23

Ideally, the explanatory variable should be the one which causes/influences the dependent variable (X causes Y)

Question 24

What is an example of a model with this dependent variable?

Answer 24

Increases in X (population density) causes Y (deforestation to increase) - not vice versa

Question 25

Why must great care be taken in interpreting regression results as reflecting causality?

Answer 25

In some cases: the assumption that X causes Y may be wrong, we may not know whether X causes Y, X may cause Y but may also cause X, and the whole concept of causality may be inappropriate

Question 26

What question does regression address?

Answer 26

How much of the variability in Y can be explained in X?

Question 27

What do good fitting models have?

Answer 27

Small residuals

Question 28

What does it mean if the residual is big for one observation?

Answer 28

Then it is an outlier

Question 29

Why is it good to look at fitted values and residuals?

Answer 29

It can be very informative

Question 30

What is the coefficient of determination?

Answer 30

The total variability in the dependent variable Y equals the variability explained in the explanatory variable (X) in the regression plus the variability that cannot be explained and is left as an error

Question 31

What is R^2 known as?

Answer 31

The most common goodness of fit statistcs

Question 32

What is one way to define R^2?

Answer 32

To say that it is the square of the correlation coefficient between y and yi

Question 33

We can split the TSS into two parts, what are these parts?

Answer 33

Explained Sum of Squares and the Residual Sum of Squares

Question 34

Where must R^2 lie between?

Answer 34

It must always lie between 0 and 1

Question 35

What does R^2 = 1 mean?

Answer 35

Perfect fit - all data points are exactly on regression line

Question 36

What does R^2 = 0 mean?

Answer 36

X does not have any explanatory power for Y whatsoever

Question 37

What does bigger values of R^2 imply?

Answer 37

That X has more explanatory power for Y

Question 38

R^2 is equal to what?

Answer 38

The correlation between X and Y squared

Question 39

What does R^2 measure?

Answer 39

The proportion of the variability in Y that can be explained in X

Question 40

How do we carry out non-linear regression?

Answer 40

Replace Y or X (or both) in the regression model by a suitable non-linear transformation (ln(Y) or X^2)