Lecture notes 1 (Two variable regression)

Question 1

How do you calculate correlation?

What is a feature about it?

Answer 1

COV(X,Y) / sqrt V(X) V(Y)

It is scale free.

Question 1

What is correlation?

Answer 1

The systematic relationship between two variables?

Question 2

How do you calculate Covariance of X,Y

Show both methods

Answer 2

Sum of (xi- xbar)(yi-ybar) / n-1

E(y-meany)(x-meanx)

Question 3

When can we view correlation as a causal relationship? (in context)

Answer 3

In clinical trials when you randomly assign different people to different groups.

Question 4

What two components is this made of yi = α + βxi + εi ?

Answer 4

yi = α + βxi + εi

yi = α + βxi (systematic part)
εi is random

Question 5

What are the 4 Classical Linear Regression assumptions in a model like:

yi = α + βxi + εi

Answer 5

1. E(εi | xi) = E(εi) = 0 ∀i (so the error term is independent of xi).
2. V (εi | xi) = σ^2 ∀i (error variance is constant (homoscedastic) – points are distributed around the true regression line with a constant spread)
3. cov(εi, εj |xi) = 0 for i ̸= j (the errors are serially uncorrelated over observations)
4. εi | xi ∼ N(0, σ^2) ∀i

Question 6

How can we interpret something as a causal effect?

Answer 6

If E(εi | xi) = 0

Question 7

What is the CLRM assumption of homoscadascity?

Answer 7

That all error terms must have a constant variance for every different explanatory variable value.

Question 8

What does this CLRM assumption mean cov(εi, εj |xi) = 0 for i ̸= j

Answer 8

It shows the information about the ith person cannot predict the jth person.

Knowing the epsilon of person i will not help in predicting person j

Question 9

What does the CLRM assumption mean εi | xi ∼ N(0, σ^2) ∀i

Answer 9

-The error terms take on a normal distribution
-Purely for mathematical convenience.

Question 10

What does the CLRM assumption mean

Answer 10

E(εi | xi) = E(εi) = 0 ∀i
for every value of x the mean of epsilon is 0
The error terms are indepedent of x

Question 11

When estimating a simple regression line for alpha, beta and standard deviation what are the estimates?

Answer 11

Alpha = a
Beta = b
Stdv = s^2

Question 12

How does the OLS estimation work?

Answer 12

yi = α + βxi + εi

yi is real value
yhat is predicted value

for OLS you want to minimise difference between real and predicted value by choosing a and b.

-Add sqaured differences of each observation and minimise it.

-Optimal solution is to minimise the residuals sum of sqaures.

Question 13

What is e?

Answer 13

e is the observed residual difference between yi and yhat

Question 14

When using OLS graphically what are we doing?

Answer 14

We are looking at a perfect straight line then looking at how far actual data point is from straight line.

Square all these differences and then minimise them.

This would then find the straight line that is closest to all data points.

Question 15

How do we estimate the variance in a regression

Answer 15

= S^2 Sample variance

1/ n-2 times by the sum of all residuals.

= RSS/ DOF

Question 16

How do we work out OLS estimator for slope coefficient?

Answer 16

b = sum of (xi - xbar)(yi-ybar) / sum of (xi-xbar)^2

Question 17

If you have an original model and then create a new model with all X times by 2 what happens to b?

How do you solve for this?

Answer 17

The new b is half the initial b

You sub in the relationship for new and old x.
To OLS formula.

Then also for this for intercept.

Question 18

How are the estimates under OLS described?

Answer 18

Best - the OLS estimators have the minimum variance, such that V (b|x) ≤ V (b∗|x), where b* is any alternative unbiased estimator.

Linear - the OLS estimators are a linear function of the error term

Unbiasedness- the OLS estimators are such that: E(a|x) = α and E(b|x) = β

Estimators.

Question 19

What must you look out for when interpreting coefficients?

Answer 19

WHAT TYPE OF MODEL IS IT?

DOES IT HAVE LOGS?

DO WE NEED TO TAKE e^ for exact if greater than 0.1

Question 20

What is the value at which you can approximate log coefficients?

Answer 20

0.1 and less can be approximated.

Question 21

What is the interpretation of coefficient with y = Blog(x)

Answer 21

A 1% increase in X causes a Beta/ 100 % increase in Y.

Or if greater than 0.1 use exact e

Question 22

What is the interpretation of coefficient with Y = BX

Answer 22

A 1 unit change in X causes a Beta unit increase in Y

Question 23

What is the interpretation of Cocefficient with log(y) = BX

Answer 23

When X increases by 1% Y increases by Beta x 100 %

when x is greater than 0.1 then exp(b) -1 x 100 corresponds to % change in Y.

Question 24

How do you interpret the coefficient for a log log model?
log(y) = Log(x)

Answer 24

when X increases by 1%, Y increases by Beta%
No need to take exact as already in logs

Question 25

What is another formula for the se(b)

Answer 25

all sqrt of s^2 / sum of xi - xbar squared

Question 25

How do you perform a hypothesis test for slope coefficent?

Answer 25

1. H0: B0
   H1: B1
2. Take significance level and DOF and find CValues
   DOF = (N- nof restrictions on residuals)
3. We then calculate t = b - B0 / se(b)
4. then compare to CV and either reject or accept H0.

Question 25

What is forecasting?
What is the issue and how is it solved?

Answer 25

Regression models often give one data point prediction but these are unlikely to be correct.

Therefore, we use confidence intervals to predict a range of values in which it will fall.

Question 25

How do you form a confidence interval?

Answer 25

Predicted value + / - CV x Standard error.

Question 25

1. What is the measure of goodness of fit and what equation explains it?
2. Break down each part of the equation

Answer 25

Measure of goodness of fit = R^2 = ESS/TSS or 1 - RSS / TSS

Sum of i=1 to n (Yi - Ybari) = Sum of i=1 to n (Yhat - Ybarhat) + sum of i=1 to n of e^2

Sum of i=1 to n (Yi - Ybari) = Total variation in the data (TSS).

Sum of i=1 to n (Yhat - Ybarhat) = Variation in the predictor values. (ESS)

sum of i=1 to n of e^2 = Variation in the regression residuals. (RSS)

Question 25

1.Is it correct to say a model with a high R^2 is a good model?

2. What is a better way to measure model adequacy?

Answer 25

1. No, it just means the model explains the total variation well.

Plus R^2 is quite an informal measure so cannot put too much emphasis.

2. if the error terms are aligned with the CLRM.

Question 26

Go on lectures notes 1 final page and explain all different parts of the stata output file.

Question 27

When do you interpret an intercept?

Answer 27

-When the explanatory variable is a feasible number