3- Classical linear regression

Question 1

What does the error term (u) capture in a regression?

Answer 1

All the effects on the dependent variable not in the explanatory variables

Question 2

What are the betas in a regression?

Answer 2

The gradients associated with each of the theoretical parameters

Question 3

What is always the first independent variable in a regression?

Answer 3

A constant β₀, it represents the y-intercept i.e. value of dependent variable when there’s no input

Question 4

What is the population regression line?

Answer 4

The sum of the independent variables; the relationship that holds between x and y on average (without error term) y = β₀ + Xβ₁

Question 5

Once x and y values are collected how is a line of best fit determined?

Answer 5

Minimising the sum of each error term squared

Question 6

What is the error term for each observation plotted?

Answer 6

The vertical distance between the population regression line

Question 7

What do the independent variable subscripts denote (xᵢₖ)?

Answer 7

The first letter is the number of observations and the second letter is the number of parameters

Question 8

What are the 6 assumptions of the linear regression model?

Answer 8

-Linearity
-Identification condition
-Exogeneity
-Homoskedasticity
-Normality
-X can be fixed or random

Question 9

Explain the linearity assumption

Answer 9

Linearity in parameters means the betas have index 1 so are not exponential

Question 10

Explain the identification condition

Answer 10

Number of observations must be at least as great as the number of parameters

Question 11

Explain the exogeneity assumption

Answer 11

Expected value of any u on X is zero E(uᵢ|X)=0 meaning no observation conveys any information about u

Question 12

Explain the homoskedasticity assumption

Answer 12

Variance of the error term is constant across observations

Question 13

Explain the normality assumption

Answer 13

Error terms are normally distributed

Question 14

What are the 3 main properties of the OLS estimator?

Answer 14

-Unbiased
-Given variance covariance matrix
-Estimator is the best in that it has minimum variance

Question 15

What does it mean that the OLS estimator is unbiased?

Answer 15

The expected value of all estimated betas will give their true value E(^βols)= β

Question 16

What are the 2 steps to show the OLS estimator is unbiased?

Answer 16

1. Sub in y to the OLS estimator expression and expand
2. Take expectations, and show only β remains

Question 17

What is the variance matrix of the OLS estimator var(βₒₗₛ)?

Answer 17

var(βₒₗₛ)=σ²(X’X)⁻¹

Question 18

When a function is homoscedastic, what is E(uu’)?

Answer 18

σ²I

Question 19

How can you prove the variance of the OLS estimator is var(βₒₗₛ)=σ²(X’X)⁻¹?

Answer 19

1. Use equation var(B̂)=E[(B̂ - E(B̂))(B̂ - E(B̂))]
2. Sub in OLS estimate and expectation and expand

Question 20

What is the OLS estimate of variance?

Answer 20

E(û’û)/n-k

Question 21

What is the Gauss-Markov theorem?

Answer 21

Unbiased linear estimators is the “Best”, in the sense that it has the minimum variance

Question 22

What are the 3 conditions for the Gauss-Markov theorem?

Answer 22

1. y=Xβ+u
2. var(u) = σ²I
3. X is full rank

Question 23

What is the identity P?

Answer 23

X(X’X)⁻¹X’

Question 24

What is the identity M?

Answer 24

I - P

Question 25

What are 3 main properties of M?

Answer 25

-Square
-Symmetric
-Idempotent

Question 26

How can you show X is uncorrelated to u?

Answer 26

E[(X’X)⁻¹X’u] = 0

Question 27

What is the variance of a given error term var(uᵢ)?

Answer 27

E(u²) = σ²

Question 28

If u has a multivariate normal distribution with zero mean, what is the distribution of uᵢ?

Answer 28

Univariate normal distribution

Question 29

What happens to the estimated coefficients (β) when X is scaled up or down?

Answer 29

Coefficients are scaled in the opposite direction to compensate

Question 30

What does the Beta variance covariance matrix look like?

Answer 30

Beta variances on principal diagonal and covariances in rest of elements

Question 31

What is a trick to find the n?

Answer 31

n is the first element of X’X

Question 32

What expression is always the error term estimate û?

Answer 32

û = (I - X(X’X)⁻¹X’)u

Question 33

What is tr(E(û’û))?

Answer 33

nσ²

Question 34

How can var(Xβ+u) be simplified?

Answer 34

var(u)

Question 35

How do betas change with operators on y?

Answer 35

Multiplication is performed on all betas in the same direction, addition is also applied to the intercept