Classical Regression Model Flashcards
What is the CRM?
The linear regression model with assumptions which ensure OLS estimators are ‘good’ estimators
What are the Gauss-Markov assumptions?
Linear cef: E(Y|X) = α + βX
Random sample {xi, yi}
Xs are non-stochastic (fixed in repeated sampling) variables with fixed non-identical values or equivalently Xs are stochastic variables distributed independently of the error
E(ε) = 0
Constant conditional variance (heteroscedasticity): Var(Y|X) = σ2
Yis are independent: Cov(Yh, Yi) = 0 for all i ≠ h
What can be said about b if the Gauss-Markov assumptions are met?
It is the Minimum Value Linear Unbiased Estimator (MVUE) or the Best Linear Unbiased Estimator (BLUE)
What are the features of conditional expectation relevant to regression?
E(B|B = b) = b and E(AB|B = b) = bE(A|B)
What do the Gauss-Markov assumptions say about Y?
It is the sum of its systematic/deterministic component E(Y|X) and its non-systematic/stochastic component ε
What is required of an efficient estimator?
It must use all available information
How do you find the expectation of the slope estimator?
First note that deviations from the mean X* = X - X̄ sum to 0
b = (ΣXY - ȲΣX)/Σ(X)2 = ΣXY/Σ(X)2
By the assumption of linearity this equals ΣX(α + βX + ε)/Σ(X)2 = β + ΣXε/Σ(X)2
By the assumption of non-stochastic Xs the expectation of b is now E(β) + ΣXE(ε)/Σ(X*)2 which using the assumption of zero expected error gives E(b) = E(β) = β
How do you find the variance of the slope estimator?
Var(b) = Var(β + ΣXε/Σ(X)2) = Var(ΣXε/Σ(X)2) which by the assumption of non-stochastic Xs equals Σ(X)2Var(ε)/(Σ(X)2)2 which by the assumption of constant variance of errors equals σ2/Σ(X*)2 which involves an unknown population parameter
What is the standard error of the regression?
SER2 = s2 = Σe2/(n-2) is the unbiased estimator for the variance of residuals/errors with ei = Yi - a - bXi
What is the standard error of the slope estimator?
(SE(b))2 = Sb2 = SER2/Σ(X*)2
This is unbiased and shows that the sampling distribution of b depends on the distribution of errors
What can be said about the true variance of errors from the assumption of non-stochastic Xs?
σ2 = V(ε|X) = E(ε2|X) - (E(ε|X))2 = E(ε2|X)
Assuming ε ~ N(0, σ2), what is the distribution of the slope estimator?
Y|X ~ N(α + βX, σ2)
Since b is a function of linear Yis, b ~ N(β, sb2)
What is the test statistic for a test of the slope of a linear regression?
t = (b - β0)/sb with dof n - 2
The CI for β is b ± σb
