What are two requirements for linear regression

1) Additive errors 2) One multiplicative parameter per term

What are some important requirements for dummy variables?

They cannot depend on Y You cannot include intercept and all dummies. Would give perfect collinearity (Xs are not linearly independent)

Derive the OLS beta estimator

What is the formula for R^2 centered?

RSS/ TSS = 1 - SSE/TSS

What is TSS, RSS and SSE

When to use centered and uncentered R^2?

When model does not contains constant, use uncentered

What are the large-sample asymptotic assumptions of regression?

1) Stationary ergodicity 2) Full rank (E[Xi'Xi]) 3) Martingale difference sequence (error times regressor) 4) Moment existence. Regressors have finite 4th moments and errors have finite variance

What is estimator bias?

Difference between expected value of estimator and true value of parameter

What is estimator consistency?

And estimator is consistent if its probability limit is the true parameter

What is the problem with ommited variables?

If the estimated excludes relevant variables that are correlated with the included variables, the parameter estimates will be biased. Also, parameter variance will not be estimated consistently. It is safe to exclude dummy variables since they are orthogonal to regressors

What happens if you include extraneous (non-relevant variables)

You parameter estimates will still be unbiased and variance will still be consistent. But you will have more variance (potentially too low t-stat)

What are the problems of working with heteroscedastic data?

Parameters will be unbiased, but variance estimator will be inconsistent. One solution is to use White's robust variance estimator. Using White's estimator on homoscedastic data will however give worse finite sample properties and increases likelihood of size distortions. Another solution to heteroscedastcity is to use GLS

How can use test for heteroscedasticity?

Use White's test. Estimate your model. Then regress squared errors on squares and cross products of all regressors (including constants). Null hypothesis: all parameters (except for intercept) are zero. Test statistic n*R^2. It is chi^2 squared distributed with df = number of regressors in auxiliary regression (excluding intercept)

What happens when errors are correlated with regressors?

If measuring regressors with noise. Regressors and errors may be correlated. This gives downwards bias. Endogeneity will also give bias.

What is a type 1 error

Rejecting a true null

What is a type 2 error?

Failure to reject the null when the alternative is true

What is the size of a test?

Pr(Type 1 error) = alpha

What is the power of a test?

Pr(1- type 2 error). Probability of rejecting a null when the opposite is true

What is a linear equiality hypothesis?

R*beta - r = 0 Where R is an mxk matrix r is mx1 vector m i number of restrictions k is number of regressors

What are the 3 types of hypothesis tests?

Wald test Lagrange Multiplier test Likelihood Ratio test

What are the differences in power between the 3 types of hypothesis tests?

W is about the same as LR which is larger than LM in terms of test statistics. Since they follow the same distribution, larger test statistics gives more power

How to implement a Wald test

Run the unrestricted regression and estimate parameters and covariances. Compute test statistics using null hypothesis. Compare against chi-squared distribution

How to implement a Lagrange Multiplier test

That is, the Lagrange Multiplier Test examines the size of the “shadow price” of the constraint regarding we are trying to test in the OLS optimization framework. Estimate the restricted model and compute its errors. Calculate the score using errors from the restricted model and the regressors from the unrestricted model Calculate the average score and the varianc of the score Compute the test statistic

How to implement a Likelihood ratio test

The Likelihood Ratio test is based on testing whether the difference of the scores, evaluated at the restricted and unrestricted parameters, is large Estimate the unrestricted model and its parameter covariance Compute the test-statistic using the restrictions Compare against CVs from Chi^2_m distribution

What are the main model selection techniques?

General to specific (GTS) Specific to general (STG) Information Criteria Cross-valiadation

Explain general to specific

Start with the largest possible number of possible models If at least one regressor is insignificant, remove the regressors with the lowest t-stat. Restimate the model. Continue until all regressors are significant

Explain specific to general

Start with the variable with the lowest p-value. At new variables sequentially as long as most recently added variable is significant

Discuss pros and cons of GTS and STG

GTS has a positive probability of including irrelevant variables, it will however never exclude relevant variables. In STG, the variance is NOT consistently estimated in the beginning, which can lead to wrong inference. For both, we have the problem that t-stats do not follow standard distributions when used sequentially.

Explain Information Criteria

If possible, search through all models and pick the one where the information criteria is lowest. Typical information criteria are AIC and BIC. If not feasible, run GTS or STG type search through models. Information criteria are of the form: - log-likelihood + penalty term for including regressors. In OLS use ln(error variance) instead of ll.

Discuss pros and cons of different information criteria

The AIC asymptotically may select a model with irrelevant regressors