What does correlation measure?
Strength of the linear relationship between two variables
= (Covx,y)/(σx*σy)
What does covariance measure?
The direction in which variables move
= [Σ(x-X)(y-Y)]/(n-1), where X and Y are the mean values
Slope of the regression line
= covariance/variance
Test for statistical significance
Standard error of estimate
- Standard deviation of the error terms
- Measures degree of variability between actual and estimate values from the regression line
= SQRT[SSE/(n-2)]
Total sum of squares (SST)
- Sum of squared differences between actual values of Y and the mean of Y
- Total variation in Y
Regression sum of squares (RSS)
- Sum of the squared differences between predicted values of Y and the mean of Y
- Variation in Y explained by X
Sum of squared errors (SSE)
- Sum of squared differences between actual values of Y and predicted values of Y
- Unexplained variation in Y
R-squared (coefficient of determination)
- Percent of total variation in the dependent variable explained by the independent variable
= (SST - SSE)/SST or RSS/SST
What does F test assess?
If at least one independent variable within a set explains variation in values of Y
F-statistic
= MSR/MSE
= (RSS/k)/[SSE/(n-k-1)]
One-tailed test
What is ρ value? How is it used?
- Smallest level of significance for which the null hypothesis can be rejected
- Reject if ρ value is less than significance level
Setting up a t-test for regression coefficients
= (estimated coefficient - hypothesized value)/standard error
Why is it necessary to adjust R squared? How?
- R squared always increases with more variables, even if contribution is insignificant
= 1 - (1-Rsq)(n-1)/(n-k-1)
Heteroskedasticity
- Variance of the residuals is not equal across all observations in the sample
- Correct using White-corrected standard errors
How to detect heteroskedasticity
- scatter plot
- Breusch-Pagan chi-square test (n*Rsq)
Serial correlation (autocorrelation)
- Residual terms are correlated with one another
- Correct using the Hansen method by adjusting the coefficient standard errors
How to detect serial correlation
Durbin-Watson test
- close to 2 if not correlated
- positively correlated if 2
- reject if less than d1 (unless model is autoregressive),
If autoregressive, use t-test on the residual autocorrelations over several lags
What is multicollinearity
Two or more independent variables are correlated with one another
How to detect multicollinearity
F test and R-sq show significance but p-values don’t
Autoregressive model
When a dependent variable is regressed against one or more lagged values of itself
Mean-reverting level for a time series
b0/(1-b1)
- indicate if time series to be covariance stationary
ARCH (autoregressive conditional heteroskedasticity)
Variance of the residuals in one period is dependent of the variance of the residuals in a previous period
What does the Dickey Fuller test check?
Covariance stationary and cointegration (two time series are economically linked)
How to test for ARCH
- Regression of squared residuals on their lagged values
- If coefficient is statistically different from zero, then time series exhibits ARCH
