Flashcards in Causal analysis Deck (35):
When does an OLS estimator capture a causal relationship? When does it not and what type of relationship is this instead?
An OLS estimator captures a causal relationship when the exogeneity assumptions holds, which states that the independent variable/regressor is uncorrelated with the error terms.
If the exogeneity assumption is violated then we are only able to interpret a relationship as an association or correlation.
Define an exogenous regressor and write this out mathematically.
An exogenous regressor is one which which is not correlated with the error terms i.e. Xi and ui are not correlated.
E(ui | Xi) = 0. Alternatively, exogeneity can be written as Cov(Xi, ui) = 0*
Define an endogenous regressor.
An endogenous regressor is one where Xi and ui are correlated. For example, if higher values of Xi are systematically associated with higher values of ui.
What are four possible reasons for endogeneity?
Reasons for endogeneity:
1. Omitted variable bias
2. Measurement error
3. Simultaneity bias
4. Selection bias
Define omitted variable bias.
Give an example of when omitted variable bias may occur.
Omitted variable bias is where a relevant variable is left out of the regression model which effects the dependent variable and is correlated with one of the regressors (is second part of this necessary?*)
For example, regressing wages on education may lead to endogeneity due to omitted variable bias. Ability is likely to affect wages and is correlated with education. The estimated coefficient for education will therefore be overestimated because ability and education are positively correlated, and the coefficient will therefore partially capture the effect of ability on wages.
Positive correlation because those who have higher ability are more likely to choose to stay on in education.
Define measurement error.
Give an example of when measurement error may occur.
Measurement error is where there is substantial error involved in measuring Xi, the independent variable.
For example, if testing the permanent income hypothesis measurement error may occur if using current income to estimate permanent income.
However, only use this answer if substantial measurement error is necessary.
Define simultaneity bias.
Give an example of simultaneity bias.
Simultaneity bias is where the dependent variable effects the independent variable, as well as the other way around. In other words, we are unsure which variable causes which, or both. They may be determined simultaneously.
For example, supply and demand; or police expenditure and crime rate.
Define selection bias.
What may cause selection bias to occur?
Selection bias is where the sample used in estimating a regression bias is not representative of the population, leading to the estimates being positively or negatively bias.
Selection bias may occur because the observations we observe may be self-selecting or skewed in some way.
The missing observations mean that we are unable to conclude whether it is a causal relationship, and estimates we derive are misleading.
What are three possible solutions to the problem of endogeneity?
2. Instrumental variables
3. Regression discontinuity design.
What is randomisation and how does this help to alleviate the problem of endogeneity?
Randomisation is when the variable of interest, Xi, is randomly assigned using, for example, a random number generator, rolling a dice, flipping a coin.
This makes it more likely for regressors to be exogenous because they will not be correlated with unobserved background characteristics i.e. randomisation smooths out disparate background characteristics.
What is the terminology used for randomisation in medical trials or by economists?
If Xi is the randomly assigned binary variable then:
Xi = 1 represents the treatment occurring
Xi = 0 represents the control group
The coefficient of Xi captures the treatment effect, which is the average change in outcome due to the treatment compared to the control group.
What are four problems or limitations that may be associated with randomised experiments?
1. Unethical to force participants to participate in some experiments, for example giving health care to some and not others.
2. Groups will often be self-selecting and therefore will have certain background characteristics, meaning they are no longer random
3. True and complete randomisation is often impossible, for example you cannot randomly assign gender, ethnicity, or having health insurance. The results often reflect having the CHOICE of receiving treatment rather than actual effect of the treatment.
4. Experiments are often time-limited and short, making it difficult to find long-term effects.
Define an instrumental variable.
Explain how instrumental variables help with endogeneity problems.
An instrumental variable is a substitute or proxy for a variable which is suspected to be endogenous or stochastic*?.
The instrument helps with endogeneity problems because it allows us to generate exogenous variation in Xi which is uncorrelated with the error terms, ui.
What three assumptions must the instrument(s) satisfy?
An instrument(s) must satisfy three assumptions:
1. Relevance assumption - the Z(s) must be correlated with X after accounting for Wi, the other exogenous variables in the regression
i.e. the relationship must be non-zero.
cov(Xi, Zi) /= 0
2. Exogeneity assumption/exclusion restriction/validity. This says that Z must not be correlated with the error terms, ui.
cov(Zi, ui) = 0
3. Z cannot be another regressor in the model, it must be an additional variable
What method is used to estimate a regression when using instrumental variables?
Describe briefly (in a sentence) how to carry out this method.
Two Stage Least Squares (2SLS) is used to estimate a regression when using instrumental variables. This involves two consecutive OLS regressions.
Describe the two stages of 2SLS.
Stage 1: Regress Xi, the independent endogenous variable, on the instrument, Zi, to obtain predictions for Xi, denoted X-hati. This captures the exogenous variation in X driven by the instrument Z.
Xi = A1 + A2Zi + A3Wi + vi
X-hati = a1 + a2Zi
where Wi are other exogenous variable(s) in the model.
Stage 2: Regress Yi, the dependent variable, on the predicted values for X. In other words replace Xi with X-hati in the original regression.
Yi = B1 + B2X-hati + ui
The estimates b1 and b2 are the IV 2SLS estimates.
Can we test the relevance assumption for an instrumental variable?
We can test the relevance assumption for an instrumental variable, simply by looking at the correlation between X and Z.
Can we test the exogeneity assumption for an instrumental variable?
We cannot directly test the exogeneity assumption for the instrumental variable because the error terms are unobserved.
However, we can indirectly test the validity/exogeneity assumption when the model is OVER-identified.
This uses the 'over-identification' or SARGAN test:
H0: All instruments are valid and exogenous
H1: At least one instrument is not valid
In Stata, the command is estat overid which is performed AFTER the two stages of 2SLS. We look at the p-value associated with the Sargan score of the chi-square distribution.
What is a problem with manually calculating 2SLS?
What is a solution for this?
A problem with manually calculating 2SLS is that the stand errors produced are wrong.
The standard errors are wrong because they do not take into account the fact that X-hati is itself an estimate from the first stage OLS regression.
This means we cannot perform hypothesis testing with these standard errors.
A solution for incorrect standard errors produced in 2SLS is to use the Stata command ivregress 2sls y (x = z1 z2) w
Stata corrects for this and produces correct standard errors.
2SLS and the coefficient of determination?
R^2, the coefficient of determination, has a different interpretation when doing 2SLS; it can even be negative.
How can we measure the strength of an instrument?
We can measure the strength of the instrument by using the F-statistic for joint significance AFTER running the first-stage of 2SLS (NOT the F-statistic of the actual regression obtained after both stages of 2SLS since this also tests the significance of the other exogenous variables, Wi).
In Stata, the F-statistic can be computed using the command test z1=0 z2=0*
If the F-statistic is LESS than 10, this implies it is a WEAK instrument.
If the F-statistic is GREATER than 10, this indicates a STRONG instrument.
What are two methods for testing whether a regressor is endogenous (one informal, one formal)?
1. Intuition and common sense - always consider the possibility of endogeneity. In particular, if CHOICE is involved then the variable is bound to be endogenous since it is likely to be systematically related to background characteristics which drive the individual to making a particular choice.
2. We can formally test whether a regressor is endogenous by performing a variant of the Hausman test which involves two stages:
First: Regress the regressor, Xi, on the instrument and the other exogenous regressors, Wi.
Xi = A1 + A2Zi + A3Wi + vi
and predict the residuals, v-hati
Second: Regress the outcome variable, Yi, on Xi (NOT X-hati), all other exogenous variables Wi, and the predicted residuals from the first stage, v-hati.
Yi = B1 + B2Xi + B3Wi + C1v-hati + ui
We then test the null hypothesis:
H0: C1 = 0
(i.e. the endogenous, problematic part of X is not related to Y, so X is not endogenous and no further instruments are needed)
H1: C1 /= 0 (X is endogenous)
If we do not reject the null this implies we do not have significant evidence to show that X is endogenous.
What types of instrumental variables are more likely to be valid and exogenous?
Instrumental variables that are randomly assigned are more likely to be valid and exogenous because they are less likely to be correlated with background characteristics.
If complete randomisation of an instrumental variable is not possible, what type of effect might we be able to capture instead?
If complete assignment under randomisation is not possible for an instrument, we can instead try to capture the Intention To Treat (ITT) effect rather than the actual treatment effect.
What does the ITT effect capture?
Give an example of an instrument that may be able to capture an ITT effect?
The ITT effect captures a CORRELATION effect rather than a causal effect since it ignores non-compliance, and therefore may over- or under-estimate the true causal effect of the treatment.
In other words, the ITT effect capture the effect of having the OPTION of being treated on the outcome variable.
For example, we may want to test the effect of watching a certain video on the test results of a set of students.
However, we cannot completely randomly assign watching the video to students.
Instead, we send around an email with the video link to a random selection of students, and use this to capture the ITT effect.
In this case, the ITT effect is the effect of receiving the email on the test scores, regardless of whether the student then went on to watch the video.
Yi = D1 + D2Zi + ei
Where Zi is the instrument, the email containing the video link, and Yi is the outcome variable, the test score.
The ITT is the coefficient of Zi, which is D2.
How can we obtain a causal effect from the ITT effect?
We can obtain a causal effect from the ITT effect (which only captures a correlation) by using the ITT effect to calculate the Local Average Treatment Effect (LATE).
What is LATE, what does it capture, and how do we calculate it?
LATE stands for Local Average Treatment Effect and captures the causal effect of a treatment by using the ITT effect.
However, LATE only captures the causal effect of a treatment on an outcome variable among those who took the treatment as a result of being given the option of treatment within the trial conditions.
To calculate LATE, we divide the ITT (D2 from card 25) by the difference in compliance rates of taking the treatment (this is the coefficient on the instrument from the first stage of 2SLS, where the endogenous variable is regressed on the instrument).
When instrumental variables are used to estimate the causal effect of an endogenous TREATMENT variable, what is the limit of what this can capture?
When instrumental variables are used to estimate the causal effect of an endogenous treatment variable, the estimation strategy only capture the Local Average Treatment Effect (LATE). This is the average treatment effect for one subset of the population - these are the 'units' which receive treatment if and only if they are induced by an exogenous instrumental variable.
What is the next best alternative if a Randomised Control Test is not possible?
If a Randomised Control Test is not possible to estimate the causal effect of a treatment variable, the next best alternatives are natural or quasi-experiments.
Natural or quasi-experiments exploit 'natural events' which are treated as sources of randomness.
For example, rainfall, changes in regulations/policies, natural disasters, unforeseen economic crises.
What is one technique that exploits 'natural' events to make causal inferences?
Regression Discontinuity (RD) experiments exploit specific rules and change in policy to make causal inferences, using these as sources of randomness.
What are RD experiments? What is the main assumption made when performing an RD experiment?
What is the variable introduced into the RD model which the treatment depends on?
Give an example of an RD model.
RD experiment exploit random/unexpected changes in specific rules or policies in order to make causal inferences.
The main assumption behind RD is that differences in the outcome between those just above and below the cut-off point are only due to the treatment - in other words the background characteristics of individuals near to either side of this threshold are assumed to be the same.
A running variable, denoted Q, is introduced into the model which the treatment depends on. Q is an observed continuous variable, such as age, weight, birth date.
For example, an RD model may estimate the causal effect of receiving neonatal care on educational attainment by exploiting the medical rule that babies weighing under 2500g receive additional neonatal care.
Another example: causal effect of drinking on mortality rate, RD could exploit the policy that individuals under 21 are not allowed to purchase or consume alcohol in US.
What are two types of RD design?
Two types of RD design:
Define what a sharp RD is.
Give an example of a sharp RD.
A sharp RD is one where the treatment status is a DETERMINISTIC function of the running variable - in other words the rule/policy is applied strictly.
For example, alcohol restrictions are applied rigorously in US to those under the age of 21. This would constitute a sharp RD design.
Define what a fuzzy RD is.
Give an example of a fuzzy RD.
A fuzzy RD is one where the treatment status is NOT A DETERMINISTIC function of the running variable. Instead, the probability or intensity of the treatment jumps at the cut-off. This may be because the rule/policy is not applied strictly, or the treatment is only affected indirectly.
For example, raising the school leaving age in England, Wales and Scotland in 1947 from 14 to 15 - however there were still some 14-year olds who left school after the policy was enacted.