Basics of experimentation

Question 1

What types of experiments have been presented in the course

Answer 1

Lab, survey, field, natural, quasi

Question 2

What are the three aspects of internal validity presented in the course?

Answer 2

Construct validity - How closely the data measure the theory. Proper measures that don’t measure all kinds of other things that we need. Experimentation doesn’t help with this.

Statistical validity - Empirical measurement between x and y. Choosing the right statistical tests and having accurate and reliable results. Assumptions and so forth; error term prerequisites. Experimentation helps a bit with this.

Causal validity This is where experiments really help; it is basically an argument of the estimate being causal in nature and thus not caused by endogeneity.

Question 3

What are the two aspects of external validity presented in the course?

Answer 3

Context dependence - How is an experiment result from Danish voters going to hold for German voters? Can I conclude about other populations than my own?

Ecological validity - How natural is the situation people are in when they are providing you with data?

Question 4

What characterises random sampling?

Answer 4

All subjects must have the same, known probability of being drawn

Question 5

What is exogeneity?

Answer 5

Generally, when something is determined outside the system of interest.

In experiments, the researcher controls an exogenous intervention. This will in theory assure that the variation in the variable of interest is exogenous.

Question 6

What is heterogenous treatment effects?

Answer 6

Variance of the treatment effect across subjects

Question 7

What is endogeneity?

Answer 7

A variable is said to be endogeneous when it is correlated with the error term of the model. Two typical sources in public administration research are 1) self selection and 2) simultaneity.

Question 8

What is simultaneity

Answer 8

“Two way causation” or reverse causation. When the root of simultaneity, statistical control is of little help. If y has a causal effect on x, we cannot control for it.

Question 9

Instrumental variable approach

Answer 9

Statistical remedy for the simultaneity problem. We must get a hold of exactly that part of x that has a causal effect on y but that is not due to y.

1. Find some variable z that is correlated with x (preferably strongly), but which we for some reason know cannot be affected by y.
2. Estimate x from z and use predicted values for x as the independent variable in a regression analysis of y
3. Since we know that y cannot affect z,w e also know that the statistical correlation cannot be due to reverse causality.

Question 10

The three characteristics of causes

Answer 10

• theoretical claim
• empirical difference
• temporal ordery