# Topic 6 - Impacts Evaluation of Development Policies and Programs Flashcards

What is Impact Evaluation?

To determine if and to what extent the change in something is due to intervention of a policy and if the change from that policy was cost effective.

Trying to establish causality

What is the issue in establishing causality between intervention (X) and output (Y)?

The diff types of causalities:

1.CAUSALITY: π β π

2.REVERSE CAUSALITY: π β X

3. Simultaneous change π β π and π β X

4. SPURIOUS CORRELATION/OMITTED VARIABLE BIAS:

ο§ Observed X/Y correlation due to some common Z.

In finding a method what is crucial?

Random selection crucial

Intervention/ Program (P)

Outcome (Y)

What is the ideal impact of an intervention?

(πΏi) = Yi (1) - Yi (0)

Change in outcome in future greater than outcome without intervention

What are some commonly used methods for impact evaluation?

- Experimental design with randomization (Randomised Control Trial (RCT) approach)
- Matching methods
- Difference-in-Differences techniques (Diff-in-diff)
- Roll-out approach and Panel analysis
- Regression discontinuity design (RDD)
- Event analysis approach
- Instrumental variables approach (IV)

What is the RCTs (Randomised Control Tests)

->Randomisation in constructing TREATMENT and CONTROL groups from an eligible population

Two random groups one placebo and one with medicine for instance

How do you measure the impact of a RCT?

Avg treatment effect:

ATE = Y bar t - Y bar c

Y bar t = Avg outcome in Treatment group

Y bar c = Avg outcome in contrl group

What are the potential behavioural problem with randomised experiments?

- Hawthorne effect - modify their behavior simply because they are being observed
- Contamination and Spill-over effects
- John Henry effect:

When control group behave different to make up for the disadv of being the control group - Dropout / Attrition effects - those not who dropped out
- Pioneer / Partial equilibrium effect

What are type 1 and type 2 errors in research (RCT)

Type I βfalse positivesβ β When the tester validates a statistically significant difference even though there isnβt one

Type II - false negative, also known as an error of omission

In what situation may RCT not be applicable?

When the event has already occurred

What is the Propensity Scoring Matching (PSM)?

Select Control (non-participants) group (C) with many observable essential characteristics as Treated group (T)

What is the process of the PSM?

- For each participant (i )

ο§ Find non-participants (mi ) with a very close propensity score (prob of participation) based on several variables (X) - Impact of program is average difference in outcomes between treated i and his/her matched control.

= Treatment on the Treated (ToT) = πoT = 1/ππβπβπ( ππ β ππ(π))

What are the assumptions the Difference-in-difference (DiD) approach?

Make crit assumption about occurence in absense of program

What are the stages the DiD approach take to calculate the approach?

- Change in outcome variable (Y) among participants over some period

βππ = Program Impact + Potential Role of other factors - Compute change in same outcome among non-participants over same period

βπππ = Potential Role of other factors

βππβ βππp = Programme Impact

What is the graph in the DiD display?

How the diff stages work

SEE GRAPH IN NOTES