# Topic 6 - Impacts Evaluation of Development Policies and Programs Flashcards

1
Q

What is Impact Evaluation?

A

To determine if and to what extent the change in something is due to intervention of a policy and if the change from that policy was cost effective.

Trying to establish causality

2
Q

What is the issue in establishing causality between intervention (X) and output (Y)?

A

The diff types of causalities:
1.CAUSALITY: π β π
2.REVERSE CAUSALITY: π β X
3. Simultaneous change π β π and π β X
4. SPURIOUS CORRELATION/OMITTED VARIABLE BIAS:
ο§ Observed X/Y correlation due to some common Z.

3
Q

In finding a method what is crucial?

A

Random selection crucial
Intervention/ Program (P)
Outcome (Y)

4
Q

What is the ideal impact of an intervention?

A

(πΏi) = Yi (1) - Yi (0)

Change in outcome in future greater than outcome without intervention

5
Q

What are some commonly used methods for impact evaluation?

A
1. Experimental design with randomization (Randomised Control Trial (RCT) approach)
2. Matching methods
3. Difference-in-Differences techniques (Diff-in-diff)
4. Roll-out approach and Panel analysis
5. Regression discontinuity design (RDD)
6. Event analysis approach
7. Instrumental variables approach (IV)
6
Q

What is the RCTs (Randomised Control Tests)

A

->Randomisation in constructing TREATMENT and CONTROL groups from an eligible population

Two random groups one placebo and one with medicine for instance

7
Q

How do you measure the impact of a RCT?

A

Avg treatment effect:
ATE = Y bar t - Y bar c

Y bar t = Avg outcome in Treatment group
Y bar c = Avg outcome in contrl group

8
Q

What are the potential behavioural problem with randomised experiments?

A
1. Hawthorne effect - modify their behavior simply because they are being observed
2. Contamination and Spill-over effects
3. John Henry effect:
When control group behave different to make up for the disadv of being the control group
4. Dropout / Attrition effects - those not who dropped out
5. Pioneer / Partial equilibrium effect
9
Q

What are type 1 and type 2 errors in research (RCT)

A

Type I βfalse positivesβ β When the tester validates a statistically significant difference even though there isn’t one

Type II - false negative, also known as an error of omission

10
Q

In what situation may RCT not be applicable?

A

When the event has already occurred

11
Q

What is the Propensity Scoring Matching (PSM)?

A

Select Control (non-participants) group (C) with many observable essential characteristics as Treated group (T)

12
Q

What is the process of the PSM?

A
1. For each participant (i )
ο§ Find non-participants (mi ) with a very close propensity score (prob of participation) based on several variables (X)
2. Impact of program is average difference in outcomes between treated i and his/her matched control.
= Treatment on the Treated (ToT) = πoT = 1/ππβπβπ( ππ β ππ(π))
13
Q

What are the assumptions the Difference-in-difference (DiD) approach?

A

Make crit assumption about occurence in absense of program

14
Q

What are the stages the DiD approach take to calculate the approach?

A
1. Change in outcome variable (Y) among participants over some period
βππ = Program Impact + Potential Role of other factors
2. Compute change in same outcome among non-participants over same period

βπππ = Potential Role of other factors
βππβ βππp = Programme Impact

15
Q

What is the graph in the DiD display?

A

How the diff stages work

SEE GRAPH IN NOTES

16
Q

Why is Impact Evaluation useful for policymaking?

A
1. Importance of Internal validity and External validity in Impact Evaluation
2. Measures to avoid βdata miningβ or βcherry pickingβ
- Prevalence of βfalse positivesβ
- Selective reporting
- Pre-analyses plan (PAP) requirement: Official independent bodies exist for PAP issues.
3. Importance of Qualitative Methods