Topic 6 - Impacts Evaluation of Development Policies and Programs Flashcards Preview

20ECC119 - Development Economics > Topic 6 - Impacts Evaluation of Development Policies and Programs > Flashcards

Flashcards in Topic 6 - Impacts Evaluation of Development Policies and Programs Deck (16)
Loading flashcards...
1
Q

What is Impact Evaluation?

A

To determine if and to what extent the change in something is due to intervention of a policy and if the change from that policy was cost effective.

Trying to establish causality

2
Q

What is the issue in establishing causality between intervention (X) and output (Y)?

A

The diff types of causalities:
1.CAUSALITY: 𝑋 β†’ π‘Œ
2.REVERSE CAUSALITY: π‘Œ β†’ X
3. Simultaneous change 𝑋 β†’ π‘Œ and π‘Œ β†’ X
4. SPURIOUS CORRELATION/OMITTED VARIABLE BIAS:
 Observed X/Y correlation due to some common Z.

3
Q

In finding a method what is crucial?

A

Random selection crucial
Intervention/ Program (P)
Outcome (Y)

4
Q

What is the ideal impact of an intervention?

A

(𝛿i) = Yi (1) - Yi (0)

Change in outcome in future greater than outcome without intervention

5
Q

What are some commonly used methods for impact evaluation?

A
  1. Experimental design with randomization (Randomised Control Trial (RCT) approach)
  2. Matching methods
  3. Difference-in-Differences techniques (Diff-in-diff)
  4. Roll-out approach and Panel analysis
  5. Regression discontinuity design (RDD)
  6. Event analysis approach
  7. Instrumental variables approach (IV)
6
Q

What is the RCTs (Randomised Control Tests)

A

->Randomisation in constructing TREATMENT and CONTROL groups from an eligible population

Two random groups one placebo and one with medicine for instance

7
Q

How do you measure the impact of a RCT?

A

Avg treatment effect:
ATE = Y bar t - Y bar c

Y bar t = Avg outcome in Treatment group
Y bar c = Avg outcome in contrl group

8
Q

What are the potential behavioural problem with randomised experiments?

A
  1. Hawthorne effect - modify their behavior simply because they are being observed
  2. Contamination and Spill-over effects
  3. John Henry effect:
    When control group behave different to make up for the disadv of being the control group
  4. Dropout / Attrition effects - those not who dropped out
  5. Pioneer / Partial equilibrium effect
9
Q

What are type 1 and type 2 errors in research (RCT)

A

Type I β€œfalse positives” – When the tester validates a statistically significant difference even though there isn’t one

Type II - false negative, also known as an error of omission

10
Q

In what situation may RCT not be applicable?

A

When the event has already occurred

11
Q

What is the Propensity Scoring Matching (PSM)?

A

Select Control (non-participants) group (C) with many observable essential characteristics as Treated group (T)

12
Q

What is the process of the PSM?

A
  1. For each participant (i )
     Find non-participants (mi ) with a very close propensity score (prob of participation) based on several variables (X)
  2. Impact of program is average difference in outcomes between treated i and his/her matched control.
    = Treatment on the Treated (ToT) = 𝑇oT = 1/π‘π‘‡βˆ‘π‘–βˆˆπ‘‡( π‘Œπ‘– βˆ’ π‘Œπ‘š(𝑖))
13
Q

What are the assumptions the Difference-in-difference (DiD) approach?

A

Make crit assumption about occurence in absense of program

14
Q

What are the stages the DiD approach take to calculate the approach?

A
  1. Change in outcome variable (Y) among participants over some period
    βˆ†π‘Œπ‘ = Program Impact + Potential Role of other factors
  2. Compute change in same outcome among non-participants over same period

βˆ†π‘Œπ‘›π‘ƒ = Potential Role of other factors
βˆ†π‘Œπ‘βˆ’ βˆ†π‘Œπ‘›p = Programme Impact

15
Q

What is the graph in the DiD display?

A

How the diff stages work

SEE GRAPH IN NOTES

16
Q

Why is Impact Evaluation useful for policymaking?

A
  1. Importance of Internal validity and External validity in Impact Evaluation
  2. Measures to avoid β€˜data mining’ or β€˜cherry picking’
    - Prevalence of β€˜false positives’
    - Selective reporting
    - Pre-analyses plan (PAP) requirement: Official independent bodies exist for PAP issues.
  3. Importance of Qualitative Methods