Experimental Evaluation by Aloysius Ssennyonjo Flashcards
(14 cards)
What are the criteria for inferring causality?
- Temporal Relationship
- Plausibility
- Strength of the Association
- Consistency
- Dose-Response Relationship
- Reversibility
These criteria help establish a causal link between an intervention and its effects.
Define internal validity.
Confidence that results accurately depict causal relationships.
Internal validity ensures that the observed effects are due to the intervention rather than other factors.
What are the threats to internal validity?
- History
- Maturation
- Testing & Instrumentation
- Selection Bias
- Loss to Follow-up
- Diffusion of Treatments
These threats can compromise the reliability of the evaluation results.
What is external validity?
Generalizability of causal relationships beyond evaluation conditions.
External validity assesses whether findings can be applied to other settings or populations.
What are the types of evaluation?
- Formative Evaluation
- Process Evaluation
- Impact Evaluation
Each type serves a different purpose in the evaluation process.
What is the strongest evaluation design for demonstrating causality?
Experimental Design.
This design typically involves random assignment to enhance internal validity.
What is a Randomized Controlled Trial (RCT)?
- Review previous knowledge
- Select participants/sites
- Create experimental and control groups
- Random allocation
- Measure outcomes
RCTs are a gold standard in research for establishing causal relationships.
What is a quasi-experimental design?
Used when random assignment is not possible, providing moderate support for causal inferences.
These designs often involve pre-test and post-test comparisons.
What are the strengths of experimental designs?
- High internal validity
- Clear causal inference
Experimental designs are robust but may face ethical concerns and cost issues.
What are the limitations of non-experimental designs?
- Cannot account for confounding factors
- Causal attribution not possible
Non-experimental designs are the weakest for demonstrating causality.
What is the purpose of constructing a plausibility argument?
To describe trends in intervention coverage, intermediary outcomes, impact outcomes, and contextual factors.
Linking these trends helps to establish a coherent narrative for the evaluation.
Fill in the blank: A _______ design is the weakest for demonstrating causality.
Non-Experimental Design
Non-experimental designs lack control groups and randomization.
What general questions should be considered for evaluation design?
- What is your evaluation question?
- What is your target population?
- How do you intend to use the results?
- What indicators do you want to measure?
- What type of inference do you want to draw?
- When do you need the results?
- Do you have a sampling frame? What shape is it in?
- How much are you willing to pay?
- Where in the program life cycle are you now?
These questions guide the evaluation planning process.
What is the role of case studies in evaluation?
They provide practical examples, such as reducing malaria transmission through IRS and antimalarial drug policy change.
Case studies help illustrate the application of evaluation designs in real-world contexts.
