Evaluation methods in disasters Flashcards
(22 cards)
What is an evaluation?
an assessment, as systematic & objective as possible, of an on-going or completed project, programme or policy, its design, implementation and results
What is the purpose of evaluations?
- Give chance to reflect on events
- Improve learning
- Improve the response
- Contribute to accountability & transparency
- Improve effectiveness within organisation
- Contribution to dialogue between different actors
- Inform decision-makers
What is monitoring?
Systematic & regular process of collecting, verifying and triangulating information to assess progress made against project outputs & activities; bearing in mind accountability to affected populations and donors
Monitoring vs evaluations
- Information gathered is used to make informed decisions & strengthen partnership and coordination.
- Monitoring helps you to do robust evaluations
What are the challenges with evaluations of humanitairan assistance?
- Urgency and Chaos
- Insecurity and lack of access or remote locations with damaged infrastructure
- Lack of baseline data → In emergencies objectives tend to change and LFA frameworks may be missing
- High staff turnover
- Conflicts and polarised perspectives → Breakdown of trust and politicised aid
- Time pressure on humanitarian workers and of affected population
- Over ambitious terms of reference (ToR) and limited resources and time
What are the types of evaluations?
- Process evaluation (activities) - formative
- Impact evaluation - summative
- Most evaluations include aspects of both impact & process
What is an impact evaluation?
- Focuses on impact of the aid, rather than on aid delivery.
- Usually, but not always, carried out some time after project completion.
What are the different levels of evaluaiton?
- thematic
- transversal
- meta-evaluation
who does the evaluations?
1. Self-evaluation
* Within the team – ideally with participation from “beneficiaries”
2. Internal evaluation by organisation
* Often Lessons Learnt focus
3. External – by consultant or donors
* Often Accountability focus
What is the evaluation criteria from ALNAP?
- Relevance & appropriateness
- Connectedness or continuity
- Coherence
- Coverage
- Efficiency
- Effectiveness
- Impact
Explain relevance criteria
Relevance - are/were the real needs addressed? → relating to:
* identified problems/REAL needs
* how were needs identified
Explain appropriateness criteria
- are/were things done right?
- Is the strategy appropriate in order to achieve the objectives?
Refers to:
* Approach & strategies in particular context
* Objectives set
Explain the coherence criteria
Need to assess security, developmental, trade and military policies as well as humanitarian policies, to ensure that there is consistency and, in particular, that all policies take into account humanitarian and human- rights considerations
Explain the connectedness or conintuity criteria
- Is a phasing out strategy designed and achieved? What does it consist of?
- Which interconnected and longer-term problems can be identified, and how have they been taken into consideration?
- Which local capacities and resources were identified? How does the project connect with those?
- Instead of sustainability
Explain the coverage criteria
- Is/was the programme reaching population groups facing life-threatening suffering wherever they are.
Explain efficiency criteria
- Were inputs and resources used appropriately and to their maximum potential?
- Were things done in the best way possible?
- Refers to: Relationship between (verifiable) input and output (qualitative and quantitative)
Explain effectiveness criteria
To what extent have the agreed objectives been achieved?
* Are/were the right things done?
Refers to:
* Progress towards objectives
* Activities and output – in relation to objectives
* Outcome
What does effectiveness criteria include?
- Timeliness
- Coordination
Explain impact criteria
- Is the program making a difference?
Refers to:
* Wider effects of the project – social, economic, technical, environmental – on individuals, communities and institutions.
* Positive and negative impact
What are the ethical framework components?
- research
- humanitarian
- evaluation
What are important factors for research?
- Respect (infomred consent)
- Beneficience (risks & benefits)
- Justice (selection of subjects)
