Lecture 5: Measuring Impact Flashcards
(21 cards)
Why is measing social value hard? (3)
- people do not agree about what the desired outcome should be…
- people’s ethics, morals, and priorities vary…
- many social value metrics are inherently unreliable…
What do most metrics assume about value?
most metrics assume that value is objective, and therefore discoverable through analysis
When can better metrics for capturing social value be created?
when people approach social value as subjective, malleable, and variable, they create better metrics to capture it
In what 4 tool categories can benefits of measuring impact be categorized? And what are the benefits per tool?
- Diagnosis tool
-What actions work best to achieve outcomes
-What unintended consequences have there been - Decision-making tool
-What needs to be adjusted to improve execution
-Where to put the resources into - Learning tool
-How and why, they are, or are not achieved
-Used to test model before scaling-up - Collaboration tool
-Provide partners with evidence & feedback
-Make a stronger case to your stakeholders (“worthy of investment”)
From 1900 on accountability (through transparency) is a way to prove that you are making a difference. How did this manifest itself?
- Demands from funders, taxpayers, citizens for transparency
- Establishing legitimacy for fund raising, for spending, and on proper governance
- Philanthropy performance, success measured by the number of grants allocated
Besides accountability, from 2000 on impact (through measurement - market approach) is another way to prove that you are making a difference. How does that manifest itself?
-Focus on outcomes
-Demonstrating results in addressing complex social problems (Poverty, inequality)
-Driven by funders, and SEs along with the establishment of norms, auditing, & greater professionalization
What is the golden standard for evaluating impact?
randomized control trials
Name different methods for assessing social/environmental performance. (7)
- Monetizing social value (Cost-benefit analysis)
- Expected return (Social Return On Investment - SROI)
- Experimental methods (Randomized control trials)
- Strategy performance approach (Balanced scorecards, KPI approach)
- Stakeholder approach (Participatory methods-beneficiaries)
- Integrative approaches (System thinking)
- Theory of Change & Logic model
What are the 6 activities in the process of calculating the Social return on investment (SROI) of Mook 2015?
o Establishing scope and identifying key stakeholders
o Mapping outcomes
o Evidencing outcomes and giving them a value
o Establishing impact
o Calculating the SROI
o Reporting…
What are the Pro’s of the SROI method?
- Useful supplementary tool to communicate value creation
- Information on inputs of resources
- Enables broad predictions
What are the cons of the SROI method?
- Arbitrary estimation (costs and outputs)
- Approximate range of impact
- Only partial outcomes measured (for two groups)
Why use the Theory of Change as a method of assessing the effectiveness / social/environmental performance?
- The foundation for what an organization does, and why it does.
- To understand how you “intend” to do good
- The causal logic or “pathway” to a long-term goal
- To identify the preconditions necessary to achieve the goal
- To identify appropriate indicators for measuring results (blueprint for evaluation)
- To communicate/agree with stakeholders
- Used as a strategic planning, monitoring & evaluation, learning, and communication tool
What are the five key steps to develop Theory of Change (ToC)
- set the ultimate goal (‘‘change”)
- analyze & identify the problems (through a problem tree)
- Identify what is needed to happen (Logic model & backward mapping of preconditions)
- Establish key assumptions
- Identify key partnes and actors
For what does the problem tree primarly help?
developing the cause-effect relationship of the problem
it helps writing a ‘‘If …then …’’ narrative
What do the roots, stem and leaves of the tree represent?
roots = causes:
- immediate cause
- secondary cause (the cause of the immediate cause)
stem = main problem
leaves = consequences
- immediate effect
What do we use the Logic Model for? (5)
- A graphic illustration of the relationship between resources, activities, and their intended effects
- Often combined with ToC
- To “understand what specific steps to do” to achieve the mission
- Limited attention to causal logic
- Used as a program (activity) planning, implementation, monitoring & evaluation, and communication tool
What are the characteristics of the logic model? (5)
- A single picture
- Defined steps to reach the goal
- Directly reflecting your activities
- Linear sequence
- Often in a pre-defined format
What do ‘inputs’, ‘activities’ and ‘outputs’ represent in the logic model?
- Inputs are the resources that go into a program or intervention
- Activities are events undertaken by the program or partners to produce desired outcomes
- Outputs are the direct, immediate results of activities
If the benefits (outcomes) are achieved, then certain changes in society/groups/communities are expected to occur. How do we call these changes in general?
impact
Aren’t ToC and Logic Model the same?
No, be they are often used interchangeably in combination
ToC vs. LM
stategic level vs. ….
how vs. …
why and how changes occur vs. ….
complex with many possible pathways vs. ….
ToC vs. LM
stategic level vs. implementation level
how vs. what
why and how changes occur vs. what you expect to do
complex with many possible pathways vs. sequential and linear
