Program Evaluation

Question 1

used to evaluate the effectiveness of a program

Answer 1

purpose of program evaluation

Question 2

• The social worker may speak with the agency Executive Director and/or program staff to identify the goals of the program. Note: no goals or objectives are created during this time, but we do need to know what the goals of the program were in order to evaluate whether they were met.

Answer 2

Step 1: Identify the goals of the program

Question 3

• Describe the types of clients served by the program, the interventions and practice
  techniques used by staff, and the organizational structure of the agency and program.
• These things will help with step 3 as we seek to understand how the program services
  are delivered as well as possible hindrances to goal achievement.

Answer 3

Step 2: Describe the characteristics of the organization

Question 4

• Process evaluation examines how a program was carried out and whether program
  activities were implemented as intended.
• This could include things like the management of the program, if the agency supported the program well, and which interventions were implemented for the program. All of these factors impact the process of implementation and will in turn end up impacting the overall program outcomes. So process evaluation has to do with implementation whereas ‘outcome’ or ‘summative’ evaluation has to do with overall outcomes (whether
  the goals were met). Here are some examples of process evaluation:
  ○ If clients were supposed to attend X number of therapy sessions, were all sessions offered and attended?
  ○ Did all clients receive the same quality of services from their providers?
  ○ If a goal was to respond to client inquiries within 12 hours, you would need to
  determine ways to find out if this occurred.

Answer 4

Step 3: Engage in process evaluation

Question 5

• Involve key program staff and others who were involved in setting the project’s goals.
• Outcome goals are comprised of data that the agency already keeps track of:
  ○ Example: Number of clients served, satisfaction with services, change in
  symptoms/diagnoses, number of service hours utilized.
• Measurable outcomes must be determined to identify whether the projects’ goals are
  being met.
• A baseline of what clients were experiencing before the program was enacted must be
  established

Answer 5

Step 4: Define outcomes to be measured

Question 6

Outcomes can be measured with two types of data: qualitative and quantitative.
○ Qualitative measurements: Information that is not in numerical form.
Ex. Open-ended surveys, unstructured interviews, observations.
○ Quantitative measurements: collecting data involving numbers that can be
statistically manipulated. Ex. Number of clients, Survey with a Likert scale gauging client satisfaction, scores on tests (such as the Beck Depression
Inventory).

Answer 6

Step 5: Measure the outcomes of the program

Question 7

• Findings should be clear and concise and may include suggestions for
  changes/improvements.
• Positive and negative feedback should both help organizations improve their programs.

Answer 7

Step 6: Report and disseminate findings