Chapter 20: Program Evaluation Techniques

Question 1

evaluation is

Answer 1

judging the worth of program services, based on an analysis of systematically collected evidence

Question 2

many programmer evaluate their program services

Answer 2

informally
there is a move to more formal evaluation procedures

Question 3

evaluating program services can serve the programmer and the organization in

Answer 3

establishing accountability, in program development, and in organizational management

Question 4

often evaluation is imposed on a programmer by

Answer 4

a board, third party funding agency or a higher level administrator

Question 5

evaluation being imposed by another party can result in it as being viewed

Answer 5

as an add on responsibility that has been imposed on the programmer or as a procedure undertaken with some trepidation because it may result in adverse view of programmer’s efforts

Question 6

evaluation data can verify that a program us

Answer 6

accomplishing its intended results

Question 7

evaluation improves the

Answer 7

credibility of the organization and its programs with its participants

Question 8

evaluation results can be reorganized and used for

Answer 8

public relations and publicity

Question 9

evaluation data can improve and refine programs by

Answer 9

determining what works, what does not work, and why things work the way they do

Question 10

evaluation data can boost

Answer 10

staff morale

Question 11

evaluation can often result in

Answer 11

the discovery of new needs

Question 12

evaluation will help managers answer the question:

Answer 12

is this the best use of these resources in this organization at this time?

Question 13

evaluation can help the organization assess which

Answer 13

programs it should keep and which should be discontinued

Question 14

programs that are continued each year without

Answer 14

sufficient review and revision become tired and outdated

Question 15

evaluation is not

Answer 15

a single activity or procedure

Question 16

most leisure service organizations will not have any

Answer 16

one evaluation procedure that can meet all of the organizations evaluation needs
it is important to have several methods operating simultaneously

Question 17

evaluation planner

Answer 17

purpose?
audience?
process?
what issues should address?
resources?
what evidence should be collected?
data gathering?
analysis?
reporting?

Question 18

purpose: why evaluate?

Answer 18

program development
organizational development
organizational accountability

Question 19

audience: who is evaluation for?

Answer 19

• consider evaluation team
• a persons position determines type of info they deem relevant
• external funding agencies, the public, clients, national association, providers of similar services

Question 20

process: how will the evaluation be conducted

Answer 20

process models: use goals and objectives
preordinate models: based on standards

Question 21

process model

Answer 21

identifies a procedure for conducting the evaluation but does not identify the criteria on which judgments are made

Question 22

in process models

Answer 22

programmer needs to provide the criteria on which judgements will be made

Question 23

preordinate models

Answer 23

provide the technique for accomplishing the evaluation and the criteria on which judgements of worth will be made
standards are the criteria

Question 24

issues: what questions should the evaluation address

Answer 24

address validity and values
questiosn asked myst be true indicators of the values asserted for the program

Question 25

what can programmers use to create evaluation questions

Answer 25

the 5 P's not advisable to evaluate everything at same time, use select areas that relate best to program

Question 26

5 P's of evaluation

Answer 26

participants program place policies/administration personnel

Question 27

participants

Answer 27

Motivations/Satisfactions, changes in attitudes, changes in knowledge, changes in skills and abilities, carryover into other situations, how individuals interact.

Question 28

program

Answer 28

Effective leadership, promotion of program, if participants gained anything, risk management

Question 29

place

Answer 29

Safety concerns, master planning, adequate facilities

Question 30

policies/administration

Answer 30

Accountability of budget, cost-benefit analysis, cost-effectiveness analysis, equitable provision of services

Question 31

personnel

Answer 31

Performance appraisal, assess training needs, provide feedback for improvement

Question 32

resources: what resources are available for evaluation?

Answer 32

- understand costs to evaluation - use data agency already collects on regular basis -evaluatiion does not always require collection of new data

Question 33

other reseources needed for evaluation

Answer 33

Staff time, printing expenses, collecting and analyzing data, preparing and distributing final reports. colleagues, universities, cooperative extension agencies and consultants.

Question 34

what evidence should be collected?

Answer 34

- any info that may be used in making judgments about issues raised - sources and quality of evidence - ensure represents a balanced and comprehensive view of program

Question 35

data gathering: how is the evidence to be collected?

Answer 35

Questionnaires, interviews, conversations, participant observation, checklists, rating scales.

Question 36

what way of data gathering is surest way to obtain a sample that is representative of the group

Answer 36

random sampling

Question 37

analysis: how can the evidence be analyzed

Answer 37

must be analyzed to determine what they indicate about a program

Question 38

if the evaluation instrument cannot be completed in less than

Answer 38

10 minutes, it is probably too long and needs to be further revised and focused.

Question 39

in all data analysis teh primary objective is to

Answer 39

develop information that provides meaningful insight about the results and effect of a program so the worth of the program is made apparent

Question 40

reporting: how can evaluation findings be reported

Answer 40

most common is written report others are testimonies, photographs, videos

Question 41

body of the report

Answer 41

Purpose of the evaluation Evaluation questions Description of the program Evaluator’s background Summary of regulations Data-gathering and analytical methods Findings Use of the report Contract Conclusions Minority Reports Recommendations Additional Considerations

Question 42

credibility is built when programmers

Answer 42

share evaluation results of all types including successes, modifications, and cancellations