Four Levels of Evaluation Flashcards

1
Q

What is effective training?

A

Training that provides relevant knowledge and skills to participants and the confidence to apply them on the job.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is training effectiveness?

A

Training and follow-up leading to improved job performance that positively contributes to key organizational results.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

How should training requests be considered?

A

As invitations to the conversation about increasing performance and improving results.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

At which stages should we involve ourselves in the training program?

A
  • Before
  • During
  • After
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What are the four levels of the New World Kirkpatrick Model?

A
  • Reaction
  • Learning
  • Behavior
  • Results
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What does the ‘Reaction’ level in the New World Kirkpatrick Model measure?

A

The degree to which participants find the training favorable, engaging, and relevant to their jobs.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What does the ‘Learning’ level in the New World Kirkpatrick Model measure?

A

The degree to which participants acquire the intended knowledge, skills, attitude, confidence, and commitment based on their participation in the training.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What does the ‘Behavior’ level in the New World Kirkpatrick Model measure?

A

The degree to which participants apply what they have learned during the training when they are back on the job.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What does the ‘Results’ level in the New World Kirkpatrick Model measure?

A

The degree to which targeted outcomes occur as a result of the training and the support and accountability package.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is the ultimate level 4 result for any organization?

A

The mission.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What question should you ask to check if you have the right mission statement?

A

Is this what the organization exists to do, deliver, or contribute to its customers and society at a high level?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are leading indicators?

A

Short-term observations and measurements that suggest critical behaviors are on track to create a positive impact on desired results.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are some examples of leading indicators?

A
  • Monthly sales volume
  • Defective rate
  • Retention
  • Satisfaction scores
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are critical behaviors?

A

Specific actions that, if performed consistently on the job, will have the biggest impact on desired results.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What are required drivers?

A

Processes and systems that reinforce, monitor, encourage, and reward the performance of critical behaviors on the job.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What percentage of learning happens on the job?

A

70%

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

List the five learning components.

A
  • Knowledge: “I know it.”
  • Skill: “I can do it right now.”
  • Attitude: “I believe this will be worthwhile to do on the job.”
  • Confidence: “I think I can do it on the job.”
  • Commitment: “I will do it on the job.”
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is often inaccurately diagnosed as a lack of knowledge and skill?

A

Poor performance.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What should customer satisfaction be used for in training?

A

To identify and eliminate barriers to learning in the program.

20
Q

What does engagement measure in the ‘Reaction’ level?

A

The degree to which the participants are actively involved in and contributing to the learning experience.

21
Q

What does relevance measure in the ‘Reaction’ level?

A

The degree to which the participants will have the opportunity to use what they learned in training on the job.

22
Q

Levels 1 and 2 of the New World Kirkpatrick Model are about _______.

A

effective training

23
Q

Levels 3 and 4 of the New World Kirkpatrick Model are about _______.

A

training effectiveness

24
Q

What are the two main considerations when designing a training program?

A

Usefulness and credibility

25
What does usefulness in a training program evaluation refer to?
Information used to make decisions related to the program
26
What does credibility in a training program evaluation refer to?
Information demonstrating program value in business terms defined by stakeholders
27
When should programs be evaluated on all four levels?
* Large in scale * Stakes are high * Expensive * Align with corporate goals
28
What are the three phases of evaluation activities for programs?
* Planning * Execution * Demonstration of value
29
What is the first step of the planning phase?
Identify the level 4 result of the organization
30
During the planning phase, what should you discuss with stakeholders after identifying the level 4 result?
How the program contributes to that result or what leading indicators will positively influence it
31
What is Return on Expectations?
What a successful training initiative delivers to key business stakeholders, demonstrating the degree to which their expectations have been satisfied
32
What should be determined with stakeholders regarding success?
What success looks like and how it will be measured
33
What are the success factors in a training program?
* Required drivers * Training program * Personal responsibility * Necessities for success
34
What is the third step of the planning phase?
Define critical behaviors and required drivers
35
What should you ask employees about during the planning phase?
* Their work * Their goals and targets for the year * Business challenges they are experiencing * Causes of performance reduction
36
What should be done once critical behaviors are known?
Discuss the interventions that might solve the challenges
37
What is the fourth step of the planning phase?
Set specific expectations for post-training and define roles and responsibilities. Create a schedule.
38
What are the necessities of success?
Prerequisite items, events, conditions, or communications that will help leverage success or avoid problems
39
What is the fifth step of the planning phase?
Design the intervention and determine if training is needed, and which critical behaviors it will focus on
40
What is the sixth step of the planning phase?
Design the evaluation tools for the program
41
What is blended evaluation?
When data is collected from multiple courses using multiple methods for all four levels
42
What is the first step of the execution phase?
Execution of the training program and associated activities
43
During the execution phase, what materials should be given to managers and supervisors?
Materials to introduce training to direct reports, outlining expectations for before, during, and after
44
How should training be tied to performance during the program?
By discussing ideas and any concerns, and emphasizing the relevance
45
How should support and accountability be ensured after training?
Managers should check in with direct reports on how the training went, what they learned, and how they will apply it
46
What should be compiled in the demonstration of value phase?
Reports of value using storytelling, visual dashboards, or graphics