Chapter Six

Training Evaluation
 Objectives
O Explain why evaluation is important

O Identify and choose outcomes to evaluate a

training program

O Discuss the process used to plan and

implement a good training evaluation

O Discuss the strengths and weaknesses of

different evaluation designs

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Key Terms
O Training effectiveness: benefits derived
training
O Training evaluation: process of
determining training effectiveness
O Training outcomes: measures to evaluate
training effectiveness
O Evaluation design: data will be collected
and which data will be collected for
training evaluation
Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Formative Evaluation
O Refers to evaluation of training that takes
place during program design

O Helps to ensure training is well organized,

runs smoothly, and that trainees learn and
are satisfied

O Provides information on how to make a

program better

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Summative Evaluation
O Refers to evaluation conducted to
determine if training has lead to desirable
outcomes

O Examines whether trainees have improved

or acquired knowledge, skills, attitudes,
behaviors, or other outcomes

O Includes examining the business impact of

training

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Importance of Evaluation
O There are multiple reasons to evaluate:
o identify a programs strengths and
weaknesses
o assess what features of training content
and context matter
o identify which trainees benefited
o gather information for marketing
training
o determine financial benefits and costs

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 The Evaluation Process

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Training Outcomes

Learning & Behavior &

Reactions Cognitive Skill-Based
Outcomes Outcomes

Affective Return on
Results
Outcomes Investment

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Reactions
O Trainees perceptions of the training
experience relating to the content,
facilities, trainer, and methods
O Key questions to consider include:
o Did the trainees like the program?
o Did the environment help learning?
o Was the material meaningful?
O Typically measured at the end of training

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Learning & Cognitive
Outcomes
O Relate to familiarity with information,
including principles, facts, techniques,
procedures, and processes
O Typically measured via paper-and-pencil
tests and self-assessments
O Tests often preferred over self-assessments

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Behavior & Skill-Based
Outcomes
O Relate to proficiency with technical skills,
motor skills, and behavior
O Include learning and transfer
O Learning often assessed via work samples
O Transfer often assessed via observation or
managerial/peer ratings

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Affective Outcomes
O Include attitudes and motivation
o e.g., self-efficacy, employee engagement,
motivation to learn, tolerance for
diversity, attitudes toward safety

O The attitude of interest depends on the

training objectives

O Affective outcomes often measured via

surveys
Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Results
O Did training have an impact on meaningful
business outcomes?

O Outcomes used to determine the benefits

of training to the company
o e.g., reduced costs, increased employee
retention, increased sales, improved
quality or customer service

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Return on Investment
O Involves comparing the training program's
benefits to its costs in monetary terms
o Benefits: value the company gains

o Direct costs: salaries for employees

involved in training, program materials,
facilities, and travel
o Indirect costs: costs not related directly
to design and delivery
Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Which Outcomes?

Relevance Reliability

Discrimination Practicality

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Relevance
O The extent to which training outcomes are
related to the learned capabilities
emphasized in the program
O Contamination refers to the inclusion of
inappropriate or irrelevant outcomes
O Deficiency refers to the omission of
important information

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Deficiency, Relevance, &
Contamination

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Reliability
O The extent to which outcomes can be
measured consistently over time

O Evaluators are concerned with consistency

over time, such that items do not change
in meaning or interpretation over time

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Discrimination
O The extent to which measured
performance reflects a true difference
O We want tests that discriminate between
high and low performance
o A test that is too easy may not
discriminate
o In this instance, both high and low
performers would do well and appear
good even though they are not
Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Practicality
O The extent to which outcomes can be
easily measured and collected

O Companies often claim that measurement

is too burdensome

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Training Evaluation Practices

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Which Outcomes?
O While collecting data on all outcomes is
ideal, it may not always be necessary
O Consider the scope of the training and
practical considerations
O Do not assume that positive reactions lead
to transfer

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Threats to Validity
O Threats to validity refer to factors that will
lead an evaluator to question the results

O Internal validity is the believability of the

study

O External validity is the generalizability of

the evaluation results to other groups and
situations

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Controlling for Threats
O There are three ways to control for threats:
o Use pre-tests and post-tests
o Use a control group
o Random assignment of employees to
control and training groups

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Post-test Only
O Involves collecting only post-training
outcomes

O Appropriate when trainees can be

expected to have similar levels of
proficiency prior to training

O Strengthened with a control group

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Pre-test / Post-test
O Involves collecting both pre-training and
post-training outcomes to determine if a
change has occurred

O Is there a difference in outcomes before

and after training?

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Pre-test / Post-test with
Comparison Group
O Includes pre-training and post-training
outcomes and use of a control group

O If the post-training improvement is greater

for the training group, there is evidence
that training was responsible

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Time Series
O Involves collecting measures at periodic
intervals pre- and post-training

O A comparison group may be used

O The strength of this design can be

improved by using reversal, which refers to
a time period when participants no longer
receive training

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Solomon Four-Group
O Combines the pre-test/post-test
comparison group design and the
posttest-only control group design
o Pre-test, treatment, post-test
o Pre-test, no treatment, post-test
o No pre-test, treatment, post-test
o No pre-test, no treatment, post-test

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 When Evaluation May Not
Be Necessary
O Time constraints

O Managers and trainees may lack expertise

O The company may view training as an

investment from which it expects little or
no return

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 When Evaluation is a Must
O The training is ongoing and has the
potential to affect many employees
O The training program involves multiple
classes and a large number of trainees
O The expertise exists to evaluate
O The cost of training is significant
O There is sufficient time and interest

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 Cost-Benefit Analysis
O Companies may desire to quantify whether
the benefits of training outweigh the costs

O Cost-benefit analysis
o Process of determining the economic
impact of training using accounting
methods that look at training costs and
benefits

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
 TUTORIAL QUESTIONS
1) Identify FIVE (5) evaluation process.
2) List and discuss SIX (6) categories of training
outcomes.
3) List FOUR (4) items to determine whether
outcomes are appropriate.
4) List FOUR (4) reason why is training cost
information important.

Copyright 2017 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.