Measuring and Improving

the effectiveness of
Training: ISTD

Dr. B. Rathan Reddy

bratanreddy@yahoo.com
9849685350
1
 HARVARD BUSINESS REVIEW
July – Aug 1990, Pages 71 - 83

 Motorola has reported that the

return on training was $33 for every
dollar spent, where absorption and
application was effective.

3
 RAY STATA
Chairman, Analog Devices, USA

 The rate at which individuals and

organisations learn may become
the only sustainable competitive
advantage.

4
 BAJAJ AUTO

 Bajaj Auto has recruited first Head

of HRD in 1997 (Personnel was
separate) when they were doing
well.

5
Pioneers of HRD in INDIA
SBI
L&T
SAIL
SUNDERAM FASTENERS
TISCO
TELCO
NTPC
TCS
6
RELIANCE EDUCATION
PROGRAMME
 16 weeks at 11M-B
10 weeks conceptual
3 weeks project
3 weeks analysis and presentation
 NMDC Executive learning
10 to 15 weeks at 11M
 14 Weeks programme at 11M-A is attended
by participants from many industries

7
 HRD INTERVENTIONS at
SCCL
Average training mandays

Workmen & clerks - 1.50 approx

Supervisory - 1.75 approx
Executives - 2.72 approx

TISCO Collieries – 7 days for all

on full day release basis

8
Recommendation by HODs & SCCL

Non Executives - 3 mandays

Executives - 8 mandays

TISCO Collieris - 7
NMDC - 3

9
Guidelines for ROI Measurement
 Create an awareness for ROI in the organization
 Build capacity for ROI by training staff to understand ROI
 Quantify information before the training in order to obtain a
baseline (e.g. number of accidents, customer complaints, etc)
 Convert this data to financial value (e.g. the cost of preventing
an accident)
 Allocate resources for ROI
 Develop a culture of measurement and accountability among
training staff and managers
 Start with only one course as a pilot programme to practice
ROI skills.
 Communicate the results to training staff and the whole
organization
 Design improvement plans for training programmes in order to
increase ROI
 Once ROI results are available, use the data to market future
learning programmes
10
Responsibility for the evaluation
of training programs
 Senior Management
 The trainer
 Line management
 The training manager
 The trainee

11
 Total Evaluation Process
 Planning, design and preparation of the training
programs against the objectives
 Provision of the agreed training programs
 Pre-course briefing meeting between learner
and line manager
 Interim validation as programme proceeds
 Completion of end-of-training programs
reactionnaire
 Assessment of ROI
 Final implementation review meeting
 Review meetings to discuss progress of
implementation

12
ΙΙ. BRAIN STORMING: EVALUATION

• WHY?

•WHO NEEDS?

•WHEN?

•METHODS OF EVALUATION?

• WHAT TO EVALUATE?
•Weather/joke-classroom 7%
•make/break(top-shopfloor)
•trainer has not taught
13
 GENERAL: OBJECTIVES
CONTENT/ RELEVANCE
READING MATERIAL
ADMINISTRATIVE ARRANGEMENTS
STRENGTH/WEAKNESS
APPLICATION:
REPEAT/PEERS/SUBORDINATES/BOSS
LENGTH OF THE COURSE
FACULTY MIX
SUGGESTIONS AND COMMENTS!

 ANY CHALLENGE YOU HAVE FACED TO PROVE

TRAINING RESULTS BY YOUR TOP MANAGEMENT?

14
 LEVELS OF EVALUATION

1. REACTION LEVEL : HOW DO YOU FEEL ABOUT THE

PROGRAM?
2. LEARNING LEVEL: WHAT SKILLS AND KNOWLEDGE
HAVE YOU ACQUIRED IN THIS
PROGRAMME?
3. BEHAVIOUR LEVEL: WHAT CHANGES IN BEHAVIOUR/
ATTITUDES CAN BE CONTRIBUTED
THROUGH THE PROGRAMME?
4.RESULTS LEVEL :* WHAT TANGIBLE GAINS TO THE
ORGANISATION?
* INCREASED PRODUCTIVITY?
* BETTER TEAM WORK? REDUCED COST?
15
5. OUTCOME LEVEL: BY-PRODUCTS WITH AS SOCIAL
CONNECTIONS, SATISFACTION OF PERSONAL
NEEDS, CAREER DEVELOPMENT, ETC.

1&2 LEVELS -INCREASED KNOWLEDGE

-IMPROVED SKILLS
-LEARNING

3,4&5 OTHER LEVELS- BETTER RESULTS

BY CHANGING BEHAVIOUR

16
 EXRCISE 1: APPROPRIATE METHODS FOR VARIOUS
LEVELS OF EVALUATION

1.REACTION LEVEL
HOW DO YOU FEEL ABOUT THE PROGRAMME?

•FORMAT -QUESTIONING PARTICIPANT

•WRITTEN COMMENT -QUESTTIONAIRE
•GET 100% IMMEDIATE
RESPONSE - NON-VERBAL COMMUNICATION
•GET HONEST RESPOSE - EXPRESSION OF FEELINGS
-OPPRTUNITY FOR
QUESTIONS
•SIGNATURE (OPTIONALS)

SECOND REACTION QUESTIONNAIRE AFTER

17
2-3 MONTHS OF TRAINING
GUEST FACULTY(LETTER)
POSITIVE REACTION
REACHES TO TOP BOSSES

LACK OF MOTIVATION TO INDIVIDUAL

LEFT ORGN-MOST SUCCESSFUL IN
THEIR OWN VENTURES--IDPL, BHEL,
ECIL

18
 2. LEARNING LEVEL

WHAT SKILLS AND KNOWLEDGE HAVE YOU

ACQUIRED IN THIS PROGRAMME ?
USE A CONTROL GROUP -Formal Examination

GET 100% RESPONSE -Questionnaire

KNOWLEDGE--WHEN? -Pre testing and post testing

SKILLS-IMPROVED? -On- the-job performance

ATTITUDE-CHANGED? -Participation in group work

LEADS TO LEVEL-3 -Evaluation by superiors

BEHAVIOUR AND BETTER MGT. PRACTICES

19
IMPROVED PROBLEM SOLVING (10-12 MONTHS)
IPE-- BRADFORD EXP. VERY SUCCESSFUL
JOB IMPROVEMENTS PLANS 2-100 TIMES-
ACTUAL SAVINGS

FIRST TWO LEVELS : (10---16 WEEKS DURATION)

INCREASE KNOWLEDGE
IMPROVE SKILLS FIRST TWO LEVELS
CHANGE ATTITUDES

ONLY SKILLS CAN BE DEVELOPED NOT VALUES

SKILL Vs VALUES-DELEGATION-ARJUNA ASHWATHAMA
-LEARNER HAS NOT LEARNED: TEACHER HAS NOT TAUGHT
-FINGER-POINT TO SOME ONE, TRAINER SELF EVALUATION

20
 3.BEHAVIOUR LEVEL

WHAT CHANGES IN BEHAVIOUR / ATTITUDES CAN BE

ATTRIBUTED TO THE PROGRAMME

SELF ASSESMENT EXERCISE *USE A CONTROL GROUP

* ALLOW TIME FOR
BEHAVIOUR CHANGE

SELF APPRAISAL *EVALUATE BEFORE AND

AFTER THE PROGRAMME
IF PRACTICAL

INTERVIEWS *GET 100%RESPONSE

21
REACTION OF JUNIOR,
PAIRS AND SUPERIORS *CONSIDER Vs BENEFITS

OBSERVATION OF *APPRAISAL OF
PERFORMANCE PARTICULARLY PERFORMANCE
ABILITY TO MANAGE CHANGE,
LEAD, MANAGE TIME, ETC.

BOSS
(1) PREVENTING, (2) DISCOURAGING, (3) NEUTRAL,
(4) ENCOURAGING,(5)REWARD--INTRINSIC--PRIDE/
STATIC-EXPERIENCE-PRAISE FROM BOSS-RECOGNITION
-MONETORY-MERIT PAY

22
TRAINER’S --- WHAT TRAINERS WILL FACE?

• ENCOURAGING OR DISCOURAGING
• GOOD INDICATION OF BEHAVIOUR
• GM- Prog. TO IAS-WE TOLD TO ÏÏ LEVEL-
VERY GOOD IMPACT!

Ex.Grievance handling trg - no change in behaviour until

grievance faced!
Q; Are you doing different because you have attended
programme? Yes or No - Why?

23
 4.RESULT LEVEL

WHAT TANGIBLE GAINS TO THE ORGANISATION?

INCREASED PRODUCTIVITY?BETTER TEAMWORK?
REDUCED COSTS?

ORGANISATION STUDIES - CONTROL GROUP

ANALYSIS OF PROBLEM - ALLOW TIME

ANALYSIS OF FINANCIAL AND -EVALUATE

NONFINANCIAL PERFORMANCE BEFORE / AFTER
OF THE ORGANIZATION

24
 WORK FLOWS STUDIES --COST Vs BENEFITS

PRODUCTIVITY INDICATORS --BE SATISFIED WITH

EVIDENCE-- IF PROOF
IS NOT POSSIBLE

PRODUCTION/SALES/PROFITS/ROI COSTS/ACCIDENTS/
TURNOVER
CHANGE ATTITUDES-NOT IN FIN.TERMS
“IMPROVED MORALE”
(delighted traineees)(ECIL-Gupta-Tangible gains to orgn?Prodn increased
Board Level Trg Head(board meeting get in?
Sales:Different parts - Trained/not trained - ROI
Q: WHAT SHOULD A TRAINER DO WHEN TOP MGT ASK FOR
A TANGIBLE EVIDENCE THAT TRG. PROGRAMME IS
PAYING OFF? ( PROOF IS IMPOSSIBLE TO GET?)
25
contd.
 5. OUTCOME LEVEL

BY PRODUCT SUCH AS SOCIAL CONNECTIONS,

SATISFACTION OF PERSONAL NEEDS,
CAREER DEVELOPMENT, ETC.
-NUMBER OF PROFESSIONAL FILL IN WHAT YOU
CONTACTS BEFORE AND NOW USE TO
AFTER TRAINING. EVALUATE
-PROMOTION AND PAY
RAISE
-OVERALL PERSONALITY ASSESSMENT
-If my customer is not happy, it is my fault?and my challenge
is please them?We can not satisfy everybody!
26
RATING IN 5 PIONT SCALE EVALUATION OF INHOUSE TRG
In- house Infra- Research/ Total Over all rating
training structure consultancy Finances score
Faculty Library

1 2 3 4 5 25
ACCEPTABLE
ECIL 1 4 2 1 3 11
MIDHAN SOMEWHAT
1 2 2 1 2 8 ACCEPTABLE
I
VER LOW
IDPL 1 1 1 1 1 5
GOOD
BHEL 3 5 3 2 3 16
ACCEPTABLE
BDL 1 2 2 1 3 10
ACCEPTABLE
HAL 3 3 3 2 3 14
ACCEPTABLE
HMT 2 2 1 1 3 10
SOMEWHAT LOW
HCL 1 2 1 1 2 7
ACCEPTABLE
IA 2 5 2 1 3 13 27
 LINE MANAGER’S

ROLE IN HRD
 Contribute your ideas to HRD
 Motivate the individual to
perform and develop
 Help in preparation of HRD
schedules
 Implementation of schedules
 Review impact and revise plan

28
29
CONTACT FOR OUR SERVICES : bratanreddy@yahoo.com

NEW TECH/ BUSINESS/ CONNECTIONS

SPECIALLY IN TOP LEVEL PROGRAMS/
MIDDLE LEVELS

SOME SPECIFIC SUGGESTIONS:

READ ISTD, ASTD MANUALS/ JOURNALS
KIRKPATRICK--EVALUATING THE PROGRAM.
(Brief about Negotiation strategies and skills prog)

30