3/1/2012

Training & Development Session 5

Usha Chirayil

Usha Chirayil

Training & Development Session 5 EVALUATION OF TRAINING Why evaluate? Methods for evaluation Criteria for evaluation

Metrics Mantra: What gets measured gets managed more effectively training evaluation process has the potential to provide useful information to multiple stakeholder groups Calculate the benefit Take corrective action By designing an effective training evaluation process, an organization can obtain the information needed to improve both training program delivery and business performance, creating opportunities for continuous organizational improvement Improve efficacy of learning delivery in organizations Demonstrate value to stakeholders & transferring ownership for learning to them Help Prioritize & Optimize learning investments Method Reaction of Participants The assessment of competency learning, whether the training material has been learned, is best conducted at the individual level. It is at the individual level that organizations are able to assess employees reactions and increased understanding. Reactions are important because, if students react negatively to a course, they are less likely to transfer what they learned to their work. Increased understanding in terms of new or improved knowledge, skills and abilities is the primary aim of a training event. Method - Data Data concerning the overall effectiveness of training procedures, appropriateness of media and instruction methods and other issues relating to possible revisions in instructional design may prove to be very valuable to the training department and will assist them in better serving the needs of the other stakeholders involved. This information can be obtained through : Questionnaire Knowledge Review Employee Portfolio Skill Gap Analysis Questionnaire a structured tool that may provide both quantitative and qualitative information about employee reactions to the training event& should focus on both training content and delivery. Training content section should target questions to ascertain whether
1

Usha Chirayil

the training materials provide useful information that will assist in performing work tasks the employee is more knowledgeable about the subject matter following the training event.

Knowledge reviews offer an objective means of determining whether training content has been learned-refer to a general group of assessment tools in which employees read questions and respond in writing. The knowledge reviews may be administered by delivering the tool at the start and end of the training event. The tool should be developed as a short answer or multiple-choice instrument using the same questions in each administration, presented in a different order. The questions on the knowledge review should adequately address the learning objectives of the training. The results of each administration of the knowledge review can be compared to measure knowledge transfer during the training event. This information will be useful in assisting the training department to improve the course material. Observation Observation is another evaluation method that provides information regarding employee reactions to the training. Training department personnel should observe employee interaction, level of engagement with training instructors and responses to course content. This evaluation technique may be informal or highly structured. Informal observation can be used to provide general information about the training structure. Highly structured observation focuses on monitoring particular points in the training event and commonly involves the utilization of a checklist of the points to be observed.

Employee portfolio The employee portfolio is an effective means of providing information to employees as well as their managers about the level of mastery of particular knowledge, skills or abilities following a training event. The employee portfolio should be designed to be used for informational purposes only. The results of the portfolio are not intended to be linked to employee performance evaluations. To implement this concept, employees should be required to identify the learning objectives of the training and within a 60-day time frame develop a work sample for review by the manager that demonstrates the application of their learning. The business unit manager should be required to ensure that within that time frame employees are assigned activities that will allow them to produce the portfolio. After receiving the portfolio, the manager should review it and provide feedback to the employee on strengths and areas for improvement.
2

Usha Chirayil

Skill Gap Analysis Skill Gap Analysis - Another means of providing employees information about their skill development is to require business unit managers to conduct an assessment of each employees level of knowledge, skills and abilities relative to the level required for successful performance in the position. The Skill Gap Analysis can be administered using a survey instrument. A skill gap occurs when an employee is rated with a lower level of skill than the position requires. This assessment should be used for informational purposes only and should not be linked to the employee evaluation process. Employees can use the results of this assessment to track their skill development over time from their manager's perspective. ROI Analysis of Organizational Performance Measures - Organizational performance measures identify the metrics against which successful business unit operations can be evaluated. These measures will enable performance tracking to identify trends and areas requiring further attention and/or interventions. As a training evaluation method, the changes in organizational performance measures should be compared to the employee Skill Gap Analysis. This comparison will identify the relationships between skill development and business performance. Information derived from this analysis will allow business unit managers to identify performance strengths and areas for improvement and make decisions about where additional training may be required. The use of organizational performance measures to assess training allows business units managers to be resultsoriented and business-focused in their prioritization of training requirements.

Usha Chirayil

Kirkpatrick Model of Training Evaluation - Refer detailed handout 1. Reaction Did they like it? How well did participants like the programme or course? Upon completion of the training session or course

2. Learning

Did they learn it? What principles, facts, and techniques were learned? What attitudes were changed?

Upon completion of the course

3. Applicati on/ Behavior

Did they use it? What changes in job behaviour resulted from the programme?

Before and after training

4. Results

Did it produce tangible business results? What were the tangible results of the programme in terms of reduced cost, improved quality, improved quantity, etc.

Before and after Training

Training Evaluation Model consists of five stages, each of which corresponds to specific data categories: Outputs refer to descriptive data about the training programs and participants, including demographic data. Pre-training assessment addresses information about the participants past experience as well as current competencies, learning needs, and expected application of learning. Post-assessment (reactions) addresses reactions to the training experience. Post-assessment (learning) is a self-assessment of knowledge/skills gained and expected application of learning. Follow-up may include several methods to assess the outcomes and effect of training programs over time