Evaluation of Training Company)

Evaluation of Training Effectiveness
Excel Soft Technologies Pvt. Ltd.
Introduction to Evaluation of Training Effectiveness
In todays era of competitive market, there are many manufactures for the same product. To ensure the named position in the market the organization needs to improve the quality of the product and provide the best quality product to the customers at the cheapest rates. This will result in more sales, more sales means more profit.
To achieve this strategy, the organization has to train and develop its most valuable resource i.e. Human Resource. Only a well-trained human resource can introduce quality products/services. Thus, huge amounts of money are spent on the training programmes. Therefore it is very important to determine, how far the training programme has been successful. With the help of evaluation, results of training programme are determined.
Evaluation of training programme has been rightly defined as any attempt to obtain information on the effects of training performance and to assess the value of training in the light of that information. The purpose of evaluation is to determine whether or not the training achieved the desired objective. The objective might have been very precise, or extremely loose, but there would have been one. Thus, an organization that invests in training programmes for employees should also conduct some form of training evaluation.
Objectives of the study:
y y y y y y
To gain an overview of the organization. To learn the functions of the HR department with respect to training programs conducted. To make a detailed study of the Training and Development programmes conducted. To determine the specific nature of training deficiencies. To gauge the training effectiveness in the organization. To determine the extent of training not needed by the participants to meet their job requirement.
1
To determine and analyze the cost effectiveness of the programme and make suggestions to improve the existing procedure and hence increase the effectiveness of the training programs.
Scope of the study:
The project study covers the various training policy and development programmes being conducted at Excelsoft Technologies Private Limited. The details collected are confined to only one organization and particularly to one department i.e., Training Department of the organization. The scope of the study is limited to the executives of the company. Here, the training effectiveness is analyzed in post training scenario.
The study covers areas such as training need identification, training process, training methodologies, organizational development and participant development. The report examines how much the employees perceive these training programs to be effective and beneficial and also to what degree the training given to these employees, cater to their general as well as specific needs.
Research methodology: y Primary Data: Primary data is collected through questionnaire which is sent to the trainees and by way of formal and informal interviews.
Secondary Data: Secondary Data is collected mainly through existing data available in the company records.
Sample Size: A sample size of 20 employees is selected from each training programme conducted.
Limitations of the Study: y The success of the project mainly depends on the extent to which the techniques of evaluation are being used by the organization. y Great results cannot be obtained in a short period. Due to time constraints the study could not go in depth. y Reluctance on the part of the company to give away their closely guarded policies of evaluating the training effectiveness. y The analysis of the training programme is confined to primary and secondary data obtained. This is the descriptive study. As the data collected includes the secondary data any mistake inherent there would have reflected in the study. y The study is purely for the academic process, the inexperience makes this project less precise when compared with a professional research work. y This project will not help to make a comparative study because training activities differ from one company to another.
Company Profile
The Excelsoft Journey - Transforming the learning paradigm
Excelsoft products address all aspects of learning/training. Excelsoft's e-Learning solutions are used by more than 500 organizations and over 7 million end users worldwide. Excelsoft's component based development and service oriented architecture ensure easy integration within all existing e-Learning environments. Excelsoft is a provider of innovative technology-based solutions in the education and training space. Excelsoft architects, designs and develops technology solutions and digital content and has established itself in a leadership position in the e-Learning business. SARAS e-Learning technology solutions framework and the product suite developed around it provides a complete infrastructure for technology enabled learning lesson planning, activity creation and delivery, in-class lesson delivery, homework and assignment management, learning community creation and management, student performance management and many other such features and functionality enables all stakeholders such as teachers, students, school management and parents effectively collaborate to deliver better efficiency and achieve higher results. Having worked with reputed clients and partners over the decade, we have acquired a deep understanding of the value Information and Communication Technology can provide in enhancing the teaching and learning efficiency. We have also gathered a significant amount of experiences and best practices in school education. This knowledge has enabled us to create powerful solutions that deliver great learning experiences and makes schools effective centres of learning. Expertise in pedagogy and technology The company has a strong grounding in pedagogy as well as in depth expertise in technology. All that they do are based on a firm grounding provided by the competency centre & the R&D department that focuses on delivering improved ways of learning to suit the contemporary environment.
4
Compliant to global standards Excelsofts products and e-Learning solutions are compliant with leading global standards such as SCORM, IEE IMS, QTI and AICC. Over 500 leading global organizations and 7 million end users have benefited from Excelsoft's e-Learning solutions. The companys service capabilities provide consulting, on-going support and comprehensive implementation of the right products and services to deliver a custom business solution that fits specific learning objectives. Team Excelsoft: Expertise clients can benefit from Excelsoft team comprises of 650 professionals who have the qualifications, talent, expertise, experience, technological know - how and thought leadership to offer the clients of Excelsoft the very best in eLearning capabilities. The team is organized to spearhead the companys strategic initiatives in eLearning and training solutions, content development and B2C initiatives. Excelsoft professionals are located across the companys global offices to service the customer base in Asia, United States and EMEA. Excelsoft has consistently believed in recruiting the best, and offering them opportunities to excel in challenging projects. The spirit of challenge, the ability to do cutting edge work and deal with some of the most exacting clientele in the world makes Team Excelsoft an attractive destination for great talent.
Products Offered: Excelsoft offers value added products and services in the e-Learning domain, that serve the interests of educational publishers, universities and schools as well as the government, defence and corporate sector. The e-Learning applications include the SARAS Virtual Learning Environment, comprising of a Learning Management System and an Assessment Management System. The company also offers SARAS Learning Spaces - an e-Learning portal that leverages Web 2.0 technologies to promote informal learning. The company offers solutions in the technology and content domain and is equally comfortable in both. Technology Solutions Technology solutions at Excelsoft include: y y y SARAS Virtual Learning Environment - an efficient and feature rich e-Learning application, SARAS AMS (Assessment Management System) LMS (Learning Management System) and
SLS (SARAS Learning Spaces) an application that leverages the power of web 2.0 tools to enhance learning.
Content Solutions
Content solutions include: y y y y y y Instructional Design, Content Writing and Copy editing, Content Design and Production, and Outsourced Content Services. Gaming based content Outsourced Knowledge Processing.
Consulting
Excelsoft also has a well established consulting practice where the company offers y y y Solutions Implementation, Custom Application Development and Mobile e-Learning applications.
Customers Overview
Excelsoft is an e-Learning company that offers its services, applications and solutions to educational publishers, universities and schools as well as the corporate, government and defense sectors. They deliver e-Learning solutions based on their consultation and solutions design practice. Their solutions involve both the domains of technology and content and are supported by their competency unit - which is their R&D establishment.
Their clients span the geographies of:

North America
y y
USA Canada
Europe
y y y y
UK Netherlands Sweden Germany
Asia Pacific
y y y y
India Singapore Australia Japan
Clientele
Leading Publishing Houses around the world Solutions rendered

y
Technology Solutions
o o o o
Content management solutions Virtual Learning environments Community based tools Content packaging solutions 8
Evaluation of Training Effectiveness Test & Assessment solutions Authoring solutions
o o
Content services
o o o
Courseware Content editing Learning objects & Simulations
Leading institutes Worldwide

y y y y
Higher education Further education Vocational Education Distance Education
Solutions rendered
y
o o o o
Virtual Learning environments Community based tools Content packaging solutions Test & Assessment solutions
Content services
o o
Courseware Learning objects & Simulations
y y y y
IT companies Shipping Industries Telecom Giants Banks & Financial sectors 9
Evaluation of Training Effectiveness Pharmaceutical Giants
Solutions rendered
y
o o o o o o
Content management solutions Virtual Learning environments Community based tools Content packaging solutions Test & Assessment solutions Authoring solutions
Content services
o o
Governmental departments Solutions rendered

y
o o o
Talent management solutions Virtual learning environments Test & Assessment solutions
Content services
o o
Leading institutes in India & Abroad

10
Solutions rendered
y
o o
Virtual Learning environments Test & Assessment solutions
Content services
o o
Solutions rendered
y
Schools in India & Abroad
Testimonials
The core of the Newsmaker product is the Newsroom - a Flash application enabling students to lay out newspaper pages, populating them with articles and photos. The application that Excelsoft has developed is well designed and intuitive, even for primary-aged students. When we conducted our own code reviews, we found that the Excelsoft developers had fully complied with the technical specifications, that we had provided at the outset and that the application was very well coded using Action script 3. The Flash application needed to allow for complex database interactions. Multiple students work collaboratively on each newspaper project, so the Flash needs to pass information back and forth with the database, on who has contributed which elements in which position. Teachers can upload text and image-based resources when setting up projects. These are then pulled into the Flash, when students begin work. Excelsoft analyzed the complexity of these requirements in a detailed way and then implemented the agreed specification very effectively. Tom Jackson Director of Digital Agency, Business and Professional, Guardian News and Media UK
11
We have been able to establish excellent working relationships with the team at Excel Soft. Working closely with members of the IMS QTI project team, Excel Soft has been able to produce a system which is second to none in its adherence to this specification vital for any organization wishing to produce future-proof banks of questions. The positive experience of working with Excel Soft on TOIA means that I will seriously consider them for future software or learning content development initiatives. Niall Sclater Head of eLearning University of Strathclyde, Scotland
We have worked with Excel Soft for two years, and have been entirely delighted with the relationship as it has evolved, to the extent that we now aim to place all our principal external software commissions with Excel Soft. This is not least because the company specializes in developing software for educational purposes. Excel Soft has developed be speaking Content Management and Syndication System for us, which is now being upgraded to version 2. We have also placed numerous contracts with Excel Soft to develop bespoke web applications, sites, and most recently a custom-designed Customer Relationship Management system. Excel Soft's staffs have always been highly responsive to our needs, all documentation has been reliable, projects well managed, and our personal relationships have been well nurtured and maintained. We would strongly recommend Excel Soft as a business partner. Colin Hughes Managing Director Learnthings Ltd., UK
12
Office Locations
Mysore, India (HQ) Excelsoft Technologies Pvt. Ltd. 1-B, Hootagalli Industrial Area, Mysore - 570 018 Tel-1 : +91 821 4282000 Tel-2 : +91 821 400220 Fax : +91 821 4282208
Bangalore, India Shiva Tulasi Complex No. 193, (13/2, 13/3), 2nd Floor 5th Main Road/Shankar Mutt Road Chamarajapet, Bangalore 560 018 Mob-1 : +91-99809 38946 Mob-2 : +91-97405 10613
Hyderabad, India MJR Arcade, 5th Floor, Plot No: 1000, Gurukul Society, Khanamet, Madhapur, Hyderabad 500081
Mob : +91-90008 77426
Noida, India C-125, Sector 65 Noida, Uttar Pradesh - 201 301
United Kingdom Kings Place 90 York Way, London N1 9GU
Singapore 118B, Telok Ayer Street, Singapore - 608 587
Tel : +65-6773 5126 Mob : +91-98737 5881 Tel-1 : +44 (0)203 353 2461 Fax : +65-6779 7957 Tel-2 : +44 (0)793 099 4194
13
Theoretical Framework
Training is a method of imparting and developing specific skills for a particular purpose. It is an act of increasing the skills of an employee for doing a particular job. Training is a process of learning sequence of programmed behaviour. This behaviour, being programmed, is relevant to a specific phenomenon that is a job. Training is an activity when an expert and learner work together effectively to cope with the impending change, to have a competitive edge over other organizations, to improve performance and job satisfaction of employees to assist in the attainment of objectives necessary for the operation of business.
Training is concerned with the future of an individual, still her/his past and present provides a crucial frame of reference. The scope of training is limited, determined by the training objectives. It is a specific and highly structured. It has relatively short-term perspective of the future of the individual. Transfer of learning can be monitored and assessed. The organization and even the training agency can exercise greater control on the process. Training has less emphasis on ideology and social values as it stresses development of competencies. Societal forces play a far less significant role than they do in education. In training group comes together for a specific purpose and disperses after they programmed objectives are assumed to have been achieved.
The aim of training is to develop potential knowledge and skills of trainees to carry out defined tasks and responsibilities or to infuse scientific thinking and planning and working methodically and efficiently. Training enhances efficiently and develops a systematic way of performing your duties and assigned tasks.
Measuring of training:
Training is a critical component in any organization's strategy, but organizations don't always evaluate the business impact of a training program. Given the large expenditures for training in many organizations, it is important to develop business intelligence tools that will help companies improve the
14
measurement of training effectiveness. These tools need to provide a methodology to measure, evaluate, and continuously improve training, as well as the organizational and technical infrastructure (systems) to
implement the methodology. Cross-functional and reporting and learning analytics provide important connections between the measures of learning effectiveness offered by a learning management system (LMS) and the larger enterprise metrics that indicate whether learning is transferred and positively affects business results.
Training Effectiveness & its Significance:
Training as an activity has been going on as a distinct field with its own roles, structures and budgets, but it is still young. This field is, however; expanding fast but controversies seem to envelop any attempts to find benefits commensurate with the escalating costs of training. Training has made remarkable contributions to the improvement of all kinds. Training is essential; but doubts arise over its contribution in practice. Complaints are growing over its ineffectiveness and waste. The training apparatus and costs have multiplied but Unhappiness persists and is growing at the working level where the benefits of training should show up most clearly. This disillusionment shows in many ways reluctance to send the most talented workforce for training, inadequate use of personnel after training, etc.
With disillusionment mounting in the midst of expansion, training has entered a dangerous phase in its development. Training is neither a panacea for all ills nor is it a waste of time. What is required is an insight into what training can or cannot do, and skill in designing and executing training successfully and cost-effectively.
At the present time, all the organizations give more trust on commercializing their activities. All the firms, in order to continue to exist in the competitive global market and to be effectual, should espouse the most modern technology, i.e., mechanization, computerization and automation. Technical know-how alone, however, does not assure success unless it is sustained by workforce possessing indispensable expertise. Hence, organizations should train the employees to enrich them in the areas of changing technical skills and knowledge from time to time.
15

Evaluation is the collection of analysis and interpretation of information about any aspects of a programme of education or training as part of a recognized process of judging its effectiveness, its efficiency and any other outcomes it may have. Hamblin defines evaluation of training as, Any attempt to obtain information (feedback) on the effects of training program and to assess the value of training in the light of that information. In other words, training evaluation means the systematic collection of data relevant to the selection, adoption, value or modification of workplace learning activities. It is a process of providing information to those involved in training regarding questions they genuinely want answers for. It is simply a feedback. Why evaluate? Training is always done with specific objectives. Hence, evaluation of training is very important for the following reasons: 1. It tells us about the quality of training and the effect it has created on the learners. 2. Systematic evaluations also help us in pointing out the weaknesses of the training program so that in future such things can be taken care of. 3. It also indicates the extent to which learners have learnt from the program and 4. The extent they have transferred their learning into their daily job.
For the evaluation to be effective, it is very important that training objectives are carefully thought and framed, the principal significant questions should be: 1. To what extent the identified training needs objectives are achieved by the programme? 2. To what extent were the learners objective achieved? 3. What did the learners learn?
16
4. What commitment have the learners made about the learning they are going to implement on their work?
In addition to the above reasons, evaluation is also done to answer the following questions: Should the program be continued? How can the program be improved? How can we ensure regulatory compliance? How can we maximize training effectiveness? How can we be sure training is aligned with strategy? How can we demonstrate the value of training?
The Kirkpatrick Model of Training Evaluation
Donald Kirkpatrick has developed a very popular evaluation model that has been used since the late 1950s by the training community. The focus is on measuring four kinds of outcomes that should result from a highly effective training program. Kirkpatricks model includes four levels or steps of outcome evaluation: y y y y Level 1 EvaluationReaction Level 2 EvaluationLearning Level 3 EvaluationBehavior Level 4 EvaluationResults
Each of these steps is discussed in detail below, including how to design these evaluations.
17
Level 1Reaction
Here the goal is to measure participants reactions to the training program. Here we measure their reactions immediately after the program. Level one evaluation should not just include reactions toward the overall program (e.g., Did you like the program?); it should also include measurement of participants reactions or attitudes toward specific components of the program, such as the instructor, the topics, the presentation style, the schedule, audiovisuals, etc. Furthermore, each of these components can be further broken down into sub-components for evaluation (e.g., you can ask participants to evaluate specific characteristics of the instructor, the presentation, etc.). In short, level one evaluation is far more than just the measurement of overall customer satisfaction. Learning (Level two outcomes) and transfer of learning (Level three outcomes) are unlikely to occur unless participants have positive attitudes toward the training program. Therefore, it is important to determine participants reactions to the training program. Also, positive reactions are important because managers are more likely to eliminate unpopular training programs. Finally, the measurement of specific aspects of the training program can provide important information about what aspects of the training program can be improved in the future. Level 1 evaluation relies on the measurement of attitudes, usually through the use of a questionnaire. It is important to include closed-ended items (including rating scales) as well as open-ended items on your questionnaire It is important to learn the weaknesses, as well as the strengths, in order to improve a program. Do not be afraid to ask about program weaknesses! When having participants fill out questionnaires, it is best not to have them put their names on the instruments because of the advantages of anonymity over confidentiality. If they do not put their names on the instruments, you can assure anonymity and they may be more likely to be more honest in their answers. Evaluators should establish performance standards on the outcomes, when possible, so that the four steps in the logic of evaluation can be utilized and evaluative judgments can be made. Also, dont forget
18
Kirkpatricks last piece of advice to communicate the results because utilization of evaluation results will not happen without dissemination and communication.
Examples of evaluation tools and methods for level1: y y y y y y Typically 'happy sheets'. Feedback forms based on subjective personal reaction to the training experience. Verbal reaction which can be noted and analyzed. Post-training surveys or questionnaires. Online evaluation or grading by delegates. Subsequent verbal or written reports given by delegates to managers back at their jobs
Guidelines for evaluating reaction: Determine what you want to find out. Design a form that will quantify reactions. Encourage written comments. Get 100% immediate response. If desirable, get delayed reactions. Develop acceptable standards. Measure future reactions against the standard.
Relevance and practicability: Can be done immediately the training ends. Very easy to obtain reaction feedback. Feedback is not expensive to gather or to analyze for groups. Important to know that people were not upset or disappointed. Important that people give a positive impression when relating their experiences to others who might be deciding whether to experience same.
19
Here are a few advantages of level one evaluation: y y y y We will know how the participants felt about the training event. It may point out content areas that trainees felt were missing from the training event. It will tell you how engaged the participants felt by the training event. It can provide information about overall participant reaction as well as participant feedback and evaluation of specific aspects of the training event. y Detailed level one evaluation can provide formative evaluation information that can be used to improve future versions of the training program (e.g., we can fix the things the participants disliked about the program and add the things they felt was missing).
Level 2Learning
Here the goal is to determine what the training programme participants learned during the training event. Because the training instructor should have specific learning objectives, one hopes to find clear learning outcomes. Learning outcomes can include changes in knowledge (e.g., what are the key differences between Windows 95 and Windows ME?), skills (Can the participant upload a new operating system to a computer), or attitudes (Have participants attitudes toward computers improved?). Some training events will emphasize knowledge, some will emphasize skills, some will emphasize attitudes, and some will emphasize multiple learning outcomes. The evaluation should focus on measuring what was covered in the training event (i.e., the learning objectives). Level two evaluations should be done immediately after the training event to determine if participants gained the knowledge, skills, or attitudes. A couple of issues here are (a) How shall one measure knowledge, skills, and attitudes? (b) What research design should be use to demonstrate improvement in level two outcomes?
20
Examples of evaluation tools and methods: y y Typically assessments or tests before and after the training. Interview or observation can be used before and after although this is time-consuming and can be inconsistent. y y y Methods of assessment need to be closely related to the aims of the learning. Measurement and analysis is possible and easy on a group scale. Reliable, clear scoring and measurements need to be established, so as to limit the risk of inconsistent assessment. y Hard-copy, electronic, online or interview style assessments are all possible.
Guidelines for evaluating learning: Measure before and after attitudes, knowledge, and/or skills. Use a paper and pencil test for knowledge and attitudes. Use a performance test for skills. Get 100% response. Use a control group, if feasible. Follow up as appropriate.
Relevance and practicability: Relatively simple to set up, but more investment and thought required than reaction evaluation. Highly relevant and clear-cut for certain training such as quantifiable or technical skills. Less easy for more complex learning such as attitudinal development, which is famously difficult to assess? Cost escalates if systems are poorly designed, which increases work required to measure and analyze.
21
Here are few advantages of level two evaluations: y y Demonstrating participant learning should help trainers in promoting their training program. Knowledge of level two evaluations can help in interpreting the results of level three evaluation (e.g., if level three results do not occur, it may because of workplace factors and not because of any flaw in the training program). y Detailed level two evaluations can provide formative evaluation information that can be used to improve future versions of the training program (e.g., you may find certain learning objectives that are not being met).
Level 3Behavior
Here the goal is to find out if training program participants change their on-the-job-behavior (OJB) as a result of their having attended and participated in the training program. If the behavior change does not occur, you also want to find out why the change did not occur. The level three question is, did the training have a positive effect on job performance? Level three evaluation specifically involves measuring the transfer of knowledge, skills, and attitudes from the training context to the workplace. Level one and level two outcomes are still important because participants generally need to react positively to the training program (level 1 outcome) and they need to learn the material (level 2 outcome) if they are going to be motivated and able to apply what they have learned when they return to their jobs. Learning is likely to transfer only if the conditions in the work setting are favorable for transfer. In addition, there are obviously many things that trainers can do to increase the likelihood of transfer. In other words, transfer is affected by training factors before trainees go back to their workplace as well as workplace factors that operate in their workplace when they return. Here are two sets of factors that make transfer more likely to occur:
22
A. These are some factors in the training program or event that can help facilitate transfer of learning: y The training or learning environment or context or culture is made to be similar to the actual work environment or context or culture of the organization (this is called situated learning). Learning that takes place in authentic contexts is more likely to be used later. y y Provide real world examples and actual experience performing and practicing the behaviors. Make sure trainees understand the general principles behind the behaviors (called transfer through principles). y Explain the importance and applicability of the future on-the-job behaviors (i.e., motivate the trainees)
B. These are some factors in the receiving organization that can help facilitate transfer of learning: y y The organizational culture and climate support change. The participants direct supervisor and others with whom he or she works support and/or facilitate the participants new behaviors through direct extrinsic rewards such as help, encouragement, praise, increased freedom and responsibility, pay increases, and recognition. y The participant gains intrinsic reinforcement from applying his or her new knowledge, skills, or attitudes (i.e., he or she likes what was learned and enjoys performing the new related behaviors). y The participant has the opportunity to use his or her new knowledge or skills on the job.
Examples of evaluation tools and methods: Observation and interview over time are required to assess change, relevance of change, and sustainability of change. Arbitrary snapshot assessments are not reliable because people change in different ways at different times. Assessments need to be subtle and ongoing, and then transferred to a suitable analysis tool.
23
Assessments need to be designed to reduce subjective judgement of the observer or interviewer, which is a variable factor that can affect reliability and consistency of measurements. The opinion of the trainee, which is a relevant indicator, is also subjective and unreliable, and so needs to be measured in a consistent defined way. 360-degree feedback is useful method and need not be used before training, because respondents can make a judgement as to change after training, and this can be analyzed for groups of respondents and trainees. Assessments can be designed around relevant performance scenarios, and specific key performance indicators or criteria. Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols. Self-assessment can be useful, using carefully designed criteria and measurements.
Kirkpatricks recommendations for designing a level three evaluation: Use a control group if possible. That is, use the strongest design that is feasible. Allow time for the behavior change to take place. Evaluate both before and after the program if practical. Again, use the strongest design that is feasible. Survey and/or interview one or more of the following: trainees, their immediate supervisor, their subordinates, and others who often observe their behavior. The more the evidence, the better. Get 100 percent response or a sampling. Repeat the evaluation at appropriate times. Consider cost versus benefits.
Evaluation questions asked at this level: y y Did the trainees put their learning into effect when back on the job? Were the relevant skills and knowledge used?
24
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles?
y y y y
Was the change in behavior and new level of knowledge sustained? Would the trainee be able to transfer their learning to another person? Is the trainee aware of their change in behavior, knowledge, skill level? Did the representative open each telephone customer dialog using his or her name and department?
Was the representative able to describe to you and categorize the customer's objections as either misinformation or valid?
y y y
Did the representative use the appropriate model answer in response to each objection? Did the representative close each sales call with a request for purchase? If the prospect did not make a purchase, did the representative end the call with specific future action steps?
Did the representative complete call history records that include summaries of who, what, where, when, and why?
Level three is often harder than level one and level two evaluations because behavior changes at the workplace are often harder to measure than reaction and learning directly after the training event. You must give the behavior time to transfer and collect data at the workplace. Probably the most common design used for level three evaluation is the one-group pretest-posttest design (i.e., get a baseline measure of the behavior you plan on training, train the participants, and then measure the participants behavior again after the training). If you are able to include a control group, you will be able to use the pretest-posttest nonequivalent control group design (i.e., in addition to measuring the training participants before and after the training, you also find a set of similar people,
25
who do not undergo training for the control group, and you measure these control group participants behavior before and after the training program). Kirkpatrick also discusses another popular design for measuring transfer of learning (i.e., for measuring level three outcomes). He never labels the approach, but it is formally called the retrospective survey design. He points out that you can survey (using questionnaires or interviews) the training participants, the participants supervisors or managers, and the participants subordinates. The design is called a retrospective design because you are asking the participants (or the others) to think back (i.e., in retrospect) to their behavior before the training program and then compare it to their current level of behavior, and, finally, to decide if the behavior has changed. You should ask for specific examples behavior changes. You can also use the retrospective survey design with the training participants managers and subordinates, asking them if the participants behavior has changed. The more corroboration you get across the different sources, the stronger the evidence of transfer. The retrospective survey design is generally is a weaker design than the experimental designs discussed earlier because the actual pretest behavior is not measured directly. However, the retrospective survey can still provide useful and sometimes convincing data. Kirkpatrick speaks highly of the retrospective design, probably because it is so simple to carry out and because it tends to be cost effective (i.e., it doesnt cost a lot of money to administer questionnaires).
Relevance and practicability: Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation. Simple quick response systems unlikely to be adequate. Cooperation and skill of observers, typically line-managers, are important factors, and difficult to control. Management and analysis of ongoing subtle assessments are difficult, and virtually impossible without a well-designed system from the beginning. Evaluation of implementation and application is an extremely important assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job, therefore evaluation in this area is vital, albeit challenging.
26
Behaviour change evaluation is possible given good support and involvement from line managers or trainees, so it is helpful to involve them from the start, and to identify benefits for them, which links to the level 4 evaluations below.
Here are a few advantages of level three evaluations: y Provides measurement of actual behavior on the job, rather than only measuring or demonstrating positive reaction and/or learning. This is important because you want to have actual on-the-job results from the training program. y Level three outcomes are required for level four outcomes (i.e., they are the intervening variables or factors that lead to level four outcomes); therefore, it is good news when level three outcomes are found. y Most level three outcomes are intrinsically useful, even if level four outcomes (i.e., final organizational or business results) are never fully demonstrated (e.g., it is useful to have managers who are effective listeners and supportive, or employees who know how to do basic work on their computers, or employees who act respectfully toward employees from different ethnic or cultural groups). y In many situations, evidence of level one outcomes, level two outcomes, and level three outcomes will be sufficient evidence of the merit and usefulness of a training program. This is especially true when all of these provide evidence of positive results of the training program.
Level 4Results
Here the goal is to find out if the training program led to final results, especially business results that contribute to the bottom line (i.e., business profits). Level four outcomes are not limited return on training investment (ROI). Level four outcomes can include other major results that contribute to the well functioning of an organization. Level four includes any outcome that most people would agree is good for the business. Level four outcomes are either changes in financial outcomes (such as positive
27
ROI or increased profits) or changes in variables that should have a relatively direct effect on financial outcomes at some point in the future.
Here are some examples of different kinds of level four outcomes: y y y y y y y y y y y y y Improved quality of work Higher productivity Reduction in turnover Reduction in scrap rate (i.e., less wasted resources) Improved quality of work life Improved human relations (e.g., improved vertical and horizontal communication) Increased sales Fewer grievances Lower absenteeism Higher worker morale Fewer accidents Greater job satisfaction Increased profits
Examples of evaluation tools and methods: It is possible that many of these measures are already in place via normal management systems and reporting. The challenge is to identify which and how relate to the trainee's input and influence. Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training, so they understand what is to be measured. This process overlays normal good management practice - it simply needs linking to the training input. Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training.
28
For senior people particularly, annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training.
Kirkpatricks recommendations for level four evaluations: 1. Use a control group if practical. In other words, use the strongest experimental design that is feasible. 2. Allow time for results to be achieved. In other words, many level four outcomes will take some time to occur. 3. Measure both before and after the program if practical. Again, use the strongest experimental design that is feasible. 4. Repeat the measurement at appropriate times. Repeated measurement (e.g., using the time-series design) can provide data on the long term pattern of results. 5. Consider costs versus benefits. You may not want to perform a level four evaluation if the costs of that evaluation are high in comparison to the potential benefits or impacts of the training program.
Relevance and practicability: Individually, results evaluation is not particularly difficult; across an entire organization it becomes very much more challenging, not least because of the reliance on line-management, and the frequency and scale of changing structures, responsibilities and roles, which complicates the process of attributing clear accountability. Also, external factors greatly affect organizational and business performance, which cloud the true cause of good or poor results.
Level four evaluation is difficult in the sense that it is quite difficult to establish firm evidence that a training program was the key or only source that produced the level four outcomes. For example, time has to elapse after the training program in order for these outcomes to occur. Other factors may also
29
occur during that time period. Second, it is hard to isolate the effect of the training program because there are usually many additional causal variables operating on the level four outcome variables (i.e., the training event is just one of many potential causes). Level four outcomes are often more distal outcomes, rather than proximal outcomes of a training program. For these reasons, the evidence obtained from level four evaluation is usually weaker than the evidence obtained from lower level evaluations, especially levels one and two which are relatively easy to document. For an example of distal, level four outcomes (increased productivity and profits) here is a potential outcome line that ends with the level four results:
Program
Reactions
Learning Behavior Productivity
Increased profits.
Level four outcomes tend to fall far down outcome lines, which mean that many intervening factors must take place in order for the level four outcomes to take place. This means that we should not be overly optimistic in expecting large level four outcomes from single training programs. In general, we should try to develop an impact theory to help us to understand the operation of level four outcomes. If you really want training program to have a positive level four outcome, it is probably wise to make the training program a component of a larger organizational performance program package that is designed to produce level four changes. A common design for documenting level four outcomes is the interrupted time-series design, although the other experimental designs with control groups are also frequently called for, depending on the kinds of outcomes you want to measure. For example, some data are frequently recorded at regular intervals (sales, turnover, accidents) making these variables easy to measure over time forming time-series data (you just have to get these secondary data). Other level four results will be more difficult to obtain because pre-training data may not be available (e.g., data on worker morale, quality of work life, improved communication). If pre-training data are not routinely recorded, you will have to make sure the data are recorded at least once before the training event is implemented. If the data are recorded multiple times before the training event, the interrupted time-series design may be used. If data are
30
recorded only once before the training event, the one group pretest-posttest design is possible or, even better, the pretest-posttest nonequivalent control group design may be feasible.
Benefits of Kirkpatricks evaluation model to the organization/department: y y Helps to determine the form of future training programmes. Assists with identifying the effectiveness of different forms of teaching (such as classroom based or web based). y y y Maximizes the training ROI (return on investment). Determines the financial benefits and costs of the programmes. Helps to compare the costs and benefits of different training programmes to choose the best program.
31
DATA ANALYSIS AND INTERPRETATION
The Communication training programme is taken for evaluation. Sample size: A sample of 20 employees is selected who attended the training programme. A questionnaire is sent to all the employees consisting of 15 questions. The questionnaire is of objective type. A question followed with four options as below: a) Strongly disagree b) Disagree c) Agree d) Strongly agree The respondent has to rate each question from 1 to 4, whichever they feel is relevant.
Analysis of questions: Each question will be individually analyzed and Training Effectiveness Index will be calculated for each question based on the responses of employees. The formula used to calculate training effectiveness index is as below:
Note: the expected rating for each question is taken as 3.
32
1. The programme helped me understand the concept of communication training.
Attribute Strongly disagree Disagree Agree Strongly agree Total
Number of Respondents 2 1 8 9 20
Question 1
10 8 6 4 2 0 Strongly disagree Disagree Agree Strongly agree
TEI for question 1 is 85%.
Out of the 20 respondents who took the survey 17 of them responded as per the expectations of the training programme i.e., they agreed that the programme helped them understand the concept of communication training while 3 responses were below the expected level. Here the responses meet the expectations of the training programme to a good extent.
33
2. The programme increased my awareness about the importance of communication training.
Question 2
Majority of them i.e., 17 respondents agreed that the programme increased their awareness about the importance of communication training. Their responses met the expectations of the training programme while most responses were above the expected level. Only 3 responses were below the expectations of the training programme.
34
3. I shared the knowledge I gained with other colleagues.
Question 3
15
10
0 Strongly disagree Disagree Agree Strongly agree
17 respondents agreed that they shared the knowledge they gained with their colleagues. Their responses met the expectations of the training programme. Only 3 responses were below the expectations of the training programme.
35
4. I recommend this course to all of my other colleagues.
Question 4
12 10 8 6 4 2 0 Strongly disagree Disagree Agree Strongly agree
18 employees responded as per the expectations of the training programme i.e., they agreed that they would recommend/suggest this course to their colleagues. It is noteworthy that only 2 employees said they would not recommend/suggest this course to their colleagues.
36
5. My job offers adequate opportunity to apply some or all that I have learnt.
Question 5
15 respondents agreed that their job offered them adequate opportunities to apply some or all that they have learnt. Their responses met the expectations of the training programme. 5 others felt that their job did not offer them any opportunities to apply what they learnt.
37
6. The programme helped me perform better at work.
Question 6
14 respondents agreed that the training programme helped them perform better at work while it was found that 6 employees responded below the expected level making this aspect/part of training only 70% effective.
38
7. The programme taught me the ways and means through which the work can be done.
Question 7
12 responses were as per the expectations of the training programme i.e., they agreed that the programme taught them the ways and means through which they can perform the work while 8 respondents did not agree with this.
39
8. My superiors and colleagues have acknowledged my improved knowledge in the field.
Question 8
Out of the 20 respondents who took the survey 13 of them responded as per the expectations of the training programme i.e., they agreed that their superiors and colleagues acknowledged their improved knowledge while 7 responses were below the expected level.
40
9. I believe I have been able to take more effective decisions after undergoing the course.
Question 9
13 of the respondents agreed that after undergoing the training programme they have been able to take more effective decisions. The 7 other responses were below the expected level i.e., they disagreed that the programme helped them in making better decisions.
41
10. I could apply some or all of the knowledge gained.
Question 10
15
10
0 Strongly disagree Disagree Agree Strongly agree
Majority of them i.e., 18 respondents agreed that they could apply some or all of the knowledge they gained. The responses met the expectations of the training programme to a great extent. It is noteworthy that only 2 responses were below the expectations of the training programme.
42
11. The programme helped me gain new ideas.
Question 11
7 6 5 4 3 2 1 0 Strongly disagree Disagree Agree Strongly agree
14 respondents agreed that the training programme helped them gain new ideas while 6 responses were below the expected level. The responses here meet the expectations of the training programme to a moderate extent.
43
12. The programme helped me in self development.
Question 12
Of all the respondents 17 responses were as per the expectations of the training programme i.e., they agreed that the programme helped them in self development. Only 3 respondents disagreed with this. The responses here meet the expectations of the training programme to a good extent.
44
13. I feel the programme was worthwhile in terms of my time away from normal job duties.
Question 13
TEI for question 13 is 85%
Majority of them i.e., 17 respondents agreed that the programme was worthwhile in terms of their time away from normal job duties. Their responses met the expectations of the training programme while most responses were above the expected level. Only 3 responses were below the expectations of the training programme.
45
14. The programme helped me to improve the quality of the work I do.
Question 14
14 12 10 8 6 4 2 0 Strongly disagree Disagree Agree Strongly agree
Out of the 20 respondents who took the survey 16 of them responded as per the expectations of the training programme i.e., they agreed that the programme helped them improve the quality of the work they do while 3 respondents did not agree to this.
46
15. I am more satisfied with my job after undergoing the course.
Question 15
Of all the respondents 13 responses were as per the expectations of the training programme i.e., they agreed that they are more satisfied with their job after undergoing the course while 7 respondents disagreed. The responses here meet the expectations of the training programme to a moderate extent.
47
Questions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Training Effectiveness Index 85% 85% 85% 90% 75% 70% 60% 65% 65% 90% 70% 85% 85% 80% 65%
The overall Training effectiveness Index for the Communication training program is found to be 77%.
48
FINDINGS:
Training is conducted effectively at Excel Soft Technologies Private Limited. Training Effectiveness Index (TEI) is found as around 77% against the targeted 75%. In the study concerning the Evaluation of Training Effectiveness at Excelsoft Technologies Private Limited, a few observations clearly found out the below mentioned facts: p While collecting the data it was observed that many people were enthusiastic about taking the survey. This shows that the employees acknowledge the efforts made for their growth. p While compiling and analyzing the data, it was observed that most of the people have responded in a similar way to all the questions. Thus the error of central tendency was observed. p Despite of the fact that most people replied promptly to the questionnaire, a few filled it without actually meaning it. This has marred the credibility of the results considerably. p Majority of the workers approved that training offered more motivation to their work; however felt that recognition of skills during the training need to be rewarded. p Most of the employees opinion is that training plays a vital role in employees work life and it has to be given to each and everyone in their respective fields for the improvement of individual/group/team performance. p A few employees felt that training helps very little. p A greater percentage of employees agreed that after undergoing the training they are more satisfied with their jobs and the training has helped them in self development. p The training programmes conducted appear to have helped the workers in improving various skills at work and the quality awareness.
49
SUGGESTIONS:
No doubt Training is a very powerful tool for the smooth functioning of the organization, but it needs to be used with care in order to derive all the benefits. Here are few suggestions/recommendations for improving training effectiveness and its evaluation: y The organization should make arrangements for all the employees to undergo training programmes for achieving 100% efficiency. y Set a deadline for the learners to complete the training. The deadline should convey a sense of urgency. y Review the key points of the training with the employees, either one-to-one or in a group, once theyve completed the training. Encourage them to ask questions about how employees will be expected to use what they have learned. y Active participation from the trainees should be encouraged. There should be a two-way communication between the trainer & trainee. y Ensure that learners have opportunities to immediately use the skills gained in their jobs, and follow up with them one-to-one to review how they are progressing. y y Make the training available to learners as a fresher even after theyve completed it. Training programmes need to be formulated by giving more focus on the needs of the company, also they should be well planned, objective based and result oriented. y Training programmes should be conducted frequently to encourage employees to build up skills in all areas, and particularly on the job training to be strengthened. y y The employees should take the training seriously with inner desire to learn. Faculty should be invited from the external sources so that the training technique also may be attractive. y Training programmes should be conducted at regular intervals so that it will help the employees to deliver skills better, to apply knowledge into practice through day to day activities. y For training to be effective, it can't stand on its own, but should be part of an overall development strategy that results in creating an infrastructure of learning. y Employees need to be engaged in the learning process and later workplace application if training is to be effective.
50
Develop a competency-based model: Trainers routinely waste time training people who arent prepared to be trained, dont need to be trained, or arent ready to be trained. A competency-based model helps eliminate this. Heres how it works:
p Have a set list of prerequisites for each training module. Prerequisites can be assessed through either successful completion of a more elementary module or through a pretest. If a person cant complete a prerequisite, that person shouldnt be in the class. p Dont train folks who dont need the training. All training modules should have post-tests to assess mastery of skills. Let people take the post-test as a pretest. If someone knows enough about a subject to pass a test without taking a course, why waste that persons time and yours? The trick to this is creating post-tests that replicate real-life scenarios. p Only train employees who are ready to use the product. This rule sounds simple. However, it seems to frequently be ignored. p Just-in-time training has its drawbacks (most notably, scheduling). But the payoff is huge. People can apply the lessons learned immediately. There is less of a chance of losing knowledge and more opportunity to assess the effectiveness of training as a method to increase on-the-job productivity.
y y
The cost incurred on the training program should not exceed its benefits The method or type of training should be very cautiously selected by the organization depending upon the organizations' resources & an employee's individual need for training.
Focus of training should be on priority development needs and to produce strong motivation to bring change in employees.
The training event should be an interesting experience for the trainees and the session needs to be sensitive to time and information overload.
For transfer to be maximized, trainees will require a clear message that they are to be held accountable for their actions and performance following the training.
51
CONCLUSION:
The Evaluation of any training programme has certain aims to fulfill. These are concerned with the determination of change in the organizational behavior and the change needed in the organizational structure. Hence evaluation of any training program must inform us whether the training programme has been able to deliver the goals and objectives in terms of cost incurred and benefits achieved. The analysis of the information is the concluding part of any evaluation programme. The analysis of data should be summarized and then compared with the data of other training programmes similar nature. On the basis of these comparisons, problems and strength should be identified which would help the trainer in his future training programmes.
While organizations have found that the Kirkpatrick framework for evaluation has met their requirements in the past, the new social and economic realities that evaluators and programorganizational developers re-examine their role and their potential contributions within organizations, and the utility of the models they apply. Given the increasing importance of rigorously demonstrating societal value-added, the Kirkpatrick framework addresses the concerns of communities and organizations, while additionally offering evaluators the opportunity to align their processes with all that an organization uses, does, produces, and delivers, as well as the value it adds to society and external clients.
Organizations invest considerable sums of money in training employees. Much of that expenditure is wasted through ineffective planning and weak linkages between the training conducted and the workplace. So often the companies set their standards and references based on outside concerns when they should start with their internal company concerns before they move ahead. Many of these items are very important and just because a company has met outside standards doesnt always mean that their inside standards are used effectively.
52
We need to go back to the who and the what situation. Who needs to know about this and what should the evaluation show? These are all important issues which need to be brought up in the evaluation situation. To conclude, people will improve in the areas that they are evaluated and measured on. The others will fade away with time.
53
BIBLIOGRAPHY:
1) Human Resource Management Shashi K. Gupta, Rosy Joshi
2) Training & Development Dr. B. Janakiram
3) Effective Training/Systems, Strategies and Practices P. Nick Blanchard, James W. Thacker
Sites Visited: www.humanresource.com www.humanresource.org
54
Evaluation of Effectiveness of the Training programme
TRAINING EFFECTIVENESS QUESTIONNAIRE
(Date: --------------)
Name: ____________________ Division _____________ Grade: _________ Dear Respondent,
Please mark your responses to the following questions as given below: ATTRIBUTE RATING
A) B) C) D)
STRONGLY DISAGREE DISAGREE AGREE STRONGLY AGREE
1 2 3 4
Ratings
QUESTIONS
(1 To 4) 1. The programme helped ______________________. me understand the concept of
2. The programme increased my awareness about the importance of ______________________.
55
3. I shared the knowledge I gained with my other colleagues.
4. I recommend this course to all of my other colleagues
5.
My job offers adequate opportunity to apply some or all that I have learnt.
6. The programme helped me perform better at work.
7. The programme taught me the ways & means through which the work can be done. 8. My superiors & colleagues have acknowledged my improved knowledge in the field.
9. The programme helped me gain new ideas.
10. I believe I have been able to take more effective decisions after undergoing the course. 11. The programme helped me in self development.
12. I could apply some or all of the knowledge gained.
13. I feel this programme was worthwhile in terms of my time away from normal job duties.
14. The programme helped me to improve the quality of the work I do.
15. I am more satisfied with my job after undergoing the course.
56
Suggestions, if any for improving the effectiveness of the training programme: ____________________________________________________________________________________ ____________________________________________________________________________________ __________________________________________________________________
57

Evaluation of Training Company)

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Evaluation of Training Company)

Transféré par

Droits d'auteur :

Formats disponibles

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Introduction to Evaluation of Training Effectiveness

Objectives of the study:

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Scope of the study:

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

The Excelsoft Journey - Transforming the learning paradigm

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Their clients span the geographies of:

UK Netherlands Sweden Germany

India Singapore Australia Japan

 Leading Publishing Houses around the world Solutions rendered

Evaluation of Training Effectiveness Test & Assessment solutions Authoring solutions

Excel Soft Technologies Pvt. Ltd.

Courseware Content editing Learning objects & Simulations

 Leading institutes Worldwide

Higher education Further education Vocational Education Distance Education

Courseware Learning objects & Simulations

IT companies Shipping Industries Telecom Giants Banks & Financial sectors 9

Evaluation of Training Effectiveness Pharmaceutical Giants

Excel Soft Technologies Pvt. Ltd.

Courseware Learning objects & Simulations

 Governmental departments Solutions rendered

Courseware Learning objects & Simulations

 Leading institutes in India & Abroad

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Virtual Learning environments Test & Assessment solutions

Courseware Learning objects & Simulations

Schools in India & Abroad

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Mob : +91-90008 77426

Noida, India C-125, Sector 65 Noida, Uttar Pradesh - 201 301

United Kingdom Kings Place 90 York Way, London N1 9GU

Singapore 118B, Telok Ayer Street, Singapore - 608 587

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Training Effectiveness & its Significance:

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Evaluation of Training Effectiveness

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

The Kirkpatrick Model of Training Evaluation

Evaluation of Training Effectiveness

Excel Soft Technologies Pvt. Ltd.

Leading Publishing Houses around the world Solutions rendered

Leading institutes Worldwide

Governmental departments Solutions rendered

Leading institutes in India & Abroad