Alternate Method for Conducting Evaluations

Alternate Method for Conducting Evaluations Problem/Need Statement Keith A. Doering Colorado State University

1|Page

Alternate Method for Conducting Evaluations

United States Air Force policy requires training providers to evaluate their courses for currency and effectiveness (Air force training program, 2010, p.96). Within the Air Force, the Air Education and Training Command (AETC) takes on responsibility to provide technical training. AETC is also responsible for developing procedures to evaluate their education and training programs. In AETC, each Training Group is required to develop a Training Evaluation Program (TEP) (Technical and basic military training evaluation, 2010). As part of the TEP, a Training Evaluation office is responsible for conducting both internal and external evaluations. AETC policy on Technical and Basic Military Training Evaluation states "Internal feedback is a self-evaluation to determine if training accomplishes the instructional objectives effectively. It helps commanders, supervisors, and instructors improve course and unit support; identifies outstanding instructors, facilities, and equipment; and allows students to attain a sense of participation and responsibility for improving training programs" and that "External feedback provides an indication of the graduates ability to perform tasks required in the career field. In addition, external feedback may indicate the need to revise training standards or courses to improve training quality, add technology, and/or identify skills and knowledge where training might be reduced or increased" (Technical and basic military training evaluation, 2010).

Training Gap/Need Due to Department of Air Force budget cuts as a result of Sequestraion, our office looked at outside ways to accomplish the mission. Our thoughts were to look for a commercial off the shelf (COTS) product to off-set some of the gaps in data-collection and analysis required to perform course evaluations. The Chief, Training Evaluation is an advocate for thinking outside the box to develop cost effective approaches to complete the mission. She was actively 2|Page

Alternate Method for Conducting Evaluations

involved during the research phase of trying to identify a solution to off-set the gap in data collection as a result of budget constraints. When approached about trying to use SurveyMonkey as an additional evaluations tool, she was on-board. She performed a cost analysis and determined the SurveyMonkey Platinum plan for $780 annually ("SurveyMonkey Plans and Pricing," 2013) was justified and was a cheaper approach than sending a team of evaluators to the field to conduct field interviews. She gave approval to make the purchase and continues to be an advocate of its use for additional survey requirements.

My involvement with the SurveyMonkey initiative from the onset resulted in me becoming the sole person in our office who can build surveys and run customized reports. Although SurveyMonkey has a user manual that provides some training; the majority of learning is done via hands-on. A needs for a customized training program tailored to the evaluation functions of my office is required to get the other training evaluators trained. Training time will be reduced by only focusing on areas relevant to our mission.

Environment Where the Need Exist Current structure in the Training Evaluation office has five civilian evaluators, two military evaluators, data technician and a Chief of Evaluations. The evaluators are assigned responsibility for different courses that are trained at the base. Evaluators often take on special projects where a need to survey students or staff members exists. In addition to generating surveys, our office has used the analysis tools to generate reports based on data that was manually placed into SurveyMonkey.

3|Page

Alternate Method for Conducting Evaluations

Why a Gap Should be Filled by an Instructional Event It is widely known in our training organization that our office has used SurveyMonkey. We have received calls from other offices and another base to assist with survey projects. Most recently, I was tasked to use SurveyMonkey to develop separate surveys to capture student feedback on quality of life issues on base, obtain student and staff thoughts/opinions about sexual assault in the military and obtain student feedback on newly revised student in-out processing events. Unfortunately, the task of creating additional surveys, performing data entry and analysis takes me away from my primary job. Developing a tailored training program

would allow the other training evaluators to not only initiate their own projects to support their courses; it would also distribute the work-load associated with the additional surveys.

Support for the Training Event

The Chief, Training Evaluations supports the need to have the other evaluators trained on how to build surveys and run the analysis tools within SurveyMonkey because the current environment relies on one evaluator to develop and analyze all the surveys in SurveyMonkey. By providing training to all training evaluators, the office's ability to support the commander with training assessments will be enhanced. As one of the senior training evaluators, the task of training new evaluators often falls to me. Having a formal training plan for SurveyMonkey would not only ensure standardized training, it could be done more efficiently and as a result, free up time for me to work on other tasked training evaluation functions. The future students in this course are fellow training evaluators. Two fellow training evaluators and our data technician have also expressed interest with receiving training via a customized training program.

4|Page

Alternate Method for Conducting Evaluations

Summary

One year of working with the SurveyMonkey program has resulted in beneficial gains for our office to evaluate not only the effectiveness of training but more importantly the training environment built to support the students. Lessons learned from the year of working with the program will be used to develop the training program. The goal would be to incorporate the locally developed SurveyMonkey training program into the training plans of all positions within the Training Evaluations office.

5|Page

Alternate Method for Conducting Evaluations

References

SurveyMonkey Plans and Pricing. (2013). SurveyMonkey Plans and Pricing. Retrieved September 15, 2013, from https://www.surveymonkey.com/pricing/?t=201309_h United States, Department of the Air Force, Air Education and Training Command. (2010, September 13). Technical and Basic Military Training Evaluation. Retrieved from http://static.e-publishing.af.mil/production/1/aetc/publication/aetci362201/aetci36-2201.pdf United States, Department of the Air Force, Secretary of the Air Force. (2010, September 15). Air Force Training Program. Retrieved September 14, 2013, from http://static.e-publishing.af.mil/production/1/af_a1/publication/afi36-2201/afi362201.pdf

6|Page