University of Maryland, Baltimore County Graduate School Emergency Health Services Department

EMS Department use of Technology Based Learning

Justin Yurong JUSTINYURONG@ME.COM August 16, 2012

Contents
Background: .................................................................................................................................... 3 Hypothesis: ..................................................................................................................................... 3 Methods: ......................................................................................................................................... 4 Results:............................................................................................................................................ 4 Figure 1 Overall Central Exam Results ................................................................................... 5 Figure 2 Number of Logins vs. Exam Score............................................................................ 5 Figure 3 Number of Logins vs. Quiz Scores............................................................................ 5 Figure 4 Number of Logins vs. Final Grade ............................................................................ 6 Discussion: ...................................................................................................................................... 7 Results:............................................................................................................................................ 7 Works Cited .................................................................................................................................... 8 Attachment 1 Examination Data .................................................................................................. 9

2 Justin Yurong, NREMT-P MSc Candidate, UMBC

Background:

Al-Ghad International Colleges of Health Sciences Emergency Medical Services department is a private college that is headquartered in Riyadh, Saudi Arabia with eight other campuses (six male and two female) that are scattered around the kingdom. The Emergency Medical Specialist (paramedic) program was created for Saudi citizens in order to address the lack of qualified prehospital providers within the kingdom. Currently, the majority of health care employees in Saudi Arabia are foreign workers (Saudi Ministry of Health, 2009), but EMS providers are predominately Saudi citizens. Compared to western style EMS systems, the Saudi system is understaffed with EMTs that have received substandard training and are not held to high standard. In the capital city of Riyadh which has approximately 6 million residents, the Saudi Red Crescent Authority (city EMS service) fields only 327 EMS providers (McKinsey and Company, 2012). Compare this to the city of Seattle, Washington which had about 600,000 citizens (1/10 the size of Riyadh) but fields 1074 EMS providers (City of Seattle, 2011) (3 times that of Riyadh) and this is only the government employees not including private ambulance services.

In the Kingdom of Saudi Arabia, the official language of healthcare is English, which is made necessary by the large amounts of foreign healthcare workers. As such, all certification examinations and training programs are mandated to be in English. This has led to a problem with the current paramedic training programs since the majority of Saudis with a decent command of the English language choose to study abroad in western higher education facilities. Students who matriculate to Al-Ghad colleges speak English as a second language with the majority of them having difficulty carrying on a normal conversation in English. The purpose of this study is to analyze whether the use of technology based learning has a positive impact on student examinations score and the students perceived ability with the subject area. Hypothesis:

EMS Students who speak English as a second language who participate in technology based learning will have higher examination scores and will feel more confident with the curriculum than students who do not participate in technology based learning.

3 Justin Yurong, NREMT-P MSc Candidate, UMBC

Methods:

During the spring of 2012 semester, 436 students participated in EMS 212 Basic medical course at nine separate campuses. The course was taught off of a standardized curriculum that was followed at each campus. American paramedic instructors (who wrote the curriculum) were present at three campuses with a total student count of 129 students. Due to data reliability issues, only students that were present at campuses with an American paramedic instructor were included in this study. The exam consisted of 25 multiple choice questions and six multipart short answer questions (attachment 1). The multiple choice portion of the exam is based off of verified questions that are published as part of the instructor package for Mosby EMT Basic 2011 Revised Second Edition. The questions were then edited by subject matter experts to decrease the English difficulty level. The short answer questions in the exam were created by subject matter experts. The multiple choice portion of the exam was graded by a Scantron device and the written portion of the exam was graded by an American subject matter expert. 23 students at the Riyadh campus were exposed to the Evolve online learning system on which quizzes, course information, animations, additional readings, and videos were available to the students in an asynchronous format. Students were required to complete online quizzes as part of the grading system. 106 students at the Jeddah and Dammam campuses who also took the course and participated in the exams DID NOT have access to the online content and will be used as the control group. Student grades for the centralized exam will be compared to log-in data from the Evolve website, in class quiz scores, and online quiz scores. Finally, interviews will be completed among participants at the Riyadh campus gauging their impressions of the centralized exam and their overall impressions of the online resources. Results: The overall results of the central exams were as follows:
Riyadh Total Students Average Max Grade Min Grade % from Average Dammam Total Students Average Max Grade Min Grade % from Average Jeddah Total Students Average Max Grade Min Grade % from Average

23 64.30 102.00 24.00 7.80

48 58.47 100.00 14.00 1.96

58 46.74 98.00 0.00 -9.77

4 Justin Yurong, NREMT-P MSc Candidate, UMBC

Total Students Average Max Grade Min Grade

129 56.50 102.00 0.00

Figure 1 Overall Central Exam Results

120.0 100.0 Exam Score 80.0 60.0 40.0 R = 0.3531 20.0 0.0 0 5 10 15 20 25 30 35 Number of Logins

Figure 2 Number of Logins vs. Exam Score

80.0 70.0 Average Quiz Score 60.0 50.0 40.0 30.0 R = 0.7298 20.0 10.0 0.0 0 5 10 15 20 25 30 35 Number of Logins

Figure 3 Number of Logins vs. Quiz Scores

5 Justin Yurong, NREMT-P MSc Candidate, UMBC

90.0 80.0 70.0 Final Grade 60.0 50.0 40.0 30.0 20.0 10.0 0.0 0 5 10 15 20 25 30 35 Number of Logins R = 0.6106

Figure 4 Number of Logins vs. Final Grade

Interviews were conducted with 6 individuals who were placed throughout the spectrum of Number of Logins (30, 24, 15, 10, 7, 4). The interview questions were as follows: 1. Do you feel that the online resources made it easier to understand the course material? Explain. 2. Do you feel that the online resources made the in-class quizzes easier to complete? 3. Do you feel that the online resources made it easier to take the centralized examination 4. Why / how did you decide how much time you spend on the online resources? 5. Are you more or less likely to utilized online resources in future courses? Following are the results of the interview: 1. 5 out of 6 students said that the online resources made it easier to understand the course material. The main reasons given for this were: a. I was able to look at the material at my own pace and was able to translate the material slowly. b. I was able to look at things that were gone over very quickly in class or not at all 2. 4 out of 6 students stated that the online material made the in-class quizzes easier. 3. 4 out of 6 students stated that the online material made the examination easier. 4. Students answers to this question were very diverse. The top answers were as follows: a. Instructor urging b. Unable to understand some in class material because of language barriers c. Desire to earn a higher grade 6 Justin Yurong, NREMT-P MSc Candidate, UMBC

5. All students stated that they will utilize online resources in the future even if not required by the instructor. It was interesting to note that that the students who only logged in 7 and 4 times respectively and had very low scores in the course stated that if given the opportunity would spend much more time utilizing the online resources. Discussion:

As noted by the overall scores, the students at the Riyadh campus had the highest scores on the exam, followed by Dammam and Jeddah. It is important to note that each individual campus has its own unique set of education challenges, so basing the results solely on the utilization of online resources is not appropriate. The Riyadh student scores were plotted on a scatter chart and then a linear fit was applied to the data with the scores being reported in a total logins vs. score with the R2 correlation coefficient being added. Finding students that would participate in the interviews proved difficult since the semester had ended several months before and the majority of students were away on vacation. Once students agreed to participate in the interviews, it was difficult (due to the language barrier) to make the students understand that were not any right or wrong answers to the questions. In the future it may be more conducive to have students complete a survey as close to the course as possible. Results:

Within the parameters established in the methods section, it appears that the hypothesis is supported and students that participated in the technology based learning (online resources) scored higher on the examination than students who did not take advantage of the online resources. Furthermore, there appears to be a direct correlation between the amount of times that a student logged into the exam and the student scores in each section. It is interesting to note that there was more of an effect on student quiz scores (R2=0.3531 for the exam score vs. R2=0.7298 for the quizzes) than there was on examination scores. Compared to students who received the course based on a standardized curriculum, students who utilized online resources scored higher (on average 7.8% higher) than students who did not have access to online material. Conclusions / Limitations: According to this project, there is a positive correlation between students who utilize online resources and examination scores and an even higher correlation between quiz scores and the number of logins. There are, however, several limitations to these results. First these results do not take in account the English ability level of the students in the courses. Theoretically all students are supposed to have a written and spoken command of the English language but in practice this is far from the truth. Second, only a small number of students were sampled as to 7 Justin Yurong, NREMT-P MSc Candidate, UMBC

their feelings and experiences with the online resources. Third, this project only analyzed the number of logins that each student made, but not the amount of time spent or the specific resources that were accessed by the students. Future projects should be conducted to analyze these topics. Works Cited City of Seattle. (2011, November 03). City of Seattle. Retrieved August 7, 2012, from Fire Department: http://www.seattle.gov/fire/deptInfo/deptProfile.htm Galbraith, M. W. (2004). Adult Learning Methods 3rd ED. Malabar, FL: Kreiger Publishing Company. Knowles, M. S. (2005). The Adult Learner, Sixth Edition. Burlington, MA: Elsevier. McKinsey and Company. (2012). Training mapped to scope of practice powerpoint. Riyadh. Saudi Ministry of Health. (2009). Health Statistics. Retrieved August 7, 2012, from Saudi Central Department of Statistics and Information: http://www.cdsi.gov.sa/english/index.php?option=com_docman&task=doc_download&g id=1152&Itemid=163

8 Justin Yurong, NREMT-P MSc Candidate, UMBC

Attachment 1 Examination Data

Exam Score / Login 2.68 3.43 2.50 3.38 1.94 3.73 4.11 3.07 5.37 2.17 5.64 5.11 4.77 3.00 7.94 11.33 6.00 3.93 7.14 11.60 4.80 11.80 7.38

Quiz Scores / login 2.15 1.97 1.91 2.05 2.29 2.39 2.59 3.28 3.10 2.45 3.48 3.98 4.12 3.92 3.60 3.21 2.03 2.74 3.47 4.67 7.28 6.82 6.33

Overall / Login 2.42 2.70 2.20 2.71 2.12 3.06 3.35 3.17 4.23 2.31 4.56 4.55 4.44 3.46 5.77 7.27 4.01 3.33 5.30 8.13 6.04 9.31 6.85

Number of Logins
30 29 29 28 26 24 22 21 19 15 14 14 11 10 9 9 8 7 7 5 5 5 4

In Class Quiz
70.0 49.4 36.0 38.6 35.2 58.0 38.0 56.8 79.0 28.0 24.0 54.0 35.4 56.0 34.0 0.0 15.2 0.0 4.4 18.0 18.0 50.0 30.0

Online Quiz
62.7 76.3 67.3 79.3 76.7 68.3 65.3 67.7 71.7 57.0 38.0 78.7 55.0 34.7 32.0 63.7 36.7 13.0 59.3 15.3 42.0 35.7 20.7

Average Quiz 64.6 57.3 55.3 57.5 59.6 57.4 57.0 68.8 58.9 36.8 48.7 55.8 45.3 39.2 32.4 28.9 16.2 19.2 24.3 23.3 36.4 34.1 25.3

Exam Written
80.5 99.5 72.5 94.5 50.5 89.5 90.5 64.5 102.0 32.5 79.0 71.5 52.5 30.0 71.5 102.0 48.0 27.5 50.0 58.0 24.0 59.0 29.5

Overall Average 72.6 78.4 63.9 76.0 55.0 73.5 73.7 66.6 80.5 34.6 63.8 63.6 48.9 34.6 52.0 65.4 32.1 23.3 37.1 40.7 30.2 46.5 27.4

9 Justin Yurong, NREMT-P MSc Candidate, UMBC