Running

Head: evaluating interactive

Evaluating Interactive Multimedia Instructional Programs By Shirley J Farrell University of Alabama

As partial credit for AIL 605 Dr. Margaret Rice

March 25, 2011

Running Head: evaluating multimedia

Introduction Free multimedia software programs from many, varied, and sometimes, unusual sources are available for teachers to enhance and enrich the learning experiences of students in their classrooms. Teachers readily accept and use the programs, but are these multimedia software programs effective in providing appropriate learning experiences? The National Aeronautics and Space Administration (NASA) produces a tremendous amount of teaching materials in many forms, including online and multimedia programs. One branch of NASA, the Jet Propulsion Lab (JPL) in Pasadena, California, provides teachers with oceanic materials in addition to space resources. For many years, the author has secured from the JPL multiple copies of one multimedia software program, A Visit to an Ocean Planet, for teachers attending professional development trainings. A Visit to an Ocean Planet shows the importance of the oceans to the climate and life on Earth. This CD-ROM focuses on the Gulf of Mexico, the 1997-1998 El Nino, satellites, and how data are used to monitor oceans and climate, and the real life activities of oceanographers. Not only does this CD contain materials, resources, and activities for teachers but also has several interactive activities for students. Upon opening the program, teachers and students have three choices: Guide, Mission, and Expedition. The guide provides teachers and students with text, audio, video, charts, diagrams, pictures and activities/experiments. Teachers or students may print out the activities and experiments to provide additional research and learning. Missions explore the joint missions between the US and France of the TOPEX/Poseidon satellite. Students are able to manipulate a 3-d satellite to explore all sections to learn about the six instruments for data collection. Then students may explore

Running Head: evaluating multimedia

the overview of the satellite or measuring ocean topography. Students interact with the content to learn about missions and play an orbit game. Through this game, students must conduct additional research Embedded into the student tasks are brief tutorials to provide guidance on how to use the multimedia (text, audio, video, charts, diagrams, pictures). Expeditions vary from simulations to data analysis. Expedition: Gulf of Mexico guides students to find and interpret data, compile interpretations, and determine results. Students plan a scientific expedition to research whales in the Gulf of Mexico by researching two different routes through the Gulf of Mexico in April and October to determine the best time to conduct the research. As students work through tasks, they record their data in a journal that is saved on the computer. The students analyze five different sets of data for each of five to six locations within each route. A second expedition focuses on El Nino. Students discover how satellites are used to detect the weather, climate and changes brought by El Nino. Data from EL Nino and La Nina are provided. Students analyze the data to answer questions and conduct additional research of weather in their local areas. The last expedition is meeting oceanographers and their contributions to society. Students may choose any or all of 17 oceanographers to learn their stories and accomplishments. These mini-biographies and research videos pro can provide additional information to complete other tasks. Throughout the different sections of the software program, students use the multimedia to explore the importance of oceans to climate and life. Learning Theory The constructivist learning theory is the foundation upon which the learning experiences are designed. Through constructivism, students are active learners working in

Running Head: evaluating multimedia

complex, realistic, and relevant environments or in open-ended learning environments (Driscoll, 2005). With this software program, students are provided tasks using the tools that professionals use in realistic environments. All tasks involve working as oceanographers or climatologists, using real data and critical and creative thinking skills in order to solve the problems within the authentic learning simulation. Scaffolding through the help button and the tutorial may be accessed as much or as little by students, as well as requesting help from the teacher. After reading about past, present and future missions, students may have questions for research. Students can access data and research at the JPL lab web site to learn more and interact with additional online simulations. Social interaction through group work is an integral part of constructivism and is applicable with the software tasks, but classroom teachers would ultimately determine if students work as individuals or groups with the Expeditions, Missions, and Guide experiments. Although multimedia software programs may be designed based on a particular learning theory, classroom teachers may not follow that particular theory nor utilize the software appropriately. Evaluation The overall design principles for this multimedia software program are good. The software program is very intuitive. Students must move the cursor over icons to navigate the software. While this screen is visually appealing, it may be difficult for all students to understand what to do to begin but an audio narration explains how to navigate. All multimedia elements (video, text, narration, charts, pictures, and audio) are balanced and used effectively for student exploration and task completion. In addition to navigation buttons, a site map is included for a complete overview of all learning opportunities and

Running Head: evaluating multimedia

options on the CD-ROM. All learning styles are accommodated on this software with visual and auditory elements. To evaluate the instructional design based on constructivism for Visit to an Ocean Planet, criteria developed by Savery and Duffy (1995) are adapted into a question format. Each question was applied to the software to determine how well this program follows the constructivist theory (Appendix A). The overall evaluative decision, based on the applied criteria reveals that this program is designed based on the constructivist theory. Although each task or activity has subparts that support the final task, the Expedition: Gulf of Mexico can be the final task of the software program. Students are able to compare the planned expedition results to the results of oceanographers. Therefore, students have been learning in a real-world environment using the tools, data, and thinking processes of real oceanographers. For gifted students this is an extremely important activity to help them explore careers using authentic activities. (Gifted students develop so many interests they may not be able to decide what they want to do in college or as adults. Exposure to careers and authentic tasks helps them explore and focus their interests and feelings.) All learning from each section is embedded into the authentic task, including accessing information regarding the work of the various oceanographers. Students develop an ownership of the learning without the sequential tasks most CDROM learning games and activities have. Students may start and stop at any point to change direction in their learning. Tasks are open-ended with no one right answer and answer keys are not embedded into the program. Students must use critical thinking skills to answer questions. For students that want a right or wrong answer, they will have to debrief with the classroom teacher on those sections. Students are able to reflect on their

Running Head: evaluating multimedia

learning in the journal of the Gulf of Mexico expedition and, if allowed to work with other students, discussions within their group. This software could be improved by designating the Expedition: Gulf of Mexico, as the final performance task of all learning and other activities from the software program. Even within this expedition, students may explore information about the different oceanographers to find answers to complete the task requirements. Most computers cannot run this program because the software was developed in 1998 running on old operating systems. One would think that the newer operating systems could handle the program but they do not. However, individuals could open folders on the CD to print the activities, watch the videos, listen to audio and analyze charts without the interactive ability of the multimedia program. For people who have kept older versions of computers or operating systems, this program would be great. NASA needs to develop this CDROM as an online simulation for students who are interested in oceanography, climatology, satellites, or the Gulf of Mexico.

Bibliography Driscoll, M. (2005). Psychology of learning for instruction. New York: Pearson Education, Inc. Savery, J. R., & Duffy, T. M. (1995). Problem based learning: An instructional model and its constructivist framework. Educational Technology, 35, 31-38.

Running Head: evaluating multimedia

Appendix A Adaptation of Savery and Duffys (1995) evaluation of constructivist instructional design evaluation.
Evaluation Item 1. Are all learning activities anchored to a larger task or problem? Do the activities/ expeditions/missions support the learner in developing ownership for the overall problem or task? Does the program provide for authentic task(s)? Comments Each mission/expedition/task has all sub-activities anchored to the overall activity. The Expedition: Gulf of Mexico does use all learning areas from the software to complete the mission. Students determine the order of completion, amount of research needed, etc. The program is not sequential or restricting navigation until a certain domain has been completed. The overall Expedition has students analyze the routes and times of year to determine the best time to collect data about the Gulf of Mexico and sea life. Once students plan the trip, they actually take a voyage through video to see the results. Students must analyze primary source documents (photos, videos, charts) in order to make critical decision and solve problems. Students may access all parts of the program to research the information or research outside of the program. Once the gulf expedition has been completed students are able to compare their analyzed data with the oceanographers data. An explanation that decisions can differ provides the open-ended quality of the task. Since none of the tasks are sequential, students can complete as much or little as they desire- giving ownership of their learning. The tasks do not have an answer key and are not recall/comprehension level tasks. The Expedition: Gulf of Mexico is analysis and evaluation. The rest of the tasks are inquiry (experiments) or analysis (mission). There is NO drill and practice on this CDROM. Each mission, tasks or expedition provides a reflection section for students. Yes/No Yes

2.

Yes

3.

Yes

4.

Did the design the of task and the learning environment to reflect the complexity of the environment the learner should be able to function in at the end of the learning? Does the program give the learner ownership of the process used to develop a solution?

Yes

5.

Yes

6.

Does the design of the learner environment to support (scaffolding) and challenge the learner's thinking? Providing opportunity for, and supporting reflection on both the content learned and the learning process.

Yes

7.

Yes YES

OVERALL Match to Constructivism