Impact of Technology on Student Achievement

GENERAL FINDINGS: Ten meta-analyses synthesized research from 946 studies, ranging from preschool to college. Each meta-analysis concluded that instructional programs that included technology show a positive impact on student achievement, resulting in higher test scores. The key findings from these studies show: 1. Classrooms in which computers were used to support instruction usually showed gains in student achievement as measured by standardized achievement tests. The magnitude of gains varied from study to study. There was usually a good match between the desired outcome of the treatment and the outcome that was measured. 2. The effectiveness of different applications of computer-assisted instruction varied by the content area and the skill being taught. In general, applications fared better if delivered in a content area with a defined structure, such as mathematics. Meta-analyses of computer-based instruction and multimedia applications indicate that the effectiveness of educational technology on improving student achievement depends on a match between the goals of instruction, characteristics of the learners, the design of the software, the technology, and the implementation decisions made by teachers (Sivin-Kachala, & Bialo, 1993). The Software Publishers Association (SPA) commissioned an independent meta-analysis of 176 studies focusing on the effectiveness of technology in schools. This report concludes that the use of technology as a learning tool can make a significant difference in, among other things, student achievement as measured by standardized tests (SivinKachula & Bialo, 1996). In a study conducted in New Zealand, researchers found the use of computers contributed, with other instructional innovations, to higher performance on English, mathematics, and science test. The study was conducted with eighth-, ninth-, and tenth-grade students (McKinnon, Nolan, & Sinclair, 1996). An extensive study of 55 New York State school districts also points to the same conclusion: increased technology supports, facilitates, and encourages student achievement (Mann & Schaffer, 1997). Research reported by the National Center for Education Statistics (NCES) indicated that the primary factor in test score differences was family/social backgrounds. The second most important factor was instructional opportunity. According to NCES, it is the way that computers are used to change and enhance curriculum that represents the most important factor in determining whether or not computers have an impact on achievement. One of the strengths of this was that the control group received a valid, well-designed instructional reading program. Analysis of the standardized test scores pre- and post-treatment indicates that all groups improved to well above the mandated cutoff score by the end of the study. Mean scores for the treatment groups were higher than the control group means (Coley, Cradler, & Endel, 1997).

LEARNING FROM TECHNOLOGY CAI computer-assisted instruction CBI computer-based instruction CEI computer-enriched instruction CMI computer-managed instruction ILS integrated learning system Kulik and Kulik (1991) conducted one of the most comprehensive studies of the effectiveness of using computers to increase student achievement. In 81% of the studies examined, the students in the computer-based instruction (CBI) classes (experimental group) had higher exam scores than students who were taught by conventional methods without computer technology (control group). One type of computer application that usually results in positive gains in achievement at the elementary and secondary levels is computer tutoring (Kulik, 1994). The West Virginia Department of Education and the Milken Exchange on Education Technology released a very comprehensive study entitled West Virginia Story: Achievement gains from a statewide comprehensive instructional technology program (Mann, Shakeshaft, Backer, & Kattkamp, 1993). Solomon, in an After word of the study, indicated that not only is there a statistically significant relationship between Basic Skills/Computer Education (BS/CE) and test scores gains; and not only can these gains be translated into effect size comparable to those of other interventions, but also, the gains from programs that update BS/CEs positive features can be achieved at a much lower cost than could similar gains from a currently very popular alternative intervention, namely class size reduction (p.50). An examination of research studies that investigate the impact of technological tutoring systems on student achievement shows mixed results (Wilson., 1993; Butzin, 2000).* Some reports suggest that CBI, CAI and ILS(s) can improve students basic skills in such disciplines as mathematics (Koedinger, Anderson, Hadly & Mark (1997).* Other studies report that, in some instances, the use of technology to teach basic skills had a negative impact on academic achievement (Wenglinsky, 1998).* Evaluations showed that Algebra students in a high school in Pittsburg, PA who used Cognitive Tutor outperformed students in traditional classes, having achievement gains up to 25 percent in skill and up to 100 percent in problem solving. Retention in mathematics classes and attendance also improved among students using this tutor. (Hubbard, 2000)* In a decade-long study, researchers investigated the impact of a computer-integrated instructional program called Project CHILD (Computers Helping Instruction and Learning Development) found that elementary students in project classrooms from kindergarten through fifth grade have consistently had higher test scores and better discipline than their counterparts in traditional, self-contained classrooms (Butzin, 2000, p3).* A variety of meta-analyses conducted between 1985 and 2000 on the impact of CBI, CAI, ILS, drill-and-practice software, and computer tutorials on student achievement report that students using computers had higher test

scores, typically measured on standardized achievement test. In 1994, for example, Kulik aggregated findings from over 500 individual studies of computer-based instruction. These studies showed percentile gains on achievement tests of 9 to 22 percent over control groups. On average, students who used computer-based instruction scored at the 64th percentile compared to students without computers who scored at the 50th percentile. Kulik also found that computer-based instruction can decrease the amount of time required for students to learn basic skill. (Kulik, 1994)* LEARNING WITH TECHNOLOGY: In a meta-analysis that examined the impact of technology on student learning, researchers found increased teacherstudent interaction, cooperative learning, and most importantly, problem-solving and inquiry (Statham & Torell, 1996). Many researchers investigating the use of technology in education have found that technology is most powerful when used as a tool for problem-solving, conceptual development, and critical thinking (Culp, Hawkins, & Honey, 1999; Sandholtz, Ringstaff, & Dwyer, 1997; Means, 1994).* One of the most powerful uses of technology in education is to tailor instruction to students individual learning needs. Technology can provide the means for students with special needs to communicate via email, and to use the Internet for research. It can also can help teachers accommodate students varying learning styles (Silverstein et al., 2000).* The Apple Classrooms of Tomorrow (ACOT) project was a 10-year study that set out to investigate how routine use of technology by teachers and students would affect teaching and learning. When compared to their non-ACOT peers, students routinely employed inquiry, collaborative, technological, and problem-solving skills uncommon to graduates of traditional high school programs (Sandholtz et al., 1997).* In a longitudinal study researchers investigated the impact of project-based learning using multimedia classroom projects were much more student-centered than non-project classrooms, and were organized around the collaborative construction of complex products (Penuel, Golan, Means, & Korbank, 2002).* A meta-analysis that examined the impact of technology on student learning found increased teacher-student interaction, cooperative learning, and most importantly, problem-solving and inquiry. One essential condition for student learning to take place: Computers should be used less for drill and practice in the classroom and more as open-ended thinking tools and content resources. (Statham & Torell, 1996). Technology supports exploration, which helps students set achievable goals, form and test hypothesis, and make discoveries of their own (Collins, 1990). Using technology to support collaborative knowledge integration includes tools that enable group thinking, problem-solving, and task orientation. Sharing real data from primary sources with experts leads to students thinking and working the way experts do (Evard, 1996; Federman & Edwards, 1997; Jonasses & Reeves, 1996; Lonergan, 1997; Scardamalia & Bereiter, 1991; Wideman & Owston, 1993). Technology encourages student collaboration, project-based learning, and higher order thinking (Penuel et al., 2000).*

Impact of Technology on Student Engagement Student Engagement Indicators: Responsibility for learning take charge, self-regulated learners Strategic develop and refine learning and problem-solving strategies Energized excitement and pleasure, intrinsically motivated Collaborative have and value skills to work with others (NCREL Learning with Technology Profile Tool) Research studies that focused on technology and students motivation to learn relied on self-reports of students attitudes toward computers and found, in general, that most students considered computer activities to be highly motivating and interesting (Gregoire, Bracewell, & Lafferriere, 1996; Heidmann, Waldman, & Moretti, 1996; Kendall & Broihier, 1992). Reports of best practice and program evaluations show that students become energized and engaged when given the leeway to explore, inquire, and make connections between their prior knowledge and new-found answers to their questions about the way the world works (Boyer, 1995; Johnson & Johnson 1996). An early review of the research concerning the effect of computers on students attitudes concluded that: Computer use most affects attitudes toward school and subject matter. Computer use appears to have a positive impact on improving students self-image and self-confidence (Roblyer, Castine, & King, 1988).

Teachers also use the Internet for student activities. Fifty percent of the teachers with classroom connections had their students use the Internet at least three times as part of a lesson. However, only 7 percent of the teachers had their students use the Internet to communicate with other students or to post their work (Becker, 1999). Except for math teachers, there was no statistical difference in use of the Internet in content areas. While studies show that math teachers are among the highest users of technology in their classroom, they were the lowest users of the Internet (Becker, 1999). Numerous studies have reported that technology can lead to increased student motivation and improves selfconcept. Software and Information Industry Association (SILA) examined 311 research reviews and reports from published and unpublished sources. They concluded that technology has been found to have a positive effect on student attitudes toward learning, self-confidence, and self-esteem (Sivin-Kachala & Bialo, 2000).* Reviews have reported that technology has been found to improve school attendance, decrease dropout rates, and have a positive impact on students independence and feeling responsibility for their own learning (Coley, 1997)* In ACOT classrooms, teachers reported that students displayed increased initiative by going beyond requirements of assignments, and that students spent more time on assignment and projects when working on computers. Students often chose to use technology during free time, and before and after school. (Sandholtz et al., 1997).* Pervasive perception among teachers is that computers have improved the climate for learning by increasing student motivation in subjects for which they use computers (Gregoire, Bracewell, & Laferriere, 1996).

Impact of Technology on Educational Systems Sustainable technologies support a classroom technology model that utilizes a broad number of highly visual, interactive technologies such as a projector, document camera, etc, combined with as little as one high powered multimedia internet connected computer in place to support the nine instructional strategies The continued use, sustainability, of these technologies in a meaningful way is possible for many years to come, for the following reasons:

Teachers build upon strategies that they already know and use, and focus on using the technology to support those strategies. Even with a 5 year replacement cycle, fewer computers will need to be replaced in the classroom (as opposed to large numbers of computers in a high-tech classroom, labs and pods), resulting in a reduced and much more sustainable technology budget. Interactive technologies will increase visual and interactive learning in the classroom to support the core curriculum, sound instruction and assessment. Schools that focus on sustainable classroom technologies narrow the range of technologies that require ongoing technical and network support.

This narrow range of hardware, software and support needs minimizes the need for additional training during replacement cycles. This model promotes a realistic approach to using technology in ways that yields big results due to the natural integration in the larger curricular framework and is not an add on the already full curriculum.

(Classroom Instruction That Works Marzano, Pickering and Pollock, 2001) The more advanced uses of technology support the constructivist view of learning in which the teacher is a facilitator of learning rather than the classrooms only source of knowledge (Trilling & Hood, 1999; Penuel & Means, 1999; Silverstein et al., 2000; Statham & Torell, 1999). In numerous studies of students learning with technology , teachers have reported that technology encourages them to be more student-centered, more open to multiple perspectives on problems, and more willing to experiment in their teaching (Knapp & Glenn, 1996).* Results of a variety of studies suggests that, over time, technology can serve as a strong catalyst for change at the classroom, school, and district level (Hawkins, Spielvogel & Panush, 1996; Means, 1994; Chang et al., 1998).* Evidence also exists that technology will have a stronger impact when technology integration is a part of the broader-based reform effort (Sandholtz et al., 1997).* Schools that have embraced technological change in instructional delivery have seen dramatic improvements consistent with school restructuring (Bozeman & Baumbach, 1995).* Technology supports exactly the kinds of changes in content, roles, organizational climate, and effects that are at the heart of constructivist educational reform movements (Means et al., 1993 p.1).* Many proponents of the current reform efforts see technology as a vital component of a new educational paradigm in which curriculum, teaching methods, and student outcomes are reconceptualized (Means, 1994).** Student engagement and academic performance remained highest when their technology use was integrated into the larger curricular framework rather than being an add-on to an already full curriculum (Sandholtz et al., 1999).*

Research suggests that when technology integrated into larger instructional frameworks, students will not only learn how to use the equipment and software, but will also gain content knowledge (Silverstein et al., 2000). Factors for Technology that Favor Desirable Outcomes INTEGRATION WITH REFORM EFFORTS: Researchers examining (Project Explore, Union City, New Jersey) the impact of the technology on student achievement found a substantial improvement in students standardized test results. While some of this improvement can be attributed to technology, researchers noted that the importance of other restructuring efforts were occurring simultaneously. The magic lay not exclusively in the technology, but in the interweaving of a systematic program of educational reform with the judicious use of technology-based resources (Change et al., 1998, p.34).* Student engagement remained high in classrooms emphasizing interdisciplinary, project-based instruction (Sandholtz et al., 1997).* Other researchers have reported similar results related to student motivation (Silverstein et al., 2000; Penuel et al., 2000).* In a study of five technology-rich schools, researchers noted goals for student learning were clearly articulated prior to the introduction of technology (Glennan & Malmed, 1996).*

PROFESSIONAL DEVELOPMENT: Statistical support for peer coaching comes from many sources. Perhaps the most easily understood data follows:
5% of learners will transfer a new skill into their practice as a result of theory 10% will transfer a new skill into their practice with theory and demonstration 20% will transfer a new skill into their practice with theory and demonstration, and practice within the training 25% will transfer a new skill into their practice with theory and demonstration, and practice within the training, and feedback 90% will transfer a new skill into their practice with theory and demonstration, and practice within the training, feedback, and coaching

Purposes which have been indicated to reinforce use of peer coaching include: increase of student learning facilitate/increase discussion between/among colleagues of professional topics/research sharing of successful practices through collaboration encouragement of and provisions for reflective practice use as a problem-solving vehicle reduce isolation among teachers promote teacher as researcher create a forum for addressing instructional problems support and assist new and beginning teachers in their practice build collaborative norms to enable teachers to give and receive ideas and receive assistance

(Dr. Bruce Joyce, "Staff Development Awareness Conference," Columbia, SC, January 1987) In 1989 the Technical Education Research Center (TERC) launched the Labnet project as a technology-supported teacher-enhancement program aimed at high school physics teachers (Roupp, Pfister, Drayton, & Gal, 1993, p.4). Researchers studying the project found that telecommunications helped teachers overcome a number of obstacles to professional development, most notably that of time for developmental activities (Honey, McMillan, Tsikalas, & Griswald, 1995). Becker was able to determine that certain variables had important independent relationships to teachers use and valuation outcomes and some did not. Three factors were especially significant: quality connectivity, computer expertise, and teacher pedagogical beliefs and practices that were constructivist-compatible. Three other factors adequate professional development, home access, and younger age of teacherswere next in importance (Becker, 1999). Success or failure of technology is more dependent on human and contextual factors than hardware or software (Ryan 1991). A variety of studies indicate that technology will have little effect unless teachers are adequately and appropriately trained (Office of Technology Assessment, 1995; Coley, Cradler, & Engel, 1997; Silverstein et al., 2000; Sandholtz, 2001)* In a report examining over 300 studies of technology use, authors concluded that teacher training was the most significant factor influencing the effective use of educational technology to improve student achievement. (SivinKachala & Bialo, 2000)* Follow-up evaluations of the BS/BC program in West Virginia showed that the greatest gains in student achievement occurred when teachers were trained in the use of technology (Schacter, 1999).* Rhode Island had 25 percent of all teachers participate in Teachers and Technology Initiative. Researchers concluded that the training (60 hours and sample units and laptop computers) gave teachers a solid foundation in the use of technology as a core component of their instructional practices (Henriquez & Riconscente, 1999, p.76).* In a paper discussing the cost, utility, and value of technology, Wahl (2000)* suggests that organizations should spend 30 percent of their budget on equipment and 70 percent on the human infrastructure to support ongoing training and technical assistance. Helping teachers to learn to integrate technology into curriculum is a critical factor in the successful implementation of technology in schools (Sivin-Kachala & Bialo, 2000).* In addition to receiving training on how to use technology instructionally, researchers also suggest that teachers need additional help in learning how to assess products created using technology (Penuel et al.,2000).* Longitudinal research examining teachers use of technology suggests that the support teachers need changes as they become more and more proficient in integrating technology into instruction (Sandholtz et el.,1997)*

This report includes a description of the Collaborative Coaching and Learning model of professional development implemented during the 2001-2002 school year in the Boston public schools and research on its effectiveness. (Barbara Neufeld and Dana Roper, Education Matters, Inc., July 2002, http://www.edmatters.org/webreports/boston/year1ofccl702final.pdf) Research about the effects of mentoring and also suggests model programs. Scroll down to the link for this article under the Departments section of the Web page. (John H. Holloway. Educational Leadership. May 2001, Vol. 58, No. 8. Association for Supervision and Curriculum Development, http://www.ascd.org/publications/ed_lead/200105/holloway.html) This article highlights one mentor program and an evaluation of its success. (Felicia Saffold. The Mentoring Leadership and Resource Network. December, 2003, http://www.mentors.net/03library/win_win.html) This study examines whether mentoring strategies are effective in the support and development of skills of beginning teachers, and if mentoring has a positive impact on students of beginning teachers. (Brenda Mahler. The Mentoring Leadership and Resource Network. May 2001, http://www.mentors.net/03library/beginningteacher.html) This article highlights the benefits that mentors derive from mentoring others. (Leslie Huling and Virginia Resta. ERIC Clearinghouse on Teaching and Teacher Education. ED460125. 2001, http://www.ericdigests.org/2002-3/mentoring.htm) Tips for mentoring and characteristics of successful mentors. Suggestions for selecting mentors, providing training, and getting started with a mentoring program are also provided. There is also a section on peer coaching and how to implement and support a peer coaching program. (Research Update, Institute for Educational Research. Reprinted by the Mentoring Leadership and Resource Network, http://www.mentors.net/03library/collab_pc.html)

ACCESSIBLE EQUIPMENT AND SUPPORT: Although studies are not conclusive about the optimal number of computers per classroom, research is clear that students and teachers are best served if they have convenient, consistent, and frequent access to technology (Mann, 1999).* Statham and Torell (1999) suggest that a 1:5 computer-to-student ratio would assure students near universal access. If students are to use computers to be better writers, researchers, and problem-solvers, they need to have access to computers when they are engaged in these processes, not only at regularly scheduled times (Knapp & Glenn, 1996).* Success or failure of technology depends on having significant critical mass numbers and types of technology

applications that are appropriate to the learning expectations of the activity [one computer to 4/5 students, located in classrooms or areas where learning is taking place, connected to the Internet, etc.] (Meers, et al, 1993; SivinKachala & Bialo, 1996). In Indiana, students participating in the Buddy project were supplied with home computers and modem access at school. Project students showed improvement in all writing skills, better understanding and broader view of math, more confidence with computer skills, an ability to teach others, greater problem-solving skills, and greater selfconfidence and self-esteem than non-project peers (Coley, 1997 p.4).*

Note: Meta-analyses are procedurally objective presentations of study features and outcomes, employing statistical methods to summarize overall findings and explore relationships between study features and outcomes. Computer-Based Technology and Learning: Evolving Uses and Expectations, NCREL 1999 *The Learning Return On Our Educational Technology Investment WestEd 2002 **Classroom Instruction in Gates Grantee Schools: A Baseline Report, Fouts and Associates 2002

Resources used by the state of Washingtons for their Technology Plan

OSPI, Washington State Educational Technology Plan: A Blueprint for Washingtons K-12 Common Schools and Learning Communities. December 2005 http://www.k12.wa.us/EdTech/techplan.aspx. Becker, H. (1999). Internet Use by Teachers: Conditions of Professional Use and Teacher-Directed Student Use. University of California, Irvine: Center for Research on Information Technology and Organizations. Retrieved September 16, 2002, from: http://www.crito.uci.edu/TLC/findings/Internet-Use/startpage.htm Bigelow, M., Jones, A., and Stead, R. Organization and Financing of Washington Public Schools. Olympia, WA: Office of Superintendent of Public Instruction. Borja, R. (2002). Internet Filtering is Balancing Act for Many Schools. Education Week, January 16, 2002. Byrom, E. (1998). Review of the Professional Literature on the Integration of Technology into Educational Programs. Retrieved September 16, 2002, from the SERVE Web site: http://www.serve.org/technology/litreview.html Dede, C. (2001). State Policy Framework for Assessing Educational Technology Implementation, Version 4. (unpublished manuscript). Education Week. (2002). E-Defining Education: How Virtual Schools and Online Instruction are Transforming Teaching and Learning. Retrieved September 16, 2002 from the Education Week Web site: http://www.edweek.org/sreports/tc02/ Friedman, D., and Erickson, K. (2000). Technology Alliance Survey: A Follow-up Analysis of Technology in Washington Schools (unpublished manuscript). Seattle, WA: Technology Alliance. Levin, D., and Arafeh, S. (2002). The Digital Disconnect: The Widening Gap Between Internet-Savvy Students and Their Schools. Retrieved September 16, 2002, from the Pew Internet and American Life Project Web site: http://www.pewinternet.org/reports/toc.asp?Report=67

Lemke, C., and Coughlin, E. (1998). Technology in American Schools: Seven Dimensions for Gauging Progress--A Policymaker's Guide. Retrieved September 16, 2002, from Milken Family Foundation Web site: http://www.mff.org/publications/publications.taf?page=158 National School Boards Foundation. (2002). Are We There Yet? Research and Guidelines on Schools' Use of the Internet. Retrieved September 16, 2002 from the NSBF Web site: http://www.nsbf.org/thereyet/fulltext.htm Newberger, E. (2001). Home Computers and Internet Use in the United States: August 2000. Washington, DC: U.S. Census Bureau. Office of Superintendent of Public Instruction. (2002). Washington State Consolidated Application For Federal Funds Under the Elementary and Secondary Education Act "No Child Left Behind." Retrieved September 16, 2006, from the OSPI Web site: http://www.k12.wa.us/ESEA/ Office of Superintendent of Public Instruction. (2006). Washington K-12 Building Technology Inventory Results. Retrieved September 16, 2006, from the OSPI Web site: http://www.k12.wa.us/EdTech/Survey.aspx Office of Superintendent of Public Instruction. (1994). Report to the Legislature: The Washington State Technology Plan for the K-12 Common School System. Retrieved September 16, 2006, from the OSPI Web site: http://www.k12.wa.us/EdTech/plan.aspx Ringstaff, C., and Kelley, L. (2002). The Learning Return on Our Educational Technology Investment: A Review of Findings from Research. San Francisco, CA: WestEd RTEC. Schacter, J. (1999). The Impact of Education Technology on Student Achievement: What the Most Current Research Has to Say. Retrieved September 16, 2002, from the Milken Family Foundation Web site: http://www.mff.org/publications/publications.taf?page=161 Smerdon, B., et al. (2000). Teachers' Tools for the 21st Century: A Report on Teachers' Use of Technology. Report No. NCES 2000-102. National Center for Education Statistics, U.S. Department of Education. Statham, D., and Torell, C. (1999). Technology in Public Education in the United States. Retrieved September 16, 2002, from http://www.tea.state.tx.us/Textbooks/archives/litrevie.htm Technology Alliance. (1998). Report and Recommendations: Findings of the Technology in Education Task Force. Seattle, WA: Author. Wells, J. and Lewis, L (2006). Internet Access in U.S. Public Schools and Classrooms: 1994-2005. White, N., Ringstaff, C., and Kelley, L. (2002). Getting the Most from Technology in Schools. San Francisco, CA: WestEd RTEC. Wilhelm, T. Carmen, D., and Reynolds, M. (2002). Connecting Kids to Technology: Challenges and Opportunities. Retrieved September 16, 2002, from the Annie E. Casey Foundation Web site: http://www.aecf.org/publications/data/snapshot_june2002.pdf