Kinect in Education: Gesture-based Learning Now and In the Future

INTE 6750 - Trends Project Professor Brent Wilson, PhD November 18, 2012 Team Tigers - Corinne Handy, Melanie Melvin and Duncan McBogg

Corinne

Melanie

Duncan

According to the 2011 Horizon Report, gesture-based computing in education is still a few years away, but its significance cannot be underestimated, especially for a new generation of students accustomed to touching, tapping, swiping, jumping, and moving as a means of engaging with information (Horizon Report, 2011). Driven by the popularity of Apples iOS and gaming systems such as the Nintendo Wii and Microsofts Xbox Kinect, gesture-based computing has strong potential in education, both for learning, as students will be able to interact with ideas and information in new ways, and for teaching, as faculty explore new ways to communicate ideas (Horizon Report, 2011). Gesture-based computing includes a handful of technologies such as touch screen and voice recognition technologies, however, this report will specifically focus on Microsofts Xbox Kinect gaming system and its potential uses in all levels of education, inside and outside of the classroom. This report will outline the inner workings of the technology and its current and future pedagogical applications, as well as the benefits, limitations and variables associated with its use in education. Kinects road to the mainstream was dependent on the general advances of digital technologies, the mass adoption of touch screen technologies, the general acceptance of avatars via gaming and popular culture. Popular culture has done a really good job of preparing us for the technology of the future. The introduction of the iPhone set the stage for mass adoption of gesture-based technologies driven by the human body alone. Apple continues to revolutionize and define the way in which gesture-

based technology is marketed, adopted and used, especially in the classroom. Microsoft is no slouch on the technology side, but the level of adoption for Kinect has yet to rival its Apple counterparts.

The Basic Elements of Kinect

Kinects gesture-based technology is a combination of several different technologies that interact with one another to create a complex technology capable of amazing feats by using the human body alone. The core technologies in the Kinect console are a digital camera sensor, a microphone array or a series of microphones, Infrared sensors, and huge amount of computing power operating in the background (See Figure 1). What separates the Kinect from its gaming competitors is its ability to capture real-time motion of the human body without needing a separate controller of any kind. According to Cong and Winters (n.d.) processing all of this data using a special artificial intelligence machine-learning algorithm allows the Kinect to map the visual data it collects to models representing people of different backgrounds, ages, heights, genders, body type, clothing and more.

Figure 1: The Inner Workings of Microsoft Kinect

The hardware that powers Kinect works in conjunction with a vast array of computer servers which operate behind the scenes. Cong and Winters summarize this interchange best: The Kinect's "brain" is really the secret. Stored in the system is enough intelligence to analyze what it sees and align that with stored collection of skeletal structures to interpret your movements. Once the brain has enough data on your body parts, it outputs this reference data into a simplified 3D avatar shape. Beyond gauging player movements, the Kinect must also judge the distances of different points on your body throughout the entire game. To do this it uses a host of sensors and analyzes all this data 30 times a second (Cong and Winters, n.d.). Data is constantly being transferred back and forth between the Kinect and the objects in its field of vision while you simply enjoy the fun of being a character in a game, without holding anything in your hands (Cong and Winters, n.d.).

The Future is Now

Some may think gesture-based gaming technology is a little far-fetched for use in education. However, only a decade or two ago the same was said about distance learning and videoconferencing and now, both are ubiquitous. Technologies such as 3D holograms and gesture-based gaming are quickly making their way into the mainstream, especially with younger generations also known as the digital native. These days, teenagers expect to see cutting edge technology as the norm, rather than the exception. Adoption of gesture-based gaming technologies in education is more a matter of convincing teachers about its potential relevance than it is convincing the students with which they serve. Popular media has already set the stage for adoption of far out technologies by showing a futuristic world with technologies that let you control a computer with only your hand, your voice and/or your physical expressions. These days, all you have to do is watch the latest television crime show to see the latest in gesture-based technology in action. Suffice it to say that TV show technology is driven by computer-assisted animation. Nonetheless, the psychology impact of these TV shows demonstrates that people are comfortable with the concept of gesture-based technologies in more areas of their life than their playrooms at home.

Scope
Explaining the intricate technical details of Kinects technology is well beyond the scope of this report. The emphasis of this report is on the current and potential uses of Kinect in education. For more information on the technology that powers Kinect, visit our website: One Stop Shop: Kinect in Education.

The Pedagogy
When researching the pros and cons of Microsoft Kinect in learning, there was an abundance of information on the benefits. Much of this information came either directly from Microsoft, or from someone quoting Microsoft. But there is also quite a bit of research done by external sources. One source used often throughout this report is KinectEDucation. Their website describes that KinectEDucation is an educator-driven community resource for developers, teachers, students, enthusiasts, and any other education stakeholder to promote the use of Kinect applications in classrooms.

The Cheerleader
While relatively new in the classroom setting, there is an abundance of information on the types of benefits Kinect can have in a learning environment. Just a few of these benefits are listed below. Cost When reviewing all the information on the benefits of using Kinect in the classroom, the biggest trend found was the cost of the device. While compared to any other gesture-based learning technology, Kinect stands alone. Kinect is relatively cheap as it costs around $149. If the classroom is equipped with a projector and a computer, Kinect can be regarded as an inexpensive add-on (Hsu, 2011, pg. 335). In other words, in order to add Kinect to a standard classroom, the setup requires a projector, computer, the Kinect device, and the software that comes with the device. Teachers also have the ability to create their own programs, or access programs other teachers have developed via the Kinect in the Classroom school activity plans page, or performing a Google search. Essentially, teachers can add this to their classes for relatively little money, and the opportunity for new lesson plan ideas is limitless. Active Learning Another benefit of using Kinect in the classroom is to promote active learning. Active learning is generally defined as any instructional method that engages students in the learning process. In short, active learning requires students to do meaningful learning activities and think about what they are doing (Prince, 2004). When using Kinect in the classroom, students dont only find out the what of what theyre learning, but also the how. They begin to connect the dots which allows for further understanding and deeper comprehension of the topic. According to the Horizon Report, gesture-based learning is an upcoming trend that will continue to be integrated into classrooms over the next few years (KinectEDucation, 2011). Kinect is easy to use and a cost effective gesture-based tool, so it is easy to see why it will become the tool of choice for teachers. Seamlessly Integrate Technology In order for technology to work effectively in a classroom, the learner needs to forget its there and focus on the learning instead. The most successful technology in the classroom is one where the learners dont even recognize that its there, and instead are engaged in the learning

itself. Kinect is seamless; as long as the software is well-programmed, user awareness of the device is rare (KinectEDucation, 2011).

Figure 2: First-grade teacher Cheryl Arnett introduced Kinect into her classroom at Sunset Elementary School in Craig, Colo., and has been delighted by the results, which include outstanding standardized test scores (Source).

Embrace Cultural Diversity Using Kinect in the classroom is a way to connect learners from many different countries, all over the world, at one time. In order to do this, Kinect allows the users to create their very own avatars. An avatar is a graphical image that represents a person over the Internet. When doing this, learners can make a visual representation of themselves, and meet in a common space to connect with others. Since the device is cheap, this excludes no population. For instance, Microsoft is piloting Kinect in Africa, a sure sign this device can reach learners from all demographics and socioeconomic statuses (KinectEDucation, 2011). Here is an example of different avatars that have been created. While using Kinect, the avatars represent how the user moves his or her body.

Figure 3: Side by side human and avatar (Source). Kinect can even recognize facial changes and the movement of eyebrow, and display them on screen as well.

Figure 4: Human and Avatar positive facial gesture comparison (Source).

Figure 4: Human and Avatar negative facial gesture comparison (Source).

For more information on the use of avatars in Kinect, please refer to the following YouTube demonstration: Avatar Kinect Microsoft CES Keynote. Establish Content Relevancy When it comes to science classes, it is not always practice to have a full, working lab for students to use. While a lab would take learning to the next level, the cost to build and maintain that lab could be far too much for the schools to afford. When combining science (and other classes) with Microsoft Kinect, these classrooms can come to life. Kinect can build a virtual classroom for students to explore. Augmented labs that integrate gestures and Kinect will bridge the gap between virtual labs and true-to-life labs (KinectEDucation, 2011). Kinect also helps students in other classes. Pilot programs in Los Angeles, Chicago, Houston, Scottsdale, Flagstaff, and Fairfax and Loudoun Counties in Virginia show that use Kinect show how teachers who use Kinect games in lesson plans are engaging students better on subjects such as mathematics, language arts, science, and social studies. They are also showing positive results in physical education, adaptive P.E., and special education (link: http://www.prnewswire.com/news-releases/teachers-are-using-kinect-for-xbox-360-to-engagestudents-and-bring-learning-to-life-141564563.html). Explore New Environments Using the Kinect system, students can explore in a hands-on way, topics that used to be just a page in a book. For example, instead of hearing about the universe or human body via a book or movie, students can now explore via Kinect. The following links are examples of how students can virtually explore the universe (http://www.youtube.com/watch?v=_qvMHAvu-yc) using Kinect, and the human body (http://www.youtube.com/watch?v=Zw_6o7AuBzk). Some experts will argue the idea of gesture-based learning in the classroom is far-fetched. It is true that this technology is new and has relatively little classroom exposure. But when we evaluate learning research, instructional needs, and the relatively inexpensive price for Kinect, it becomes further evident that this technology will be a 21st century tool to facilitate instruction (KinectEDucation, 2011). Special Needs Another benefit of using a tool like Kinect in the classroom is that it opens many doors for students with a variety of disabilities. Lacina (2009) summarizes the benefits of integrating technology into the classroom including meeting the needs of visual learners, more interactively teaching whole-class lessons, and better engaging students (Hsu, 2011).

Figures 5 and 6: The Power of Kinect in Special Education (Source).

Yeah but... the flip side

Kinect in the classroom is intriguing and exciting, but most of all it is emerging. Of course we want students to be engaged, active, multi-cultural, and given every opportunity to explore the world beyond the classroom. In theory the addition of Kinect to the classroom will meet the 4 steps in John Kellers ARCS Model of Motivational Design. To promote the learning process and sustain motivation, the ARCS Model starts with gaining the learners attention, which the Kinect activities will easily accomplish gaining the attention of the class. After gaining attention, the material needs to be relevant for the learner outside of the classroom. The Kinect will allow the student to make relevant connections using avatars and lively game-like environments. Succeeding in the activities and the ability to explore freely develops confidence. And the last step, satisfaction from this success is achieved with new exciting rewards such as the celebration of the students avatar on the screen after they successfully complete an activity (Keller, 2010). Who wouldnt want this? Of course we agree that the cost for the Kinect system in the classroom is relatively inexpensive. But is it realistic? What is the flip side of this coin? Even Microsoft admits a few things are needed for Kinect be successful in education: a reformed classroom model, software and a paradigm shift. Lack of Space Creating a classroom space that will best display the Kinect is the easiest of the three needed items, and that is not always easy. We will discuss the other two below. Teachers are used to moving desks and students, a classroom layout as suggested by KinectEDucation would be simple if there were empty space in the classroom, which is doubtful. Because the system is active, there must be space for the students to move freely and white space to project the game. There has to be enough space in the room for the screen and the sensor to detect the students. And even then, it takes time for the Kinect to track a new user in the classroom (Hsu, 2011). If even possible, there would be a need to rearrange classroom furniture, which would at the least take time and could be more of a distraction to students than the benefit of the activity.

Lack of Material Because it is just emerging, the likelihood of the average teacher being interested in adding a Kinect lesson to their immediate plans is slight. The setup of the Kinect system is not plug and play and the availability of developed content is minimal. The average teacher would need to spend time setting up the system at the least and would need to be quite tech savvy to write a game. Kinect in the Classroom website has over 200 ready-to-go activity plans. Which may sound like plenty. But this is for every subject, every age and every possible Kinect game. Many lesson/games require additional purchases which can add to the initial purchase price. For example, there are less than twenty math games for ages 5-10. The limited amount of material leaves teachers with hours of setup time to play a limited number of games with his or her class. There are few educators that have the ability or are going to take the time to develop software. Creating games takes a higher-level computer skill. The launch of the Kinect for Windows Software Development Kit (SDK) has made game development more user friendly. Microsoft has also added Kodu, which allows students to participate in the game creation using an iconbased programming language for designing, programming and testing games. These additions are helpful, but the setup is still not simple. Lack of Time Calibration of the Kinect takes a long amount of time, which can take away from instructional time or planning time. When it is time to use the Kinect with the students, the system may need to be recalibrated for each student to be recognized or the game may stall while the system is recalibrating. Delays such as this in a classroom of excited students can be disastrous. The Novelty Factor There is no doubt that students will be more engaged in learning and excited when they see the Kinect set up in their classroom. There is also no disputing the Kinect system allowing educators to appeal to all learning styles. At question is whether the addition of the system will promote the lesson or be a distraction? Charlie Osborn (2012) refers to this threat as the novelty factor. Playing a game with the Kinect, while fun, is not always the best way to convey the lesson material to the learner. Teachers have to decide if it is the best method for their material, their learners and their classroom setup. The curriculum still has to be taught and the tests still have to be taken. Is the addition of this cool technology going to supplement the current systems and processes or is it going to be a hindrance? Administrative Opinion and Approval No doubt, it is a great value in a world full of declining budgets. But if the tool is not used or understood by the teachers, it is money wasted. Most school systems are still offering training on the use of email and the interactive white boards if they are lucky enough to even have them. One would assume that they are offering this training because it is needed if this is the case,

these schools are not ready for a Kinect in every classroom. Fiscal decision-makers have to choose technology that is beneficial to the most students and the most teachers. Speaking of politics and budgets, The US Department of Education cites the number one reason that teachers quit or are likely to quit as a constant battle with administration. This is the same administration charged with convincing early adopting teachers that they will be adding value to their lesson plans by bringing in a gaming system to the classroom. If the teachers are already in a constant battle, it seems that they dont want to start another one. Regarding the paradigm shift, the KinectEDucation blog agrees that just the association with videogames is a hurdle for educators to get passed before the system can be adopted. The administrators and those that have to justify purchases will seriously question the ingenuity of purchasing a Kinect. The cost may not be as much of an issue as the perception of having a video game in the classroom.

Conclusion
The potential of Kinect in education seems fairly limitless. Kinect has every aspect of an effective education tool. Kinect is engaging, high quality, immersive, and customizable. The idea of meeting kids where they are and introducing learning modules on devices they are already using sets the stage for future widespread adoption, both in the classroom and in online teaching environments. As with any technology, educators should not assume that everyone who wants access to it actually gets the access they so desire, especially those that cannot afford it. We can see why the Horizon Report named gesture-based technology as a technology to watch in the upcoming years. With Kinect, Microsoft has made it possible for human beings to control technology like never before. However, Kinect gestures, and more importantly the learning curve associated with learning these gestures has a little ways to go before the average teacher will adopt it as a learning tool in the classroom. In the short term, we are convinced the typical educator will not spend the amount of time it takes to learn, configure, and lesson plan around Kinect. We can see niche applications for the Kinect in education, e.g., teaching with technology, data visualization, and special needs applications. In the end, we found a fair amount of subjective high-level promotional materials, but very few concrete case studies to support Kinects effectiveness in a real world classroom setting with actual students. As the Horizon Report suggests, the topic is in its infancy, so we expect the level of rigor to substantially increase over time. At present, the technology savvy teacher may be excited about using a Kinect activity in their classroom. They may even spend their weekends developing software. However, the average teacher is too overwhelmed, and the Kinect is under-supported to make this technology anything more than emerging and novel. We posit that if professional course development were funded with grants and Microsoft dollars, it could be more useful for the masses. Otherwise, the technology may remain in the classrooms of a few highly motivated teachers.

References
Carmody, T. (2010). How motion detection works in Xbox Kinect: Wired Magazine Online. Retrieved from http://www.wired.com/gadgetlab/2010/11/tonights-release-xbox-kinect-howdoes-it-work/ Cong. R., & Winters, R. (n.d.). How it works: gaming technology takes a giant leap: JameCo Electronics: Retrieved from http://www.jameco.com/Jameco/workshop/howitworks/xboxkinect.html?sp_rid=MjU5NDM3O DA5MjkS1&sp_mid=3910917 De Ruimte School (n.d.). The power of Kinect in Special Education: Retrieved from https://www.heliomare.nl/download/Word%20en%20PDF%20bestanden/Onderwijs/Folder% 20Kinect%20%20%20%208%203%202012.pdf Hsu, H.J. (2011). The Potential of Kinect as Interactive Educational Technology: - IPEDR vol.13. IACSIT Press, Singapore. Retrieved from: http://www.ipedr.net/vol13/64-T10050.pdf Inglish, P. (2012). Top 5 reasons many teachers quit: Retrieved November 7, 2012 from: http://pattyinglishms.hubpages.com/hub/Teachers_Quit

Keller, J.M. (2010). Motivational Design for Learning and Performance: The ARCS Model Approach. Retrieved from: http://www.arcsmodel.com/Mot%20dsgn%20A%20model.htm KinectEDucation. (2012). 5 benefits of using Kinect in education: Retrieved from http://www.kinecteducation.com/blog/2011/07/02/5-benefits-of-using-kinect-in-education/ KinectEDucation (2012). Three things Kinect needs to be successful in education: Retrieved from: http://www.kinecteducation.com/blog/2011/07/30/3-things-kinect-needs-to-besuccessful-in-education/ Michael. P. (2004). Does Active Learning Work? A Review of the Research: Journal of Engineering Education, 93(3), 223-231 (July, 2004). Retrieved from: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.6544&rep=rep1&type=pdf Osborn, Charlie. (2012). Gesture-based tech: A future in education? Retrieved from: http://www.zdnet.com/blog/igeneration/gesture-based-tech-a-future-in-education/15514. State of Victoria Department of Education and Early Childhood Development (2010). The impact of web 2.0 technologies in the classroom: Retrieved from: http://csamarktng.vo.msecnd.net/kodu/pdf/web20_technologies_in_the_classroom_kodu.pdf