Dev Project Report

Smart Home Technology for elderly using ANN
ABSTRACT
The control of elderly people has always been a major concern in hospitals as in homes.
Our aim is to control and surround elderly people with comfort, safety in their homes while
respecting their privacy. In this project we propose a monitoring system based on Kinect sensor
to control and monitor elderly people with less mobility through hand gestures in a smart home.
The system recognizes hand gestures and communicates them through a network.
The basic idea behind this project is to find out how a Smart Home can be controlled by using
hand gestures. Here we give gestures by using hands which is recorded by the kinect camera, by
using matlab codes, and is sent to the computer, which is further interfaced with a home
automation kit and thus we define the codes for the function of smart home on our hand
gestures.
Dept. of Telecommunication 2015-16 Page 1

TABLE OF CONTENTS
CHAPTER 1: INTRODUCTION
1.1 Background 08
1.2 Ageing Population: Statistics and Growth Rates 08
1.3 Increasing Life Expectancy 10
1.4 Socio-Economic and Health related Determinants 11
1.5 Elderly Population in India 12
1.6 Need for Early Detection of Ageing Changes 13
1.7 Our Approach and Solution 14
1.8 Conclusion 15
CHAPTER 2: KINECT WORKING AND PRINCIPLE
2.1 Introduction 16
2.2 Kinect camera 18
2.3 Kinect for Windows 19
2.4 Kinect Versions 20
2.5 Software 21
2.6 Kinect for XBOX ONE 22

2.7 Reception 23
2.8 Parts of Kinect Camera 25
2.8.1 Depth Sensor 26
2.8.2 Color Sensor 27
CHAPTER 3: GESTURE RECOGNITION
3.1 Introduction 28
3.2 Problem Statement 29
3.3 Previous Work and Modification 30
3.4 Hand Gestures 31
3.5 Images Captured in Kinect 33
CHAPTER 4: ARTIFICIAL NEURAL NETWORK

4.1 Introduction 34
4.2 Neuron in Brain 37
4.3 Characteristics of ANN 37
4.4 Activation Function 40
CHAPTER 5: HARDWARE IMPLEMENTATION

5.1 Open Source Microcontroller 41
5.1.1 Power 43

5.1.2 Input and Output 44
5.1.3 Duty Cycle 46
5.1.4 Getting Started 47
5.1.5 General Code Structure 49
5.2 Relay Switches 51
CHAPTER 6: PROGRAM CODES
6.1 Main Hardware Program 52
6.2 ANN Training Program 58
CHAPTER 7: ADVANTAGES AND DISADVANTAGES
7.1 Advantages 59
7.2 Disadvantages 59
CHAPTER 8: APPLICATIONS AND CONCLUSIONS
8.1 Applications 60
8.2 Conclusions 61
REFERENCES 62

LIST OF FIGURES
CHAPTER 1
1.1 Countries, Population Share by Age Group 10
1.2 Life Expectancy around the World 11
CHAPTER 2
2.1 XBOX with Television Set 24
2.2 Kinect Camera 25
CHAPTER 3
3.1 Gesture Display 28
3.2 Accelerometer Gloves 29
3.3 Tracking of Gesture 1 31
CHAPTER 4
4.1 Neural Network 34

4.2 Characteristics of ANN 37
4.3 Gaussian Function 38
4.4 Threshold Function 38
4.5 Sigmoid Function 39
4.6 Flowchart of ANN Working 40
CHAPTER 5
5.1 Arduino uc type 41
5.2 PWM Duty Cycle 46
5.3 Relay Switches 50
5.4 5-point Relay 51
CHAPTER 8
8.1 Gestures used in Gaming Implementation 60

LIST OF TABLES
1. Percentages of Older Ages by Regions 09
2. Comparison between Different Microcontrollers 42

Chapter 1
INTRODUCTION
1.1. Background
The population throughout the world is growing. It is considered as a human success story,
through triumph of public medical and health advancements. But this ageing process also
puts a lot of challenges regarding national development, issues concerning heath of the
elderly individual, the sustainability of families, and the ability of health care system to
provide for ageing populations. The terms “Elderly”, “Older population” and “Senior
citizens” are generalized to refer to people aged 60 years or older. People aged 80 years or
older is referred as the oldest of old. Between 2000 and 2050, the share of the population aged
60 and over is projected to increase in every country in the world; the same is true for the
80+ population. The number of people over the age of 60 is expected to reach 1 billion by
2020 and almost 2 billion by 2050 (representing 22 percent of the world’s population).
The proportion of individuals aged 80 or over (the so-called “oldest old”) is projected to rise
from 1 percent to 4 percent of the global population by 2050. Practically in all regions of the
world the older population is growing in an unprecedented rate than the normal population and
also the difference in the growth rates is ever increasing. In the current situation, ageing is not
just limited to an individual or community, but as a world.
1.2. Ageing Population: Statistics and Growth rate

The term population ageing is defined as the percentage of population aged 65 or older to
the given population. The world’s elderly population has been growing at an accelerated pace
and projections suggest that the annual net gain will continue to exceed 10 million over the
next decade-more than 850,000 each month. Today, one in every nine persons in the world is
60 years or older.

In 2009, an estimated 737 million persons were aged 60 years or over and constituted the
“older population” of the world, nearly two thirds of whom lived in developing countries.
Their number is projected to increase to 2 billion in 2050, by which time older persons
will outnumber children (persons aged 0 to 14 years). Today, more than half of the older
population lives in Asia (54 per cent) and a fifth lives in Europe (21 per cent). The
projections suggest that elderly population around the world will outnumber the children
under age 5 for the first time in the history in the coming 20 years. Also the oldest old that
is people aged 80 and above is rising as shown in Table 1.1.
Table 1.1: Percentage of population in older ages by region, 2008, 2020, and 2040

1.3. Increasing Life Expectancy

The demographic projections of population ageing indicate that the world is experiencing
a historically unprecedented phenomenon. During the recent years the number of persons
aged 60 or over has increased from 200 million in 1950 to around 670 million. The ageing
people accounts for a substantial proportion of the total world population. Many countries are
forecast to see a higher share of people aged 60+ in the coming years. Even in the
developed world much of the population already has large number of aged people. People
aged 60 and over consist of twenty percent of the total population in developed countries
today, and this will rise to over 30 percent in the next four decades as shown in Figure 1.1.
In the developing world, less than 10 percent of the population is over the age of 60 years.
By 2050, however, the proportion is expected to more than double, and the 60+ age group
will comprise 20 percent of India’s population and 30 percent of China’s by the middle of
this century—a total of over 760 million people.
Figure 1.1. Countries, population share by age group
Over the past century, the life expectancy of human has increased and has become almost
doubled. The maximum life span—the longest number of years a human being has lived has
increased spectacularly as well. There is little disagreement over these facts.The growth of the
world population can be seen in the Figure 1.2, as is shown below.
Figure 1.2: Life Expectancy around the world
1.4. Socio-Economic and Health Related Determinants
Diverse studies have identified a number of socio-economic factors that affect health and
longevity. Elderly people who are living alone will get affected by their life
satisfaction, health, and chances of institutionalization. Availability of kin, personal
wealth, health, and individual preferences are key determinants of older persons living
arrangements. Also in many places the cultural norms and money are equally important in
deciding whether an older person lives alone or with family members. A Majority of older
people, especially older women in developed countries often live alone, while living with
kin is still the norm of developing nation.

According to the Australian Institute of Health and Welfare in 2008, elderly women
and the “oldest old” are likely to be overrepresented among long-term care residents.
These long-term care facilities usually include nursing homes, assisted living/residential
care facilities, and sometimes hospice centres. There is an increased demand in
institutional residence, as it predominantly becoming a lone option for older people
in developed countries who have difficulties with activities of daily living or who
require specialized medical services. These facilities come with a price and not all
elderly people can afford. This can cause an immense stress on the scarce resources of
a country to care for the elderly. According to current estimates, health care in many
countries including US, EU and much of Asia is now a significant part of the economy.
It is estimated that the predicted health cost in US alone will be 2.9 trillion dollars in
2015, which is approximately 17.8% of the GDP.
Coupled with this, the present workforce shortage poses difficulty with the cost of
elderly care, since 55-60% of the costs incurred during health care is mainly the high
labor costs. Therefore, the importance of enabling the elderly to live in their own
home as long as possible is crucial. However, if they prefer to live alone they do
require constant monitoring so that medical help can be provided immediately in
times of dire needs.
1.5 Elderly Population in India
A report jointly brought out by United Nations Population Fund (UNFPA) and Help Age
International says the following: India’s population is likely to increase by 60 per cent
between 2000 and 2050 but the number of elders, who have attained 60 years of age, will
shoot up by 360 per cent and the government should start framing policies now else its
consequences are likely to take it by surprise. India has around 100 million elderlies at
present and the number is expected to increase to 323 million, constituting 20 per cent of the
total population, by 2050. We take this opportunity to understand some fundamentals about
the elderly population of India, their rights and some terms for your examination.
Definition of a Senior Citizen Elderly or old age consists of ages nearing or surpassing the
average life span of human beings. The boundary of old age cannot be defined exactly
because it does not have the same meaning in all societies. Government of India adopted

‘National Policy on Older Persons’ in January, 1999. The policy defines ‘senior citizen’ or
‘elderly’ as a person who is of age 60 years or above. Also, as per Maintenance and Welfare
of Parents and Senior Citizens Act, 2007, senior citizen means any person being a citizen of
India, who has attained the age of 60 years or above.
1.6 Need for Early Detection of Ageing Changes
The increase ageing of population in the world is associated with the increase of
suffering with many disabilities. So there is a need for an in-home pervasive network
which assists and helps the residents by controlling home appliances, medical data look
back and also communicating in case of an emergency. Too often are headlines such as
these seen in the paper: “Elderly man lay dead for days”, and “Woman found starved in
flat”. It is shocking to know that no one was looking after these people and imagine they
must have an uncaring family. At the same time however, as a society we value our
rights to live independently and keep control of our own lives. Many dread the thought
of being forced to live with their adult children, or in a rest home or other sheltered
living arrangement yet at the same time they know there is a high risk of death because of
a collapse, a fall, or stroke. There are many people in our community who because of
age or some infirmity, or perhaps because their memory and judgment can no longer be
totally relied upon, are having pressure put on them by loving relatives to leave their home
and give up their precious independence.
Any abnormal events which can occur to an old person in a home often leads to more
serious illnesses or even death. But in a case of reduced mobility or other factors
which leads to this kind of situation should also be considered for effective monitoring,
which will make a significant impact on the health of the elderly people. Surely, with the
technology of today, there is a better way for these people to resolve this problem. This
technological assistance or monitoring of a person in the home is achieved using few but
effective wireless sensors, which are centralized in structure and distributed around the
house. The smart home concept is a promising way to improve the living standards of
elderly by improving the home care for the elderly. The smart home monitoring can

circumvent institutionalizing the older persons and can help them live at home in safety
and provide independence. The smart homes target to improve comfort, quality of life,
safety, monitoring, monitoring mobility and physiological parameters. Modern sensor not
only assists and monitors people with reduced physical functions but helps to resolve the
social isolation they face. They are capable of providing assistance without limiting or
disturbing the resident’s daily routine, giving him or her greater comfort, pleasure, and
well-being. A number of smart homes have now been developed around the world by
many institutes and researchers. The smart home is based on smart and intelligent
sensors, which are developed, fabricated and configured around a wireless network. It is
expected that these smart homes can reduce escalating medical costs.
1.7 Our Approach and Solution
Recent advances in sensor technology, communication systems, and information

technologies have created ample opportunities to develop novel tools enabling remote
monitoring in-case of emergency conditions, and to care the elderly. In-home monitoring
has the added benefit of evaluating individualized health status and reporting it to care
providers, and caregivers alike; allowing timelier and individually targeted preventive
interventions. The smart homes equipped with the wireless sensor networks will benefit
both health care providers and their patients. By monitoring the patient continuously and
reporting any abnormal situation, the system frees human labour and thus reducing
labour costs and increasing efficiency by notifying doctors or health care provider
quickly. The data collected from various sensors over the network in a smart home can
be stored and integrated into comprehensive health record. Other issues like quality of life
for elder people, such as privacy, independence, dignity and convenience are supported
and enhanced by the ability to provide services in the patient’s own home. It is a known
fact that the use and the way of implementation of wireless sensor networks drastically
change from one application to other application; the need for in-depth study on
performance parameters is vital. While several sensors are readily available off the shelf,
making them “intelligent” in the context of a specific application (such as monitoring of
the elderly) is always a challenging task. For example, an intelligent wireless sensor

system will not only detect the usage pattern of the daily appliances, it will have
capabilities to collate the data and analyze them. Depending on the data analysis it can
detect any abnormality.
To achieve this, I have developed and implemented a SMART component-based

system by integrating various sensors and communicating via standard radio frequency
protocols. The system depends on a set of selected number of wireless sensors and
controller which relies on inputs from sensors. This system will consist of a proof-of-
concept that what I have developed is feasible, reliable, practical and scalable.
1.8 Conclusion
An insight of elderly population growth across the world is discussed in this chapter.
According to the statistics there is increasing number of elderly people around the
world. Since elderly people are more prone to accidents if they prefer to live alone in their
own home, a need of technological assistance and monitoring is paramount.
By integrating sensors in the house hold of an elderly person they can be monitored
constantly. The smart home monitoring using wireless sensors concept is a promising way
to improve the living standards of elderly by improving the home care for the elderly.

Chapter 2
KINECT WORKING AND PRINCIPLE
2.1 Introduction
Kinect is a line of motion sensing input devices by Microsoft for Xbox 360 and Xbox
One video game consoles and Windows PCs. Based around a webcam-style add-
on peripheral, it enables users to control and interact with their console/computer without the
need for a game controller, through a natural user interface using gestures and spoken
commands. The first-generation Kinect was first introduced in November 2010 in an attempt
to broaden Xbox 360's audience beyond its typical gamer base. A version for Windows was
released on February 1, 2012. Kinect competes with several motion controllers on other home
consoles, such as Wii Remote Plus for Wii and Wii U,PlayStation Move/PlayStation
Eye for PlayStation 3, and PlayStation Camera for PlayStation 4. Microsoft released the
Kinect software development kit for Windows 7 on June 16, 2011. This SDK was meant to
allow developers to write Kinecting apps in C++/CLI, C#, or Visual Basic .NET.
We love Microsoft's Kinect 3D sensor, and not just because you can play games with it. At a
mere $150, it's a dirt-cheap way to bring depth sensing and 3D vision to robots, and while
open-source USB drivers made it easy, a forthcoming Windows SDK from
Microsoft promises to make it even easier.
Kinect, which is actually hardware made by an Israeli company called Prime Sense, works by
projecting an infrared laser pattern onto nearby objects. A dedicated IR sensor picks up on the
laser to determine distance for each pixel, and that information is then mapped onto an image
from a standard RGB camera. What you end up with is an RGBD image, where each pixel
has both a colour and a distance, which you can then use to map out body positions, gestures,
motion, or even generate 3D maps. Needless to say, this is an awesome capability to
incorporate into a robot, and the cheap price makes it accessible to a huge audience.
Kinect was first announced on June 1, 2009 at E3 2009 under the code name "Project
Natal". Following in Microsoft's tradition of using cities as code names, "Project Natal" was
named after the Brazilian city of Natal as a tribute to the country by Brazilian-born Microsoft
director Alex Kipman, who incubated the project. The name Natal was also chosen because
the word natal means "of or relating to birth", reflecting Microsoft's view of the project as
"the birth of the next generation of home entertainment".
It was rumored that the launch of Project Natal would be accompanied with the release of a
new Xbox 360 console (as either a new retail configuration a significant design revision
and/or a modest hardware upgrade ). Microsoft dismissed the reports in public and repeatedly
emphasized that Project Natal would be fully compatible with all Xbox 360 consoles.
Microsoft indicated that the company considers it to be a significant initiative, as fundamental
to Xbox brand as Xbox Live, and with a launch akin to that of a new Xbox console platform.
Kinect was even referred to as a "new Xbox" by Microsoft CEO Steve Ballmer at a speech
for Executives' Club of Chicago. When asked if the introduction will extend the time before
the next-generation console platform is launched (historically about 5 years between
platforms), Microsoft corporate vice president Shane Kim reaffirmed that the company
believes that the life cycle of Xbox 360 will last through 2015 (10 years).
On June 16, 2011, Microsoft announced its official release of its SDK for non-commercial
use. On July 21, 2011, Microsoft announced that the first ever white Kinect sensor would be
available as part of "Xbox 360 Limited Edition Kinect Star Wars Bundle", which also
includes custom a Star Wars-themed console and controller, and copies of Kinect
Adventures and Star Wars Kinect. Previously, all Kinect sensors had been glossy black.

On October 31, 2011, Microsoft announced launching of the commercial version of Kinect
for Windows program with release of SDK to companies. On February 1, 2012, Microsoft
released the commercial version of Kinect for Windows SDK and told that more than 300
companies from over 25 countries are working on Kinect-ready apps.
2.2 Kinect Camera

Microsoft had an advertising budget of US$500 million for the launch of Kinect, a larger sum
than the investment at launch of Xbox console. The marketing campaign You Are the
Controller, aiming to reach new audiences, included advertisements on Kellogg's cereal
boxes and Pepsi bottles, commercials during shows such as Dancing with the
Stars and Glee as well as print ads in various magazines such as People and InStyle.
On October 19, Microsoft advertised Kinect on The Oprah Winfrey Show by giving free
Xbox 360 consoles and Kinect sensors to the people in the audience. Two weeks later,
Kinect bundles with Xbox 360 consoles were also given away to the audience of Late Night
with Jimmy Fallon. On October 23, Microsoft held a pre-launch party for Kinect in Beverly
Hills. The party was hosted by Ashley Tisdaleand was attended by soccer star David
Beckham and his three sons, Cruz, Brooklyn, and Romeo. Guests were treated to sessions
with Dance Central and Kinect Adventures, followed by Tisdale having a Kinect voice chat
with Nick Cannon. Between November 1 and 28,Burger King gave away a free Kinect
bundle "every 15 minutes".
A major event was organized on November 3 in Times Square, where singer Ne-
Yo performed with hundreds of dancers in anticipation of Kinect's midnight launch. During
the festivities, Microsoft gave away T-shirts and Kinect games.
Kinect was launched in North America on November 4, 2010, in Europe on November 10,
2010, in Australia, New Zealand and Singapore on November 18, 2010, and in Japan on
November 20, 2010. Purchase options for the sensor peripheral include a bundle with the
game Kinect Adventures and console bundles with either a 4 GB or 250 GB Xbox 360
console and Kinect Adventures.

2.3 Kinect for Windows

On February 21, 2011 Microsoft announced that it would release a non-commercial
Kinect software development kit (SDK) for Windows in spring 2011, which was released for
Windows 7 on June 16, 2011 in 12 countries. The SDK includes Windows
7 compatible PC drivers for Kinect device. It provides Kinect capabilities to developers to
build applications with C++, C#, or Visual Basic by using Microsoft Visual Studio 2010 and
includes following features:
1. Raw sensor streams: Access to low-level streams from the depth sensor, color camera
sensor, and four-element microphone array.
2. Skeletal tracking: The capability to track the skeleton image of one or two
people moving within Kinect's field of view for gesture-driven applications.
3. Advanced audio capabilities: Audio processing capabilities include
sophisticated acoustic noise suppression and echo cancellation, beam formation to
identify the current sound source, and integration with Windows speech
recognition API.
4. Sample code and Documentation.
In March 2012, Craig Eisler, the general manager of Kinect for Windows said that almost
350 companies are working with Microsoft on custom Kinect applications for Windows.

2.4 Kinect Versions

Version 1.5
In March 2012, Microsoft announced that next version of Kinect for Windows SDK would be
available in May 2012. Kinect for Windows 1.5 was released on May 21, 2012. It adds new
features, support for many new languages and debut in 19 more countries.
1. Kinect for Windows 1.5 SDK would include 'Kinect Studio' a new app that allows
developers to record, playback, and debug clips of users interacting with applications.
2. Support for new "seated" or "10-joint" skeletal system that will let apps track the
head, neck, and arms of a Kinect user - whether they're sitting down or standing;
which would work in default and near mode.
3. Support for four new languages for speech recognition – French, Spanish, Italian, and
Japanese. Additionally it would add support for regional dialects of these languages
along with English.
4. It would be available in Hong Kong, South Korea, and Taiwan in May and Austria,
Belgium, Brazil, Denmark, Finland, India, the Netherlands, Norway, Portugal,
Russia, Saudi Arabia, Singapore, South Africa, Sweden, Switzerland and the United
Arab Emirates in June.
Version 1.6, 1.7 and 1.8
Kinect for Windows SDK for the first-generation sensor was updated a few more times, with
version 1.6 released October 8, 2012, version 1.7 released March 18, 2013, and version 1.8
released September 17, 2013.
Version 2
See also: Kinect for Xbox One
The second-generation Kinect for Windows, based on the same core technology as Kinect for
Xbox One, including a new sensor, was first released in 2014.

2.5 Software
Requiring at least 190 MB of available storage space, Kinect system software allows users to
operate Xbox 360 Dashboard console user interface through voice commands and hand
gestures. Techniques such as voice recognition and facial recognition are employed to
automatically identify users. Among the applications for Kinect is Video Kinect, which
enables voice chat or video chat with other Xbox 360 users or users of Windows Live
Messenger. The application can use Kinect's tracking functionality and Kinect sensor's
motorized pivot to keep users in frame even as they move around. Other applications with
Kinect support include ESPN, Zune Marketplace, Netflix, Hulu Plus andLast.fm. Microsoft
later confirmed that all forthcoming applications would require them to have Kinect
functionality for certification.
Games that require Kinect have a purple sticker on them with a white silhouette of Kinect
sensor and "Requires Kinect Sensor" underneath in white text, and also come in purple
packaging. Games that have optional Kinect support (meaning that Kinect is not necessary to
play the game or that there are optional Kinect minigames included) feature a standard green
Xbox 360 case with a purple bar underneath the header, a silhouette of Kinect sensor and
"Better with Kinect Sensor" next to it in white text.

2.6 Kinect for Xbox One

Xbox One consoles ship with an updated version of Kinect; the new Kinect uses a wide-
angle time-of-flight camera, and processes 2 gigabits of data per second to read its
environment. The new Kinect has greater accuracy with three times the fidelity over its
predecessor and can track without visible light by using an active IR sensor. It has a 60%
wider field of vision that can detect a user up to 3 feet from the sensor, compared to six feet
for the original Kinect, and can track up to 6 skeletons at once. It can also detect a
player's heart rate, facial expression, the position and orientation of 25 individual joints
(including thumbs), the weight put on each limb, speed of player movements, and track
gestures performed with a standard controller. Kinect's microphone is used to provide voice
commands for actions such as navigation, starting games, and waking the console from sleep
mode.
All Xbox One consoles were initially shipped with the Kinect sensor included—a holdover
from a previously-announced, but retracted mandate requiring Kinect to be plugged into the
console at all times for it to function. In June 2014, bundles without Kinect were made
available, along with an updated Xbox One SDK allowing game developers to explicitly
disable Kinect skeletal tracking, freeing up system resources that were previously reserved
for Kinect even if it was disabled or unplugged.
A standalone Kinect for Xbox One, bundled with a digital copy of Dance Central Spotlight,
was released on October 7, 2014.

2.7 Reception
IGN gave the device 7.5 out of 10, saying that "Kinect can be a tremendous amount of fun for
casual players, and the creative, controller-free concept is undeniably appealing", though
adding that for "$149.99, a motion-tracking camera add-on for Xbox 360 is a tough sell,
especially considering that the entry level variation of Xbox 360 itself is only $199.99".
Game Informer rated Kinect 8 out of 10, praising the technology but noting that the
experience takes a while to get used to and that the spatial requirement may pose a barrier.
Computer and Video Games called the device a technological gem and applauded the gesture
and voice controls, while criticizing the launch lineup and Kinect Hub.
CNET's review pointed out how Kinect keeps players active with its full-body motion
sensing but criticized the learning curve, the additional power supply needed for older Xbox
360 consoles and the space requirements. Engadget, too, listed the large space requirements
as a negative, along with Kinect's launch lineup and the slowness of the hand gesture UI. The
review praised the system's powerful technology and the potential of its yoga and dance
games. Kotaku considered the device revolutionary upon first use but noted that games were
sometimes unable to recognize gestures or had slow responses, concluding that Kinect is "not
must-own yet, more like must-eventually own." TechRadar praised the voice control and saw
a great deal of potential in the device whose lag and space requirements were identified as
issues. Gizmodo also noted Kinect's potential and expressed curiosity in how more
mainstream titles would utilize the technology. Ars Technica's review expressed concern that
the core feature of Kinect, its lack of a controller, would hamper development of games
beyond those that have either stationary players or control the player's movement
automatically.

The mainstream press also reviewed Kinect. USA Today compared it to the futuristic control
scheme seen in Minority Report, stating that "playing games feels great" and giving the
device 3.5 out of 4 stars. David Pogue from The New York Times predicted you will feel a
"crazy, magical, omigosh rush the first time you try the Kinect [sic]." Despite calling the
motion tracking less precise than Wii's implementation, Pogue concluded that "Kinect’s
astonishing technology creates a completely new activity that’s social, age-spanning and even
athletic." The Globe and Mail titled Kinect as setting a "new standard for motion control."
The slight input lag between making a physical movement and Kinect registering it was not
considered a major issue with most games, and the review called Kinect "a good and
innovative product," rating it 3.5 out of 4 stars.
FIG 2.1 XBOX WITH TELEVISION SET

2.8 Parts of Kinect Camera

Kinect builds on software technology developed internally by Rare, a subsidiary of Microsoft
Game Studios owned by Microsoft, and on range camera technology
by Israeli developer PrimeSense, which developed a system that can interpret specific
gestures, making completely hands-free control of electronic devices possible by using
an infrared projector and camera and a special microchip to track the movement of objects
and individuals in three dimensions. This 3D scanner system called Light Coding employs a
variant of image-based 3D reconstruction.
Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is
designed to be positioned lengthwise above or below the video display. The device features
an "RGB camera, depth sensor and multi-array microphone running proprietary software",
which provide full-body 3D motion capture, facial recognition and voice recognition
capabilities. At launch, voice recognition was only made available in Japan, United Kingdom,
Canada and United States. Mainland Europe received the feature later in spring 2011.
FIG 2.2 KINECT CAMERA

2.8.1 Depth Sensor

The depth sensor consists of an infrared laser projector combined with a monochrome CMOS
sensor, which captures video data in 3D under any ambient light conditions. The sensing
range of the depth sensor is adjustable, and Kinect software is capable of automatically
calibrating the sensor based on game play and the player's physical environment,
accommodating for the presence of furniture or other obstacles.
Described by Microsoft personnel as the primary innovation of Kinect, the software

technology enables advanced gesture recognition, facial recognition and voice recognition.
According to information supplied to retailers, Kinect is capable of simultaneously tracking
up to six people, including two active players for motion analysis with afeature extraction of
20 joints per player. However, PrimeSense has stated that the number of people the device
can "see" (but not process as players) is only limited by how many will fit in the field-of-
view of the camera.
In 2000 GestureTek transformed the way people interact with computing displays by
inventing and patenting 3D video gesture control. Our pioneering of this technology over the
last 10 plus years has resulted in a multi-patented3D-vision gesture control system, which is a
highly precise and reliable user interface for interacting with any display screen from any
distance. Whether on a personal computer, set top box, television set, mobile device, game
console, digital sign or interactive kiosk, GestureTek's ground-breaking depth tracking
software enables users to control onscreen interaction with simple hand motions instead of a
remote control, keyboard or touch screen. This technology has been used for custom
installations for Fortune 500 companies over the last 10 years, and GestureTek continues to
offer this advanced state of the art technology and service to its clients. Some of the
technology and patents that are the basis of this technology have been licensed to major
consumer electronics companies, like Microsoft, for use on the XBOX.

2.8.2 Color Sensor

Reverse engineering has determined that the Kinect's various sensors output video at a frame
rate of ~9 Hz to 30 Hz depending on resolution. The default RGB video stream uses 8-bit
VGA resolution (640 × 480 pixels) with a Bayer color filter, but the hardware is capable of
resolutions up to 1280x1024 (at a lower frame rate) and other colour formats such as UYVY.
The monochrome depth sensing video stream is in VGA resolution (640 × 480 pixels)
with 11-bit depth, which provides 2,048 levels of sensitivity. The Kinect can also stream the
view from its IR camera directly (i.e.: before it has been converted into a depth map) as
640x480 video, or 1280x1024 at a lower frame rate. The Kinect sensor has a
practical ranging limit of 1.2–3.5 m (3.9–11.5 ft) distance when used with the Xbox software.
The area required to play Kinect is roughly 6 m2, although the sensor can maintain tracking
through an extended range of approximately 0.7–6 m (2.3–19.7 ft). The sensor has
an angular field of view of 57° horizontally and 43° vertically, while the motorized pivot is
capable of tilting the sensor up to 27° either up or down. The horizontal field of the Kinect
sensor at the minimum viewing distance of ~0.8 m (2.6 ft) is therefore ~87 cm (34 in), and
the vertical field is ~63 cm (25 in), resulting in a resolution of just over 1.3 mm (0.051 in)
per pixel. The microphone array features four microphone capsules and operates with each
channel processing 16-bit audio at a sampling rate of 16 kHz.
Because the Kinect sensor's motorized tilt mechanism requires more power than the Xbox
360's USB ports can supply, the device makes use of a proprietary connector combining USB
communication with additional power. Redesigned Xbox 360 S models include a special
AUX port for accommodating the connector, while older models require a special power
supply cable (included with the sensor ) that splits the connection into separate USB and
power connections; power is supplied from the mains by way of an AC adapter.

Chapter 3
GESTURE RECOGNITION
3.1 Introduction
Gesture recognition pertains to recognizing meaningful expressions of motion by a human,
involving the hands, arms, face, head, and/or body. It is of utmost importance in designing an
intelligent and efficient human-computer interface. The applications of gesture recognition
are manifold, ranging from sign language through medical rehabilitation to virtual reality. In
this paper, we provide a survey on gesture recognition with particular emphasis on hand
gestures and facial expressions. Applications involving hidden Markov models, particle
filtering and condensation, finite-state machines, optical flow, skin color, and connectionist
models are discussed in detail. Existing challenges and future research possibilities are also
highlighted.
Fig 3.1: gesture display
3.2 Problem Statement

Basic gesture recognition go through following problems:
 It requires external hardware like glows, accelerometer, color caps etc. to perform
gesture recognition.
 Most of the gesture recognition techniques fails in complex background.

 With more system interface the speed become slow and gesture reaches with delay,
which may result in robot collision.
3.3 Previous Work and Modification
 Earlier normal camera were used instead of kinect cameras .Earlier we used
accelerometer gloves but now we use bare hands.
 In the normal cameras we didn’t have good light conditions and it could sense any
random and minute objects also which was considered to be the gesture for the input
to be provided.
Fig 3.2 :Accelerometer gloves

3.4 Hand Gestures

HAND gesture recognition is of great importance for human-computer interaction (HCI),
because of its extensive applications in virtual reality, sign language recognition, and
computer games . Despite lots of previous work, traditional vision-based hand gesture
recognition methods are still far from satisfactory for real-life applications. Because of the
nature of optical sensing, the quality of the captured images is sensitive to lighting conditions
and cluttered backgrounds, thus optical sensor based methods are usually unable to detect and
track the hands robustly, which largely affects the performance of hand gesture recognition.
To enable a more robust hand gesture recognition, one effective way is to use other sensors to
capture the hand gesture and motion, e.g., through the data glove . Unlike optical sensors,
such sensors are usually more reliable and are not affected by lighting conditions or cluttered
backgrounds. However, as it requires the user to wear a data glove and sometimes requires
calibration, it is inconvenient to use and may hinder the natural articulation of hand gesture.
Also, such data gloves are usually more expensive than optical sensors, e.g., cameras. As a
result, it is not a very popular way for hand gesture recognition.
Thanks to the recent development of inexpensive depth cameras, e.g., the Kinect sensor , new
opportunities for hand gesture recognition emerge. Instead of wearing a data glove, using the
Kinect sensor can also detect and segment the hands robustly, thus it provides a valid base for
gesture recognition. In spite of many recent successes in applying the Kinect sensor to
articulated face recognition , human body tracking and human action recognition , it is still
an open problem to use Kinect for hand gesture recognition. Due to the low-resolution of the
Kinect depth map, typically, of only 640 X 480, although it works well to track a large object,
e.g., the human body, it is difficult to detect and segment a small object from an image with
this resolution, e.g., a human hand which occupies a very small portion of the image with
more complex articulations. In such a case, the segmentation of the hand is usually
inaccurate, thus may significantly affect the recognition step.

3.5 Images captured in Kinect
Fig 3.3: Tracking of gesture 1
Fig 3.4: Tracking of gesture 2

Fig 3.5: Tracking for gesture 3
Fig 3.6: Tracking for gesture 4

Chapter 4
ARTIFICIAL NEURAL NETWORK
4.1 Introduction
First wave of interest in neural networks (also known as connectionist models or parallel
Distributed processing emerged after the introduction of simplified neurons by McCulloch
and Pitts in 1943 (Mc- Culloch & Pitts 1943) .These neurons were presented as models of
biological neurons and as conceptual components for circuits that could perform
computational tasks. When Minsky and Papert published their book Perceptrons in 1969
(Minsky and Papert 1969) in which they showed the deficiencies of per- ceptron models most
neural network funding was redirected and researchers left the field.Only a few researchers
continued their efforts, most notably Teuvo Kohonen, Stephen Grossberg, James Anderson,
and Kunihiko Fukushima.The interest in neural networks re-emerged only after some
important theo- retical results were attained in the early eighties (most notably the discovery
of error back-propagation) and new hardware developments increased the processing
capacities. This renewed interest is reflected in the number of scientists, the amounts of
funding, the number of large conferences, and the number of journals associated with neural
networks, nowadays most universities have a neural networks group, within their psychology,
physics, computer science, or biology departments. Artificial neural networks can be most
adequately characterized as computational models with particular properties such as the
ability to adapt or learn, to generalize, or to cluster or organize data, and which operation is
based on parallel processing. However, many of the above-mentioned properties can be
attributed to existing (non-neural) models, the intriguing question is to which extent the
neural approach proves to be better suited for certain applications than existing models. To
date an equivocal answer to this question is not found. Often parallels with biological
systems are described. However, there is still so little known (even at the lowest cell level)
about biological systems, that the models we are using for our artificial neural systems seem
to introduce an oversimplification of the biological models.

Data mining is process of identify patterns and establish relationships Data Mining
defined as The nontrivial extraction of implicit, previously unknown, and potentially useful
information from data. Data mining is the process of analyzing large amount of data stored is a
data warhorse for useful information which makes use of artificial intelligence techniques
,neural network ,and advance statistical tools (such as cluster analysis) to reveal trends,
patterns and relationship, which otherwise may be undetected.
4.2 Neuron in brain
Although heterogeneous, at a low level the brain is composed of neurons.A neuron receives
input from other neurons (generally thousands) from its synapses.Inputs are approximately
summed
Figure 4.1: Neural Network
When the input exceeds a threshold the neuron sends an electrical spike that travels that travels
from the body, down the axon, to the next neuron(s)

 Brains learn
 Altering strength between neurons
 Creating/deleting connections
Hebbs Postulate (Hebbian Learning)
When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes
part in firing it, some growth process or metabolic change takes place in one or both cells such
that A’s efficiency, as one of the cells firing B, is increased.
 Long Term Potentiation (LTP)

 Cellular basis for learning and memory
LTP is the long-lasting strengthening of the connection between two nerve cells in response to
stimulation. Discovered in many regions of the cortex
A neuron (neuron or nerve cell) is an electrically excitable cell that processes and transmits
information through electrical and chemical signals. These signals between neurons occur via
synapses, specialized connections with other cells. Neurons can connect to each other to form
neural networks. Neurons are the core components of the nervous system, which includes
the brain, and spinal cord of the central nervous system (CNS), and the ganglia of the
peripheral nervous system (PNS). Specialized types of neurons include: sensory neurons
which respond to touch, sound, light and all other stimuli affecting the cells of the sensory
organs, that then send signals to the spinal cord and brain; motor neurons that receive signals
from the brain and spinal cord, to cause muscle contractions, and affect glandular outputs, and
interneurons which connect neurons to other neurons within the same region of the brain or
spinal cord, in neural networks. A typical neuron possesses a cell body (soma), dendrites,
and an axon. The term neurite is used to describe either a dendrite or an axon, particularly
in its undifferentiated stage. Dendrites are thin structures that arise from the cell body, often
extending for hundreds of micrometers and branching multiple times, giving rise to a

complex ”dendritic tree”. An axon is a special cellular extension that arises from the cell
body at a site called the axon hillock and travels for a distance, as far as 1 meter in humans
or even more in other species. The cell body of a neuron frequently gives
rise to multiple dendrites, but never to more than one axon, although the axon may branch
hundreds of times before it terminates. At the majority of synapses, signals are sent from
the axon of one neuron to a dendrite of another. There are, however, many exceptions to
these rules: neurons that lack dendrites, neurons that have no axon, synapses that connect
an axon to another axon or a dendrite to another dendrite, etc. All neurons are electrically
excitable, maintaining voltage gradients across their membranes by means of metabolically
driven ion pumps, which combine with ion channels embedded in the membrane to generate
intracellular-versus-extracellular concentration differences of ions such as sodium, potassium,
chloride, and calcium. Changes in the cross-membrane voltage can alter the function of
voltage-dependent ion channels. If the voltage changes by a large enough amount, an all-or-
none electrochemical pulse called inaction potential is generated, which travels rapidly along
the cell’s axon, and activates synaptic connections with other cells when it arrives. Neurons do
not undergo cell division. In most cases, neurons are generated by special types of stem cells.
A type of glial cell, called astrocytes (named for being somewhat star-shaped), have also been
observed to turn into neurons by virtue of the stem cell characteristic pluripotency. In humans,
neurogenesis largely ceases during adulthood but in two brain areas, the hippocampus and
olfactory bulb, there is strong evidence for generation of substantial numbers of new neurons.
Since the 1960’s, database and information technology has been evolving systematically from
primitive le processing systems to sophisticated and powerful databases systems. The
research and development in database systems since the 1970’s has led to the development of
relational database systems
data modeling tools, and indexing and data organization techniques. In addition, users gained
conve- nient and edible data access through query languages, query processing, and user
interfaces. E- Client- methods for on-line transaction processing (OLTP), where a query is
viewed as a read-only transaction, have contributed substantially to the evolution and wide

acceptance of relational technology as a major tool for e-client storage, retrieval, and
management of large amounts of data.
4.3 Characteristic of ANN
Basically Computers are good in calculations that basically takes inputs process then and after
that gives the result on the basis of calculations which are done at particular Algorithm which
are programmed in the softwares but ANN improve their own rules, the more decisions they
make, the better decisions may become.The Characteristics are basically those which should
be present in intelligent System like robots and other Artificial Intelligence Based
Applications. There are six characteristics of Artificial Neural Network which are basic and
important for this technology which are showed with the help of diagrams.
4.4 Activation Function
Activation Functions are basically the transfer function which is output from a artificial
neuron and it send signals to the other artificial neuron. There are four form of Activation
Functions Threshold, Piecewise Linear, Sigmoid and Gaussian all are different from each
other. In Below figures you can see the Activation function with its demonstration.
Fig 4.2: Characteristics of ANN

Figure 4.3: Gaussian function
Figure 4.4- Threshold function

Fig 4.5: Sigmoid function

Flowchart
Fig 4.6 :Flowchart of ANN working

Chapter 5
HARDWARE IMPLEMENTATION
5.1 Open Source Microcontroller

An embedded system is a computer system with a dedicated function within a larger
mechanical or electrical system, often with real-time computing constraints. It is embedded as
part of a complete device often including hardware and mechanical parts. To control our
system we will use a microcontroller, which is a small computer on a single integrated circuit
containing a processor core, memory, and programmable input/output peripherals.
Microcontrollers are designed for embedded applications, in contrast to the microprocessors
used in personal computers or other general purpose applications. Like all good things, this
powerful component is basically very simple. It is made by mixing tested and high- quality
"ingredients" (components) as per following receipt:
1. The simplest computer processor is used as the "brain" of the future system.
2. Depending on the taste of the manufacturer, a bit of memory, a few A/D converters, timers,
input/output lines etc. are added
3. All that is placed in some of the standard packages.
There are many platforms that can be used for embedded systems. Three models are the
Arduino, Raspberry Pi and BeagleBone. Those three are all readily available, affordable and
about the same size (just larger than 2 x 3). Left to right: (Arduino Uno, BeagleBone,
Raspberry Pi)
Fig 5.1 Arduino uc Type

The Arduino Uno is a staple for the maker community. Arduinos come in various
sizes and flavors, but we chose the Arduino Uno as an example of the prototypical Arduino
for comparison. It has an easy to use development environment, an avid user base and is
designed to be easy to interface all sorts of hardware to. The Raspberry Pi is the newcomer to
the game. It isn’t really an embedded computer. It is actually a very inexpensive full-on
desktop computer. The BeagleBone is the perhaps the least known of these platforms, but an
incredibly capable board worthy of consideration for many projects. It is a powerful Linux
computer that fits inside an Altoid’s mint container.
Table 2: comparison between microcontroller
Why use Arduino? We chose to use Arduino for the following reasons:
1. It is an open source project with umpteen, different kits, shield’s. It is a very accessible
platform with extended and customized possibilities for both software and hardware sides.
2. It has the largest community of users, the most tutorials and sample projects.

3. The Arduino makes it the easiest of any of the boards to interface to external hardware and
sensors. There are different versions of the board that operate at different voltages (3.3v vs
5v) to make it easier to connect to external devices.
4. It has a much simpler microcontroller-based approach making it more suitable for

applications which demand a very high degree of reliability.
5. It has a friendly programming environment, with easy programming using a C-like

programming language. It comes supplied with a friendly Arduino development environment
(IDE) and libraries which have been designed for ease of use and to support “non-
programmers”, and within minutes of installing these you can be blinking LEDs and reading
temperature sensors.
6. It has low power consumption (<0.5 W), capable of even running micro amps with very
low clock. It is made to be running 24 hrs a day, all year and is suited for permanent
installations. It will do it’s job day and night for years without requiring any updates. That
also makes it suitable for battery powered applications.
7. It is inexpensive, around 30 euro per board and comes with free authoring software
8. Has a cross-platform - The Arduino software runs on Windows, Macintosh OSX, and
Linux operating systems. Most microcontroller systems are limited to Windows.
Raspberry Pi and Beagle bone have their advantages such as including Ethernet interfaces
and USB, enabling connecting them to the network relatively painlessly, unlike the Arduino
which even with a shield that includes the ability to connect to Ethernet, has fairly limited
access to the networking functions. Yet for our system purpose we found that the arduino
advantages make it the most suitable.
5.1.1 Power
The Arduino Mega can be powered via the USB connection or with an external power
supply. The power source is selected automatically. External (non-USB) power can come
either from an AC-to-DC adapter (wall-wart) or battery. The adapter can be connected by
plugging a 2.1mm center-positive plug into the board's power jack. Leads from a battery can
be inserted in the Gnd and Vin pin headers of the POWER connector. The board can operate
on an external supply of 6 to 20 volts. If supplied with less than 7V, however, the 5V pin may
supply less than five volts and the board may be unstable. If using more than 12V, the voltage
regulator may overheat and damage the board. The recommended range is 7 to 12 volts. The
power pins are as follows:
 VIN. The input voltage to the Arduino board when it's using an external power source
(as opposed to 5 volts from the USB connection or other regulated power source).
You can supply voltage through this pin, or, if supplying voltage via the power jack,
access it through this pin.
 5V. This pin outputs a regulated 5V from the regulator on the board. The board can be
supplied with power either from the DC power jack (7 - 12V), the USB connector
(5V), or the VIN pin of the board (7-12V). Supplying voltage via the 5V or 3.3V pins
bypasses the regulator, and can damage your board. We don't advise it.
 3V3. A 3.3 volt supply generated by the on-board regulator. Maximum current draw
is 50 mA.
 GND. Ground pins.
 IOREF. This pin on the Arduino board provides the voltage reference with which the
microcontroller operates. A properly configured shield can read the IOREF pin
voltage and select the appropriate power source or enable voltage translators on the
outputs for working with the 5V or 3.3V.
5.1.2 Input and Output
Each of the 16 digital pins on the UNO can be used as an input or output, using
pinMode, a function which configures pin behavior. They operate at 5 volts. Each pin can
provide or receive a maximum of 40 mA and has an internal pull-up resistor (disconnected by

default) of 20-50 kOhms. The digitalWrite and digitalRead functions are used to send digital
output and receive digital input. Some of the digital pins have additional specialized
functions. The 8 analog input pins, each provide 10 bits of resolution (i.e. 1024 different
values). By default they measure from ground to 5 volts, though is it possible to change the
upper end of their range using the AREF pin and analogReference function. Receiving analog
input is done using the analogRead function.
Arduino analog pins are only used to receive analog input; there are no pins that give out true
analog output. The Arduino analogWrite rather puts out a signal that pulses back and forth
between 0 and 5 volts rapidly. The percentage of the time it is ON vs Off allows it to simulate
the effect of analog signals in some applications.

5.1.3 Duty Cycle

When the signal is high, we call this on time. To describe the amount of on time we use the
concept of duty cycle. Duty cycle is measured in percentage. The percentage duty cycle
specifically describes the percentage of time a digital signal is on over an interval or period of
time. This period is the inverse of the frequency of the waveform.
If a digital signal spends half of the time on and the other half off, we would say the digital
signal has a duty cycle of 50% and resembles an ideal square.
Fig 5.2 PWM duty cycle
wave. If the percentage is higher than 50%, the digital signal spends more time in the high
state than the low state and vice versa if the duty cycle is less than 50%. At the right is a
graph that illustrates these three scenarios.

We will need to use PWM for the following controls:
Variation of LED brightness: You can control the brightness of an LED by adjusting the duty
cycle. The frequency of the square wave does need to be sufficiently high enough when
controlling LEDs to get the proper dimming effect. A 20% duty cycle wave at 1 Hz will be
obvious that it’s turning on and off to your eyes meanwhile, 20% duty cycle at 100 Hz or
above will just look dimmer than fully on. Essentially, the period can’t be too large if you’re
aiming for a dimming effect with the LEDs
5.1.4 Getting started
Get the latest version of the Arduino environment from the download page. When the
download finishes, unzip the downloaded file. Make sure to preserve the folder structure.
Double-click the folder to open it. There should be a few files and sub-folders inside.
 Plug in your board and wait for Windows to begin it's driver installation process.
After a few moments, the process will fail, despite its best efforts
 Click on the Start Menu, and open up the Control Panel.
 While in the Control Panel, navigate to System and Security. Next, click on System.
Once the System window is up, open the Device Manager.
 Look under Ports (COM & LPT). You should see an open port named "Arduino UNO
(COMxx)". If there is no COM & LPT section, look under "Other Devices" for
"Unknown Device".
 Right click on the "Arduino UNO (COmxx)" port and choose the "Update Driver
Software" option.
 Next, choose the "Browse my computer for Driver software" option.

 Finally, navigate to and select the driver file named "arduino.inf", located in the
"Drivers" folder of the Arduino Software download (not the "FTDI USB Drivers"
sub-directory). If you are using an old version of the IDE (1.0.3 or older), choose the
Uno driver file named "Arduino UNO.inf"
 Windows will finish up the driver installation from there.
 Double-click the Arduino application.
 Make sure to select the entry in the Tools > Board menu that corresponds to your
Arduino ( Arduino UNO)

 Select the serial device of the Arduino board from the Tools | Serial Port menu. This
is likely to be COM3 or higher (COM1 and COM2 are usually reserved for hardware
serial ports). To find out, you can disconnect your Arduino board and re-open the
menu; the entry that disappears should be the Arduino board. Reconnect the board and
select that serial port.
 Now, simply click the "Upload" button in the environment to upload the
opened/written program to the Arduino board. Wait a few seconds - you should see
the RX and TX leds on the board flashing. If the upload is successful, the message
"Done uploading." will appear in the status bar.
5.1.5 General code structure
 Libraries and global variables Here we declare all libraries used as well as any
global variables.
 setup() Here we layout the initial conditions for the program so that the Arduino
knows what kind of work it’s going to be doing. In this part we usually initialize
variables and set pin modes using ‚pinMode()‛ instruction. The setup function will
only run once, after each powerup or reset of the Arduino board
 loop() This function does precisely what its name suggests; it loops consecutively,
allowing your program to change and respond. It is used to actively control the
Arduino board.
5.2 Relay Switches

A relay is a device that uses an electromagnet to mechanically pull two connections
together to complete a circuit, in the exact same way your finger mechanically pushes two
contacts together in a toggle switch. They are used as remote control switches. The relay has
two inputs (control circuit), one for the positive of our input signal and the other for the
negative. When a signal passes through the input pins of the relay, current passes through the
electromagnet coil which pulls the metal connector towards the magnet.

Fig 5.3 Relay switch
A typical relay generally has 5 pins on it (SPDT - single pole double throw switch) These are
referred to as, Input 1, Input 2, COM, N/O, N/C Load circuit pins: COM: is short for common
ground, in most cases you connect the negative of the power you are switching to this. If your
application uses the same power source, you can connect Input 2 and COM together. COM is
the middle terminal, and is connected to the pin that the contact gets pulled towards.
N/C: Short for normally closed, this terminal is not connected to COM when there is no
signal, but when there is a signal, the pin inside the relay is pulled down until it touches N/C,
which would connect it to COM.
N/O: short for normally open, uses a small spring to make it always touching COM when the
magnet is off. N/O is used in most cases to turn on the standby light, as it switches on when
there is no power.

Fig 5.4 5 Point relay
5.2.1 Benefits:
Relays can work with AC and DC, while transistors are DC only. - Relays have zero
closed resistance, while transistors have a forward voltage drop that wastes a little to lots of
power. - Relays have infinite open resistance, while transistors have a leakage that can affect
attached electronics. - Relays offer complete isolation between the activating circuit and the
load.
We chose to use an available multiple relay switch board ( 5 pin relays – ULN
2803AG IC ) in this project since the accompanied delay in switching is not of essence, we
don’t need to vary switched current and relay has its previously mentioned benefits. The
board requires 12V to operate, but features low current draw, and has 4 pins per relay switch
to control it, one for control, common, N/C (connected when switch was on) and N/O
(connected when switch is off).
To use a 5V input from arduino to activate a single 5 pin relay, we can use a transistor
to add gain to the 5V such that it will be enough to activate the switch. We will also use a
clamp diode to protect the relay from inverse current.

Chapter 6
PROGRAM CODES
6.1 Main Hardware

try
fclose(Sr1);
end
clc;
clear all;
close all;
%%
Sr1 = serial('COM3','Baudrate',9600);
fopen(Sr1);
fwrite(Sr1,'A');
% Delte webcam object if it is already accessing the camera
delete(imaqfind)
load ANN
% Create the objects for the color and depth sensors.
% Device 1 is the color sensor and Device 2 is the depth sensor.
vid = videoinput('kinect',1);
vid2 = videoinput('kinect',2);
% Get the source properties for the depth device.

srcDepth = getselectedsource(vid2);
% set tracking mode to skeleton

set(srcDepth,'TrackingMode','Skeleton')
% Set the frames per trigger for both devices to 1.

vid.FramesPerTrigger = 1;
vid2.FramesPerTrigger = 1;
% Set the trigger repeat for both devices to 200, in

% order to acquire 201 frames from both the color sensor and the depth sensor.
vid.TriggerRepeat = 5000;
vid2.TriggerRepeat = 5000;
% Configure the camera for manual triggering for both sensors.

triggerconfig([vid vid2],'manual');
% Start both video objects.

start([vid vid2]);
% Enable flag
Eflag = 0;
Ecounter = 0;

% Status
status = zeros(1,4);
DC = zeros(1,4);
count2 = 0;
beep on
% Trigger the devices, then get the acquired data.% Trigger 200 times to get the frames.
for i = 1:5000
% Trigger both objects.

trigger([vid vid2])
% Get the acquired frames and metadata.

[imgColor] = getdata(vid);
[imgDepth, ts_depth, metaData_Depth] = getdata(vid2);
% Find which ID is tracked

ix = metaData_Depth.IsSkeletonTracked; % Find nonzero terms
% Get x y cordinates according to pixels

Co = metaData_Depth.JointImageIndices;
% Get x y and z cordinates (normalised)

Co2 = metaData_Depth.JointWorldCoordinates;
% Get tracked skeleton points

S = Co(:,:,ix);
S2 = Co2(:,:,ix);
% Identify the person who nearer

if sum(ix)>1
% find out minimum z value for shoulder center

[v, pix] = min(S2(3,3,:),[],3);
dix = find(ix);
pix = dix(pix);
% Take the cordinates which are nearer
S = Co(:,:,pix);
S2 = Co2(:,:,pix);
end
subplot(221)
imshow(imgColor)
title('Color image')
skeletonViewer(S,1)

% GEt y value of spine

spiney = S2(1,2);
fally = heady-spiney;
if fally<0.3
count2 = count2+1;
if count2>12
disp('Fall detected')
beep
pause(0.5)
end
else
coutn2 = 0 ;
end
end
if sum(ix)>0
% Get right hand x,y value
Rx = S2(12,1,1);
Ry = S2(12,2,1);
Rz = S2(12,3,1);
% Get left hand y value

Lx = S2(8,1,1);
Ly = S2(8,2,1);
Lz = S2(8,3,1);
% Get hip center y value

Hy = S2(1,2,1);
% Calculate y distance
dy1 = Ry-Hy;
dy2 = Ly-Hy;
% get shoulder center cordinates

Sx = S2(3,1,1);
Sy = S2(3,2,1);
Sz = S2(3,3,1);
% Find maximum distance

if max([dy1,dy2])>0
if dy1>dy2 % Right hand up then left hand

xim = S(12,1,1);
yim = S(12,2,1);
else % Left hand is up then right hand
x1 = Lx;
y1= Ly;
z1 = Lz;
xim = S(8,1,1);
yim = S(8,2,1);
end
% Calculate distance with shoulder center

Dsc = sqrt((Sx-x1)^2+(Sy-y1)^2+(Sz-z1)^2);
if Dsc>0.3
% Calculate z distance from shoulder center

dz = Sz-z1;
if dz>0.4
Ecounter = Ecounter + 1;
if Ecounter== Fth
Eflag = ~Eflag;
if Eflag==0
disp('Disabled');
else
disp('Enabled');
end
end
else
Ecounter = 0;
end
if Eflag == 0
continue;
end
% highlight with circle
subplot(221)
hold on
plot(xim,yim,'yo','markersize',10,'markerfacecolor','c')

op = round(op);
subplot(221)
if op==1
n = 1;
DC(n) = DC(n)+1;
ixx = zeros(1,length(DC));
ixx(n) = 1;
DC = DC.*ixx;
if DC(n)== cth
status(n) = ~status(n);
if status(n)==1
fwrite(Sr1,'A');
disp('Light 1 ON');
text(15,20,'Light 1 ON','fontsize',12,'color','r')
else
fwrite(Sr1,'B');
disp('Light 1 OFF');
text(15,20,'Light 1 OFF','fontsize',12,'color','r','fontweight','bold')
end
end
elseif op ==2
n = 2;
DC(n) = DC(n)+1;
ixx(n) = 1;
DC = DC.*ixx;
if DC(n)== cth
if status(n)==1
fwrite(Sr1,'C');
disp('Light 2 ON');
else
fwrite(Sr1,'D');
disp('Light 2 OFF');
end
end
elseif op==3
n = 3;
DC(n) = DC(n)+1;
ixx(n) = 1;
DC = DC.*ixx;

end
end
elseif op==4
n = 4;
DC(n) = DC(n)+1;
ixx(n) = 1;
DC = DC.*ixx;
if DC(n)== cth
if status(n)==1
fwrite(Sr1,'E');
disp('Charger ON');
else
fwrite(Sr1,'F');
disp('Charger OFF');
end
end
end
subplot(2,2,[3 4])
bar(F);
title('Features')
end
else
Ecounter=0;
end
end
end
fclose(S);
clear S

6.2 ANN Training Code
clc;
clear all;
close all;
%% Training
load Features;
% Initialise fitnet with 20 neurons.

net = fitnet(20);
% Train neural network

net= train(net,Features,Labels);
save ANN net
%% Testing
op = sim(net,Features);
op = round(op);
sum(op~=Labels)
%Show weights
a = net.IW;
weights = a{1,1}

Chapter 7
ADVANTAGES AND DISADVANTAGES
7.1 Advantages
 It is easy to implement.
 It can help elder people live an independent life.
 It can be used to alert someone in case of an emergency like a person falls.
 It is very user friendly.
 No wearable sensors are required to be worn by the person.
 It can be used in home as well as office.
 It reduces the dependency on others.
7.2 Disadvantages
 The kinect camera is very costly.
 The system is less compact.
 The person cannot be too far or too near the kinect sensor.
 It is still in its development phase.

Chapter 8
APPLICATION AND CONCLUSION
8.1 Applications
 It is used in homes or hospitals in monitoring of elderly person.

 It can be used in offices and home automation systems.
 It can be used in security systems in case of emergency.
 It can be used in gaming implementation.
Fig 8.1 Gesture used in gaming implementation
Page 60
Dept. of Telecommunication 2015-16
8.2 Conclusion
A variety of smart-home technologies are available that are readily accepted by older
adults and their family members, healthcare professionals and carriers. The feasibility and
utilization of smart-technologies can be improved by addressing issues related to safety
and privacy. In addition, exploring how feelings of safety and more control over one’s life
contributes to social and emotional well-being as well as the capacity to continue
participating in outside interests and activities. While the outcomes and cost effectiveness
of these forms of technology remains to be assessed, they appear to show some
potential for helping older adults to live longer, safely and independently in their own
homes.
Page 61
REFERENCES
1. Organisation for Economic Co-operation and Development (2001) World Population

Ageing, Paris.
2. Behr R, Sciegaj M, Walters R, Bertoty J, Dungan R (2011) Addressing the housing
challenges of an aging population: Initiatives by Blueroof Technologies in McKeesport,
Pennsylvania. J Archit Eng 17: 162-169.
3. Ackerman MJ (2009) The smart home. J Med Pract Manage 25: 68-69.
4. Morris ME (2012) Preventing falls in older people. BMJ 345: e4919.
5. Dementia across Australia 2011-2050 (2011) Deloitte Access Economics for
Alzheimer’s Australia.
6. Caring for Older Australians (2011) Australia Productivity Commission Inquiry Report.
Melbourne.
7. Agoulmine N, Deen MJ, Jeong-Soo L, Meyyappan M (2011) U-health smart home:
Innovative solutions for the management of the elderly and chronic diseases. IEEE
Nanotechnology Magazine 5: 6-11.
8. Morris M, Ozanne E, Miller K, Santamaria N, Pearce A, et al. (2012) Smart
technologies for older people: A systematic literature review of smart technologies that
promote health and wellbeing of older people living at home. IBES, The University of
Melbourne, Australia.
9. McLean A (2011) Ethical frontiers of ICT and older users: cultural, pragmatic and
ethical issues. Journal of Ethics and Information Technology 13: 313-326.
10. Menschner P, Prinz A, Koene P, Kobler F, Altmann M, et al. (2011) Reaching into
patients’ homes-Participatory designed AAL services-The case of a patient- centered
nutrition tracking service. Electronic Markets 63-76.
Page 62

Dev Project Report

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Dev Project Report

Transféré par

Droits d'auteur :

Formats disponibles

Smart Home Technology for elderly using ANN

Dept. of Telecommunication 2015-16 Page 1

1.2 Ageing Population: Statistics and Growth Rates 08

1.3 Increasing Life Expectancy 10

1.4 Socio-Economic and Health related Determinants 11

1.5 Elderly Population in India 12

1.6 Need for Early Detection of Ageing Changes 13

1.7 Our Approach and Solution 14

CHAPTER 2: KINECT WORKING AND PRINCIPLE

2.2 Kinect camera 18

2.3 Kinect for Windows 19

2.4 Kinect Versions 20

2.6 Kinect for XBOX ONE 22

Dept. of Telecommunication 2015-16 Page 2

2.8 Parts of Kinect Camera 25

2.8.1 Depth Sensor 26

2.8.2 Color Sensor 27

CHAPTER 3: GESTURE RECOGNITION

3.2 Problem Statement 29

3.3 Previous Work and Modification 30

3.4 Hand Gestures 31

3.5 Images Captured in Kinect 33

CHAPTER 4: ARTIFICIAL NEURAL NETWORK

4.2 Neuron in Brain 37

4.3 Characteristics of ANN 37

4.4 Activation Function 40

CHAPTER 5: HARDWARE IMPLEMENTATION

Dept. of Telecommunication 2015-16 Page 3

5.1.2 Input and Output 44

5.1.3 Duty Cycle 46

5.1.4 Getting Started 47

5.1.5 General Code Structure 49

5.2 Relay Switches 51

CHAPTER 6: PROGRAM CODES

6.1 Main Hardware Program 52

6.2 ANN Training Program 58

CHAPTER 7: ADVANTAGES AND DISADVANTAGES

CHAPTER 8: APPLICATIONS AND CONCLUSIONS

Dept. of Telecommunication 2015-16 Page 4

1.1 Countries, Population Share by Age Group 10

1.2 Life Expectancy around the World 11

2.1 XBOX with Television Set 24

2.2 Kinect Camera 25

3.1 Gesture Display 28

3.2 Accelerometer Gloves 29

3.3 Tracking of Gesture 1 31

3.4 Tracking of Gesture 2 31

3.5 Tracking of Gesture 3 32

3.6 Tracking of Gesture 4 32

4.1 Neural Network 34

Dept. of Telecommunication 2015-16 Page 5

4.2 Characteristics of ANN 37

4.3 Gaussian Function 38

4.4 Threshold Function 38

4.5 Sigmoid Function 39

4.6 Flowchart of ANN Working 40

5.1 Arduino uc type 41

5.2 PWM Duty Cycle 46

5.3 Relay Switches 50

5.4 5-point Relay 51

8.1 Gestures used in Gaming Implementation 60

Dept. of Telecommunication 2015-16 Page 6

1. Percentages of Older Ages by Regions 09