An Innovative Communication System For Deaf Dumb and Blind People

An Innovative Communication System for Dumb, Deaf and Blind People 2016-17
CHAPTER 1
INTRODUCTION
Nowadays, embedded system is emerging as an important trend in all applications. More
recently developed embedded applications are changing our lifestyle in a smart way. Sign
language is an expressive and natural way for communication between normal and dumb
people. The intension of the sign language translation system is to translate the normal sign
language into speech and to make easy contact with the dumb people. In order to improve the
life style of the dumb people, the proposed system is developed. Sign. The physical gesture
communication consist of hand gestures that convey respective meaning, the nonphysical is
head movement, facial appearance, body orientation and position. Sign language is not a
universal language and it is different from country to country. Research in the sign language
system has two well-known approaches. First the image processing technique using the
camera to capture the image/video and second research approach is a sign language
recognition system using a data glove. The user need to wear glove consist of flex sensor,
accelerometer and motion tracker. These Sensor output signals are fed to the computer for
further process to recognize the hand gesture and to produce speech/text. The main
characteristics of human hand gestures can be summarized by their dynamic, multi-attribute
property. To utilize hand gestures as a way of interaction, it is necessary to analyse the
motion patterns for each of the gesture attributes and finally to extract the whole
interpretation by integrating the relevant factors across time Gesture based applications
widely range from replacing the traditional mouse as a position device to virtual reality and
communication with the deaf. A complete vision-based system consisting of hand gesture
acquisition, segmentation, filtering, representation, and classification is developed to robustly
classify hand gestures.
Approximately 285 million people are judged to be visually impaired worldwide in which 39
million are blind and 246 are said have low vision. Approximately 90% of this world’s
visually impaired is from the dispirited income people and 82% of people living with
blindness aging persons and above. The numbers of people visually impaired from eye
related diseases have been brought down in the past 20 years according to global estimated
work. In which 80% of all visual restitution can be prevented or cured. India is considered to
be the home for the world’s largest act of blind people. In lodge to facilitate these people we
have developed the assistive device for blind people who does not want the assistance of
other neighbours.
Dept. of ECE, SJBIT Page 1

In all around the world about 9.1 billion people are deaf and mute. In their daily life they
face plenty of problems on their communication. Sign language is a linguistic process which
is employed for communication among the normal people and handicapped people. Sign
language relies on sign patterns such as body language of the person and movements of the
arm to facilitate the discernment between the great unwashed. The deaf and vocally
impaired people don't simply have to learn the customized sign language, but the core issue
is that they can communicate with the usual sort of multitude in the society. It is similarly
not possible for all the masses to learn the sign language to understand whatever is said
through gestures. Therefore, the communication gaps still exist between the deaf and dumb
people. Dumber people can simply tilt the message by sign language which could not be
understandable by other people.
In resolving these difficulties with visually and vocally impaired people we have used the
tiny credit card sized computer named raspberry pi. By this device we provide the solution
for blind, deaf and dumb people e. For blind people the image is converted to voice by using
Tesseract software, the deaf people received their content by message as soon as the
opposite person speaks out it displayed as a message. The dumb persons conveyed their
message through text instead of sign language which is delivered via espeak.
They can, however write and communicate easily. But it is not feasible. They suffer a lot in
face-to-face communication. Generally, these individuals make use of sign language for
interaction; but they discover it gesture is of our concern as they are very popular among
these individuals and they find it easy to interact using these gestures but hard to deal with
the one who do not understand this language. Hence, there is a need of a proper interpreter.
The number of training institute of sign language in India is less so, we require a framework
which can act as an instructor or as well. The sign language is difficult to interpret by the
individuals who are not well-aware of it. Gesture is a non-verbal communication, which
includes the movements of head, hand and other body parts. Here hand gesture is of our
concern as they are very popular among these individuals and they find it easy to interact
using these gestures despite of large number of dumb and deaf peoples, very less research is
done in order to bridge the communication barrier. In an attempt to bridge the
communication barrier, we propose a system which helps normal and deaf and dumb people
to effectively communicate with each other.
Sign language is widely used by people who cannot speak and hear or people who can hear
but cannot speak. These gestures are used by the deaf people to express their thoughts. But
the use of these gestures are always limited in the deaf-dumb community, normal people

never try to learn the sign language. This causes a big gap in communication between the
deaf–dumb people and the normal people. Usually deaf people seek the help of sign
language interpreters for translating their thoughts to normal people and vice versa. But
this system is very costly and does not work throughout the life period of a deaf person. So as
a system that automatically recognizes the sign language gestures is necessary. Such a system
can minimize the gap between deaf people and normal people in the society.
There are various sign languages across the world. The sign language used at a particular
place depends on the culture and spoken language at that place. Indian sign language
(ISL) is used by the deaf community in India. It consists of both word level gestures and
finger spelling. Finger spelling is used to form words with letter by letter coding. Letter
by letter signing can be used to express words for which no signs exist, the words for which
the signer does not know the gestures or to emphasis or clarify a particular word. So
the recognition of finger spelling has key importance in sign language recognition. The
finger spelling in Indian sign language consists of both static as well as dynamic
gestures which are formed by the two hands with arbitrarily complicated shapes. This
paper presents a method for the automatic recognition of static gestures in Indian sign
language alphabet and numerals. The signs considered for recognition include 26 letters of
the English alphabet and the numerals from 0-9.Numeral sign indication is shown in
Figure.1.1.
Figure 1.1 ASL numbers (1-10)
It is commonly developed in deaf and dumb communities which include deaf people, their
parents and friends. Sign language helps the deaf and dumb people to communicate with each
other. In the 1600’s, sign language was recognized as an accepted language, a well-known
language. People usually used their hands and arms to convey their messages to others. In
1620 Juan Pablo de bonet published first book that helps the deaf and dumb people to learn

sign language. In 1750 first school for Deaf and dumb people was opened by Abede I Epee in
France. Before 1600’s it was very difficult for deaf and dumb people to communicate. Deaf
people classified as “mentally defective” due to lack of their education and also lack of the
development of sign language. During 1600’s, people felt that there is only one thing that
separate the human beings and animals that is trait of “speech”. So due to this fact people
who are deaf also called as “animals” because they are not able to speak. At that time deaf
people had no faith on hearing people, they had faith only on themselves. In families where
there was one or more than one person who is deaf they started to find a way to communicate
with each other using their hands and arms. This informal form of communication passes
many years and now accepted as a sign language. Sign language used in India is called Indian
sign language. Sign language varies from country to country like spoken language. Sign
Language used in America is called American Sign Language. Sign language used in German
is called German sign language. Sign language does not have any written form. There must
be some notation system that helps to write sign language. In 1984, such system was
developed named as HamNoSys (Hamburg Notation System) at the University of Hamburg.
It is stoke based notation system used to write signs HamNoSys Notations are same
throughout the world. HamNoSys helps to generate SiGML that further helps to generate
animation corresponding to words. HamNoSys have a particular form to write signs. In
HamNoSys, signs are elaborated in forms of signing parameters. Signing parameters contains
hand shapes, hand location, hand orientation and hand movement. Representation of ISL
alphabets is shown in figure 1.2.
Figure 1.2 Representation of ISL Alphabets

This paper is going to concentrate on the above mentioned fact and tries to develop a new
instrument which can help differently abled people (Blind, Deaf and Dumb) to communicate
easily in the living world with other normal persons or the persons of their own kind. The
main aim of this paper is to bridge the gap in communication and bring forward some
technology that can help out the people who are suffering from Blindness or Deafness or
Dumbness or any combination of these three.
1.1 EXISTING SYSTEM

In the earlier days the blind people are catered with the basic learning’s of the Braille system.
Braille is a scheme of raising symbols which the great unwashed who are blind or partially-
sighted have been employed worldwide for over 150 years. The language in the Braille will
go from left to right across the page, just like printed words. The symbols which represent
each letter are prepared up of between one and six dots based on the figure of six dots which
we would pick up on a dice or a domino. Later in the evolution of the Braille system, there
exists a system of screen reader which is a computer program that enables the blind masses to
interpret what is shown on the screen through speech. Braille Scheme representation is shown
in Figure 1.3.
Figure 1.3 Braille Scheme
Using the keyboard, he/she moves using the arrow keys and a screen reader interprets what is
displayed on the cover. In the presence of screen reader there also exists another reader which
makes the blind masses to read text easier which is known as finger reader. Finger Reader
Diagram is shown in Figure 1.4.

Figure 1.4 Finger Reader
The finger reader is a device that helps visually impaired users with understanding texts and
language. It is essentially a ring type model usually wears on their index finger, which houses
a tiny camera and some haptic actuators for feedback. People with vision impairment who set
up 2.8% of the population, which naturally rely on tactile feeling through their fingers to
learn Braille and gather data about the surroundings. On applying the finger –wearable device
that is ready to use and gives them real time feedback which would fix their lives more
comfortable. With the elongation of these gimmicks, there exists a Bar code scanner for blind
people. Finger view is shown in Figure 1.5.
Figure 1.5 Finger View
The Bar code consists of bars and spaces that differ in width. The bars and spaces of a bar
code keep in touch to numbers and letters that epitomize the evocative data. Human existence
is a social being that can interact with other people to shares to their estimates, judgments,
views and information in the form of verbal and non-verbal message. But communication for
a soul is difficult who cannot hear is visual, but not auditory. In order to communicate with
the masses, the dense and deaf people in earlier days have used the sign language to
communicate. Bar Code Reader view is shown in Figure 1.6.

Figure 1.6 Bar Code
The sign language has developed through smart devices like sign language translator with the
help of gloves which is processed by gesture recognition. This is the basic form of all deaf
and dumb people who can convey their information through sign languages with the help of
different scientific devices.
Figure 1.7 Sign Language Representation
In the above said existing system has certain disadvantages which are as listed beneath In
Braille system, it can induce harm to the surface which would be like scribbling with
permanent marker over visual writing, and any injury to the finger would be like
experiencing an eye patch on to a seeing person. So, Braille is slightly, more susceptible to
problems preventing any reading.
 In Screen reader visually impaired people only take heed to a screen reader reading
the text displayed on the cover, and they don’t hold the opportunity to recognize the
correct spelling of certain words like medical terms and other scientific terms.
 In Finger reader the chief defect is it can only access to only English languages and
could not able to make out.
 In Bar code reader the main causal agents are that it could be that they are easily
broken, if so void and also it is sometimes hard for the scanner to recognize the
words.

CHAPTER 2
LITERATURE SURVEY
[1]. Title: “Full Duplex Communication System for Deaf & Dumb People"
Conference: International Journal of Emerging Technology and Advanced Engineering
(IJETAE)
Year: 2015
Author: Shraddha R. Ghorpade, Surendra K. Waghamare.
One of the important problems that our society faces is that people with disabilities are
finding it hard to cope up with the fast growing technology. The access to communication
technologies has become essential for the handicapped people. Generally deaf and dumb
people use sign language for communication but they find difficulty in communicating with
others who don’t understand sign language. Sign language is an expressive and natural way
for communication between normal and dumb people (information majorly conveyed through
the hand gesture). So, we need a translator to understand what they speak and communicate
with us. The sign language translation system translates the normal sign language to speech
and hence makes the communication between normal person and dumb people easier. But the
question arises, how the deaf person understands the speech of a normal person and hence we
need a system which converts the speech of normal person to text and the corresponding
gesture is displayed on display. So, the whole idea is to build a device that enables two way
communications between deaf-mute person and a normal person.
[2]. Title: “Sign Language Recognition System”

Conference: International Journal of Computational Engineering Research (IJCER)
Year: 2013
Author: Er. Aditi Kalsh, Dr. N.S. Garewal
Communication is the process of exchanging information, views and expressions between

two or more persons, in both verbal and non-verbal manner. Hand gestures are the nonverbal
method of communication used along with verbal communication. A more organized form of
hand gesture communication is known as sign language. In this language each alphabet of the
English vocabulary is assigned a sign. The physically disabled person like the deaf and the
dumb uses this language to communicate with each other. The idea of this project is to design
a system that can understand the sign language accurately so that the less fortunate people
may communicate with the outside world without the need of an interpreter. By keeping in

mind the fact that in normal cases every human being has the same hand shape with four
fingers and one thumb, this project aims at designing a real time system for the recognition of
some meaningful shapes made using hands.
[3]. Title: “Sign Pro-An Application Suite for Deaf and Dumb"
Conference: IJCSET
Year: 2012
Author: Ashish Sethi, Hemanth S,Kuldeep Kumar,Bhaskara Rao N,Krishnan R
This application helps the deaf and dumb person to communicate with the rest of the world
using sign language. Suitable existing methods are integrated in this application. The key
feature in this system is the real time gesture to text conversion. The processing steps include:
gesture extraction, gesture matching and conversion to speech. Gesture extraction involves
use of various image processing techniques such as histogram matching, bounding box
computation, skin colour segmentation and region growing. Techniques applicable for
Gesture matching include feature point matching and correlation based matching. We have
come up four different approaches based on the methods used for gesture extraction and
matching. A Comparative study of these approaches is also carried out to rank them based on
time efficiency and accuracy. The other features in the application include voicing out of text
and text to gesture conversion.
[4]. Title: “Design of Communication Interpreter for Deaf and Dumb Person"
Conference: International Journal of Science and Research (IJSR)
Year: 2015
Author: Pallavi Verma, Shimi S. L., Richa Priyadarshani.
We describe gesture based device for deaf and dumb person as communication for a person,
who cannot hear is visual, not auditory. Generally dumb people use sign language for
communication, but they find difficulty in communicating with others who don’t understand
sign language. So there is a barrier in communication between these two communities. This
work aims to lower this barrier in communication. The main aim of the proposed project is to
develop a cost effective system which can give voice to voiceless people with the help of
Smart Gloves. With the proposed work sign language is converted into text and speech using
flex sensor and microcontroller. It means that using smart gloves communication will not be
a barrier between two different communities.

[5]. Title: “AAWAAZ: A Communication System for Deaf and Dumb"
Conference: ICRITO
Year: 2016
Author: Anchal Sood, ,Anju Mishra
The paper proposes a framework for recognizing hand gesture which would serve not only as
way of communication between deaf and dumb and mute people, but also, as an instructor.
Deaf and dumb individuals lack in proper communication with normal people and find it
difficult to properly express themselves. Thus, they are subjected to face many issues in this
regard. The sign language is very popular among them and they use it to express themselves.
Thus, there is a need of a proper translator. The deaf and dumb are not idle as past, they are
working outside and doing great at it. So an efficient system must be set up, to interact with
them, to know their views and ideas. The framework here, act as a communication system for
deaf and dumb individuals. It would take the sign language as an input which would display
the result not only in the form of text but also in the form of audio. Similarly, if there is any
input in the form of text, it would display the corresponding image.
[6]. Title: “Two Way Communicator between Deaf and Dumb People and Normal People"
Conference: International Conference on Computing Communication Control and
Automation
Year: 2015
Author: Prashant G. Ahire, Kshitija B. Tilekar, Tejaswini A. Jawake, Pramod B. Warale
One of the most precious gift of nature to human beings is the ability to express himself by
responding to the events occurring in his surroundings. Every normal human being sees,
listens and then reacts to the situations by speaking himself out. But there are some
unfortunate ones who are deprived of this valuable gift. This creates a gap between the
normal human beings and the deprived ones. This application will help for both of them to
communicate with each other. The system is mainly consists of two modules, first module is
drawing out Indian Sign Language(ISL) gestures from real-time video and mapping it with
human-understandable speech. Accordingly, second module will take natural language as
input and map it with equivalent Indian Sign Language animated gestures. Processing from
video to speech will include frame formation from videos, finding region of interest (ROI)
and mapping of images with language knowledge base using Correlational based approach
then relevant audio generation using Google Text-to-Speech (TTS) API. The other way
round, natural language is mapped with equivalent Indian Sign Language gestures by

conversion of speech to text using Google Speech-to-Text (STT) API, further mapping the
text to relevant animated gestures from the database.
2.1 MOTIVATION
According to a survey conducted by disabled welfare department, about eight lakh people in
India are deaf and dumb. Despite of this large number, very less research is done in order to
bridge the communication barrier. The issues confronted by the deaf and dumb individuals in
India can be comprehensively grouped into classes like social connection, communication,
behavioural issues, psychological well-being, and safety concerns. Their issues have been
intensified by the absence of a legitimate gesture based communication translator in India.
They can however write and communicate easily. But it is not feasible. And they suffer a lot
in face to face communication. Generally, these individuals make use of sign language for
interaction; but they discover it gesture is of our concern as they are very popular among
these individuals and they find it easy to interact using these gestures hard to deal with the
ones who do not understand this language. Hence, there is a need of a proper interpreter. The
number of educators trained in Indian Sign Language is less so we require a framework
which can act as an instructor as well. If these individuals are not trained at an early age i.e.
between 3 to 5 years, their social survival gets to be troublesome.
About nine million people in the world are considered to be a deaf or dumb or both.
Communications between deaf-dumb and a normal person have always been a challenging
task. Generally dumb people use sign language for communication but they find difficulty in
communicating with others who don’t understand sign language. This project aims to lower
this barrier in communication by developing a portable electronic device that act as a smart
assistant. The device can act as an artificial ear and tongue. Through this device the disabled
persons can even make a phone call communication, which is defined as an impossible thing
for them. Thus our primary goal in this project is to provide a standard lifestyle for deaf and
dumb peoples as normal ones.
2.2 OBJECTIVE
The objective is to develop a compact system which can be used by deaf, dumb and blind
person. The main objective of system is to bridge a communication gap between the
physically disabled person and the common person. This system will help the deaf and the
dumb in a large scale. The sign language is difficult to interpret by the individuals who are
not well-aware of it. Here, hand gesture is of our concern as they are very popular among
these individuals and they find it easy to interact using these gestures. The Hand Gesture

Recognition is referred to as the process of tracking human gestures and then translating them
in the form of some meaningful commands. Objectives of the Project:
 The visually impaired people can able to understand the words easily by Tesseract
OCR software.
 The vocally impaired people can communicate their message through text which can
be read out by eSpeak.
 The deaf people can able to hear others speech from text.

CHAPTER 3
HARDWARE AND SOFTWARE REQUIREMENTS

3.1 HARDWARE REQUIREMENTS
3.1.1 RASPBERRY PI
The Raspberry Pi is a bargain basement priced, credit-card sized computer which can be
easily plugged into a computer monitor or TV. We utilize a standard keyboard and black eye.
It is a minute device that enables people of all ages to research computing, and to learn how
to program in languages like Scratch and Python. It is capable of doing everything we would
expect a desktop information processing system to perform, surfing from the net and playing
high-definition videos, to making databases, word-processing, and live games. Raspberry pi
has the capability to intercommunicate with the exterior world, and has been used in a
spacious array in digital projects, from music equipment and sensors to weather stations and
chirping birdhouses with infrared cameras.
3.1.1.1 THE IDEA TO CREATE THE RASPBERRY PI

The idea behind a tiny and affordable computer for kids came in 2006, when Eben Upton,
Rob Mullins, Jack Lang and Alan Mycroft, based at the University of Cambridge’s Computer
Laboratory, became concerned about the year-on-year decline in the numbers and skills
levels of the A Level students applying to read Computer Science. From a situation in the
1990s where most of the kids applying were coming to interview as experienced hobbyist
programmers, the landscape in the 2000s was very different; a typical applicant might only
have done a little web design.
Something had changed the way kids were interacting with computers. A number of
problems were identified: majority of curriculums with lessons on using Word and Excel, or
writing webpages; the end of the dot-com boom; and the rise of the home PC and games
console to replace the Amiga’s, BBC Micros, Spectrum ZX and Commodore 64 machines
that people of an earlier generation learned to program on.
There isn’t much any small group of people can do to address problems like an inadequate
school curriculum or the end of a financial bubble. But those students felt that they could try
to do something about the situation where computers had become so expensive and arcane
that programming experimentation on them had to be forbidden by parents; and to find a
platform that, like those old home computers, could boot into a programming environment.

Thus came the idea of creating the device which kids could buy and learn programming or
hardware on – The Raspberry Pi.
3.1.1.2 INITIAL DESIGN CONSIDERATIONS
From 2006 to 2008 they created many designs and prototypes of what we now know as the
Raspberry Pi. One of the earliest prototypes is shown below:
Figure 3.1 Earliest Prototype of the Pi
These boards use an Atmel ATmega644 microcontroller clocked at 22.1MHz, and a 512K
SRAM for data and frame buffer storage. By 2008, processors designed for mobile devices
were becoming more affordable, and powerful enough to provide excellent multimedia, a
feature which would make the board desirable to kids who wouldn’t initially be interested in
a purely programming-oriented device. The project started to look very realisable and
feasible. Eben (now a chip architect at Broadcom), Rob, Jack and Alan, teamed up with Pete
Lomas, MD of hardware design and manufacture company Norcott Technologies, and David
Braben, co-author of the BBC Micro game Elite, to form the Raspberry Pi Foundation to
make it a reality. Three years later, the Raspberry Pi Model B entered mass production
through licensed manufacture deals with Element 14/Premier Farnell and RS Electronics, and
within two years it had sold over two million units.
Raspberry Pi is a credit-card sized computer manufactured and designed in the United

Kingdom by the Raspberry Pi foundation with the intention of teaching basic computer

science to school students and every other person interested in computer hardware,
programming and DIY-Do-it Yourself projects.
Figure 3.2 Raspberry Pi Board
The Raspberry Pi is manufactured in three board configurations through licensed

manufacturing deals with Newark element14 (Premier Farnell), RS Components and
Egoman. These companies sell the Raspberry Pi online. Egoman produces a version for
distribution solely in China and Taiwan, which can be distinguished from other Pis by their
red colouring and lack of FCC/CE marks. The hardware is the same across all manufacturers.
The Foundation provides Debian and Arch Linux ARM distributions for download. Tools are
available for Python as the main programming language, with support for BBC BASIC (via
the RISC OS image or the Brandy Basic clone for Linux), C, Java and Perl.
3.1.1.3 COMPONENTS OF THE PI.
 Processor: The Raspberry Pi has a Broadcom BCM2837 System on Chip module. It has
quad core ARMv8 CPU .It can run at 1.2GHz with 400MHz Video Core IV multimedia
GPU. The Broadcom SOC used in the Raspberry Pi is equivalent to a chip used in an old
smartphone (Android or iPhone). While operating at 1.2 GHz by default, the Raspberry
Pi provides a real world performance roughly equivalent to the 0.041 GFLOPS. On the
CPU level the performance is similar to a 300 MHz Pentium II of 1997-1999 but the
GPU, however provides Gpixel/s, 1.5 Gtexel/s or 24 GFLOPS of general purpose
compute and the graphics capabilities of the Raspberry Pi are roughly equivalent to the
level of performance of the box of 2001. The Raspberry Pi chip operating at 1.2 GHz by
default, will not become hot enough to need a heatsink or special cooling.

 OS: Boots from SD card, running a version of the Linux (Raspbian) operating system.
 POWER: The Pi requires 5V power supply. It is powered by a Micro USB charger or
the GPIO header. Any good smartphone charger will do the work of powering the Pi.
 GPIO (General Purpose Input Output): General-purpose input/output (GPIO) is a
generic pin on an integrated circuit whose behaviour, including whether it is an input or
output pin, can be controlled by the user at run time. GPIO pins have no special purpose
defined, and go unused by default. The idea is that sometimes the system designer
building a full system that uses the chip might find it useful to have a handful of
additional digital control lines, and having these available from the chip can save the
hassle of having to arrange additional circuitry to provide them.The production
Raspberry Pi board has a 40 pin 2.54 mm (100 mil) expansion header, marked as P1,
arranged in a 2x13 strip. They provide 17 GPIO pins plus access to I²C, SPI UART), as
well as +3.3 V, +5 V and GND supply lines. Pin one is the pin in the first column and
on the bottom row.
Figure 3.3 GPIO connector on R-Pi
 SD Card: The Raspberry Pi does not have any on board storage available. The
operating system is loaded on a SD card which is inserted on the SD card slot on the

Raspberry Pi. The operating system can be loaded on the card using a card reader on
any computer GPIO.
 Video outputs: HDMI, composite video via 3.5 mm audio jack and Composite video port
(PAL and NTSC), Full size HDMI CSI camera port for connecting the Raspberry Pi camera,
DSI display port for connecting the Raspberry Pi touch screen display. The Display Serial
Interface (DSI) is a specification by the Mobile Industry Processor Interface (MIPI)
Alliance aimed at reducing the cost of display controllers in a mobile device. It is
commonly targeted at LCD and similar display technologies. It defines a serial bus and
a communication protocol between the host and the device. A DSI compatible LCD
screen can be connected through the DSI connector, although it may require additional
drivers to drive the display.
 Audio Jack: A standard 3.5 mm TRS connector is available on the RPi for stereo audio
output. Any headphone or 3.5mm audio cable can be connected directly. Although this
jack cannot be used for taking audio input, USB mics or USB sound cards can be used.
 USB 2.0 Port: USB 2.0 ports are the means to connect accessories such as mouse or
keyboard to the Raspberry Pi. There is 1 port on Model A, 2 on Model B and 4 on
Model B+. The number of ports can be increased by using an external powered USB
hub which is available as a standard Pi accessory.
 Ethernet: Ethernet port is available on Model B and B+. It can be connected to a
network or internet using a standard LAN cable on the Ethernet port. The Ethernet ports
are controlled by Microchip LAN9512 LAN controller chip.
 CSI connector: Camera Serial Interface is a serial interface designed by MIPI (Mobile
Industry Processor Interface) alliance aimed at interfacing digital cameras with a mobile
processor. The RPi foundation provides a camera specially made for the Pi which can be
connected with the Pi using the CSI connector.
 HDMI: HDMI – High Definition Multimedia Interface.HDMI 1.3 a type a port is
provided on the RPi to connect with HDMI screens.
3.1.2 SYSTEM ON CHIP (SOC)

Since smartphones and tablets are basically smaller computers, they require pretty much the
same components we see in desktops and laptops in order to offer us all the amazing things
they can do (apps, music and video playing, 3D gaming support, advanced wireless features,
etc.).
But smartphones and tablets do not offer the same amount of internal space as desktops and
laptops for the various components needed such as the logic board, the processor, the RAM,

the graphics card, and others. That means these internal parts need to be as small as possible,
so that device manufacturers can use the remaining space to fit the device with a long-lasting
battery life,Thanks to the wonders of miniaturization, SOC manufacturers, like Qualcomm,
Nvidia or Texas Instruments, can place some of those components on a single chip, the
System on a Chip that powers smartphones.
A system on a chip or system on chip (SoC or SOC) is an integrated circuit (IC) that
integrates all components of a computer or other electronic system into a single chip. It may
contain digital, analog, mixed-signal, and often radio-frequency functions—all on a single
chip substrate. SoCs are very common in the mobile electronics market because of their low
power consumption. A typical application is in the area of embedded systems.
Figure 3.4 Hardware Required to Design The System
The contrast with a microcontroller is one of degree. Microcontrollers typically have under
100 kB of RAM (often just a few kilobytes) and often really are single-chip-systems, whereas
the term SoC is typically used for more powerful processors, capable of running software
such as the desktop versions of Windows and Linux, which need external memory chips
(flash, RAM) to be useful, and which are used with various external peripherals. In short, for
larger systems, the term system on a chip is a hyperbole, indicating technical direction more
than reality: increasing chip integration to reduce manufacturing costs and to enable smaller
systems. Many interesting systems are too complex to fit on just one chip built with a process
optimized for just one of the system's tasks.

A typical SoC consists of:
 A microcontroller, microprocessor or DSP core(s). Some SoCs—called multiprocessor

system on chip (MPSoC)—include more than one processor core.
 memory blocks including a selection of ROM, RAM, EEPROM and flash memory.
 timing sources including oscillators and phase-locked loops.
 peripherals including counter-timers, real-time timers and power-on reset generators.
 external interfaces, including industry standards such as USB, FireWire, Ethernet,
USART, SPI.
 analog interfaces including ADCs and DACs.
 voltage regulators and power management circuits.
 A bus - either proprietary or industry-standard such as the AMBA bus from ARM
Holdings - connects these blocks. DMA controllers route data directly between external
interfaces and memory, bypassing the processor core and thereby increasing the data
throughput of the SoC.
The benefits of SOC are:
 Increase performance.
 Low power consumption yields increased reliability.
 Trim size.
3.1.3 LOGITECH CAMERA

Logitech camera is a plug and play setup and soft to apply. With simple plug and play
arrangement, we can make video calls in no time on most IMS and Logitech video in sky-
scraping definition. With this camera VGA sensor the video captured appears in natural
colour. It has a 5 Megapixel camera with high results. The camera selects the built in Mic
with noise reduction.
Figure 3.5 Logitech Camera

XVGA video recording system has a reach of about 1024*768 resolutions. It is a camera with
motion detection and universal clip and has a broadband of 256 kbps for uploading. It is
provided with 512 MB RAM or more and 200MB hard drive space. We can interfaces hi-
speed USB to other devices for easy utilities.
3.2 SOFTWARE REQUIREMENTS

In our project the Raspberry Pi is interfaced with computer monitor by using the 5v power
cable. Through this line, we operate the kit with the following software. They are divided into
categories as listed below.
3.2.1 MECHANIZE
Mechanize gives you a browser like object to interact with web pages. Mechanize Browser
implements the urllib2.OpenerDirector interface. Browser objects have state, including
navigation history, HTML form state, cookies, etc. The set of features and URL schemes
handled by Browser objects is configurable. The library also provides an API that is mostly
compatible with urllib2. Features of Mechanize browser include: ftp:, http: and file: URL
schemes, browser history, hyperlink and HTML form support, HTTP cookies, HTTP-EQUIV
and Refresh, Referer header, robots.txt, redirections, proxies, and Basic and Digest HTTP
authentication. Mechanize works with Python 2.4, Python 2.5, Python 2.6, and Python 2.7.
3.2.2 CHROMIUM
Chromium is an open-source browser project that forms the basis for the Chrome web
browser. Chromium is started by Google, to provide the source code for the
proprietary Google Chrome browser. When Google first introduced Chrome back in 2008,
they also released the Chromium source code on which Chrome was based as an open-source
project. That open-source code is maintained by the Chromium Project while Chrome itself is
maintained by Google.
Google takes Chromium and then adds the following:
 AAC, H.264, and MP3 Support. Chrome includes licensed codecs for these proprietary
media formats, giving you access to a wider variety of media content—particularly sites
using HTML5 video to stream H.264 videos. Both browsers include the basic, free
codecs: Opus, Theora, Vorbis, VP8, VP9, and WAV.
 Adobe Flash (PPAPI). Chrome includes a sandboxed Pepper API (PPAPI) Flash plug-
in that Google automatically updates along with Chrome. Even on Windows and Mac,

you’re better off with the sandboxed PPAPI Flash plugin from Chrome rather than the
older NPAPI Flash plug-in available from Adobe’s website.
 Google Update. Windows and Mac users of Chrome get an extra background app that
automatically keeps Chrome up-to-date. Linux users use their standard software
management tools.
 Extension Restrictions. For Chrome, Google disables extensions that are not hosted in
the Chrome Web Store.
 Crash and Error Reporting. User of Chrome can opt to send statistics on crashes and
errors to Google for analysis.
 Security Sandbox Google also notes that some Linux distributions may disable
Chromium’s security sandbox, so you’ll want to navigate to about: sandbox in
Chromium to ensure the sandbox is enabled and functioning by default. This is one of
Chromium best features.
On Linux, you can often install Chromium directly from your Linux distribution’s software
repositories. On Ubuntu Linux, for example, you can install it by opening the Ubuntu
Software Centre, searching for Chromium, and then clicking Install. Chromium gets updated
with security updates through your Linux distribution’s software repositories.
3.2.3 TESSERACT OCR
Tesseract is an optical character recognition engine for various operating systems. It is free
software, released under the Apache License, Version 2.0, and development has been
sponsored by Google since 2006. In 2006 Tesseract was considered one of the most accurate
open-source OCR engines then available.
Optical Character Recognition is the process of electronically extracting text from images and
reusing it in a variety of ways such as document editing, free-text searches, or compression.
Optical Character Recognition is a technology that enables you to convert different types of
documents, such as scanned paper documents, PDF files or images captured by a digital
camera into editable and searchable data.
Python-Tesseract is an optical character recognition (OCR) contrivance for python. It will be

familiarized and understand the text embedded in icons. It is a binding for goggle’s OCR. It is
too functional and a stand-alone incantation script to Tesseract, as it can read all facsimile
types reinforced by the Python Imaging collections and others, whereas Tesseract-occur by

default, it ropes tiff and bmp. To boot, if practiced as a script, Python-Tesseract will print the
recognized text instead of writing it to a file.
The Tesseract engine was originally developed as proprietary software at Hewlett
Packard labs in Bristol, England and Greeley, Colorado between 1985 and 1994, with some
more changes made in 1996 to port to Windows, and some migration from C to C++ in 1998.
A lot of the code was written in C, and then some more was written in C++. Since then all the
code has been converted to at least compile with a C++ compiler. It was then released as
open source in 2005 by Hewlett Packard and the University of Nevada, Las Vegas (UNLV).
Tesseract development has been sponsored by Google since 2006.
There are two parts to install in tesseract OCR the engine and the training data for a language.
Tesseract is available directly from many Linux distributions. Packages are also generally
available for language training data unpack it, and copy the trained data file into the 'Tess
data' directory. You can install Tesseract using either Mac-Ports or Homebrew.
Tesseract has Unicode (UTF-8) support, and can recognize more than 100 languages "out of
the box". It can be trained to recognize other languages. Tesseract supports various output
formats: plain-text, html, pdf. In order to get better OCR results, we need to improve the
quality of the image you are giving Tesseract.The latest stable version is 3.05.00. Developers
use lib tesseract C or C++ API to build their own application.
Tesseract Features:
 Page layout analysis.
 More languages are supported.
 Improve forecast accuracy.
 Add UI.
3.2.4 OPENCV
OpenCV is an open source computer vision library. The library is written in C and C++ and
runs under Linux, Windows and Mac OS X. There is active development on interfaces for
Python, Ruby, Matlab, and other languages. OpenCV was designed for computational
efficiency and with a strong focus on real- time applications. OpenCV is written in optimized
C and can take advantage of multicore processors. If you desire further automatic
optimization on Intel architectures, you can buy Intel’s Integrated Performance Primitives
(IPP) libraries, which consist of low-level optimized routines in many different algorithmic
areas. OpenCV automatically uses the appropriate IPP library at runtime if that library is

installed. One of OpenCV’s goals is to provide a simple-to-use computer vision infrastructure

that helps people build fairly sophisticated vision applications quickly. The OpenCV library
contains over 500 functions that span many areas in vision, including factory product
inspection, medical imaging, security, user interface, camera calibration, stereo vision, and
robotics. Because computer vision and machine learning often go hand-in- hand, OpenCV
also contains a full, general-purpose Machine Learning Library (MLL). This sub library is
focused on statistical pattern recognition and clustering. The MLL is highly useful for the
vision tasks that are at the core of OpenCV’s mission, but it is general enough to be used for
any machine learning problem.
OpenCV is a compilation of encoding functions primarily aimed at factual-time computer
vision, originally shaped by Intel research and subsequently supported by Willow Garage and
now maintained by Itseez. It is the collection of cross-platform and free for use under the
open source. OpenCV includes foremost changes to the C++ interface, aiming at ease, more
case-safe designs, and better implementations for existing ones in terms of execution. It is
used in diverse purposes for Facial recognition system, Gesture recognition, Motion
understood.
OpenCV has played a role in the growth of computer vision by enabling thousands of people
to do more productive work in vision. With its focus on real-time vision, OpenCV helps
students and professionals efficiently implement projects and jump-start research by
providing them with a computer vision and machine learning infrastructure that was
previously available only in a few mature research labs.
The CV component contains the basic image processing and higher-level computer vision
algorithms, ML is the machine learning library, which includes many statistical classifiers
and clustering tools. High GUI contains I/O routines and functions for storing and loading
video and images, and CX Core contains the basic data structures and content.
3.2.5 OPENCV-PYTHON
Python is a widely used high-level1programming language for1general-purpose
programming. It became popular programming language because of its simplicity and code
readability. It enables the programmer to express his ideas in fewer lines of code without
reducing any readability.
Compared to other languages like C/C++, Python is slower. But another important feature of
Python is that it can be easily extended with C/C++. This feature helps us to write
computationally intensive codes in C/C++ and create a Python wrapper for it so that we can
use these wrappers as Python modules. This gives us two advantages: first, our code is as fast

as original C/C++ code since it is the actual C++ code working in background and second, it
is very easy to code in Python. This is how OpenCV-Python works, it is a Python wrapper
around original C++ implementation.So, OpenCV-Python is an appropriate tool for fast
prototyping of computer vision. Reading a good Python program feels almost like reading
English. This pseudo-code nature of Python is one of its greatest strengths. It allows you to
concentrate on the solution to the problem rather than the language itself. Python has an
extraordinarily simple syntax, Python is an example of a FLOSS (Free/Libré and Open
Source Software). In simple terms, you can freely distribute copies of this software, read its
source code, make changes to it, and use pieces of it in new free programs. FLOSS is based
on the concept of a community which shares knowledge. When you write programs in
Python, you never need to bother about the low-level details such as managing the memory
used by your program, etc. All your Python programs can work on any of platforms without
requiring any changes at all if you are careful enough to avoid any system-dependent
features. You can use Python on GNU/Linux, Windows, FreeBSD, Macintosh, Solaris,
OS/2,Amiga, AROS, AS/400, BeOS, OS/390, z/OS, Palm OS, QNX, VMS, Psion, Acorn
RISC OS, VxWorks, PlayStation, and Sharp Zaurus Python, on the other hand, does not need
compilation to binary. You just run the program directly from the source code. Internally,
Python converts the source code into an intermediate form called bytecodes and then
translates this into the native language of your computer and then runs it. All this, actually,
makes using Python much easier since you don't have to worry about compiling the program,
making sure that the proper libraries are linked and loaded, etc. This also makes your Python
programs much more portable, since you can just copy your Python program onto another
computer and it works Extensive Libraries the Python Standard Library is huge indeed. It can
help you do various things involving regular expressions, documentation generation, unit
testing, threading, databases, web browsers, CGI, FTP, email, XML, XML-RPC, HTML,
WAV files, cryptography, GUI (graphical user interfaces), and other system-dependent stuff.
Besides the standard library, there are various other high-quality libraries which you can find
at the Python Package Index. Python is indeed an exciting and powerful language. It has the
right combination of performance and features that make writing programs in Python both
fun and easy.
Python is created by Guido van Rossum and first released in 1991. An interpreted language,
Python has a design philosophy which emphasizes code readability and a syntax which
allows programmers to express concepts in fewer lines of code than possible in languages
such as C++ or Java. The language provides constructs intended to enable writing clear

programs on both a small and large scale. Python features a dynamic type system and
automatic memory management and supports multiple programming paradigms,
including object-oriented, imperative, functional programming, and procedural styles. It has a
large and comprehensive standard library.
Python 2.0 was released on 16 October 2000 and had many major new features, including
a cycle-detecting garbage collector and support for Unicode. With this release the
development process was changed and became more transparent and community-backed.
Python 3.0 (which early in its development was commonly referred to as Python 3000 or
py3k), a major, backwards-incompatible release, was released on 3 December 2008, after a
long period of testing. Many of its major features have been backported to the backwards-
compatible Python 2.6.x and 2.7.x version series.
The End of Life dat for Python 2.7 was initially set at 2015, then postponed to 2020 out of
concern that a large body of existing code cannot easily be forward-ported to Python 3. In
January 2017, Google announced work on a Python 2.7 to Go trans compiler, which The
Register speculated was in response to Python 2.7's planned end-of-life but Google cited
performance under concurrent workloads as their only motivation.
Python uses dynamic typing and a mix of reference counting and a cycle-detecting garbage
collector for memory management. An important feature of Python is dynamic name
resolution (late binding), which binds method and variable names during program execution.
The design of Python offers some support for functional programming in the Lisp tradition.
The language has map (), reduce () and filter () functions; list comprehensions, dictionaries,
and sets; and generator expressions. The standard library has two modules (intercools and
functions) that implement functional tools borrowed from Haskell and Standard ML.
Python is used by hundreds of thousands of programmers and is used in many places.

Sometimes only Python code is used for a program, but most of the time it is used to do
simple jobs while another programming language is used to do more complicated tasks.
Its standard library is made up of many functions that come with Python when it is installed.
On the Internet there are many other libraries available that make it possible for the Python
language to do more things. These libraries make it a powerful language; it can do many
different things.

Some things that Python is often used for are:
 Web development
 Game programming
 Desktop GUIs
 Scientific programming
 Network programming.
3.2.6 VNC VIEWER

Virtual Network Computing (VNC) is a graphical desktop sharing system that uses
the Remote Frame Buffer protocol (RFB) to remotely control another computer. It transmits
the keyboard and mouse events from one computer to another, relaying the
graphical screen updates back in the other direction, over a network. VNC is platform
independent .There are clients and servers for many GUI-based operating systems and
for Java. Multiple clients may connect to a VNC server at the same time. Popular uses for this
technology include remote technical support and accessing files on one's work computer from
one's home computer, or vice versa. VNC was originally developed at the Olivetti & Oracle
Research Lab in Cambridge, United Kingdom. There are a number of variants of VNC which
offer their own particular functionality; e.g., some optimized for Microsoft Windows, or
offering file transfer etc. Many are compatible with VNC proper in the sense that a viewer of
one flavor can connect with a server of another; others are based on VNC code but not
compatible with standard VNC.
The VNC server is the program on the machine that shares some screen and allows the client
to share control of it. The VNC client (or viewer) is the program that represents the screen
data originating from the server, receives updates from it, and presumably controls it by
informing the server of collected local input. The VNC protocol (RFB protocol) is very
simple, based on one graphic primitive from server to client and event messages from client
to server. In the normal method of operation a viewer connects to a port on the server. In its
simplest form, the VNC protocol can use a lot of bandwidth, so various methods have been
devised to reduce the communication overhead. The VNC protocol allows the client and
server to negotiate which encoding will be used VNC is commonly used as a cross-platform
remote desktop system. For example, Apple Remote Desktop for Mac OS X.
VNC Connect is the latest version of our remote access software for personal and commercial
use. It utilizes VNC technology to connect you to a remote computer wherever you are,
watch its screen in real-time, and take control. VNC Connect consists of a VNC Server app

for the computer you want to control, and a VNC Viewer app for the device you want to
control from. VNC uses the RFB protocol to transmit screen pixel data from one computer to
another over a network, and send control events in return. VNC Server captures the desktop
of the computer in real-time and sends it to VNC Viewer for display. VNC Viewer gathers
your input (mouse, keyboard, or touch) and sends it for VNC Server to inject and
actually achieve remote control. Two main use cases are unattended access (connecting in to
your own computer from elsewhere) and remote helpdesk (an administrator or customer
service agent connecting in to your computer to help you with a problem).
RealVNC is a company that provides remote access software. The software consists of a
server and client application for the Virtual Network Computing (VNC) protocol to control
another computer's screen remotely. Andy Harter (CEO of RealVNC Limited) and other
members of the original VNC team at AT&T founded RealVNC Limited in 2002. For a
desktop-to-desktop connection RealVNC runs on Windows, Mac OS X, and Unix
like operating systems. A RealVNC client also runs on the platform, Apple iPhone, iPod
touch iPad and Google Android devices. In November 2016, RealVNC released the updated
version of their software, now called VNC Connect (version 6.0). The new version introduces
a cloud connection option using a subscription-based pricing model. Users can choose
between three subscription levels; Home (free for non-commercial use), Professional and
Enterprise. Home and Professional subscriptions are cloud connections only. The Enterprise
subscription supports hybrid connections that include the traditional direct (peer to peer)
connections and/or cloud connections.
RealVNC clients using VNC viewer can run in full-screen mode; they use the F8 function
key as the default key for bringing up an options menu (which includes the option to, among
other things, switch off full screen mode or to forward a Control-Alt-Delete key .The server
component of RealVNC allows a computer to be remotely controlled by another computer.
RealVNC have ported their VNC server and viewer applications to Pi, and they are now
integrated with the system. To enable the server, select the option on the Interfaces tab in
Raspberry Pi Configuration; you’ll see the VNC menu appear on the taskbar, and you can
then log in to your Pi and control it remotely from a VNC viewer.
3.2.7 ESPEAK
Espeak is a compact open source software speech synthesizer for Linux, Windows another
platforms. It uses a formant synthesis method, providing many languages in a small size.
Much of the programming for Espeak language support is done using rule files with feedback

from native speakers. Because of its small size and many languages, it is included as the
default speech synthesizer in the NVDA open source screen reader for Windows,
Android, Ubuntu and other Linux distributions. Its predecessor Espeak was used by Google
Translate for 27 languages in 2010; 17 of these were subsequently replaced by commercial
voices. Espeak can be used as a command-line program, or as a shared library. It
supports Speech Synthesis Markup Language (SSML). Language voices are identified by the
language's ISO 639-1 code. They can be modified by voice variants". These are text files
which can change characteristics such as pitch range, add effects such as echo, whisper and
croaky voice, or make systematic adjustments to formant frequencies to change the sound of
the voice. For example, "af" is the Afrikaans voice. "af+f2" is the Afrikaans voice modified
with the "f2" voice variant which changes the formants and the pitch range to give a female
sound. Espeakng uses an ASCII representation of phoneme names which is loosely based on
the Kirshenbaum system. Installation is done as follows:
Step 1: Install Python:
If you haven't installed Python already, install it via apt-get:
sudo apt-get install python-dev
Step 2: Install Espeak
To install the TTS engine (Espeak) and the Python module, use the following command:
sudo apt-get install Espeak
sudo apt-get install Espeak python-Espeak
Step 3: Speak!
To speak a sentence, use the command -From Espeak import Espeak
Espeak. Synth ("Hello Instructables!")
This allows many languages to be provided in a small size. The speech is clear, and can be
used at high speeds, but is not as natural or smooth as larger synthesizers which are based on
human speech recordings.
Espeak is available as:
 A command line program (Linux and Windows) to speak text from a file or from stdin.
 A shared library version for use by other programs. (On Windows this is a DLL).
 A SAPI5 version for Windows, so it can be used with screen-readers and other
programs that support the Windows SAPI5 interface.
 Espeak has been ported to other platforms, including Android, Mac OSX and Solaris.

Features of Espeak:
 Includes different Voices, whose characteristics can be altered.
 Can produce speech output as a WAV file.
 SSML (Speech Synthesis Mark-up Language) is supported and also HTML.
 Compact size. The program and its data, including many languages, totals about 2
Mbytes.
 Can be used as a front-end to MBROLA diaphone voices, see mbrola.html. eSpeak
converts text to phonemes with pitch and length information.
 Can translate text into phoneme codes, so it could be adapted as a front end for another
speech synthesis engine.
 Potential for other languages. Several are included in varying stages of progress. Help
from native speakers for these or other languages is welcome.
 Development tools are available for producing and tuning phoneme data.
 Written in C.
The Espeak speech synthesizer supports several languages, however in many cases these are
initial drafts and need more work to improve them. Espeak edit is a GUI program used to
prepare and compile phoneme data. Originally known as speak and originally written for
Acorn/RISC_OS computers starting in 1995. This version is an enhancement and re-write,
including a relaxation of the original memory and processing power constraints, and with
support for additional languages Espeak uses speech engine in the libespeak shared library.
The libespeak library must first be installed.
Espeak uses the Port Audio sound library (version 18), so you will need to have
the libportaudio0 library package installed. Some Linux distributions have version 19 of Port
Audio which has a slightly different API. The speak program can be compiled to use version
19 of port Audio by copying the file portaudio19.h to port audio before compiling.
Linux will talk like a fast robot, and you might need to pay careful attention to understand
what is being said. However, there are many options available that adjust the voice.
Enter man eSpeak for details.The default speaking speed of 160 words per minute is too fast
to be intelligible. Use the -s option to change the voice speed. Use the -v option followed by
the language (e.g. for British English) plus a type. We can be lowered pitch with the -
p option. We can use en+f5 or en+m2 to adjust the voice types for different effects (f is
female and m is male), but exact information regarding this is scarce. Trial and error seems to
be the best solution.These are text files, not audio files, and they specify how voices should

be generated. You can even create your own, so consult the eSpeak documentation for
details. These voices require the -v option in order to use them.
We can even combine voices (that is what the + sign was for in en+m5). Choose a language,
such as en or en-us (en is British English, which lends a slight accent). Useful variations are
found in the lv directory. This is where we found the m1-m7 and f1-f5 voices. There are
others, such as croak and whisper.
Append one of these to the en or en-us base. Use the + sign without spaces.
eSpeak -s 125 -v en-us+f2 'msg'
Some text, but different speaker. We hear a pseudo-female speaking.We can try croak.
espeak -s 125 -v en+croak 'Listen to my lovely voice as I try not to cry'espeak -s 125 -v
en+whisper 'Please be quiet in the sanitarium'
espeak -s 125 -v en+f5 'Please be quiet in the sanitarium'
espeak -s 125 -v en+f5+whisper 'Please be quiet in the sanitarium'
This only produces the default robotic voice, not a whispering female. For that, we must
use whisperf (whisperf is the female whisper effect).
espeak -s 125 -v en+whisperf 'Please be quiet in the sanitarium'
Sounds like a gas pipe leak to me, but oh, well. Using the klatt3 effect adds a special robotic
touch.
Fortune eSpeak -s 125 -v en+klatt3
The key to remember is the <language>+<type> combination.
Espeak GUI. Limited, but fun to show what eSpeak can do. The GUI lets you change the
speed, male/female voice, but that is all. For advanced details, use the eSpeak command at
the command line with custom options.
3.3 OPERATING SYSTEM

The Raspberry Pi primarily uses Linux kernel-based operating systems. The ARM11 is based
on version 6 of the ARM which is no longer supported by several popular versions of Linux,
including Ubuntu. The install manager for Raspberry Pi is NOOBS. The OSs included with
NOOBS are:
 Archlinux ARM
 OpenELEC
 Pidora (Fedora Remix)
 Raspbmc and the XBMC open source digital media centre
 RISC OS – The operating system of the first ARM-based computer

 Raspbian (recommended) – Maintained independently of the Foundation; based on

ARM hard-float (armhf)-Debian 7 'Wheezy' architecture port, that was designed for a
newer ARMv7 processor whose binaries would not work on the Rapberry Pi, but
Raspbian is compiled for the ARMv6 instruction set of the Raspberry Pi making it work
but with slower performance. It provides some available deb software packages, pre-
compiled software bundles. A minimum size of 2 GB SD card is required, but a 4 GB
SD card or above is recommended. There is a Pi Store for exchanging programs. The
'Raspbian Server Edition (RSEv2.4)', is a stripped version with other software packages
bundled as compared to the usual desktop computer oriented Raspbian.
3.3.1 BOOT PROCESS
The Raspberry Pi does not boot as a traditional computer. The Video Core i.e. the Graphics
processor actually boots before the ARM CPU.
Figure 3.6 Boot process of Raspberry Pi

The boot process of the Raspberry Pi can be explained as follows:
 When the power is turned on, the first bits of code to run is stored in a ROM chip in the
SoC and is built into the Pi during manufacture. This is the called the first-stage
bootloader.
 The SoC is hardwired to run this code on start-up on a small RISC Core (Reduced
Instruction Set Computer). It is used to mount the FAT32 boot partition in the SD Card
so that the second-stage bootloader can be accessed. So what is this ‘second-stage
bootloader’ stored in the SD Card? It’s ‘bootcode.bin’. This file can be seen while
mount process of an operating system on the SD Card in windows.
 Now here’s something tricky. The first-stage bootloader has not yet initialized the ARM
CPU (meaning CPU is in reset) or the RAM. So, the second-stage bootloader also has to
run on the GPU. The bootloader.bin file is loaded into the 128K 4 way set associative
L2 cache of the GPU and then executed. This enables the RAM and loads start.elf which
is also in the SD Card. This is the third-stage bootloader and is also the most important.
It is the firmware for the GPU, meaning it contains the settings or in our case, has
instructions to load the settings from config.txt which is also in the SD Card. We can
think of the config.txt as the ‘BIOS settings’.
 The start.elfalso splits the RAM between the GPU and the ARM CPU. The ARM only
has access the to the address space left over by the GPU address space. For example, if
the GPU was allocated addresses from 0x000F000 – 0x0000FFFF, the ARM has access
to addresses from 0x00000000 – 0x0000EFFF.
 The physical addresses perceived by the ARM core is actually mapped to another
address in the Video Core (0xC0000000 and beyond) by the MMU (Memory
Management Unit) of the Video Core.
 The config.txt is loaded after the split is done so the splitting amounts cannot be
specified in the config.txt. However, different .elf files having different splits exist in
the SD Card. So, depending on the requirement, the file can be renamed to start.elf and
boot the Pi. In the Pi, the GPU is King.
 Other than loading config.txt and splitting RAM, the start.elf also loads cmdline.txt if it
exists. It contains the command line parameters for whatever kernel that is to be loaded.
This brings us to the final stage of the boot process. The start.elf finally
loads kernel.img which is the binary file containing the OS kernel and releases the
reset on the CPU. The ARM CPU then executes whatever instructions in the
kernel.img thereby loading the operating system.

 After starting the operating system, the GPU code is not unloaded. In fact, start.elf is not
just firmware for the GPU, It is a proprietary operating system called VideoCore OS
(VCOS). When the normal OS (Linux) requires an element not directly accessible to it,
Linux communicates with VCOS using the mailbox messaging system.
3.3.2 RASPBERRY PI COMPATIBLE OPERATING SYSTEMS
Table 1 List of supported Operating Systems
Distribution Type Memory Packages

footprint
Arch Linux ARM Linux 8,700
BerryTerminal Linux
Bodhi Linux Raspbian 35,000+

ARMHF
Debian ARM Linux 20,000+
Fedora Remix Linux 16,464?
Gentoo Linux Linux ~23 MiB
IPFire Linux ~20 MiB 144
I2PBerry Linux 20,000+
Meego MER + XBMC Linux (embedded) ~34 MiB + ~320 (core)

XBMC
Moebius Raspbian ~20 MiB (core) + Raspbian

Repositories
Nos Linux ~90 MiB 35,000+
OpenSUSE Linux 3.11 28 MiB (inc. 6300

X11)

OpenWRT Linux 3,3MiB 3358
PiBang Linux Linux_3.6.11

&SystemD
PwnPi Linux 20,000+
QtonPi Linux
VPNbian Linux ~40 MiB w/o 35,000+

desktop
Raspbian Linux ~30 MiB w/o 35,000+

desktop
OpenELEC Linux 3.10.16 95 MiB (incl. ~140 (+ 7 via xbmc)

(embedded) XBMC)
XBian Raspbian 35,000+
Raspbmc Raspbian 20,000+
RISC OS RISC OS
Aros hosted on Raspbian Mixed Debian6 and <~50 MiB

Limited Demo Aros
Plan9 Plan 9
SlaXBMCRPi Linux 3.10.36+ 476

(+ Official SlackwareARM
14.1 Packages)
PiMAME Linux
PiBox Linux/Buildroot
pipaOS Raspbian ~32 MiB 37.500
Raspberry WebKiosk Raspbian
Volumio Raspbian
Nard SDK Embedded Linux ~40 MB

3.3.3 RASPBIAN OS
Raspbian is a Debian-based computer operating system for Raspberry Pi. It is now officially
provided by the Raspberry Pi Foundation, as the primary operating system for the family of
Raspberry Pi single-board computers. Raspbian was created by Mike Thompson and Peter
Green as an independent project. The initial build was completed in June 2012. The operating
system is still under active development. Raspbian is highly optimized for the Raspberry Pi
line's low-performance ARM CPUs. Raspbian is the Best All-Around Operating
System. Raspbian is the “official” operating system of the Raspberry Pi and because of that,
it's the one most people will want to start with. Raspbian is a version of Linux built
specifically for the Raspberry Pi.Raspbian is a free operating system based on Debian
optimized for the Raspberry Pi hardware. An operating system is the set of basic programs
and utilities that make your Raspberry Pi run.
Raspbian uses PIXEL, Pi Improved Xwindows Environment, Lightweight as its main desktop
environment as of the latest update. It is composed of a modified LXDE desktop environment
and the Open box stacking window manager with a new theme and few other changes. The
distribution is shipped with a copy of computer algebra program Mathematica and a version
of Minecraft called Minecraft Pi as well as a lightweight version of Chromium as of the latest
version.
The goal of Raspbian is to become the leading OS of choice for all users of the Raspberry Pi.
This goal has been largely achieved. There are still some rough edges but these are generally
more related to the Pi's hardware and kernel than Raspbian itself. Raspbian images are
produced by various people. For newcomers we recommend the images provided by the
Raspberry Pi foundation, these are available from their download page Raspberry Pi
Foundation's download page. Raspbian tries to stay as close to Debian as reasonably possible.
Debian is used by millions of users around the globe on a daily basis and there is a vast store
of knowledge and documentation about using Debian across the web. Any information you
find that applies to Debian will almost certainly apply to the same version of Raspbian.
Information for earlier versions of debian will often apply too but may need some tweaks just
as it would when using it on a newer version of debian. Raspbian is an unofficial port of
Debian wheezy armhf with compilation settings adjusted to produce code that uses "hardware
floating point", the "hard float" ABI and will run on the Raspberry Pi. The port is necessary
because the official Debian wheezy armhf release is compatible only with versions of the
ARM architecture later than the one used on the Raspberry Pi (ARMv7-A CPUs and higher,
vs the Raspberry Pi's ARMv6 CPU). The Debian squeeze image issued by the Raspberry Pi

foundation was based on debian armel which uses software floating point and the "soft float"
ABI. The foundation used the existing Debian port for less capable ARM devices. Therefore,
it did not use of the Pi's processor's floating point hardware - reducing the Pi's performance
during floating point intensive applications - or the advanced instructions of the ARMv6
CPU.
Debian is a free operating system for your computer and includes the basic set of programs
and utilities that make your computer run along with many thousands of other packages.
Debian has a reputation within the Linux community for being very high-quality, stable and
scalable. Debian also has an extensive and friendly user community that can help new users
with support for practically any problem. This makes Debian an ideal operating system for
the Raspberry Pi that will be used by children and many others using Linux for the first time.
Raspbian is a operating system optimized for the Raspberry Pi hardware.
However, Raspbian provides more than a pure OS: it comes with over 35,000 packages, pre-
compiled software bundled in a nice format for easy installation on your Raspberry
Pi. Raspbian is based on Debian, which is a Linux distribution. It's calledRaspbian because
it's a portmanteau of Raspberry and Debian; much in the same way as Rasbmc or Pidora.
Figure 3.7 Raspbian OS Interface

To update an existing Jessie image, type the following at the command line:
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install -y rpi-chromium-mods
sudo apt-get install -y python-sense-emu python3-sense-emu
sudo apt-get install -y python-sense-emu-doc realvnc-vnc-viewer and then reboot.
The RealVNC server is used to remotely access Pi, type the following:
sudo apt-get install -y realvnc-vnc-server
Figure 3.8 Application View
3.3.4 SETTINGS AND CONFIGURATION OF PI
There are a couple of new settings dialogs in this release, found under the Preferences entry
in the main menu. The first is Raspberry Pi Configuration – this is A GUI version of the old
raspi-config command-line application, which provides all the same functionality in a nicer
interface. (The old raspi-config is still on the system and can be accessed from the command
line by typing “sudo raspi-config”, but it shouldn’t be necessary to do so any more.The new
Raspberry Pi Configuration allows you to enable and disable interfaces, tweak performance

and configure internationalisation options, such as timezone and keyboard. It also allows
some more control over boot options than was available in the past, with the option to
automatically log in as the “pi” user available when booting to both CLI and desktop.
Figure 3.9 Raspberry Pi Configuration
There is a new keyboard setting dialog, accessed from the Localisation tab, but hopefully
many people won’t need this – the system will detect some common keyboards sold for use
with Pi and set up the GUI keyboard driver correctly. If that doesn’t happen, it’s now easy to
choose the right country and keyboard type in this dialog.

CHAPTER 4
DESIGN AND IMPLEMENTATION

4.1 BLOCK DIAGRAM
Figure 4.1 Block Diagram
The proposed system consists of two units. A subsystem which can clipped on the collar of
the shirt (as a lavalier microphone). This consists of input – condenser microphone and
output – speaker sections and a camera to capture and record human face for future reference
with their voice modulation. The main system is a portable device where the user sends and
receives messages as text. Also the device acts as mobile phone and the communication is
performed as text chatting. The communications between these two units are via Bluetooth.
When a normal person tries to interact with a disabled person using this device, the device
starts vibrating informing that someone is around you for interaction. It takes inputs from the
lavalier microphone. Then the device does speech to text (STT) conversion and displays it on
the device screen, based on what the normal person conveys. The user can give reply as text
message and the device does text to speech (TTS) conversion. The output is obtained from a
small and powerful speaker. An advanced prediction texting concept will be provided, so that

the delay in communication can be reduced. Finally contacts of every person can be
stored/recorded as voice modulation and face photographs based on image processing
technique deployed with the device camera.
4.2 FLOW DIAGRAM

The entire operation of the project is performed using the flow chart as below:
Figure 4.2 Flow Diagram

4.3 PROJECT OVERVIEW

In our task we have identified the above software which has been initially employed in the
procedure before it has taken into account. The Jessie is a terminal window which is used to
serially transmit the data to another window by way of commands through command prompt
window. After this the RealVNC takes place which moves as a remote desktop between the
raspberry pi and Putty. It transmits the files in a safe way. Here, that the visually and vocally
impaired people are catered with the raspberry pi based assistive devices and the system
follows three important processes for blind, deaf and dumb people.
The total project is catered with the source code of Python. It is the easiest programming
language to interface with the raspberry pi. The total project is run by the source code of
python to assisting the blind, deaf and dumb masses in a single device which is so compact
and easy for them to manage. The output window is termed as AAWAAZ, It exposes the
entire output screen to the user and which process is to run out in a childlike way.
We aim for developing the prototype model for blind dumb and deaf people by employing in
a single compact device. The project provides a unique solution for these people to manage
their sites by themselves. The project is catered with the source code of Python. It is the
easiest programming language to interface with the Raspberry Pi. The project is run by the
source code of Python to assist blind dumb and deaf people in a single device which is so
compact and easy for them to manage.
Figure 4.3 Software Interface

The system is provided with 4 switches. Each switch has different functions. We have
choosen the switch for necessary conversion:
 Text to speech (TTS) using (SW0)
 Text to speech using camera (TTSC) using (SW1)
 Speech to text(STT) using (SW2)
 Gesture control using(SW3)
4.3.1 TEXT TO SPEECH (TTS)

The first process text to speech conversion is done for the dumb masses who cannot speak.
The Dumb people convert their thoughts to text which could be transferred to a voice signal.
The converted voice signal is speak out by espeak synthesizer. After pressing the switch SW0
the OS and sub process imported. Call text to speech function and enter the text as input.
Enter The Text” is speak out by espeak for raw input. After entering the text from keyboard,
the espeak synthesizer converts text to speech. The process also provided with the keyboard
interrupt ctrl+C.
4.3.2 TEXT TO SPEECH USING CAMERA(TTSC)
The second process is developed for blind people who cannot read normal text. When the
visually impaired people have to buy some products or read any wordings in the image,they
face problem to read. In order to help blind people, we have interfaced the Logitech camera
to capture the image by using OPENCV tool. The captured image is converted to text using
Tesseract OCR and save the text to file out1.txt. Open the text file and split the paragraph
into sentences inorder to understand the meaning of the language and save it to file out1.txt.
In OCR, the adaptive thresholding techniques are used to change the image into binary
images and the are transferred to character outlines. The converted text is read out by the
espeak. The process also provided with the keyboard interrupt ctrl+C.
4.3.3 SPEECH TO TEXT (STT)

The third process is developed for the hearing impairment people who cannot understand the
words of normal people. In order to help them, our project is provided with a switch which is
used to convert the voice of the normal people text. This procedure is held out by assigning
the minimum threshold voltage to recognize the voice signal and begin entering the voice
signal through the microphone and later obtaining the signal it convert them into message to
other individuals. We have used a chromium browser which is automatically connected to
URL speechtexter.com. The process is performed by assigning a minimum threshold voltage

to recognize the voice signal. The input is given through a microphone which is converted
into a text format. The URL supports a variety of languages. If the voice signal recognizable
it will print the text else it gives the error signal. The process also provided with the keyboard
interrupt ctrl+C.
4.3.4 GESTURE CONTROL
The fourth process is developed for the vocally impaired people who cannot exchange the
thoughts to the normal people. Dumb people uses gesture to communicate with normal
people which are majorly cannot be understandable by normal people. The process started
with the capturing the image and read it. Cropped the useful portion of image to recognize the
gesture of hand. Convert the RGB image into gray scale image for better functioning of hand
gesture process. The next step is to Blur the cropped image using Gaussian blur function
which blurr the useful portion of the image and pass it to the threshold function to get the
highlighted part of the image. Find the contours and an angle between two fingers. By using
convex hull function we can implement the finger point. Count the number of angles which is
less than 90 degree which gives the number of defects. According to the number of defects,
the text is printed on display and read out by the Speaker. The process also provided with the
keyboard interrupt ctrl+C.
4.3.4.1 APPLICATION OF GESTURE CONTROL
The applications of hand gesture recognition include:
 Sign language- The problems of deaf and dumb have been worsened by the absence of
a proper sign language interpreter in India. The number of teachers, instructor strained
in Indian Sign Language are very low so we need a system which teaches this sign
language. It narrows the communication gap with the normal people.
 Medical Applications- Doctors can manipulate digital images during medical

procedures using the simple hand gestures instead of touch screens or computer
keyboards, which would be very beneficial. It allows the doctors to remain in place
during an operation and not move for any procedure, without the need to operate
traditional computer-based interfaces. They even have the option of zooming in and out
a particular image by moving the hand clockwise or counter clockwise.
 Automated Home- The smart TV support motion control as we can easily switch
channels, change volume, play games, pause videos, and even surf the web all using
hand gestures. Similarly, the smart refrigerators, air conditioners and microwave ovens

can be controlled using hand gestures. Another important application is controlling the
Wi-Fi-connection using hand gestures. Of all these, this paper focuses on the application
of sign language which would prove beneficial to the society. It is a great step towards
the progress and development of the society.

CONCLUSION AND FUTURE SCOPE

The deaf and dumb are not sitting in the house as past, they are working outside and doing
great. So an efficient system must be set up, to interact with them. Sign language is very
popular among them and hand gesture recognition is just a small part of it.
The proposed system aims at bridging the communication gap between two strata of the
society. Considerable amount of work has been done previously in this domain, but this paper
adds in complete both way communication in an efficient manner because the system will be
implemented as a handy mobile application. So, it truly serves its needs in all aspects. The
above specified strategies prove efficient in terms of time and accuracy. Further
enhancements, can be made in terms of implementing the communicator with other sign
languages like American Sign Language, accent recognition for various ascents throughout
the globe, emotion recognition in sign language and language translation.
By this paper, we have designed the prototype model for blind, deaf and dumb people by
employing a single compact device. The important key factor of this project to facilitate these
people and to fix them more confident to manage their sites by themselves. The primary
advantage is that the device can be taken away easily and is of about less weight.
To further this project can be followed out with any other advanced devices by using simple
coding language to get it less complicated. The complication can be reduced by a tiny gadget
which could be more useful those people in this electronic world.
The proposed system fulfils the hand gesture recognition process but with some limitations as
both the hands cannot be used with this technique, the results are not that efficient. Proper
light conditions help in easy detection of region of interest. The system can be further
expanded for alphabets. Like numbers, each alphabet has its own unique gesture. Thus, the
detection and recognition would be efficient using this technique. Not only this, the input can
be taken in the form of videos. The videos can then be divided into frames. From those
frames, the necessary hand region could be extracted using bounding box, and then following
the same procedure as mentioned in the proposed framework. Every language has its own
grammar. The sign language has it as well. By recognizing the frame images and using
proper parsing algorithm, a grammatical structure could be formed. This would take the
system of hand gesture recognition step ahead. This system can be made handy by
incorporating it in a mobile phone. Just a click, the image is captured and corresponding
result is obtained. Thus, a more efficient way of interaction could be achieved.

The system can be further expanded for the alphabet, numbers in gesture control. The input
can be also taken in the form of videos and the videos are divided into frames and the frames
can be converted into text. We can also add grammatically structure for sign language. The
system can be made handy by incorporating it into a mobile phone.

REFERENCES
[1]. Shraddha R. Ghorpade, Surendra K. Waghamare, “Full DuplexCommunication System for
Deaf & Dumb People,” International Journal of Emerging Technology and Advanced
Engineering (IJETAE), Volume 5, Issue 5, May 2015, ISSN 2250-2459.
[2]. Er. Aditi Kalsh, Dr. N.S. Garewal, “Sign Language Recognition System,” International
Journal of Computational Engineering Research (IJCER), Volume 03, Issue 6, June 2013.
[3]. Vishal Bhame, R. Sreemathy, HrushikeshDhumal, “Vision Based Hand Gesture
Recognition Using Eccentric Approach for Human Computer Interaction,” International
Conference on Advances in Computing, Communications and Informatics (ICACCI),
2014.
[4]. Cordelia Schmid, Roger Mohr, Christian Bauckhage, “Evaluation of Interest Point
Detectors,” International Journal of Computer Vision (IJCV), 151–172, 2000.
[5]. Qing Chen, Nicolas D. Georganas, Emil M. Petriu, “Real-time Vision based Hand
Gesture Recognition Using Haar-like Features,” Instrumentation and Measurement
Technology Conference (IMTC) Warsaw, Poland, May 1-3, 2007.
[6]. SawantPramada, Deshpande Saylee, NalePranita, NerkarSamiksha, Mrs. Archana S.
Vaidya “Intelligent Sign Language Recognition Using Image Processing,” IOSR Journal
of Engineering (IOSRJEN), Volume 3, Issue 2, Feb. 2013, PP 45-51
[7]. http://www.lifeprint.com/asl101/pagessigns/ n/numbers1-10.htm
[8]. M.K. Bhuyan, Debanga Raj Neog and Mithun Kumar Kar, “Fingertip Detection for Hand
Pose Recognition,” International Journal on Computer Science and Engineering (IJCSE),
Vol. 4No. 03 March 2012, ISSN: 0975-3397.
[9]. Ashish Sethi, HemanthS,Kuldeep Kumar, Bhaskara Rao N, Krishnan R, Sign Pro-An
Application Suite for Deaf and Dumb, IJCSET —May 2012— Vol 2, Issue 5,1203-1206
ISSN 2231-0711.
[10]. Ms. Rashmi D. Kyatanavar, Prof. P. R. Futane ,Video Gesture Classification Using
Fourier Descriptors and Genral Fuzzy Min Max Neural Network, International Journal of
Emerging Technology and Advanced Engineering. ISSN 2250-2459, Volume 2, Issue 5,
May 2012.
[11]. Ma De-yi, Chen Yi-min, Li Qi-ming, Huang Chen, Xu Sheng, ”Region Growing by
Exemplar- Based Hand Segmentation under Complex Backgrounds”.

[12]. ShilpaKamdi, R.K. Krishna, Image Segmentation and Region Growing Algorithm,
International Journal of Computer Technology and Electronics Engineering (IJCTEE)
Volume 2, Issue 1 ISSN 2249-6343.
[13]. Zeno GeradtsJurrienBijhold Rob Hermsen, Use of Correlation algorithms in a
database of spent cartridge cases of firearms.
[14]. D. Sasirekha, E. Chandra Text to speech International Journal of Soft Computing and
Engineering (IJSCE) ISSN: 2231-2307, Volume-2, Issue-1, March 2012.
[15]. P. Rajesh Kumar P.V.V. Kishore Segment, Track, Extract, Recognize and Convert
Sign Language Videos to Voice/Text, (IJACSA) International Journal of Advanced
Computer Science and Applications, Vol. 3, No.6, 2012.
[16]. Kamalpreet Sharma1, Naresh Kumar Garg1, kamalpreet Sharma hand gestures
recognition for deaf and dumb International Journal of Computer Application and
Technology (s), May - 2014, pp. 1013 ISSN 2349-1841.
[17]. Aditi Kalsh, N.S. Garewal Sign Language Recognition for Deaf and Dumb
International Journal of Advanced Research in Computer Science and Software
engineering Volume 3, Issue 9, September 2013 ISSN: 2277 128X.
[18]. AnupNandy, SoumikMondal, Jay Shankar Prasad, Pavan Chakra borty and G.C.
Nandi Recognizing and Interpreting Indian Sign Language Gesture for Human Robot
Interaction Intl Conf. on Computer and Communication Technology ICCCT10.
[19]. Chucai Yi, Student Member, IEEE, Yingli Tian, Senior Member, IEEE, and Aries
Arditi “Portable Camera-Based Assistive Text and Product Label Reading from Hand-
Held Objects for Blind Persons” 2013 IEEE.
[20]. Vasanthi.G and Ramesh Babu.Y Department of ECE, DMI College of Engineering,
Chennai, India. “Vision Based Assistive System for Label Detection with Voice
Output”Jan-2014.
[21]. S.Hariharan,K.AbdulRahim,T.Ajith Prakash G. Ganesh Kumar,” A New Concept Of
Electronic Braille display Interfaced With Computer” in April-2009.
[22]. Bachar Y.R, Gupta. R, Pathan W.A (E&T Dept. SIER NASIK, SPP University, Pune,
India)”Smart Speaking Gloves For Speechless “.
[23]. DharanikumarChowdary. M, M. Narasimhulu, G. Subrahmanya Sharma, “Advanced
Embedded Chatter Box for Physically Challenging Persons” in Jan 2012.
[24]. BhavinaPatel,Vandana Shah, Ravindra k Shirsagar, “Microcontroller Based gesture
RecoginitionSyatem For the Handicap People”in Dec-2011.

[25]. Pallavi Varma, Shimi.S.L, Richapriyadarshini,”Design of Communication Interpreter

for Deaf and Dumb person” in Jan-2015.
[26]. S.B.Shroke, NandarDehpande, Prashant Deshmukh, SA=njay Kumar Mathapati.
”Assistive TransulatorFor Deaf and Dumb People”-in July-2014.
[27]. Michael McEnancy”Finger Reader Is audio reading gadget for Index Finger “in July
2014.

An Innovative Communication System For Deaf Dumb and Blind People

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

An Innovative Communication System For Deaf Dumb and Blind People

Transféré par

Droits d'auteur :

Formats disponibles

An Innovative Communication System for Dumb, Deaf and Blind People 2016-17

Dept. of ECE, SJBIT Page 1

Dept. of ECE, SJBIT Page 2

Figure 1.1 ASL numbers (1-10)

Dept. of ECE, SJBIT Page 3

Figure 1.2 Representation of ISL Alphabets

Dept. of ECE, SJBIT Page 4

1.1 EXISTING SYSTEM

Figure 1.3 Braille Scheme

Dept. of ECE, SJBIT Page 5

Figure 1.4 Finger Reader

Figure 1.5 Finger View

Dept. of ECE, SJBIT Page 6

Figure 1.6 Bar Code

Figure 1.7 Sign Language Representation

Dept. of ECE, SJBIT Page 7

[2]. Title: “Sign Language Recognition System”

Communication is the process of exchanging information, views and expressions between

Dept. of ECE, SJBIT Page 8

Dept. of ECE, SJBIT Page 9

[5]. Title: “AAWAAZ: A Communication System for Deaf and Dumb"

Dept. of ECE, SJBIT Page 10

Dept. of ECE, SJBIT Page 11

Dept. of ECE, SJBIT Page 12

HARDWARE AND SOFTWARE REQUIREMENTS

3.1.1.1 THE IDEA TO CREATE THE RASPBERRY PI

Dept. of ECE, SJBIT Page 13

3.1.1.2 INITIAL DESIGN CONSIDERATIONS

Figure 3.1 Earliest Prototype of the Pi

Raspberry Pi is a credit-card sized computer manufactured and designed in the United

Dept. of ECE, SJBIT Page 14

Figure 3.2 Raspberry Pi Board

The Raspberry Pi is manufactured in three board configurations through licensed

3.1.1.3 COMPONENTS OF THE PI.

Dept. of ECE, SJBIT Page 15

Figure 3.3 GPIO connector on R-Pi

Dept. of ECE, SJBIT Page 16

3.1.2 SYSTEM ON CHIP (SOC)

Dept. of ECE, SJBIT Page 17

Figure 3.4 Hardware Required to Design The System

Dept. of ECE, SJBIT Page 18

A typical SoC consists of:

 A microcontroller, microprocessor or DSP core(s). Some SoCs—called multiprocessor

3.1.3 LOGITECH CAMERA

Figure 3.5 Logitech Camera

Dept. of ECE, SJBIT Page 19

3.2 SOFTWARE REQUIREMENTS

Dept. of ECE, SJBIT Page 20

3.2.3 TESSERACT OCR

Python-Tesseract is an optical character recognition (OCR) contrivance for python. It will be

Dept. of ECE, SJBIT Page 21

Dept. of ECE, SJBIT Page 22

installed. One of OpenCV’s goals is to provide a simple-to-use computer vision infrastructure

Dept. of ECE, SJBIT Page 23

Dept. of ECE, SJBIT Page 24

Python is used by hundreds of thousands of programmers and is used in many places.

Dept. of ECE, SJBIT Page 25

Some things that Python is often used for are:

3.2.6 VNC VIEWER

Dept. of ECE, SJBIT Page 26

Dept. of ECE, SJBIT Page 27

Dept. of ECE, SJBIT Page 28

Dept. of ECE, SJBIT Page 29