HAND GESTURE RECOGNITION SYSTEM FOR DUMB

PEOPLE

by
AMIT SINGH (1316110024)

ANKIT CHOUDHARY (1316110031)

ARUN KUMAR ATTRI (1316110045)

ASHUTOSH SINGH (1316110052)

Submitted to the Department of Computer Science and Engineering

in partial fulfillment of the requirements

for the degree of

Bachelor of Technology

in

Computer Science and Engineering

Krishna Engineering College

Dr. A.P.J. Abdul Kalam Technical University, Lucknow

May, 2017
 DECLARATION

I hereby declare that this submission is my own work and that, to the best of my knowledge and
belief, it contains no material previously published or written by another person nor material which
to a substantial extent has been accepted for the award of any other degree or diploma of the
university or other institute of higher learning, except where due acknowledgment has been made
in the text.

AMIT SINGH (1316110024)

ANKIT CHOUDHARY (1316110031)

ARUN KUMAR ATTRI (1316110045)

ASHUTOSH SINGH (1316110052)

Date:

ii
 CERTIFICATE

This is to certify that Project Report entitled “Hand Gesture Recognition System For Dumb
People” which is submitted by Amit Singh (1316110024),Ankit Choudhary (1316110031),
Arun Kumar Attri (1316110045), Ashutosh Singh (1316110052) in complete fulfillment
of the requirement for the award of degree B. Tech. in Department of Computer Science
and Engineering of Dr. A.P.J. Abdul Kalam Technical University, is a record of the
candidate own work carried out by him under my supervision. The matter embodied in this
is original and has not been submitted for the award of any other degree.

Date: Supervisor

Dr. Dileep Yadav

iii
 ACKNOWLEDGEMENT

It gives us a great sense of pleasure to present the report of the B. Tech Project undertaken during
B. Tech. Final Year. We owe special debt of gratitude to Assistant Professor, Dr. Dileep Yadav
Department of Computer Science & Engineering, Krishna Engineering College, Ghaziabad and for
his constant support and guidance throughout the course of our work. His sincerity, thoroughness
and perseverance have been a constant source of inspiration for us. It is only his cognizant efforts
that our endeavors have seen light of the day.

We also take the opportunity to acknowledge the contribution of Professor Dr. Mayank Singh,
Head, Department of Computer Science & Engineering, Krishna Engineering College, Ghaziabad
for his full support and assistance during the development of the project.

We also do not like to miss the opportunity to acknowledge the contribution of all faculty members
of the department for their kind assistance and cooperation during the development of our project.
Last but not the least, we acknowledge our friends for their contribution in the completion of the
project.

AMIT SINGH (1316110024)

ANKIT CHOUDHARY (1316110031)

ARUN KUMAR ATTRI (1316110045)

ASHUTOSH SINGH (1316110052)

Date:

iv
 ABSTRACT

In our country around 2.78% of peoples are not able to speak (dumb). Their
communications with others are only using the motion of their hands and expressions. We
proposed a new technique called artificial speaking mouth for dumb people. It will be very
helpful to them for conveying their thoughts to others. Some peoples are easily able to get
the information from their motions. The remaining is not able to understand their way of
conveying the message. In order to overcome the complexity the artificial mouth is
introduced for the dumb peoples. A camera records a live video stream, from which a
snapshot is taken with the help of interface. The system is trained for each type of count
hand gestures (one, two, three, four, and five) at least once. After that a test gesture is
given to it and the system tries to recognize it. A research was carried out on a number of
algorithms that could best differentiate a hand gesture. It was found that the diagonal sum
algorithm gave the highest accuracy rate. In the preprocessing phase, a self-developed
algorithm removes the background of each training gesture. After that the image is
converted into a binary image and the sums of all diagonal elements of the picture are
taken. This sum helps us in differentiating and classifying different hand gestures.
Previous systems have used data gloves or markers for input in the system. I have no such
constraints for using the system. The user can give hand gestures in view of the camera
naturally. A completely robust hand gesture recognition system is still under heavy
research and development; the implemented system serves as an extendible foundation for
future work.

v
 TABLE OF CONTENTS Page

DECLARATION ii
CERTIFICATE iii
ACKNOWLEDGEMENTS iv
ABSTRACT v
LIST OF TABLES vii
LIST OF FIGURES viii
LIST OF SYMBOLS ix
LIST OF ABBREVIATIONS x
CHAPTER 1 - INTRODUCTION 1
1.1 Problem Statement 2
1.2 Problem Definition 3
1.3 Expected Outcomes 3
CHAPTER 2 - LITERATURE SURVEY 7
CHAPTER 3 - PROPOSED METHODOLOGY 17
3.1 Project Constraintss 17
3.2 The Web Cam System 18
3.3 Brief Outline Of The Implementation System 19
3.4 ADVANTAGES 36
3.5 REQUIREMENT SPECIFICATION 37
3.5.1 Hardware and Software Requirement 37
CHAPTER 4 - FEATURE EXTRACTIONS 26
4.1 NEURAL NETWORKS 26
4.2 Row Vector Algorithm 28
4.3 Results of Concentration Analysis 42
CHAPTER 5 - CONCLUSION 43
REFERENCES 46
APPENDIX 48

LIST OF TABLE Page

Table 4.1 Table for Average Time per Matching 42

vi
 LIST OF FIGURES Page

Figure 2.1 LIGHTING CONDITION AND BACKGROUND 4

Figure 2.2 HAND GESTURE RECOGNITION FLOW CHART 5
Figure 2.3 THE EFFECT OF SELF SHADOWING (A) AND CAST SHADOWING (B) 10
Figure 2.4 SYSTEM IMPLEMENTATION 23
Figure 3.1 DIFFERENT ETHNIC GROUP SKIN PATCHES 25
Figure 3.2 REMOVAL OF BACKGROUND 26
Figure 3.3 LABELING SKIN REGION 27
Figure 3.4 REAL TIME CLASSIFICATION 28
Figure 3.5 NEURAL NETWORK DIGRAM 29
Figure 3.6 NEURAL NET BLOCK DIGRAM 30
Figure 3.7 NN FOR ROW VECTOR AND EDGINE VECTOR 31
Figure 3.8 NN FOR MEAN STANDARD DEVIATION 32
Figure 3.10 ROW VECTOR FLOW CHART 33
Figure 3.11 DFD Level 0 – Automated Attendance System 32
Figure 3.12 DFD Level 1 – Student Face Registration Module 33
Figure 3.13 DFD Level 1 – Student Face Recognition Module 34
Figure 3.14 DFD Level 1 – Concentration Analysis Module 35
Figure 4.1 EDGING AND ROW VECTOR FLOW CHART 39
Figure 4.2 MEAD NAD SD FLOW CHART 39
Figure 4.3 DIAGONAL SUM 40
Figure 4.4 DIAGONAL SUM FLOW CHART 40
Figure 4.5 GRAPHIC USER INTERFACE 41
Figure 4.6 PERFORMANCE CHART
Figure 4.7 GRAPHICAL USER INTERFACE OUTPUT
Figure 4.6 PERFORMANCE PERCENTAGE
Figure 4.6 DEGREE OF ROTATION CLASSIFICATION
Figure 4.6 DEGREE OF ROTATION MISCALSSIFICATION
Figure 4.6 ETHNIC GROUP SKIN DETECTION
Figure 4.6 CLASSIFICATION PERCENTAGE

42

vii
 LIST OF SYMBOLS

∑ Summation

∫ Integrate

 Belongs to

α Alpha

ψ Psi

Ф Phi

λ Lambda

μ Mu

√ Square root

Ω Omega

Δ Delta

Θ Theta

viii
 LIST OF ABBREVIATIONS

CQ Concentration Quotient

DFD Data Flow Diagram

HDD Hard Disk Drive

PC Personal Computer

PCA Principal Component Analysis

RAM Random Access Memory

ix