GESTURE BASED ROBOTICS

VEHICLES USING SIXTH SENSE

TECHNOLOGY

O.V.CHANUKYA
2210416337
ECE A3
WHAT IS A GESTURE?

• A gesture is a form of non-verbal communication or non-vocal

communication in which visible bodily actions communicate
particular messages, either in place of, or in conjunction with,
speech.

• Gestures include movement of the hands, face, or other parts

of the body.
FUNCTIONAL FLOWCHART

• Inbuilt webcam of the system is used to capture the real time

video of gestures.
• Colour markers are used to interpret the gesture
movements.
• It shows the basic flow of the system. To transmit the gesture
commands wirelessly ZigBee series2 module is used .
• Zigbee coordinator is connected to the serial port of the
system through USB Explorer board. Zigbee router is mounted
on the voltage regulator to regulate the voltage to 3.3V. In
order to control the robotic vehicle Arduino board is used
VIDEO FOR WEB CAM
• The user wears coloured tapes to provide input gestures to
the system. The inbuilt webcam of the computer is used as
a sensor to detect the hand movements of the user.
• It captures the real time video at a fixed frame rate and
resolution which is determined by the hardware of the
camera.
GESTURE RECOGNITION
• Gesture recognition can be seen as a way for computers to
begin to understand human body language, thus building
a richer bridge between machines and humans than
primitive text user interfaces or even GUIs (Graphical User
Interface).
• Gesture recognition enables humans to interface with the
robot and interact naturally without any mechanical
devices. In gesture recognition based on the colour
detected and its position it is interpreted into gestures.
• Based on the corresponding gesture, robotic movement is
done.
IMPLEMENTATION
• Acquiring images
• RGB to grayscale conversion
• Extraction of red color
• Gray to binary scale conversion
• Finding centroid of an object
• Serial communication between MATLAB and Arduino
• Controlling robotic vehicles
APPLICATIONS
The applications of the gesture based controlled robot include

• These robots are used in military applications to operate

robots
• These robots are used in medical applications for the
purpose of surgery
• These robotics are used in the construction field
• These robotics are used in industries to control trolly and lift
ADVANTAGES
• Speed and sufficient reliable for recognition system
• Good performance system with complex background
• Simple, fast and easy to implement
• No training is required
DISADVANTAGES
• Irrelevant object might overlap with the hand
• Wrong object extraction appeared if the objects larger than
the hand
• Ambient light affects the color detection threshold
THANK YOU