A Project Report

on

Object Detecting Pick and Place Robot

Submitted By:

1.Amarsinh Dhepe
2.Rohit Gadhave
3.Ashok Chavhan

Under the guidance of

Mr.A.D.Sonavane

In fulfillment of
B.E.(E & TC)
Degree of Savitribai Phule Pune University,Pune
(2018-2019)

JSPM’s
Rajarshi Shahu College of Engineering,
Pune-411033
Department of Electronics & Telecommunication

1
 CERTIFICATE
This is to certify that Project Report entitled

“ Object Detesting Pick and Place Robot ”

By
1.Amarsinh Dhepe
2.Rohit Gadhave
3.Ashok Chavhan

Is a record of bonafied work carried out by them in the Department of

Electronics & Telecommunication Engineering, under my guidance in the
fulfillment for the award of completion of final year of engineering in
Electronics & Telecommunication Engineering of Savitribai Phule Pune
University, Pune.

Mr.A.D.Sonawane
Project Guide

Dr.B.D.Jadhav Dr.R.K.Jain
HOD (E & TC) Principal

JSPM’s
Rajarshi Shahu College of Engineering, Pune-33

2
 ACKNOWLEDGEMENT

It gives us immense pleasure in presenting the report on

(“Object Detecting Pick and Place Robot”)

We wish to express our sincere thanks to Dr.R.K.Jain, Principal, Rajarshi Shahu

College of Engineering Tathawade for providing us all the necessary facilities.
We would like to place on record our deep sense of gratitude to Dr.B.D.Jadhav,
HOD,Department of Electronics & Telecommunication Engineering, for his
stimulating guidance and continous encouragement.
We also wish to extend our thanks to Mrs.C.V.Rane, Project co-ordinator for her
supervision throughout the course of present work.
We are extremely thankful to Mr.A.D.Sonavane for his insightful comments and
constructive suggestions to improve the quality of project work.
Lastly we are thankful to Teaching and non-teaching faculty of the Department
for their continous co-operation.

Students Name
1. Amarsinh Dhepe
2. Rohit Gadhave
3. Ashok Chavhan

3
 INDEX

LIST OF FIGURE
LIST OF TABLE
ABSTRACT
1. INTRODUCTION 1
2. LITERATURE REVIEW 3
3. PROBLEM DEFINATION 5
4. SYSTEM DESIGN 7
4.1 BLOCK DIAGRAM 8
4.2 POWER SUPPLY 9
4.3 ARDUINO 9
4.4 RASPBERRY PI 10
4.5 MOTOR DRIVER CKT 11
4.6 DC MOTOR 12
4.7 SERVO MOTOR 12
4.8 CAMERA 13
5. SOFTWARE DESIGN 14
5.1 OpenCV 15
5.2 ARDUINO(IDE) 15
5.3 ALGORITHM 16
5.4 FLOW CHART 17
6. RESULT AND CONCLUSION 18
6.1 OBSERVATION TABLE 19
6.2 CONCLUSION 19
6.3 APPLICATION 19
REFERENCES 20

4
 List of Figures

Figure Number Figure Name Page Number

Fig.1.1 Robotics Arm 2
Fig.4.1 Block Diagram 8
Fig.4.2 Power Supply 9
Fig.4.3 Arduino 9
Fig.4.4 Raspberry pi 10
Fig.4.5 Motor Driver ckt 11
Fig.4.6 DC Motor 12
Fig.4.7 Servo Motor 12
Fig.4.8 Camera 13
Fig.5.1 Flow Chart 17

List of Tables
Number of Table Name of Table Page Number
Tab.6.1 Observation Result 19

5
 Abstract
The mankind always strived to automate the things in order to find substitutes
for himself to carry out his orders and also work in hostile environment. The
popular concept of robot is of a machine that looks and works like human being.
The industry is moving from the current state of automation to Robotization, to
increase productivity and to deliver uniform quality. The pick and place robot is
one of the technologies in manufacturing industry and designed to perform pick
and place functions. The system is very important to eliminate human errors and
to get more precise work. It can also save the cost in long term and help to solve
problems and tasks that cannot be done such as on high temperature area,
narrow area and very heavy load thing. The rotation of this robot is 180 degree
(clockwise) and -180 degree (counter clockwise). The arm will move
horizontally to pick up and hold the object from base A and places it to base B.
This robot is used to pick and place the object only in their specifications
(between 10g and 50g). The benefit is the robot is that the construction cost is
very low.
The robotics arm kit is made of two section. The bottom driving unit takes
the robot to left, right, forward and backward motion. The top gripper unit is to
pick and place any object. The driving unit has two motors and also the gripper
unit has two motors. The aim of this project is to design an autonomous robot
with complete system allow the robot to interact with certain object and pick
and place appropriate object.

6
 Chapter 1

Introduction

1
 Introduction

In this highly developing society time and man power are critical constrains for
completion of task in large scales. The automation is playing important role to
save human efforts in most of the regular and frequently carried works. One of
the major and most commonly performed works is picking and placing of jobs
from source to destination.
The pick and place robot is a microcontroller based mechatronic
system that detects the object, picks that object from source location and places
at desired location. The project consist of Robotics arm, Raspberry pi, Arduino,
camera, servo motors, dc motors, motor driving ckt, conveyor belt etc...
The Raspberry pi and camera is used for video processing purpose,
Arduino is used to control motors and to control robotics arm. The robotics arm
kit is made of two section. The bottom driving unit takes the robot to left, right,
forward and backward motion. The top gripper unit is to pick and place any
object. The driving unit has two motors and also the gripper unit has two
motors. The aim of this project is to design an autonomous robot with complete
system allow the robot to interact with certain object and pick and place
appropriate object.

Fig1.1 Robotics Arm

2
 Chapter 2

Literature Survey

3
 Literature Survey

1. IEEE paper by Andrew B. Cushing, Jackrit Suthakom and Gregory S.

Chirrikjian from Department of Mechanical Engineering, Johns Hopkins
University Baltimore, Maryland, on implemention of the “An
Autonomous Self-Replicating Robotic System”.In this paper they
describe the concept of self-replicating machines was introduced a fully
autonomous self-replicating robot has not been implemented until now.
They describe a fully autonomous prototype that demonstrates robotic
self replication. This work builds on results in remote-controlled robotic
replication and semi-autonomous replicating robotic systems.

2. IEEE paper named “Pick and place industrial robot controller with
computer vision” by Pratiksha Andhare and Sayali Rawat gives us idea
about finding the x, y coordinate of object. This is found by converting
pixel co-ordinate into real world co-ordinate with the help of 2D
transformation. We incorporate robotic arm controller based on location
and orientation of object. Fundamentally the problem taken in this work
is to pick and place objects which are placed under vision sensor.
Computer vision is the visual discernment segment of an eager
motivation to copy human insight and to enrich robots with savvy
conduct.

3. IEEE paper named “Object detection and recognition for a pick and place
Robot” by Rahul kumar, Sunil Lal, Sanjesh Kumar and Praneel Chand
helps us to improve image processing algorithm. This paper is directed
towards the development of the image processing algorithm which is a
pre-requisite for the full operation of a pick and place Robotic arm
intended for object sorting task For this type of task first the objects are
detected, and this is accomplished by feature extraction algorithm. Then
extracted image is sent to the classifier to recognize what object it is and
once this is finalized, the output would be the type of the object along
with it's coordinates to be ready for the Robotic Arm to execute the pick
and place task The major challenge faced in developing this image
processing algorithm was that upon making the test subjects in
compliance with the classifier parameters, resizing of the images
conceded in the loss of pixel data. Therefore, a centered image approach
was taken.

4
 Chapter 3

Problem Defination

5
 Problem Defination

Aim:-

The aim of this project is to design an autonomous robot with complete

system allows the robot to interact with certain objects that it encounters with
the help of camera to detect appropriate object and to pick and place appropriate
object.

In order to achieve the aim of this project, several objectives are needed to be
completed.

Objective:-

1. To study the concept on how pick and place robot function and operate.
2. To design the hardware for pick and place robot.
3. To make the system run continuously with less rest.
4. To make an interface between the programming of robot arm and the
camera.
5. To make the robot’s gripper has the function of pick and place.
6. To make the robot capable to stop at the desire location accurately.

6
 Chapter 4

System Design

7
 System Design

Block Diagram and Description :-

Fig.4.1 Block Diagram

The Block Diagram mainly consist of

1.Power Supply

2.Arduino

3.Raspberry pi

4.Motor Drive

5.DC Motor

6.Servo Motor

7.Camera

8
1.Power Supply:-

In our project we require 12V and 5V regulated power supply. This is a simple
approach to obtain a 12V and 5V DC power supply using a single circuit. The
circuit uses two ICs 7812 and 7805 for obtaining the required voltages. The
AC mains voltage will be stepped down by the transformer T1, rectified by
bridge B1 and filtered by capacitor C1 to obtain a steady DC level .The IC1
regulates this voltage to obtain a steady 12V DC. The output of the IC1 will be
regulated by the IC2 to obtain a steady 5V DC at its output. In this way both
12V and 5V DC are obtained. Such a circuit is very useful in cases when we
need two DC voltages for the operation of a circuit. By varying the type number
of the IC1 and IC2, various combinations of output voltages can be obtained.

Fig.4.2 Power Supply

2.Arduino:-

In our project we are using Arduino to control servo Motor and DC Motor.
Servo Motor are used in movement of Robotics Arm and DC motor is used in
movement of belt. Arduino is a microcontroller board based on the
ATmega328P. It has 14 digital input/output pins (of which 6 can be used as
PWM outputs), 6 analog inputs, a 16 MHz quartz crystal, a USB connection, a
power jack. It contains everything needed to support the microcontroller.

Fig.4.3 Arduino
9
 Arduino Specification

 Microcontroller: ATmega328P
 Operating Voltage: 5V
 Input Voltage (recommended): 7-12V
 Input Voltage (limit): 6-20V
 Digital I/O Pins: 14 (of which 6 provide PWM output)
 PWM Digital I/O Pins: 6
 Analog Input Pins: 6
 DC Current per I/O Pin: 20 mA
 DC current for 3.3V Pin: 50 mA
 Flash Memory: 32 KB (ATmega328P) of which 0.5 KB used by bootloader
 SRAM: 2 KB (ATmega328P)
 EEPROM: 1 KB (ATmega328P)
 Clock Speed: 16 MHz
 Length: 68.6 mm
 Width: 58.4 mm
 Weight: 25 g

3.Raspberry pi:-
In our project we are using Raspberry Pi for video processing purpose. Video
processing is done with the help of camera and Raspberry pi. Video is captured
with camera and different processing algorithms performed on data and object
detection is done on the basis of colours. Raspberry pi provide us 1.4GHz 64-bit
quad-core processor, dual-band wireless LAN, Bluetooth 4.2, faster Ethernet,
and Power-over-Ethernet support.

Fig.4.4 Raspberry pi
10
Specifications of Rasberry Pi:-

o SoC: Broadcom BCM2836 (CPU, GPU, DSP, SDRAM)

o CPU: 900 MHz quad-core ARM Cortex A7 (ARMv7 instruction set)
o GPU: Broadcom VideoCore IV @ 250 MHz
o Memory: 1 GB (shared with GPU)
o USB ports: 4
o Video input: 15-pin MIPI camera interface (CSI) connector
o Video outputs: HDMI, composite video (PAL and NTSC) via 3.5 mm
jack
o Audio input: I²S
o Audio outputs: Analog via 3.5 mm jack; digital via HDMI and I²S
o Storage: MicroSD
o Network: 10/100Mbps Ethernet
o Peripherals: 17 GPIO plus specific functions, and HAT ID bus
o Power rating: 800 mA (4.0 W)
o Power source: 5 V via MicroUSB or GPIO header

4.Motor Driver ckt :-

Motor drivers acts as an interface between the motors and the control circuits.
Motor require high amount of current whereas the controller circuit works on
low current signals. So the function of motor drivers is to take a low-current
control signal and then turn it into higher current signal that can drive a
motor.In our project we are using motor driver circuit to control DC motor,
motor driver ckt consist of MOSFET IRF540.

Fig.4.5 Motor Driver ckt

11
5.DC motor:-

In our project DC motor is used for rotation of conveyor belt with the help of
two wheels. A DC motor is an electrical machine which converts electrical
energy into mechanical energy. The working of DC motor is based on the
principle that when a current carrying conductor is placed in a magnetic field, it
experiences a mechanical force. The direction of the mechanical force is given
by Fleming’s Left-hand Rule and its magnitude is given by F = BIL Newton. In
our project we are using DC motor with speed 120 RPM.

Fig.4.6 Dc Motor

6.Servo Motor:-

In our project Servo motor is used for movement of robotics arm. Servo motors
are controlled with the help of Arduino. Servo motor works on the PWM ( Pulse
Width Modulation ) principle, which means its angle of rotation is controlled by
the duration of pulse applied to its control PIN. Basically servo motor is made
up of DC motor which is controlled by a variable resistor (potentiometer) and
some gears. A servo motor is a rotary actuator or a motor that allows for a
precise control in terms of the angular position, acceleration, and velocity.
Basically it has certain capabilities that a regular motor does not have.
Consequently it makes use of a regular motor and pairs it with a sensor for
position feedback. The servo motor is most commonly used for high technology
devices in the industrial applications like automation technology.

Fig.4.7 Servo Motor

12
7.Camera:-

In our project camera is used for video processing purpose. Camera is used to
capture video data and send data to processor to process the data, with the help
of captured data then processor detect an object on basis of colour of object.
Digital cameras look very much like ordinary film cameras but they work in a
completely different way. When you press the button to take a photograph with
a digital camera, an aperture opens at the front of the camera and light streams
in through the lens. So far, it's just the same as a film camera. From this point
on, however, everything is different. There is no film in a digital camera.
Instead, there is a piece of electronic equipment that captures the incoming light
rays and turns them into electrical signals. This light detector is one of two
types, either a charge-coupled device (CCD) or a CMOS image sensor.

Fig.4.8 Camera

13
 Chapter 5

Software Design

14
 SOFTWARE DESIGN:-

Software used:-

1 .OpenCV:-

In our project OpenCV software is used for video processing. Object detecting
process is done with the help of camera and OpenCV. OpenCV (Open Source
Computer Vision Library) is an open source computer vision and machine
learning software library. OpenCV was built to provide a common
infrastructure for computer vision applications and to accelerate the use of
machine perception in the commercial products. Being a BSD-licensed product,
OpenCV makes it easy for businesses to utilize and modify the code.
The library has more than 2500 optimized algorithms, which includes a
comprehensive set of both classic and state-of-the-art computer vision and
machine learning algorithms. These algorithms can be used to detect and
recognize faces, identify objects, classify human actions in videos, track camera
movements, track moving objects, extract 3D models of objects, produce 3D
point clouds from stereo cameras, stitch images together to produce a high
resolution image of an entire scene, find similar images from an image database,
remove red eyes from images taken using flash, follow eye movements,
recognize scenery and establish markers to overlay it with augmented reality,
etc. OpenCV has more than 47 thousand people of user community and
estimated number of downloads exceeding 18 million. The library is used
extensively in companies, research groups and by governmental bodies.
2. Arduino(IDE):-

In our project to write program for ATmega328p and to upload program we are
using Arduino (IDE). ). The open-source Arduino Software (IDE) makes it easy
to write code and upload it to the board. It runs on Windows, Mac OS X, and
Linux. The environment is written in Java and based on Processing and other
open-source software. With Arduino IDE, users can easily access contributed
libraries and receive up-to-date support for the latest Arduino boards, so they
can create sketches that are backed by the newest version of the IDE. Arduino
IDE has more than 700 libraries integrated. These were written and shared by
members of the Arduino community that other users can utilize for their own
projects without having to install anything. This enables programmers to add a
different dimension to their sketches.

15
Algorithm:-
1. Start
2. Select an object with specific colour to detect.
3. Convert RGB image to HSV image
4. Masking Operation.
5. Object Detection.
6. Stop movement of conveyor belt.
7. Pick detected object.
8. Place object at desire position.
9. End

16
Flow Chart:-

Fig.5.1 FLOW CHART

17
 Chapter 6

Result and Conclusion

18
 Result and Conclusion

Observation Result:-

Observation Time taken to Time taken to pick Time taken to

Number detect object object place object

1 3 sec 7 sec 4 sec

2 4 sec 8 sec 4 sec

3 5 sec 7 sec 5 sec

4 3 sec 6 sec 5 sec

Tab.6.1 Observation Result

Conclusion:-
By working on this project we have learned need of robot in situations
where we have to perform same task repeatedly in same manner and importance
of automation in industry to save time and energy.

Applications:-
 In industry for packing purpose.

 In industry as a seperator.

 As assembly robot.

 To pick ripe fruit.

 To sorting purpose.

19
 References:-

IEEE Papers:-
[1] Andrew B. Cushing, “An Autonomous Self-Replicating Robotic System”.
[2] Pratiksha Andhare, “Pick and place robot controller with computer vision” .
[3] Rahul kumar, “Object detection and recognition for a pick and place Robot”.

Websites:-
1. www.wikepedia.com
2. www.datasheetcatlog.com
3. www.ieeeXplore.com
4. www.electronicsforyou.com

20