1 INTRODUCTION

Bar code is a Iast, easy, accurate and automatic data

collection method. Barcode enables products to be tracked
eIIiciently and accurately at speeds not possible using
manual data entry systems. The barcode reader only be used
to recognize the barcode, and the price oI two dimensional
bar codes reader is expensive. Now mobile phones can
implement many new kinds oI applications such as taking
photos, and movie shooting by using embedded camera
devices. So an interesting approach is capturing bar-codes
with their cameras and decoding them with soItware
running on the phone |1|. Recently, the mobile industry
began to pay more attention to barcode applications in
m-commerce because 2D-barcodes not only provide a
simple and inexpensive method to present diverse
commerce data, but also improve mobile user experience
by reducing their inputs |2|.
Previous research work has shown that recognition oI 2D
barcode is a researching hotspot and very diIIicult in
various conditions. These conditions include highlight
spots, skew projection, low contradistinction,
non-homogeneous lighting, and various mixed conditions.
Ohbuchi et al. |3| presented an algorithm capable oI the
real-time recognition oI barcodes on a mobile phone. The
most important drawback oI the method is the Iact that it
has been hand tailored Ior one certain hardware device. It
relies on access to a powerIul but also very speciIic
hardware element, which is not accessible by normal
application developers: the built-in signal processor oI a
device |4|. Besides that, the algorithm proposed by
Ohbuchi relies on two speciIic conditions: Prior to the
code`s decoding, the barcode`s position is detected using a
spiral scanning algorithm that runs on the device`s signal
processor. This algorithm makes the assumption that the
point in the middle oI the screen is located in the code. This
assumption is not realistic in usually environment.
Ying et al. |5| presented an algorithm capable oI
recognizing the PDF 417 barcodes in real time with a

This work is supported by Doctor Foundation oI Jinan University
(B0609).
mobile phone. This method includes three steps. The Iirst
step detects the code region using the Otsu algorithm and
the Least Square Method. The second method searches Ior
the cut-oII rules with a scanning approach. In the third step
symbol characters are segmented Irom the original image.
So the successIul binarization oI the code areas during the
Iirst step is the most key step in his algorithm. This is mean
that the recognition result will rely on the eIIect oI the Otsu
method. But the Otsu method could be Iailed in complex
lighting conditions, including highlight spots, low
contradistinction, non-homogeneous lighting, and various
mixed conditions |6|. So the recognition rate oI the
algorithm proposed by Ying is not high especially in
complex lighting conditions. Additionally, many
constraints are added in this method, so the application
scope is decreased.
Sun et al. |7| introduce an algorithm to analyze and correct
the distorted image oI QR Code. The algorithm consists
gray-scale image transIormation, Canny edge detection,
external contours Iinding, inverse perspective
transIormation and cell grids generating. In this method, the
recognition time will be cost more than Ying`s method
because there is no binarization operation; gray image is
processed in all steps oI Sun`s method. The Processing time
oI the black-white image is less than the processing time oI
gray image.
In this paper we describe a new image recognition
algorithm which applied to capturing image in various
lighting conditions. The paper is organized as Iollows:
section two simply introduces the structure oI QR Code;
section three describes the proposed approach Ior QR Code
reading with mobile phones; its experiment results are
given in section Iour. Conclusions are given in section Iive.
2 THE STRUCTURE OF QR CODE
QR Code (Quick Response Code) is developed in Japan by
Denso Corporation in 1994, and later recognized as a
standard. QR Code has been approved as an AIM Standard,
a JIS Standard and an ISO standard |8|. So QR Code is
being used in a wide variety oI applications, such as
manuIacturing, logistics, and sales applications. In 2000

Recognition of QR Code with Mobile Phones
Yue Liu, Ju Yang, Mingjun Liu
School oI InIormation Science and Engineering, University oI Jinan, Jinan 250022, China
E-mail: iseliuyujn.edu.cn

Abstract: Quick Response Code has been widely used in the automatic identiIication Iields. In order to adapting various
sizes, a little dirty or damaged, and various lighting conditions oI bar code image, this paper proposes a novel
implementation oI real-time Quick Response Code recognition using mobile, which is an eIIicient technology used Ior
data transIerring. An image processing system based on mobile is described to be able to binarize, locate, segment, and
decode the QR Code. Our experimental results indicate that these algorithms are robust to real world scene image.
Key Words: Quick Response Code, recognition, mobile phone

203
978-1-4244-1734-6/08/$25.00 c 2008 IEEE

years, QR Code is being issued as National standard in
China |9|. There are 40 versions in QR Code, Iour levels oI
error correction, and the maximum symbol size (the highest
version) can encoding 7089 numeric data or 4296
alphanumeric data. The Reed-Solomon is used in QR Code,
and the highest level oI error correction allows recovery oI
30 oI the symbol code words. In addition to, QR Code has
a clear Ieature that a Iinder pattern composed oI three
square patterns in the upper leIt and right and the lower leIt
corners.
Each QR Code symbol consists oI an encoding region and
Iunction patterns, as shown in Fig. 1. Function patterns
include Iinder, separator, timing patterns and alignment
patterns. The Iinder patterns located at three corners oI the
symbol intended to assist in easy location oI its position,
size and inclination.

Fig 1. The structure oI QR Code
3 OUR APPROACH
In this Section, the recognition algorithm oI QR Code is
introduced, which is used in various conditions. The
algorithm consists oI several steps, as shown in Fig. 2, gray
scale image conversion, binarization, Iilter, orientation
(Iinder patterns or timing patterns location), alignment
patterns location, cell grids generating, error correction and
decoding. The input is an RGB color image which is
captured by mobile phone and the output is a decoding
result.

Fig. 2 The Ilowchart oI our approach

The real line oI Fig. 2 expressed to implement in a sequence
in the algorithm. The dotted line showed that the step is
used in other algorithm, but our approach skipped the step.
3.1 Gray Conversion
QR Code symbol is captured by mobile phone with camera,
and images are captured in RGB 24bit Iormat by most
phones, but QR Code symbol is a set oI dark and light
pixels. It is needless to deal with color inIormation and the
gray image calculated quickly with little space, so gray
conversion is needed to do Iirstly.
3.2 Binarization
Selection oI a proper binarization method is critical to the
perIormance oI barcode recognition system. In binarizing
an image, a simple and popular method is threshold.
Among more than 20 threshold methods, ReI. |10|
concluded that Otsu`s method is the best, which chooses the
threshold that minimizes within-group variance. But in our
test, Otsu`s algorithm is not satisIactory perIormance in
uneven illumination and is not real-time implementation.
Ohbuchi |3| propose a global threshold method. First he
divided the center 60*60 areas oI 320*240 images into nine
blocks. Then calculate the gray histogram Ior each block,
and sort the gray value. The middle value is chooses as the
threshold oI each block. Finally the smallest value oI these
middle values is the global threshold oI this whole image.
This method is existed two drawbacks: (1) the barcode
symbol must be in the center oI the captured image,
otherwise the method will Iailed. (2) This method results in
excessive segmentation. Separate a part oI barcode image
as background and lead to decode Iailure.
Using a global thresholding method, iI an image has
variable lighting conditions, the resulting binary image will
be very bad |11|. In this case, a local thresholding method
perIorms better. The main problems with a local threshold
method are hard to set a right window size, eliminate the
block eIIect, and reduce the execution time. However,
memory restrictions and embedded system requirements
preclude the use oI binarization algorithms that require a
priori knowledge oI the Iull image and large execution time,
thus a number oI well-known locally adaptive algorithms
cannot be use. So it is diIIicult to binarize the bar code in
various illumination using one method.
An adaptive multi-level thresholding algorithm is
purposed, which integrate the local threshold with global
threshold. The algorithm can achieve higher recognition
rate under the condition oI lower illumination, contrast and
uneven illumination. This method including Iollowing
steps. Firstly, caculate the histogram oI gray image. In order
to decreasing the eIIect oI noise, Iilter the histogram, and
analysis the Ieature oI histogram peak. II the Iiltered
histogram is bimodal distribution, the lowest oI trough or
the middle value oI Ilat trough will be used as the global
threshold. In common lighting conditions, this global
threshold is used to process image with the satisIying result.
II the histogram oI image is single peak histogram, and
single peak area in lower gray area, it means that barcode
image is in weak illumination, otherwise the barcode image
is in strong illumination. We adopt iterative threshold
method, which Iorm the threshold with mean between old
global threshold and center oI dark area or light area. II
histogram shows multipeak distribution, it means that the
image is in the case oI uneven lighting conditions or
complex background. The local threshold algorthm is used.
The multi-level threshold method integrate global threshold
and local threshold method, it is meeted the real-time
binarization in common lighting condition, and also
satisIied the binarization in special illumination condition.
3.3 Filter
Standard opening and closing techniques are applied to the
204 2008 Chinese Control and Decision Conference (CCDC 2008)

bitmap to remove noise. AIter the Iilter, edge detection is
used in most recognition algorithm. But the QR Code has
its special Ieatures; three Iinder patterns will provide the
quick orientation. So the step oI edge detection is omitted in
our approach as shown in Fig.2. This will increase the
recognition rate.
3.4 Orientation
There are three identical position detection patterns located
at three oI the Iour corners oI QR Code, as shown in Fig. 1.
Three dark-light-dark squares are overlapped in every
Iinder pattern, and the dark-light ratio is 1:1:3:1:1. There is
the oII chance that similar graphic existed in barcode
symbol. So the approximately ratio area should be quickly
Iind. But when one oI the Iinder patterns is partially dirty or
damaged, we can use the timing patterns. The timing
patterns provide the secondary inIormation which can help
us to locate the symbol, decide the rotation direction oI
symbol and the width oI module. Rotated images are
handled aIter orientation in common, Ior example in ReI. |1|
and ReI. |4|. But the interpolation operation is used in
rotating operation. The amount oI calculation is great and is
not accurately. So we skip this step, as shown in Fig. 2. A,
modular distance oIIset algorithm was used without
rotating symbol. AIter located the Iinder patterns, the
module width and height and the angle oI rotation were
known. We get the grid moving along the line which
connects the central oI three Iinder patterns. This method
avoids the rotation and interpolation, increases the
computing speed.
3.5 Alignment Patterns Location
DiIIerent Irom PDF 417 barcode, in order to correct the
contorted QR Code symbol, there are many alignment
patterns in symbol. With the version increasing, the number
oI alignment patterns adding, as shown in Fig.3. When
version is 3, the number oI alignment patterns is 1, but
when version is 7, the number oI alignment patterns is 6.
Link the central point oI the alignment patterns and three
position detected patterns, the small sampling grid is
Iormed. In small sampling grid, distortion is omitted.
ThereIore, located the central coordinates oI alignment
pattern is critical Ior recognition barcode.

Fig. 3 The number oI alignment patterns
Using the known alignment patterns and detection patterns
can estimate the central coordinate. In International
standard oI QR Code, a reIerence method that locates the
central coordinates oI alignment patterns is provided. But
the precondition oI this method is estimated the central
coordinate just inside oI the alignment patterns. A large oI
actual captured image are tested, the probability oI
estimated central value inside oI the alignment patterns is
only 70. The other 30 image will be located Iailure
using this method. A new estimated method is provided
which used eight directions oI the estimated point. The
method scan the nearly eight directions oI the estimated
point to Iind the central coordinate oI alignment patterns
which are satisIied condition. This method expands the
scanning range, and 99.3 captured QR Code image is
successIully decoded.
3.6 Grid Generation
When the Iinder patterns and alignment patterns were
located successIully, the segment is easy to do, and the grid
is easy to generate. Then the corresponding pixels were
getting to ready decoding.
3.7 Error Correction and Decoding
The error correction and decoding process is the last step oI
recognition barcode. It employs the Reed-Solomon error
correction to enable accurate reads even when substantial
parts oI the code are distorted. Decoding is just the reverse
oI the encoding procedure and the decoding steps can
reIerence to international standard oI QR Code.
4 EXPERIMENT RESULTS
This application is implemented in our test hardware
architecture. An Intel XScale processor oI Motorola,
Motorola A1200r is run at up to 312MHz. In addition to the
processor core, its associated caches and memory
management units, including additional 512MB Micro-SD,
and camera interIace (twenty thousand pixel CMOS
sensors) and LCD controller etc.. The barcode image
processing algorithm is embedded in this mobile.
A sample set oI 100 unconstrained QR Code barcode image
were tested to veriIy the various binarization methods. All
original images are acquired by camera embedded in a
mobile phone. The images captured have a resolution oI
640*480 pixels. The experiments were done in real
conditions. In order to evaluate the validity oI our method,
these images were divided into Iour groups: the Iirst group
has normal lighting conditions; the second group has
highlighted spots and background noise; the third group has
low contrast or non-homogeneous lighting conditions; the
Iourth group has damaged barcode image.
All kinds oI illumination conditions were applied to QR
Code image. The multi-level binarization results were
compared with those obtained using other techniques:
Otsu`s algorithm; the reIerence algorithm oI QR Code
standard; Ohbuchi`s method; the original image is captured
by common lighting condition.
The experimental results are summarized in Table 1. In the
multi-level thresholding method, there are three kind oI
execution time corresponding to processing step. The
81.4 image can be binarized with once global
thresholding, 15.7 image need secondary thresholding
selection, and 0.028 image need locally thresholding
method. From Table 1, we Iound that the multi-level
thresholding method has satisIactory recognition rate and
execution time.
2008 Chinese Control and Decision Conference (CCDC 2008) 205

Table1. The Binarization Method Comparison
Binarization method
Recognition
rate
Execution time
ms
ReIerence algorithm oI QR
Code Standard
80 56
Otsu`s method 90 70
Ohbuchi`s method 85 45
Our method

99.8
5581.4
6015.7
750.028
In order to conIirm the eIIectiveness oI the proposed
recognition method, which including multi-level threshold
algorithm, determination oI the location and orientation oI
Iinder pattern, alignment patterns location Irom eight
directions, and decoding. We evaluated it with the whole
recognition perIormance. For the evaluation, we captured
more than 100 barcode image and run recognition
algorithm taken by our mobile. The recognition rate and
executing time were compared in Table 2. The
experimental results have shown that the barcode
recognition rate is higher (the Iirst group image), average
97, and the recognition time is litter in the normal lighting
conditions, about average 400ms. With the lighting
conditions became weak or uneven, the captured barcode
image can be recognized slower and the recognition rate
decreased. The Iour group is the damaged barcode image,
so the recognition rate is the slowest, decode a image will
cost average more 500ms. In a whole, the experiment
veriIied the proposed method is eIIicient, and higher
recognition rate was obtained.
Table2. Compare oI DiIIerent Group
Group
The average oI total
recognition time(ms)
Recognition rate
The Iirst group 400 97.5
The second group 430 95
The third group 470 91
The Iour group 540 87
5 CONCLUSION
With the mobile phone with camera device is getting more
popular, recognition barcode based on mobile phone is
getting more important and practical. We proposed a new
high-speed, high-accuracy automatic recognition method
Ior recognizing QR Code symbol in various illumination
conditions. In contrast to the algorithm proposed by
Ohbuchi, our method has no hardware speciIic
requirements and is able to run as a standard application on
most oI the mobile phones. From the experiment, the
proposed method produced better results than other
method. The recognition test also showed the proposed
method is eIIective Ior the QR Code image recognition
based on mobile phone.
. ACKNOWLEDGMENT
I would like to thanks proIessor Mingye Liu, Dr. Zhenhong
Shang Ior his guidance during the course oI this work in
ASIC Laboratories oI Beijing Institute oI Technology.
REFERENCES
|1| Tasos Falas, Hossein Kashani, 'Two-Dimensional Bar-code
Decoding with Camera-Equipped Mobile Phones, Proceedings of
the Fifth Annual IEEE International Conference on Pervasive
Computing and Communications Workshops (PerComW07), 19-23,
March, 597 600,2007.
|2| Gao, Jerry Zeyu, et al, 'Understanding 2D-BarCode Technology and
Applications in M-Commerce, The 31st Annual International
Computer Software and Applications Conference (COMPSAC
2007). Volume 2, July 24 27, 49 56, 2007.
|3| Eisaku Ohbuchi, Hiroshi Hanaizumi, and Lim Ah Hock, 'Barcode
Readers using the Camera Device in Mobile Phones, Proceedings
of the 2004 International Conference on Cvberworlds (CW04), 11,
260-265,2004.
|4| Robert Adelmann, ETH Zurich, 'Mobile Phone Based Interaction
with Everyday Products On the Go, The 2007 International
Conference on Next Generation Mobile Applications, Services and
Technologies (NGMAST 2007). 12-14, Sept., 63 69, 2007.
|5| Ying-Hong Liang, et al, 'Real Time Recognition oI 2D Bar Codes in
Complex Image Conditions, Proceedings of the Sixth International
Conference on Machine Learning and Cvbernetics, Hong Kong,
19-22, Aug., 1699-1704,2007.
|6| Yue Liu, Mingjun Liu, 'Automatic Recognition Algorithm oI Quick
Response Code Based on Embedded System, Proceedings of the
Sixth International Conference on Intelligent Svstems Design and
Applications (ISDA06), Volume 02, Nov., 783 - 788, 2006.
|7| Aidong Sun, Yan Sun, 'The QR-code reorganization in illegible
snapshots taken by mobile phones, Fifth International Conference
on Computational Science and Applications. 26-29, Aug., 532-538,
2007.
|8| ISO/IEC 18004:2000. InIormation technology-Automatic
identiIication and data capture techniques-Bar code Symbology-QR
Code, 2000.
|9| GB/T 18284-2000. National standard oI the People`s Republic oI
China: Quick Response Code (in Chinese), Issued by China State
Bureau oI Quality and Technical Supervision, 2000.
|10| P.K. Sahoo, S. Soltani, and A.K.C. Wong, 'A survey oI thresholding
techniques, Computer Vision, Graphics, and Image Processing, vol.
41, issue 2, Feb., 233-260, 1998.
|11| P. Wellner. Adaptive thresholding on the digitaldesk. EuroPARC
Technical Report EPC-93-110, 1993.
|12| E. Ottaviani, et al, 'A common image processing Iramework Ior
two-dimensional barcode reading, Seventh International
Conference on Image Processing and its Applications, Vol.2, 13-15
July, 2652-655, 1999.

206 2008 Chinese Control and Decision Conference (CCDC 2008)