A Vehicle Surveillance System for

Face Detection
Zhichao Ji , Huabiao Qin
College of Electronic and Information Engineering
South China University of Technology
Guangzhou, China
mealyee@gmail.com, eehbqin@scut.edu.cn

Abstract—Face detection is the process of locating the position
where faces are present in an image. Not all proposed face
detection methods are suitable for direct hardware
implementation. This paper proposes an improved skin-color
based face detect method and outlines its transition from a
software to hardware-based implementation. The hardware
implementation of the face detect algorithm is examined by the
FPGA (ALTERA CYCLONE EP1C6, Field Programmable Gate
Arrays).
Results show that it is robust to challenges of cluttered
backgrounds, partial occlusion and varying illuminations. Its
high detection and tracking performance makes it quite suitable
for real-time applications, especially for power-limited devices,
such as video surveillance systems for automated security,
environment, transportation and health-care monitoring
applications.
In this paper we propose a low cost FPGA system for face
detection and tracking. First, we present a high performance
face detection and tracking method, which being robust to
illumination and geometric variations, and entails a low
computational load. That makes it suitable for simple
microprocessor as well as custom hardware implementations.
We then present an FPGA realisation of this method. Our
proposed FPGA face detection and tracking system combines
very low logic and memory costs with high processing rates at
low clock speeds, making it ideal for small power-limited
devices of vehicle surveillance.

Keywords—vehicle surveillance,,facee detect, skin-color, FPGA

I. INTRODUCTION
Face detection is defined as the process of locating the
position where faces are present in an image. It is the first step
towards face recognition. Several methods have been proposed
thus far to achieve face detection (see Fig.1 [1]). They involve
techniques utilising knowledge-based methods (pattern
detection) [2], feature-invariant methods [3], face localization
methods [4] and colour segmentation. Many of these
techniques may not be suitable for direct hardware
implementation due to a high level of complexity. They may,
for example, require floatingpoint operations that would
necessitate the use of additional hardware and consequently
increase the cost of hardware implementation. Colour is a
powerful descriptor that has practical use in the extraction of Fig.1 Face detection methods
the face location, mainly because colour information is used to
segment regions rather than edges. Colour segmentation is a The rest of the paper is organized as follow: In section II
popular method used for face detection due to its low we describe the design methodology. In section III we will
computational requirements and its ease of implementation. introduce the the image processing algorithms and their
implementations. In section IV the whole system architecture
Currently, software-based face detection algorithms lack
is briefly discussed. Finally, section IV we make some
the computational ability to support real-time video streams[5]
conclusions .
. In recent years, FPGAs have proven to be invaluable in image
processing applications [6-8], because they combine the II. DESIGN METHODOLOGY
reprogrammability advantage of general purpose processors
with the parallel processing and speed advantages of custom One of the most important goals to be achieved in this work
hardware. was to get a system not dependant on any external PC
(Personal Computer). This design decision was made according
to the following performance criteria: 1) Cost reduction, as a
This work was sponsored by Key Science Project of Guangdong Province
key factor in the automobile industry. 2) High reliability 3)
NO.2006A10503002 and Fundamental Nature Science Program of
Guangdong Province NO.04020073 Portability and flexibility.

1-4244-1266-8/07/$25.00 ©2007 IEEE.

 The expertise of the design team (image processing [9] and We can see that the skin-color distribution in YCbCr space
hardware impelementation [10] ) has been crucial to reach the can be approximated by a Gaussian model. Namely, the
proposed goals in the short term. The proposed design cycle distribution of Cb and Cr is a planar Gauss distribution:
(depicted in fig.2 ) performs the following steps:
1) Problem definition and study of feasibility. Basic
requirements: cost and feasibility estimation. .
2) Functional modeling. To test the accuracy of the
algorithm using an ANSI C compiler.
3) DSP C modeling and test. The TMS320C6000TM DSP
Imaging Developer's Kit has been used in order to check the
image processing algorithms designed.
4) VHDL modeling and simulation, using MENTOR
GRAPHICS MODELSIM SE V6.1D. Fig.3 Skin color distribution in YcbCr space

5) FPGA implementation and test, using ALTERA

1 ( cb − µ 1 ) 2 (2.1])
QUARTUS II (Version. 5.0 ) and UP3 board of EP1C6Q240 f ( cb ) = exp[ − ]
2πσ 2σ 12
device. 1

1 (cb − µ 2 ) 2 (2.2)
6) System design. PCB implementation of the whole f (cr ) = exp[− ]
system. 2πσ 2 2σ 22

Where, µ1 the mean of Cb， µ 2 the mean of Cr

σ1 the variance of Cb, σ 2 the variance of Cr

There is a σ rule for Gauss distribution variable X:
p{µ − 3σ < X < µ + 3σ } = 0.9974 (2.3)
The face detection method is based on the above principle.
Differrent to some common skin color matching methods[10],
here we make some reasonable improvements to meet the
challenges of illumination and geometric variations, and
suitable for hardware impletation.
Our skin color based face detection method can be divided
in to three steps: 1) establishment of skin color module ; 2)skin
spatial filter; 3)face detecting/tracking.
A. Establishment of skin color module
The first step is the establishment of skin color module,
which is used to decide the classification of the pixel as skin or
non-skin in a video frame. Here, this detection relies on a
statistical histogram-based skin colour model. More
specifically, a statistical skin colour model has been created
offline, during a training phase, by projecting manually
segmented skin pixels onto a histogam. The training database
encompasses the skin tones of different people and of different
ethnicities, e.g. Caucasian, Asian, African, etc, captured under
various illumination conditions. During skin detection, an
unknown pixel is projected onto the table to retrieve a skin
colour distribution region as in fig.4.

Fig.2 Design flow methodology

III. FACE DETECTION IMPLEMENTATION

It is common to think that different people have different
skin-colors due to the existence of different races. However,
what really occurs is a larger difference in brightness, but not in
color. It is verified that the human skin-colors are clustered in
color space, and the chrominance values in the facial regions
are narrowly distributed [11], implying that skin colour is fairly
Fig.4 Skin color distribution in Ycb and YCr
uniform.as illustrated in Fig.3..
 After that we set up a skin color module using look-up table of W is smaller than ‘3’, we reset the skin pixel to non-skin ;
method, which will be easy implemented by FPGA .We stastic and a non-skin pixel to skin if the W values are bigger than ‘7’.
the value range of Cb, Cr for each value of Y during training. In this way, the noise could also be easily eliminate.
We use Ymin, Ymax to represent the minimum and maximum
intensity of the skin region, So Cb_min[i], Cb_max[i] will
represent the minmum and maximum value of Cb in array of
Cb_min[Ymin…Ymax] and Cb_max[Ymin…Ymax]. We
clould establish the same LUT of Cr in the same way. So the
skin color module could be present in such data form:
Cb _ min[ Y min ⋅ ⋅ ⋅ Y max] Cb _ max[ Y min ⋅ ⋅ ⋅ Y max] (2.4)
Cr _ min[ Y min ⋅ ⋅ ⋅ Y max] Cr _ max[ Y min ⋅ ⋅ ⋅ Y max]

The advantages of this skin-color module are not only

generally robust to geomebic transformations, such as scale,
orientation and viewpoint changes, since such transformations
do not affect the colour of skin, but also to complex
backgrounds and illumination variations.
In the YcbCr space, for each pixel, if its Y component is
between Ymin and Ymax, and the Cb,Cr components also in
range of this skin-color module, we will evaluate this pixel to
1, otherwise to 0, in this way we got a binary face image.
The architecture of this algorithm illustrated in fig.5. Here
we use a 256*32 read only memory(ROM) to store the skin
color LUT data, and the Y component as the ROM’s address Fig.6 Skin spatial filter architecture
input and Cb_min, Cb_max, Cr_min, Cr_max as the data
output, the output of ROM datas comparing to the Cb, Cr and C. Face detecting / tracking.
finally got the evaluate results. In the face tracking process, we do some some statistics for
each region. These statistics are the zeroth and first order
moments [11], given by

m00 = ∑∑ f ( x, y)
x y

m10 = ∑∑x ⋅ f (x, y) (2.6)

x y

m01 = ∑∑ y ⋅ f (x, y)
x y
Where f(x,y) is the skin map
1... for " skin " pixels
f ( x, y ) = { (2.7)
0" non " skin " pixels
Then, the centroid of each region is calculated as
m10 m
x= , y = 01 (2.8)
m00 m00
Fig.5 Color module architure
The tracking process is based on distance and mass
measurements. The distance between the centroid of each
B. Skin spatial filter
region and the centroid of the face of the previous frame is
From observation, it is not difficult to find skin color pixels calculated as the maximum axis distanced, using
are nor present one alone, and they always cramed into a
region. When evaluating a single pixel, we’d better take into d = max(dx, dy )
account of the pixels arroud, Which is more compatible with
the skin color pixels ditribution law. With dx = x − x prev , dy = y − y prev (2.9)
Here we use a 3*3 window filter to accumulate the
evaluating results which could be marked as W0, W1,…, W8,
Where ( x, y ) is the centroid of a skin region of the
then caculate the weighted average of W: current frame and ( x prev , y prev ) the centroid of the face of the
W=(2W0+W1+…+W8)/10 (2.5) previous frame
The architecture of this window filter illustrated in fig.6.. The skin group which produces the smallest distance d is
Here we use a column of shift registers to generate a dealy of a selected as the tracked face for the current frame. The ten
row pixel time, then put all the numbers into the Integral frame hysteresis function employed prevents flickering among
generator to calculate the number of value ‘1’. When the value skin regions of similar sizes.
 IV. SYSTEM ARCHITECTURE
The choice of hardware has been made as a step to meet the
constraints on cost and size of each unit mentioned below. The
whole constructrue of the vehicle surveillance system is shown
in fig.7.
There is one standard video camera for image input
(OV7620 of OminiVision). The current resolution of the
camera is VGA (640*480) and frame rate is 60HZ. A FIFO
memory block is generated by the IP core of Altera in the
SOPC builder for video image data buffering. In this way the
cost will be decreased comparing to an extra FIFO chip.
The NIOS II processor is the central processor of the whole
system. All the peripherals are connected to the Avalon Switch
Fabric. There are 16-Mbyte SDRAM and 8-Mbyte FLASH on
board as the main memory of the system. Our main Face detect
algorithm implemented as custom components including
establishment of skin color module, skin spatial filter amd face
tracking .
Fig.7 System structure
The user interface will connected to LCD, key board, and
voice chip(such as ISD25120). The LCD and keybord will be
used to initialize the parameters of the detect algorithm
components, and the vioce chip is used to make warning voice.
Table 1 shows the statistic for the implementation of
proposed face detect method on an ALTERA Cyclone
EP1C6Q240 device.
Table.1 Design summary for FPGA implementation
Devic ALTERA EP1C6Q240
Total pins 46/185 (25%)
Total logic elements 1,267/5,980 (21%)
Total registers 627
Total memory bits 24,576/92,120 (17%)
Total PLLs 1/2 (50%)
CLK frequence 60MHz
All architectures above are piplinedd and implement in the
FPGA device. All the algorithms used only about 1,267 LEs
and 2/20 RAM bolks. Most important, it can operate following
the camera data flow (VGA 640*480, and the frame rate is
60HZ) after a block buffering to meet the requirment of real-
time.
V. CONCLUSION AND DISCUSSION
A FPGA based face detection system of vehicle
surveillance is discussed in this paper. Here, we present a high
performance face detection and tracking method, then
discussed the FPGA realization of this method.
The face detection system has such advantages: 1) cost
reduction: FPGA solution is much cheaper than ASIC and DSP
solution; 2) portability: this system is independent of any
external PC except for the configuration time; 3) flexibility:
The adoption of an improved skin-colour based face detection
method achieves a very high performance and robustness both
to illumination variations and to geometric changes, such as
rotation, scale and viewpoint, since skin colour is invariant to
such transformations; 4)The algorithms implementation in
FPGA is completely reconfigurable. So a new superior
algorithm can been easily implemented according to the user's
specification. What's more, the algorithm modules can be
considered as special-purpose IP (Intellectual Property) which
would be reused by other applications.
REFERENCES
[1] Ming-Hsuan Yang and David J. Kriegman, “ Detecting Faces in
Images: A Survey ” , IEEE Trans. Pattern Analysis and Machine
Intelligence, Vol. 24, no. 1, pp 34-58, Jan 2002.
[2] G. Yang and T.S. Huang, “ Human Face Detection in Complex
Background”, Pattern Recognition, vol. 27, no. 1, pp. 53-63, 1994.
[3] K.C. Yow and R. Cipolla, “Feature-Based Human Face Detection”,
Image and Vision Computing, vol. 15, no. 9, pp.713-735, 1997.
[4] B. Takacs and H. Wechsler, “Face Location Using a Dynamic Model
of Retinal Feature Extraction”, Proc. First Int’l Workshop Automatic
Face and Gesture Recognition, pp.243-247, 1995.
[5] Theocharides, T.; Link, G.; Vijaykrishnan, N.; Irwin, M.J.; Wolf, W.,
"Embedded hardware face detection",Proceedings. 17th International
Conference on VLSI Design, 2004 pp. 133 - 138, 2004
[6] Swenson, R. L., Dimond, K. R., “A Hardware FPGA Implementation of
a ZD Median Filter Using a Novel Rank Adjustment Technique”, In
Proceedngs of 7th IEEE Intemationol Confermce on Imge Processing
and ils AppIicoIiom IPA99, 1999, vol.l, pp. 103-106.
[7] Turk, M., Pentland, A., “Eigenfaces for Recognition”, Journal of
Cognitive Neuroscience, 1991, vol. 3, no. 1, pp.71-86.
[8] Yang, M. H., Ahuja, N., Kriegman, D., “Face Detection Using a Mixhlre
of Linear Subspaces”, In Proceedings of 4th IEEE Infemalioml
Conterence on Aulomaric Face and GesIureReeogniIiaFGUU, 2000, pp.
70-76.
[9] Zhichao Tian; Huabiao Qin; Real-time driver's eye state detection.
Vehicular Electronics and Safety, 2005. IEEE International Conference
on 14-16 Oct. 2005 Page(s):285 – 289
[10] Fei Wang; Huabiao Qin; A FPGA Based Driver Drowsiness Decteting
System. Vehicular Electronics and Safety, 2005. IEEE International
Conference on 14-16 Oct. 2005 Page(s):358– 363
[11] Douglas Chai and King N. Ngan, “Face Segmentation Using Skin-
Color Map in Videophone Applications”, IEEE Trans. Circuits and
Systems for Video Technology, Vol. 9,no. 4, June 1999.