10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010)

MULTIMODAL BIOMETRIC SYSTEM USING FACE, EAR AND GAIT

BIOMETRICS
i
^!OurYuzdunpunuh

IurmuOz
1
usOu!^mr!uUuh
Member of Young Researchers Club Department of Electrical Engineering
Islamic Azad University ofNajaf Abad Amirkabir University of Technology
Digital Signal Processing Research Lab
Department of Electrical Engineering
Isfahan University of Technology
Isfahan, Iran. Fattahi@cc.iut.ac.ir
Isfahan, Iran Tehran, Iran
ali.p@iaun.ac.ir kfaez@aut.ac.ir
ABSTRACT
In this paper, a novel multimodal biometric recognition
system using three modalities including face, ear and
gait, based on Gabor+ PCA feature etraction method
with fusion at matching score level is proposed The
performance of our aroach has been studied under
three diferent normalization methods (min-ma, median
MD and z-score) and two diferent fusion methods
(weighted sum and weighted product. Our new method
has been successfully tested using 360 images
corresponding to 120 subjects fom three databases
including ORL face database, |51 ear database, and
CAS gait database. Because of these biometric traits,
our proposed method requires no signicant user co
operation and also can work fom a long distance.
According to the experimental results our proposed
method exhibits excellent recognition performance and
outperforms unimodal systems. The best recognition
performance that our proposed method achieved is
%97.5.
Index Terms Multimodal Biometrics, Face Biometric, Ear
Biometric, Gait Biometric, Gabor+PCA.
. INTRODUCTION
Nowadays multimodal biometrics is getting more and
more concer and becoming a hot research topic in
biometric area. Personal identity authentication system
based on any unimodal biometric may not be acceptable
to a particular user group or in a particular situation. So
the multimodal biometrics technology is being developed
to alleviate the problems intrinsic to unimodal biometric
systems [1]. Multimodal biometric systems are expected
to be more reliable due to the presence of multiple pieces
of evidence and also able to meet the stringent
performance requirements imposed by various
applications [2]. Multimodal systems address the problem
of non-universality and also provide anti-spoofng
measures by making it difcult for intruders to spoof
multiple biometric traits simultaneously. By asking the
user to present a random subset of biometric traits, the
system ensures that a live-user is indeed present at the
point of acquisition. However, an integration scheme is
required to fse the information presented by the
978-1-4244-7167-6/101$26.00 2010 IEEE 251
individual modalities. In additional, multimodal biometric
system takes more than one single feature into account
[3,4]. This helps in identifing and verifing the person
with more accuracy even if one of the features gives less
matching scores [5]. In a multimodal biometric system
that uses different biometric modalities, different fsion
levels are possible: fsion at feature extraction level,
fsion at matching score level or fsion at decision level.
Fusion at feature extraction level is diffcult because
feature vectors which are used by different biometric
traits may be inaccessible or incompatible. Also it is
diffcult to integrate the information at decision level,
because at this level, only restricted value of information
is available. Hence, due to easily accessing and easily
integrating, fsion at matching score level are generally
prefered. In this paper, multimodal biometric
identifcation system which is based on features extracted
fom three biometric modalities including face, ear and
gait, using Gabor+PCA method is proposed. The main
reason of using these three biometric traits is that, our
system requires no signifcant user co-operation and also
because of the gait biometric feature, our system can
work fom a long distance. In this method, the fsion of
the unimodal systems into a multimodal one has been
carried out at a matching score level and the decision of
the multi modal system will come fom the fsion of the
different unimodal scores. We applied our proposed
method on the face images fom ORL face database [6]
and ear images fom USTB ear database [7] and gait
silhouette images fom CASIA Gait Database [8].
The rest of the paper is organized as follows: section 2
describes a Gabor+PCA feature extraction method. Our
proposed multimodal biometric system is introduced in
Section 3. Experimental results are reported in Section 4
and conclusions are drawn in Section 5.
2. GABOR+PCA FEATURE EXTRACTION
In this section, for extracting efcient feature fom each
modality, we propose Gabor+PCA method as an effcient
feature extraction method for increasing multimodal
recogmtIOn performance. In this feature extraction
method, we frst extract Gabor wavelet representation
fom each modality and then we apply principle
component analysis (PCA) on them. The feature which is
extracted by this method can enhance the recognition
performance.
2. . Extract Gabor Features
The Gabor wavelets, whose kerels are similar to the 2-D
receptive feld profles of the mammalian cortical simple
cells, exhibit desirable characteristics of spatial locality
and orientation selectivity. They are localized in the space
and fequency domains optimally. Gabor wavelets
(kerels, flters) [9] are defned as
( )
_ ll
_
l" f

,,, ,,
.i-
,
_
-.i
]
(1)
Z - 2

(
Where 1 and v are the orientation and scale of Gabor
kerels and
Z
=
(
x,
y)
. 11.11
denotes the norm operator and
q is the standard deviation of the Gaussian envelope
determining the number of oscillations. Wave vector
_
/

is defned as
(2)
Where ] =] fvandn =]
v
max f
[
__
IS the
maximum fequency and f is the spacing factor between
the kerels in fequency domain. Each kerel is a product
of an elliptical Gaussian envelope and a complex plain.
The Gabor wavelets mostly use in fve different scales
VE {O, ... ,4} and eight different orientation = {0, ... ,7}
[10]. Figure 1, shows the Gabor kerels real part in fve
different scales and eight different orientations. If we use
these parameters: q=27,
[
__
=X],and f =. ,
kerels show desirable properties like orientation
selectivity, spatial locality and spatial fequency.
Fig. I. Real part of Gabor Kerels at eight orientations and fve scales.
The Gabor wavelet representation of an image is the
convolution of the image with each of Gabor kerels as
defned by (1). Let I(x,y) be the gray level distribution
of a biometric image. The convolution of image f and a
Gabor kerel (p,v ) is defned as.
GI,V(Z)
=
1(z) * CI,v(z) (
3
)
Where * shows the convolution operator and
Z
=
(
x,
y)
and GI,)z) are the convolution of Gabor
252
kerels at orientation 1 and scale v with image I. Thus,
the set N={GI
,
v(z)
: 1 E {O, ... , 7}, VE {O, ... ,4} include 40
different Gabor wavelet representations of image 1(z)
Each of these forty representations can use as our Gabor
wavelet representation of the image.
2.2. Principle Component Analysis
P
C
A is a technique used to reduce multidimensional data
sets to lower dimensions for analysis and also involves
the computation of the eigenvalue decomposition or
Singular value decomposition of a data set, usually afer
mean centering the data for each attribute. Only the N
eigenvectors associated with the largest eigenvalues are
used to defne the subspace, where N is the desired
subspace dimensionality. In our experimental results
N=20. We use eigenvectors for classifcation. Matching
scores achieve by computing the normalized Euclidean
distance between the eigenvectors of templates and the
input image. The normalized Euclidean distance between
two feature vectors Xl and x
2
is defned as
Where N is the dimensionality of the feature vectors.
Figure 2, shows a scheme of our overall feature
extraction method.
l(:) G [,)
P
C
A
I
Extracted eaure
- Gabor 1 p,v .1 __- ---- .
Fig. 2. A scheme of our overall feature extraction method
J. MUL TIMODAL BIOMETRIC SYSTEM
Biometric systems have four important levels including:
sensor level, feature extraction level, matching score level
and decision level. Fusion in multimodal biometric
systems can happen at the last three levels. In our
proposed method, information fsion happens at
matching score level. By fsion at this level, each system
provides a matching score indicating the proximity of the
feature vector with the template vector and these scores
can be combined to recognize the claimed identity. But
before the fusion step, in order to combine the matching
scores, we should frst normalize these scores.
Normalization includes mapping the scores obtained fom
each modality into a common domain. This step called
normalization step. In our proposed method we introduce
thee different normalization and two different fusion
methods.
J. . Normalization Methods
Good normalization method should be robust and
effcient. In [11], Huber describes the notion of
robustness and effciency of statistical process.
Robustness relates to insensitivity to the existence of
outliers and effciency relates to proximity of the
achieved estimate to the optimal estimate. But for
efciency parameter we should already know the
distribution of data. We have compared three different
kinds of normalization techniques in our experimental
studies.
Min-max is the frst normalization technique in our
experimental studies. If the maximum and minimum
values, which generated by matchers, are known, this
method can work very well. Before applying min-max
method, we should fnd the minimum and maximum
values of the data set. Min-max method transform scores
into a common range [0,1]. Also this method is not
robust; it means that it is sensitive to the existence of
outliers. The normalized scores are given by
S;
= Sf !
(
5
)
!aX!m
Where Sf are the matching scores and S; are the
normalized scores.
The second method used to normalize the matching
scores is median-MAD (median absolute deviation)
method. This normalization scheme is a robust method
and has an average efciency but it does not keep the
input distribution and it does not transform the scores into
a common numerical range. The normalized scores by
this method are given by
S
f = St mcd|un (
6
)
t
J1
Where MAD=median St mcd|unl )
The third normalization technique is z-score. This method
computed using the arithmetic mean and standard
deviation of the data set. Due to sensitivity of both mean
and standard deviation to outliers, this method is not
robust and it may not transform the scores of our three
modalities into a common numerical range, but it has
high efciency. The normalized scores are given by
S;
= Sf
P
(7)
(
Where ( is the standard deviation and
P
is the mean of
the data set.
J.2. Fusion Methods
In order to combine the scores reported by the three
matchers we used two different techniques including
weighted sum method and weighted product method.
A. Weighted Sum
In order to combine the input scores to achieve a fnal
score, one of the simplest fusion methods is weighted
sum method. This method has low computational cost
and decision in this method is computed by comparing
these fnal scores with a threshold.
B. Weighted Product
Weighted product is another algorithm that combines the
scores using weighted multiplication of each modality
253
scores to obtain a fnal score. Decision in this method is
the same as the previous method that computed by
comparing these fnal scores with a threshold. But the
computational cost of this method is more than the
previous method.
Fig. 3 shows the block-diagram of the proposed
multimodal biometric system based on the fusion of face,
ear and gait features at the matching score level.
Fig. 3. Block-diagram of the multimodal biometric system based on
maching score fsion
4. EXPERIMENTAL RESULTS
4. . Databases
In this method three different databases for three
modalities (face, ear and gait) have been used. For face
images, we have used OR face database [6] including
120 images corresponding to 40 subjects with three
images per person which were taken at different times
and under different lightning condition and different
facial expression (FigA).
Fig. 4. Sample images for one person fom ORL face database [6].
For ear images we have used USTB ear database [7].
From this database, we have used 120 images
corresponding to 40 subjects with thee images per
person which were acquired under standard condition
with a little change (Fig.5).
Fig. 5. Saple images for one person fom USTB ear database [7].
For gait silhouette images we have used CASIA gait
database [8]. From this database 120 average silhouette
images over one gait cycle are extracted fom 120 video
sequences corresponding to 40 subjects with three
average silhouette images per person (Fig.6).
Fig. 6. Sample average silhouette images for one person from CASIA
gait database [8].
4.2. Experiments
Multimodal database in our experiments consist of three
separate databases (Face, Ear and Gait). We have 40
users, for each user we have face, ear and gait images
(three images for each modality). This data was used to
construct 120 (40x3) genuine scores and 4680
(40x3x39) imposter scores for each modality. Each
subject in face database was randomly paired with one
subject in ear database and then each of these pairs was
randomly paired with one subject in gait database. Due to
the mutual independence assumption of the biometric
traits, we can randomly pair the users fom the two sets.
The recognition performance of a biometric system can
be measured by reporting its false acceptance rate (FAR)
and genuine acceptance rate (GAR) at various thresholds.
Table I shows the recognition performance of three
unimodal systems (face, ear and gait systems) with three
different Gabor Kerel Parameters (1 =orientation,
v=scale) at a FAR of 0.1%. The best performance for
unimodal systems obtained when we use Gabor feature
with these parameters: =7, +=4. We can see that the
performance of these unimodal systems are low. The
performance of our multimodal system has been studied
under different normalization and fsion methods at a
FAR of 0. 1 % (table 2). In table 2, in order to achieve the
best recognition performance, we use these Gabor kerel
parameters: =7, +=4. We used thee different
normalization techniques (min-max, median-MAD and z
score) and applied two fusion methods (weighted sum
and weighted product). From table 2 we observe that z
score normalization method outperform other
normalization technique in our experimental studies.
Table 1: Geniune acceptance rate (GAR) of three unimodal
system under different Gabor kernel parameters at the 0.1 %
false acceptance rate (FAR).
Unimodal =7,+=4 =6, + =4
=5,+=4
System
Face 65% 57.5% 52.5%
Ear 82.5% 80% 77.5%
Gait 72.5% 70% 65%
Table 2 also shows that weighted product method provide
better performance than weighted sum method. Hence,
we can conclude that when the scores normalized using z
score method and combined using weighted product
254
method, the best recognition performance (97.5%) are
achieved.
Table 2: Geniune acceptance rate (GAR) of our multimodal
system under diferent normalization and diferent fusion
h d h 0 1Of1 (FAR) met 0 s at t e 0 a se acceptance rate
Normalization Fusion Methods
Methods Weighted Weighted
sum Product
Min-max 92.50% 94.16%
Median-MAD 89.16% 90.83%
z-score 95.83% 97.5%
5. CONCLUSIONS
In this paper, a new multimodal biometric recognition
system using three modalities including face, ear and gait,
based on Gabor+PCA feature extraction method with
fsion at matching score level is proposed. Also the effect
of different fusion method and different score
normalization method on the recognition performance of
our multimodal biometric system are studied in this
paper. We show that our approach also exhibits excellent
recognition performance and outperforms unimodal
systems on a variety of image databases including face,
ear and gait images
REFERENCES
[1] Arun Ross, Ani! Jain, "Multimodal Biometrics: An
Overview," Proc. of 12th European Processing Conerence, pp.
1221-1224, 2004.
[2] Zuev, Y., Ivanon, S., "The voting as a way to increase the
decision reliability. In: Foundations of Information/Decision
Fusion with Applications to Engineering Problems,"
Washington, DC, USA. pp. 206-210. 1996.
[3] Jain, A.K., Hong, L., Pankanti, S,"Can multibiometrics
improve performance?" In Proc. AutoiD

99, Summit, NJ, USA.

pp. 59-64. 1999.
[4] Jain, A.K., Hong, L., Kulkari, Y "A multimodal
biometric system using fngerprint, face and speech." In Second
Internat. Con on AVBPA, Washington, DC, USA. pp. 182
187. 1999.
[5] L. Xu, A. Kryzak and C.Y. Suen, "Methods of Combining
Multiple Classifers and Their Application to Handwriting
Recognition," IEEE Trans. on Systems, Man and Cybernetics,
vol. 22, no. 3, pp. 418-435, May-June 1992.
[6] F. Samaria and A. Harter, "Parameterisation of a
stochastic model for human face identifcation," in Proc. of the
2nd IEEE Workhop on app. of com. vision, USA, 1994.
[7j Li Yuan, Zhichun Mu, Zhengguang Xu, "Using Ear
Biometrics for Personal Recognition," International Workhop
on Biometric Recognition Systems, IWBRS 2005, pp. 221-228,
2005
[8] Chinese Academy of Sciences, Institute of Automation
Gait Database: http://www.sinobiometrics.com/
[9] Ch. Liu and H. Wechsler, "Gabor Feature Based
Classifcation Using the Enhanced Fisher Linear Discriminant
Model for Face Recognition," IEEE Trans. on Image
Processing, vol. II, no. 4, pp. 467-476, April 2002.
[10] 1. Jones and L. Palmer, "An evaluation of the two
dimensional Gabor flter model of simple receptive felds in cat
striate corex," J Neurophys., pp. 1233

1258, 1987.
[II] P.J. Huber, Robust Statistics, Wiley, New York, 1981.