‘

Æ


‘

 

Automatic
ace etection is a complex problem in image processing. any methos exist to solve this
problem such as template matching, isher inear iscriminant, eural etworks,  , an 
.
uccess has been achieve with each metho to varying egrees an complexities.
‘
‘


igure c shows the
ace etection algorithm that we evelope.
‘
‘
‘
¬nput ¬mage
‘
‘
‘
‘
olor ase ask eneration
‘
‘
‘
‘

egion ining an eparation
‘
‘
‘
‘
aximal
ejection lassi
ier 
 Algorithm applie to otential aces
‘
‘
‘
‘
n rocessing uplicate
ejection an ener
ecognition
‘
‘
igure c lock iagram o
our ace etection Algorithm
‘
‘
he olor ase ask eneration will be iscusse in ection ,
egion ining an eparation in
ection , the 
 Algorithm in ection , an the n rocessing in ection .

‘ c
 ÿ
    
re woul like to reuce the number o
locations in the image that nee to be searche
or
aces. olor-
base mask generation picks out the pixels which are most likely to be
aces which can then be searche
using more avance techniques.
‘
he
irst step in our algorithm is to assign the probability o
being a
ace to every pixel in the image. re
begin by using the training set to etermine the istribution in
-space o
the
ace pixels an the
backgroun pixels. igure  shows the bouning regions
or
ace pixels an backgroun pixels base on
the training images. ote that the
 values range
rom  to  in igure to reuce the amount o

memory require.
‘
‘

igure  - space spanne by ace an on-
ace

 color vectors
‘
‘
‘
he simplest option
or
ace etection woul be to
in only those pixels which are containe in the
bouning region
or
ace pixels. owever, there is a noticeable overlap between the
ace pixels region an
the backgroun pixels region.
‘
‘
¬
we take transverse slices o
the  plot shown above, we get igure which shows the istribution o

ace
pixels re, backgroun pixels green, an where they overlap yellow. re woul like to assign a high
probability o
being a
ace pixel to pixels which resie in the re region, a meium probability to those
which resie in the yellow region, an a low probability to those which lie in the green/black regions.

‘ 
 igure  ransverse slices through - olor
 color space show in ig. 
‘
‘
‘
or each location in
 space, we use the
ollowing
ormula to calculate the associate probability o

being a
ace
‘
robability   o

ace pixels /  o

ace pixels   o
backgroun pixels  weight
‘
he weight is use to compensate
or the
act that there are many more backgroun pixels than
ace pixels
in the training images. ¬t also provies a simple way to bias the
unction towars
ining or rejecting
backgroun pixels. or example, a low weight will assign a higher probability to
ace pixels also in the
backgroun region, but will also cause pixels in the backgroun to have a higher probability. reight
values o
. to .8 were
oun to work quite well.
‘
igure  shows transverse slices through the resulting -imensional probability
unction. or every

value, we can assign a probability between  an c. his p
is then smoothe by a - box kernel in orer
to reuce sensitivity to speci
ic lighting conitions. ortunately, this p
can be compute once an then
just loae
rom a
ile.

‘ 
 igure  ransverse slices through - probability
unction
or a given speci
ic image
‘
‘
sing this probability
unction, we look up the probability value
or every pixel in the given image. igure
 shows the results o
this operation
‘

‘
igure  ¬mage showing the probability that a given pixel is a ace-ixel. rhite is
higher probability.

‘ 
 learly, the
aces are assigne high probabilities, while the backgroun is assigne meium to low
probabilities. re can now exploit the high spatial coherence o
the
aces, by convolving with ovals
approximately the same size o
a
ace. onvolving with an oval
ollowe by thresholing results in the
mask shown in igure .
‘
‘

igure  ask that results
rom
iltering probability image in igure  with an oval. he
result is then threshol to get the above mask.
‘
‘
‘
At this point, we have create a mask which reuces the number o
pixels which must be searche
or
aces
by at least an orer o
magnitue.

‘ 
 
 Æ      
.c
egion ining
nce we have a mask showing potential
aces, we nee to split this mask up into regions which can be
searche
or
aces. A
ter some very basic erosion an hole-
illing steps, most o
the
aces are nicely
containe in a single contiguous set o
pixels. hese sets can easily be
oun an labele. n
ortunately,
some contiguous regions contain more than one
ace. ¬eally, these regions coul be separate so that the
more avance algorithms nee only output a yes/no answer rather than counting the number o

aces in a
region.
‘
igure shows an example o
such a region which requires separation. rhile it may seem possible to
separate these regions by
urther erosion, over-erosion causes some o
the smaller or obstructe
aces to
isppear entirely. here
ore, we nee a better way o
separating connecte regions into single
ace-shape
regions.
‘

‘
igure  xample o
several connecte
aces in the mask image, even a
ter per
orming
erosion an ilation to separate.
‘
.
egion eparating
he
unamental problem is that the
aces come in such i

erent sizes. ome
aces are very large an
inclue necks, while other are obstructe or are very small.
‘
re use the
ollowing algorithm to separate the regions
‘
c. onvolve the mask with a large hea-an-neck shape template examples are shown in igure 8

‘ 
 . in the peak value resulting
rom the convolution, an subtract a ilate version o
the template
rom
the location o
the peak value.
.
epeat steps c an  until the peak value
alls below a certain threshol
.
epeat steps c to  with smaller an smaller hea-an-neck shape templates.
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
igure 8 hree i

erent sizes o
our !hea-an-neck´ shape template consist o
an
oval as the hea an a square as the neck.
‘
‘
his algorithm sequentially removes the hea shapes
rom the mask until there are none remaining. ¬t uses
templates o
many sizes an is there
ore insensitive to size i

erences in the heas, an because it removes
large heas
irst, it results in very
ew !ouble etections´.
‘
Ajusting the threshols ownwar results in more
alse-etections i.e. more hans, arms, etc«, but also
results in better etection o
obstructe
aces. ¬eally, algorithms later in the chain woul be capable o

rejecting the
alse-etections.
‘
igure " shows the results o
this region separation algorithm. ¬t etects both large an small heas, an in
this particular case results in a per
ect score, even be
ore oing any actual
ace etection.
‘

igure "
esults o
the
egion eparation Algorithm. ote
or this example that all

aces were properly segmente.

‘
 -
   
 !

he reaer is re
erre to #c$
or the etaile theory behin 
 an its application to
ace etection. he
lectures an slies given by ichael la in class #$ were also help
ul to our unerstaning o

. ¬n
this section we will escribe the basic etails o

 an our implementation o
it
or this project.
‘
.c heory-raining
re are given two sets, the non-
ace image set %&k} an the
ace image set %'k}. he non-
ace image set
is assume to be much larger than the
ace set. he images are assume to be gray-value images. he
images can be projecte onto a one-imensional line by a kernel (, which is easily
oun
rom the image set
statistics he projection o
the
ace set will occupy a certain range c an  on the line, while the
projection o
the non-
ace set will occupy another range hope
ully much larger. he goal o
this
projection is to minimize the number o
non-
ace images projecte onto the #c,$ range. his eliminates
as many non-
ace images as possible while keeping all the
ace images. he remaining images complete

ace set an non-
ace images that weren¶t rejecte are then use to
in another ( to project onto, where a
new #c,$ is
oun to reject as many non-
aces as possible. he iterations are repeate an more (¶s are

oun until practically all o
the non-
ace images have been eliminate.
‘
he calculation o
( is very simple an only requires
ining the mean x an y an covariance
matrices 
x an
y o
each image set

‘
he objective
unction can be written as
‘
‘ X  %# x   y $# x   y $ 
x 
y}X


%X }

X 
xX

‘
he goal is to maximize the above
unction, which has the e

ect o
maximizing the istance between the
s o
the
ace set an non-
ace set. aking the erivative o
the objective
unction an setting to zero
we
in
%# x   y $# x   y $ 
x 
y}X 
xX

‘
his equation is o
the
orm
X  )X , which is a generalize eigenvalue problem. ( is then the
eigenvector that correspons to the largest eigenvalue.
‘
. heory-etecting aces
nce an aequate number o
(¶s an their corresponing #c,$ ranges have been
oun 
or our algorithm
we acquire c (¶s, we apply the kernels to a test image to
in
aces. he basic iea is to go through the
image taking consecutive blocks o
a certain size same as all the ata set images. he
irst ( is applie to
the block, an we check i
the projection
alls within the speci
ie #c,$ range. ¬
it oes, then we apply
the secon ( an check i
the projection
alls within the corresponing #c,$. As long as the block keeps
‘ 8
satis
ying the #c,$ range
or each kernel, we apply the next (. ¬
we run through all the kernels an the
block satis
ies every one, the block is classi
ie as a
ace. ¬

or any ( the block
alls outsie the !
ace´ range,
the block is iscare an classi
ie as non-
ace. his is one
or every block in the image. he key here is
that we must work with multiple resolutions o
the image. ince the training ata is o
a certain size say
cxc, it is geare to
in
aces that are o
the same size. ¬n a given image, the
aces coul be o
all i

erent
sizes, so
aces will be
oun on i

erent resolutions o
the input image. la recommens working with
resolution ratios o
cc..
‘
. ¬mplementation etails
re chose a block size o
cxc
or our training ata as recommene in #c$. re gathere the
ace set

rom the seven provie training images. re selecte each
ace in atlab using a program we wrote
which ha the user click on the
ace in the training set; the program woul then cut the
ace o


rom just
above the eyebrows to just above the chin an resize the
ace to cxc. re assume that the
orehea an
chin area probably wouln¶t a much in
ormation to a
ace; most o
the etail is in the eyes, nose an
mouth. rom the training images we gathere c
aces. re then
lippe each one across the vertical axis
to generate c more
aces
or a total o
8
aces. ¬n orer to represent a larger range o

aces, we
per
orme a proceure calle regularization, aing a value o
r  ¬ with r   to the
ace covariance
matrix #$.
‘
re gathere our non-
ace images
rom raining ¬mage  maske with the groun truth ata to get ri o

the
aces. re wrote a program that went through the image an took successive blocks as non-
ace
images. re then went to a lower resolution an repeate the same process. re actually starte with an
image that was one-thir resolution o
the original an then went through
ive aitional lower resolution
versions to generate our ata set. he reason we starte with a lower resolution image is that
aces in the
training images were all at least x pixels, so when
ining
aces we probably wouln¶t start looking
or

aces until we own-sample the input image by at least three since our
ace images are only o
size
cxc. hus we only care about non-
ace images that are on the same scale. re were able to generate
c8,"88 non-
ace images. ¬eally we woul want our non-
ace image set to be as large as possible to cover
every possible case o
a non-
ace. owever, generating these non-
ace sets took a long time in atlab, an
an even bigger problem was storing the images. ¬t wasn¶t practical to generate a matrix to store every
image, so we kept track o
blocks by their positions in the training image.
‘
here were a couple o
very help
ul pre-processing proceures that we per
orme on the input blocks
be
ore classi
ying them as
ace/non-
ace. he
irst was removal o
the mean
rom the block. his
processing helpe to eal with i

erent skin colors an lighting on
aces. he secon proceure was
masking out some o
the pixels in the input block, namely the bottom le
t an right corners. his remove
some o
the backgroun pixels that coul be
oun in the
ace block. ¬t turns out that these proceures can
be applie to the kernel itsel
instea o
each block, thereby saving innumerable calculations.
‘
. Avantages an isavantages
ne o
the avantages o

 is that it uses simple an there
ore
ast classi
iers. he classi
iers are
weak so we nee to use many o
them, but this isn¶t a problem because each runs through the algorithm
quickly. 
 is simple to implement once the classi
iers an threshols have been acquire. ¬t is also
very accurate.
‘
ne problem with 
 is that i
given a very large input image, traversing it with cxc blocks at several
resolutions coul take a long amount o
time. Another logistical problem lies in combining the results
rom
multiple resolutions. ¬
you
in a
ace on one resolution, then
in it again on another resolution, how to
know i
you
oun a new
ace or i
it¶s the same one? his problem is actually not i

icult to solve. he
biggest problem with 
 is that it obviously will have some
alse etections. owever, when we
combine 
 with the color segmentation/region
ining algorithm we were able to severely cut own
on these
alse etections
‘
. ¬ntegration o

 an olor egmentation/
egion ining
re use color segmentation/region
ining 
rom now on re
erre to as just color segmentation to segment
the input image into i

erent blocks; the color segmentation algorithm !claims´ each selecte block as

‘ "
 having one
ace. re then take each o
these selecte blocks an per
orm 
 to etermine i
there really
is a
ace in the block or maybe it¶s just a
lesh-colore orange. he 
 works with the block in
ive
i

erent resolutions to
in a
ace; once it
ins a
ace in the block at a certain resolution, it stops an labels
the block as a
ace. ¬t then moves on to the next block selecte by color segmentation an oes the same
thing until all the selecte blocks have been processe.
‘
he bene
its o
integrating 
 an color segmentation are numerous. oing color segmentation then

 saves time because the 
 algorithm no longer has to go through every successive block in the
input image. ¬nstea it only has to go through the  or so blocks assuming -
aces in image
ienti
ie by the color segmentation. Another avantage is that 
 can get ri o

alse etections
rom
the color segmenting. A
lesh colore shirt that is segmente looks nothing like a
ace to 
 an is
eliminate. re on¶t have to worry about 

alse etections since the color segmenting gets ri o
so
much o
the input image, ecreasing the chances o
a
alse etection that woul have occurre i

 ha
to traverse the entire image. &et another bene
it is that now we on¶t have to worry about
ining the
same
ace at multiple resolutions. re know that there¶s either one
ace or no
aces in the selecte block.
o once we
in a
ace, there¶s no nee to continue processing at lower resolutions, so we can continue
with the next color-segmente block. he input blocks into the 
 algorithm never containe more than
one
ace
or the test images we ran, so we on¶t have to worry about multiple
aces per block.
he only real problem with this integration is that sometimes there really is a
ace in the block that color
segmentation ienti
ies, but 

ails to
in it. his o
ten occurs i
the
ace is partially obstructe or
severely rotate. owever, we were willing to not
in these rare cases in orer to avoi the
alse
etections that 
 alone or color segmentation alone woul have etecte.
‘
igure c below shows the input color-segmente an region separate images that are input to the 

algorithm. igure cc shows the output o
the 
 algorithm. he missing images at locations ,,
,, an , are not present in the output are those that 
 i  classi
y those as
aces. his was
a per
ect etection image.
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
igure c xample o

ace caniates passe
rom
egion eparating Algorithm to
the 
 algorithm. ote the non-
aces at positions ,, ,, an ,

‘ c
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
igure cc utput o
our 
 Algorithm. ote the non-
aces
rom igure c have
been remove, but nothing else has been. his is per
ect etection.
‘
‘
‘
D
"  #

A
ter the 
 algorithm makes the
inal
ace selection, we per
orm two more steps in our algorithm.
‘
irst, we take the results o
the 
 an i
two
aces are too close together, we eliminate the secon
ace.
re
oun that the previous steps o
our algorithm can sometimes ouble count certain
aces, an that this
technique success
ully eliminate those extra
aces.
‘
econ, we tag each
ace that the 
 algorithm
ins with its average alue 
rom  color space.
¬nstea o
oing gener recognition, we only search
or one speci
ic
emale in the set o
etecte
aces.
he
ace with the lowest alue
rom  space is labele as a
emale. ne o
the three
emales in the
class has much arker skin than anyone else, so she usually has the lowest average alue in  color
space, epening on lighting conitions.
‘
‘
†
     
e
ore settling on a
inal algorithm we also consiere template matching as a simpler approach to the

 algorithm. re create a template by averaging together all the
aces across all seven training
images. rior to averaging the
aces together we normalize the istance between the eyes so that each

ace was approximately the same size an
eatures such as the eyes, nose, an mouth line up. ur
template can be seen in igure c below.

‘ cc
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
‘
igure c emplate mae
rom averaging all
aces in test images
‘
‘
‘
‘
sing a single template to try to match
aces
rom the output o
the region
ining an separation step
actually gave lower scores than oing nothing. o improve upon this we scale the template to several
i

erent resolutions, which gave much better results than be
ore. owever, there were still several
alse
positives an negatives in each image. ther improvements that we mae inclue constraining the peak
o
the correlation to be near the center o
the input image. Also, i
template matching prouce results that
in¶t have a well-e
ine peak but instea just a large high-value area, we iscare those results too.
igh-value correlations without a well-e
ine central peak were typically ue to arms, necks, or other
non-
aces.
‘
‘
‘
‘
A
!

‘
#c$ ichael la, &acov el-r,
enato *eshet, !
ejection ase lassi
ier
or ace etection´, ewlett
ackar aboratories, ¬srael, attern
ecognition etters, April c.
#$ ichael la, 8 ecture, c" ay  an c ay .
#$ reeric arrat, !
ejection-base inear lassi
ier´, pring   8 roject
eport.
‘
ther
onzalez an roos, igital ¬mage rocessing, rentice all, ew +ersey, .

‘ c