監控視訊 ch9

Face Detection
Definition
Face detection is concerned with finding whether or not
there are any faces in a given image (usually in gray scale)
and, if present, return the image location and content of
each face.
This is the first step of any fully automatic system that
analyzes the information contained in faces (e.g., identity,
gender, expression, age, race and pose).
Department of Electrical Engineering

MMLab
Find the faces in an image

MMLab
Face Recognition

MMLab
Face Recognition

MMLab
Face Detection
Skin color
Face structure
Eye
Lip
`
`
`
`

MMLab
Variation in illumination conditions may cause problems

Use illumination-invariant color models
Color space
`
`
`

MMLab
Find out human face features
`
`
`
Eyes
Lips
Geometrical relations

MMLab
Skin Detection
Skin detection is the process of finding skin-colored pixels
and regions in an image or a video.
This process is typically used as a preprocessing step to
find regions that potentially have human faces and limbs in
images
A skin detector typically transforms a given pixel into an
appropriate color space and then use a skin classifier to
label the pixel whether it is a skin or a non-skin pixel.
pixel. A skin classifier defines a decision boundary of the
skin color class in the color space based on a training
database of skin-colored pixels.
`
`

MMLab
Density plots of Asian skin in different color

spaces
10

MMLab
Density plots of Asian, African and Caucasian

skin in different color spaces
11

MMLab
Methods of Skin Detection

`
Pixel-Based Methods
`
`
Classify each pixel as skin or non-skin individually,

independently from its neighbors.
Color Based Methods fall in this category
Region Based Methods

`
12
Try to take the spatial arrangement of skin pixels into account

during the detection stage to enhance the methods
performance.
Additional knowledge in terms of texture etc are required

MMLab
Skin Color based methods - Advantages

Allows fast processing
Robust to geometric variations of the skin patterns
Robust under partial occlusion
Robust to resolution changes
Eliminate the need of cumbersome tracking devices or
artificially places color cues
Experience suggests that human skin has a
characteristic color, which is easily recognized by
humans.
`
`
`
`
`
`
13

MMLab
Issues with skin color

`
`
`
`
`
`
Are Skin and Non-skin colors seperable?

Illumination changes over time.
Skin tones vary dramatically within and across individuals.
Different cameras have different output for the identical image.
Movement of objects cause blurring of colours.
Ambient light, shadows change the apparent colour of the
image.
What colour space to be used?

How exactly the colour distribution has to be
modelled?
`
`
14

MMLab
How exactly the colour distribution has to be

modelled?
Non parametric Estimate skin color distribution
from the training data without deriving an explicit
model of the skin.
`
`
Look up table or Histogram Model

Bayes Classifier
Parametric Deriving a parametric model from the

training set
`
`
15
Gaussian Model

MMLab
What colour space to be used?

Different Color Models
RGB
Normalized RGB
HIS, HSV, HSL
`
`
`
`
Fleck HSV
TSL
YcrCb
Perceptually uniform colors
`
`
`
`
CIELAB, CIELUV
Others
`
`
16
YES,YUV,YIQ, CIE-xyz

MMLab
RGB Red, Green, Blue

`
`
`
Most common color space used to represent images.

Was developed with CRT as an additive color space
[1] Rehg and Jones have used this color space to study
the separability of the color space
17

MMLab
Normalized RGB rg space

`
2D color space as b component is redundant

`
b=1gr
Invariant to changes of surface orientation relatively to

the light source
18

MMLab
HSV, HSI, HSL (hue, saturation,

value/intensity/luminance)
`
`
`
`
`
19
High cost of conversion

Based on intuitive values
Invariant to highlight at white light sources
Pixel with large and small intensities are discarded as HS
becomes unstable.
Can be 2D by removing the illumination component

MMLab
Y Cr Cb
`
YCrCb is an encoded nonlinear RGB signal, commonly

used by European television studios and for image
compression work.
Y Luminance component, C Chorminance
20

MMLab
Perceptually uniform colors

`
`
`
skin color is not a physical property of an object, rather

a perceptual phenomenon and therefore a subjective
human concept.
Color representation similar to the color sensitivity of
human vision system should
Complex transformation functions from and to RGB
space, demanding far more computation than most other
colorspaces
21

MMLab
`
`
`
The skin colors form a separate cluster in the RGB color space. Hence
skin color can be used as a cue for skin detection in images and videos.
The performance of different color space may be dependent on the
method used to model the color for skin pixel.
For the common methods Look up table, bayes classifier, gaussian the
results are
` Look up table HS performs the best followed by normalized RGB
` Bayes is not largely affected by the the color space
` Gaussian No general result can be derived from the papers under
consideration
Removing the illumination component does increase the overlap
between skin and non skin pixels but a generalization of training data is
obtained
22

MMLab
Skin color and lip pixels detection

`
`
23
RGB and YCbCr color space

Particular chromatic distribution

MMLab
Eyes detection
`
`
`
24
Brightness factor
Darkest area
Histogram equalization

MMLab
Color Space
25

MMLab
Color Space
26

MMLab
Color Space
27

MMLab
28

MMLab
Luminance
Regulation
Contour
Extraction
Color
Segmentation
Geometric
Correction
Face
Information
Input Image
29

MMLab
Computes
Mean Faces
Dimension
Reduction
Face vectors
Feature
vectors
30

MMLab
Skin Color Detection

Distribution of skin-color pixels in normalized RGB color space
31

MMLab
Distribution of skin-color pixels in YCbCr

color space
32

MMLab
Distribution of skin-color pixels in normalized

YCbCr color space
0.0225x 2 + 6.1251x 290
0.0284 x 2 9.1477x + 836
33

MMLab
Skin Color Models

`
RGB, YCbCr , and HSV(HSI)
RCb=[77,127] and RCr=[133,173]
Original Image
34
Cb
Cr
Cb and C r

MMLab
R G B Variance
(255,255,82) = 6782.33
pseudo skin-color
35
R G B Variance
(255,223,247) = 506.66
skin-color
MMLab
Histogram of RGB variance for skin color

1200
2000
1800
1000
1600
1400
Pixel Number
Pixel Number
800
600
400
1200
1000
800
600
400
200
200
0
0
0
500
1000
1500
2000
2500
3000
500
1000
RGB Variance
2000
2500
3000
2500
1200
1000
2000
Pixel Number
800
Pixel Number
1500
RGB Variance
600
1500
1000
400
500
200
0
0
500
1000
1500
RGB Variance
36
2000
2500
3000
500
1000
1500
2000
2500
3000
RGB Variance

MMLab
Skin Color Detection

`
Using color information of Rcr =[133,173] as skin color

region
Denote the value of RGB as x , y (c0 , c1 , c2 )

2
1
Calculate the RGB variance Var ( x , y ) = (ci ci ) 2
3 i =0
255, Cr RCr Var ( x , y ) [th1, th 2]

Map skin color ( x, y ) =
otherwise
0,
37

MMLab
Original image
Cb and Cr
Cr
Cr and RGB variance

MMLab
(a)
(b)
(a) Original image
(b) Result of [14]
(c) Result of [11]
(d) Result of proposal
(c)
(d)

MMLab
Video Demo

MMLab

MMLab
Face Detection
Skin Color
Extraction
Input
Image
Face Localization
Color
Segmentation
Frame Diff.
Enhancement
Face Verification
Eye
Detection
Eye Pair
Validation
Motion Region
Segmentation
Output
Face Image
42

MMLab
Frame difference is the most simple way to find motion

areas
Two defects
`
`
43
The frame difference always appears on the edge areas

Frame difference becomes very weak when the object does
not move much

MMLab
Infinite Impulse Response type (IIR-type) filter is

applied to enhance the frame difference
`
Feedback loop
Ot (x, y) = It (x, y) + Ot1(x, y)

It(x,y)
Ot(x,y)
44
: the original t-th frame difference

: the t-th enhanced frame difference
: weight

MMLab
Frame Difference Image
45
Enhanced frame difference

Image
MMLab
Mean filter
`
`
Find the high possible motion

region and eliminate noise
1 1 1
M(x, y) = Ot (x + i, y + j)
9 i=1 j=1
Binarized by a predifined
threshold and denoted as O1(x,y)
Dilation
`
`
`
46
Extend the frame difference

region
Obtain a bitmap O2(x,y)
Structure element
of dilation operator

MMLab
Enhanced frame
difference
47
Mean filter (O1)
Dilation(O2)

MMLab
Motion scan procedure (cont.)

Vertical scan
48
1.
Find the top boundary and the bottom boundary of the

motion pixel in each column
2.
Each pixel between top boundary and bottom boundary is

set to be a motion pixel
3.
The residual pixels outside these two boundaries are set to

be non-motion pixel
4.
After all, a bitmap O3(x,y) is obtained

MMLab
Mean filter and Dilation (O2)
49
Vertical Scan (O3)

MMLab
Horizontal scan
`
Left-to-right scan
O3 (x, y) = 0, if (O2 (x, y) = 0 O3 (x 1, y) = 0)

`
Right-to-left scan
O3 (x, y) = 0, if (O2 (x, y) = 0 O3 (x +1, y) = 0)

`
50
If the pixel does not meet the criterion, the value of the
pixel in O3(x,y) will not be changed.

MMLab
Noise Elimination
`
Any group of less than six horizontally or vertical

connected motion pixels will be eliminated
Ensure a correct geometric shape of the motion region is

obtained
51

MMLab
Salesman # 8
Mean filter and Dilation
Horizontal Scan
52
Vertical Scan
Eliminate Noise
MMLab
Combining motion and skin color
Motion Region Image
Face Candidate
Skin Color Image
53

MMLab
Eye Detection
`
The analysis of the chrominance components indicated that

high Cb values are found around the eyes
(a) Salesman
54
(b) Claire
(c) Suzie

MMLab
Eye detection
`
Peak detector
`
P(x,y)={[Cb2(x,y)\g(x,y)]g(x,y)}Cb2(x,y)
`
g(x,y)
`
`
55
: Structural element
: Dilation operator
: Erosion operator

MMLab
Example
Cb2 image
Erosion
Dilation
Eye Candidate
56

MMLab
Elimination of error eye candidates

Area
Shape
Height to width
ratio
Density
Luminance
intensity
57

MMLab
Eye pair validation

Slope (45 )
Area ratio
Distance to area ratio
Face rectangle
`
`
`
`
58
Each eye pair candidate will

correspond to a face
rectangle
If the face rectangle is in face
candidate, it will be seemed
as a correct face rectangle

MMLab
Sampled eye image

13
Diff =
`
mask
19
Y(x, y) Y(x, y) + Y(x, y)

x=6 y=0
80
x=0 y=0
x=14 y=0
120
Using Diffup and Diffdown to eliminate the error eye pair

candidate
59

MMLab
Decision of overlap eye pairs
60
Eye Candidates
Face Rectangle
Sampled Face Rectangle
Result
MMLab
Decision of overlap eye pairs

`
Eye image (2010)

`
E : Edge score
`
The edge pixel number in the sampled image
S : Symmetry score
`
Small S mean high symmetry and high possible face region

9
S = (Yi, j Yi,19 j ) /(Ymax Ymin +1)

i=0 j =0
Yman: maximum luminance value

Ymin : minimum luminance value
61

MMLab
Face Score
`
Winner
`
The largest face score
Edge Pixels E
FaceScore =
=
Symmetry
S
62

MMLab
Results
(a) Suzie
(b) Claire
(c) Carphone
(d) Salesman
(e) Test 1
(f) Test 2
63

MMLab
64
100 frames
DR
FR
Suzie
Claire
Carphone
Salesman
Test 1
Test 2
Average
91.0%
86.0%
91.0%
86.0%
93.0%
80.0%
87.8%
4.2%
9.5%
5.2%
1.1%
5.1%
14.0%
6.6%
MMLab
65
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
MMLab
(a)
(b)
(c)
(d)
(e)
(f)
Average DR
Average FR
66
DR
97.2%
98.4%
90.4%
99.2%
96.8%
97.2%
FR
1.2%
0.4%
6.2%
0.8%
2.0%
1.2%
(g)
(h)
(i)
(j)
(k)
(l)
95.0%
1.9%
DR
91.6%
94.4%
95.6%
88.0%
94.0%
96.6%
FR
5.0%
1.3%
2.0%
3.9%
1.1%
0.2%

MMLab
Experimental results (with glasses)
(a)
(e)
DR
FR
67
(b)
(a)
90.4%
11.4%
(b)
93.9%
3.9%
(c)
(c)
94.4%
0.8%
(d)
(d)
96.8%
3.6%
(e)
96.0%
2.4%
Average
94.3%
4.6%

MMLab
Automatic zooming a face area
Face region
68
A Close-up picture

MMLab
Suzie
Salesman
Rotation
Speak
Hand
Hand
Skin Color
Background
69

MMLab
Raise up
Rotation
Tilt
Expression
Speak
Hand
70

MMLab
Expression
Tilt
Rotation
Glasses
Backward
71

MMLab
ROI Video Coding

Original Image
Face Detection
Face Region?
Foreground
Low-Pass Filter
Background
Combined Image
Video Codec
72

MMLab
Objective Comparison (Foreman, 96Kbps)
PSNR
PSNR of whole
frame (dB)
of ROI (dB)
Average BitRate (bits)
Skipped
frame
w/o ROI
29.99
29.68
3722
14
WA
32.92
31.23
3604
11
TF
33.35
30.46
3603
11
PR
33.91
30.16
3605
11
73

MMLab
PSNRROI by WR:
32.25 dB
74
PSNRROI by WA:
34.05 dB
PSNRROI by TF:
34.52 dB
PSNRROI by PR:
35.77 dB

MMLab
Face-based Region of Interesting (ROI) video coding

`
Encoded by H.263+ at bit rate of 64 kb/s
Detected face region

75
Without ROI
With ROI

MMLab
Without ROI
Fixed ROI
Face ROI
Carphone.qcif@64Kbps
76

MMLab
Multiple ROI
Without ROI
77
Multiple ROI

MMLab
Fixed and Face ROI
Without ROI
Fixed and Face ROI
Carphone.qcif@64Kbps
78

MMLab

監控視訊 ch9

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

監控視訊 ch9

Transféré par

Droits d'auteur :

Formats disponibles

Face Detection

Department of Electrical Engineering

Find the faces in an image

Department of Electrical Engineering

Department of Electrical Engineering

Department of Electrical Engineering

Department of Electrical Engineering

Variation in illumination conditions may cause problems

Department of Electrical Engineering

Find out human face features

Department of Electrical Engineering

Department of Electrical Engineering

Density plots of Asian skin in different color

Department of Electrical Engineering

Density plots of Asian, African and Caucasian

Department of Electrical Engineering

Methods of Skin Detection

Classify each pixel as skin or non-skin individually,

Region Based Methods

Try to take the spatial arrangement of skin pixels into account

Department of Electrical Engineering

Skin Color based methods - Advantages

Department of Electrical Engineering

Issues with skin color

Are Skin and Non-skin colors seperable?

What colour space to be used?

Department of Electrical Engineering

How exactly the colour distribution has to be

Look up table or Histogram Model

Parametric Deriving a parametric model from the

Department of Electrical Engineering

What colour space to be used?

Department of Electrical Engineering

RGB Red, Green, Blue

Most common color space used to represent images.

Department of Electrical Engineering

Normalized RGB rg space

2D color space as b component is redundant

Invariant to changes of surface orientation relatively to

Department of Electrical Engineering

HSV, HSI, HSL (hue, saturation,

High cost of conversion

Department of Electrical Engineering

YCrCb is an encoded nonlinear RGB signal, commonly

Department of Electrical Engineering

Perceptually uniform colors

skin color is not a physical property of an object, rather

Department of Electrical Engineering

Department of Electrical Engineering

Skin color and lip pixels detection

RGB and YCbCr color space

Department of Electrical Engineering

Department of Electrical Engineering

Department of Electrical Engineering

Department of Electrical Engineering

Department of Electrical Engineering

Department of Electrical Engineering

Department of Electrical Engineering

Department of Electrical Engineering

Skin Color Detection

Department of Electrical Engineering

Distribution of skin-color pixels in YCbCr

Department of Electrical Engineering