A Method of Driving Fatigue Detection Based on Eye Location

Liling Li, Mei Xie, Huazhi Dong College of Electronic Engineering University of Electronic Science and Technology of China Chengdu, China E-mail: 471320791@qq.com

Abstract-For a driver fatigue monitoring system, one of the most important problems to solve is eye location. This paper presents a method of eye location based on AAM and a method of driver fatigue detection based on calculate an improved method to Appearance Model)

PERCOLS

AAM

(Active

model is composited by the appearance model and fitting algorithm which is an efficient algorithm to extract facial feature points,. First establish the AAM model, position of eye, next calculate the then use the physical fitting algorithm to detect the position of face and locate the

PERCLOS which is used to measure the driver fatigue to

determine the current state of driver is fatigue or not.

improved

AAM; PERCLOS

Keywords- Drowsy driving; fatigue detection; eye location;

of the shape control points which used to characterize the position of feature points; then define a energy function with the square of the picture difference between the model example and the input image, energy function tis used to evaluate the fitting degree; fitting in the localization process, according to the model of linear expression, through an effective change model fitting algorithm parameter set to control the shape change of position of control points generated the current model instance , get the current energy function value, and then update the model parameters, This iteration repeated in order to achieve minimization of energy function, to fit the model instance with the purpose of the input image, and ultimately obtained the position of the shape control points to describe the characteristics of the current object position. A. Statistical shape model

I.

INTRODUCTION

One of the main factors that cause traffic accedients is drowsy driving.To research a valid driver fatigue detection and monitoring system which could give an alarm in time, for reducing traffic accidents and the protection of state property has very important significance. In early stage the contact detection system is not practical because it is inconvenient to use, now the non-contact detection system repalce it. Non-contact inspection system through the camera to get the driver's face and operational behavior to determine whether he or she is in a state of fatigue driving, then make treatment appropriate. Among them, by acquiring the driver's face video to detect the state of human eye is widely used as a non-contact detection method[I-3]. When detect the human eye's state to judge the state of driver fatigue, the detection of eye location and status trequires accuracy and real-time. In this paper, through establish AAM (active appearance model)[4] appearance model and baced on face detection to locate the location of the eye accurately. And the traditional method has been improved then calculate the physical of discriminant driver fatigue to determine the current state is in the drowsy driving or not. II. AAM APPERANCE MODEL

Shape is preserved the most essential geometric information about the target objects after removing the effetion of translation, scaling and rotation transformation [5].AAM is calculated by a set of training images manually calibrated, assuming the image has n landmark points, face shape can be a vector as follows: S =

(XpYI, X2'Y2,,Xn,Yn)T

(1)

Which (x,y) are the coordinates of landmark points, in order to eliminate rotation, translation and scaling effects, use the Procrustes[6] method to shape aligned to the uniform vector coordinate system, then use the principal component analysis of aligned shape reduce the dimension of vector, any shape model can be expressed as follows:
S = S 0

i= 1

iS i s;

(2) are the eigenvectors

Which So is the average shape, which are corresponding to shape parameters.

B.

k largest enginvalues, hi are the

Statistical texture model

AAM model is a dynamic appearance model of object, the apperance model is established baced on comblination the shape model and the texture model of object. Dynamics is embodied in the AAM fitting calculation, using PCA(Principal component Analysis) to describe the motion

In order to get a better description, we need to add texture (the gray face value of a certain region or ROB value) information. First, the shape of the standardization sample images that distort the image of each sample, then match the samples shape and the average shape face
So

to eliminate

978-1-61284-486-2/111$26.00 2011 IEEE

180

differences in texture caused by differences in shape, and then sampling the average normalized shape image

So where

K(N(W(p);q))) ,even get the smallest difference

between the two:

area are covered by the texture image, so that the texture image can be obtained. Form this we can see texture is defined in the average shape pixels p within the shape image

So that composed by all the so' in order to eliminate

Lgo(P) + Lgi(P) - J(N(W(P);q)) i=1

III.

(6)

illumination factors, changing the texture image use the method PCA, and arbitrary normalized texture image can be obtained as follows:

PERCLOS--MEASUREMENT PRINCIPLE AND PROCEDURE OF DRIVING FATIGUE

Which

g(p) = go + LAigi(P)'v"p E So i=1

A.

PERCLOS

(3)

go is the average texture, gi (p) are the

eigenvectors which correspond to the I largest eigenvalues,

PERCLOS (Percentage of Eyelid Closure Over the Pupil Over Time) [8] is percentage of eyes closing time over a particular time. Figure (1) shows that PERCLOS commonly used measurement principle, it is calculated as follow formula diagram:

Ai are the texture parameters.

Modelling instance

t3 -t2

C.

Given a set of shape parameters b texture parameters

=

(b1, b2
1

A (A p A2 ,... ,Af) ,Follow(2)and(3) AAM shape vectors S and the texture vectors g(p) can be
the shape M(p) , follow this way all the pixels on the parameters in a given location can be obtained. Beside these we need to make the similarity transformation about M (p) to get the actual image coordinate frame of face shape, to (x,y) : obtained. Piecewise affine So to

, bk) T and

Which s is the percentage of eyes closing time over a particular time, that is PERCLOS value. Eye opened biggest

t4 -t1

xlOO%

(7)

S on the shape can obtain

N(Pl=
Which rotation angle

C:
l'
=

-a
is

(ty,t y ) is the translation coordinates, B is the k

the scaling proportion, parameters, are attitude

)(:J (:: J
+

(4)

Figure I.

PERCLOS measurement principle

( a, b,t x ,t y ) kcosB-l,b

final position on the face shape is N(p0 ;q), on the place of

a=

ks i n B,For any point Po on the so' the

Po the texture is g(po) , so the final AAM model

=

instance can be expressed as follows:

In specific tests there are three measurement scales: P70, P80 and EYEMEA(EM). P70: Percentage of eyelid closure time over pupil over time more than 70%; P80: Percentage of eyelid closure time over pupil over time more than 80%; EM: Mean square value of the percentage of eye closure speed . Judging from the present study, P80 is the best measure of fatigue standards.
B.

K(N(W(p);q)))
Which

g(p)\:/pE

So

(5)

Steps f f or atigue detection

K is a two-dimensional images that sizable with N(W(p;q)) .

Input image match to the model image

D.

Use the classical reverse synthesis algorithm[7] to match input image I(p) to the model image

Firstly, acquire continuous driver's head image with the vehicle camera , then use the image processing module and the AAM algorithm described above to complete the positioning of the driver face and eye, even extract the eye feature parameters, including detect and track eye opening and closing movement of the opening and closing state , and make statistics in a certain period of time the driver eyes

181

continuously opening and closing time, in the end calculate PERCLOS values to judge the driver's fatigue.
C.

Improved calculation ofPERCLOS

To achieve the purpose of fatigue detection, we must first ensure the vehicle equipment (camera) can detect the human eye successful, assuming that the eye has been detected while due to the human eye would have on small texture values combined with the eyes of different people of different sizes, even with the vehicle moving the action followed the driver's body shaking make the camera to capture the range of the human eye may be timely to change, we can see the distance between vehicle camera and the human eye is extremely important. As we all known, proportion can eliminate the effect of distance, In order to eliminate the distance influencing factors between the camera and the eye, give the following improved method PERCLOS as shown in Figure.2:
-' ,"

Figure 3.

the rough location of the human face and eye:(a)Face detection (b)Rough eye location

B.

The accurate location of the human eye

By establishing the AAM model to match the training samples with the model can accurately determine the location of the human eye, shown in Figure.4:

i,;.'''''

Figure 4.
C.

Accurate eye locaiton

Detection of Opened or closed eye state

Figure 2.

Improved calculation of PERCLOS

First, let the driver's eye look into the camera opened for 5 seconds, monitoring devices will automatically record this information to initialize the initial value as that record the current ratio of height to width of the human eye ,doing this the influencing of distance between camera and the human eye can be eliminated ,and improve the detection accuracy, on this basis calculate the PERCLOS value can be used to quantify eyes closed, and further determine whether the driver is fatigue. In this paper , use the PSO measurement standard. When the ratio of human eye's height and width is between 20% - 80% , according 0.2S1
<

A complete fatigue driving detection process, including image acquisition, image processing, face detection, eye location, eye tracking, eye closure decision, fatigue determined, use the method in this paper, according to this process can accurately detect the eyes closed situation to achieve the purpose of determine fatigue. Follow Figure.5 we can see by this means state of opened eye or closed can be detected accurately, even if when human wearing glasses.

S2

<

0.8S1 ,we

consider that the eye is opened; When the ratio of human eye's height and width is less than 20% ,according S3 is closed. IV. A. RESULTS AND ANALYSIS OF EXPERIMENT
<

0.2S1

'

we consider that the eye

The rough location of the human eye

Use the Adaboost[9] algorithm module in the AAM algorithm can detect the human face, and thus the initial detection of human eye orientation can be obtained, as shown in Figure.3:

182

Figure 6.

Determine the characterization of the fatigue: (a)Does not fatigue state(b) Fatigue state

V.

CONCLUSIONS

This paper describes a driver fatigue detection method, by establishing the appearance model AAM ,on the basis of human face detection locate the position of the eye accurately, and improve the traditional method to calculate PERCLOS, by calculating the ratio of eye's height and width eliminates the influence of distance between the driver and the impact of the camera , and then calculate the improved physical quantities PERCLOS that used to determine the driver fatigue to determine the current state of the driver is fatigue or not. 1) it can capture 7 f rames pixel per second, very good to detect the eyes closed condition with a good real-time; 2) AAM model f non-rigid objects (such as eye, etc. ) or fit the contours has a high accuracy; 3) The limitations PERCLOS theory itself can only be applied on the basis of certain conditions, such as interior lighting, light changes and movement of the head can cause detection errors, etc. , so in f uture studies f atigue detection method will combine a variety of ways to improve the accuracy of judgments.
ACKNOWLEDGMENT Figure 5. Detection of eye closure:(a ) eyes closed when wearing glasses(b)eyes opened when wearing glasses (c)eyes opened when not wearing glasses(d)eyes closed when not wearing glasses.

The authors sincerely appreciate Xie Mei and Dong Hua zhi for their help.
REFERENCES [I]

D.

Determination of f atigue Record the number of eye closure and the ratio of eye's

height and width , then calculate the ratio

= _

S3 S2

[I] Vural E, Cetin M, Ercil A, et al. Drowsy Driver Detection Through Facial Movement Analysis[C]//Proc. of IEEE International Workshop on Human Computer Interaction. [So I.]: IEEE Press, 2007:6-18. Lin Guangyu, Wei Lang , Based on digital image driver while driving the car identification Computer Engineering, 2007,33 (22):193-194. Zhangzu Tao, Zhang Jiashu,UKF-based nonlinear tracking the human eye detect driver fatigue, Southwest Jiaotong University,2008,43 (6) 697-702. Cootes T F, Edwards G J, Taylor C J. Active Appearance Models. Proceedings of the European Conference on Computer Vision, 1998, 2484-498. Dryden I L, Mardia Wiley&Sons, 1998. K V. Statistical Shape Analysis. John

When s > 0.15, this shows the driver is sleepy state that belongs to drowsy driving. The results as shown in Figure.6:

[2] [3]

[4]

[5] [6]

Goodall C. Procrustes Methods in the Statistical Analysis of Shape. Journal of the Royal Statistical Society B, 1991, 53(2)285-339.

183

[7]

Cootes T F, Edwards G J, Taylor C J. Active Appearance Models. IEEE Transactions on Pattern Analysis and Machine Intelligence,200 I, 23(6)681-685 Song Zhenghe and Zhou Yiming, Motor Driver Fatigue recognition algorithm based on PERCLOS. China Agricultural University,2002,7 (2) 104-109.

[9]

Viola P ,.Iones M. Robust real-time face detection[J ] International Journal of Computer Vision ,2004 ,57 (2) :137 - 154.

[8]

[10] Yang Quefen, Hu Weihua,An innovation algorithm in the eye location for the driver fatigue, Computer Engineering and Applications,2008,44(6) :20-24

181