Driver Head Monitoring using

Multi-core ARM processor
By : Goutham A.H.
2nd year M.Tech in VLSI Design
and Embedded Systems, DSCE.
under the guidance of
Prof. K.P. Shashikala



 Talking, eating, phone usage,
drowsiness, fatigue etc.

potential for intelligent
driver assistance systems(IDA)
to alert the driver of potential


driver behavior is hence
becoming an increasingly important
component if IDA’s

 Head motion cues.Importance of Driver Head Pose Driver head and eye dynamics derive where or at what the driver is looking. . when compared with eye-gaze cues. along with lane position and vehicle dynamics are used to predict a driver’s intent to turn. Head motion. were shown to better predict lane change intent 3 seconds ahead of the intended event.

a driver spends 95% of the time facing forward. It is during those 5% non-frontal glances that are of special interest when interesting events. Higher the speed.Driver Head Pose in Critical Situations  During a typical ride. . which are critical to driver safety occur. narrower is the drivers visual range and the driver is not expected to turn away from the frontal pose even for a fraction of a second.

Head movements during a merge event .

Single camera perspectives are susceptible to occlusion of facial features from spatially large head movements from the frontal pose. . Distributed camera framework for head movement analysis can operate even during large head movements.

Advantage is the covering of large pose space. .Multi-Perspective Framework Each camera is tracks the head independently.

Multi-perspective data selection .

) .  Use the knowledge of shape to constrain the search.  Patch model: Describes each individual item. etc. eyes are above nose. nose above mouth.CLM Algorithm  Local Models to search for individual items. such as nose. mouth (patches)  Shape Model: Describes where each patch can appear (for a normal person. eyes.

Architecture .


Image annotations .

◦ q is a vector of parameters that controls non rigid shape. nose tip. such as eye corners. mouth contour and outer face contour and the Point Distribution Model is given by: ◦ is the mean value of the ith feature. nose corners.CLM algorithm for facial feature tracking ◦ Facial features refer to salient landmarks on the face. ◦ Φi is a principal component matrix. .

ty]T is the translation term ◦ R2D is the rotation matrix . ◦ t = [tx .◦ s is a scaling term that is used to controls the distance from camera (s = f/Z here f is the focal length and Z is the depth from camera).

R2D . t .Let p CLM = {s . q } fitting algorithm look for maximum a posteriori probability (MAP) of the parameters p .

. Head Pose Estimation ◦ Given n (n ≥ 3) 3D reference points in the object framework and their corresponding 2D projections the perspective-n-point (PnP) problem solving algorithm determines the orientation and position of a camera. ◦ A Direct Least Squares (DLS) method is used to solve the PnP and to obtain the head rotation in 3 degrees of freedom. ◦ The CLM algorithm outputs 68 landmark points that are considered for PnP problem.B.

CLM Tracker Image sequence Bounding Shape box Face initializati parameters detection on Fitting successful failed Validatio n .


0 OTG x 1.0 Host x 1.0 GHz quad core and Cortex™-A7 1.1 for display .4 GHz quad core CPUs  2Gbyte LPDDR3 RAM at 933MHz (14.0 HS400 Flash Storage  USB 3.9GB/s memory bandwidth)  eMMC5. USB 2. USB 3.0 Host x 4  HDMI 1.4a and DisplayPort1. Samsung Exynos5422 Cortex™-A15 2.

.Test bed for on road evaluation Camera placement on the car dashboard.

OpenCV's application areas include:  2D and 3D feature toolkits  Facial recognition system  Gesture recognition  Human–computer interaction (HCI)  Mobile robotics  Motion understanding  Object identification  Segmentation and Recognition  Stereopsis Stereo vision: depth perception from 2 cameras  Motion tracking .OpenCV for image processing  OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at realtime computer vision.

and taskbased applications.Intel TBB Threading Building Blocks (TBB) is a C++ template library developed by Intel for writing software programs that take advantage of multi-core processors. . TBB library makes parallel performance and scalability easily accessible to write loop.

Results and Evaluations .

of events 6 7 3 8 4 3 31 .Naturalistic driving events Events Right turns Left turns Stop sign Right lane change Left lane change Merge Total No.