Real-time vehicles tracking based on Kalman filter in an ITS

ZOU Xiaohonga LI Dongmeib LIU Jichuanb

a
College of Information Science and Engineering, Yanshan University, Qinhuangdao 066004, China
b
Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China

ABSTRACT

Tracking vehicles is an important and challenging problem in video-based Intelligent Transportation Systems, which has
been broadly investigated in the past. A robust and real-time method for tracking vehicles is presented in this paper. The
proposed algorithm includes two stages: vehicle detection, vehicle tracking. Vehicle detection is a key step. The concept
of tracking vehicle is built upon the vehicle-segmentation method. According to the segmented vehicle shape, a predict
method based on Kalman filter is proposed. By assuming that the vehicle moves with a constant acceleration from the
current frame to the next, a Kalman filter model is used to tracking and predicting the trace of a vehicle. The model can
be used in the real traffic environment, and can track multi-targets in a big area. So it is practical in the vehicle tracking.
The proposed method has been tested on a number of monocular traffic-image sequences and the experimental results
show that the algorithm is robust and can meet the real-time requirement.

Key words: Intelligent Transportation System, vehicle detection, vehicle tracking, Kalman filter, correlation matching

1. INTRODUCTION

In recent years, Intelligent Transportation Systems (ITS) has been investigated by many researchers. Video cameras
are considered as being a suitable sensor device for capturing and recognizing spatio-temporal aspects of road structures
and traffic situations due to their high spatio-temporal resolution. It is well recognized that vision-based surveillance
system are more versatile for traffic parameter estimation than the others.
In the last several years, an extensive research work has been done and many vehicles monitoring systems have been
exploited [1] [2]. In the traditional target tracking methods, the filter algorithms are α-β algorithm, the Gauss sum filter and
Kalman filter. Among them, α-β algorithm is a simple method and has little calculation, but the precision is not high.
Although the precision of Gauss sum filter is higher, its calculation is complex. Kalman filter is a commonly used
method to estimate the state of linear dynamic system. It is widely used to solve the problem of the estimation of linear
dynamic system’s state, and enable it to have a quite perfect solution.
This article uses a target tracking method based on Kalman filter. The method makes a full use of the prediction
function of Kalman filter to predict the region where the next frame possibly appears, then carries on the correlation
match operation in the smaller forecast region, finds the best correlation match spot and makes the target tracking more
initiatively.

2. VEHICLE DETECTION

International Symposium on Photoelectronic Detection and Imaging 2007: Image Processing, edited by Liwei Zhou,
Proc. of SPIE Vol. 6623, 662306, (2008) · 0277-786X/08/$18 · doi: 10.1117/12.791270

Proc. of SPIE Vol. 6623 662306-1

 It is a very important step to extract the vehicle shape out of the background and other vehicles accurately without the
effect of moving cast shadows, where the concept of tracking vehicle is built upon. The vehicle-detection method is
requested to automatically segment every vehicle so that there can be a unique tracking associated with the vehicle [3]. It
includes five steps: background estimation, background updating, background subtraction, moving cast shadow
elimination, and vehicle detection. In this phase, we will solve several problems as follows:
1) Extract the background image automatically from a sequence of road traffic images and update the background
continually according to the change of ambient lighting, weather, etc.
2) Select an adaptive filter to eliminate abnormal moving object in the binary background subtraction image so that
the system can be more robust.
3) Suppress moving cast shadows to avoid the overlapping between adjacent vehicles.
4) Detect vehicles from the binary background subtraction image.

3. VEHICLE TRACKING

Based on the segmented vehicle shape, which can be represented by a simple square model, The Kalman filter is used
[4]
to track and predict the trace of vehicles. Kalman filter is not require to the preserve the past survey data . When the
recent data is obtained, according to the recent data and previous estimated parameter, with the aid of the state shift
equation of the system itself and a set of recurrence formulas, the new estimated parameter can be figured out. This can
greatly reduces the filter device’s reserves and computation, and break through the limit produced by stationary random
process.

3.1 The Kalman filter model of vehicles tracking

(1) Supposition of vehicles’ constant acceleration motion in straight line
In the situation that the image collecting rate is quite high and the interval between the frames is small, the distance
vehicle moved is not far between two consecutive frames. The most vehicles travel at normal conditions (namely does
not appear the movements such as a sudden braking, backing a car, making a turn on a great angle), so the movement
between two consecutive frames is approximately considered as constant acceleration motion in straight line; But these
non-uniform accelerated movement situations are approximated to the constant acceleration motion in straight line,
because the interval between consecutive frames is small and the actual change of the acceleration is not big. Therefore,
this paper supposes that the movement of vehicles between two consecutive frames is a constant acceleration motion in
straight line. According to the principle of Kalman filter and the character of moving vehicles, this paper has established
the Kalman filter model of vehicles’ constant acceleration motion [5].
(2) State variable
In the Kalman filter model of vehicles tracking, the state variable is the muster of square apexes which indicate the
vehicles’ outline, including their positions, velocities and accelerations. Expresses a model point’s vertical and horizontal

coordinates with a complex vector: z kt = xkt + jykt ，1 ≤ k ≤ 4 ，t=1,2…。Here k means a serial number of model points at

current time; t means the time series. Then the state variable can be expressed as:

Proc. of SPIE Vol. 6623 662306-2

  z1t z 2t z3t z 4t 
 
x =  z&1t
t
z& 2t z&3t z& 4t  (1)
 &z&t &z&2t &z&3t &z&4t 
 1 
In the actual operation, we may establish two Kalman filters for convenience. Their state variable separately is model
points’ vertical or horizontal coordinates, their velocity and acceleration along the x or y axis. Two filters’ state transfer
equations and survey equations are completely same.
(3) State transition equation
The establishment of state transition equation (namely the assurance to the goal movement rule) is the key of the
model. If the state transition equation’s error is oversized, it can influence the performance of Kalman filter greatly.
Under the supposition of vehicles’ constant acceleration motion in a straight line, not considering the additional noise,
we obtains the following state transition equations,
X t +1 = AX t (2)

1 1 0.5
According to the supposition of movement model, we set A = 0 1 1  , and then the state transition equation is
0 0 1 

 z1t +1 z 2t +1 L z M t +1 
1 1 0.5  z1
t
z 2t L z M t 
 t +1   
 t t 
 z&1 z& 2t +1 L z& M
t +1
 = 0 1 1   z&1 z& 2t L z& M , M=4 (3)
 &z&t +1 &z&2 t +1 t +1 
L &z&M 0 0 1   &z&1t &z&2t L &z&Mt 
 1    
In the actual operation, we can calculate the vertical and horizontal coordinates separately.
(4) Measurement equation
The measurement equation is the points’ positions which represent the figure of the moving vehicle. The equation

is Y t = C k X t + ϖ t .

According to the movement model, we set Ck = [1 0 0] , so we get

Y t = [1 0 0]X t + ϖ t (4)

We suppose that ϖ t is the Gauss white noise whose additional mean value is 0 and the variance is σ 2 in the survey
process.
In above model, the noise is determined by experience and the statistics study. The experiment indicated that, the
introduction form of above noise conform to the need of the practical application basically.

3.2 Models’ initialization

(1) The choice of model parameter P0
Regarding to the choice of the filter’s initial error covariance matrix P0, we set P0 = diag {1,1,1} .
)
(2) The choice of original state X 0

Proc. of SPIE Vol. 6623 662306-3

 )
If we want to estimate the initial value of state variable X 0 , we need to know the outline point’s initial position,

velocity and acceleration. Therefore, we can calculate the value according to the moving vehicles’ position which
appears in the first three frames based on of the following suppositions.
Supposition 1: there are enough spacing between observed goal and other vehicles when initialization happens;
Supposition 2: the observed object’s velocity in the first 3 frames is not too high, namely, the distance that vehicle
moves between two consecutive frames is not too far. Therefore, the vehicle’s region examined in each frame has quite
many superpositions, which may quite easily to match.
Thus, making use of the positional information of the first 3 frames where the vehicles turn up, the average velocity

and acceleration can be got by calculating the positional variation of the vehicle center, respectively given z&i0 , &z&i0

( 1 ≤ i ≤ 4 ). In addition, the position of the vehicle’s outline point in frame 3 will be regarded as the initial position
)
z i0 ( 1 ≤ i ≤ 4 ). Then we can obtain the initial value X 0 of the model state variable.

3.3 Vehicle tracking algorithm based on Kalman filter

Algorithm beginning

Background Initialization
-
+ Obtain images
+
Background Updating
Background Subtraction
Eliminate the distortion caused by
shadows and headlights
Abstract the vehicle’s region and
descript the shape Initialize the parameters of
Kalman filter

Match vehicle’s position Predict the vehicle’s position

Update the parameters of

Kalman filter

Track of moving vehicle

Figure 1: Vehicle tracking algorithm flowchart

Figure 1 is the flow chart of vehicles tracking algorithm. First, the moving vehicles examination algorithm will be
adopted, which is based on the fixed background image differences. It can remove the noises caused by the shadows in
images and the vehicles’ headlight projection. It also can draw the moving vehicles’ region and descript the square model.
Then, we compare the positions of these vehicles with the positions forecasted by Kalman filter, to identify the match

Proc. of SPIE Vol. 6623 662306-4

relations of every target vehicles in this frame and previous frame. We consider the matching result as the new
observation variable of Kalman filter, update the parameters of the Kalman filter, regard the filter value of observation
variable output by the Kalman filter as the position of this vehicle, and calculate the tracking feature point (center of
square), then join the point to the trace of the vehicles. The traces of the vehicles that ultimately described by the
algorithm are taken as assemble of vehicles’ square centers which appear in the every frame.
It is essential to mention that there will be a certain deviation in the practical calculation. Because comparing with
actual location of vehicles, the vehicles’ contours determined by the moving vehicles examination algorithm are unfixed
in every frame. Therefore, the moving information of outline points (velocity, acceleration) is not the real moving
information of vehicles. Thus, we will carry on the tracking prediction after the Kalman filter model obtains the filter
value in k time and the speed as well as the acceleration is replaced by their mean values.
Besides, the vehicles’ positions getting by the filter rather than by matched and examined will be adopted as final
vehicles’ positions. In this way, it is equivalent to revise the results of the examining algorithm to make sure that the
moving paths of the vehicles is smoother and more approaching to the real situation.

4. EXPERIMENT RESULTS

(a) (b) (c) (d)

input image background background vehicle
extraction subtraction detection

Figure 2: Vehicle detection

Figure 3. Vehicle tracking

Proc. of SPIE Vol. 6623 662306-5

 Figure 2 represents the results of vehicle detection. And figure 3 represents the result of vehicle tracking based on the
Kalman filter. The moving paths of the vehicles are well obtained by above algorithm. It can be seen from the figure that
the vehicles’ paths are not very smooth in some place, particularly in the beginning. The main reason is that parameters
of Kalman filter are not stable in the beginning, so there are some differences between the calculated acceleration as well
as speed value and the real ones. However, due to the revision function of the examination algorithm towards the Kalman
filter, it is guaranteed that the track can still carry on in this situation, and the parameters of Kalman filter can be steady
quite quickly. Then the examined results will be revised by the Kalman filter parameter after it is steady for the sake of
smoothing the trace. In this way, the trace is not so steady in the beginning, however, later on, it will be smoothed
quickly.
The overall error of our system was measured by the following three criteria:
(1) Wrongly measured regions. This error occurs when vehicles were very close to each other, or when the region
includes more than two vehicles.
(2) Wrongly split regions. This error often occurs when vehicles appear at the lower side or higher side of the image
frame. In these cases, only part of the vehicle is visible in that frame. In addition, wrongly split regions happen when a
bus is mistaken with two vehicles.
(3) Losing vehicles in tracking. This error occurs in different situations. For example, vehicles are too far away from
camera.
The total error of our tracking algorithm is 5.4% that is the summation of the three error (i.e. 2.4% + 1.6% + 1.3% =
5.4%).

5. CONCLUSIONS
In this paper, the model of constant acceleration motion based on Kalman filter to track vehicle in a video-based ITS is
proposed. The experiment results show that the algorithm is effective and real-time. The model is low cost and high
accuracy. The assumptions of the Kalman filter conform to the actual situation, and the tracking algorithm can meet the
need of practical application.

REFERENCES
1. D.M.Ha, J.M.Lee, Y.D.Kim. “Neural-edge-based vehicle detection and traffic parameter extraction”. Image and
Vision Computing. Vol.22, pp.899-907, 2004.
2. Chengcui Zhang, Shu-Ching Chen, Mei-Ling Shyu and Snnivas Peeta. “Adaptive Background Learning for Vehicle
Detection and Spatio-Temporal Tracking”. ICICS-PCM 2003, Volume 2, pp.797-801. Singapore, 2003.
3. Michael Piovoso, Phillip A. Laplante. “Kalman filter recipes for real-time image processing”, Real-Time Imaging,
Vol. 9(6), pp. 433-439, 2003.
4. Peihua Li, Tianwen Zhang, Bo Ma. Unscented Kalman filter for visual curve tracking. Image and Vision Computing,
Vol. 22(2), pp.157-164, 2004.
5. Roya Rad, Mansour Jamzad. “Real time classification and tracking of multiple vehicles in highways”, ELSEVIER,
Vol.26, pp.1597-1607, 2005.
6. S.A.Vigus, D.R.Bull,C.N.Canagarajah. “Video objects tracking using region split and merge and a Kalman filter
tracking algorithm”. International Conference on Image Processing, Vol.10, pp.650-653, 2001.
7. Suchendra M. Bhandarkar, Xingzhi Luo. “An Efficient Background Updating Scheme for Real-time Traffic

Proc. of SPIE Vol. 6623 662306-6

 Monitoring”, 2004 IEEE Intelligent Transportation Systems Conference, WeA1.3, pp.859-864, Washington, D.C.,
USA, 2004.
8. P. Viola, M. J. Jones, D. Snow, “Detecting pedestrians using patterns of motion and appearance”, Proc. Ninth IEEE
International Conf Computer Vision, Vol. 2, pp.734 - 741, Nice, France, 2003.

Proc. of SPIE Vol. 6623 662306-7