2008 Congress on Image and Signal Processing
A J2ME-Based Wireless Intelligent Video Surveillance System
Using Moving Object Recognition Technology
Lizhong Xu , Zhong Wang , Huibin Wang , Aiye Shi , Chenming Li
College of Computer and Information Engineering
Hohai University
Nanjing, P. R. China 210098
lzhxu@hhu.edu.cn / wz1980@gmail.com
Abstract
A low-cost intelligent mobile phone-based wireless
video surveillance solution using moving object
recognition technology is proposed in this paper. The
proposed solution can be applied not only to various
security systems, but also to environmental
surveillance. Firstly, the basic principle of moving
object detecting is given. Limited by the memory
consuming and computing capacity of a mobile phone,
a background subtraction algorithm is presented for
adaptation. Then, a self-adaptive background model
that can update automatically and timely to adapt to
the slow and slight changes of natural environment is
detailed. When the subtraction of the current captured
image and the background reaches a certain threshold,
a moving object is considered to be in the current view,
and the mobile phone will automatically notify the
central control unit or the user through phone call,
SMS (Short Message System) or other means. The
proposed algorithm can be implemented in an
embedded system with little memory consumption and
storage space, so it’s feasible for mobile phones and
other embedded platforms, and the proposed solution
can be used in constructing mobile security monitoring
system with low-cost hardware and equipments. Based
on J2ME (Java2 Micro Edition) technology, a
prototype system was developed using JSR135 (Java
Specification Requests 135: Mobile Media API) and
JSR120 (Java Specification Requests 120: Wireless
Messaging API) and the test results show the
effectiveness of proposed solution.
1. Introduction
The increasing need for intelligent video
surveillance in public, commercial and family
applications makes automated video surveillance
systems one of the main current application domains in
computer vision. Intelligent video surveillance systems
deal with the real-time monitoring of persistent and
transient objects within a specific environment.
978-0-7695-3119-9/08 $25.00 © 2008 IEEE
DOI 10.1109/CISP.2008.235
Intelligent surveillance system has been evolution to
third generation, known as automated wide-area video
surveillance system [1]. Combined computer vision
technology, the distributed system is autonomic, which
can also be controlled by remote terminals.
A low-cost intelligent wireless security and
monitoring solution using moving object recognition
technology is presented in this paper. The system has
good mobility, which makes it a useful supplement of
traditional monitoring system. It can also perform
independent surveillance mission and can be extended
to a distributed surveillance system.
Limited by the memory consuming and computing
capacity in a mobile phone, background subtraction
algorithm [1-6] is presented to be adopted in mobile
phones. In order to be adapted to the slow and slight
changes of the natural environment, a self-adaptive
background model updated automatically and timely is
detailed. When the subtraction of the current captured
image and the background reaches a certain threshold,
a moving object is thought to be in the current view,
and the mobile phone will automatically notify the
central control unit or the user through phone call,
SMS, or other means.
Based on J2ME technology, we use JSR135 and
JSR120 to implement a prototype.
2. Background Subtraction Technology
Background subtraction is a commonly used class
of techniques for segmenting out moving objects of
interest in a scene for applications such as surveillance.
It involves comparing an observed image with an
estimate of the image if it contained no objects of
interest. The areas of the image plane where there is a
significant difference between the observed and
estimated images indicate the location of the objects of
interest. The term “background subtraction" comes
from the simple technique of subtracting the timely-
updated background template from the observed image
and then thresholding the result to generate the objects
of interest [6].
281
 The flow chart is shown in Figure 1.
Figure 1: Background Subtraction Algorithm
2.1. Background Template Construction
Before the moving objects can be identified, a
background template must be built.
Generally, background and foreground (moving
objects) are mixed together such as waving leaves in
the garden and running automobiles on high way. The
foreground can not be removed so the ideal
background image can not be retrieved. But the
moving objects do not exist in the same location in
each frame of a real-time video sequence. An
“average” frame of the video sequence can be retrieved
to approach the ideal background image. The gray
values of pixels which have the same location in each
frame of the video sequence are averaged to represent
the gray value of the pixel which located in the same
place in the approximate background. An average
value of pixels in the same location of each frame in a
video sequence is calculated. To simplify, the
approximate background is also called “background
template”, “background” or “template” in the
following contents.
Frame1 + Frame2 + ... + Framei
Background first = two pixels have the same location in their frames.
i
(1)
In our prototype, the first 10 frames are captured to
calculate the background template (i=10). Moving
objects can not be identified in these frames. If the
moving objects move too slowly, i should be increased
to reduce the tolerance.
2.2 Moving Object Recognition
After the background template has been
constructed, the background image can be subtracted
from the observed image. The result is foreground
(moving objects). Actually, the background is timely-
updated. The update algorithm is detailed in the next
section.
Foreground j =| Frame j − Background j | (j>i)
In case of some random disturbances, each pixel
will fluctuate in a small range even there is no
expected moving objects in the scene. So there must be
a strategy to judge it. A threshold is defined in the
system. If the difference of one pixel between real time
frame and template is more than 10, then add 1 to the
threshold. When differences of all pixels in the frame
are all calculated, moving objects is thought to appear
if the threshold is more then 3 percent of the total
number of pixels in the frame.
2.3 Background Template Update
Due to the sun light changing very slowly, the
background template must be updated timely.
Otherwise the foreground can not be correctly
identified anymore. Add 1 to the pixels value in the
background template if the corresponding pixels value
is more than it in the template, or subtract 1 if it is less.
This algorithm is more efficient than “Moving Average
Algorithm” because it only uses addition and
subtraction operation, and do not need much memory
storage.
The algorithm is presented as follow:
For each pixel in Frame j:
If ( Pixel k > Pixel background k )
then {Pixelbackground k + +};
(3)
elseIf ( Pixel k < Pixel background k )
then {Pixelbackground k − −};
Pixelk is a pixel in frame j, and Pixelbackground k is the
corresponding pixel in background template. These
With such method, the background template can
adjust automatically according to environment change.
3. J2ME Technology
In this article, we have implemented a prototype on
mobile telephones based on J2ME technology.
Java™ Platform, Micro Edition (Java ME) is the
most ubiquitous application platform for mobile
devices across the globe. It provides a robust, flexible
environment for applications running on a broad range
of other embedded devices, such as mobile phones,
PDAs, TV set-top boxes, and printers. Applications
based on Java ME software are portable across a wide
range of devices, yet leveraging each device's native
capabilities.
(2)
282
3.1 Mobile Media API (JSR-135)
The Mobile Media API (MMAPI) extends the
functionality of the J2ME platform by providing audio,
video and other time-based multimedia support to
resource-constrained devices [7] [9].
3.1.1 Getting a Video Capture Player The first step
in taking pictures (officially called video capture) in a
MIDlet is obtaining a Player from the Manager.
Player mPlayer =
Manager.createPlayer("capture://video");
The Player needs to be realized to obtain the
resources that are needed to take pictures.
mPlayer.realize();
3.1.2 Showing the Camera Video The video coming
from the camera can be displayed on the screen either
as an Item in a Form or as part of a Canvas. A
VideoControl makes this possible. To get a
VideoControl, just ask the Player for it:
VideoControl mVideoControl =
(VideoControl)mPlayer.getControl("VideoControl");
already be set by the implementation (the address is
taken from the URL that was passed when the client
connection was created). Before sending the text
message, the method populates the outgoing message
by calling setPayloadText().
TextMessage tmsg
=(TextMessage)mc.newMessage(MessageConnection.
TEXT_MESSAGE);
tmsg.setPayloadText(msg);
mc.send(tmsg);
The main title (on the first page) should begin 1-3/8
inches (3.49 cm) from the top edge of the page,
centered, and in Times 14-point, boldface type.
Capitalize the first letter of nouns, pronouns, verbs,
adjectives, and adverbs; do not capitalize articles,
coordinate conjunctions, or prepositions (unless the
title begins with such a word). Leave two 12-point
blank lines after the title.
4. Prototype
The system architecture is shown in Figure 2.

3.1.3 Capturing an Image Once the camera video is

shown on the device, capturing an image is easy. All
you need to do is call VideoControl's getSnapshot()
method. The getSnapshot() method returns an array of
bytes, which is the image data in the format you Figure 2: System Architecture
requested. The default image format is PNG (Portable
Network Graphic). In the prototype system, if the difference between
byte[] raw = mVideoControl.getSnapshot(null); real-time frame and template reaches a predefined
Image image = Image.createImage(raw, 0, threshold, moving objects are considered to appear.
raw.length); Then the handset will send out an alert SMS.
Since the device has good mobility, it can be put
3.2 Wireless Message API (JSR-120) anywhere including those area not covered by other
surveillance system. And it can be deployed rapidly in
The J2ME Wireless Toolkit supports the Wireless emergency.
Messaging API (WMA) with a sophisticated
simulation environment. WMA 1.1 (JSR 120) enables
MIDlets to send and receive Short Message Service
5. Experiment
(SMS) or Cell Broadcast Service (CBS) messages.
The prototype has been implemented on Motorola
WMA 2.0 (JSR 205) includes support for MMS
E680 GSM phone and Motorola ic902 CDMA1X
messages as well [8] [10].
phone. The phone fact sheet is listed below.
3.2.1 Creating a MessageConnection To create a
client MessageConnection just call Connector.open(), Table 1: Phone Fact Sheet
passing a URL that specifies a valid WMA messaging E680 IC902
protocol. CPU Intel xScale™ Qualcomm MSM6550
MessageConnection mc = 300 MHz 225 MHz ARM9™
(MessageConnection)Connector.open(addr); OS Linux CDMA Brew
Screen 240*320 65K colors
3.2.2 Creating and Sending a TextMessage The Camera 0.3 Mega pixel 2.0 Mega pixel
connection is a client, the destination address will Java MIDP 2.0 JSR135 JSR120 etc.

283
Data GPRS Class 10 CDMA1X of several frame time. The “Template time” is the total
Service 32-48kbps 400-700kbps time to construct the background template.
In this instance, the “Snapshot time” , ”DIP time”
Figure 3 is the UI (User Interface) of the prototype and “Frame time” of the 100th frame are presented.
application. The first picture in the form is real time And the “Frame time Average” of the 100 frames and
frame, which is got from the camera originally. The “Template time” are also given.
second image is the template image. If there are some
moving objects being detected, the third picture will be Table 2: Performance (“ms” is millisecond)
displayed on the form. And some real time information
E680 IC902
is displayed below the pictures.
Image size 192*192 160*120
Snapshot time (100th frame) 1213 ms 1565 ms
DIP (digital image process) 278 ms 296 ms
time (100th frame)
Frame time (100th frame) 1491 ms 1861 ms
Frame time Average (100 1542 ms 1792 ms
frames)
Template Time (first 10 20422 ms 25639 ms
frames)

As shown in Table 2, the background template can

be built in less than half a minute. And the total time to
Figure 3: Java Demo calculate a frame is around one and a half seconds. It
meets the requirements to be a family security
Since the first several frames must be stored to monitoring system and an anti-theft system. The
calculate the template, a big memory heap size is experiment demonstrates the feasibility of the proposed
needed. For E680, the image size is 192*192.The Java system.
Virtual Machine heap is big enough to store the Some frames are magnified to be seen more clearly
frames. But for ic902, the image size is 640*480.The (See Figures 4 - 6). Figure 4 is the self-adaptive
JVM heap can’t provide so much memory. There are background template. As shown in Figure 5 and 6, a
two methods to solve this problem. First, the frames person run into the scene was identified immediately.
can be stored to the EFS (Embedded File System).
Second, the image size can be reduced. The first
method can provide high resolution image data, which
contains more detailed image features. But storing to
EFS will take a much longer time than storing to
memory, and much longer time is need in the
following calculating processes. The second method
will lost some detailed image features, but it can fully
operated in memory and reduce much processing time.
Considered the requirements of the real time ability,
the second method is adopted. The image size is
reduced to 160*120, which is still enough to be used to
identify the moving objects.
The performance for 100 frames is detailed in Table
2. The term “Snapshot time” is the time length to get
the image though J2ME MMA API. It mainly depends
on the capability of Hardware, Operating System and Figure 4: Background Template
Java VM. The “DIP (digital image process) time” is
the time length to perform the background subtraction
algorithm, including images compare and template
update. The “Frame time” is the total time to process a
frame which equals to the sum of “Snapshot time” and
“DIP time”. The “Frame time Average” is an average

284

Figure 5: Real Time Frame
Figure 6: Foregrounds (Moving Object)
6. Conclusion
The moving object recognition technology led to
the development of autonomous systems, which also
minimize the network traffic.
With good mobile ability, the system can be
deployed rapidly in emergency. And can be a useful
supplement of traditional monitoring system.
With the help of J2ME technology, the differences
of various hardware platforms are minimized. All
embedded platforms with camera equipped and
JSR135/JSR120 supported can install this system
without making any changes to the application.
285
Also, the system can be extended to a distributed
wireless network system. Many terminals work
together, reporting to a control center and receiving
commands from the center. Thus, a low-cost wide-area
intelligent video surveillance system can be built.
Further more, with the development of embedded
hardware, more complex digital image process
algorithms can be used to give more kinds of
applications in the future.
7. Acknowledgements
This work was supported by the National High
Technology Research and Development Program of
China (863 Program) (2007AA11Z227).
8. References
[1] M Valera, SA Velastin, Intelligent distributed
surveillance systems: a review. IEE Proceedings on Visual
Image Signal Processing, April. 2005, vol. 152, vo.2, pp.192-
204.
[2] M. Piccardi, Background subtraction techniques: a
review, IEEE International Conference on Systems, Man and
Cybernetics, Oct. 2004, vol. 4, pp. 3099–3104.
[3] T. Horprasert, D. Harwood and L.S. Davis, A Robust
Background Subtraction and Shadow Detection, Proc.
Proceedings of the Fourth Asian Conference on Computer
Vision, January 2000, vol. 1, pp. 983-988.
[4] Y Ivanov, A Bobick, J Liu, Fast Lighting Independent
Background Subtraction, International Journal of Computer
Vision, Jun. 2000, vol. 37, no. 2, pp. 199–207.
[5] A Elgammal, D Harwood, L Davis, Non-parametric
Model for Background Subtraction, Proceedings of the 6th
European Conference on Computer Vision-Part II, 2000, pp.
751-767
[6] Alan, M. McIvor, Background Subtraction Techniques,
Proceedings of Image & Vision Computing New Zealand
2000 IVCNZ’00, Reveal Limited, Auckland, New Zealan,
2000.
[7] C. Enrique Ortiz, The Wireless Messaging API
developers.sun.com, 2002
[8] Jonathan Knudsen, Taking Pictures with MMAPI
developers.sun.com, 2003.
[9] Sun Microsystems, Inc., Nokia Corporation., JSR135
MMAPI 1.2 (Final Version) Specification, 2006.
[10] Sun Microsystems, Inc., JSR 120 Wireless Messaging
API (WMA) specification, 2002.