Computers and Electronics in Agriculture 106 (2014) 111119

Contents lists available at ScienceDirect

Computers and Electronics in Agriculture

journal homepage: www.elsevier.com/locate/compag

Automatic video tracking of Chinese mitten crabs based on the particle

lter algorithm using a biologically constrained probe and resampling
Yu Jiang a,b,c, Zhenbo Li a,b,c, Jingjing Fang a,b,c, Jun Yue d, Daoliang Li a,b,c,
a

China-EU Center for Information and Communication Technologies in Agriculture, China Agricultural University, Beijing 100083, PR China
Beijing Engineering and Technology Research Center for Internet of Things in Agriculture, China Agricultural University, Beijing 100083, PR China
c
Key Laboratory of Agricultural Information Acquisition Technology, Ministry of Agriculture, Beijing 100083, PR China
d
College of Information and Electrical Engineering, Ludong University, Yantai 264025, PR China
b

a r t i c l e

i n f o

Article history:
Received 11 February 2013
Received in revised form 1 May 2014
Accepted 7 May 2014

Keywords:
Chinese mitten crabs
Behavioral analysis
Tracking
Particle lter

a b s t r a c t
The behavioral patterns of crabs affect the quality and quantity of their production. Currently there are no
methods to track behavioral patterns using imaging-based algorithms. This study provides a precise
pathway tracking method for the research of the relationship between behavioral patterns of crabs
and their living environment. The Particle Filter algorithm is an appropriate framework to handle nonGaussian movement, the biological constraints are used to decrease the computational complexity and
improve the accuracy of tracking the crabs pathways and a biological probe is utilized to determine
the area of movement. Then, this newly determined area replaces the entire image frame as the search
space to reduce the complexity of the computation. In the resampling step, a traditional Gaussian particle
distribution is substituted by a fusiform particle distribution, which better matches the crabs biological
motion patterns, to represent the probability of the crab movement. This strategy allows the crab
positions to be covered using fewer particles, which is more accurate for analyzing abrupt motion or
long-term stationary situations than traditional particle distributions. To determine the robustness and
accuracy of the results, 3000 and 12,000 frames were used, respectively. The coverage ratio and accuracy
increased by 28.79% and 5.75%, respectively, compared with the color histogram-based particle lter
(CHPF) and by 69.57% and 37.66% compared with the ssion bootstrap particle lter (FBPF). The experimental results show that the proposed tracking method is feasible and can be used as an efcient tool to
get the pathway of crabs under water.
2014 Elsevier B.V. All rights reserved.

1. Introduction
Chinese mitten crabs are widely harvested crustaceans in China,
with production reaching approximately 650,000 tons in 2010
(FAO, 2010). To increase the quantity and improve the quality of
crabs, intensive and automatic aquaculture strategies are needed.
However, further improvement in the crab aquaculture is limited
by insufcient knowledge of crab behavior. Understanding the
relationships between different crab behaviors and water quality
factors such as dissolved oxygen (DO) and pH is important (Chen,
2007). Therefore, behavioral analysis methods and tracking platforms are required to analyze the crab activity, aiding researchers
determine the relationships between different behaviors and the
environmental drivers in the crab water habitat.
Corresponding author. Address: P. O. Box 121, China Agricultural University, 17
Tsinghua East Road, Beijing 100083, PR China. Tel.: +86 10 62737679; fax: +86 10
62737741.
E-mail address: dliangl@cau.edu.cn (D. Li).
http://dx.doi.org/10.1016/j.compag.2014.05.013
0168-1699/ 2014 Elsevier B.V. All rights reserved.

With the development of computer vision technology, many

researchers have designed various systems to automatically
observe, store and analyze animal behaviors based on visual
inspection (Zion, 2012). An imaging-based system was built to
acquire images to study the behavioral rhythms of crustacean
decapods inhabiting depths where the sun light is absent
(Menesatti et al. 2009). The results showed that video analysis provides a promising direction for animal behavioral search. Later on,
a video solution was presented for assessment of benthic populations and biodiversity due to rhythmic behavior (Aguzzi et al.,
2012). Besides, the relationship between animal shape and their
genetics can be potential to predict animal behaviors (Costa
et al., 2011). Although all aforementioned methods can be used
to explore the relationship between animal behaviors and their living environment, they do not estimate the pathway of the animal
under water overtime since it is hard to track animal movements
using traditional pattern recognition algorithms. However,
recently developed video tracking algorithms provide pathway
information of tracking objects.

112

Y. Jiang et al. / Computers and Electronics in Agriculture 106 (2014) 111119

Video tracking is the method which can obtain the positions of a

target of interest (Khan et al., 2005; Veeraraghavan et al., 2008). In
the last few decades, video tracking has been widely applied to
track pedestrians, cars, and other targets. Generally, these algorithms are divided into two types: the linear (Gaussian, unimodal
problems) and the non-linear (non-Gaussian, multimodal problems). Kalman lter has been proven successful in solving linear
problems, and uses a series of measurements of observed overtime
to estimate the status of the object in the future (Crisan and
Doucet, 2002). However, its performance is reduced in non-linear
situations. It is not suitable for tracking animals because movement of animals is typically non-linear. Researchers have presented various strategies, such as the extended Kalman lter
(EKF), the unscented Kalman lter (UKF), and the particle lter
(PF), among others, to correctly track objects in non-linear situation (Cheng and Zhang, 2008).
With advances in computational ability, the particle lter
(Gordon et al., 1993) has been deemed an efcient model for a
non-linear, non-Gaussian and multi-model state space. The particle lter algorithm is based on a sequential Monte Carlo method,
so the importance sampling (IS), and the sequential importance
sampling (SIS) are two critical factors of the particle lter algorithm. However, the simple SIS was used in all these particle lter
algorithms, causing a degeneracy problem after several iterations
(Chen, 2003). Gordon et al. proposed the landmark method termed
the bootstrap particle lter (BPF), a novel resampling strategy that
is based on utilizing the SIS to solve the degeneracy problem,
which, in turn, causes the problem of sample impoverishment
(Gordon et al., 1993). According to Gordon, many novel resampling
strategies such as the Markov Chain Monte Carlo (MCMC) shifting
(Khan et al., 2005), Kernel smoothing (Arulampalam et al., 2002),
regularization (Cheng and Zhang, 2008), and ssion BPF
(Punskaya et al., 2001) have been presented. Recently, Han et al.
utilized the immune genetic algorithm (IGA) to increase the number of meaningful particles to remedy the sample impoverishment
and ensure diversity in the particle set (Han et al., 2011).
In addition to the resampling strategy, the transition is another
crucial part of the particle lter. Some researchers redirected their
methods to nd variety and ensure a correct transition. Yan et al.
presented a particle lter using scale-invariant feature transform
(SIFT) to enhance their accuracy and robustness (Yan et al.,
2011). Along with the development of object detection procedures
(Felzenszwalb and Huttenlocher, 2004; Liu et al., 2011; Alexe et al.,
2012), a joint detection and particle lter method to track colored
objects was proposed (Czyz et al., 2007), making two distinct contributions: (1) target detection and deletion are embedded in the
particle lter without relying on an external track initialization
and cancellation algorithm; and (2) the algorithm is able to track
multiple objects that share the same color description while maintaining the attractive properties of the original color particle lter.
To examine the tracking abilities of these animal behavior analysis systems based on computer vision, two premises are adopted:
(1) The particle lter is an efcient way to track an animals motion
due to its prominent effect on non-linear, non-Gaussian and multimodal cases, which occur frequently in nature; and (2) the basis of
the particle lter is an effective and robust sample (transition) or
resampling strategy.
The overall objective of this paper is to address the issue of
accurate tracking of the pathway of the crab movement under
water. To achieve the objective, a 2-D imaging system was constructed to acquire video sequences of crabs under water, and a
biologically constrained based particle lter (BCPF) was applied
to obtain the position of crabs on a frame by frame basis, thus
providing the pathway of the crab overtime. Moreover, the BCPF
capability was extended to mitigate the tracking shifts caused by
abrupt movement and/or extended stationary periods of crabs,

and reduce the tracking errors accumulated in each particle matching iteration.
2. Materials and methods
2.1. Chinese mitten crab behavioral analysis platform
An automated Chinese mitten crab behavioral analysis platform
based on 2-D computer vision technology was designed consisting
of ve parts, including an enclosed box, a uorescent lamp (T5HO
47W, GE Lighting, U.S.), a water tank, a water-cycling device
(HW303B, Sunsun Corporation, P.R. China), a CCD (DH-HV3151UC,
Daheng Imaging Corporation, P.R. China) color camera, and a host
computer (Fig. 1). The enclosed box blocks out any external interference, such as luminance or noise. The uorescent lamps boost
the lighting conditions to improve the quality of the video
sequences captured in our articially closed environment. The
water-cycling device ensures the water purity and keeps the water
temperature stable.
A CCD color camera is positioned on the top of the water tank to
acquire video data. The data are transmitted to the host computer
through a USB interface. The host computer used for image processing was built around an Intel Core i5 CPU with a 2.8 GHz processor and 4 GB of RAM, running Windows 7 Service Pack 1 (SP1)
with Visual Studio 2010. The Open CV 2.4 library is utilized as
the video processing framework.
2.2. Video acquisition
Our study monitored Chinese mitten crabs obtained from Yixing, Jiangsu Province, which is one of the main breeding areas in
China. More than 33 h of video sequences used in the experiment
were captured by the CCD color camera with a frame resolution
of 720  540 at 20 frames per second (fps). To capture high-quality
video sequences, the water in the tank was approximately 10 cm
depth, with the bottom of the tank covered by white marble sand
to create uniform background thus eliminate problems caused by
shadow or reection of the glass materials and background complexity. Each video contains the motions of four mature crabs
(Fig. 2), however the BCPF was applied to motions of individual
crabs, because multiple object tracking is beyond the scope of
research presented herein.
2.3. Dynamic model for crab tracking
Crab video tracking is regarded as a Bayesian estimation problem and a representative dynamic state model needs to be dened
prior processing the video data. The crab shell is used to represent
the entire crab, which means that claws and legs are ignored during the tracking.
Eq. (1) represents the state vector of a single crab in the non-linear dynamic state space:

X t xt ; yt ; W t ; Ht 

in which t denotes the frame number. The term (xt, yt) denotes the
middle point coordinate of the target region (rectangle in ourcase)
in which the targets color histogram and object template are computed. The terms Wt and Ht denote the width and height of the track
region, respectively. The particles are predicted in accordance with
the second-order autoregressive dynamics of a random walk as
shown by Eq. (2):

X t A0  X t1  X 0 A1  X t2  X 0 X 0 B  wt

in which X0 represents the initial state of the crab tracker A0, A1, and
B are the coefcients dened by 2.0, 1.0, and 1.0, respectively; and

Y. Jiang et al. / Computers and Electronics in Agriculture 106 (2014) 111119

113

Fig. 1. Schematic of the 2-D imaging-based behavioral analysis platform.

Fig. 2. Movement of four crabs captured by the 2-D imaging system.

w denotes the process noise, which is assumed to be white, zeromean and Gaussian (Cheng and Zhang, 2008).
2.4. Color histogram-based particle lter
Particle lter algorithms are sequential Monte Carlo techniques
specically designed for sequential Bayesian estimation when the
systems are non-linear and their random elements are non-Gaussian (Gordon et al., 1993). In general, it uses amount of sample
points (particles) to describe potential positions of the object in
the search space, and the particle most similar to the object is considered as the position of the object. The color histogram-based
particle lter (CHPF) consists of three steps described as follows
and run recursively (Fig. 3a):
Step 1: Initialization
Particle lters approximate the posterior density using a
weighted set of stochastic samples, namely distribution of particles
randomly according to the selected object tracker, when the frame
number is 0.

We consider the difference in the color histogram between the

initial target region and the tracker region as the weight score
described in Step 2. The difference and its normalization value
are calculated with Eq. (3)(5):

Dt

r
XN q
j j
1
h0 ht
j1
j

nweightt
Z

XN
j1

1
j
weightt
Z
j

weightt

4
5

where D represents the distance between two histograms derived

from the Bhattacharyya similarity coefcient, and j denotes the
jth particle with N as the total number of particles and h indicating
the histogram of the target to be selected initially or tracked automatically. The weight and the n weight represent the weight of the
previous particle and of the new particle, respectively. The term Z
is the normalization coefcient.
2.5. Biologically constrained probe

Step 2: Importance sampling

When the frame number is greater than 1, the most likely state
transition is executed via Eq. (2). Thus, the particle weights are calculated and normalized.
Step 3: Resampling
Taking into account the sorting of the normalized weights, the
particles are resampled, and particles with higher weight obtain
additional new sub-particles.

Because traditional particle transitions follow a Gaussian distribution, a sufcient number of particles need to be generated in
each iteration to represent the target. Furthermore, due to the exibility of the color feature under different circumstances, the particles cannot be translated accurately. Some particle lter methods
with other accurate and robust features are more computationally
complex. To achieve both accuracy and low-complexity computations, the biological constraints of the crabs are utilized. This strategy can amend the accumulated errors caused by color histogrambased function.

114

Y. Jiang et al. / Computers and Electronics in Agriculture 106 (2014) 111119

Fig. 3. Flow chart of the processing procedure for crab tracking: (a) typical color histogram based particle lter and (b) the biological constraints based particle lter (BCPF).

The length of crab shell in our video sequences is approximately

0.81 cm, and the ratio between the video data and the real world
is approximately 1:6.7. Firstly, a rectangle, or sometimes square, is
used as the track to cover the crab shell. Furthermore, according to
the maximum velocity of a crab is approximately 10 cm per second. Considering the frame rate of our video data (20 fps), the longest distance of crab movement is approximately 0.5 cm.
Therefore, the target tracked in our video may depict a region that
has the same midpoint as the tracker with double its length and
width, changing the target search space from the entire frame to
a sub-frame. The computational cost of searching the target simultaneously decreases signicantly.
The particle with the highest weight in the last round is chosen
as the probe to calculate this sub-frame using Eq. (6).

F sub GetSubF; xprobe ; yprobe ; 2  W probe ; 2  Hprobe

T 0 i ; j Ti ; j 
0

w  h 
0

1
P

i00 ;j00 Ti

00

00

;j

F 0 i i ; j j Fi i ; j j


w  h 

i00 ;j00 Fi

00

00

i ;jj

x; y arg maxRi; j

10

Consequently, Eq. (11) utilizes the obtained positional information to adjust all particle positions prior to the next transition
procedure.

dx ; dy xprobe;new ; yprobe;new  xprobe;old ; yprobe;old

11

The coordinates (dx, dy) indicate the deviation between the

probe coordinates and the template-matched probe coordinates.
2.6. Biologically constrained resampling

The GetSub function is used to obtain a region with a midpoint

at (x, y), a width W and a height H. If F is the frame picture, then
(xprobe, yprobe) are coordinates for the probe particle tracker midpoint, and Wprobe and Hprobe are the probe particle tracker width
and height, respectively.
Furthermore, a template matching function is applied to determine the targets position and matching score in the sub-frame
using Eqs. (7)(9) as follows:

P
0
0
0 0 0
0
i0 ;j0 T i ; j  F i i ; j j

q
Ri; j P
P
0
0 2
0 0 0 2
0
i0 ;j0 T i ; j 
i0 ;j0 F i i ; j j

(i, j) as the midpoint with the width and the height similar to the
tracker. The term T represents the target template. Eq. (10) can be
used to calculate the maximum relative x and y coordinates.

in which R(i, j) is the relative coefcient between the matching

region and the template. The matching region uses the pixel point

The SIS method reduces particle diversity, and tracking accuracy signicantly. To handle the deciency in the particle variety
caused by resampling, the typical particle lters are applied as
the new resampling strategies mentioned in Section 1 can increase
the diversity of trajectories by directly increasing the number of
particles or by indirectly smoothing the particle distribution. However, crab activity differs from that of humans through characteristics such as abrupt changes in the direction of movement.
Therefore, these methods cannot be adapted to our tracking
strategy.
The biological clues (movement direction and patterns), as a
new resampling rule, were applied to simultaneously maintain
the particle variety and smooth distribution. The crab motions
can be classied in two types: rotation and translation. Rotation
activities cause a change in the direction, and translation activities
lead to a change in position.
Only translational activities can cause a crabs position to
change and can affect the particle distribution. In addition, unless
it moves in a circular motion, a crab can only move along a straight
line. Therefore, translation without rotation cannot lead to directional change except a linear directional change: from start to
end or vice versa. Considering these biological motion characteristics, we assume that the crab tracked in our algorithm can move in

Y. Jiang et al. / Computers and Electronics in Agriculture 106 (2014) 111119

115

Step 3: Importance sampling with probe adjustment

Under the condition of a frame greater than 1, all particles are
adjusted using Eq. (13), the transition of the most likely state is
executed via Eq. (2). Finally, the particle weights are calculated
and normalized:

8
>
< xp;old ; yp;old dx ; dy ; Rx; y P 0:8
xp;new ; yp;new xp;old ; yp;old ;
Rx; y < 0:8
>
:
Fig. 4. Biological movement direction assumption: (a) eight-direction encoded
activity sketch map and (b) example of biological direction based particle
distribution.

eight directions (Fig. 4a). Before resampling, the direction of the

crabs movement is calculated using Eq. (12). Then, the particles
are resampled by both their direction and weight score. The calculated direction and the two directions nearest the positive resampling direction are calculated and allocated 70% of the particles to
them by their normalized weight, and the rest of the particles are
distributed along the three relative negative directions. Finally, a
new fusiform-based distribution is created (Fig. 4b).

directionp;t

xp;t  xp;t1
GetDirection
yp;t  yp;t1

!
12

The direction indicates the crab orientation of particle p in the

tth frame; GetDirection is a function for obtaining the direction
code via gradient.
2.7. Particle lter with a biologically constrained probe and resampling
A new biological constraint-based transition and resampling for
the design of a typical color histogram-based particle lter is proposed (Figs. 3b and 5). The detailed steps are as follows:
Step 1: Initialization
The particle lters approximate the posterior density by a
weighted set of stochastic samples, namely a random distribution
of particles according to the midpoint of the object tracker selected
manually, when the frame number is 0.
Step 2: Probe calculation
Prior to the particle transition, a probe is emitted to determine
the sub-frame and to match the template.

13

Step 4: Resampling with biological distribution

Taking into account the sorting by normalized weight, the particles are resampled by both their direction and the normalized
particle weight.
2.8. Evaluation standards
To evaluate the algorithms mentioned in this paper, target coverage ratio and position deviation were used to provide the performance information of different algorithms. Eq. (14) is used to
calculate the ratio between the number of crab pixels covered
within our tracker and other methods trackers:

Rcov erage

Scit
Scs

14

where Rcoverage is the coverage ratio between the areas in which the
crabs appear in the tracker calculated by algorithms and labeled by
humans. The term Scit is the area in which the crabs appear in the
tracker, and Scs is the manually outlined area of the standard. More
target pixels covered in the tracker, namely a higher coverage ratio,
indicates that more information regarding the target is obtained by
the tracker. Therefore, higher the ratio, the more accurate position
obtained.
Eqs. (15) and (16) are used to calculate the deviation between
the position obtained via tracking algorithms and those marked
by humans:

Dev t

q
xtr;t  xsd;t 2 ytr;t  ysd;t 2

8
>
< 1;
Accuracyt 0;
>
:

15

Dev t < r
Dev t P r

16

The term Dev is the deviation between the standard position

and the tracker; xtr, xsd denote the x-coordinate of the tracker midpoint in tth frame. The terms ytr, ysd denote the y-coordinate of
tracker midpoint in tth frame. The term r is the threshold that

Fig. 5. Procedure of biological constraints based probe and resampling, including initialization, tracker matching, probe amendment, movement direction calculation, and
resampling distribution.

116

Y. Jiang et al. / Computers and Electronics in Agriculture 106 (2014) 111119

measures the accuracy of the tracker. In our case, r is 7 due to the

deviations of the x- and y-coordinates, which are each 5 pixels (The
size of a crab onscreen is approximate 50  50 pixels. One-tenth of
the size is considered to be the maximum deviation). Thus, the
Accuracy is 1 when the Dev value is less than 7, which is considered
an accurate position.
3. Results and discussion
A quantied test requires people to label or manually verify the
data, leading to a high cost of labor. Therefore, considering our
labor conditions, we selected 3000 frames for the coverage ratiobased measurement and 10 min video sequences for the position-based measurement separately. The frames and video
sequences were chosen randomly for an unbiased rectication of
performance.
To prove that our algorithm can efciently perform under conditions of low computational capacity, the CPU utilization ratio and
the memory usage for the tracking program was limited to 25% and
100 MB, respectively. Our algorithm was compared with two other
typical particle lters, the color histogram-based particle lter
(CHPF) (Gordon et al., 1993) and the ssion bootstrap particle lter
(FBPF) (Punskaya et al., 2001) with the same video sequences.
3.1. Probe adjustment accuracy
First, the target coverage ratio was used to verify the performance under extended stationary situation. In this case, a crab
stayed in a xed place for a long time. The tracker obtained by
our method usually covered the entire crab in the results, while
the tracker obtained by CHPF and FBPF sometimes exhibited poor
coverage (Fig. 6 and Table 1). This occurred because both CHPF
and FBPF use color information as matching feature. Although
there is no movement, the color feature is sensitive to uctuating
light, especially in the water. Moreover, the color histogram-based
matching function cannot properly calculate the statistical object
color characteristic under the water. Therefore, the target color histogram can misdirect the particle transition to increase the accumulated errors, which may invalidate particles identication and
disable the tracker. Finally, the tracker obtained by CHPF and FBPF
have low target coverage ratio. Usually a shifting phenomenon can
be caused by this low coverage ratio. On the contrary, the tracker
obtained by our method matches the object using template-matching algorithm, which is not sensitive to minor light changes caused
by the wave of water. Particle transition can be adjusted using the
amended position given by template matching method. Therefore,
the proposed method can reduce, or even eliminate errors accumulated by previous iteration to provide higher target coverage ratio.
Meanwhile, the higher ratio can maintain correctness of tracker to

Table 1
Frame coverage ratio of Fig. 6 calculated with two typical particle lters and the BCPF.
Frame
number

Coverage ratio
(CHPF) (%)

Coverage ratio
(FBPF) (%)

Coverage ratio
(BCPF) (%)

1
20
120
160
240

100.00
94.33
78.62
76.33
81.97

100.00
84.17
98.27
93.45
87.13

100.00
100.00
100.00
100.00
100.00

avoid the shift issue, providing an accurate pathway of the object

we tracked.
In addition, the coverage ratio was used to test the performance
of crabs motions. In that video, from frames 40 to 175, movement
occurred with acceleration; from 175 to 220, movement was decelerated; and from 220 to 235, the motion abruptly ceased. The proposed method provides higher coverage ratio than CHPF and FBPF,
especially for abrupt motion (Fig. 7 and Table 2). Because, for the
CHPF, the particle transition relies on a color-based matching function, so this method appears plausible under relatively steady
lighting conditions, e.g. high coverage ratio at frame 1, 4, and
220. However, when the abrupt motions occur, the color feature
is unstable and cannot track the crab accurately, and then the ratio
is decreased, e.g. frame 175 and 235. The FBPF demonstrated its
deciencies in handling abrupt motion or stopped motion as well.
In frames 40175, due to acceleration, most of the particles moved
along the accelerated direction, obtaining the maximum probability using the initialized target. With this process, despite the crabs
beginning to slow down (from frame 175 to 220), most of the particles kept on transiting along the accelerated direction, but not
change to the decelerated direction. Meanwhile, the error of the
tracker accumulates. Finally, when the crab stops suddenly, these
particles are transited into the error space with their previous inertia, and the coverage ratio was decreased signicantly, e.g. frame
220 and 235. However, the proposed method searches the object
in the probe area instead of the whole image space, which allows
it to use template-matching function instead of color-based histogram method. The particles are then transited to the correct position instead of shifting to other places, namely, it reduces the
accumulated errors. Therefore, when the crab abruptly moves or
stops, the proposed method still correctly tracks it instead of shifting along the previous tracking direction.
Overall, the biological probe uses template-matching function
to reduce the accumulated error in particle transition, improving
the accuracy of tracking algorithm under either extended stationary condition or abrupt motions. No matter which coverage ratio
(minimum, maximum or average), the probe method exhibits
excellent performance compared with the CHPF and FBPF (Table 3).

Fig. 6. Representative results of target covered in tracker of two typical particle lters and probe amendment for long stationary condition. (a), (b) and (c) are results of BCPF,
FBPF and CHPF respectively.

Y. Jiang et al. / Computers and Electronics in Agriculture 106 (2014) 111119

117

Fig. 7. Representative results of target covered in tracker of two typical particle lters and probe amendment for movement condition. (a), (b) and (c) are results of BCPF, FBPF
and CHPF respectively.

Table 2
Frame coverage ratio of Fig. 7 calculated with two typical particle lters and the BCPF.
Frame
number

Coverage ratio
(CHPF) (%)

Coverage ratio
(FBPF) (%)

Coverage ratio
(BCPF) (%)

1
40
175
220
235

100.00
92.58
81.77
90.30
80.63

100.00
97.31
93.68
70.09
53.94

100.00
100.00
100.00
100.00
100.00

Table 3
Minimum, maximum and average coverage ratios of 3000 selected frames.
Algorithm

Minimum coverage
ratio (%)

Maximum coverage
ratio (%)

Average coverage
ratio (%)

CHPF
FBPF
BCPF

19.33
68.09
86.97

100
100
100

65.39
88.43
94.18
Fig. 9. Deviation of crab position tracked by CHPF, FBPF and BCPF in rapid
movement video sequence.

3.2. Biologically constrained resampling accuracy

Resampling is an essential part of our algorithm and is crucial
for accurate tracking. To evaluate its performance, the coordinate
deviation is measured under rapid and back-and-forth motions
separately.
Figs. 8 and 9 present the results for rapid motion tracking. In the
case of rapid motion tracking, the crab moves much faster than in
other videos (Fig. 8). Compared with the CHPF and the FBPF, the
proposed method is always in lower position deviation (Fig. 9).
Because the particles in our method were distributed in a fusiform
area to enlarge the searching space during re-sampling. Therefore,
despite the position of a crab changing signicantly between two
frames, our method can nd it in the larger searching space. However, the CHPF and the FBPF methods are using the Gaussian distribution, which cannot resample enough searching space to
guarantee adequate samples to represent the crab motion probability. In addition, the CHPF resampling is known to cause a sample
deciency, meaning that after a certain number of iterations, most
particles are concentrated around one point, thus losing sample
diversity and causing a high position error. For the FBPF resampling, although a ssion strategy can resample particles while
maintaining the sample variety, the distribution of the sample is
Gaussian as well, and are not large enough for the rapid crab

activities. Therefore, the FBPF has higher accuracy than the CHPF
but lower one than the proposed method.
More specically, a proper and large resampling distribution
improves the accuracy of prediction of particle position. The
resampling distribution of our method is adaptively tted to the
natural patterns of crabs movement, e.g. crab more moves along
horizontal direction (Fig. 10). Therefore, our method always has
lower position deviation. On the contrary, the CHPF and the FBPF
only provide Gaussian distribution resampling. Both of them have
much higher tracking error. However, the FBPF has lower position
deviation than the CHPF, as the size of area covered by FBPF is larger than that of the CHPF. The proposed method has much lower
position deviation than other two existing methods over all video
sequences (Table 4).

3.3. Deciency of the biologically constrained particle lter

Although the biological constraint is successful in the aforementioned situations, two problems are left unresolved using this
method:

Fig. 8. Representative results of resampling of two typical particle lters and BCPF for rapid movement condition. The purple, the red and the yellow tracker represent results
of BCPF, FBPF and CHPF respectively. (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of this article.)

118

Y. Jiang et al. / Computers and Electronics in Agriculture 106 (2014) 111119

Fig. 10. Particle distribution of different resampling strategy: (a) crab motion direction (b) resampling distribution of BCPF (c) resampling distribution of FBPF and (d)
resampling distribution of CHPF.

Table 4
Minimum, maximum and average deviation of tracker midpoint coordinates and
accuracy rate of tracking for 12,000 frames.
Algorithm

Minimum
deviation

Maximum
deviation

Average
deviation

Accuracy
rate (%)

CHPF
FBPF
BCPF

0
0
0

33.73
25
12.73

16.53
12.60
1.03

22.46
54.37
92.03

video interacts with other crabs, the target deforms or rotates to

reduce similarity with the original template. In some cases, the original target template may be matched with new crabs, which join
via interaction, rather than the originally monitored target.
Because our approach is designed to track a single crab pathway,
a tracking framework based on a joint probability model should
be applied to track multiple crabs including their interactions.

4. Conclusions
3.3.1. Template nullication
The sub-frame region has an intense light shadow or the target
template is deformable (Fig. 11a). Due to inability to match a
potential template region to a sub-frame, our probe strategy
degraded into a typical particle lter and had no effect on the
amendment.
The light problem is caused by the design of the experimental
platform and can be avoided by changing the position of the illuminant. The current light source is positioned at the top of experimental platform, and as the light propagates through water, it
increases in intensity. Given its exibility in installation location,
an LED light source could be used instead of the current uorescent
light, or a polarized light could be selected to solve the problem.
For the circular motion case, the number of calculations
increases signicantly in all directions if the rotation is considered
in the template matching stage. Because the template-matching
method can resist subtle circular motions of the crab movement,
it can be ignored to maintain a low level of computational
complexity. However, matching invalidation can occur under conditions of a large circular motion. Considering experiences with
tracking human movement, more motion cues are utilized as constraints for tracking crab movement to account for the rotation
without leading to increased computation.

3.3.2. Interaction activity

When two or more crabs interact, overlap and occlusion sometimes appears (Fig. 11b). Because the original object tracked in the

To address the natural motions of the Chinese mitten crabs, an

approach based on the biological constraints model was developed
with the particle lter algorithm using a probe strategy with a
resampling method.
With the new approach including the biological constraints proposed in this paper, the natural motions of Chinese mitten crabs
can be tracked efciently and effectively using a particle lter algorithm. A biologically constrained strategy provides efcient tracking by utilizing information like the sphere of activity, the style
of movement, the limitation of speed, etc.
An appropriate resampling distribution signicantly improves
the resampling performance and ultimately the entire particle lter algorithm. The fusiform-distributed resampling is adapted to
both the characteristics of natural crab activitytransverse walking without rotationand the representative probability of the
samples with the maximum coverage ratio.
In the future, the focus will be on adding new, efcient and
robust features to the present target and to develop a new matching function to correctly calculate the probability of crab movement. In addition, an attempt will be made to establish a 3-D
system to obtain specic crab movement information from videos
captured using two cameras, and study interactions between crabs.
Acknowledgements
This research is nancially supported by the National
Science and Technology Support Program (2011BAD21B01 &
2012BAD35B07), the Fundamental Research Funds for the Central
Universities (2012RC022). The authors would like to thank Dr.
Adnan Mustac (University of Georgia, Athens, GA) for improving
the quality of writing in English.
References

Fig. 11. Deciency of our algorithm: (a) failure of template matching caused by
strong light disturbance and (b) failure of interactive activity.

Aguzzi, J. et al., 2012. Challenges to the assessment of benthic populations and

biodiversity as a result of rhythmic behaviour: Video solutions from cabled
observatories. Oceanogr. Mar. Biol. Ann. Rev., 233284.
Alexe, B., Deselaers, T., Ferrari, V., 2012. Measuring the objectness of image
windows. IEEE Trans. Pattern Anal. Mach. Intell. 99, pp. 11.
Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T., 2002. A tutorial on particle
lters for online nonlinear and non-Gaussian Bayesian tracking. IEEE Trans.
Signal Process. 50 (2), 174188.
Chen, W., 2007. Research on technique of healthy aquaculture of river crab in large
scale. Master Thesis. Nanjing Agricultural University.
Chen, Z., 2003. Bayesian Filtering: From Kalman Filters to Particle Filters, and
Beyond.
Cheng, S.Y., Zhang, J.Y., 2008. A review of particle lter. J. Astronaut. 4, 10991111.

Y. Jiang et al. / Computers and Electronics in Agriculture 106 (2014) 111119

Costa, C. et al., 2011. Shape analysis of agricultural products: a review of recent
research advances and potential application to computer vision. Food
Bioprocess. Technol. 4 (5), 673692.
Crisan, D., Doucet, A., 2002. A survey of convergence results on particle ltering
methods for practitioners. IEEE Trans. Signal Process. 50 (3), 736746.
Czyz, J., Ristic, B., Macq, B., 2007. A particle lter for joint detection and tracking of
color objects. Image Vision Comput. 25 (8), 12711281.
FAO global production statistics, 2010.
Felzenszwalb, P.F., Huttenlocher, D.P., 2004. Efcient graph-based image
segmentation. Int. J. Comput. Vision 59 (2), 167181.
Gordon, N.J., Salmond, D.J., Smith, A.F.M., 1993. Novel approach to nonlinear and
non-Gaussian Bayesian state estimation. IEEE Proc. Radar Signal Process. 140
(2), 107113.
Han, H., Ding, Y.S., Hao, K.R., Liang, X., 2011. An evolutionary particle lter with the
immune genetic algorithm for intelligent video target tracking. Comput. Math.
Appl. 62 (7), 26852695.
Khan, Z., Balch, T., Dellaert, F., 2005. MCMC-based particle ltering for tracking a
variable number of interacting targets. IEEE Trans. Pattern Anal. Mach. Intell. 27
(11), 18051819.

119

Liu, T., Yuan, Z.J., Sun, J., Wang, J.D., Zheng, N.N., Tang, X.O., Shum, H.Y., 2011.
Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. 33 (2),
353367.
Menesatti, P., Aguzzi, J., Costa, C., Garca, J.A., Sard, F., 2009. A new morphometric
implemented video-image analysis protocol for the study of social modulation
in activity rhythms of marine organisms. J. Neurosci. Methods 184 (1), 161
168.
Punskaya, E., Andrieu, C., Doucet, A., Fitzgerald, W.J., 2001. Particle Filtering for
demodulation in fading channels with non-Gaussian additive noise. IEEE Trans.
Commun. 49 (4), 579582.
Veeraraghavan, A., Chellappa, R., Srinivasan, M., 2008. Shape and behavior encoded
tracking of bee dances. IEEE Trans. Pattern Anal. Mach. Intell. 30 (3), 463
476.
Yan, Y., Wang. J.L., Li, C.Z., Wu, Z.X., 2011. Object tracking using SIFT features in a
particle lter. IEEE International Conference on Communication Software and
Networks (ICCSN).
Zion, B., 2012. The use of computer vision technologies in aquaculture a review.
Comput. Electron. Agr. 88, 125132.