Pattern Analysis and Applications manuscript No.
(will be inserted by the editor)
Understanding People Motion in Video Sequences Using Voronoi
Diagrams
Detecting and Classifying Groups
Julio Cezar Silveira Jacques Jr., Adriana Braun, John Soldera, Soraia Raupp Musse, Cláudio Rosito
Jung
Graduate School of Applied Computing
Universidade do Vale do Rio dos Sinos
Av. Unisinos 950, 93022-000 São Leopoldo, RS, Brazil
phone: +55 51 3591-1122 ext. 1626 and fax: +55 51 3590-8162
Received: September 30, 2005
Abstract This work describes a model for understand-
ing people motion in video sequences using Voronoi Di-
agrams, focusing on group detection and classification.
We use the position of each individual as a site for the
Voronoi Diagram at each frame, and determine the tem-
poral evolution of some sociological and psychological
parameters, such as distance to neighbors and personal
spaces. These parameters are used to compute individ-
ual characteristics (such as perceived personal space and
comfort levels), that are analyzed to detect the forma-
tion of groups and their classification as voluntary or
involuntary. Experimental results based on videos ob-
tained from real life as well as from a crowd simulator
were analyzed and discussed.
1 Introduction
The understanding of people motion in video sequences
can be very useful in several applications. For example,
motion analysis is useful for video surveillance, pedes-
trian monitoring in traffic applications, measurement of
athletic performance and virtual reality systems, among
other potential applications.
Several approaches have been proposed for action/behavior
understanding from video sequences with different levels
of analysis, ranging from global (crowd events without
individuality) to local (individual behaviors and pose de-
tection). For middle-level analysis, people interactions
may provide useful cues. In particular, group detection
and classification may be useful in a variety of appli-
cations. For instance, an approach behavior followed by
Correspondence to: C. Jung
e-mail: crjung@unisinos.br
grouping can represent a theft or kidnap, and the detec-
tion of large involuntary groups may indicate the pres-
ence of strangle points in the environment (people form
groups due to lack of physical space).
In this work, we explore and quantify some concepts
related to personal space, proxemics and spatial relation-
ship among people, that are mainly described in other
research fields like psychology and sociology. These con-
cepts are then used to detect and classify groups, and
are briefly discussed next.
The term proxemics has been firstly proposed by Ed-
ward Hall [1] in order to describe the social use of space
(in particular, personal space). Personal space is related
with the area with invisible boundaries surrounding an
individual’s body. This area states as a comfort zone dur-
ing interpersonal communication, and can disappear in
specific environments or situations (e.g. elevators, dense
crowds). Hall proposes four main distances (see Table 1)
observed in American interactions. Each distance has a
particular meaning, in terms of the kind of interaction
allowed. Hall argues that those meanings depends on
the culture, and also shows how distance constrains the
types of interaction that are likely to occur.
Robert Sommer [2], an American sociologist, calls
such spatial overcoat as “portable space”, meaning that
each organism carries his/her own space wherever he/she
goes. Each individual person has an invisible “personal
space” around himself/herself, and either consciously or
unconsciously, he/she tries to maintain that space. The
desired physical distance between individual persons may
differ very much depending on culture. However, in spite
of such cultural differences, people implicitly recognize
and respect the individual personal spaces of others.
This work describes a model in which the personal
space and the distance from each individual to his/her
neighbors can be easily estimated through Voronoi Di-
2 Julio Cezar Silveira Jacques Jr. et al.
agrams (VDs). In the proposed approach, the position
of each person is captured using computer vision algo-
rithms at each frame, and it is used as a site to compute
a VD. As people move, the positions of the sites are mod-
ified in time, generating a temporal evolution of Voronoi
polygons (called in this work Dynamical Voronoi Dia-
gram, or DVD). The geometry of Voronoi polygons is
explored to extract and quantify sociological and psy-
chological individual characteristics, which are used to
detect the possible kinds of interactions proposed by
Hall [1]. Such interactions are then analyzed to detect
group formations. We also propose a variation of the per-
sonal space: the Perceived Personal Space (PPS), which
takes into account the vision field of each individual to
characterize group types. The main contribution of this
paper is related to the understanding of motion and
behavior of individuals and groups using Voronoi Dia-
grams.
This paper is organized as follows. Next Section presents
some works related to pattern recognition used in crowd
motion understanding. Section 3 presents the proposed
model, while Section 4 discusses some results of the pro-
posed model applied to video sequences filmed in real
public environments as well as to video sequences re-
sulting from a crowd simulator. Finally, Section 5 draws
final remarks and some ideas for future work.
2 Related Work
Several authors have presented methods for understand-
ing the motion of people filmed by static cameras. There
is a great variety of approaches and applications, rang-
ing from a global analysis (crowd behavior) to very lo-
cal (tracking of individual trajectories or body parts).
Within this range of applications, there are methods
that rely on people interactions for motion understand-
ing, and some of these techniques are briefly revised next
(a more comprehensive review on human motion under-
standing and surveillance can be found in the survey
papers [3,4]).
Stauffer and Grimson [5] worked with statistical meth-
ods to track people and other objects, which are clas-
sified and recognized through prior models. The system
learns with such recognitions and accumulated prototype
motion histogram from filmed recognized sequences, that
allow the system to detect unusual events. An inspector
watching may help to classify the level of danger of that
scene. In such work, they are more concerned with people
recognition and their interaction with objects for surveil-
lance applications, not much with interactions among
people.
Buxton and Gong [6] described a method to deter-
mine individual people behavior using Bayesian networks.
In this approach, the objects dynamics are tracked and
their behaviors are described as Bayesian networks, which
contain information such as time and events. To classify
events, the system detects the proximity of agents as not
near, nearby, close, very close and touching. Hosie and
collaborators [7] proposed a method for group behav-
ior detection based on surveillance data. Their approach
rely on pair primitives, which are pre-defined movements
that can occur between two targets in the scene over one
time sample. Such primitives are used to detect simple
events, such as convergence, divergence or stationarity.
These two methods are more focused on pairwise prim-
itives, and are not suited for analyzing larger groups.
Oliver et al. [8] used computer vision tracking algo-
rithms and Coupled Hidden Markov Models (CHMMs)
to model and recognize human tasks. Du and collabo-
rators [9] proposed a similar approach using Dynamic
Bayesian Networks (DBNs) instead of CHMMs. These
approaches are able to detect some kinds of pre-defined
human interactions (such as follow, or approach + talk
+ continue together ), but are also limited mostly to in-
teractions between only two persons.
Gong and Xiang [10,11] explored Dynamic Proba-
bilistic Networks (DPNs) for modeling the temporal rela-
tionships among a set of different object temporal events
in the scene for a coherent and robust scene-level be-
havior interpretation. Although grouping is embedded
in their approach, the main focus is activity recognition.
Also, psychological aspects are not considered for group-
ing purposes.
Liu and Chua [12] presented a new method to model
and classify multi-agent activities based on the obser-
vation decomposed hidden Markov models (ODHMMs),
also proposed by the authors. In their approach, pre-
defined activities with interacting people can be trained,
using the relative distances between any two people as
feature vectors. Despite the good results achieved for de-
tecting “Snatch Thefts”, the proposed method presents
a high computational cost as the number of agents in-
creases. Furthermore, tests were performed using man-
ual tracking of people, making its practical application
difficult to evaluate.
Fuentes and Velastin [13,14] proposed event detec-
tion algorithms based on trajectories and foreground
blobs designed for closed-circuit television (CCTV) surveil-
lance systems. In their approach, some pre-defined events
involving two or more persons can be detected (such as
fights, attacks and vandalism), but the concept of group-
ing was not explicitly used.
In this work, we propose a new method that uses the
trajectory of each tracked person to compute a temporal
sequence of VDs. Then, Voronoi polygons are explored
to extract individual characteristics based on psycholog-
ical concepts, such as personal space (proxemics) [1] and
perceived personal space (a new concept introduced in
this paper). Such individual characteristics are used to
detect and characterize group formations, which can be
further explored for higher-level event detection.
The proposed model is described in the next Section.
Understanding People Motion Using Voronoi Diagrams
3 The Model for Understanding People Motion
The main goal of the proposed model is to extract infor-
mation from video sequences and to use such information
to recognize patterns of individual and groups’ behaviors
using VDs. First of all, we will justify our decision con-
cerning the usage of VDs.
According to Hall [1], Personal Space (PS) is related
with the region with invisible boundaries surrounding an
individual’s body. Mathematically, this concept can be
quantified as the set of points that are closer to this per-
son than to any other individual. In fact, this property
arises naturally when computing the Voronoi Diagram
using the position of each person as a site: any point be-
longing to the polygon related to a given site is closer to
this site than to any other site used in the Voronoi tes-
sellation. Furthermore, according to Sommer [2], the PS
is in fact a “portable space”, meaning that each individ-
ual carries his/her own space wherever he/she goes. As
people move, their new positions are used as sites to re-
compute the VD, and the PSs are updated accordingly.
In other words, the time-varying positions tracked people
are used as sites to compute a series of time-varying VDs.
As people move, there is one VD for each frame of the
video sequence, and the temporal evolution of the VDs is
called a Dynamic Voronoi Diagram (DVD). Such DVD
provides the PS for each tracked person across time.
Besides of providing the temporal evolution of per-
sonal spaces, the Voronoi tessellation can be also used to
quantify other psychological concepts. In particular, the
following assumptions and definitions are taken in this
work:
• Perceived Personal Space (PPS): this concept is in-
troduced in this paper, and it is defined as the por-
tion of the PS that lies within the vision field of the
person;
• People comfort (PC): we define different “comfort
levels”, based on the PPS and the sociological inter-
personal distances proposed by Hall [1].
The temporal evolution of these parameters (given
by the DVD) is then used to detect group formations.
We assume that a set of people form a group if they
keep close enough for a certain period of time (the defi-
nition of “close enough” is provided by Hall’s classifica-
tion, shown in Table 1, while “a certain period of time”
is quantified in terms of a grouping period Tg , described
in Section 3.3). When a group is detected, it is further
classified as:
• Voluntary group: is a group of individuals that keep
an intimate distance during a certain period of time,
with sufficiently large PPSs (this means that, although
individuals have perceived enough space in their vi-
sion fields, they chose to stay in short distance with
other individuals);
• Involuntary group: is a group of individuals that keep
an intimate distance during a certain period of time,
3
but having small PPS (this usually indicate that they
form a group because they have no free space left to
go);
More details on group formation and classification
will be provided in next Sections. However, we first de-
scribe an algorithm for automatic people tracking in
filmed video sequences suited for top-view images in monochro-
matic video sequences, which is used to obtain the tra-
jectory of each person (and, hence, the DVD). Although
color cameras are becoming very popular nowadays, we
decided to use grayscale video sequences because there
are still scenarios (e.g. surveillance cameras) that employ
monochromatic cameras (or cameras with poor color
definition). Furthermore, the procedure described next
can be applied to the luminance component of color
cameras, whereas most tracking algorithms designed for
color cameras can not be applied to grayscale video se-
quences.
3.1 An Algorithm for Automatic People Tracking in
Top-View Cameras
Several vision-based techniques for people tracking in
video sequences have been proposed in the past years (a
recent survey can be found in [4]), most of them with
surveillance purposes. In these applications, an oblique
(or almost lateral) view of the scene is required, so that
faces of the individuals can be recognized. However, such
camera setups often result in occlusions, and mapping
from image pixels to world coordinates may not be ac-
curate (due to camera projection). Since our main goal
is to extract trajectories for each individual, we chose a
camera setup that provides a normal view with respect
to the ground (thus, reducing perspective problems and
occlusions). In fact, the general plane-to-plane projective
mapping is given by:
au + bv + c du + ev + f
x= , y= , (1)
gu + hv + i gu + hv + i
where (u, v) are world coordinates, (x, y) are image co-
ordinates, and a, b, c, d, e, f , g, h, i are constants. In
top-view camera setups, it is easy to show that Equa-
tion (1) reduces to:
x = au, y = ev, (2)
and the mapping from image to world coordinates is
trivial. Furthermore, Equation (2) indicates that per-
sons have the same dimensions at all positions, indicat-
ing that an area thresholding technique may be used for
people detection.
It is important to notice that the expected longitudi-
nal projection of a person in oblique-lateral views is ex-
plored by several tracking algorithms, such as [15–17,14].
However, such hypothesis clearly does not apply for top-
view cameras, requiring a different strategy for people
4 Julio Cezar Silveira Jacques Jr. et al.
tracking. In fact, a person’s head is a relatively invariant
feature in top-view (or almost top-view) camera setups,
indicating that tracking can be performed through tem-
plate matching.
To improve the accuracy of template matching, we
also employ a background subtraction technique to iso-
late foreground pixels from the static background. Al-
though there are several algorithms for background sub-
traction reported in the literature [14–24], just a sub-
set deal with shadow suppression in grayscale video se-
quences, such as [18,21–24]. In this work, we adopted the
algorithm described in [24], due to its simplicity, speed
and adaptation to illumination changes (including shad-
ows and highlights).
In the first frame that a new person enters the scene,
it is expected only a portion of his/her body to be visi-
ble. As the person keeps moving, a larger portion of the
body will be visible, until he/she enters completely in
the viewing area of the camera. Hence, each new fore-
ground “blob”1 that appears is analyzed, and its area is
computed in time. If this blob is related to a real person,
then its area will increase progressively, until it reaches a
certain threshold that represents the minimum area al-
lowed for a valid person (such threshold is determined a
priori, based on average person sizes in a specific camera
setup).
Once a person is fully detected in the scene, it must
be tracked across time. For that purpose, we use a simple
and robust tracking procedure based on feature correla-
tion [25]. To find the approximate position of the center
of the head (which is the center of the correlation tem-
plate T ), the Distance Transform (DT) is applied to the
negative of each foreground blob (i.e. the blob exterior
is considered the foreground object). The global maxi-
mum of the DT corresponds to the center of the largest
circle that can be inscribed in the blob, and it provides
an estimate of the person’s head center.
If the blob area exceeds a certain threshold (in this
work, we used three times the minimum area value), than
such blob is probably related to two or more connected
persons, as in the rightmost blob in Fig. 1(b). In such
case, not only the global maximum of the DT is ana-
lyzed, but also the local maxima with largest values. If
the distance between the global maximum and a cer-
tain local maximum is larger than the diameter of the
template (so that templates do not overlap), such lo-
cal maximum is also considered a head center, as shown
in the rightmost blob of Fig. 1(c). Fig. 1(a) illustrates
the corresponding frame, along with head centers and
correlation templates. It is important to note that the
procedure for detecting individual templates belonging
to the same blob may fail when people enter in the scene
forming a compact cluster, which generates a blob with-
out a recognizable geometry. In such cases, individual
1
A blob is a set of connected foreground pixels
persons will only be detected when the cluster breaks
apart.
The next step is to identify template T in the follow-
ing frame. Although there are several correlation metrics
designed for matching a small template within a larger
image, Martin and Crowley [25] indicated that the Sum
of Squared Differences (SSD) provides a more stable re-
sult than other correlation metrics in generic applica-
tions, leading us to choose the SSD as the correlation
metric. It should be noticed that more generic object
tracking methods (such as the covariance tracking de-
scribed in [26]) can be used instead of the SSD, but with
additional computational cost.
Also, it does not make sense to compute the corre-
lation over the entire image, since a person moves with
limited speed. We use a reduced correlation space, tak-
ing into consideration the frame rate of the sequence,
and the maximum speed allowed for each person. For
example, if the acquisition rate is 15 frames per second
(FPS) and the maximum allowed speed is 5 m/s, than
the center of the template cannot be displaced more that
5 × 1/15 = 0.33 meters in the subsequent frame (for the
camera setup shown in Fig. 1, 0.33 meters correspond to
approximately 35 pixels). Since the template center must
be moved to a foreground pixel, the correlation space
is further reduced by removing all background pixels.
The SSD between the template T and the correlation
space is then computed, and the center of the template
is moved to the point related to the global minimum of
the SSD. Such correlation procedure is repeated for all
subsequent frames, until the person disappears from the
camera view.
Although the head is a good choice for the correlation
template, head tilts and illumination changes may vary
the graylevels within the template. Also, the procedure
for selecting the initial template may not detect exactly
the center of the head. To cope with such situations, T
is updated every N frames (we used N = 5 for sequences
acquired at 15 FPS).
As a result of our tracking procedure, we can deter-
mine the trajectory and velocity of each person captured
by the camera. We use the position of each person to ob-
tain the Voronoi sites for each frame, and the associated
velocity to compute the individuals orientation (which
are used to obtain PPS). The usage of VDs for obtaining
individual and group characteristics is explained next.
3.2 Extracting Individual Information
We argue that group events can only be precisely ex-
tracted if individual information are consistently treated.
We defined that the personal space (PS) for an individ-
ual is the area of the corresponding Voronoi polygon. In
fact, such choice matches the psychological principle of
personal space, since all the points in the interior of a
Voronoi polygon are closer to the site that generated this
polygon than to any other site.
Understanding People Motion Using Voronoi Diagrams
We can also use the Voronoi polygon of each per-
son to compute his/her distance to his/her neighbors.
In fact, a point located at the edge of a Voronoi poly-
gon is equidistant from two sites. Thus, the orthogonal
distance from the site of a VD to its polygon edges rep-
resents half of the distance between this site and a neigh-
boring site. With this distance, we can classify the kind
of interactions that each two neighbors have according
to Hall’s distances, shown in Table 1. An example of a
VD and the computed distances is illustrated in Fig. 2.
In this Figure, values d1 , d2 and d3 are half-distances
from the person marked with a small square with re-
spect to his/her neighbors. It should be noticed that the
VD for a set of N sites can be computed with complexity
O (N log N ) using a divide and conquer algorithm [27],
which is cheaper than performing an exhaustive search
to compute pairwise distances.
The DVD provides the temporal evolution of VDs
as people move, allowing the computation of the PS and
distance from neighbors for each individual as a function
of time. The temporal analysis of such information can
be used to determine both individual and group infor-
mation (as it will be discussed in Section 3.3).
An important individual characteristic that can be
assessed using the temporal evolution of the PS is the
level of comfort of people, according to Sommer [2]. Al-
though it is intuitive to think that a person feels comfort-
able if the personal space around him/her is sufficiently
large, we believe that the PS is not the best metric to
estimate the level of comfort. In fact, the psychological
concept of PS is static, meaning that it considers a region
around the individual, disregarding his/her direction of
movement. A person can have lots of personal space be-
hind him/her, but he/she may not feel comfortable if
there is an obstacle or another person right in front of
his/her desired path. The proposed Perceived Personal
Space (PPS) provides a much better metric to estimate
the level of comfort, because it takes into account the
vision field of the individual and his/her PS.
The vision field is modeled as a circular sector with
angle α, symmetric with respect to the velocity vector of
the individual. According to Vaughan et al. [28], the hu-
man vision field has an aperture angle of approximately
170◦ in the horizontal plane, but can be reduced due
to several factors, such as eye disfunctions (e.g. myopia)
or movement at higher speeds (e.g. driving a car on a
highway). Also, the region of attentional focus is consid-
erably smaller, approximately 40◦ [29]. Consequently, if
one is focused on a particular direction, then objects in
the peripherical vision field will be out of focus, or blurry
(in fact, the perception of objects decreases as they get
closer to the boundaries of the vision field). In this work,
we used α = 120◦ as the perception angle, making a com-
promise between the attentional focus (40◦ ) and the full
viewing angle (170◦ ). The PPS is then defined as the
area of the region formed by the intersection of the vi-
sion field and the corresponding Voronoi polygon, shown
5
as darker regions in Fig. 3. It should be noticed that α
is a parameter in our algorithm, and can be adjusted to
increase or decrease the synthetic vision field, or even
be chosen randomly for each individual within a certain
angular range.
We believe that a person feels comfortable if there
is enough space for him/her in the PPS, within a cer-
tain distance range Rc . We use Hall’s values (Table 1) to
determine different comfort levels depending on Rc . For
instance, using Rc = 3.5 meters would provide a com-
fort level with respect to the public distance (highest
comfort level), while using Rc = 0.5 meters would pro-
vide a comfort level with respect to the intimate distance
(lowest comfort level). Mathematically, we say that an
individual is comfortable at frame t if:
απ 2
PPS(t) ≥ R , (3)
360◦ c
απ 2
where 360 ◦ Rc is the area of a circular sector with angle
α and radius Rc , and PPS(t) is the perceived personal
space at frame t. In particular, we use the intimate com-
fort level (Rc = 0.5) to determine voluntary group for-
mations, as it will be explained in Section 3.3.
According to the proposed metric for comfort, indi-
vidual 7 is more comfortable than individual 2 in Fig. 3.
In fact, there is no one close to person 7 in his/her vision
field, while person 1 is in front of person 2.
Next, we describe our approach for group formation
and characterization based on inter-person distances and
comfort levels.
3.3 Extracting Groups Information
To detect group formation, we keep track of the dis-
tance from each person to his/her neighbors (such dis-
tances are provided directly by the VD) across time. If
two or more people keep short distances among them in
a certain period of time (let us denote this time period
Tg , measured in frames, and called grouping period ), we
consider that they form a group. In practice, even a very
strong group (e.g. a married couple) can be apart during
some frames, when avoiding obstacles and/or other peo-
ple, but still keeping the group link. To cope with this
kind of situation, we consider that two individuals form
a group if they keep an intimate distance for at least a
fraction p of the grouping period Tg , where 0 ≤ p ≤ 1.
Formally, let us consider two individuals Ii (t) and
Ij (t) at frame t, and define a binary function:

1 if d(Ii (t), Ij (t)) ≤ Dintimate
g(i, j, t) = , (4)
0 otherwise
where d(Ii (t), Ij (t)) represents the distance between agents
Ii and Ij at frame t, and Dintimate = 0.5 meters is the
distance for intimate relationship, as defined in Table 1.
6 Julio Cezar Silveira Jacques Jr. et al.
Then, Ii (t) and Ij (t) are considered a group at frame t
if:
X t
g(i, j, k) ≥ pTg . (5)
k=t−(Tg −1)
Our experimental results indicated that 5 seconds
is enough time for group formation (leading to Tg =
150 frames in video sequences acquired at 30 frames per
second), and p = 0.8. However, such parameters can be
fine tuned for specific applications.
We also impose our grouping property to be transi-
tive, meaning that if Ii and Ij are grouped and Ij and Ik
are also grouped, than Ij and Ik must be also grouped.
In this case, Ii , Ij and Ik will be in the same group. Such
transitive property is important, because we can detect
large groups using only pairwise comparisons.
An example of group formation is shown in Fig. 5.
This Figure illustrates four frames of a video sequence,
and people belonging to two different groups according
to Equation (5) are highlighted. The “trail” behind each
tracked person shows his/her positions in the previous
frames.
After detecting the formation of a group, we want
to characterize it as voluntary or involuntary. We be-
lieve that there are two main causes for group formation:
people can form a group whether because they want to
(e.g. friends) or because they are forced to, due to lack of
space (e.g. exiting a crowded football stadium). It is rea-
sonable to think that such group characterization is re-
lated to the perceived personal spaces of the individuals
in the group: friends walking together in a non-crowded
environment may have plenty of PPS, but they choose to
stay close; on the other hand, if several people approach
a door at the same time, their PPS will be small, and
they have no choice but to stay close to other people.
To characterize a group as voluntary or involuntary,
we evaluate the PPSs of all individuals of the group
(which is equivalent to evaluating the comfort of individ-
uals). It is expected that, in voluntary groups, most peo-
ple should have large PPSs during the grouping time Tg .
However, some individuals may walk a little behind oth-
ers, and consequently their PPSs would be small, even if
they belong to a voluntary group. Thus, to detect volun-
tary group formations, we check if at least half of persons
in the group have sufficiently large PPSs.
More specifically, let us consider a group with N per-
sons, formed by individuals I1 , I2 , · · · , IN . Let ci (t) be
a binary “comfort” function with respect to the public
distance, that returns 1 if Equation (3) is satisfied with
Rc = 1.25, and 0 otherwise. Individual Ii (t) is said to be
comfortable in the previous Tg frames if:
t speed, so that the two persons walk side by side, at an in-
X
ci (t) ≥ pTg , (6)
k=t−(Tg −1)
where p is the same parameter used in Equation (5). The
group is qualified as voluntary at frame t if at least N/2
elements of the group satisfy Equation (6). Otherwise,
the group is characterized as involuntary.
4 Experimental Results
In this Section we show some experimental results of our
model, and indicate some higher-level events that could
be detected using group formation and classification. We
analyzed filmed sequences in real life as well as videos
generated by a crowd simulator (in the latter, we do not
apply the tracking algorithm, since the positions of all
agents are known). The reason to include data gener-
ated by a crowd simulator is the flexibility to generate
controlled situations with a variety of known parame-
ters and events, such as the generation of groups, their
size, people velocity, environment restrictions, locations
of strangle points, etc. Furthermore, tracking algorithms
may present erroneous results in very crowded video se-
quences.
Fig. 4 illustrates four frames of a filmed video se-
quence taken a few frames apart from each other. In
this scenario, a group of two persons move from right to
left, and a single person walks in the opposite direction
(PPSs are marked as brighter polygons). In particular,
the PPSs of the single person and the top person in the
group decrease as they approach each other, as well as
the distance between them until they cross each other.
Such relative distance is classified according to Hall [1]
as social in Fig. 4(a), personal in Fig. 4(b) and intimate
in Fig. 4(c). After they crossed, their relative distance
increases, as shown in Fig. 4(d). Since an intimate dis-
tance was kept for only a few frames (just before and
after crossing), the single person was not included in the
group. The two persons walking close to each other from
right to left are classified as a voluntary group after Tg
frames, since they enter the scene with intimate distance
and keep such distance during the whole sequence.
An example of group formation is shown in Fig. 5.
Fig. 5(a) illustrates two “clusters” of people that walk
together. After some frames, the cluster with four per-
sons is identified as a group (Fig. 5(b)), and a few frames
later the second cluster is also characterized as a group
(Fig. 5(c)), until they disappear from the viewing area of
the camera. It should be noticed that both groups were
detected as voluntary, since most individuals maintained
their PPS sufficiently large in at least a fraction p of the
previous Tg frames.
An example of dynamic group formation is illustrated
in Fig. 6. In this example, a person rapidly approaches
and reaches another person. Then, he/she reduces his/her
timate distance. After some time (grouping time), a vol-
untary group is detected. A higher-level interpretation
of this grouping behavior could be simply two friends
meeting, or maybe a kidnapping situation. A finite au-
tomaton could be easily implemented in the proposed
Understanding People Motion Using Voronoi Diagrams
model to detect sequences of events that could be re-
lated to suspect behavior, such as “approach” followed
by “voluntary group”.
The evaluation of group formation and classification
is a challenging task. In fact, the concept of grouping
is a little subjective, and the classification of a group
as voluntary or involuntary is difficult without a pri-
ori knowledge. To validate our results, a set of 20 short
movies was shown to 20 Computer Science undergradu-
ate and graduate students, and they had to answer for
each movie whether there was grouping or not. In case of
grouping, they had to chose between voluntary or invol-
untary. We analyzed their opinions by contrasting with
the results generated by our algorithm, and such com-
parison is discussed below.
According to our algorithm, most of short movies
(70%) contained from 1 to 5 voluntary groups, a few
(15%) contained involuntary groups (from 1 to 2), and
the remaining (15%) contained no group at all. From all
the groups (voluntary or involuntary) that were detected
by our algorithm, 74% was also pointed out as groups
by human subjects; 25% of answers provided by human
evaluators indicated a larger number of groups, and only
1% of answers indicated a smaller number of groups.
From the total of short movies where no group was de-
tected by our algorithm, 90% of them were in agreement
with human subjects, and 10% of the answers pointed
at least one group.
The evaluation of group classification as voluntary or
involuntary was also performed. From the total of volun-
tary groups detected by the method, 86% were also clas-
sified by humans; 3% detected more voluntary groups,
while 11% detected less voluntary groups. From the total
of involuntary groups, 68% were also detected as invol-
untary by evaluators; 31% of all answers pointed out a
larger number of groups, and the remaining 1% indi-
cated less involuntary groups. The worst case concern-
ing the classification of groups occurred in a short movie
where a group formed by two persons is climbing up a
small set of stairs, and a third person suddenly follows
the same path with higher velocity. There is no space
left for the third person to deviate from the group, so
he/she slows down and stays behind the group (during
enough time for grouping in our method) until there is
free space to accelerate again. In such case, the algorithm
detected only one voluntary group, since more than half
of the persons in the group keeps high PPS. On the other
hand, 70% of subjects defined this group as involuntary,
because they could perceive the temporary situation of
grouping behavior, as people group and ungroup during
the filmed sequence.
A fast and easy way to generate video sequences with
knowledge of ground truth is to use crowd simulators, in
which crowd and group events can be controlled and
handled in different virtual environments. At our re-
7
search Laboratory2 , we have developed specific studies
on crowd simulation, and developed algorithms that are
able to populate virtual environments [30–32].
In the first experiment, a 90 × 20 meters virtual rect-
angular environment was created. People arise in one
end of the largest dimension, and their goal is to move
to other end of the room along this dimension. The sim-
ulation was executed with a different number of agents
(100, 150 and 200, ten times each), and 80% of the agents
forming groups (in a total of 24 groups in all experi-
ments). When a certain agent is detected as belonging
to a group, a positive match is triggered; otherwise, a
negative match is triggered. A quantitative evaluation
of the proposed algorithm was performed in terms of
true positives (TP), true negatives (TN), false positives
(FP) and false negatives (FN). The evolution in time of
average TP, TN, FP and FN for simulations with differ-
ent number of agents is illustrated in Figs. 7(a)-(c), and
their averages for all the three experiments are shown in
Table 2. The number of groups as a function of time is
depicted in Fig. 7(d)3 , and the average number of groups
for the experiments with 100, 150 and 200 agents is 23.5,
22.9 and 24.8, respectively. It can be observed that some
false positives are detected because some agents that
were not supposed to form groups walk close to others
for enough time (grouping period), and a grouping event
is detected. If the grouping period Tg is increased, the
number of false positives tends to decrease (but actual
groups will be detected after a larger period of time).
In another simulated experiment, a virtual environ-
ment with length of 120 meters was created, containing a
narrow corridor in the middle with length of 20 meters.
Such environment was populated with 300 agents us-
ing our CSHuV Simulator [33], and their goal is to move
from left to right. Few of them are part of a family, mean-
ing that they know each other and try to keep together
during the whole simulation. Fig. 8 illustrates a snapshot
of the simulation, and it can be noticed that individuals
belonging to the same family were correctly identified as
voluntary groups (black dots) in non-crowded regions.
In this simulation, it is also possible to recognize large
involuntary groups in the corridor (dots in dark gray),
because the PPS of agents are drastically reduced due to
environment constraints in that region. Individuals that
were not grouped are marked as light gray dots. The sud-
den formation of large involuntary groups may indicate
the presence of spatial constraints or traffic jams that
compromise the flow of people, and could be included in
systems for higher-level event detection.
For the same simulation, we have also analyzed the
comfort level of agents. More specifically, we highlighted
individuals that do not have a personal comfort level
2
CROMOS Lab: http://www.inf.unisinos.br/
~cromoslab
3
In Fig. 7, frame 0 represents the first frame when a group
was detected.
8 Julio Cezar Silveira Jacques Jr. et al.
(i.e., agents for which Equation (3) is not satisfied for
Rc = 1.25 meters). Such individuals are marked as darker
dots in Fig. 9. It can be observed that few agents present
low level of comfort in regions A and C, opposed to the
situation in region B. As expected, the level of comfort
was drastically reduced in the corridor, since agents get
crowded. Some videos illustrating the proposed method
applied to filmed sequences and crowd simulators can be
accessed at http://www.inf.unisinos.br/~cromoslab/
voronoi_analysis.html.
5 Conclusions
Visual evaluation of results in filmed video sequences is
in agreement with expected, meaning that persons that
appear to know each other and walk together were iden-
tified as voluntary groups. In our controlled experiments
with crowd simulators, most generated families were cor-
rectly labeled as groups. Furthermore, large involuntary
groups were detected in strangle points, where unknown
people stay very close to each other for enough time.
As future work we intend to further explore group
detection and characterization for automatic detection
of higher level events, and also to obtain physical pa-
rameters from the sensed environment (such as obstacles
and strangle points). We also plan to study a variety of
filmed video sequences in order to adapt Hall’s distances
for other cultures.
6 Originality and Contribution
This paper presented a model to recognize individual
and groups information in video sequences. We described
a model using Dynamic Voronoi Diagrams (DVDs) in
which interpersonal distances and personal spaces (PSs)
of people are detected and analyzed across time. In ad-
dition, we proposed a new parameter: the perceived per-
sonal space (PPS), which we consider more adequate for
measuring comfort levels. A model for group formation
was proposed using sociological interpersonal distances
defined by Hall [1], and groups were classified as volun-
tary or involuntary based on the PPSs of the individuals
in the group during a certain time interval.
7 Acknowledgments
This work was developed in collaboration with HP Brazil
R&D. The authors would like to thank the anonymous
reviewers for their fruitful contributions to improve this
work.
References
1. E. T. Hall. The Silent Language. Doubleday Company,
Garden City, NY, 1959.
2. R. Sommer. Personal Space: The Behavioural Basis of
Design. Prentice Hall, Englewood CLiffs, New Jersey,
1969.
3. M. Valera and S.A. Velastin. Intelligent distributed
surveillance systems: a review. IEE Vision, Image and
Signal Processing, 152(2):192–204, April 2005.
4. T. B. Moeslund and A. Hilton and. V. Kruger. A survey
of advances in vision-based human motion capture and
analysis. Computer Vision and Image Understanding,
104, 2006.
5. C. Stauffer, W. Eric, and L. Grimson. Learning patterns
of activity using real-time tracking. IEEE Transactions
on Pattern Analysis and Machine Intelligence, 22(8):747
– 757, August 2000.
6. H. Buxton and S. Gong. Advanced visual surveillance
using bayesian networks. In IEEE International Con-
ference on Computer Vision, Cambridge, Massachusetts,
June 1995.
7. R. Hosie, S. Venkatesh, and G.A.W. West. Classify-
ing and detecting group behaviour from visual surveil-
lance data. In IEEE International Conference on Pattern
Recognition, pages Vol I: 602–604, 1998.
8. N. M. Oliver, B. Rosario, and A. P. Pentland. A bayesian
computer vision system for modeling human interactions.
IEEE Transactions on Pattern Analysis and Machine In-
telligence, 22(8):831–843, August 2000.
9. Y. Du, G. Chen, W. Xu, and Y. Li. Recognizing in-
teraction activities using dynamic bayesian network. In
IEEE International Conference on Pattern Recognition,
volume 1, pages 618–621, August 2006.
10. S. Gong and T. Xiang. Recognition of group activities
using dynamic probabilistic networks. In IEEE Interna-
tional Conference on Computer Vision, page 742, Wash-
ington, DC, USA, 2003. IEEE Computer Society.
11. T. Xiang and S. Gong. Beyond tracking: Modelling activ-
ity and understanding behaviour. International Journal
of Computer Vision, 67(1):21–51, 2006.
12. X. H. Liu and C. S. Chua. Multi-agent activity recogni-
tion using observation decomposed hidden markov mod-
els. Image and Vision Computing, 24(2):166–175, Febru-
ary 2006.
13. L. M. Fuentes and A. Velastin. Tracking-based event
detection for CCTV systems. Pattern Analysis and Ap-
plications, 7(4):356–364, 2004.
14. L. M. Fuentes and S. A. Velastin. People tracking in
surveillance applications. Image and Vision Computing,
24(11):1165–1171, November 2006.
15. I. Haritaoglu, D. Harwood, and L. S. Davis. W4: Real-
time surveillance of people and their activities. IEEE
Transactions on Pattern Analysis and Machine Intelli-
gence, 22(8):809–830, August 2000.
16. A. M. Elgammal, R. Duraiswami, D. Harwood, and
L. S. Davis. Background and foreground modeling us-
ing nonparametric kernel density estimation for visual
surveillance. Proceeedings of the IEEE, 90(7):1151–1163,
2002.
17. F. H. Cheng and Y. L. Chen. Real time multiple ob-
jects tracking and identification based on discrete wavelet
transform. Pattern Recognition, 39(6):1126–1139, June
2006.
18. S.-Y. Chien, S.-Y. Ma, and L.-G. Chen. Efficient moving
object segmentation algorithm using background regis-
Understanding People Motion Using Voronoi Diagrams 9

tration technique. IEEE Transactions on Circuits and

Systems for Video Technology, 12(7):577–586, 2002.
19. R. Cucchiara, C. Grana, M. Piccardi, and A. Prati. De-
tecting moving objects, ghosts, and shadows in video
streams. IEEE Transactions on Pattern Analysis and
Machine Intelligence, 25(10):1337–1342, October 2003.
20. H. Ning, T. Tan, L. Wang, and W. Hu. People tracking
based on motion model and motion constraints with au-
tomatic initialization. Pattern Recognition, 37(7):1423–
1440, July 2004.
21. D. Xu, X. Li, Z. Liu, and Y. Yuan. Cast shadow detec-
tion in video segmentation. Pattern Recognition Letters,
26(1):5–26, 2005.
22. Y. Wang, T. Tan, K. F. Loe, and J. K. Wu. A prob-
abilistic approach for foreground and shadow segmenta-
tion in monocular image sequences. Pattern Recognition,
38(11):1937–1946, November 2005.
23. Y. L. Tian, M. Lu, and A. Hampapur. Robust and
efficient foreground analysis for real-time video surveil-
lance. In IEEE Computer Vision and Pattern Recogni-
tion, pages I: 1182–1187, 2005.
24. J. C. S. Jacques Jr., C. R. Jung, and S. R. Musse. A
background subtraction model adapted to illumination
changes. In IEEE International Conference on Image
Processing, pages 1817–1820. IEEE Press, 2006.
25. J. Martin and J. L. Crowley. Comparison of correlation
techniques. In Conference on Intelligent Autonomous
Systems, Karsluhe, Germany, March 1995.
26. F. Porikli, O. Tuzel, and P. Meer. Covariance tracking
using model update based on lie algebra. In IEEE Com-
puter Vision and Pattern Recognition, pages I: 728–735,
2006.
27. F. Aurenhammer. Voronoi diagrams: a survey of a funda-
mental geometric data structure. ACM Computing Sur-
veys, 23(3):345–405, 1991.
28. D. G. Vaughan, T. Asbury, and P. R. Riordan-Eva. Gen-
eral Ophtalmology. Lange Medical Publications, New
York, 1995.
29. E. Fuchs. Text-book of ophthalmology. D. Appleton, 1898.
30. A. Braun, B. E. J. Bodmann, L. P. L. Oliveira, and S. R.
Musse. Modelling individual behavior in crowd simula-
tion. In Proceedings of Computer Animation and Social
Agents 2003, pages 143–148, New Brunswick, USA, 2003.
IEEE Computer Society.
31. L. M. Barros, A. T. da Silva, and S. R. Musse. Petrosim:
An architecture to manage virtual crowds in panic situa-
tions. In Proceedings of Computer Animation and Social
Agents 2004, pages 111–120, Geneva, Switzerland, 2004.
32. N. Courty and S. R. Musse. Simulation of large crowds in
emergency situations including gaseous phenomena. In
Proceedings of Computer Graphics International 2005,
pages 206–212, Stony Brook, NY, 2005.
33. A. Braun, B. E. Bodmann, and S. R. Musse. Crowd
simulation in emergency situations. In Short Paper in
ACM Symposium on Computer Animation 2004, Greno-
ble, France, 2004.
10 Julio Cezar Silveira Jacques Jr. et al.

d1
d2
d3

Fig. 2 Voronoi Diagram and distances from one person to

his/her neighbors.

Fig. 3 Voronoi polygons and Perceived Personal Space

(PPSs, shown as translucent regions).
Understanding People Motion Using Voronoi Diagrams 11

Hall’s Classification Approximate distance Kind of interaction

Intimate distance
Personal distance
Social distance
Public distance
Table 1 Hall’s classification for Personal Space.
up to 0.5 meters Comforting, threatening
0.5 to 1.25 meters Conversation between friends
1.25 to 3.5 meters Impersonal business dealings
more than 3.5 meters Addressing a crowd

(a) (b) (c)

P N P N P N
T 77.3 7.5 T 119.3 13.5 T 158.2 12.9
F 13.5 1.7 F 16.5 0.7 F 27.1 0.8
Table 2 Average number of true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN) for the
simulated scenarios with (a) 100, (b) 150 and (c) 200 agents.

(a) (b) (c)

Fig. 1 (a) Frame of a video sequence, with detected center heads and correlation templates. (b) Result of background
subtraction. (c) Distance Transform and its maxima, that are used to detect head centers.

(a) (b) (c) (d)

Fig. 4 Illustration of PPS and evolution of interpersonal distances

(a) (b) (c) (d)

Fig. 5 Groups recognized at the filmed sequence.

12 Julio Cezar Silveira Jacques Jr. et al.

(a) (b) (c) (d)

Fig. 6 Example of dynamic formation of a voluntary group.

(a) (b) (c) (d)

120 160 35
80

70 140 30
100
60 120 25
TP TP

number of groups
TP
number of agents

number of agents
number of agents

TN 80 TN
50 100 TN
FP FP FP 20
FN FN FN
40 60 80
15
30 60
40
10
20 40
20 5 100 agents
10 20 150 agents
200 agents
0 0 0 0
0 500 1000 1500 0 500 1000 1500 0 500 1000 1500 0 500 1000 1500
frame frame frame frame

Fig. 7 Evolution in time of true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN) for group
detection in a simulated environment using (a) 100 agents (b) 150 agents and (c) 200 agents. (d) Shows the temporal evolution
of the number of groups for the three experiments.

Fig. 8 Voluntary and involuntary groups recognized in the Fig. 9 Individual comfort highlighted in the “narrow cor-
“narrow corridor” simulation sequence. ridor” simulation sequence.