Académique Documents
Professionnel Documents
Culture Documents
1
ing the motion
ow elds seen by multiple cameras, an where ! and t denote the 3D angular velocity and
algorithm without solving the spatial point correspon- translational velocity of the ego-motion. This 3D rela-
dence problem is proposed to estimate 3D ego-motion tive motion can be expressed with respect to the k-th
parameters of the moving object. camera coordinate system C : k
P_ = ! P t ; (3)
2 Ego-Motion Estimation Algorithms k k k k
where
Let us consider an arbitrary multi-camera congura-
tion shown in Fig. 1. Without loss of generality, their ! R ! and t R [(! b ) + t] : (4)
k
T
k k
T
k k
attached to the moving object is used to describe the image point p of the 3D point P is
k k
_
uk3 p_ = (! p ) PP p P1 t :
k k (6)
k
zk
k k
zk zk
u13 Ok
e3 u k1 Applying cross product () to both sides of Eq. (6)
e2 bk by p , we have
k
uk2
O1
b1
O
e1 p [p_ + (! p )] = P1 (p t ): (7)
k k k k k k
zk
k k (8) k k k
Figure 1: An arbitrary conguration of multiple cameras. Since we want to estimate ! and t using the observa-
tions from all the K cameras, the above fundamental
At any time instance, we can compute a
ow eld for equation is re-expressed in terms of ! and t by using
each of the K cameras. The k-th
ow eld is composed Eq. (4):
of N image feature points and their corresponding
ow
k h i
vectors. Our goal is to compute the 3D ego-motion R fp p_ + (R ! p ) g (! b + t) = 0: (9)
T
A point P in 3D space can be respectively denoted It is Eq. (9) that we use to determine the 3D motion
by P and P in the two coordinate systems C and C .
k g k parameters without solving the point correspondence
These coordinate vectors must satisfy problem.
2 3 There are N
ow vectors associated with the k -th
P camera. We use p and p_ to represent the i-th point
k
P4 P 5=R P +b :
xk
yk (1)
k k k
ki
and
ow associated with the k -th camera. Eq. (9) can
ki
P zk
be rewritten to:
Due to the 3D ego-motion, a point P in the static envi- m (h + t) = 0;
T
(10)
ronment will have an instantaneous 3D motion relative ki k
2
Because the optimal solution !^ can not be obtained
h !b : (12) by searching a close-to-singular error function, we have
to dene a new error function to solve these degenerate
k k
t = M 1c;
K
M m m and c
T
m m h: T
X Xk
K N
J20 (!; t ) = t ( m m )t
ki ki ki ki k
T T
k =1 i=1 k =1 i=1 n n ki ki n
(15) k =1 i=1
In the following, the matrix M is sometimes written = t Mt T
= (t )2 = ; (19)
By substituting Eq. (14) into the t in Eq. (13), we n
have a new error function J1 which only depends on where the matrix M is dened in Eq. (15), and is
the unknown rotation parameter !: an eigenvalue of M (!).
Given an estimate of !, the best estimate of t
X Xk
n
J1 (!) c T
M c+ 1
(m
K N
T
h): 2
(16) should be the eigenvector of M (!) corresponding to
ki k
the smallest eigenvalue. If we substitute the above op-
k =1 i=1
timal t into J20 , a new error function J2 which only
n
Therefore the optimal estimate of ! (denoted by depends on the unknown ! can be dened as
!^ ) is the one that minimizes the error function J1 (!). J2 (!) the smallest eigenvalue of M (!): (20)
Once we have !^ , the optimal estimate of t, denoted ^t,
can be easily obtained by using Eqs. (14) and (15). Here, the 3-by-3 matrix M is a function of !.
Therefore, the optimal estimate of !, denoted by !^ ,
Algorithm 2: Degenerate Case is the one which minimizes the error function J2 (!),
When M (!) is not full-rank, Eq. (14) fails to solve and the optimal estimate of t (denoted by ^t ) is n n
the corresponding t, which also causes a singularity the eigenvector of M (^!) corresponding to the smallest
when calculating J1 . eigenvalue.
By considering Eqs. (10) and (14), we know the
following conditions will make the matrix M (!) have 3 Experimental Results
a close-to-zero eigenvalue. They include:
This section will show some results of real experi-
(1) ! 0, i.e., there is almost no rotational motion. ments. We use a binocular head (referred to as the IIS
head) to simulate a moving object with two cameras
true
(2) All of the h 's are very close to zero. In this case,
k
mounted on it. The IIS head is built for experiments
the whole multi-camera vision system is just like a of active vision, which has four revolute joints and two
monocular vision system with K separate view elds, prismatic joints, as shown in Fig. 2. The two joints
which provides a much larger eld of view. on top of the IIS head are for camera verge or gazing.
The next two joints below them are for tilting and pan-
(3) All of the h 's are parallel to t
k true . ning the stereo cameras. All of the above four joints
3
Table 1: True motion parameters used in Experiment 1.
rotation translation
! (deg/frame) direction mag.(mm)
!x ! y ! z t xn tyn tzn ktk
0.00 0.00 0.00 -0.017 .045 1.00 20.00
LCCS
yr Table 3: Translational parameters estimated in Experi-
Left Camera
xr ment 1. (^t ; t n ) is dened as the angle between ^t
n;true n
and t .
t (translational direction)
n;true
90
o
n error
(^t ; t
RCCS
Or
zr
t^
xn t^
yn t^
zn n n;true )
Global Coordinate System = LCCS Right Camera Both .040 .054 1.00 3:30
Left .062 .049 1.00 4:54
Figure 3: The coordinate systems and camera congura- Right -0.99 -0.051 .10 83:39
tion of the real experiments.
4
Table 4: True motion parameters used in Experiment 2.
Table 6: Translational parameters estimated in Experi-
ment 2. (^t ; t
n ) is dened as the angle between ^t
n;true n
3.2 Experiment 2
(a) (b)
In experiment 2, we let the IIS head pan with a small
angle. Table 4 is the true motion parameters used in this
experiment. Again, we estimate the ego-motion by using
the left camera only, the right camera only, and both the
left and right camera, respectively. The scenes viewed from
the left and right cameras are the same as Figs. 4(a) and
4(b). The
ow elds observed from left and right cameras
are shown in Fig. 5(a) and 5(b), respectively.
The result of this experiment is given in Tables 5 and
6. As expected, the performance of using both the left (c) (d)
and right cameras is the best. The ambiguity problem Figure 4: The images and the
ow elds used in experiment
occurs again when using the right camera only, which 1: (a) The scene viewed from the left camera. (b) The
is because the depth of the scene viewed from the right scene viewed from the right camera. (c) The optical
ow
camera is as far as ve meters and hence the optical eld obtained from the left camera. (d) The optical
ow
ow eld looks very similar to the one caused by small eld obtained from the right camera. The
ow vectors in
pure translation. Notice that the errors of the trans- the gures are enlarged by a factor of two.
lational direction are larger than the ones obtained in
Experiment 1. This is because the signals of transla-
tion in this experiment are very small. Whenever there
is noise, the signals of translation will be seriously cor-
rupted, and it is hard to estimate the translational pa-
rameters with high accuracy.
4 Conclusion
In this paper, we propose a method for ego-motion
estimation using a multiple-camera vision system. By (a) (b)
combining the information contained in the multiple Figure 5: The
ow elds used in experiment 2: (a) The
optical
ows observed from dierent cameras, some optical
ow eld obtained from the left camera. (b) The
ambiguity problems can be avoided and the accuracy optical
ow eld obtained from the right camera. The
ow
can be improved. Our algorithm considers two cases vectors in the gures are also enlarged by a factor of two.
separately: non-degenerate case and degenerate case.
5
One potential application of our multiple-camera ap- [8] M. K. Leung and T. S. Huang, \Estimating Three-
proach is the \inside-out" (or \outward looking") head Dimensional Vehicle Motion in an Outdoor Scene Us-
tracker for virtual reality. The current outward look- ing Stereo Image Sequences," International Journal of
ing head tracker requires structured environments, e.g. Imaging Systems and Technology, Vol. 4, pp. 80-97,
regular pattern in the ceiling. Our approach does not 1992.
require specially-designed environment, as long as the [9] L. Li and J. H. Duncan, \3D Translational Motion and
environment have enough features for computing opti- Structure from Binocular Image Flows," IEEE Trans-
cal
ow. actions on Pattern Analysis and Machine Intelligence,
Dierent camera congurations have dierent per- Vol. 15, pp. 739-745, 1993.
formance on ego-motion estimation. In this paper, we [10] L. G. Liou and Y. P. Hung, \Vehicle-Type Ego-Motion
have not analyzed the performance between dierent Estimation Using Multiple Cameras," Proceedings of
camera congurations. Some analysis on nding the Asian Conference on Computer Vision, pp. 166-170,
optimal camera conguration can be found in another Singapore, 1995.
paper [10]. [11] Y. Liu and T. S. Huang, \Vehicle-Type Motion Esti-
mation from Multi-frame Images," IEEE Transactions
Acknowledgements on Pattern Analysis and Machine Intelligence, Vol. 15,
pp. 802-808, 1993.
This work was supported in part by the National [12] K. Prazdny, \Determining the Instantaneous Direction
Science Council of Taiwan, under Grants NSC 86-2745- of Motion from Optical Flow Generated by a Curvilin-
E-001-007, NSC 86-2212-E-002-025, and by Mechanical early Moving Observer," Computer Graphics and Im-
Industry Research Laboratories, Industrial Technology age Processing, Vol. 17, pp. 238-248, 1981.
Research Institute, under Grant MIRL 863K67BN2. [13] T. Vieville, E. Clergue, and P. E. D. S. Facao, \Com-
putation of Ego-Motion and Structure from Visual and
Inertial Sensors Using the Vertical Cue," Proceedings
References of International Conference on Computer Vision, pp.
591-598, Berlin, Germany, 1993.
[1] G. Adiv, \Inherent Ambiguities in Recovering 3D Mo- [14] J. Weng, P. Cohen, and N. Rebibo, \Motion and Struc-
tion and Structure from a Noisy Flow Field," IEEE ture Estimation from Stereo Image Sequence," IEEE
Transactions on Pattern Analysis and Machine Intel- Transactions on Robotics and Automation, Vol. 8, pp.
ligence, Vol. 11, pp. 477-489, 1989. 362-382, 1992.
[2] N. Ayache and F. Lustman, \Trinocular Stereo Vision
for Robotics," IEEE Transactions on Pattern Analysis
and Machine Intelligence, Vol. 13, pp. 73-85, 1991.
[3] T. Broida and R. Chellappa, \Estimation of Object
Motion Parameters from Noisy Images," IEEE Trans-
actions on Pattern Analysis and Machine Intelligence,
Vol. 8, pp. 90-99, 1986.
[4] T. Broida and R. Chellappa, \Estimating the Kine-
matics and Structure of a Rigid Object from a Se-
quence of Monocular Images," IEEE Transactions on
Pattern Analysis and Machine Intelligence, Vol. 13,
pp. 497-512, 1991.
[5] W. Burger and B. Bhanu, \Estimating 3D Egomotion
from Perspective Image Sequences," IEEE Transac-
tions on Pattern Analysis and Machine Intelligence,
Vol. 12, pp. 1040-1058, 1990.
[6] D. J. Heeger and A. D. Jepson, \Subspace Methods
for Recovering Rigid Motion I: Algorithm and Im-
plementation," International Journal of Computer Vi-
sion, Vol. 7, pp. 95-117, 1992.
[7] R. Hummel and V. Sundareswaran, \Motion Param-
eter Estimation from Global Flow Field Data," IEEE
Transactions on Pattern Analysis and Machine Intel-
ligence, Vol. 15, pp. 459-476, 1993.