COMPUTER VISION: THREE-VIEW GEOMETRY

IIT Kharagpur
Computer Science and Engineering,
Indian Institute of Technology
Kharagpur.
1 / 85
Trifocal tensor Three-View Geometry
Encapsulates the projective geometry relations between 3 views.
Independent of scene structure.
Analogous to fundamental matrix.
Depends only on the relative pose between the three cameras
and the internal parameters of the cameras.
Can be uniquely determined by

Camera matrices
OR

Point correspondences between the images.

USAGE OF TRIFOCAL TENSOR
Transfer points from a correspondence in two views to the
corresponding point in a third view.
Transfer lines .....
2 / 85
Trifocal tensor Three-View Geometry
WHAT WE ARE INTERESTED IN?
Homography between two of the views induced by a plane
back-projected from a line in the other view.
Relations between image correspondences between points and
lines.
Retrieval of the fundamental matrices.
Retrieval of the camera matrices.
3 / 85
Trifocal tensor Three-View Geometry
Consider the set of corresponding lines l l

.
The planes back-projected from l l

are incident on the space

line L.
This is the GEOMETRIC INCIDENCE RELATION for corresponding
lines.
4 / 85
Trifocal tensor Three-View Geometry
P = [ I | 0] P

= [A | a
4
] P

= [B | b
4
]
The camera matrices for the 3 views are taken as P P

.
a
4
and b
4
are the epipoles in views 2
n
and 3
r
, arising from the 1
st
camera.
These epipoles are denoted as e

and e

.
e

= P

C e

= P

C
C is the center of the 1
st
camera.
The back-projected planes can be written as:
= P

l =

l
0

= P

4
l

= P

4
l

5 / 85
Trifocal tensor Three-View Geometry
INTERSECTION OF PLANES:
= P

l =

l
0

= P

4
l

= P

4
l

All the 3 planes intersect in a common line.

Algebraically it means that the 4 3 matrix M = [

] has
rank 2.
Points on a line can be represented as: X = X

+ X
2
where X

and X
2
are linearly independent.
For the line of intersection L of the 3 planes we have

X =

X =

X = 0
Given M = [

] , we have M

X = 0.
Hence M

= M

X
2
= 0
6 / 85
Trifocal tensor Three-View Geometry
INTERSECTION OF PLANES:
= P

l =

l
0

= P

4
l

= P

4
l

M
43
= [m

m
2
m
3
] =

l A

0 a

4
l

4
l

= M

X
2
= 0
The condition M

= M

X
2
= 0 for two linearly independent
vectors X

and X
2
implies that M has a two dimensional null
space.
This implies there is linear dependence on the columns of M, i.e.
m

= m
2
+ m
3
Applying this to M gives: 0 = a

4
l

+ b

4
l

.
Thus = kb

4
l

) and = ka

4
l

)
7 / 85
Trifocal tensor Three-View Geometry
INTERSECTION OF PLANES:
= P

l =

l
0

= P

4
l

= P

4
l

M
43
= [m

m
2
m
3
] =

l A

0 a

4
l

4
l

= M

X
2
= 0
Applying values of = kb

4
l

) and = ka

4
l

)
l = A

+ B

= b

4
l

)A

4
l

)B

= l

b
4
)A

a
4
)B

l
i
= l

b
4
a

i
)l

a
4
b

i
)l

= l

a
i
b

4
)l

a
4
b

i
)l

8 / 85
Trifocal tensor Three-View Geometry
INTERSECTION OF PLANES:
= P

l =

l
0

= P

4
l

= P

4
l

M
43
= [m

m
2
m
3
] =

l A

0 a

4
l

4
l

= M

X
2
= 0
l
i
= l

a
i
b

4
)l

a
4
b

i
)l

T
i
= a
i
b

4
a
4
b

i
l
i
= l

T
i
l

The set of 3 matrices {T

1
T
2
T
3
} constitute the trifocal tensor.
9 / 85
Trifocal tensor Three-View Geometry
l
i
= l

a
i
b

4
)l

a
4
b

i
)l

T
i
= a
i
b

4
a
4
b

i
l
i
= l

T
i
l

The set of 3 matrices {T

1
T
2
T
3
} constitute the trifocal tensor.
The ensemble of matrices [T
1
T
2
T
3
] can be denoted as [T
i
] .
l

= l

[T
1
T
2
T
3
] l

where
l

[T
1
T
2
T
3
] l

represents l

T
1
l

T
2
l

T
3
l

)
10 / 85
Trifocal tensor Three-View Geometry
l

= l

[T
1
T
2
T
3
] l

= l

[T
i
] l

= l

[T

i
] l

= l

[T

i
] l

The three tensors [T

i
] [T

i
] [T

i
] exist, but are distinct.
All three tensors can be computed from any one of them.
Matrix elements [T
i
] are independent of the form of cameras.
The simple formula for computing the trifocal tensor
T
i
= a
i
b

4
a
4
b

i
is valid only for chosen canonical cameras:
P = [ I | 0] P

= [A | a
4
] P

= [B | b
4
]
11 / 85 12 / 85
Trifocal tensor Three-View Geometry
DEGREES OF FREEDOM
l

= l

[T
1
T
2
T
3
] l

= l

[T
i
] l

= l

[T

i
] l

= l

[T

i
] l

The trifocal tensor consists of three 3 3 matrices: [T

i
] [T

i
] [T

i
] .
Three 3 3 matrices have 27 dofs = 26 independent ratios.
Three camera matrices have 11 dofs each. Hence 33 dofs.
The projective world frame is not to be specied for trifocal tensor.
Hence 15 dofs can be subtracted from 33.
We are left with 33-15 = 18 dofs
Number of independent algebraic constraints satised by the
trifocal tensor: 26-18 = 8.
13 / 85
Homographies induced by a plane Three-View Geometry
14 / 85
Homographies induced by a plane 3-View Geometry
Consider a 3D line L and its projection as image plane lines
l l

. The trifocal tensor satises the line incidence relation:

l
i
= l

T
i
l

A line in the 2
n
view can be back-projected to a plane in 3-space.
This plane induces a homography between the 1
st
and the 3
r
views.
x

= Hx l

= H

l l = H

15 / 85
Homographies induced by a plane 3-View Geometry
Using l
i
= l

T
i
l

and l = H

we get
H = [h
1
h
2
h
3
] with h
i
= T

i
l

This H is the homography H

13
between the 1
st
and the 3
r
views
induced by the line l

in the 2
n
image.
H
13
l

) =

1
T

2
T

Likewise the homography between the 1

st
and the 2
n
view,
induced by a line in the 3
r
view is given as:
H
12
l

) = [T
1
T
2
T
3
] l

16 / 85
Point and line Incidence 3-View Geometry
The trifocal tensor relation l

= l

[T
1
T
2
T
3
] l

involves
homogeneous quantities and holds only up to scale.
To make this relation independent of the scale factor we can take
the cross product:

[T
1
T
2
T
3
] l

[l]

= 0

Likewise we can have:

1
T

2
T

[l]

= 0

We shall discuss 3 types of INCIDENCE RELATIONS:

Point-line-line correspondence
Point-line-point correspondence
3-point correspondence
17 / 85
Point-line-line Relationship 3-View Geometry
A 3D line L maps to l

and l

in the 2
n
and 3
r
images and to a
line passing through x in the 1
st
image.
The point x on the line must satisfy x

l =

i
x
i
l
i
= 0
18 / 85
Point-line-line Relationship 3-View Geometry
The point x on the line must satisfy x

l =

i
x
i
l
i
= 0
Since l
i
= l

T
i
l

, we have

i
x
i
l

T
i
l

= 0, i.e.
l

i
x
i
T
i

= 0
where

i
x
i
T
i
) is simply a 3 3 matrix.
There exists a 3D
point X which maps
to x in the 1
st
image
and to points on the
lines l

and l

in the
2
n
and 3
r
images.
19 / 85
Point-line-point Relationship 3-View Geometry
The 3D point X maps to points x and x

on 1
st
and 3
r
images and
to a point on the line l

in the 2
n
image.
20 / 85
Point-line-point Relationship 3-View Geometry
The 3D point X maps to points x and x

on 1
st
and 3
r
images and to a
point on the line l

in the 2
n
image.
x

= H
13
l

) x =

1
l

2
l

3
l

x =

i
x
i
T

i
_

_
l

This is valid for any line l

passing through x

in the 2
n
image.
The homogeneous scale factor may be eliminated by post
multiplying the transpose of both sides by [x

[x

= l

i
x
i
T
i

[x

= 0

21 / 85
Point-point-point Relationship 3-View Geometry
[x

i
x
i
T
i
_

_
[x

= 0
33
22 / 85
Point-point-point Relationship 3-View Geometry
[x

i
x
i
T
i

[x

= 0
33
How?
Any line l

passing through x

can be written as:

l

= x

= [x

for some point y

on l

By the point-line-point relation we have:

l

i
x
i
T
i
_

_
[x

= y

[x

i
x
i
T
i
_

_
[x

= 0

This is true for all lines l

through x

, hence independent of y

.
[x

i
x
i
T
i

[x

= 0
33
23 / 85
Summary of relations 3-View Geometry
Line-line-line: l l

[T
1
T
2
T
3
] l

= l

[T
1
T
2
T
3
] l

[l]

= 0

Point-line-line: x l

i
x
i
T
i

= 0
24 / 85
Summary of relations 3-View Geometry
Point-line-point: x l

i
x
i
T
i

[x

= 0

Point-point-line: x x

[x

i
x
i
T
i

= 0
Point-point-point: x x

[x

i
x
i
T
i

[x

= 0
33
25 / 85
Point to note Caution
The lines l l

are projections of 3D line L.

The points x x

are projections of 3D point X.

l l

Implies that there exists a 3D line L which projects to l l

in the
1
st
, 2
n
, 3
r
views respectively.
x x

Implies that there exists a 3D line X which projects to x x

in
the 1
st
, 2
n
, 3
r
views respectively.
26 / 85
Point to note Caution
The lines l l

are projections of 3D line L.

The points x x

are projections of 3D point X.

x l

Implies that there exists a 3D line L which projects to l

in the
2
n
, 3
r
views, and to a line passing through x in the 1
st
view. The
3D point X corresponding to x may or may not lie on the 3D line L.
x l

Implies that there exists a 3D point X which projects to x x

in the
1
st
and 3
r
views and to a point lying on line l

in the 1
st
view. The
line l

is a projection of some 3D line L.

We cannot say whether X lies on L
27 / 85
Incidence of X and L Caution
Entities satisfying a tensor relation do not guarantee incidence in
3-space.
Incidence of L and X is not guaranteed for x l

relation
28 / 85
Three-view Geometry
Next

Extracting Epipolar Lines

Extracting Fundamental matrix
Retrieving Camera matrices
29 / 85
Epipolar Geometry 3-View Geometry
Consider the plane

back-projected from l

. If this passes
through the 1
st
camera center C then it is the epipolar plane for the
1
st
and 2
n
views.
Suppose X is a point on

. The image of this point is x x

in the
two views.
30 / 85
Epipolar Geometry 3-View Geometry
A plane

back-projected from a line l

in the 3
r
image will
intersect the plane

in a line L.
The ray back-projected from point x must intersect this 3D line L
We shall use the correspondence x l

31 / 85
Epipolar Geometry 3-View Geometry
Point-line-line: x l

i
x
i
T
i

= 0
If the 3D line L corresponding to back-projection of l

and l

lies
on the epipolar plane

for the 1
st
and 2
n
views, then the above
relation will be satised for any line l

. Hence:
l

i
x
i
T
i

= 0

This is valid even when the roles of l

and l

are reversed.

i
x
i
T
i

= 0

32 / 85
Epipolar Geometry 3-View Geometry
l

i
x
i
T
i

= 0

i
x
i
T
i

= 0

The two relations indicate that the epipolar lines can be computed
as the left and right null vectors of the matrix

i
x
i
T
i

.
The epipole can be computed as the intersection of 3 different
epipolar lines. Choose 3 points x
x

i
x
i
T
i
1 0 0)

T
1
0 1 0)

T
2
0 0 1)

T
3
The left null spaces of T
1
T
2
T
3
would give the 3 epipolar lines.
The epipole e

in the 2
n
image is the common intersection of
these epipolar lines
33 / 85
Epipolar Geometry 3-View Geometry
The epipole e

The common intersection of

lines represented by left null
vectors of the T
i
s
The epipole e

The common intersection of

lines represented by the right
null vectors of the T
i
s.
34 / 85
Algebraic Properties T
i
matrices
The left null vector of T
i
is l

i
= e

a
i
This gives the epipolar line in the 2
n
view for the points
x = 1 0 0)

0 1 0)

0 0 1)

as i = 1 2 3
The epipole e

is the common intersection of the epipolar lines l

i
for i = 1 2 3
The right-null vector of T
i
is l

i
= e

b
i
This gives the epipolar line in the 3
r
view for the points
x = 1 0 0)

0 1 0)

0 0 1)

as i = 1 2 3
The epipole e

is the common intersection of the epipolar lines l

i
for i = 1 2 3
35 / 85
Algebraic Properties T
i
matrices
Each matrix T
i
has rank 2. This is because T
i
= a
i
e

i
is
the sum of two outer products.
The sum of the matrices

i
x
i
T
i

also has rank 2.

The left null vector of the sum

i
x
i
T
i

is the epipolar line l

of x
in the 2
n
view.
The right null vector of the sum

i
x
i
T
i

is the epipolar line l

of
x in the 3
r
view.
36 / 85
Three-view Geometry
Next

Extracting Epipolar Lines

Extracting Fundamental matrix
Retrieving Camera matrices
37 / 85
Extracting Fundamental matrices T
i
matrices
Consider a point x in the 1
st
view.
A line l

in the 3
r
view induces a homography H
12
from the 1
st
to
the 2
n
view as given by: refer slide 16
x

= [T
1
T
2
T
3
] l

) x
The epipolar line corresponding to x is the line joining x

to the
epipole e

.
l

= [e

[T
1
T
2
T
3
] l

) x
Hence
F
21
= [e

[T
1
T
2
T
3
] l

F
21
is the F matrix between 1
st
and 2
n
views.
F
31
is the F matrix between the 1
st
and 3
r
views.
38 / 85
Extracting Fundamental matrices T
i
matrices
F
21
= [e

[T
1
T
2
T
3
] l

This formula for F

21
is valid for any choice of l

. However we must
avoid the degenerate case when T
i
l

= 0, i.e. l

lies in the null

space of any of the T
i
.
The right-null vector of each T
i
is the epipolar line l

i
= e

b
i
.
Hence if we choose the vector e

for l

, then we are guaranteed

that l

will be perpendicular to the right null space of each T

i
.
F
21
= [e

[T
1
T
2
T
3
] e

Likewise we have the formula for F

31
F
31
= [e

1
T

2
T

39 / 85
Three-view Geometry
Next

Extracting Epipolar Lines

Extracting Fundamental matrix
Retrieving Camera matrices
40 / 85
Retrieving camera matrices P P

matrices
Trifocal tensor is independent of the 3-D projective
transformations.
Hence the camera matrices can be retrieved only up to a
projective ambiguity.
The rst camera can be chosen as P = [ I | 0]
Since F
21
= [e

[T
1
T
2
T
3
] e

is known, the second camera can

be taken as:
P

= [[T
1
T
2
T
3
] e

| e

]
RECALL
If the F matrix can be written as F = [[m]

M] then the cameras can be

chosen as P = [ I | 0] and P

= [M | m]
41 / 85
Retrieving camera matrices P P

matrices
P = [ I | 0] P

= [[T
1
T
2
T
3
] e

| e

]
We choose P P

to be consistent with fundamental matrix F

21
.
Having chosen P P

, the two cameras P P

have now established

some projective world frame.
The third camera P

must be consistent with this projective frame.

P = [ I | 0] P

= [A | a
4
] P

= [B | b
4
]
Since we have chosen P

, each a
i
= T
i
e

and a
4
= e

Also, since F
31
= [e

1
T

2
T

, we know b
4
= e

How do we choose P

= [B | b
4
] ?
42 / 85
Retrieving camera matrices P P

matrices
P = [ I | 0] P

= [A | a
4
] P

= [B | b
4
]
We know that: a
i
= T
i
e

, a
4
= e

and b
4
= e

what is b
i
?
Substituting this in the trifocal tensor relation
T
i
= a
i
b

4
a
4
b

i
we get T
i
= T
i
e

i
This gives e

i
= T
i
e

I )
Multiplying on the left by e

we get: e

i
= e

T
i
e

I )
Normalizing such that e

= 1 gives b

i
= e

T
i
e

I )
Taking the transpose of both sides: b
i
= e

I ) T

i
e

43 / 85
Retrieving camera matrices P P

matrices
P = [ I | 0] P

= [A | a
4
] P

= [B | b
4
]
We found
b
i
=

i
e

and b
4
= e

Hence
P

i
e

P = [ I | 0] P

[T
1
T
2
T
3
] e

All these 3 cameras are projectively equivalent, i.e. they are consistent
in terms of the projective world frame.
44 / 85
Next

Tensor Notation
45 / 85
Tensor Notation Introduction
Representation of geometric entities (like points, lines, etc.)
depends on the basis vectors.
The way we represent a quantity with a tensor depends on the
way it gets transformed when the basis gets transformed
BASIS VECTORS e
i
Consider a set of basis vectors e
i
, i = 1 . . . 3 for a 2-dimensional
projective space IP
2
.
POINTS x
With respect to this basis, a point in IP
2
is represented by a set of
coordinates x
i
,
x =

3
i1
x
i
e
i
We represent this as x =

x
1
x
2
x
3

[superscript index]
46 / 85
TRANSFORMATION OF BASIS
We transform e
i
to a new basis e
j
=

i
H
i
j
e
i
where H is the basis
transformation matrix with entries H
i
j
.
TRANSFORMATION OF POINT
With respect to the new basis the point x transforms to
x =

x
1
x
2
x
3

x = H
1
x
Hence points transform according to H
1
.
TRANSFORMATION OF LINE
A line in IP
2
is represented by l = l
1
l
2
l
3
) in the original basis.
The transformed line

l = H

l
Coordinates of line transform as H

.
47 / 85
Tensor Notation Introduction
Superscript/ Subscript/ convention
Indices which transform according to H
1
are written as
super-scripts. They are the contravariant indices.
Indices which transform according to H or H

are written as
sub-scripts. They are covariant indices.
Tensor Summation Convention
An index repeated in upper and lower positions in a product
represents summation over a range of the index.
x
i
=

H
1

i
j
x
j

l
i
= H
j
i
l
j
48 / 85
Tensor Notation Introduction
Transformation of Projective mapping Example
Let P be a matrix representing a mapping between projective (or
vector) spaces.
Let G and H represent basis transformations in the domain and
range spaces.
With respect to the new bases, the new mapping is represented
as:

P = H
1
PG
IN TENSOR NOTATION:

P
i
j
=

H
1

i
k
G
l
j
P
k
l
49 / 85
Tensor Notation Introduction
The number of indices of a tensor are called as the valency of the
tensor.
The sum over an index, e.g. H
j
i
l
j
is referred to as contraction. In
this case the tensor H
j
i
is contracted with the line l
j
.
50 / 85
The tensor
rst
Introduction
The tensor
rst
is dened for r s t = {1 2 3} as follows:

rst
=
_

_
0 unless r s and t are distinct
+1 if r s t have even permutation of 123
1 if r s t have odd permutation of 123
The tensor
ijk
(or its contravariant counterpart
ijk
) is connected
with the cross-product of two vectors.
If a and b are 2 vectors, and c = a b, then
c
i
= a b)
i
=
ijk
a
j
b
k
The tensor
ijk
is related to determinants:
For 3 contravariant tensors a
i
b
j
c
k
, it can be veried that
a
i
b
j
c
k

ijk
is the determinant of the 3 3 matrix with rows a
i
b
j
and c
k
51 / 85
The tensor
rst
Introduction
The skew symmetric matrix [a]

is written in tensor notation as:

[a]

)
ik
=
ijk
a
j
if a is contravariant
[a]

)
ik
=
ijk
a
j
if a is covariant
RECALL:
Cross product matrix: e = e
1
e
2
e
3
)
[e]

=
_

_
0 e
3
e
2
e
3
0 e
1
e
2
e
1
0
_

_
Any skew symmetric 3 3 matrix may be written in the form [e]

for a
suitable vector e.
52 / 85
Next

Using the Tensor Notation for Trifocal Tensor

53 / 85
Tensor Notation
Image points are represented by column 3-vectors.
x =

_
x
1
x
2
x
3
_

_
Lines are represented by homogeneous row 3-vectors.
l = l
1
l
2
l
3
)
The i j-th entry of a matrix A is denoted by a
i
j
i is the row (contra-variant) index
j is the column (covariant) index
54 / 85
Tensor Notation
The equation x

= Ax is equivalent to:
x
i
=

j
a
i
j
x
j
written as x
i
= a
i
j
x
j
An index repeated in the contra-variant and covariant positions
would mean summation over that index.
The trifocal tensor T
i
= a
i
b

4
a
4
b

jk
i
= a
j
i
b
k
4
a
j
4
b
k
i
55 / 85
Tensor Notation
The trifocal tensor
jk
i
has one covariant and two contravariant
indices.
The arrangement of indices for the trifocal tensor implies the
transformation rule:

jk
i
= F
r
i

G
1

j
s

H
1

k
t

st
r
where F G H indicate the basis transformations in the 3 images.
56 / 85
Tensor Notation
The line-line-line relation l

[T
1
T
2
T
3
] l

= l

is written in tensor
notation as:
l
i
= l

j
l

jk
i
= l

jk
i
l

k
since l

j
l

jk
i
=

jk
l

j
l

jk
i
=

jk
l

jk
i
l

k
= l

jk
i
l

k
This relation is used for line transfer from one view to another
57 / 85
HOMOGRAPHY BETWEEN 1
st
AND 3
r
VIEWS Point transfer
The plane dened by back-projecting the line l

induces a
homography between the 1
st
and the 3
r
views.
l
i
= l

j
l

jk
i
= l

jk
i

= l

k
h
k
i
where h
k
i
= l

jk
i
Here h
k
i
are the elements of the homography matrix H
13
.
x
k
= h
k
i
x
i
= x
i
l

jk
i
58 / 85
Trifocal Tensor
PROPERTIES
A homography is obtained from the trifocal tensor by contraction
with a line, i.e. l

extracts a 3 3 matrix from the line.

A pair of important tensors are
ijk
and
ijk
This tensor is used to represent the vector product.
The skew-symmetric matrix [x]

is written as x
i

irs
The line joining two points x
i
and y
j
is equal to the cross product
x
i
y
j

ijk
= l
k
59 / 85
Incidence relations Trifocal tensor
Line-line-line: l l

[T
1
T
2
T
3
] l

= l

[T
1
T
2
T
3
] l

[l]

= 0

l
r

ris

j
l

jk
i
= 0
s
Point-line-line: x l

i
x
i
T
i
_

_
l

= 0
x
i
l

j
l

jk
i
= 0
60 / 85
Incidence relations Trifocal tensor
Point-line-point:
x l

i
x
i
T
i
_

_
[x

= 0

x
i
l

x
k

kqs

jq
i
= 0
s
Point-point-line:
x x

[x

i
x
i
T
i
_

_
l

= 0
x
i

x
j

jpr

pk
i
= 0
r
Point-point-point: x x

[x

i
x
i
T
i

[x

= 0
33
x
i

x
j

jpr

x
k

kqs

pq
i
= 0
rs
61 / 85
Incidence relations Trifocal tensor
Point-line-point:
x l

i
x
i
T
i
_

_
[x

= 0

x
i
l

x
k

kqs

jq
i
= 0
s
Point-point-line:
x x

[x

i
x
i
T
i
_

_
l

= 0
x
i

x
j

jpr

pk
i
= 0
r
Point-point-point: x x

[x

i
x
i
T
i

[x

= 0
33
x
i

x
j

jpr

x
k

kqs

pq
i
= 0
rs
62 / 85
Incidence relations Trifocal tensor
Consider the point-line-line relation
x
i
l

j
l

jk
i
= 0
This relation is a contraction of the tensor over all 3 of its indices.
This relation is linear in the 3 image entities involved: x
i
l

j
l

k
.
Likewise all other tensor relations are also linear in the image
entities involved.
For example, in the point-point-point relation
x
i

x
j

jpr

x
k

kqs

pq
i
= 0
rs
if x
1
and x
2
satisfy this relation, then so does x = x
1
+ x
2
.
The right hand side of the above equation is 0
rs
which is a 3 3
matrix with indices r s. i.e. r = {1 2 3} s = {1 2 3} .
63 / 85
Incidence relations Trifocal tensor
TRILINEARITIES FOR POINT-POINT-POINT
x
i

x
j

jpr

x
k

kqs

pq
i
= 0
rs
Clearly there are 9 different equations possible for different
choices of r and s. We examine 1 equation with r = 1 and s = 2.
Choosing r = 1 in the 2
n
view and expanding x
j

jpr
results in
l

p
= x
j

jp1
=

0 x
3
x

which is a horizontal line in the 2

n
view through x

.
Choosing s = 2 in the 3
r
view and expanding x
k

kqs
results in
l

q
= x
k

kq2
=

x
3
0 x
1

64 / 85
Incidence relations Trifocal tensor
TRILINEARITIES FOR POINT-POINT-POINT
x
i

x
j

jpr

x
k

kqs

pq
i
= 0
rs
l

p
= x
j

jp1
=

0 x
3
x

q
= x
k

kq2
=

x
3
0 x
1

Trilinear point relation now reduces to:
0 = x
i
x
j
x
k

jp1

kq2

pq
i
= x
i

x
3

x
3

21
i
x
1

23
i

+ x

x
3

31
i
x
1

33
i

This is just one of the 9 tri-linearities.
65 / 85
Incidence relations Trifocal tensor
0 = x
i
x
j
x
k

jp1

kq2

pq
i
= x
i

x
3

x
3

21
i
x
1

23
i

+ x

x
3

31
i
x
1

33
i

WHY DO WE CALL IT TRILINEARITY
Prex tri indicates that every monomial in the relation involves a
coordinate from each of the 3 image elements involved. (in the
above case a p-p-p relation has 3 points, x x

)
The relations are linear in each of the algebraic entities involved.
This is just one of the 9 tri-linearities. Why? There are 9
trilinearities for 3 choices of r and 3 choices of s.
Further 3 equations for each i for the
xx
i
entries.
66 / 85
3-view geometry
Next

Transfer of geometric entities from one/two views

to another
Point Transfer
Line Transfer Transfer
67 / 85
The Transfer problem
Given a pair of matched points in two views, what will be its
position in the 3
r
view?
How to transfer the lines ?
This problem can be solved...
If the camera matrices P P

are known. Reconstruct the 3D

point X by back-projecting rays from two views, and then project X
on to the 3
r
view.
By using fundamental matrices: F
21
F
31
F
32
.
By using trifocal tensor [T
1
T
2
T
3
] .
68 / 85
Epipolar Transfer Using F matrices
Let x x

be matched points in the two images.

We know F
31
, hence we compute the epipolar line in the 3
r
view
corresponding to x.
We know F
32
, hence we compute the epipolar line in the 3
r
view
corresponding to x

.
Intersection of the two epipolar lines gives x

= F
31
x) F
32
x

)
However there are certain degenerate conditions.
69 / 85
Epipolar Transfer Using F matrices
DEGENERATE CONDITIONS
One can dene a plane passing through the 3 camera centers
C C

. This called as the trifocal plane.

What if the 3D point X lies on this plane?
The epipoles e
32
e
31
and the point x

will lie on the same line

If the C C

are collinear, then the trifocal plane does not exist.

In this case e
32
= e
31
70 / 85
Using Trifocal Tensor Points Transfer
We are given a point correspondence x x

We choose a line l

passing through point x

in the 2
n
view.
As shown in slide 58, the point x

may be computed as:

x
k
= x
i
l

jk
i
This transfer is not degenerate for general points X lying on the
trifocal plane.
DEGENERATE CONDITIONS
If l

is the epipolar line corresponding to x, then x

i
l

jk
i
= 0
k
.
Hence the point x

is undened. see slide 36)

A good choice for l

is the line perpendicular to F

21
x
71 / 85
Using Trifocal Tensor Points Transfer
STEPS: (How to choose the line l

)
Compute F
21
from the trifocal tensor and correct x x

to the
exact correspondence x x

.
Compute the line l

through x

and perpendicular to l

e
= F
21
x. If
l

e
= l
1
l
2
l
3
)

and x

= x
1
x
2
1) then
l

= l
2
l
1
x
1
l
2
+ x
2
l
1
)

DEGENERATE CONFIGURATION
What if the 3D point X lies on the base-line joining the 1
st
and 2
n
cameras?
The points x and x

correspond with the epipoles in the two

images.
It is not possible to identify a line passing through x

which is not
the epipolar line.
72 / 85
3-view geometry
Next

Line Transfer using Trifocal tensor

73 / 85
Using Trifocal Tensor Line Transfer
Line transfer is not possible using fundamental matrix.
We are given corresponding lines l, l

in the 1
st
and 2
n
views.
What will be the position of this line in the 3
r
view?
As shown in slide 57, the line l

may be computed by solving

l
i
= l

j
l

jk
i
This is equivalent to solving the set of equations:

l
r

ris

j
l

jk
i
= 0
s
74 / 85
Using Trifocal Tensor Line Transfer
DEGENERATE CONDITIONS
The 3D line L is an intersection of planes and

The line L is undened when and

are coincident.
This will happen when l and l

are corresponding epipolar lines.

75 / 85
Using Trifocal Tensor Line Transfer
DEGENERATE CONDITIONS
Line transfer from 1
st
to 3
r
: Degenerate if given l and l

are
corresponding epipolar lines.
Line transfer from 1
st
to 2
n
: Degenerate if given l and l

are
corresponding epipolar lines.
When the lines l l

L are all on the trifocal plane.

76 / 85
3-view geometry
Next

The fundamental matrices F

21
F
31
F
32
for the 3
views
77 / 85
Fundamental matrices F
21
F
31
F
32
The 3 fundamental matrices F
21
F
31
F
32
are not independent but
satisfy:
e

23
F
21
e
13
= e

31
F
32
e
21
= e

32
F
31
e
12
= 0
The fundamental matrices are said to be compatible if they satisfy
these 3 relations.
There always exist 3 camera matrices P
1
P
2
P
3
such that F
ij
is
the fundamental matrix corresponding to the pair

P
i
P
j

78 / 85
Fundamental matrices F
21
F
31
F
32
UNIQUENESS OF CAMERA MATRICES GIVEN F
21
F
31
F
32
The 3 fundamental matrices determine the conguration of the
three cameras uniquely up to a projective ambiguity.
The camera matrices P P

can be determined from F

21
by
two-view techniques.
How to determine the 3
r
camera P

consistent with the same

projective world frame.
79 / 85
Fundamental matrices F
21
F
31
F
32
COMPUTING P

GIVEN P P

AND F
21
F
31
F
32
(Triangulation)
The 3
r
camera P

has to be determined in the same projective

frame. This is done as follows:
1 Select a set of matching points x
i
x

i
in the rst two images.
Determine the corresponding 3D point using triangulation.
2 Use epipolar transfer slide 69) to determine the corresponding
point x

in the 3
r
image, using F
31
and F
32
.
3 Solve for the camera matrix P

from the set of 3D-2D

correspondences X
i
x

i
Ensure that X
i
is not on the tri-focal plane.
Ensure that the three cameras are not collinear.
80 / 85
Fundamental matrices F
21
F
31
F
32
COMPUTING P

GIVEN P P

AND F
21
F
31
F
32
(Linear method)
The 3
r
camera P

must satisfy that P

F
31
P and P

F
32
P

be
skew-symmetric.
Each of these equations give rise to 10 linear equations in the
entries of P

.
Thus we get a total of 20 equations in 12 entries of P

.
The method assumes that the three fundamental matrices are
compatible and the cameras are non-collinear.
81 / 85
Trifocal Tensor v/s Fundamental matrices
Which one is better to describe
the 3-view geometry ?
Trifocal tensor
It is well dened and uniquely
determines geometry even for
collinear cameras.
Relative displacements
between the cameras are built
into the trifocal tensor.
Trifocal tensor can be used to
determine the fundamental
matrices which satisfy
compatibility. slide 78)
Fundamental Matrix
Views being collinear leads to
degenerate conditions
because the trifocal plane is
not unique.
There is no direct constraint
on the relative displacements
between the 3 cameras.
F matrices determined using
point correspondence for
individual views may not
satisfy compatibility.
82 / 85
Trifocal Tensor v/s Fundamental matrices
Trifocal tensor is clearly the better choice.
However if we are to use fundamental matrices F
21
F
31
F
32
then
we can determine the cameras up to a projective ambiguity in the
world frame.
What if we use just two fundamental matrices F
21
F
31
?
Can two F matrices capture 3 view projective conguration ?
83 / 85
Using F
21
and F
31
3-View Geometry
USING THE COUNTING ARGUMENT:
Two fundamental matrices have 7 dofs each. Hence total 14.
Three arbitrary cameras have 3 11 = 33 dofs.
Specifying the 2 fundamental matrices gives 14 constraints. This
leaves 33-14=19 dofs for specifying the camera matrices.
This is 4 more than the projective ambiguity (15 dofs).
In what form do we get 4 more dofs ?
84 / 85
Using F
21
and F
31
3-View Geometry
Given F
21
we can specify the cameras P P

.
Given F
31
we can specify the cameras P P

.
In both cases we choose P = [ I | 0]
Having xed the rst two cameras P P

compatible with F
21
, we
have xed the projective coordinate frame.
The 3
r
camera may now be chosen P

H
2
.
In that case the pair PH
2
P

H
2
will be compatible with F
31
.
The projective frame xed by P P

will be unmodied as long as

PH
2
= P = [ I | 0].
To ensure PH
2
= P = [ I | 0] the form of H
2
must be restricted to:
H
2
=

I 0
v

i
k
i

Here v
i
is a 3-vector and k
i
is a scalar. (4 additional dofs)
85 / 85