Vous êtes sur la page 1sur 61

Reconstructing Camera

Positions from
Visibility Information

Elif Tosun
Honors Thesis Presentation
Advisor: Ileana Streinu
May 1st, 2002
Outline
 Motivation
 General Problem
 Some Definitions
 Main Problems of Interest
 Background in Rigidity
 Solutions in 2D
 Further Work
 Summary
Motivation:
MIT City Scanning Project
 Goal: To reconstruct a 3D model of an
urban environment using numerous
images.
Motivation: MIT City Project
 Method:
– Robot with a mounted hemispherical camera
– Images acquired at different *unknown*
positions of an urban environment
– Images put together to get “image feature
points” using methods in Computer Vision and
Computer Graphics.

A corner may
be an image
feature point
 Method (Cont’d):
– Slopes between cameras and image features
are inferred from use of hemispherical
cameras with known local orientations.

– Once the set of slopes are obtained, solution


for camera and image feature positions is
based on solving a linear system of equations
However…
 A linear system of equations may be over or under
determined.
 This information is NOT known a priori
 Therefore, system *sometimes* may fail to give a
solution.
 A SUBSET of edges that give a unique solution is
needed.

“… determination of the minimal set of distinct pairs


needed for a non-degenerate solution is significantly
more complicated and depends entirely on the topology
of the adjacency graph. ” [Antone-Teller, 2000]
The Problem
 Given: Visibility information as a set of
slopes at n positions of a hemispherical
camera
 Find: Absolute coordinates of camera
locations and positions of image
features
 Why? May be used to reconstruct a
model of environment.
 Problem? Over- or under-determined
system of equations. Need a subset.
This thesis focuses on:
 Reformulating this problem and
modeling it based on its
geometric/combinatorial aspects
 Exploring it in 2 and 3 dimensions
 Isolating several feasible sub problems
and giving algorithmic solutions
 Providing implementations for some
General Camera Registration Problem
 Formulation using “direction networks”
Data set:
3 cameras ,4 image
feature points, slope
known for 13 pairs

Direction Network:
7 (=3+4) vertices
13 edges
Direction Network?

A direction network is a graph G=(V, E)


together with slope information for each
edge:
D = (V, E, S) with
S = {se: se is the slope information for
an edge e  E}
General Camera Registration Problem
Given a direction network D and the
location of a subset of vertices V’ of V,
find the location of all vertices v  V of G.

Notes:
1. We don’t treat image features and camera
positions differently. Both sets together form the
set of vertices V of G.
2. Location of a subset of vertices is needed to fix a
frame of reference.
Definitions
 Realization of a direction network: A
geometric embedding of G of D.
– i.e. set of points P s.t. if (ij)E then the
slope between points pi and pj is sij
Definitions
 Realization of a direction network: A
geometric embedding of G of D.
– i.e. set of points P s.t. if (ij)E then the
slope between points pi and pj is sij
y-axis

(0,0) x-axis
Definitions
 Mixed “direction and length” network: A direction network
of the following form:
MD = (V, v1, E1, E2, S, L)
where:
E1 = set of edges for which slopes are given
E2 = set of edges for which lengths are given
S = {se: se is the slope for edge e  E1}
L = {le: le is the length for edge e  E2}
v1 = a unique vertex that is pinned down
(at origin)
How to solve for an MD system

To solve for a mixed “direction and length”


network…
A) Associate a linear equation to each slope
information…
i.e. for each slope sij between points
pi=(xi, yi) and pj=(xj, yj):

(yj-yi) – sij(xj-xi) = 0
B) And associate a quadratic equation to each

length information

i.e. for each length lij between points


pi=(xi, yi) and pj=(xj, yj):

(xj-xi)2 + (yi-yj)2 = lij2


A particular case…
The particular case of an MD that we will be
considering is the following:
There exists an edge e, s.t.
– both the length (le) and the slope information
(se) is known
– one end point of e is pinned down
– e is the only edge in E2
i.e. it’s as if two vertices are pinned down:
se So, we’re only dealing
le with a linear system of
equations
More Definitions
 Tight direction network: A direction
network where there is a unique
solution to the system.
 Loose direction network: A direction
network where the system is under-
determined.
 Loose direction network: A direction
network where the system is under-
determined.

It has a parallel
redrawing!
 Over-determined direction network: A
direction network where there are more
edges than needed to have a unique
solution:
 Non-realizable direction network: A
direction network with no solution where
the system is over-determined and the
constraints are incompatible
Main Problems

 Problem 1: Given a direction network,


decide whether it’s tight or not.

 Problem 2: Given a loose direction


network, find a set of edges, that needs
to be added, so that it becomes tight. -
if possible
 Problem 3: Given an over-determined
direction network:
– Find a set of edges that need to be
removed so that it becomes tight – if
possible
– Find the set of edges to be removed to
get ‘the best’ tight framework.
– Get an approximation of ‘the best’ tight
framework.
 Why “if possible” for Problems 2 and 3:
– The data set might not have the edges
necessary for tightness
All induced sub-graphs
are under-determined

– The given direction network might be


partially over- or under-determined
 Why “if possible” for Problems 2 and 3:
– The data set might not have the edges
necessary for tightness
All induced sub-graphs
are under-determined

– The given direction network might be


partially over- or under-determined
Under-determined
sub-graph
No edge removal
can make it tight
 Why “if possible” for Problems 2 and 3:
– The data set might not have the edges
necessary for tightness
All induced sub-graphs
are under-determined

– The given direction network might be


partially over- or under-determined
Over-determined
sub-graph
No edge addition
can make it tight
Background in Rigidity
Geometric Rigidity
 2 dual categories:
– Fixed Edge Lengths
– Fixed Directions (“parallel redrawing”)

 Duality in 2D: Properties of one hold


for the other as well!
Fixed Length Rigidity:
Definitions
 A framework is a graph G=(V, E)
together with length information for each
edge:
F = (V, E, L) with
L = {le: le is the length
information for
le
an edge e  E}
Fixed Length Rigidity:
Definitions
 A minimally rigid framework

 A flexible framework
(infinitesimal rigidity – velocities)
Fixed Length Rigidity:
Definitions
 A minimally rigid framework

 A flexible framework
(infinitesimal rigidity – velocities)
Fixed Length Rigidity:
Definitions
 An over-braced framework

 An unrealizable framework
Fixed Length vs. Fixed Direction
 Framework  Direction Network
 Minimally Rigid  Tight

 Flexible  Loose
Fixed Length vs. Fixed Direction

 Over-braced  Over-determined

 Unrealizable  Unrealizable
Fixed Length vs. Fixed Direction
 Find a non-trivial set  Find point coordinates
of velocities
 Rigidity Matrix (M)
(2nxm).  Parallel Redrawing
 Info on derived from Matrix (M) (2nxm).
point coordinates  Info on slopes
Fixed Length vs. Fixed Direction
 Solution: Mv = b  Solution: Mv = b
V = column vector of unknowns, b = column vector of 0s.
 Why? (fixed rods,  Why? (line equations)
perpendicular velocity (xi-xj)mij –(yi-yj) = 0
vectors)
(pi-pj).(vi-vj) = 0 pj

mij
pi
Fixed Length vs. Fixed Direction
 Solution: Mv = b  Solution: Mv = b
V = column vector of unknowns, b = column vector of 0s.
 Why? (fixed rods,  Why? (line equations)
perp. velocity) (xi-xj)mij –(yi-yj) = 0
(pi-pj).(vi-vj) = 0
pj

mij
pi
Rigidity in the Plane:
Combinatorial Rigidity
 Based on underlying graph, no
numeric data
 Laman’s Theorem:
Given a minimally rigid graph G=(V, E) with n
vertices, m edges:
(i) m = 2n-3
(ii) For every sub graph G’ of k vertices
has <= 2k-3 edges
Rigidity in the Plane:
Combinatorial Rigidity
 Based on underlying graph, no
numeric data
 Laman’s Theorem:
Given a minimally rigid graph G=(V, E) with n
vertices, m edges:
(i) m = 2n-3
(ii) For every sub graph G’ of k vertices
has <= 2k-3 edges
Generic Rigidity & Tightness
 A generic minimally rigid framework(or
tight dir. network) : minimally rigid (or
tight) for generic positions of points.

Generic Non-generic
Generic Rigidity & Tightness
 A generic minimally rigid framework(or
tight dir. network) : minimally rigid (or
tight) for generic positions of points.

Generic Non-generic
Solution to Problem 1 Given a direction
network, decide if
(Decision Problem) tight or not

Geometric Approach :
Using parallel redrawing matrix

If there exists a solution to Mv=B


Check dimension, if =0, TIGHT
if > 0, UNDER-DETERMINED
Else
UNREALIZABLE
Decision Problem
Geometric Approach (Cont’d)
 PROBLEM:
In case of NOISE, geometric
approach may return: Unrealizable
(although if erroneous edges
are not used, there might be a
solution)

 So : we need a subset of these edges


necessary and sufficient for a unique solution

 Use Combinatorial Approach to get a subset,


then use Geometric Approach
Solution to Problem 1
(Decision Problem)
 Combinatorial Approach
Main Idea:
- Tight direction network  has a unique
solution *most* of the time
- Underlying graph of a tight D.N. is a minimally
rigid graph
- Find a way of checking if a given graph is
minimally rigid or not
Decision Problem:
Combinatorial Approach
 Sugihara’s Algorithm, based on bipartite
matchings
– Associate a bipartite graph to original
– Modify it for each edge and check for
complete bipartite matching

 Mathematica Notebook
Decision Problem:
Combinatorial Approach: Sugihara’s Alg.
 Complexity
O(n1.5).O(n2) =O(n 3.5)
Solution to Problem 2
(Extension)
 Problem: Given an under-determined
direction network, find a set of edges to
extend it to a tight graph (if possible).
Extension Problem
The Algorithm
 Natural Approach: A Greedy Algorithm
 Add one edge at a time, use “decision”
algorithm to test

 Proof of Correctness!
Matroid Theory and Rigidity
Definition:
A matroid is an ordered pair M=(S, l) satisfying
the following conditions
1. S is a finite non empty set
2. l is a non empty family of subsets of S
called the independent subsets, s.t. if B l
and AB, then A l
3. If A l, and |A|<|B|, then there exists an elt
x  (B-a) s.t. A  {x} l
Matroid Theory and Rigidity
2 Important Properties:
1. Result by Sugihara:
“Frameworks demonstrate
matroidal properties.”

2. “For matroidal structures, greedy


algorithms return optimal solutions
correctly”

Because of these two properties, our


Greedy Approach returns a correct
solution
Solution to Problem 3
(Extraction)
 Given an over-determined direction
network, find a set of edges to be
removed to get a tight one(if possible).
Extraction: Part 1
The Algorithm
 Greedy Algorithm
 Remove one edge at a time, use
“decision” algorithm to test
 Correctness – based on Matroidal
Property
For Extension and Extraction Problems:
 How about if we assign weights to
edges?
 Weights can mean…
– Easiest to sample
– Most reliable
 Slight modification in code
 They help in recognizing a better/best
tight direction network
Extraction Problem : Part 2
 Connection b/n Combinatorial and
Geometric Rigidity
 The Problem: Given a tight subset of a
data set, find another (better?, w/out a
non-generic embedding?) tight subset.
Data Original Output New Output
C C C

A A B A
B
D D D
Further Work
 Motivating problem is in 3D.
 Our algorithms…
– Emphasis not on efficiency but to show
that they are implementable for the sake of
experimentation
 …and implementations
– Emphasis was on underlying theory. Used
what was already available to us
(Mathematica)
Summary
 Reformulation of Problem
Summary
 Reformulation of Problem

 Underlying Theory
Summary
 Reformulation of Problem

 Underlying Theory

 Implementations
Credits
 Seth Teller, for suggesting the problem

 Ruth Haas,
 Brigitte Servatius,
 Jack Snoeyink,
 Ileana Streinu, and
 Walter Whiteley , for their ideas and references
Special Thanks…

 Ileana Streinu
 Joseph O’Rourke
 Smith CS Dept
 Friends and Family
QUESTIONS
?

Vous aimerez peut-être aussi