C OMPUTER V ISION : E STIMATION OF 2D

H OMOGRAPHY

IIT Kharagpur

Computer Science and Engineering,

Indian Institute of Technology
Kharagpur.

(IIT Kharagpur) Estimation Jan ’10 1 / 23

 We consider a set of point correspondences xi ↔ x0i between two
images.
Such point correspondences may be putative correspondences,
i.e. they are believed to be correct, but some of them may be
wrong also.
Our problem is to compute the 3 × 3 matrix H such that

Hxi = x0i

Issued to be examined:
The number of measurements xi ↔ x0i required.
I Should it be just 4?
Approximate Solutions? How to compute the homography if there
are multiple point correspondences given, some of which may be
incorrect or mismatches. Which cost functions should be
minimized while estimating the homography?

(IIT Kharagpur) Estimation Jan ’10 2 / 23

Direct Linear Transformation (DLT)
Algorithm
x0i = Hxi x0i × Hxi = 0

The vector cross product is zero x0i × Hxi = 0 because the

vectors x0i has the same direction as Hxi .
The j th row of the matrix H is denoted by hjT
 1T 
 h xi 
 2T 
Hxi =  h xi 
 
h3T xi

(IIT Kharagpur) Estimation Jan ’10 3 / 23

Writing x0i = (xi0 , yi0 , wi0 ) the cross product x0i × Hxi = 0 may be
explicitly stated as:
 0 3T
 yi h xi − wi0 h2T xi 


x0i × Hxi =  wi0 h1T xi − xi0 h3T xi 

 
 0 2T
xi h xi − yi0 h1T xi


We have hj T xi = xiT hj
 0 T 3
 yi xi h − wi0 xi T h2


0
xi × Hxi =  wi0 xi T h1 − xi0 xi T h3
 

 0 T 2 
xi xi h − yi0 xi T h1


(IIT Kharagpur) Estimation Jan ’10 4 / 23

  0T −wi0 xiT yi0 xiT  
 
h1 

   
   
0T
  
 w 0 xT 0 T
−xi xi   2
h  = 0

 i i
   
   
T 3
−yi0 xiT xi0 xiT
   
0 h
 1   
 h   h1 h2 h3 
Ai h =0 h =  h2 H =  h4 h5 h6 
   

 3  
h h7 h8 h9


Aiis a 3 × 9 matrix. h is a 9-vector made up of the entries of the

matrix H.
The equation Ai h = 0 is an equation linear in the unknown h.

(IIT Kharagpur) Estimation Jan ’10 5 / 23

Although there are 3 equations, only 2 of them are linearly
independent. Thus each point correspondence gives 2 equations
in the entries of h.
 T 0 xT 0 xT
  h1 
 0 −wi i
y i i   

  
   2 
   h  = 0
 w 0 xT
 i i 0T −xi0 xiT   

   
h3

Now Ai is a 2 × 9 matrix in Ai h = 0
The equations hold for any homogeneous coordinate
representation (xi0 , yi0 , wi0 )T of the point x0i . One can choose the
homogeneous coordinate wi0 = 1

(IIT Kharagpur) Estimation Jan ’10 6 / 23

Solving for H
Each point correspondence gives rise to two independent
equations in the entries of H.
Given a set of 4 such correspondences we obtain a set of
equations Ah = 0 where A is a matrix of equation coefficients built
from the matrix rows Ai .
The 8 × 9 matrix A has a rank 8, and thus has a 1-dimensional null
space which provides for the solution for h.
The solution h can only be determined up to a scale.

(IIT Kharagpur) Estimation Jan ’10 7 / 23

Over-determined solution
What if more than 4 point correspondences xi ↔ x0i are available?
The set of equatoins Ai h = 0 is over-determined.
If the measurement of image coordinates is exact, then there is an
exact solution to h.
If the image measurements are noisy, there will not be any exact
solution apart from the h = 0 solution.
Instead of looking for an exact solution, we look for an approximate
solution, i.e. a vector h that minimizes a suitable cost function.
We attempt to minimize the norm ||Ah|| with constraint ||h|| = 1.
This is identical to finding the minimum of the quotient ||Ah||/||h||
The solution is the (unit) eigenvector of A T A with least eigen value.
The solution can also be obtained as the unit singular vector
corresponding to the smallest singular value of A.

(IIT Kharagpur) Estimation Jan ’10 8 / 23

Degenerate configurations
Suppose that of the given point correspondences, 3 of the points
x1 , x2 , x3 are collinear.
If the correspondences x10 , x20 , x30 If the correspondences x10 , x20 , x30
are all on the same line, it would are not all on the same line, then
mean that the homography is not there can be no projective
sufficiently constrained, and there transformation taking xi to x0i . In
will exist a family of homographies such a case the solution matrix
mapping xi to x0i . It would follow which you will get will be a rank 1
that the 8 equations used for matrix and hence does not
computing H are not independent. represent a projective
transformation.
A situation where a configuration does not determine a unique solution
for a particular class of transformations is termed as degenerate.

(IIT Kharagpur) Estimation Jan ’10 9 / 23

Data Normalization for DLT algorithm
The result of the DLT algorithm for computing 2D homographies
depends on the coordinate frame in which the points are
expressed.
The result of the DLT algorithm is not invariant to similarity
transformations of the image.
For computing a 2D homography, some coordinate systems turn
out to be in some way better than others.
Data normalization is carried out prior to homography
computation. Such a normalization typically consists of translation
and scaling of image coordinates.

Data normalization is a compulsory step.

An algorithm which incorporates an initial data normalization step will
be invariant with respect to arbitrary choices of scale and coordinate
origin.

(IIT Kharagpur) Estimation Jan ’10 10 / 23

Data normalization steps:
The points are translated so that their centroid is at the origin.
The points are then scaled (isotropic√scaling) so that the average
distance from the origin is equal to 2. This means that the
“average" point is equal to (1, 1, 1)T
Given two images for homography computation, a (similarity)
transformation is applied to each of the two images independently.

S1 S2
xi −→ x̃i x0i −→ x̃0i

Let the transform computed for the first image be denoted as S1

and that for the second image be denoted as S2 .
Solve the homography H̃ for the point correspondences:

x̃i ↔ x̃0i

(IIT Kharagpur) Estimation Jan ’10 11 / 23

Data normalization: Denormalization
The resulting 2D homography has to be corrected so that it is with
respect to the original coordinate system.

H = S−1
2 H̃S1

(IIT Kharagpur) Estimation Jan ’10 12 / 23

Symmetric Transfer Error
X
d(xi , H−1 x0i )2 + d(x0i , Hxi )2
i

Reprojection Error
We are given a set of correspondences xi ↔ x0 i
Our objective is to estimate the homography, as well as a
“correction" for each correspondence.
We are seeking a homography Ĥ and pairs of perfectly matched
points x̂i and x̂0i that minimize the total error function
X
d(xi , x̂i )2 + d(x0i , x̂0i )2 subject to x̂0i = Ĥx̂i ∀i
i

(IIT Kharagpur) Estimation Jan ’10 13 / 23

(IIT Kharagpur) Estimation Jan ’10 14 / 23
Robust Estimation
The set of correspondences {xi ↔ x0i } may have errors.
The source of error may be the measurement of the point’s
position. Such an error can be modeled using a Gaussian
distribution.
Errors will also arise if the points are mismatched.
The mismatched points are outliers to the Gaussian distribution.
These outliers can severely disturb the estimated homography.
Robust estimation techniques tend to discard the outliers and
retain the inliers.
Such techniques are robust (tolerant) to outliers.

(IIT Kharagpur) Estimation Jan ’10 15 / 23

RANSAC
It is a robust estimator (RANdom SAmple Consensus) algorithm.
It is able to cope with a large proportion of outliers.

(IIT Kharagpur) Estimation Jan ’10 16 / 23

Consider the problem
Given a set of 2D data points, find the line which minimizes the
sum of squared perpendicular distances, subject to the condition
that none of the valid points deviates from this line by more than t
units.
The problem involves two parts: a line fit to the data; and a
classification of data into inliers and outliers.

(IIT Kharagpur) Estimation Jan ’10 17 / 23

RANSAC Basic Idea
Two of the points are selected randomly; these points define a line.
The support for this line is measured by the number of points that
lie within a distance threshold.
This random selection is repeated a number of times and the line
with most support is deemed the robust fit.

The points within the threshold distance are the inliers (consensus
set).
The intuition is that if one of the points (of the minimal subset) is
an outlier then the line will not gain much support.
Scoring a line by its support has the advantage of favouring a
better fit.

(IIT Kharagpur) Estimation Jan ’10 18 / 23

RANSAC In general
We wish to fit a model to data, and the random sample consists of
a minimal subset of the data, sufficient to determine the mode.
For a planar homography, the minimal subset consists of 4 point
correspondences. The model is a 3 × 3 matrix.

(IIT Kharagpur) Estimation Jan ’10 19 / 23

Objective:

Robust fit of a model to a data set S which contains outliers. (RANSAC)

Algorithm:

(i) Randomly select a sample of s data points from S and instantiate

the model from this subset.
(ii) Determine the set of data points Si which are within a distance
threshold t of the model. The set Si is the consensus set of the
sample and defines the inliers of S.
(iii) If the size of Si (the number of inliers) is greater than some
threshold T , re-estimate the the model using all the points in Si
and terminate.
(iv) If the size of Si is less than T , select a new subset
and repeat the above.
(v) After N trials the largest consensus set Si is selected, and
the model is re-estimated using all the points in the subset Si .

(IIT Kharagpur) Estimation Jan ’10 20 / 23

RANSAC for homography computation
Identify by some means a set of putative correspondences. Some
of these correspondences may be mismatches.
The point matches can be computed automatically based on
proximity and similarity of their intensity neighbourhood.
RANSAC is designed to
1 Estimate the homography
2 Set of inliers consistent with the estimate (the true
correspondences),
3 Outliers (mismatches).

(IIT Kharagpur) Estimation Jan ’10 21 / 23

Objective:

Compute the 2D homography between two images.

Algorithm:

(i) Repeat the following for a considerably large number N of samples:

(a) Select a random sample of 4 correspondences and compute the
homography H
(b) Calculate the distance d⊥ for each putative correspondence.
(c) Compute the number of inliers consistent with H by the
number of correspondences for which d⊥ < t
Choose the H with the largest number of inliers.
In the case of ties, choose the solution that has the lowest
standard deviation of inliers.

(IIT Kharagpur) Estimation Jan ’10 22 / 23

Compute the 2D homography between two images.

Final Steps:

(i) Optimal Estimation: Restimate H from all correspondences

classified as inliers. This can be done by minimizing a suitable
cost function.
(ii) Guided Matching: Further interest point correspondences are
now determined using the estimated H to define a search region
about the transferred point position.

These steps can be iterated until the point correspondences

stabilize.

(IIT Kharagpur) Estimation Jan ’10 23 / 23