Ransac

Cameraparameterestimationfor
imagebasedmodeling
JaechulKim
Demopresentation Visualrecognitionandsearch,Mar21,2008
Purpose Purpose
Introduce a basic procedure of camera Introduceabasicprocedureofcamera
parameterestimationfrommultipleimages
and its application to imagebased modeling anditsapplicationtoimage basedmodeling
Overview of general procedure Overviewofgeneralprocedure
Step 1 : Point matches and epipolar geometry Step1:Pointmatchesandepipolar geometry
estimation(i.e.Fundamentalmatrix
computation) computation)
Step2 : Estimation of camera parameters Step2:Estimationofcameraparameters
Focallength,Cameraposition,Viewingdirection
etc etc.
Step 3: 3D reconstruction & Texture mapping Step3:3Dreconstruction&Texturemapping
Step1
FeaturePointsMatching&Epipolar GeometryEstimation
General procedure Generalprocedure
Findpointcorrespondencesbetweenimages
From point correspondences compute Frompointcorrespondences,compute
fundamentalmatrices(Fmatrices)between
images images
Outliersinpointcorrespondencesarerejectedduring
FmatrixcomputationusingRANSAC
Output:Fmatrix(i.e.projective
reconstruction)
Demopresentation Visualrecognitionand
search,Mar21,2008
Step1
FeaturePointsMatching
Three methods are tested in this demo Threemethodsaretestedinthisdemo
Harriscornerdetector&Windowcorrelation+
RANSAC RANSAC
SIFTdetector&SIFTdescriptor+RANSAC
Manual matching Manualmatching
search,Mar21,2008
Step1
l Epipolar geometry
Projective geometry between two views Projectivegeometrybetweentwoviews
X e1,e2:epipoles
l1,l2:epipolar lines
l2
l1
x1 x2
C1 C2
e1 e2
search,Mar21,2008
Step1
d l Fundamentalmatrix
Encode epipolar geometry between two views Encodeepipolar geometrybetweentwoviews
Rank2matrix(det(F)=0)thatcanbe
computed from at least 7 point computedfromatleast7point
correspondences
0
T
Fx x
Defineepiploar lineforagivenpointx1 orx2
0 =
1 2
Fx x
2
T
1
1 2
x F l
Fx l
=
=
search,Mar21,2008
2 1
x F l
Step1
( d l ) RANSAC(RANdom SAmple Concensus)
Robustestimationtechniqueunderthepresenceof q p
outliers
Algorithmoutline
Given putative correspondences sample 7 or 8 Givenputativecorrespondences,sample7or8
correspondencesandthencomputetheFundamental
matrix
Using the computed Fundamental matrix count the UsingthecomputedFundamentalmatrix,countthe
numberofinliers
Ifthenumberofinlier isamaximumamongiterations,
store the Fundamental matrix and inliers storetheFundamentalmatrixandinliers.
Repeatthesampling.
search,Mar21,2008
Step1
Feature point detection & matching Featurepointdetection&matching
Harriscorner+Windowcorrelation+RANSAC
Harris corner detector Harriscornerdetector
Parameterstobeused
Harristhreshold,Mc, is500
Kappa is set to 0 04 Kappaissetto0.04
Gaussiansmoothingwithsigma1isappliedtoimagebefore
cornerdetection
Window size (u v) is 1 Windowsize(u,v)is1
search,Mar21,2008
Step1
d l Windowscorrelation
Foradetectedcornerpoint(x,y)intheimage1,
h h ( ) h h searchthecornerpoint(x,y)intheimage2with
theminimumSSDerror
P t t b d Parametertobeused
Correlationwindowsize15
Search area in the image 2 is set to 300 by 300 (1/4 size Searchareaintheimage2issetto300by300(1/4size
oftheimage)centeredto(x,y)
search,Mar21,2008
Step1
d Harriscornerdetection
I iti ll d t t d i t Initiallydetectedcornerpoints
search,Mar21,2008
Step1
Window correlation + RANSAC Windowcorrelation+RANSAC
Putativematches(626) InliersafterRANSAC(23,4%)
search,Mar21,2008
Step1
E l f f l t h Examplesoffalsematches
Step1
l f f l h Examplesoffalsematches
search,Mar21,2008
Step1
Moreexamples(Harris+RANSAC)
Initiallydetectedcornerpoints
search,Mar21,2008
Step1
l ( ) Moreexamples(Harris+RANSAC)
search,Mar21,2008
Step1
Moreexamples(Harris+RANSAC):Goodresult
Step1
l Moreexamples(Harris+RANSAC):Goodresult
search,Mar21,2008
Step1
Harris + RANSAC Concl sion Harris+RANSAC Conclusion
Weaktomatchingtwoimageswithlarge
viewpoint change viewpointchange
Confusioninrepetitivetextures
S f i i h i F i SomeofimagepairshaveincorrectFmatrices
Harriscornerdetectionseemstobemoreproper
to ideo based camera parameter tracking here tovideobasedcameraparametertrackingwhere
imagechangebetweenconsecutiveframesis
small small
search,Mar21,2008
Step1
SIFT+RANSAC
SIFT+RANSAC
Parametertobeused
Sigma:0.5
Numberofoctaves:6
N b f l l t 3 Numberoflevelsperoctave:3
SIFTdescriptor:128dimensions
Putative matches are found using nearest Putativematchesarefoundusingnearest
neighborbetweentheSIFTdescriptors
search,Mar21,2008
Step1
SIFT+RANSAC
SIFT + RANSAC SIFT+RANSAC
50 50
100
150
200
100
150
200
250
300
350
400
250
300
350
400
450
500
550
450
500
550
100 200 300 400 500 600 700 100 200 300 400 500 600 700
InitiallydetectedSIFTfeaturepoints
Step1
SIFT+RANSAC
SIFT+RANSAC
search,Mar21,2008
Step1
SIFT+RANSAC
SIFT + RANSAC Good res lt SIFT+RANSAC:Goodresult
search,Mar21,2008
Step1
SIFT+RANSAC
d l SIFT+RANSAC:Goodresult
search,Mar21,2008
Step1
SIFT+RANSAC
l l ( ) Failureexamples(SIFT+RANSAC)
search,Mar21,2008
Step1
SIFT+RANSAC
( ) Failureexamples(SIFT+RANSAC)
search,Mar21,2008
Step1
SIFT+RANSAC
SIFT RANSAC C l i SIFT+RANSAC Conclusion
MorerobusttotheviewpointvariancethanHarris
corner corner
Insomecases,automaticmatchingusingSIFT
providesareliableFmatrix p
But,itstillinvokesfalsematchesinrepetitive
texturedareas
Forbagoffeatures,thismaybenotacriticalproblem
But,forFmatrixcomputation,theaccuratelocation
between matches is very important betweenmatchesisveryimportant
search,Mar21,2008
Step1
Conclusion
A i f hi f F i AutomaticfeaturematchingforFmatrix
computation
Both Harris + RANSAC and SIFT + RANSAC dont BothHarris+RANSACandSIFT+RANSACdon t
providethereliableresultspersistentlyovermany
imagestakenfromthewiderangeofimaging
conditions in practice conditionsinpractice
But,SIFT+RANSACismorepowerful
Ifmanyofimageswithsimilarappearancesaregiven,
SIFT+RANSAC can provide reliable Fmatrices estimation SIFT+RANSACcanprovidereliableF matricesestimation
Or,someprogressivewayliketheoneusedinreading
assignmentpapercouldfixtheproblem
The higher the inlier rate is the more reliable the Thehighertheinlier rateis,themorereliablethe
matchresultis.
search,Mar21,2008
Step1
Conclusion
Manualassignmentofcorrespondences
In all of my trials automatic matches fail to Inallofmytrials,automaticmatchesfailto
provideaconvergentestimationofcamera
parameters
Therefore,allexperimentsoncameraparameter
estimationareperformedonthedatasetwith
manuallyassignedcorrespondences
search,Mar21,2008
Step2
Cameraparameterestimation
The implemented method Theimplementedmethod
EXIFinformationbasedparameterinitialization+
Parameter optimization using Bundle adjustment ParameteroptimizationusingBundleadjustment
search,Mar21,2008
Whyneedcameraparameters?
Projectiveambiguity
Projective Geometry Hierarchyoftransformations
GeneralImaging
I
n
c
r
e
a
Orthographiccamera
a
s
i
n
g
f
o
c
a
l
,
i
n
C
a
m
e
r
a
c
a
Frontoparallelviewing
camera
n
c
r
e
a
s
i
n
g
d
i
s
t
a
a
l
i
b
r
a
t
i
o
n
camera
Fullycalibratedcamera
a
n
c
e
} { X H PH
1
FromMultipleviewgeometryinComputerVision,1
st
ed.pp.59)
} { X H PH,
Step2
Camera model : Pinhole projection + CCD Cameramodel:Pin holeprojection+CCD
model
Z
R, t
X
Y
R,t
K
3 4
[ | ] =
+
x K R t X
X RX t KX
Homogeneous coordinate (linear)
N h di t
3 3 3 3
,
c w c
= + = X RX t x KX
Non-homogeneous coordinate
Step2
Intrinsic parameters : CCD p
= 0
x
o f
o s f
K
f
=
1 0 0
0
y
o f K
Extrinsic parameters : coordinate transformation
R, t
search,Mar21,2008
Step2
EXIF information EXIFinformation
Metafileinformationstoredinimagefileby
digital cameras digitalcameras
Containsfocallength,fnumber,whitebalance,
model name, maker name, etc modelname,makername,etc
search,Mar21,2008
Step2
HowtoinitializeacamerausingEXIF g
Getafocallengthf(mm)fromEXIFinformation
e.g.10mm
Estimate a CCD size from model name in EXIF information EstimateaCCDsizefrommodelnameinEXIFinformation
e.g.20mmby20mmforcanonEOS300d
Converttheunitoffocallengthfrommmtopixels
e.g.imagesize1000by1000,then1pixel=20/1000mm,f=10mm
=10/(20/1000)=500pixels
Formoreaccuratecomputation,wecanconsiderthe
numberofeffectivepixels
e.g.if10MpixelsDigitalcamerahas8Meffectivepixels,thenCCD
sizeshouldbeconsideredusingthereducedsizeby8/10.
search,Mar21,2008
Step2
Parameter optimization using bundle Parameteroptimizationusingbundle
adjustment
InitializetheinternalparametersusingEXIF p g
information
InitializetheexternalparametersusingFmatrix
d h i i i li d i l andtheinitializedinternalparameters
Given,Fandinternalparameters,cameramotioncan
becomputedvialinearequation. p q
Minimizethereprojectionerrorsusingnonlinear
leastsquareoptimization
search,Mar21,2008
Step2
dl d Bundleadjustment
Iterativenonlinearleastsquaretechniqueto
fitthemodeltothemeasurement
LevenbergMarquardtalgorithmisgenerallyused.
Variables:3Dreconstructedpoints+Camera
2
) min
j i
i j
j i,
X P (x

projectionmatrices
Measurement:2Dpointcorrespondences
Errormeasure:reprojectionerrorof3D
reconstructedpointsfrom2Dobservedpoints
search,Mar21,2008
Step2
dl d Bundleadjustment
Speedupviausingasparseness
P1 P2 P3 X1 X2 X3
J I) J (J
T T
= +
X11
X12
X13 X13
X21
X22
x23
x31
X32 X32
X33
Step2
Result 1 Chateau cattle images (7 images) Result1 Chateaucattleimages(7images)
search,Mar21,2008
Step2
Result2 Triumphal Arch images (6 Images) Result2 TriumphalArchimages(6Images)
search,Mar21,2008
Step2
Result3 Projector images (12 images) Result3 Projectorimages(12images)
search,Mar21,2008
Step2
Sensitivity to initialization (projector dataset) Sensitivitytoinitialization(projectordataset)
8%
fy
6% 4%
fx
10%
Convergence region
search,Mar21,2008
Convergenceregion
Step2
Sensitivity to initialization (Chateau cattle Sensitivitytoinitialization(Chateaucattle
dataset)
12%
fy
6% 6%
fx
10%
Convergence region
search,Mar21,2008
Convergenceregion
Step2
Sensitivity to initialization (Triumphal Arch Sensitivitytoinitialization(TriumphalArch
dataset)
8%
fy
6% 8%
fx
8%
Convergence region
search,Mar21,2008
Convergenceregion
Step2
Camera parameter estimation Cameraparameterestimation
Conclusion
EXIF based approach EXIFbasedapproach
Provideapracticalwaytoinitializingcamera
parameters parameters
InitializationisveryimportanttoBundle
adjustment i e non linear optimization adjustment,i.e.nonlinearoptimization
Costfunctioninthebundleadjustmentisnonlinear,
andnonconvex.
Wheninitialparametersaredistorted,itdoesnot
convergetothesolutionanymore.
search,Mar21,2008
Conclusion Conclusion
SIFT outperforms Harris corner under the SIFToutperformsHarriscornerunderthe
largeviewpointchanges
But, automatic matching doesnt provide But,automaticmatchingdoesn tprovide
consistentlyreliableresultinpractice
Camera parameter estimation is a nonlinear, Cameraparameterestimationisanon linear,
nonconvexproblem
Goodinitializationisveryimportant. y p
EXIFinformationisapracticalwaytoinitialize
cameraparameters.
search,Mar21,2008
References References
HarriscornerdetectorandRANSAC Matlab
http://www.csse.uwa.edu.au/~pk/research/matlabfns/
E i l t ti M tl b Epipolar geometrycompuation Matlab
http://www.robots.ox.ac.uk/~vgg/hzbook/code/
SIFT Matlab
http://vision ucla edu/~vedaldi/code/sift/sift html http://vision.ucla.edu/ vedaldi/code/sift/sift.html
Linearalgebra(GSL) C++
http://www.gnu.org/software/gsl/
Bundleadjustment C++
http://www.ics.forth.gr/~lourakis/sba/
EXIFparser C++
http://www.codeproject.com/KB/graphics/cexif.aspx
Multiple View Geometry in Computer Vision 1
st
ed Richard Hartley and MultipleViewGeometryinComputerVision1
st
ed,RichardHartleyand
AndrewZisserman
Chateaucattledatasetareobtainedfromthetutorialimagesusedin
ImageModeler S/WbyREALVIZ.
search,Mar21,2008

Ransac

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Ransac

Transféré par

Droits d'auteur :

Formats disponibles

Cameraparameterestimationfor

Vous aimerez peut-être aussi