26/4/2009

ComputerVision

Today!
Definition RelatedDisciplines WhyisComputerVisionDifficult? RecognitionMethodology TheThreeProcessingLevels Applications

Prof.GeorgeBebis,Computer VisionLaboratory,Universityof Nevada

26/4/2009

Computer visions great trick is extracting descriptions of the world from pictures or sequences of pictures
(Forsyth/Ponce: Computer Vision)

ComputerVision:
Isanattempttorealizehumanvisualabilityby engineering Itisasonofartificialintelligence Itisabrotherofimageprocessingaswellaspattern recognition,butdefersbecause Itthinksmuchofthreedimensional informationaswell asknowledge ofthesubjectinthescene as knowledge of the subject in the scene

26/4/2009

ComputerVision
Human visual system Object recognition Virtual heritage

1978

1980

1986

1996

Shape-from-shading

Assembly plan from observation Modeling from reality

SomeRelatedDisciplines
ImageProcessing ComputerGraphics PatternRecognition Robotics ArtificialIntelligence

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

26/4/2009

ImageProcessing

ImageRestoration(e.g.,correctingoutfocusimages)

Prof.GeorgeBebis,Computer VisionLaboratory,Universityof Nevada

ComputerGraphics
Geometricmodeling

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

26/4/2009

ComputerVision

Prof.GeorgeBebis,Computer VisionLaboratory,Universityof Nevada

RoboticVision
Applicationofcomputervisionin robotics. b ti Someimportantapplicationsinclude:
Autonomousrobotnavigation Inspectionandassembly

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

26/4/2009

PatternRecognition
Hasaverylonghistory(researchworkinthisfield startedinthe60s). started in the 60s). Concernedwiththerecognitionandclassificationof2D objectsmainlyfrom2Dimages. Manyclassicapproachesonlyworkedundervery constrainedviews(notsuitablefor3Dobjects). Ithastriggeredmuchoftheresearchwhichledto todaysfieldofcomputervision. todays field of computer vision Manypatternrecognitionprinciplesareusedextensively incomputervision.
Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

ArtificialIntelligence
Concernedwithdesigningsystemsthatareintelligent andwithstudyingcomputationalaspectsofintelligence. and with studying computational aspects of intelligence. Itisusedtoanalyzescenesbycomputingasymbolic representation ofthescenecontentsaftertheimages havebeenprocessedtoobtainfeatures. Manytechniquesfromartificialintelligenceplayan importantroleinmanyaspectsofcomputervision. Computervisionisconsideredasubfieldofartificial Computer vision is considered a sub field of artificial intelligence.

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

26/4/2009

WhyisComputerVision Difficult?
Itisamanytoone mapping
A a iety of u fa e Avarietyofsurfaceswithdifferent material a d ith diffe e t ate ial and geometrical properties,possiblyunderdifferent lighting conditions,couldleadtoidenticalimages Inversemappinghasnonuniquesolution(alotof informationislost inthetransformationfromthe3D worldtothe2Dimage)

Itiscomputationallyintensive Wedonotunderstand therecognitionproblem

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

Factorsthatinfluencetheimage
Illuminationcondition Objectshape Cameracharacteristics

26/4/2009

Image
Aimageisamatrixof g pixels Eachpixel
brightness Color Distance
KatsushiIkeuchi,cvl sec@cvl.iis.utokyo.ac.jp

PracticalConsiderations
Imposeconstraints torecoverthescene
G h Gathermoredata(images) d ( ) Makeassumptionsabouttheworld

Computabilityandrobustness
Isthesolutioncomputableusingreasonableresources? Isthesolutionrobust?

Industrialcomputervisionsystemsworkverywell
Makestrongassumptionsaboutlighting conditions Makestrongassumptionsabouttheposition ofobjects Makestrongassumptionsaboutthetype ofobjects

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

26/4/2009

RecognitionMethodology
imageformatione.g.perspectiveor orthographicprojection conditioning labeling grouping extracting a i g matching

DC&CVLab. CSIE NTU

Conditioning
Conditioningisbasedonamodelthatsuggeststhatthe Conditioning is based on a model that suggests that the observedimageiscomposedofaninformativepattern modifiedbyuninterestingvariationsthattypicallyadd toormultiplytheinformativepattern. e.g.noisesuppression, backgroundnormalization b k d l

DC&CVLab. CSIE NTU

26/4/2009

Labeling
Labelingisbasedonamodelthatsuggeststhatthe Labeling is based on a model that suggests that the informativepatternhasstructureasaspatial arrangementofevents,eachspatialeventbeingasetof connectedpixels. e.g.thresholding,edgedetection, cornerfinding f d

DC&CVLab. CSIE NTU

Grouping
Thegroupingoperationidentifiestheeventsby The grouping operation identifies the events by collectingtogetheroridentifyingmaximalconnected setsofpixelsparticipatinginthesamekindofevent. beforegrouping:pixels aftergrouping:setsofpixels e.g.segmentation,edgelinking

DC&CVLab. CSIE NTU

10

26/4/2009

Extracting
Theextractingoperationcomputesforeachgroupof pixelsalistofitsproperties. pixels a list of its properties. exampleproperties:centroid,orientation, area,spatialmoments e.g.regionholes,arccurvature g g

DC&CVLab. CSIE NTU

Matching
Matchingoperationdeterminestheinterpretationof Matching operation determines the interpretation of somerelatedsetofimageevents,associatingthese eventswithsomegiventhreedimensionalobjectortwo dimensionalshape. e.g.templatematching

DC&CVLab. CSIE NTU

11

26/4/2009

Typicalschemeofprocessing
feature extraction make description

input

matching

memorize (learn)

AnIndustrialComputerVisionSystem

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

12

26/4/2009

TheThreeProcessingLevels
Lowlevelprocessing
Standardproceduresareappliedtoimproveimagequality Proceduresarerequiredtohavenointelligentcapabilities.

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

TheThreeProcessingLevels(contd)
Intermediatelevelprocessing
E Extractandcharacterizecomponentsintheimage d h h Someintelligentcapabilitiesarerequired.

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

13

26/4/2009

TheThreeProcessingLevels(contd)
Highlevelprocessing
Recognition and interpretation Recognitionandinterpretation. Proceduresrequirehighintelligentcapabilities.

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

RecognitionCues
Scene interpretation, even of complex, cluttered scenes is a straightforward task for humans.

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

14

26/4/2009

RecognitionCues(contd)
How are we able to discern reality and an image of reality? What clues are present in the image? What knowledge do we use to process this image?

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

Theroleofcolor
What is this object? Does color play a role in recognition? Might this be easier to recognize from a different view?

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

15

26/4/2009

Theroleoftexture
Characteristicimagetexturecanhelpusreadilyrecognize objects.

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

Theroleofshape

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

16

26/4/2009

Theroleofgrouping

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

Illusionduetotheprojection

KatsushiIkeuchi, cvlsec@cvl.iis.utokyo.ac.jp

17

26/4/2009

MathematicsinComputer Vision
Intheearlydaysofcomputervision,visionsystems employedsimpleheuristicmethods. l d i l h i i h d Today,thedomainisheavilyinclinedtowards theoretically,wellfoundedmethodsinvolvingnontrivial mathematics.
Calculus LinearAlgebra ProbabilitiesandStatistics SignalProcessing ProjectiveGeometry ComputationalGeometry OptimizationTheory ControlTheory
Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

ComputerVisionApplications
Industrialinspection/qualitycontrol Surveillanceandsecurity Facerecognition Gesturerecognition Spaceapplications Medicalimageanalysis Autonomousvehicles Virtualrealityandmuchmore...
Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

18

26/4/2009

VisualInspection

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

CharacterRecognition

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

19

26/4/2009

DocumentHandling

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

SignatureVerification

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

20

26/4/2009

Biometrics

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

FingerprintVerification/ Identification

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

21

26/4/2009

FingerprintIdentificationResearchat UNR
Minutiae Matching

DelaunayTriangulation

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

ObjectRecognition

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

22

26/4/2009

ObjectRecognitionResearch atUNR
referenceview1referenceview2

novelviewrecognized

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

IndexingintoDatabases
Shapecontent

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

23

26/4/2009

IndexingintoDatabases
Color,texture

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

TargetRecognition
DepartmentofDefense(Army,Airforce, Navy) N )

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

24

26/4/2009

InterpretationofAerial Photography
Interpretation of aerial photography is a problem domain in both computer vision and photogrammetry photogrammetry.

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

AutonomousVehicles
Land,Underwater,Space

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

25

26/4/2009

TrafficMonitoring

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

FaceDetection

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

26

26/4/2009

FaceRecognition

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

FacialExpressionRecognition

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

27

26/4/2009

HumanActivityRecognition

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

MedicalApplications
skincancer breastcancer

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

28

26/4/2009

InsertingArtificialObjectsintoaScene

Prof.GeorgeBebis, ComputerVisionLaboratory, University of Nevada

Conclusion

Computervisiontechniqueisusefulin variou field Itwillyieldmoreinthefuture

29