MakingPHPSee

Confoo2011
MichaelMaclean
mgdm@php.net
http://mgdm.net

Youwanttodowhat?

PHPhasmanywaystocreategraphics
Cairo,ImageMagick,GraphicsMagick,GD...

Youwanttodowhat?

Therearen'tthatmanywaystoprocessgraphics

IntroducingOpenCV

TheOpenComputerVisionLibraryhas>500
algorithms,documentationandsamplecodefor
realtimecomputervision.

IntroducingOpenCV

OriginallydevelopedbyIntel,releasedasopen
sourceundertheBSDlicence

IntroducingOpenCV

http://opencv.org

Why?

BecauseICan

Why?

Iwantedtolearnaboutit,butusealanguageI
wasmorecomfortablewith

Why?

Ithoughtitmightbeusefultoothers

Applications

Objectrecognition
Thisincludesfacedetection!

Applications

Gesturetracking(perhapsnotthatusefulin
PHP...)

Applications

Structurefrommotion
Creating3Dmodelsofobjectsfrom2Dinputs

Applications

Structurefrommotion
Again,maybenotinPHP,butoneday...

Applications

OthercoolthingsI'venotyetthoughtof

ABriefDisclaimer

It'snotmyaimforthistalktogiveashort
courseoncomputervision

ABriefDisclaimer

Rather,I'mgivingatourofaninterestingbitof
software

ABriefDisclaimer

(I'mcertainlynotthebestqualified,anyway!)

ABriefDisclaimer

Iwillnotexplainthetheorybehindmanythings
(asImightnotunderstanditmyself)

ABriefDisclaimer

I'mnotreallyamathsgeek,soifthereareany
inaccuracies,pointthemout

Gettingit

It'swillbeonGithubveryshortly
http://github.com/mgdm

Gettingit

You'llneedtheOpenCVlibrary,whichisinmost
Linuxdistributionsthesedays
YouneedPHP5.3

Gettingit
I'venotdoneaWindowsbuildyetit'sonthe
TODO
Patchesarewelcome

Installing
It'sthenormalPHPextensionbuildsystem
phpize
./configure
make
make install

Addextension=opencv.sotophp.ini

Basicusage

EverythinginthelibraryisundertheOpenCV
namespace
Ifyou'renotfamiliarwithnamespaces,check
http://php.net/namespaces

Basicusage

Let'sstartbyloadingatestimage

LoadingandSavingImages
$image = OpenCV\Image::load("test.jpg");
$image ->save("test2.jpg");

YounowhaveanImageobject
Thesameimagehasbeensavedastest2.jpg
Ifthathasworked,thelibraryissetupcorrectly.
Congratulations

BasicsofOpenCV

Ittreatsimagesasbeingfundamentallymatrices
ofnumbers

BasicsofOpenCV

So,manyofthesameoperationsyou'dperform
onamatrixcanalsobeperformedonanimage

BasicsofOpenCV

Thisincludesthingsliketransposition,adding,
multiplying,addingscalars,etc

BasicsofOpenCV

Thereisaverylargelistofthesebasicoperators
NotallofwhichIunderstand...

ImageProcessing

OpenCV(predictably)hasmany,manyfunctions
thatdovariousthingstoimages
Theserangefromthefairlymundanetothevery
powerful

Smoothing

Notveryexciting,butagooddemonstrationa
Gaussianblur

Smoothing

$dst = $image->smooth(
OpenCV\Image::GAUSSIAN, 31, 0, 0, 0);

$dstwillnowcontainaslightlyfuzzyversion
oftheimage

The input

The result

Smoothing,continued

Whywouldyouwanttodothat?
Perhapstoremovenoisefromanimagetakeninlow
light

Smoothing,continued

Variousmethodsareavailable,accessiblevia
differentparameterstosmooth()

Medianblur

Gaussian

Bilateralfiltering

ImageMorphology

(Soundscool,doesn'tit?)

ImageMorphology
Theseareaclassofoperatorswhichconvolvean
imagewithakernel
Thekernelislikeamask,whichisscanned
acrosseverypointintheimage
Ithasananchorpoint,whichrepresentsthepixel
beingcurrentlytransformed

Kernels
0

Forsomeoperations,eachnumber
representstheamountofinfluencethat
pixelhasontheoutput

Dilation

Akernelconsistingofa3x3square,withan
anchorinthecentre,isscannedovertheimage
Foreachpointthattheanchorgoesover,the
maximumvaluecoveredbythekernelisfound

Dilation

Thismaximumvaluebecomesthevalueofthe
pixelatthatpositioninthenewimage
Theupshotofthisis,brightareasgetlarger

Erosion
Thisisfundamentallytheoppositeofdilation
Insteadofthemaximumvalueunderthekernel,
welookfortheminimum
Brightareasgetsmaller

What'sthepoint?

Brightareasmightbepartofthesameobject,
butsplitintoseveralpartsintheimage
(Shadows,obstructions)

What'sthepoint?

Theseoperationswillcausetheseareastojoin
up,makingthemeasiertoidentify

Edgedetection
OpenCVfeaturesseveralalgorithmsforedge
detection
YoumightwellhaveseenthesebeforeinGIMPor
Photoshop

Youcanusethemtogetoutlinesofobjects

Sobel
TheSobeloperatorisawaytogetaderivative
ofanimage
(Mathsgeeksmaypointoutthisistechnically
incorrectjustrunwithitfornow)

Sobel

Youendupwithanimagewhichhasbright
areaswherethereislotsofcontrast

Theresult

Canny

Amorecomplexalgorithm,whichtakes
derivativesinxandy
Somemoreprocessingresultsinafarclearer
line

Theresult


Andnowforsomething
completelydifferent
moreinteresting

Templatematching

YoucanuseOpenCVtotryandlocateanimage
withinanotherimage
You'dusuallydothiswithasmallimageofan
object,andalargersearchimage

Mytestimage

(Itisonly59pixelswide,excusethefuzziness)

Myotherimage

ThankstoMichaelangelovanDamfortheCreativeCommonspicture
http://www.flickr.com/photos/dragonbe/3411273755/

Theresult

Theresult

It'snotperfect,butit'sastart

Theresult

ThereasonitmatchesmostoftheElePHPantsis
becausethesmallimageis,well,small,andhas
beenresavedasaJPEG

Theresult

Thus,thepixelsitissearchingwithareno
longeridenticaltotheoriginals

Histograms
Histogramsareawayoffittingvaluesintoslots,
orbins,asOpenCVcallsthem
Theyarefundamentallythesameasanarray,
withacountofvalues,andcanbegraphedasa
barchart

I'mnotgoingtodoanything
quitethatdull

Histograms

OpenCVuseshistogramstorecordasignature
offeaturesinanimage

Histograms

Forexample,youcanusethemtorecordthe
coloursinasectionofanimage
(Stayawakeattheback)

Backprojection

IfIconvertthisimagetoHSVformat,Icangeta
versionoftheimagewhichencodesthehueas
onechannel

Backprojection
ThenIcanconvertthisintoahistogram,toget
ahashofsortsoftheproportionofcoloursin
theimage
Then,OpenCVwilltakethishistogramandlook
foritinanotherimage

Anaside...

Thisiswhatanimagelookslikewhenit'sconvertedtoHSVandsaved
asifitwasstillRGB

Thecode
<?php
use OpenCV\Image as Image;
use OpenCV\Histogram as Histogram;
echo "Loading sample\n";
$i = Image::load("elephpant_sample.jpg",
Image::LOAD_IMAGE_COLOR);
$hsv = $i->convertColor(Image::RGB2HSV);
$planes = $hsv->split();
$hist = new Histogram(1, 32,
Histogram::TYPE_ARRAY);
$hist ->calc($planes[0]);
echo "Loading main image\n";
$i2 = Image::load("dragonbe_elephpants.jpg",
Image::LOAD_IMAGE_COLOR);
$hsv2 = $i2->convertColor(Image::RGB2HSV);
$planes2 = $hsv2->split();
$result = $planes2[0]->backProject($hist);
$result ->save("bp_output.jpg");

Theresult

Theresult

Asyoucansee,theresultisabinaryimage
showingtheplaceswherethehistogram
matchedthecoloursintheimage

Theresult

Youcouldprobablygetabettermatchwitha
bettersample

Theresult

Youcanthengoonanddootherthingswiththe
morphologyoperators,tofindtheextentsof
eachmatch

Theresult

Oryoucanusethatimageasamaskforother
operations

Otherusesforhistograms

Incombinationwiththeedgedetectionstuff
fromearlier,youcanmakeahistogramofthe
edgegradients

Otherusesforhistograms

Thismeansthatyoucanthentryandmatch
particularshapesusingahistogram
Thisisusedbysomeforgesturerecognition

Machinelearning

Inadditiontotheimageprocessingfunctions,
OpenCVincludessomemachinelearning
capability

Machinelearning

Thealgorithmsaregeneral,andnotreally
specifictocomputervision

Machinelearning

Generally,thealgorithmsaretrainedusinga
largedataset,andthentestedagainstanother

Machinelearning

Therearetwomainwaysofimplementingthis

Supervisedlearning,inwhichtheinputdatais
labelled

Unsupervisedlearning,wherethedatahasnolabels

Supervisedlearning

Inthismethod,thealgorithmknowsthatthe
featureitisbeingtrainedoniseitherpresentor
notpresent

Supervisedlearning

Itcanthenlearnthatcertaincharacteristicsare
distinctiveamongthosewherethefeatureis
present

Supervisedlearning

Forexample,youcanhaveadatasetof10,000
images,inwhich8,000containfacesand2,000
donot

Unsupervisedlearning

Thesealgorithmsarereferredtoasclustering
algorithms
Theyareusedtomatchupsimilarimages,
withoutknowledgeofwhatiswithin

Training

OpenCVcomeswithtoolsfortrainingthe
variousalgorithmsitsupplies
I'mnotgoingtoexplainthemhere

Haarclassifier

Thisisafairlyspecializedimplementationofa
learningalgorithm

Haarclassifier

Itisusedtodetectmostlyrigidobjects
Likefaces!

Haarclassifier

OpenCVcomeswithapretrainedHaarclassifier
capableofrecognisingfaceswithinanimage

Haarclassifier
<?php
use OpenCV\Image as Image;
use OpenCV\Histogram as Histogram;
$i = Image::load("sailing.jpg",
Image::LOAD_IMAGE_COLOR);
$result = $i->haarDetectObjects("
haarcascade_frontalface_default.xml");
foreach ($result as $r) {
$i->rectangle($r['x'], $r['y'],
$r['width'], $r['height']);
}
$i ->save("sailing_detected.jpg");

Theinputimage

Theresult

Otherfeatures

Thereareotherinterestingfeaturesofthe
librarythatI'venotimplementedyet,butthey
willbetheresoon

Inpainting
Repairingdamagetoimagesforexample

strippingwatermarks

fillingintextureaftertheimagehasbeenrotated

Todothisproperly,youhavetocreateamask,
andI'venotworkedoutareasonableAPIfor
that

Contourdetection

OpenCVcanuseitsedgedetectionalgorithmsto
findthecontoursofanimage
Thesearestoredinasequence,whichcanbe
iteratedover

Imagecapture

OpenCVcangrabimagesfromacamera
Iwillnowattempttodemothis...

Thanksforlistening
Pleaseratemytalk!
http://joind.in/2830
Getintouch
mgdm@php.net
http://mgdm.net
@mgdm