AmortizedAnalysisExplained byRebeccaFiebrink PrincetonUniversity ThisisaversionofaWikipediastylereportsubmittedaspartofCOS423underRobertTarjanin Spring2007.Myintentioninpostingthistothewebistobenefitfuturestudentsbyofferinga comprehensivediscussion,selectedindepthexamplesaccessibletostudentswithoutpriorknowledge ofparticulardatastructures,andareviewofapplicationsrelevanttoanalgorithmscourse.

Contents Introduction Overview Keyideas History Comparisontootheranalysistechniques Thebasics Thethreeapproachestoamortizedanalysis Theaccountingmethod Overview Example Thepotentialmethod Overview Example Comparingtheaccountingandpotentialmethods Indepthexamples Movetofront Splaytrees Moreexamples Redblacktrees Fibonacciheaps Disjointsets Maximumflow Dynamicarrays/hashtables Scapegoattrees Conclusions References

Introduction Overview Amortizedanalysisisatechniqueforanalyzinganalgorithm'srunningtime.Itisoftenappropriate whenoneisinterestedinunderstandingasymptoticbehavioroversequencesofoperations.For example,onemightbeinterestedinreasoningabouttherunningtimeforanarbitraryoperationtoinsert anitemintoabinarysearchtreestructure.Incasessuchasthis,itmightbestraightforwardtocomeup withanupperboundby,say,findingtheworstpossibletimerequiredforanyoperation,then multiplyingthisbythenumberofoperationsinthesequence.However,manyrealdatastructures,such assplaytrees,havethepropertythatitisimpossibleforeveryoperationinasequencetotaketheworst casetime,sothisapproachcanresultinahorriblypessimisticbound!Withabitofcleverreasoning aboutpropertiesoftheproblemanddatastructuresinvolved,amortizedanalysisallowsatighterbound thatbetterreflectsperformance. Keyideas Amortizedanalysisisanupperbound:it'stheaverageperformanceofeachoperationinthe worstcase. Amortizedanalysisisconcernedwiththeoverallcostofasequenceofoperations.Itdoesnot sayanythingaboutthecostofaspecificoperationinthatsequence.Forexample,itisinvalidto reason,TheamortizedcostofinsertionintoasplaytreewithnitemsisO(logn),sowhenI insert'45'intothistree,thecostwillbeO(logn).Infact,inserting'45'mightrequireO(n) operations!Itisonlyappropriatetosay,WhenIinsertmitemsintoatree,theaveragetimefor eachoperationwillbeO(logn). Amortizedanalysisisconcernedwiththeoverallcostofarbitrarysequences.Anamortized boundwillholdregardlessofthespecificsequence;forexample,iftheamortizedcostof insertionisO(logn),itissoregardlessofwhetheryouinsertthesequence'10,''160,''2'orthe sequence'2','160','10,''399',etc. Thetwopointsaboveimplythatbothamortizedandworstcaseboundsshouldbeunderstood whenchoosinganalgorithmtouseinpractice.Becauseanamortizedboundsaysnothingabout thecostofindividualoperations,itmaybepossiblethatoneoperationinthesequencerequires ahugecost.Practicalsystemsinwhichitisimportantthatalloperationshavelowand/or comparablecostsmayrequireanalgorithmwithaworseamortizedcostbutabetterworstcase peroperationbound. Amortizedanalysiscanbeunderstoodtotakeadvantageofthefactthatsomeexpensive operationsmaypayforfutureoperationsbysomehowlimitingthenumberorcostof expensiveoperationsthatcanhappeninthenearfuture. Ifgoodamortizedcostisagoal,analgorithmmaybedesignedtoexplicitlyperformthisclean upduringexpensiveoperations! Thekeytoperformingamortizedanalysisispickingagoodcreditorpotentialfunctionthat captureshowoperationswithdifferentactualcostsaffectadatastructureandallowsthedesired boundstobeshown. History Infinance,amortizationreferstopayingoffadebt,suchasaloanormortgage,bysmallerpayments madeovertime.Thepaperintroducingamortizedanalysisforalgorithmsasageneraltechniquewas publishedbyTarjanin1985,thoughherecognizespriorapplicationofthebanker'smethodinworkby BrownandTarjan(1980)andHuddlestonandMelhorn(1981,1982)andcreditsSleatorwiththe

conceptofthepotentialmethod.Aggregateanalysisisanoldermethod(Ahoetal.1974)thatcannow beunderstoodasaformofamortizedanalysis. Earlyapplicationsofamortizedanalysisincludedthemovetofrontlistupdatingheuristic (SleatorandTarjan1985a),selfadjustingandbalancedbinarytrees(Tarjan1985,SleatorandTarjan 1985b),andpathcompressionheuristicsforthedisjointsetunionproblem(Tarjan1983,1985). Amortizationplaysanimportantroleintheanalysisofmanyotherstandardalgorithmsanddata structures,includingmaximumflow,Fibonacciheaps,anddynamicarrays.Alltheseapplicationsof amortizedanalysisarediscussedbelow.Amortizedanalysishasrecentlybeenusedindiverse applicationsincludingsecurity(Maoetal.2006),databases(Agarwaletal.2006),anddistributed computing(FomitchevandRuppert2004),anditcontinuestobeimportanttotheoreticalworkon algorithmsanddatastructures(e.g.,Mendelsonetal.2006;Georgiadisetal.2006). Comparisontootheranalysistechniques Asmentionedabove,worstcaseanalysiscangiveoverlypessimisticboundsforsequencesof operations,becausesuchanalysisignoresinteractionsamongdifferentoperationsonthesamedata structure(Tarjan1985).Amortizedanalysismayleadtoamorerealisticworstcaseboundbytaking theseinteractionsintoaccount.Notethattheboundofferedbyamortizedanalysisis,infact,aworst caseboundontheaveragetimeperoperation;asingleoperationinasequencemayhavecostworse thanthisbound,buttheaveragecostoveralloperationsinanyvalidsequencewillalwaysperform withinthebound. Amortizedanalysisissimilartoaveragecaseanalysis,inthatitisconcernedwiththecost averagedoverasequenceofoperations.However,averagecaseanalysisreliesonprobabilistic assumptionsaboutthedatastructuresandoperationsinordertocomputeanexpectedrunningtimeof analgorithm.Itsapplicabilityisthereforedependentoncertainassumptionsaboutprobability distributionsonalgorithminputs,whichmeanstheanalysisisinvalidiftheseassumptionsdonothold (orthatprobabilisticanalysiscannotbeusedatall,ifinputdistributionscannotbedescribed!)(Cormen etal.2001,923).Amortizedanalysisneedsnosuchassumptions.Also,itoffersanupperboundonthe worstcaserunningtimeofasequenceofoperations,andthisboundwillalwayshold.Anaveragecase bound,ontheotherhand,doesnotprecludethepossibilitythatonewillgetunluckyandencounteran inputthatrequiresmuchmorethantheexpectedcomputationtime,eveniftheassumptionsonthe distributionofinputsarevalid.Thesedifferencesbetweenprobabilisticandamortizedanalysis thereforehaveimportantconsequencesfortheinterpretationandrelevanceoftheresultingbounds. Amortizedanalysisiscloselyrelatedtocompetitiveanalysis,whichinvolvescomparingthe worstcaseperformanceofanonlinealgorithmtotheperformanceofanoptimalofflinealgorithmon thesamedata.Amortizationisusefulbecausecompetitiveanalysis'sperformanceboundsmusthold regardlessoftheparticularinput,whichbydefinitionisseenbytheonlinealgorithminsequencerather thanatthebeginningofprocessing.SleatorandTarjan(1985a)offeranexampleofusingamortized analysistoperformcompetitiveanalysis. Thebasics Thethreeapproachestoamortizedanalysis Thereexistthreemainapproachestoamortizedanalysis:aggregateanalysis,theaccountingmethod, andthepotentialmethod(Cormenetal.2001,p.405).Aggregateanalysisisasimplemethodthat involvescomputingaboundonasequenceofoperations,thendividingbythenumberofoperationsto obtaintheamortizedcost.Alloperationsareassignedthesameamortizedcost,eveniftheyareof differenttypes(e.g.,addingoneitemtoastackorpoppingmultipleitemsfromastack)(p.40610).

Theothertwomethodsaresomewhatmoreinterestingandpowerful,andbyreasoningaboutparticular localorglobalpropertiesofthedatastructures,theyallowforassigningdifferentamortizedcoststo differenttypesofoperationsinthesamesequence. Theaccountingmethod Overview Theaccountingmethod,orthebanker'sview,assignschargestooperationsasifthecomputerwere coinoperated.Onecanthinkofeachoperationalwaysbeingaccompaniedbyinsertingoneormore coinsintothecomputertopayfortheoperation,accordingtoapredeterminedchargeforthatoperation type(e.g.,astackpushmighthaveonecharge,andapopofseveralitemsfromthestackmighthave anothercharge).Thechargedoesnotnecessarilycorrespondtotheactualtimerequiredforthe particularoperation;itispossiblethattheoperationwillcompleteinlesstimethanthetimecharged,in whichcasesomepositiveaccumulationofcreditisleftafterthetimerequiredfortheoperationtofinish issubtractedfromtheoperationpayment.Or,itispossiblethattheoperationwillneedmoretimethan thetimecharged,inwhichcasetheoperationcanbepaidforbypreviouslyaccumulatedcredit. Intheaccountingmethod,theamountchargedforeachoperationtypeistheamortizedcost forthattype.Aslongasthechargesaresetsothatitisimpossibletogointodebt(i.e.,onecanshow thattherewillneverbeanoperationwhoseactualcostisgreaterthanthesumofitschargeplusthe previouslyaccumulatedcredit),theamortizedcostwillbeanupperboundontheactualcostforany sequenceofoperations.Therefore,thetricktosuccessfulamortizedanalysiswiththeaccounting methodistopickappropriatechargesandshowthatthesechargesaresufficienttoallowpaymentfor anysequenceofoperations. Moreformally,denotetheactualcostoftheithoperationby c i andtheamortizedcost(charge) oftheithoperationby c i .If c ic i ,theithoperationleavessomepositiveamountofcredit, credit i=c ici thatcanbeusedupbyfutureoperations.Andaslongas
n i=1 n

ci c i
i=1

,(1)

thetotalavailablecreditwillalwaysbenonnegative,andthesumofamortizedcostswillbeanupper boundontheactualcost(Cormenetal.2001,p.410). Itiscommontokeeptrackofstoredcreditbyassigningittoaparticularlocationinthedata structure,forexamplesomepartofthestructurethatwillbealteredlaterbysomeexpensiveoperation thatwas(partially)madepossiblebytheoperationthatprovidedthecredit.Thislocationcouldbea nodeinatree,apositioninastack,orsomethingelse.Differenttypesofoperationsthatprovidecredit mightstoretheircreditindifferentstructures,butasingleoperationtypewillhaveasingle,well definedlocationtostoreitscredit.Reasoningaboutwhencreditisproduced,whereitisstored,and howitisusedupiskeytoshowingthatEquation1holdsforalloperationsequences.Finally,remember thatamortizationisananalyticaldeviceonly!Thecreditsstoredinthedatastructureareimaginary toolsweusetoreasonaboutthebehaviorofanalgorithmordatastructure,notsomethingthatis explicitoraccessibleinadatastructureabstractionoritsimplementationcode. Example Thisfirstexamplesuppliesatrivialillustrationoftheaccountingmethod,adaptedfromTarjan(1985) andCormenetal.(2001,p.410).Supposeyouhaveasimplestackofitems,whereyoucanpushanitem ontothestackorpopanitemfromthestackinconstanttime.Thestackisconstrainedtonaturally

contain0ormoreitemsatalltimes.Supposethereisonlyoneoperationtypedefinedforthisstack, calledOP,whichinvolvesapplyingzeroormorepopsfollowedbyonepush:
OP(n){ //requiresatleastnitemsareonthestack popnitemsfromstack push1itemontostack

Figure1:OP(3)appliedtoastack Alsosupposethatmsuchoperationshavebeenperformed.TheworstcaserunningtimeforanOP operationinthesequenceisO(m),whichoccursifoperations1throughm1haveeachbeenOP(0) (eachpoppingnoitemsandpushingone),andthemthoperationisOP(m1)(poppingallitemsfrom thestack,thenpushingone).Asimpleworstcaseanalysiswouldconsidermoperationswithaworst caseperoperationcostofO(m)tohaveanupperboundofO(m2). ItshouldbeintuitivelyobviousthatnoteverysingleOPoperationcanhaveacostofm,since previousoperationshavetopushitemsontothestackbeforetheycanbepopped!Amortizedanalysis capturesthisintuition.ConsiderassigningeachOPachargeof2.ThefirstOPmustbeOP(0),popping nothingfromtheemptystackandpushingoneitem.Theactualcostofthisoperationis1,andthe amortizedcostis2,sothecreditremainingis21=1.Thiscreditof1isstoredwiththenewitemin thestack.Ifthisitemispoppedfromthestackinthefuture,thepopwillbepaidforbythisstored1 credit.Inthisway,anysequenceofOPsmaintainstheinvariantthatthereisonecreditstoredper stackeditem.Thealgorithmwillneverrunoutofcredit,becauseeachpopisalreadyprepaid,andeach pushonlyusesup1ofthetwocreditspaidtheOP.TheamortizedcostofeachOPis2,sotheamortized costofanysequenceofmOPsis2m,whichisO(m).ThisisquiteanimprovementovertheO(m2) analysis!

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1

Figure2:Intheaccountingmethod,onecreditisstoredwitheachstack item.OP(3)paysfor3popsusingthepoppeditems'credits,paysfor thepushwithoneofOP'sown2payments,andstorestheotherofits2 paymentsasacredittiedtothenewitem.

Ifadatastructurepossesseslittleornostoredcredit,thisisanindicationthatthebadness resultingfromasubsequentparticularoperationhavingactualcostthatexceedsitsamortizedcostis limited,asthestoredcreditcannotfallbelow0.Ontheotherhand,ifadatastructurepossessesmuch storedcredit,itmightbepossiblethatasubsequentoperationmayhaveactualcostthatfaroutweighs itsamortizedcost,asitmayeatuptheprepaymentsofthepreviousoperations. Thepotentialmethod Overview Thepotentialmethod,orphysicist'sview,definesafunctionthatmapsadatastructureontoareal valued,nonnegativepotential(Tarjan1985;Cormenetal.2001,p.412).Thepotentialstoredinthe datastructuremaybereleasedtopayforfutureoperations.Inthisway,thepotentialissimilarto consideringthetotalcreditstoredthroughoutadatastructureusingtheaccountingmethod.However, thepotentialmethodhasafewotherconceptualdifferencesfromtheaccountingmethod,asdiscussed below. Letusrefertothedatastructureattimei(oralternatively,justafteroperationi),asDi.The potentialfunction( i)mapsDiontoarealvalue.Again,denotetheactualcostofoperationias c i . D Inthepotentialmethod,theamortizedcostofoperationiisequaltotheactualcostplusthe increaseinpotentialduetotheoperation: c i=c i Di Di 1 (2) Thatis,inperforminganalysis,theamortizedcostandpotentialfunctionmustbedefinedsothatthis equivalencealwaysholds.Thisequivalenceimpliesthatanoperationwhoseactualcostislessthanits amortizedcostresultsinanoverallincreaseinpotential,andanoperationwhoseactualcostisgreater thanitsamortizedcostrequiresadecreaseinpotential.Adecreaseinpotentialthereforepaysfor particularlyexpensiveoperations,justasadecreaseincreditsdidfortheaccountingmethod. Justasintheaccountingmethod,wemustenforceanadditionalconditiontoensurethatthe totalamortizedcostofasequenceisalwaysanupperboundonthetotalactualcost.Byequation2,we canderivearelationshipbetweenthetotalamortizedandactualcosts:
n i=1 n

ci= c i Di Di 1
i=1

(3)

Notethatall( i)termsexcept( 0)and( n)cancelout,leaving: D D D

n i=1

ci= c i Dn D 0
i=1

(4)

Toensurethatthetotalamortizedcostisanupperboundonthetotalactualcost,forasequenceofany length,wemerelymustensurethat Di D0 foralli.Typically,( 0)isdefinedtobe0,and D thepotentialisdefinedsothatitisalwaysnonnegative(Cormenetal.2001,p.413). Example Consideragainthesimplestackexampleintroducedabove.Anaturalpotentialfunctionis( i)=the D numberofitemsinthestackafteroperationi.ConsiderperformingOP(k)asoperationi,which performskpopsandonepush.OP(k)hasanactualcostofc=k+1.Ifthestackoriginallycontained nitems,thepotentialbeforetheoperationis( i1)=n,andthepotentialaftertheoperationis( i)= D D (nk+1).TheamortizedcostofOP(k)istherefore:

c i=c i Di D i 1 =k 1 nk 1n =2 .Furthermore,itisclearthatthepotential definedthiswayisnevernegative.Therefore,wecanconcludethattheamortizedcostofOPis2 (O(1)),andthetotalamortizedcostforanysequenceofOPoperationsisanupperboundontheactual costofthesequence. Inpractice,potentialfunctionsareoften,unsurprisingly,abitmorecomplicated.Forexample, onemightwishtodefinethepotentialbasedonmorecomplicatedpropertyofthedatastructure.Also, itmaybenecessarytoenumerateallthepossibleoutcomesofanoperationtoshowthatequation2 holdsforeachofthem.Suchslightlymorecomplicatedapplicationsofthepotentialmethodappearin theindepthexamplesbelow.

Comparingtheaccountingandpotentialmethods Theaccountingmethodchargeseachoperationacertainamountaccordingtoitstype(e.g.,insertor delete),puttingthefocusoneachoperation'sprepaymenttooffsetfutureexpensiveoperations.The potentialmethodinsteadputsthefocusontheeffectofaparticularoperationataparticularpointin time,basedontheeffectsoftheoperationonthedatastructureandthecorrespondingchangein possibilityforfutureoperationstoincurexpense.Theaccountingmethodstorescreditinparticular areasofadatastructure,suchasnodesofatreeoritemsinastack,whereasthepotentialmethod generallyconsiderspropertiesoftheentiredatastructure(suchashowmanynodesareinatree,orhow manyitemsareinthestack)tocomputethepotential. Theaccountingmethodandpotentialmethodareequivalentintermsoftheirapplicabilityto particularproblemsandtheboundstheyprovide(Tarjan1985).However,oneortheothermaybeeasier toapplytoagivenproblem.Tarjansuggeststhatthepotentialmethodmaybemorenaturalwhen fractionalamountsoftimemustbeconsidered,whiletheaccountingmethodislessabstract. Indepthexamples Inthissection,wetakeastepbysteplookatapplyingamortizedanalysistosomerealproblems.The readerisalsoencouragedtoreadChapter17ofCormenetal.(2001),whichprovidesadditionalsimple examples. Movetofront Thefirstexampleappliesthepotentialmethodtothemovetofront(MTF)heuristicforlistaccess andupdating,adaptedfromTarjan1985andSleatorandTarjan1985a(thereaderisreferredtothe originalsourcesformoreinformationonthebackgroundofthisproblem,andaverydetailedanalysis anddiscussionofthealgorithm).AspectsofthisexamplearealsodrawnfromtheCOS423Spring 2000handout(Tarjan2000). TheproblemMTFaddressesisthis:Consideralinearlistofitems(suchasasinglylinkedlist). Toaccesstheitemintheithpositionrequirestimei.Also,anytwocontiguousitemscanbeswappedin constanttime(notincludingthetimetoaccesstheminthelist).Thegoalistoallowaccesstoa sequenceofnitemsinaminimalamountoftime(oneitemmaybeaccessedmanytimeswithina sequence),startingfromsomesetinitiallistconfiguration.Ifthesequenceofaccessesisknownin advance,onecandesignanoptimalalgorithmforswappingitemstorearrangethelistaccordingtohow oftenitemsareaccessed,andwhen.However,ifthesequenceisnotknowninadvance,aheuristic methodforswappingitemsmaybedesirable.Thisproblemisoneformulationofthestandard dictionaryproblem(SleatorandTarjan1985a,p.203). TheMTFisaheuristicthattakesadvantageofthefactthat,forrealproblemssuchaspaging,if itemiisaccessedattimet,itislikelytobeaccessedagainsoonaftertimet(i.e.,thereislocalityof

reference).MTFworksbymovingitemitothefrontofthelistwhenitisaccessed.Thismoveis performedviasuccessiveswapsfrompositioniallthewaydowntoposition1.Therefore,whentheith itemisaccessed,thecostisitoaccesstheitem,plusi1tomovetheitemtothefirstposition,fora totalcostof2i1(Figure3).

a b c d e

a b c d e

a c b d e

c a b d e

Figure3:Movetofront:Toaccess'c'inthe originallist(left),walkdownfrom'a',thenmove 'c'tofrontbyswappingwith'b'then'a'(right) AmortizedanalysiscanbeusedtoshowthatMTFalwaysperformswithinafactorof4ofany algorithmevenanoptimalalgorithmthatknowstheaccesssequenceinadvance,andevenwithout makingassumptionsaboutlocalityofreference.Here,toanalyzewithrespecttoanarbitrary(perhaps optimal)algorithmA,definethepotentialofMTFattimetasthetwotimesnumberofpairsofitems whoseorderintheMTF'slistdiffersfromtheirorderinA'slistattimet.Forexample,ifMTF'slistis ordered(a,b,c,e,d)andA'slistisordered(a,b,c,d,e),thenthepotentialforMTFwillbeequalto2, becauseonepairofitems(dande)differintheirorderingbetweenA'slistandMTF'slist.Thepotential att=0is0,asbothalgorithmsbeginwiththesamelistbydefinition.Also,itisimpossibleforthe potentialtobenegative.Therefore,thetotalamortizedcostisanupperboundonthetotalactualcostof anyaccesssequence. Now,considertheanalysisofaccessingasingleitem,x.LetxbeatpositionkinMTF'slistand atpositioniinA'slist.ThecosttoMTFofaccessingxandmovingittothefrontis2(k1).Thecost forAtoaccessxisi.Notethatmovingxtothefrontofthelistreversestheorderingofallpairs includingxandanitemoriginallyinlocation1tok1(i.e.,k1pairsintotal).Therelativepositions ofallotherpairsareunchangedbythemove.InA'slist,therearei1itemsaheadofx;allofthesewill bebehindxinMTF'slistoncexismovedtothefront.Therefore,thereareatmostmin{k1,i1} pairinversions(i.e.,disagreementsinpairorderbetweenMTFandA)thatareaddedbythemoveto frontofx.Allotherorderingreversals(atleastk1min{k1,i1})mustresultinpairinversion removals(i.e.,thepairorderingsnowagreebetweenMTFandA). Therefore,thepotentialchangeincurredinthissingleaccessandmovetofrontisbounded aboveby2(min{k1,i1}(k1min{k1,i1}))=4min{k1,i1}2(k1).Byequation 2,theamortizedcostofthissingleaccessisboundedaboveas: c = c 2 k 14min {k 1,i 1} 2 k 1 4min {k 1,i1} 4 i So,theamortizedcostofasingleaccessandmovetofrontbyMTFisboundedabovebyfourtimesthe costoftheaccessbyA. OneadditionalconsiderationisthatAmightindependentlyperformswapsinresponsetoanew accessrequest,andthisisnottakenintoaccountabove.SayAdoesswaptwoitems.Thisincursno additionalactualcostonthepartofMTF,butitwillincreaseordecreasethenewpotentialby2,andit

willincreasethecostofaccessforAby1.TheboundonMTF'samortizedcoststillholds,sincethe amortizedcostisincreasedbyatmost2,buttheboundisincreasedby4.Thisistruenomatterhow manyswapoperationsAperforms. Splaytrees Thenextexampleappliesamortizedanalysistosplaytrees.Thispresentationoftheexampleassumes nopriorknowledgeaboutsplaytrees,astheyarequitestraightforwardstructureswhoseessential behaviorisexplainedbelow.Thereadershouldhaveabasicunderstandingofbinarysearchtrees, however,andshouldunderstandtheconceptofatreerotation.(Ifnothingelse,rememberthatarotation isamodificationtosomesubtreeofthesearchtree,whichpreservesthebinarysearchtreeorder propertybuthastheeffectofmovingsomenodestowardstherootandsomenodesawayfromtheroot by1.Asinglerotationcanbeperformedinconstanttimeusingastandardbinarytreeimplementation.) SplaytreesareaselfadjustingbinarysearchtreeintroducedbySleatorandTarjan(1985b).The valuesofthenodesareorderedsymmetrically,meaningthattheleftsubtreeofnodexcontainsnodes whosekeyvaluesarestrictlylessthanx,andtherightsubtreecontainsnodeswhosevaluesaregreater thanx.Splaytreesarenotbalancedtreeslikeredblacktrees;theirstructuremaybearbitrarilybad, (e.g.,Figure7),meaningthatanaccessonatreewithnnodesmayrequirenoperations.However, muchlikethemovetofrontheuristicdescribedabove,splaytreesmoveanelementtotherootofthe treethroughaseriesofsplaysteps(rotations).Thismovehastheeffectoflocalcleanupalongthe pathfromtheaccessednodetotheroot,whichtendstoreducethedepthofthetree,anditiswhat makessplaytreesselfadjusting.Splaytreeshavesomenicepropertiesincomparisontobalanced treessuchasredblacktrees,forexamplesimplerimplementationandanalysis.Usingamortized analysis,itispossibletoshowthatsplaytreesarejustasefficientasbalancedtreesforthestandard operationssuchasaccess,insert,anddelete(SleatorandTarjan1985b).Thefollowingexampleshows thisinastepbystepanalysis,whichiscompiledfromSleatorandTarjan(1985b)andTarjan(1985),as wellascoursematerialsavailableonlinebyWayne(2001),TarjanandMulzer(2007),andKargerand Zhang(2005)Thereaderisencouragedtoconsultthesesourcesforadditionaldiscussionand clarification. First,letusintroducetheSPLAY(x,S)operationonsomeelementxandasplaytree,S.This operationhastheeffectofrotatingxuptotherootofSifxisinS,orelserotatingthenextbiggestor nextsmallestelementtoxuptotheroot(dependingontheimplementation).SPLAYworksby repeatedlyperformingrotations.Ateachiterationofthewhileloop,tworotations(cases2ad)are performedtoreplacex'sgrandparentwithx,oronerotation(cases1ab)isperformedtoputxatthe root,ifx'sparentistheroot.Figures4through6illustratetherotationsforcases2a,2c,and1a;cases 2b,2d,and1baresymmetric.
SPLAY(x,S){ whilexisnotrootofS{ ifx'sparentisroot{ ifxisaleftchild,doarightrotation(case1a) elsexisaleftchild,sodoaleftrotation(case2a) //xisnowroot }else{ ifxisaleftchildanditsparent,y,isaleftchild,do2rightrotations(case2a) elseifxisarightchildandyisarightchild,do2leftrotations(case2b) elsexisarightchildandyisaleftchild,sodo1leftthen1rightrotation(case2c) elseifxisaleftchildandyisarightchild,do1rightthen1leftrotation(case2d) } } }

z y x C D A

x y B C B z A D y x C A A y

z D x A y B

x z C D

B A Figure4:SPLAYcase2a

C B B A Figure5:SPLAYcase2c x y C B

B A B A Figure6:SPLAYcase1a

SPLAYmovesxuponelevelinthetreeeachtimearotationisdone.Thiscouldresultinatotal of0rotations,inthecasewherexisalreadytheroot,oritcouldben1rotations,whenthetree containsnnodesinasinglechainandxisthelowestinthechain(e.g.,item1inFigure7).Therefore,a worstcaseboundontheSPLAYoperationisO(n)fornnodes. Figure7showsthesequenceoftreeconfigurationsthatoccuroverthewhileloopin SPLAY(1,S).Indeed,9rotationsareperformedonthetreewith10nodes.However,noticethatthe treeisshallowerafterthisaccessandsplay;thereisnowaytodoanotheraccessthatwillresultin SPLAYtaking9operations.Infact,considertheoperationSPLAY(2,S)onthisnewtree(Figure8). Thissplayonlyrequired5rotations,anditleftthetreeevenmorebalanced.Amortizedanalysisis usefulforreasoningaboutthefactthatperformingaveryexpensiveoperation,suchasSPLAY(1,S) ontheoriginaltreeinFigure7,performssomesortofcleanupthatlimitstheexpenseoffuture operations. 10 9 6 7 10 9 6 4 3 7 5 10 9 1 2 4 3 9 10 10 1 8 6 9 4 7 2 5 3 1 6 5 8 10 9

1 Figure7:SPLAY(1,S)ona"bad"tree(fromWayne2001)

8 6

4 2

1 2

4 3

1 6 5 8

10 9

1 3

2 4 6 5

3 Figure8:SPLAY(1,S)ontreefrom Figure7 TheSPLAYoperationiscalledonceforeachInsert,Delete,Join,andFindoperation.ToFind elementxintreeS,forexample,wefirstcallSPLAY(x,S),tomakextherootofSifxisinthetree. ThecostoftheSPLAYoperationisthereforecrucialtoanalyzingallotheroperations. Now,wedefineapotentialfunctioninordertoperformamortizedanalysis.LetSi(x)denotethe subtreeofSrootedatnodexattimei(specifically,aftertheithiterationoftheSPLAYwhileloop,for example),and|Si(x)|denotethenumberofnodesinthatsubtreeatthattime.Definetherankfunction tobe r i x=logS i x .Letri(x)indicatethelocalpotentialofnodexinthetreeSattimei. Finally,letouroverallpotentialfunctionattimeibe i = r i x . ItcanbeshownthattheamortizedcostofanysplaysteponnodexisO(logn).Toprovidethis bound,wecanenumerateallthepossiblechangestothetreethatasingleiterationoftheSPLAYwhile loopmightcause.Then,foreachcase,wecomputethepotentialfortheentiretreebeforeandafterthe loopiteration,andtheactualcostrequiredtoperformtheiteration.Thisgivesusaboundonthe amortizedcostofasingleloopiteration,whichwecanthensumoveralliterationsofthelooptoobtain aboundontheamortizedcostoftheentireSPLAYoperation. Choosecase2afromSPLAYtostartwith.Figure4showstheconfigurationbeforeandafterthe tworightrotations.Thetotalactualcostofthe2rotationsis2.Thechangeinpotentialintroducedby thesetworotationsisthesumofthechangesinpotentialateachnode: i i1= r i x r i1 x = r i x r i1 x Notethatcase2aonlychangesthepotentialofnodesx,y,andz:thesizesofallothernodes'subtrees remainunchanged.Furthermore,observethat r i x=r i1 z , r i 1 y r i 1 x ,and r i yr i x .Byequation2,theamortizedcostofthissinglecaseis:
c i = c i i i1 = 2r i x r i1 x r i yr i1 yr i zr i1 z = 2r i xr i1 zr i yr i zr i1 x r i1 y 20r i x r i zr i1 x r i1 x = 2r i x r i z 2 r i1 x . (5)
x tree x tree xtree x tree

10 9

Wewouldliketogetthedependenceonzoutofthisexpressionsomehow.Wecanaccomplishthisby takingadvantageofthefollowingpropertyoflogarithms: log alog b ab log 2 2 So,

r i 1 xr i z = log S i1 x log S i z 2 log

S i 1 xS i z 2

Usingthepropertythat S i 1 xS i zS i x , S i x r i 1 xr i z 2 log = 2 log Si x 2log2 2 r i x2 2

so, r i z 2 r i x r i 1 x 2 . Substitutingthisbackintoequation5,weseethat
c i 2r i x 2 r i x r i1 x 22 r i1 x = 3 r i x 3r i1 x = 3r i xr i1 x

(6).

Theanalysisofcases2b,2c,and2darenearlyidentical.Forcases1aand1b,thereasoningissimilar, buttheactualcostciisonly1,not2,sotheboundis c i 3r i x r i 1 x1 . Now,wehaveanupperboundontheamortizedcostforanyindividualiterationofthewhile

m

loopofSPLAY.ThetotalamortizedcostofanentireSPLAYoperationis c = ci ifthewhileloop
i=1

executesmtimes.Usingequation6,andtakingadvantageofthetelescopingpropertyandthefactthat eithercase1aor1bwillonlybeexecutedonce,weget:
m m

c =

ci
i=1

3r i x r i 1 x 1=3r m x r 0 x 1
i =1

Notethatrm(x)=nbecause,attimem,xistheroot,soallnnodesareinitssubtree.Noteagainthat r0(x)maybeassmallas1,ifxisataleaf.Therefore,ouramortizedboundforoneSPLAYoperation, c ,isO(logn). Finally,thisboundonSPLAYcanbeusedtoeasilyshowamortizedboundonalloperationsof splaytrees.Trivially,theFind(x)operationrequiresoneSPLAY,soitscostisalsoO(logn).Other operationsareslightlymoreinvolved;thereaderisdirectedtoKargerandZhang(2005),Sleatorand Tarjan(1985b),orTarjan(1983,p.536)formoredetails. Moreexamples Theexamplesaboverequirelittlebackgroundknowledgeofthedatastructuresorproblemarea.This sectionprovidesbriefoutlinesoftheuseofamortizedanalysisinseveralotheralgorithmsanddata structuresthataresomewhatmoreinvolved.Thissectionisprovidedsothatreadersfamiliarwiththese topicsmayappreciatetheroleamortizationhasplayedintheiranalysisand/ordesign,andsothat readerscomfortablewiththeexamplespresentedthusfarmayconsultsourcesforthefollowingtopics inordertolearnmore. Redblacktrees Redblacktreesareatypeofbalancedbinarysearchtreethatenforcesbalancebycoloringeachnodein thetreeeitherredorblack,thenreadjustingthetreeafteroperationssuchasinsertanddelete, accordingtothecolorsofthenodes(Cormenetal.2001,Chapter13).Insimple(unbalanced)binary

searchtrees,basicoperationssuchasinsertanddeletecanruninO(n)timeonatreeofsizen.Forred blacktrees,theseoperationsruninO(logn)time.However,thereisincreasedoverheadinvolvedinthe structuralmodificationsperformedtomaintaintreebalance.Intheworstcase,anyoperationcancause O(logn)modifications.However,amortizedanalysiscanbeusedtoshowthatanysequenceofminsert anddeleteoperationscausesO(m)structuralmodificationsintheworstcase(Cormenetal.p.428).It cansimilarlybeshownthatthetotaltimeformconsecutiveinsertionsinatreeofnnodesisO(n+m) (Tarjan1985). Tarjan(1985)usestheaccountingmethodtoshowthissecondbound.Theanalysisshowsthat theinvariantismaintainedthateveryblacknodeinthetreecontains0,1,or2credits,dependingon whetherithasoneredchild,noredchildren,ortworedchildren,respectively.Initially,O(n)creditsare addedtothetree.Theanalysistakesthefollowingform:allpossiblelocaleffectsofarebalancing operationfollowinganinsertoperationareenumerated.Theinvariantisshowntoholdforallofthese cases,wheretheterminatingcasesrequirepaymentofO(1)credits,andnonterminatingcases(which requireoneormoreadditionalrebalancingoperations)useupavailablecreditinthetreetoperform theiroperations.Becauseeachoperationincludesonlyoneterminatingcasebydefinition,eachinsert operationmustpayonlyaconstantamountofcredit.Therefore,theoverallamortizedcostofasingle insertoperationisO(1),andsotheamortizedcostofanysequenceofminsertoperationsisO(n(to inserttheinitialncredits)+m). Fibonacciheaps Fibonacciheapsareaheapdatastructurerepresentedbyaforestofheaporderedtrees(Fredmanand Tarjan1987).TheyallowdeletionfromaheapwithnitemsinO(logn)amortizedtimeandsupport otherheapoperations(insert,findmin,meld,decreasekey)inO(1)amortizedtime.Fibonacciheaps havebeenusedasunderlyingdatastructurestoachievebettertheoreticalboundsonminimumspanning treesandshortestpathproblems. Cormenetal.(2001,Chapter20)employthepotentialmethodtoanalyzetherunningtimeof Fibonacciheapoperations.Thepotentialfunctionusedis H =t H 2 m H wheretisthe numberoftreesintherootlistofheapHandm(H)isthenumberofmarkednodesinH.Withthis potentialfunction,itisfairlyeasytoshowO(1)amortizedtimeforeachnondeleteoperation individually,andO(lgn)timeforextractminanddelete(seethebookfordetails). Disjointsets Disjointsetdatastructuresmaintainacollectionofdisjointdynamicsets(Cormenetal.2001,Chapter 21)andsupportoperationstomakeaset,joint(union)twosets,andfindthesetthatcontainsa particularelement.Thestraightforwardunionalgorithmforadisjointsetforestimplementationtakes O(n)timefornelements,usingbothamortizedandworstcaseanalysis.However,usingtheunionby rankandpathcompressionheuristics,onecanshowthatanysequenceofmmake,union,andfind operationsrunsinO(m( n))time,whereistheslowlygrowingAckermannfunction.Tarjan performsthisanalysisusingtheaccountingmethod(1983,p.2430),andCormenetal.(2001,p.5138) useapotentialfunction.Bothanalysesrequireunderstandingofthedetailsofpathcompressionthatlie outsidethescopeofthispaper,butthecuriousreaderisdirectedtotheTarjanandCormentextbooks formoredetailedtreatment. Maximumflow Thegoalofmaximumflowproblemsistofindthemaximumflowthroughagraph,whereonenodeis asource,onenodeisasink,andeachedgeinthegraphhasaknowncapacity(Cormenetal.2001,

Chapter26).Thiswellstudiedproblemhasseveralsolutionapproaches.Onesuchapproach,called pushrelabel,includesmanyoftheasymptoticallyfastestsolutionstothemaximumflowproblem (Cormenetal.,p.609).Ananalysisofpushrelabelhingesonfindingaboundonthenumberof nonsaturatingpushesperformedbythealgorithm.Thisisdoneusingapotentialfunctiondependenton theheightsoftheverticesinthegraph(Cormenetal.,p.6789). Dynamicarrays/hashtables Adynamictableisexpandedorcontractedasdataisaddedtoorremovedfromit.Itistherefore necessarytodesignandreasonaboutalgorithmsformanagingtablesize.Usingonesimplealgorithm describedinCormenetal.(2001,p.41624),asequenceofmoperationscanbeshowntotakeO(m) amortizedtime. Scapegoattrees Scapegoattreesareanothertypeofbalancedbinarysearchtree(GalperinandRivest1993).Unlikered blacktrees,theydonotstoreadditionalinformationateachnode.Amortizationusingapotential functionbasedonthesumoflocalnodepotentialsyieldsO(logn)insertionanddeletion. Conclusions Amortizedanalysisisausefultoolthatcomplementsothertechniquessuchasworstcaseand averagecaseanalysis.Ithasbeenappliedtoavarietyofproblems,anditisalsocrucialtoappreciating structuressuchassplaytreesthathavebeendesignedtohavegoodamortizedbounds. Tounderstandtheapplicationofamortizedanalysistocommonproblems,itisessentialtoknow thebasicsofboththeaccountingmethodandthepotentialmethod.Theresourcespresentedhere supplymanyexamplesofbothmethodsappliedtorealproblems. Toperformanamortizedanalysis,oneshouldchooseeithertheaccountingmethodorthe potentialmethod.Theapproachesyieldequivalentresults,butonemightbemoreintuitively appropriatetotheproblemunderconsideration.Thereisnomagicformulaforarrivingatapotential functionoraccountingcreditschemethatwillalwayswork;themethoduseddependsonthedesired boundsandthedesiredcomplexityoftheanalysis.Somestrategiesthatsometimeswork,however, includeenumeratingthewaysinwhichanalgorithmmightoperateonadatastructure,thenperforming ananalysisforeachcase;computingthepotentialofadatastructureasasumoflocalpotentialsso thatonecanreasonabouttheeffectsoflocalchangeswhileignoringirrelevantandunchanging componentsofthestructure;designingthepotentialmethodaroundthedesiredformoftheresult(e.g., relatingthepotentialtothelogofthesubtreesizeforsplaytrees,asthedesiredoutcomeisa logarithmicbound);andreasoningabouteachtypeofoperationinasequenceindividuallybefore comingupwithaboundonanarbitrarysequenceofoperations. Anunderstandingofamortizedanalysisisessentialtosuccessinanalgorithmscourse,to understandingtheimplicationoftheoreticalboundsonrealworldperformance,andtothoroughly appreciatingthedesignandpurposeofcertaindatastructures.Thereaderisthereforeagainurgedto consultanyofthesourcesmentionedheretoimprovehisorherunderstandingofamortizedanalysis andtoexplorethesealgorithmsingreaterdepth.

References Aho,A.V.,J.E.Hopcroft,andJ.D.Ullman.1974.Thedesignandanalysisofcomputeralgorithms. AddisonWesley. Agarwal,P.K,J.Xie,J.Yang,andH.Yu.2006.Scalablecontinuousqueryprocessingbytracking hotspots.Proceedingsofthe32ndInternationalConferenceonVeryLargeDatabases32:31 42. Brown,M.R.,andR.E.Tarjan.1980.Designandanalysisofadatastructureforrepresentingsorted lists.SIAMJournalonComputing9(3):594614. Cormen,T.H.,C.E.Leiserson,R.L.Rivest,andC.Stein.2001.Introductiontoalgorithms,2nded. MITPress. Fomitchev,M.,andE.Ruppert.2004.Lockfreelinkedlistsandskiplists.Proceedingsofthe23rd AnnualACMSymposiumonPrinciplesofDistributedComputing,509. Fredman,M.L.,andR.E.Tarjan.1987.Fibonacciheapsandtheirusesinimprovednetwork optimizationalgorithms.JournaloftheACM34(3):596615. Galperin,I.,andR.L.Rivest.1993.Scapegoattrees.ProceedingsoftheFourthAnnualACMSIAM SymposiumonDiscreteAlgorithms,16574. Georgiadis,L.,R.E.Tarjan,andR.F.Werneck.2006.Designofdatastructuresformergeabletrees. Proceedingsofthe17thAnnualACMSIAMSymposiumonDiscreteAlgorithms,394403. Huddleston,S.,andK.Melhorn.1981.RobustbalancinginBtrees.5thGIConferenceon TheoreticalComputerScience:LectureNotesinComputerScience104.NewYork:Springer Verlag.23444. Huddleston,S.,andK.Melhorn.1982.Anewdatastructureforrepresentingsortedlists.Acta Informatica,17:15784. Karger,D.,andX.Zhang.2005.Splaytrees.ScribenotesforSeptember12,6.854Advance Algorithms,MIT.Available:http://courses.csail.mit.edu/6.854/05/scribe/dzhangsplaytrees.ps. Mao,Y.,Y.Sun,M.Wu,andK.J.R.Liu.2006.JET:Dynamicjoinexittreeamortizationand schedulingforcontributorykeymanagement.IEEE/ACMTransactionsonNetworking14(5): 112840. Mendelson,R.,R.E.Tarjan,M.Thorup,andU.Zwick.2006.Meldingpriorityqueues.ACM TransactionsonAlgorithms2(4):53556. Sleator,D.D.,andR.E.Tarjan.1985a.Amortizedefficiencyoflistupdateandpagingrules. CommunicationsoftheACM28(2):2028.

Sleator,D.D.,andR.E.Tarjan.1985b.Selfadjustingbinarysearchtrees.JournaloftheACM32(3): 65286. Tarjan,R.E.1983.Datastructuresandnetworkalgorithms.CBMSNSFRegionalConferenceSeries inAppliedMathematics. Tarjan,R.E.1985.Amortizedcomputationalcomplexity.SIAMJournalonAlgebraicandDiscrete Methods6(2):30618. Tarjan,R.E.2000.Amortizedanalysisofmovetofront(MF)listrearrangement.Coursehandoutfor COS423,TheoryofAlgorithms,PrincetonUniversity.Available: http://www.cs.princeton.edu/courses/archive/spr00/cs423/handout2.pdf. Tarjan,R.E.,andW.Mulzer.2007.ScribenotesforFebruary17,COS423,TheoryofAlgorithms, PrincetonUniversity.Available: http://www.cs.princeton.edu/course s/archive/spring07/cos423/lectures/lecture4.pdf. Wayne,K.2001.Amortizedanalysis.CourseslidesforCOS423,TheoryofAlgorithms,Princeton University.Available:http://www.cs.princeton.edu/~wayne/cs423/lectures/amortized4up.pdf.