FrequentlyAskedQuestionsonTAPENADE

Lastmodified:TueApr1010:36:09CEST2012 bythedevelopersofTapenade

Weencourageyoutosubscribe tothe"tapenadeusers"mailinglisttosharewithotherusers,togetthelatestnews,andto submit yourquestionsandremarks!

Problemswiththewebinterface:
Problemswithwebbrowsers(Macintosh) Multiplesessionsinparallel Approximativesourcecodecorrespondence

ProblemsrelatedtoInstall:
DownloadTapenade OldversionsofTapenade Systemerrorsofthe"notfound"sort JAVAVersionrequired TapenadeandWindows Tapenade,Windowsandcygwinproblem TapenadewithCprogramsonWindows,problemwithcpp

Differentiationcrashes:
Directoryseparatorlinux/windows LanguagesacceptedbyTapenadeparsers,Syntaxerrors Syntaxofdirectivesthatmodifydifferentiation($AD...) JAVAerrors:NullPointerException,ArrayOutOfBoundsException,etc Systemerrorsofthe"outofmemory"sort Errorsondifferentiateddeclarations Errorsoncomplicatedstatements

Differentiatedprogramdoesn'tcompile:
MissingdifferentiatedroutinesforintrinsicorBlackBoxroutines

Differentiatedprogramcrashes:
DivisionsbyZeroandNaN's PUSH/POPmechanism Wrongexternal's Somecasesof"segmentationfault"

ReturnedDerivativesarewrong(orjustneedtobevalidated):
Basicandextremelyusefulcodepreparationbeforedifferentiation Validationofthedifferentiatedprograms

Functionnalities:
Buildingthecontextforcallingthedifferentiatedcode Copiedproceduresinthedifferentiatedcode ToomanyWarningandErrormessages IncludesandComments Typemismatches Aliasing LanguageacceptedbytheA.D.engineLimitations Blackboxprocedures,unknownexternalprocedures Domainofvalidityofderivatives OriginalstatementsandresultsdisappearingfromtheReversecode Differentiatingmultipletop/headprocedures,DifferentiatingLibraries IndependentandDependentsets.Initializationandusageofthedifferentiatedparameters Defaultsizesofbasicdatatypes Parameterizedarraysizes Multidirectionalmodes(i.e.option u t ) mli GettingtheJacobianusingtheMultidirectionalmode SecondDerivativesandHessians

PerformanceImprovements:
Controllingcheckpointinginthereversemode IndependentIterationsLoops($ADIILOOP) BinomialCheckpointing($ADBINOMIALCKP) Userdefinedadditionalcheckpoints($ADCHECKPOINTSTARTand$ADCHECKPOINTEND)

LinearSolvers

Documentation:
Acronym AllWarningorErrormessages OtherDifferentiationModes OtherADtools DifferenceswithOdysse Documentation,ArchivesoftheMailinglist Sendingremarks,questions,andbugreports

Acronym:
WhatdoesTAPENADEmean?Whydidyouchoosethisname? Well,weareallsearchingfortheultimateAutomaticDifferentiationtool,butwewouldbesillytoclaimthatwefoundit!Therefore, TAPENADEcouldmean"TangentandAdjointPENultimateAutomaticDifferentiationEngine". Honestly,wearrangedthisacronymafterwechosethename!WesimplylikedthenameTAPENADEbecauseitisatypical ingredientofourlocalcuisine.Itismadewithcapers,olives,andanchovies,anditmakesaperfectapritifinsummer,onsmall toastswithaglassofcoldroswine.Justincaseyoucouldn'tfindsomeinyourlocalgrocery,hereisarecipeofTapenadefroma famousbookofprovenalcuisine.

LanguagesacceptedbyTapenadeparsers,SyntaxErrors:
Aftercallingfordifferentiationofmyfiles,Igetthe "S m t i g w n r n u i g d f e e t a i n o e h n e t w o g d r n i f r n i t o "webpage,whichtellsmethatthereisasyntaxerrorinmyfiles. WhatcanIdo? Yes,thishappensinmanycases.Firstnoticethattherearemanysortsofsocalled"syntaxerrors",whichresultinslightlydifferent errormessages.Youmaygetoneormanyofthefollowing: Ss:fNuhflrdrcoy ytf(oscieoietr) S s : F r r n P r e : f o s c i e o i e t r : I t r u t d a a y i : ytotaasrfNuhflrdrcoynerpenlss Somerequiredfilewasnotfound.Typicallyani c u e n l d filewasnotgiven. To:Ukonsfiief olnnwufxflf T o : N a s r f r f . F l k p e : olopreofiesipd TAPENADEtriestoguessthelanguageofthefilefromitssuffix.Thenitlooksforaparserforthislanguage.Eitherstepmay fail. F l : F r r n P r e : L x c l e r r a i e n n f l f . :Thereissomecharacterorwordat ieotaasreiarotlnniief,.. thislinennwhichisnotunderstoodinthecurrentfile'sprogramminglanguage(e.g.F r r n o t a ). F l : F r r n P r e : S n a r o t l n n i i e f : ieotaasrytxerraiennflf Atthislineissomesyntaxconstructionwhichisnotpermittedbythecurrentfile'sprogramminglanguage(e.g.F r r n o t a ). Ss:Uepceoreoeao:ox)nerpenlss ytnxetdsucprtrp(x:Itrutdaayi S s : U e p c e o r e l s : o s ( x : I t r u t d a a y i : ytnxetdsucitpx)nerpenlss Thisinternalerrorsaysthatoneoperatorofthecurrentfile'sprogramminglanguagewasforgotteninapreliminarystageof thetool.Pleasesendusthiserrormessagesothatwecanfixitquickly. Ss:Ntateprtret y t o r e o e a o : t x : Theparserhasfailed,andsentanincorrecttreetothedifferentiator.Thisisprobablyaninternalerror,becausethenormal messageshouldbethewellknown"S n a r o "above. ytxerr Fl:Teflosntcnanatppoeue i e h i e d e o o t i o r c d r :strangelyenough,thismessagemayappearasaresult fromasyntaxerror.Syntacticallycorrectfilesmustdeclaresubroutinesorfunctions.If,foranyreason,noroutinecanbe found,differentiationcannotstart. Fl:FrrnPre:Lxclerraie1irga.,Ukoncaatr''i ieotaasreiarotlnnpormfnnwhrcemn clmo5 o u n 1 t :Ifyourprogramusesthefreeformatstyleandthefile'ssuffixis. or. ,youmustselectfortran95as f F inputlanguagewiththe n u l n u g o t a 9 optioninordertoparseitwithoutsyntaxerror. iptagaefrrn5 Firstofall,makesurethattherearenomissingoremptyfiles.Thencheckthateachfile'ssuffixcorrespondstoitsactuallanguage: . or. forF r r n f F ota, . 9 ,. 9 ,. 9 ,or. 9 forF r r n 0 f0 F0 f5 F5 o t a 9 orF r r n 5 ota9, . or. forC c C Otherwisethisisa"real"syntaxerror. OurF r r n o t a parseracceptsalloftheF r r n 7 o t a 7 standard,plusmostoftheusualconstructor'sextensions,presentorpast(vax, sun,gould,connexionmachine,cray).Bewareofpossibleconflictsbetweenthestandardandtheextensions!Forexampleinthe standard,whitespacesarenotsignificant:ifthetextf n t o a isfoundinthecode,itisseenasthekeywordf n t o ucinl ucin followedbytheidentifiera ,probablyleadingtoasyntaxerror!Thereforeyoumustn'tcalloneofyourcodeidentifiers l f n t o a ors b o t n s ucinl u r u i e .Wecouldn'tchangethisrulebecausesomeoldprogramsdidstripawaywhitespaces!Check thatyourfilereallycomplieswiththestandardortheconstructor'sextension.Ifthisisthecase,Tapenadeshouldparseitcorrectly. Ifitdoesnot,thereisprobablyabuginourparser.Pleasesendusamessagesothatwecanfixitquickly. OurF r r n 5 o t a 9 parseracceptsofcourseF r r n 0 o t a 9 ,andalsotheH F P extensions.Becarefulwithcommentsdelimitorinthe freeformatstyle:commentsmuststartwitha"! ".OurC parseracceptsANSIC plusafewcommonextensions,BUTitdoesn't acceptC + + input.

Ifyoureallythinkyourprogramcomplieswiththesestandards,andourparserrejectsit,pleasesendusamessagesothatwecan fixourparser.Noticefurthermorethatourparserdoesnotsupportthec p p directives.Forinstance,wecannotparsefiles containing# e i e d f n or# f e directives.Pleaserunyourfilesthroughc p idf p beforesendingthemtoTapenade. Iftheerroryougetisaninternalerror,pleasesendusamessagesothatwecanfixit!

Syntaxofdirectivesthatmodifydifferentiation($AD...):
IgetthefollowingerrorwhenTapenadeparsesmysource: "F l : F r r n P r e : L x c l e r r a i e n n f o f 5 n n w e w r $ d ieotaasreiarotlnnio.9,Ukonkyod:ca ncekon" ohcpit Moregenerally,whatistheexactsyntaxofthedirectivesthatIcanuseinmysourcetoalteritsdifferentiation? Tapenadeunderstandsasmallnumberofdirectivesthatcanbeusedtoalterthedifferentiatedresult.Thesedirectivesmustbe providedinthesourcecodeassinglelinecommentsthatbeginwith$ D A .Sincethesyntaxofcommentschangeswiththe language,thesyntaxoftapenadedirectiveschangesaccordingly,e.g.fordirective"n c e k o n ": ohcpit InFortran77: C$DNCEKON AOHCPIT TheCmustbeincolumnzero.Therecanbeanynumberofwhitespace(ornone)beforethe$.Caseisinsignificant. InFortran9095: !$DNCEKON AOHCPIT The!canbeinanycolumn.Therecanbeanynumberofwhitespace(ornone)beforethe$.Caseisinsignificant. In"notreallyANSI"C: /AOHCPIT /$DNCEKON Thefirst/canbeinanycolumn.Therecanbeanynumberofwhitespace(ornone)beforethe$.Caseisinsignificantbutwe suggestyouuseuppercaseforabettervisibility. InC: /AOHCPIT* *$DNCEKON/ Theinitial/canbeinanycolumn.Therecanbeanynumberofwhitespace(ornone)beforethe$.Caseisinsignificantbut wesuggestyouuseuppercaseforabettervisibility. Thecorrectlocationofadirectiveinthesourcedependsontheactualdirective.Atthemoment,theavailableTapenadedirectives are: $ D N C E K O N ,describedhere AOHCPIT $DIOP A I L O ,describedhere $ D B N M A K ,describedhere AIOILCP $ D C E K O N T R ,describedhere AHCPITSAT $ D C E K O N N ,describedhere AHCPITED

LanguageacceptedbytheA.D.engineLimitations:
IcanseethatTapenadedifferentiatesFortran77programs.WhataboutFortran95orC?Whatabouttheirvariousconstructors' dialects?Doyouhaveplansforobjectorientedlanguages?Doyoudifferentiateprogramswithpointers,withrecords,with modules? Wearetalkinghereofactualdifferentiation,notparsing.Aboutparsing,seethe"Syntax"question.Wetriedtodifferentiatecorrectly thestandardfeaturesofFortran(77and95)andofC,plusmostoftheusualconstructor'sextensions,presentorpast(vax,sun, gould,connexionmachine,cray).Asfarasdifferentiationisconcerned,mostoftheseextensionshavelittleeffectanyway.Of coursetherearepossiblebugsandafewrestrictionsthatwillbeliftedprogressively.WedonotdifferentiatepureC++constructs (Tapenadedoesn'tevenparsethem!).Inthefollowinglist,partsinblackarewhatTapenadeshoulddifferentiatecorrectly,partsin redarewhatisnotdifferentiated: Fortran95freesourceform:ourF r r n 5 o t a 9 handlesit.Becarefulwithcommentsdelimitorinthefreeformatstyle: commentsmuststartwitha"! " modules,B O K A A L C D T :acceptedanddifferentiated. overloading:acceptedanddifferentiated. keywordargumentsandoptionalarguments:acceptedanddifferentiated. derivedtypes,structuredtypes:acceptedanddifferentiated SWITCH/CASEstatements:acceptedanddifferentiated pointers:accepted,analyzedanddifferentiatedexceptinsomecasesinreversemode dynamicallocation,malloc's,free's:acceptedanddifferentiatedintangentmode.Inreversemode,therearesome limitations(thatwemustdocumentsomeday)e.g.correspondingallocationsandfreesmustliveinthesameprocedure.In anycase,webelievetheseconstructsshouldpreferablybekeptoutofthecomputationalkernel,whichshouldcontainthe fragmenttodifferentiate.

namelist:acceptedanddifferentiated. vectornotation,W E E H R andF R L constructs,mask's,S M OAL U 'sandF R L 's:acceptedanddifferentiated. OAL argumentpassingcallbyvalue,callbyvalueresult,callbyreference:acceptedanddifferentiated. interleavingdeclarationsandinstructions,declarationswithinitializations:acceptedanddifferentiated.Theattached commentsareapproximativelyreplacedattheircorrectpositions. Cifexpressions,e.g.x = 1 + ( < ? y y y 0 : ) :parsedbutincorrectlydifferentiated,especiallyinreversemode. arrayconstructorse.g.A : / ( ) X 1 , ( ) Y 1 / :arrayConstructorareparsedbutincorrectlydifferentiated ()=(X2()Y2()) inreversemode.

Specificallyforthereversemode:aprogrampiecethatwillbecheckpointed(e.g.aprocedurecall)mustbereentrant.Thismeans thattheremustbeawaytocallittwice,atthecostofsomemanipulationstorestoretheinitialstate.Forinstanceaprocedurethat modifiesaninternalhiddenvariableorstate(thinkofIOpointersinfiles,orprivatecountervariables...)isnotreentrant.Inthiscase, youmusteithertransformtheprogrampiecetomakeitreentrant,ortellTapenadenottocheckpointit. Differentiationofobjectorientedlanguagesisstillnotinourimmediategoals,andneitherarethetemplatesofC++.

Blackboxprocedures,unknownexternalprocedures:
Tapenadecomplainsaboutunknownexternalorintrinsicprocedures,howcanIteachitaboutthem? Ineedtodefineanewsetoflibraryfilesormodifytheexisting*Libfiles. Whatisthesyntaxinthereandwhatdoesitmean? IftheprogramyouwanttodifferentiatecontainscallstoEXTERNALs,INTRINSICs,library,orother"BlackBox"routines,thenyoumay needtoprovideTAPENADEwithsomeinformationabouttheseBlackBoxroutines.Youmayalsohavetodothisifweforgottoaddinto thestandardlibraryfilestheinformationonsomeintrinsicsubroutinesthatyouactuallyuse. Itcanalsohappenthatyouhavethesourceofaprocedure,butyoudon'twanttogiveittoTAPENADE,becauseTAPENADEwouldthen differentiateitinitsownstandardmanner,andyouknowhowtodifferentiateitbetterbyhand!SomakeitaBlackBoxbyjustforgettingto provideitssource. Someintrinsicfunctions,suchasM X A ,arenotwelltreatedwhentheyhaveanunusualnumberofargument.Forinstance,M X x + , A(,yz 0 0 willnotbeanalyzed,nordifferentiatedcorectly.TheworkaroudiseithertoreplacethisbytwonestedcallstoM X .) A ,ortotreatthe3 argumentsM X A asanotherBlackBoxfunction. So,forwhateverreason,TAPENADEhasnosourceforasubroutineorfunction(supposeitiscalledB O ).Thishassomeunfortunate BX consequences.First,duringtypechecking,TAPENADEcannotcomparethenumberandtypesoftheactualargumentstothenumber andtypesoftheformalargumentsofB O .Furthermore,itdoesn'tknowwhichargumentswillbereadand/oroverwritten.More BX specificallyforAD,TAPENADEdoesn'tknowhowtopropagateactivitythroughB O ,i.e.whichoutputsofB O dependina BX BX differentiablemanneronwhichinputsofB O .Lastbutnotleast,TAPENADEdoesn'tknowthenameofthedifferentiatedversionof BX B O ,inthecasewhereyouhaveprogrammeditalready. BX Noticethat,althoughallthisinformationismissing,TAPENADEwilloftengenerategoodenoughcode,thankstoanumberof conservativeassumptions: Thetypesandthenumberofargumentsarededucedfromtheactualusages.Iftheseusagesdon'tmatch,therewillbeanerror message. Bydefault,ablackboxroutineissupposedtocommunicatewiththecallingcodeonlythroughthecallingarguments:TAPENADE supposesthattheblackboxroutinedoesn'tuseargumentsthroughCOMMON'sandotherglobals.Warning:thisassumptionis notconservative:iftheblackboxroutineactuallyreferstoaglobalandnothingisdonetotellTAPENADEaboutthat, TAPENADEmayproducewrongcode! Allactualargumentsaresupposedtobepossiblyreadinsidetheblackboxroutine,exceptofcoursethereturnedresultofa function. Allactualvariablereferenceargumentsaresupposedtobepossiblyoverwritteninsidetheblackboxroutineifitisanexternaloran intrinsicprocedure.Ifitisanintrinsicfunction,wesupposenoargumentisoverwritten. Anyvariableargumentofadifferentiabletypeisassumedtoreturnanoutputvaluethatdepends(inadifferentiableway)onthe inputvalueofeveryargumentofadifferentiabletype. ThedifferentiatedroutineissupposedtohavethesamenameandshapeaswhatwouldhavebeencreatedbyTAPENADE,i.e. B O _ orB O _ . BXD BXB Ifyouwanttooverridetheassumptionsabove,allowingTapenadetoproducebettercode,youmayprovidesomeknowledgeofthe BlackBoxroutinesthatareusedintheprogram.The*Lib(ortheobsolescent*ADLibfiles)areonewaytogiveitthisknowledge.Another wayistowritedummyroutines.

SPECIFYINGBLACKBOXROUTINESIN*LibFILES:
The"*L b i "filescontaininformationaboutlibrary,intrinsic,externalroutinesoranyotherBlackBoxroutine.ThiswillhelpTapenade producemoreefficientcode.Forinstance,thereisonesuchfile"F 7 e e a L b 7 G n r l i "alreadyinyouinstallation"l b "directory.Youmay i/ modifyit.Youmayalsocreateyourownfilessuchas"M A p i e e a L b y p l G n r l i ",inany"M D r c o y y i e t r ",andthenyouwillpassthesefile namestoyourt p n d commandusingthe x option. aeae et tpndetMDrcoyMApieeaLb.. aeaexyietr/yplGnrli. Eachentryisaboutonesubroutineorfunction,andgivesinfoaboutnumber,type,intentofarguments,andpossiblyinforelatedtoAD. Forexampleyoumighttype,forasubroutinenamedB O andforafunctioncalledF B X BX BO: sbotnbx uruiebo:

etra: xenl sae(aa, hp:prm1 aa, prm2 omn/1[,[ cmoc/0*, omn/2[,6, cmoc/01[ omn/2[616, cmoc/1,7[ aa, prm3 aa, prm4 aa) prm5 tp(dnel ye:ietra, dnel ietra, raTp(dneldmoosdmoo(oe)nn()) aryyeietra,iCln(iClnnn(,oe)), oiidyeaemdfesietdul)dnel, mdfeTpNm(oiir(dnobe,ietra) raTp(oiidyeaemdfesietdul)dnel, aryyemdfeTpNm(oiir(dnobe,ietra) iCln(iClnnn(,oe)), dmoosdmoo(oe)nn()) dnola, ietboen raTp(dnhrce,iCln(iClnnn(,oe)), aryyeietcaatrdmoosdmoo(oe)nn()) dnhrce) ietcaatr RaNtrte:(,0,0,0,1 edoWitn1,0,0,1) Ntedhnrte:(,0,0,0,0 oRaTeWitn0,1,0,0) RaTeWitn:(,1,1,1,0 edhnrte0,0,1,0) dp:(d esi, ,1,0,0,0 1,0,0,0, ,0,1,0,0 1,0,0,0, ,0,0,0,0 0,0,0,0, ,1,1,0,0 0,0,1,0, ,0,0,0,0 0,0,0,0, d i, d i) fntobo: ucinfbx itisc nrni: saeprm1aa,rsl) hp:(aa,prm2eut tp:(dneldneloiidyeaemdfesietdul)dnel) yeietra,ietra,mdfeTpNm(oiir(dnobe,ietra) RaNtrte:110 edoWitn(,,) Ntedhnrte:001 oRaTeWitn(,,) NtedoWitn010 oRaNtrte:(,,) drvtv:bnr(alietfbxxrsin(eaa,mtvrY) eiaieiaycl(dnbo,epesosmtvrXeaa), u(els.,mtvrX, mlraCt20eaa) xrsin(iaymtvrXoe)d(nCt5eaa), epesosbnr(eaa,nn(,adits,mtvrY) iaymtvrYoe)i(els.,mtvrY)) bnr(eaa,nn(,dvraCt10eaa)) The"e t r a : x e n l "linestatesthattheprocedureisexternal.Foranintrinsic,justreplacethislineby"i t i s c ". nrni: The"s a e "partisnecessary,becauseitdefinesthe(arbitrary)orderinwhichargumentswillbereferredtointhesequel.Itssyntaxis hp: eitherp r m N a a thatdesignatestheNthformalparameter,orr s l forthereturnedvalueifB O isafunction,orc m o eut BX omn /x[,[ x / 0 8 foranargumentwhichisfoundfromoffset0tooffset8incommon/ x . x/ Allnextlinesareoptional.Iftheyaremissing,thecorrespondingconservativeassumptionsaremade. The"t p : y e "partgivesthetypesofeachargument,inthesameorder.Thesyntaxofatypeisratherheavy:wedidn'tfindthetimeto makeitcleaneryet,sorry!Actuallyit'stheinternaltreerepresentationofthetypeinsideTapenadememory.InBNFlike,aTYPEcanbe: TP:dnAE YE:=ietNM (AEblnigtra,cmlxnee,caatrola} NMeogno{elope,itgrhrce,boen, ormtvye) frpiiietps TP:oiidyeaemdfesMDFES,TP) YE:=mdfeTpNm(oiir(OIIR)YE (OIIRenitootyietaditsuhaobeo6 MDFESbigalsfmsldnnnCtscsdulr1 ormtvyeszoiir) frpiiietpiemdfes TP:raTp(YEiCln(IS) YE:=aryyeTP,dmoosDM) (oraswtieubroiesos fraryihafxdnmefdmnin) DM:I IS:=DM DM:IIS IS:=DM,DM DM:=dmoo(ON,BUD I:iClnBUDON) BUD:=nn( ON:oe) (onuseiiddmninlwropeon) franpcfeiesooeruprbud BUD:=its ON:nCtN TP:raTp(YEiCln() YE:=aryyeTP,dmoos) (oraswtnubroiesos fraryihaynmefdmnin) TP:oneTp(YE YE:=pitryeTP) (ooneoaohrtpaalbeierftr) frapitrtnteye(vialnnauue) TP:od) YE:=vi( (odtp,mslenoo'attaeaoti) viyeotymasyudntwnocrbut Forexample,r a * iswritten el8 mdfeTpNm(oiir(nCt8,ietra) oiidyeaemdfesits)dnel Arraysarebuiltwiththecombinator"a r y y e r a T p ",thatbindsthearrayelements'typewithalistofdimensions("d m o o ")whichare iCln pairsofthelowerandupperbound.Sofar,theseboundsarenotveryuseful,soyoumightjustaswellleavethemas"n n ( "oreven oe)

usethearrayType(TYPE,dimColons())form.TakealookatF77GeneralLibtofindmoreexamples. Thismechanismisfarfrombeingcomplete.Forexample,thereisstillnosyntaxforstructuredargumenttypes("derivedtypes"),although theyarehandledwellinsideTapenade.Nothingexistsfornamedarguments,norforavariablenumberofarguments.Foravariable numberofargument,youmayhavetocreatea*Libentryforeachnumberofargumentsthatactuallyoccursinyourcode. Inadditiontothis,thereisour"metavar"type: TP:eaaAE YE:=mtvrNM Thismetavartyperepresentsanytype.IfitappearswiththesameNAMEseveraltimesinthelistoftypesofthepresentprocedure,each occurrencemustmatchthesametype.ThevalueofNAMEjustindicateswhich"sort"oftypeshouldbethere,i.e.ifactualpattern matchingfailstofindasolutiontypeforthemetavar,thenTAPENADEwillchooseidentNAME.Otherwiseyoumaynameyourmetavar "foo"insteadof"real",anditshouldworkjustthesame.Thismechanismisbynomeanscomplete,butithelps. Lookatfunction"SUM"inF77GeneralLibforalesstrivialuseofmetavars:inSUM,themetavarletsyouspecifythatittakesanarrayof REALandreturnsaREAL,ortakesanarrayofREAL*8andreturnsaREAL*8,orevenanarrayofINTEGERandreturnsanINTEGER. The"R a N t r t e , N t e d h n r t e , R a T e W i t n o R a N t r t e "infogivesthereadwrite edoWitnoRaTeWitnedhnrte,NtedoWitn signatureofeachargument.Thereis1foroneargumentwhenthereexistsatleastoneexecutioncaseoftheprocedureforwhichatleast apartofthisargumentis"affectedInThisWay",where"affectedInThisWay"isthenameoftheinfo(i.e.oneofR a N t r t e , edoWitn N t e d h n r t e , R a T e W i t n o R a N t r t e ).Thisisanewnotation,validfromversion3.6on.Thisnew oRaTeWitnedhnrte,NtedoWitn notationreplacesthepreviousnotation(R R , N )whichhadadifferentmeaning,lessaccurate.Althoughthepreviousnotationis ,N,WW stillaccepted,weadviseyoutoswitchrapidlytothenewnotationinyourhomemadelibraries. The"d p : e s "entryisrelatedtoAD.Itisthedifferentiabledependencymatrix,ofeachoutputvaluewithrespecttoinputvalues,foreach argument.Thematrixissquare(i elementscountforonerow),andtheorderoftherowsistheorderoftheparametersaslistedinthe d "s a e "entryforthisprocedure.Thesameholdsfortheorderofthecolumns.Forexample,theelementinrow2,column5specifies hp: whethertheoutputvalueofp r m 2 a a dependson(i.e."hasanontrivialzeroderivativewithrespectto")theinputvalueofthevariable fromoffset16to176inc m o c / o m n / 2 .Herethereisazero,meaningatrivialzeroderivative,i.e.nodependency.Forexamplehere,the resultinparameter2dependsontheinputparameters1and2,theresultinthefirstvariableinc m o c / o m n / 2 doesn'tdependonany input,andtheresultinthetailarrayof/ 2 dependsontheinputsinparameter2andinthewholeof/ 2 .An"i "entryinsteadofa c/ c/ d completelinestatesthattheargumentisleftunmodified,whichisastrongerinformationthanalinewithjustone"1",becauseitimplies thatthepartialderivativeiscertainly"1.0".Thisisthecasehereforparameters1,4,and5.Ontheotherhand,parameter3isoverwritten, butofcoursedependsonnobody,sinceithasanondifferentiabletype.Forthesamereason,nobodydependsonparameters3,4,and5. Noticealsothatnobodydependsontheinputin/ 1 :itisanoutputonly. c/ Theotherthingyoumaywanttospecifyistheactualderivativeoftheblackboxprocedure.Threemainpossibilities: Itmayhappenthatafteractivityanalysis,TAPENADEfindsoutthatthedifferentiatedprogramdoesnotactuallyneedtodifferentiate theblackbox.Inotherwords,noargumentoftheblackboxisactive.Thenyou'reverylucky,thereisnothingyouneedtodo! Iftheroutinereallyhasactivearguments,thenitmayhappenthatitisactuallyafunction,andasimpleonesothatyoucandefine thepartialderivativesofitsoutputwithrespecttoeachinput,asasimplecheapexpression.Thiscanincludecallstoanother intrinsic.TAPENADEcanusethosepartialderivativesbothintangentandreversemode.ThinkforexampleofC S X .There's O() onlyonepartialderivativeyouneedtospecifywhichisthepartialderivativeoftheresultwrtthefirst(andonly)argument,andthis derivativeis I ( ) S N X .Tospecifythis,the" e i a i e " d r v t v : entryinthe"*G n r l i "filegivesapatternthatmustmatchthe eeaLb originalcall,andpatternsthatdefineseachpartialderivative.Hereagain,thesyntaxisuglybecausewedidn'tfindthetimetoclean ityet.Sorryagain!Let'srecallwhatwewroteforthefunctionf b x b o above: fntobo ucinfbx drvtv:bnr(alietfbxxrsin(eaa,mtvrY) eiaieiaycl(dnbo,epesosmtvrXeaa), u(els.,mtvrX, mlraCt20eaa) xrsin(iaymtvrXoe)d(nCt5eaa), epesosbnr(eaa,nn(,adits,mtvrY) iaymtvrYoe)i(els.,mtvrY)) bnr(eaa,nn(,dvraCt10eaa)) Thepartsingreenarecompletelymeaningless,butmustberespectedbecauseTAPENADEexpectstofindthemthere. Inthisexample,onecanfind,inorder,thepatternthatmustmatchtheactualcall,thereforeinstantiatingthemetavariablesXandY, thenanexpressionwhichisacommonfactortoallpartialderivatives(whichcanbe"n n ( "ifthereisnocommonfactor),then oe) thelistofeachpartialderivativeofthefunction'sresultwithrespecttoeachmetavariable:foreachmetavariablekey,thereisan expressionthatdefinesthecorrespondingpartialderivativeexpression,whichhastobemultipliedbythecommonfactorifpresent. Forinstanceintheaboveexample,wehaveexpressedthatthepartialderivativeofF B X X Y withrespecttoX BO(,) isthe expression2 0 X Y 5 anditspartialderivativewithrespecttoY .*(+) istheexpression2 0 X Y . * / .Noticethatwedidn'tgivea "d p : e s "entry,becausethedefaultforanintrinsicfunctionisjustwhatwewant!Moreexamplescanbefoundinthe F7eeaLb 7 G n r l i file,forinstancethespecificationofthepartialderivativesofC S O mentionedabove. Ifthereisnowaytodefinethepartialderivativesexplicitly,thenyoumustwriteandprovidethederivativecodeoftheblackbox routineforthecurrentdifferentiationmode(e.g.tangent,adjoint,...).FollowthemessageissuedbyTAPENADEtoknowwhich inputs(resp.outputs)oftheblackboxmustbeconsideredactive.FollowthesyntaxthatTAPENADEhaschosenwhengenerating thecallstothisdifferentiatedblackboxinthedifferentiatedcode.Thensimplywritethesedifferentiatedexternalroutinesbyhandin aseparatefile,usingyourpreferredmethod/algorithm.Wheneverythingelsefails,youmayevenusedivideddifferences,alwaysat thecostofsomeaccuracy.Finallylinktheresultingcodewiththerestattheendofcompilationtime.

Pleasenoticethatthese*Libfilesarenotabletorepresentallyouwouldliketo.Forexample,thereisnosupportforprocedureswith variablenumberofargumentsornamedarguments.Wehopeitcanbeusefulinmostsituations,though,untilwedeviseabetterandmore completesyntaxforthese*Libfiles.

SPECIFYINGBLACKBOXROUTINESWITHDUMMYROUTINES:
Sometimesthe*Libfilescannotexpresssomebehavioroftheblackboxprocedure,Tapenadedoesnotunderstandthisinformationwell

anddoesn'tdifferentiatetheblackboxprocedurecorrectly.Sometimesalso,thesyntaxofthe*Libfilesisreallypainfulandonelooksfora simplermethod.Hereisapossiblesimplermethodthatyoucanuse.Writeadummydefinitionoftheproblematicblackboxprocedure, justlikeastandardprocedure,withstatementsthatjust"represent"thedatadependencesoftheactualprocedure.Wemeanthatthe computationsinsidethedummyproceduremaybetotallywrongandmeaningless,providedtheyimplementthesamedifferentiabledata dependenciesbetweentheinputsandtheoutputs.Inotherwordsagain,foreachoutputz thatdependsinadifferentiablewayoninputs ,say,x andy ,justwriteastatementlike z=xy * inthedummyprocedure.Givethisdummydefinitiononlytothedifferentiationprocess,andremovethedifferentiateddummyprocedure fromthedifferentiatedcodereturnedbyTapenade.Finally,writebyhandthecorrectdifferentiatedblackboxprocedureandlinkitwiththe differentiatedcodereturnedbyTapenade.

OtherDifferentiationModes:
Ineedhigherorderderivatives,Hessians,Taylorseries. IneedthefullJacobianMatrix. Initspresentform,Tapenadebuildsprogramsthatcomputedirectionalderivativesandgradients,i.e.inourjargontheforwardand thereversemodeofA.D.TapenadedoesnotprovideJacobians,norhigherorderderivatives,norHessians,norTaylorseries. However,therearewaystocopewiththat.Forexample,onecanusetheforwardmodetwice,togetdirectionalsecondderivatives. WeknowofsomepeoplewhohavetriedthatwithTapenade,andapparentlyitworked.Noticehoweverthatthisiscertainlyless efficientthanaspecificmodeforsecondderivatives.Onereasonforthatisthatifyouusetheforwardmodetwice,yougettwo redundantcopiesofeachcomputationofthefirstderivatives! Then,fromthehigherorderdirectionalderivatives,youcanderivethehigherorderHessiantensors.Thisissometimesmoreclever thancomputingthesetensorsfromscratch,becauseonemaytakeadvantageofthetensor'ssymmetries.Thisquestionis discussedforexampleinGriewankUtkeWalther97 TapenadedoesnothingaboutTaylorseriesnorintervals.ThesetwocouldbeinterestingnewmodestoincludeintoTapenade,but thisisnotinourcurrentplans. ManyapplicationsneedtheJacobianmatrix.Herearesomewaystogetit: Therecouldbeaspecialpurposedifferentiationmode,propagatingapartialJacobianthroughouttheexecutionoftheoriginal program.ThiswasimplementedonceintoOdyssee,andtheresultswereinteresting,thoughslightlydisappointing.This modewasnotmaintainedinthenextversionofOdyssee(1.7).Howeverthereisapotentialsourceofoptimizationcoming fromarrayanalysisinregularloops,asexploredinTadjoudineEyssetteFaure98 Onecancomputeitcolumnbycolumn,byrepeatedexecutionsoftheforwardmode,oneexecutionforeachdirectioninthe originspace.i.e.foreachelementoftheCartesianbasisoftheinputspace.Actuallyfewerdirectionsarerequiredwhenthe Jacobianmatrixcanbeshowntobesparseenough(seethelitterature,e.g.theGriewankWalther2008book[GrieWalt08]). Thiscanbeimprovedbycomputingthederivativesformanydirectionsinthesamerun.Thisisthesocalledvectormodeor multidirectionalmode,firstfoundinADIFORandavailableinTapenade.Multidirectionalmodecanexhibitsome parallelism,asshowninBuckerLangMeyBischof01.AscomparedtoADIFOR,Tapenadebringsthenoveltythatdifferentiated instructions,whicharenowloops,aregatheredusingaspecialdatadependencyanalysis.Thisreducestheoverheaddueto thenumerousoneinstructionloops. Onecancomputeitlinebyline,byrepeatedexecutionsofthereversemode.Duetotheintrinsicdifficultiesofthereverse mode,thisisrecommendedonlywhenthenumberofinputsislargerthanthenumberofoutputs,butontheotherhandthis canbenefitfromspecificimprovementsofthereversemodesuchasabettterdetectionofcommonsub=expressionsin derivatives.Atthepresenttime,Tapenadedoesn'tprovideamultidirectionalmodefortheadjointmode,butthiscould changeifmoreapplicationswoulduseit.

Typemismatches:
TapenadeTypeCheckersendsmealotofmessagesoftheform: (C6yemsacnasgmn:RA*eevsDULRCSO T1)TpimthisinetEL8rcieOBEPEIIN Whatdotheymean?Aretheyimportant?WhatmustIdo? Thesemessagesmaybeimportant,butinmanycasestheyareonly"paranoiac"warnings.Yourcompilermayperfectlywellaccept thatanargumentofkindD U L R C S O ispassedtoaprocedurethatexpectsaR A * orviceversa.Howeverthisis OBEPEIIN EL8 againsttheFortranstandard,anditmaybethecauseofportabilityproblemsinthefuture.ThisiswhyTapenadecomplainswhile yourcompilerdoesn't. Forinstanceg77doesnotcomplainon: ra*a el8vr vr=100 a.d butTapenadedoesbecausethestandardsaysthat1 0 0 O B E P E I I N . d isD U L R C S O andnotR A * . EL8 MoreimportantlyconcerningADinthereversemode,theP S andP P UH O proceduresthatareusedintensivelyrelyonanexact knowledgeofthesizeinbytesoftheirargument.Soattheminimumyoushouldcheckthatthesizesofthesetypesmatchonthe machine/compileryouwilluse,andthatthesearethesizesthatTapenadewilluse.Tocheckthesizesonyourmachine/compiler, refertothebeginningoftheFAQsectiononvalidationofthedifferentiatedprograms .TomodifythesizesusedbyTapenadeto adaptthemtotheresultsoftheabovecheck,usethecommandlineoptions 8 r etc. Anyway,alltheseprecisionproblemsareanigntmare.WeagreethatmostofthemessagesthatTapenadesendsaboutthisarea

nuisance.Mostprobablyyoucanneglectthem.Maybesomedayweshallofferanoptiontohidethemaway. Neverthelessitstilloccurssometimes,especiallywithFortran95,thatthesemessagesindicatebugsinTapenade.Forinstance Tapenademayhavelostadimensionofanarray.Inthiscase,itisunlikelythatdifferentiationwillwork.Inthissituation,whenyou seethatyourprogramisobviouslycorrectandTapenadecomplains,pleasesendusamessagesothatwecanimprove Tapenade.

Aliasingproblems:
IgetthefollowingmessagefromTapenade: (F2oetalaigialnucinFn,btenagmnsagmns D0)Ptnilaisnncligfntouceweruetruet Whatdoesitmeanandisitimportant? ThismeansthatTapenadedetectedamemoryoverlapbetweenargumentsofranksn andn ofacalltoasubroutineorfunction. 1 2 Whatisaliasing?Supposeasubroutinehasmanyarguments(i.e.formalparametersorcommonsorglobals...)Ateachcallsite, eachoftheseformalparametersisgivenan"actualparameter",i.e.areferencetoanactualvariable,scalarorarray,oran expression,oraconstant...Aliasinghappenswhenthesameactualparameterisgiventotwodifferentformalparameters.andone ofthesetwoformalparametersisoverwrittenbythesubroutine.Moregenerally,ithappenswhentwoactualparametersoverlapin memory,andatleastonemaybeoverwritten.PleasenoticethatsuchaliasingresultsincodethatdoesnotconformtotheFortran standard.e.g.seeMetcalfReid96section5.7.2page91.Thisisanimportantwarning,becauseTapenade,likemoststaticanalysis tools,analyseseachsubroutineinitsownlocalcontext,i.e.assumesthateachformalparameterisadifferentmemorylocation. Andthisassumptionleadstoageneratedprogram(differentiated)thatwillfailifthereisaliasing.Forexample: sbotn(elaelb uruieFra,ra){ a=3b * } wouldbedifferentiatedinthereversemodeas(grossly): sbotn_(elaela,ra,rab uruieFBra,rabelbelb){ bb+3a b=b*b a. b=00 } andthisworksfineaslongastheactuala andb aredifferentvariables.Butifa b x doesx = 3 x = = ,F * ,andwhatyouwantforF B _ isx * b b = 3 x .Ifyoucheck,thisnotwhatyougetbycallingtheaboveF B _ .Similarproblemsariseinmostprogram transformations,suchasparallelizationorpartialevaluation,orothers... Noticehoweverthataliasing,heredetectedbyastaticanalysis,isbyessenceundecidable.ThereforeTapenademaydetect aliasingwhenthereisnone!Also,itisfrequentthat,althoughthereisaliasing,thedifferentiatedprogramwillrunfine.Itisthe responsibilityoftheenduser,whoknowstheprogram,tocheckthathisaliasingisharmless.Usually,whenaliasingisreallya problem,onecaneasilyfixthatbyintroducingtemporaryvariables,sothatmemorylocationsoftheparametersdonotoverlap.

MissingdifferentiatedroutinesforintrinsicorBlackBoxroutines:
Ican'tcompilemycodebecauseitmissesthedifferentiatedroutinesforsomeintrinsics,externals,orBlackBoxroutines. IgetacompilermessagesayingitneedstolinkwithD M L _ CPXB HowdoIdefinetheadjointofanintrinsicroutine? Thisisperfectlynatural. InyouroriginalcodewereBlackBoxroutines,suchaD M L ,i.e.eitherINTRINSICsorEXTERNALsorotherroutinesforwhich CPX youdidn'tprovidethesource.Asaconsequence,Tapenadehasplacedinthedifferentiatedcodeacalltothedifferentiatedroutine D M L _ intangentmodeorD M L _ inadjointmode,andsentyouamessagerequestingyoutodefinethesenewroutines. CPXD CPXB Nowisthetimetodoit. TaketheexampleofD M L with2arguments: CPX cCPXr,i c=DML(ri) ThiscallbuildsaD U L O P E numberwithrealpartr andimaginaryparti ,andputstheresultintovariablec . OBECMLX r i c ThevariablesonwhichD M L operatesarer ,i andc .Toconsideronlyrealnumbers,let'ssplitc intoitsrealand CPX r i c c imaginarypartsc .randc .i.Sincer andi maybeusedlater,weconsiderthemalsoas(unchanged)outputsofD M L . c c r i CPX Similarlyc canbeseenasan(unused)input.Allinall,D M L operatesfromR4toR4,withsomeconventionalorderinge.g.: c CPX ric.ci (ricrc.) WecanthuswritetheJacobianmatrixofD M L ,whichisinthiscasesimply: CPX 1000 () 0100 () 1000 () 0100 ()

astheoutputc .rreceivesexactyr ,theoutputc .ireceivesexactyi ,andneitherr nori aremodified. c r c i r i ThetangentdifferentiationofD M L justimplementsthematrixtimesvectorproductwiththisJacobian,i.e. CPX rd)()(r (r1000rd) id)()(i (i0100id) cdr)=()*(c. (c.1000cdr) cdi)()(c. (c.0100cdi) hencethedefinitionofD M L _ youmustprovide: CPXD fntoCPXDr,rdii,c) ucinDML_(rr,i,idc DULRCSOrrdi,i OBEPEIINr,r,iid DULOPEcCPXD OBECMLXc,DML_ DML_CPXrdid CPXD=DML(r,i) cCPXr,i c=DML(ri) ed n

TheadjointdifferentiationofD M L justimplementsthematrixtimesvectorproductwiththetransposedJacobian,i.e. CPX rb)()(r (r1010rb) ib)()(i (i0101ib) cbr)=()*(c. (c.0000cbr) cbi)()(c. (c.0000cbi) hencethedefinitionofD M L _ youmustprovide: CPXB sbotnCPXBr,rbii,cb uruieDML_(rr,i,ibc) IPIIOE MLCTNN DULRCSOrrbi,i OBEPEIINr,r,iib DULOPEc OBECMLXcb rb=rb+BEcb rrDL(c) ib=ib+DMGcb iiIA(c) cb=DML(.0 cCPX0d) ed n

Theaboveequationisa"vectorial"assignmentinR4,whichisdone"inparallel".Thereforewewerecarefulnottoreinitializec b c beforeitisusedtoupdater b r andi b i .Also,sincethisisadjointmode,itisnousetoreturnthevalueofc c

DivisionsbyZeroandNaN's:
ThedifferentiatedprogramreturnsNaN'sinthederivativevariables.Howevermyoriginalprogramiscleananddoesn'treturn NaN's! DifferentiationcanintroduceNaN'sintheresultingprogram,becauseprogramsaresometimesnondifferentiable.(Youmaytake alookattheFAQitemaboutthedomainofvalidityofderivatives)Ingeneral,ADisintroublewhenitcomestodifferentiatinga functionatplaceswhereitisnotdifferentiable. Inthissituation,wepreferthatthedifferentiatedprogramreturnNaN,whichatleastshowsthatsomethingwentwrong.Thisis probablybetterthansilentlyreturningawrongderivativethatmayleadtootherwrongresultslater.Theendusedcanthenanalyse theoriginalprogram,andmaybetransformittomakeitdifferentiable.

HowevertherearesituationswhereTapenadecandoaslightlybetterjob,automatically.Nondifferentiablilityoftenoccursfor S R ,L G Q T O ,andexponentiations.ForS R ,wecandosomethingalittlemorecleverthanreturningNaN,butonlyinoneparticular QT case: Supposeinstructionis a=SR() QTb thetangentderivativeinstructionis ad(.*QTb) d=b/20SR() thatreturnsNaNifb is0 0 . .Butifb isalso0 0 d . atthistime,thenwefeelitissafetoreturn a. d=00 Ontheotherhand,wethinkitisnotsafetotestonb ora being0 0 . orcloseto0 0 . .Thereasonisthatinthiscasethederivativeis reallyundefined,soit'suptotheusertodosomethingaboutit.Itwouldbedangeroustoreturna . automatically.Onlythe d=00 usercandoitbyhand,andcheckthatit"works"forthecurrentapplication.

Somethingsimilarisdoneforexponentiations.NothingisdoneforL G O .WeshouldprobablydothesameasforS R ! QT Asimilarreasoningappliestothereversemode:Tapenadegeneratesstatementsthatdetectspecialcasesandbuild0 0 . derivatives. HoweverthereisaweaknessinTapenadewhenitcomestoreturningNan's:unlikewhathappensinAdifor,thereisnowarning messageprintedatthemomentwhentheNanappears.Therefore,intheveryfrequentcasewherethecompiledcodedoesnot stopwhentheNanappears,computationgoesonasusualanditisverydifficulttofindtheplacewheretheproblemoriginated.We areawareofthisproblemandthisshouldbeimproved.

NaN'scanalsoappearforaslightlydifferentreason:forsomegiveninputs,somefunctionshaveanoutputwhichislarge,butnot yetinoverflow,whereasthederivativeisdefinitelyinoverflow.Thinkof1 x / ,whosederivativeis / * 2 1 x * .Againthisrequiresa specialtreatment.Onewaycouldbetoswitchtodoubleprecision.

WerecentlyranintoaveryannoyingNaNproblemrelatedtouninitializedvariables.Pleaseavoidusinguninitializedvariablesin yourcode!Tapenadetriestosendmessagesaboutuninitializedvariables,butthisisatypicalundecidablequestion,sowecannot findallofthem.Thefollowingsummarizestheproblemthatwefoundonaverylargecode... sbotnO(,) uruieTPxy ra, elxy raA() elBD2 clU(A,) alSBBDx y=BD1 A() ed n sbotnU(A,) uruieSBBDx ra,BD2 elxA() BD1* A()=xx BD2A()x A()=BD2* ed n YoucanseethatarrayB D A isnotinitialized.B D 2 mayverywellcontainsomethingbadlikeaNaN.Howeverthishasnovisible A() impactonthisoriginalcodebecausetheresultingx andy arenotpollutedbytheNaN.Stillthiscodeisgoingtosilentlycompute onNaN's,andyoudon'twantthis.Nowifwedifferentiatethiscodeinreversemode,thefollowingstatementappearsinS B B U_: xb+bd2*ab2 b=xa()bd() andiftheinitialB D 2 isadirtyNaN,thenthiscomputesNaNtimes0.0,whichonsomecompilersevaluatestoNaN,andthe A() resultingx getscorrupted! b

Domainofvalidityofderivatives:
WhatdoesADcomputewhenthefonctionisnondifferentiable? WhyistheTapenadederivativealways0.0formydichotomybasedresolutionprogram? TowhatextentcanItrustthederivativescomputedbyTapenade? Isthereadiscontinuityinmyfunctionclosetothecurrentinput? HowcanIevaluatethedifferentiabilitydomainaroundmycurrentinput? IsthereanysupportinTapenadefornondifferentiablefunctions? IftheTapenadedifferentiatedcodeisexecutedrightonanondifferentiableinput,itmayhappenthatthedifferentiatedprogram returnsNaN'sasderivatives.SeetheFAQitemaboutdivisionsbyZeroandNaN's. Also,althoughitishardtoadmit:),theremaybebugsinTapenade:(Therefore,youprobablyneedtogothroughthevalidation processtomakesurethatthederivativesarecorrect,comparedtoDividedDifferences.Althoughinaccurateandexpensive, DividedDifferencesarerobustandthereforereturngoodenoughderivativesforvalidation. However,itmayhappenthatthedifferentiatedprogramjustdoesnotnoticethatitison(orcloseto)adiscontinuity.Thisisnota bugofTapenadethiscomesfromthenatureofAD.Thinkofadichotomybasedprogramthatsolvesf x y = ,returningy (,)0 asa functionofx .Thederivativeofy withrespecttox isingeneralnontrivial.RobustDividedDifferenceswillactuallyreturn(an approximationof)thecorrectderivative.ButADwillreturn0 0 . ,andthiscanbeexplained: thedichotomybasedprogramactuallyseesf asapiecewiseconstantfunction,whereeachpieceisofthesizeofmachine precision.Inotherwords,forvaluesofx thatdifferonlyveryslightly,thevalueofy isexactlythesame,andthusthederivativeis rightly0 0 . .Onlythisisnotwhatyouhadinmindwhenyouwrotetheprogram! Apartfromtheimmediateruleofthumbwhichis"don'teventhinkofdifferentiatingdichotomybasedcode,don'tevenwriteit!",the conclusionisthatprogrammingoftenintroducesdiscontinuitiesintoperfectlydifferentiablemathematicalfunctions,andADcan'tdo muchaboutit(true,DividedDifferencesbehavebetteronthisquestion!).Whenthishappens,thederivativescomputedbyADare justuselessorplainlywrong.Allwecandoiswarntheuserwhenthishappens,whichiswhenthereisanondifferentiability "close"tothecurrentpoint,i.e.aroundthecurrentinputvalues. InTapenade,thereisanexperimentaldifferentiationmodethatevaluatesthedomainofdifferentiabillityaroundthecurrentpoint, followingagivendirectionintheinputdomain.Thecommandlineargument i e t a i buildsaspecialdifferentiated drcVld programwhich,givenapointintheinputspaceandadirectionintheinputspace(verymuchlikeinthetangentmode),evaluates theintervalofdifferentiabilityalongthisdirection.ThisusessubroutinescalledV L D T _ O A N . . , ,whicharedefined AIIYDMI_.()

inthefilev l d t T s . providedintheADFirstAidkit.Afterexecutionofthedifferentiatedprogram,theintervalof aiiyetf differentiabilityaroundthegiveninputpointandfollowingthegivendirectionisfoundin CMOvldt_etcmo/gi,ga,ifi,ifa OMN/aiiyts_omnmnmxnmnnmx whereg i (resp.g a )isthelower(resp.upper)bound,andifthereisnoconstraintonthelower(resp.upper)bound,then mn mx i f i (resp.i f a )is. R E nmn nmx TU. Caution:If,insidethesameexecution,youneedtoperformseveralindependentcallstothe"directionalvalidity"differentiated code,fordifferentinputsorfordifferentdirections,thendon'tforgettoresetthevalidityintervalto]infinity,+infinity[betweentwo successivecalls.Dothisbyresettingbothi f i andi f a to. R E nmn nmx TU. UsingthismodeineverydirectionoftheCartesianbasisoftheinputspacewillgiveaboxinsidewhichderivativesandderivatives basedoptimizationarevalid.

DifferenceswithOdysse:
IhavebeenusingOdysseinthepast.WhatarethemodificationsfromOdyssetoTapenade? Detailsonthisquestioninthehistoryofsuccessiveversions. TAPENADEisthesuccessorofOdysse.Therefore,TAPENADEisanincrementalimprovementtoOdysse.However,wehadto radicallymodifytheinternalstructuresofOdyssetotakeprofitofcompilationtheoryandtechnology.Theinternalrepresentationof aprogramisnowaCallGraph,withonenodepersubroutine,andeachsubroutineisnowaFlowGraph,whosenodesareBasic Blocks.EachBasicBlockisequippedwithaSymbolTable,andSymbolTablesarenestedtocapturescoping.BasicBlocks containinstructions,representedasAbstractSyntaxTrees.Staticanalysesnowrunonthisstructure,andthisyieldsanenormous speedupindifferentiationtime,withrespecttoOdysse.Rightfromthebeginning,thisinternalrepresentationofprogramswas designedindependentlyfromaparticularimperativelanguage,likeFortranorC.Forexampledatastructuresarealreadymanaged. Inprinciple,adaptiontoC,say,justrequiresaCfrontendandbackend,i.e.aCparserandaCprettyprinter. Aboutthedifferentiationmodel,TAPENADEisverysimilartoOdysse.Itstillprovidesyouwiththeforwardandreversemodeof A.D.Therearesomeimprovementsthough: Theactivityanalysis,astaticanalysisthatfindsthevariablesthatneednotbedifferentiated,isnowbidirectional.Itdetects thevariablesthatdonotdependontheindependents,andthevariablesthatdonotinfluenceonthedependents.Thisleads tofewerdifferentiatedvariables. WeincludedtheTBRanalysis,astaticanalysisforthereversemode.Itfindsvariableswhich,althoughtheyareoverwritten duringtheforwardsweep,neednotbesavedbecausetheywillnotbeusedduringthereversesweep. Weimplementedanewstoragestrategyforthereversemode,namedtheglobaltrajectory.Thisstoresintermediatevalues onanexternalstackratherthanonstaticarraysthatthadtobedimensionedbyhandbytheuserofOdysse Snapshotsinthereversemodearesmaller,becausetheyuseInOutanalysistofindoutvariablesthatarereallynecessary forduplicateexecutionofthecheckpointedpieceofcode. Theinversionofthecontrol,inthereversemode,isdoneontheFlowGraph.Thisgivesabetter,morereadabletransformed source,thatthecompilercancompilebetter. TAPENADEnowprovidesamultidirectionalforwardmode,similartotheADIFOR'svectormode.Itcomputesthedirectional derivativesinmanydirectionssimultaneously.Derivativeloopsaregatheredusingdatadependencyanalysis. DataFlowanalysisisusedtooptimizethetransformedsource,atthelevelofeachBasicBlock. WeprovidetwowaysofusingTapenade:eitherbyconnectingtothewebserver,orbydownloadingittoinstallitlocally.Thereisa graphicaluserinterfacebasedonXHTML,andasimplecommandlineinterface.

Documentation,ArchivesoftheMailinglist:
WhatistheavailabledocumentationonAutomaticDifferentiationingeneralandspecificallyonTapenade? HowcanItakeadvantageoftheexperiencefromprevioususersofTapenade? Hereisashortintroductionto"whatisAD?" ThewebsiteoftheADcommunitywww.autodiff.org containsthemostcomprehensivedocumentationonAD. AboutTapenade,apartfromthetutorialhere,youshouldstartbylookingattheTapenadeuser'smanual. AnothergoodstartingpointisourDocumentspage. Ifyousubscribetothetapenadeusersmailinglist,youwillhaveaccesstothearchiveofpreviousdiscussionsattheaddress: https://listssop.inria.fr/wws/arc/tapenadeusers.

PUSH/POPmechanism:
WherearethePUSHandPOProutinesdefined? Weprovideyouwiththesourceoftheseroutines,whichcomesastwofilesa B f e . anda S a k c dufrf d t c . .Youmaydownloadit byclickingonthe"DownloadPUSH/POP"buttoninthedifferentiationresultpage,orelse
DownloadFirstAidKit

Insomecases,thecompilermaycomplainthatsomerequiredPUSH/POPprocedureforsomeunusualtypeisnotin a B f e . ,e.g._pushinteger16.Inthiscaseedita B f e . andeitheruncommenttherequiredprocedureitifitisthereor dufrf dufrf orcreateitasindicated(seeinsidefilea B f e . ). dufrf

OriginalstatementsandresultsdisappearingfromtheReversecode:
MyoriginalprocedurecomputedanoutputYwhereasthereversedifferentiatedproceduredoesnotreturnthecorrectvalueofY. Somestatementsthatarepresentintheoriginalcodehavesimplydisappearedfromthereversecode(thismayevenresultin

emptyloopsorifbranches). Bydefault,thereversemodeofTapenadeperformsadjointlivenessanalysis.Thismeansthatsomestatementsfromtheoriginal program,whichwere"live"i.e.whichcomputedsomethingneededbytherestoftheprogram,arenotliveanymoreinthereverse differentiatedprogram.Thisoftenoccursaroundtheendofaforwardsweep,justbeforethebackwardsweep.Thereasonis:a prioritheonlyresultyouexpectfromthereversedifferentiatedprogramisthederivatives.Inparticulartheoriginalresultofthe originalprogramisnotconsideredasanecessaryresultofthereversedifferentiatedprogram.Thereforestatementsthatare neededonlyforthisoriginalresultbecomedeadcode.MoreexplanationsandasmallexampleareintheTAPENADE2.1user's guide,specificallyonpage3334.Acommandlineoptioncantogglethismechanismoff,see" o p i d o n l v n s " notmajitiees onpage38. Foruserswhowantthereversedifferentiatedprogramtopreservetheoriginaloutputsoftheoriginalprogram,weadvisefirstto toggleadjointlivenessanalysisoff.Howeverthisisnotenough,becausethePUSH/POPmechanismwillnaturallydestroyseveral oftheprogram'soriginalresults.Weadvisethatyoumodifythedifferentiatedprogrambyhand,savingthevariablesyouwantinto temporaryvariablesattheendoftheforwardsweep,andrestoringthesevariablesattheendofthebackwardsweep.

Basicandextremelyusefulcodepreparationbeforedifferentiation
IhaveacodeIwanttodifferentiate.HowshouldIproceed? Just"Differentiatingacode"doesnotmeanmuch,unlessyouknowpreciselywhichpartofthecodeisthemathfunctionyouwant todifferentiate,whichareits"independent"inputsand"dependent"outputs,andmaybewhichdifferentiationmodeyouwanttouse. Weadviseyoutofollowthestepsbelow.Thiswayyouwillbeforcedtoanswertheabovequestionsevenbeforeyoustartplaying withTapenade.Thesestepswillalsovastlyeaseyourlifewhenitcomestovalidatethedifferentiatedprograms.Finally,theywill makeourlifeeasierifwearetohelpyou,andifwehavetodebugTapenade! 1. Firstthingistodecidewhichisthepartofthecodethatdefinesthefunctionyouwanttodifferentiate.Itiscertainlynotyour completeprogram!Itmustbeaconnexpieceofcode(callittheinside)withauniquebeginning(callittheentry)andaunique end(callittheexit).Thispieceofcodecanverywellcontaincallstootherprocedures.Everyrunofyourcodethatgoes throughtheentrymustthenreachtheexit.Inparticular,theremustbenocontroljumpsfromoutsidetotheinside,exceptto theentry.Similarly,theremustbenocontroljumpsfromtheinsidetotheoutside,exceptfromtheexit.Makethis"inside" pieceofcodeasubroutineorfunction(inFORTRAN)oraprocedure(inC).Youoriginalcodeisthereforemodified:you havecreatedanewprocedure,andnowthereisacalltothisprocedureinplaceoftheoriginalpieceofcodetodifferentiate.It nowlookslike(FORTRANstyle): ..waeeoeieoe.. .htvrcdsbfr. CBgnigohucinIwnodfeetae einnftefntoattifrnit ALMFN(12..12.. CLYUCx,x,.,y,y,.) CEdohucinIwnodfeetae nftefntoattifrnit ..waeeoecmsatr.. .htvrcdoefe. 2. Itgoeswithoutsaying,butmakesureyouusesomemechanismtomakenecessaryvariablestraveltoandfromthenew procedure.Youmayuseparameterpassing,orFORTRANcommon,orglobals...asyouwish.Weshallcallthesevariables the"parameters"oftheprocedure(eveniftheyareindeedglobals).Itisveryusefulthatyouknowthesevariableswell,what theymean,whattheyareneededtoo,aretheyjustusedorjustwrittenandreturned,orboth,etc...Nowyouseewhythis preparationworkmustbedonebysomeonewhoknowsthecode! 3. ApplyTapenadeinthe"nodifferentiation"mode,whichiscommandlineoption"p".Thiswilltriggerseveralanalysisbutwill generateasourcecodethatshouldbeequivalenttotheoriginalcode,withoutanydifferentiation.Theinterestistocheckthat Tapenadeisabletoparseyoursourcefilescorrectly.AnotherinterestistoseethewarningmessagesissuedbyTapenade. Someofthesemessagesmayindicatesituationswherethingscangowrong.Therefore,weadviseyoutotypesomething alongthelinesof: >tpndpoifrnitdsucieuoicuefls $aeaeondfeetae"oreflsbtnnldie" Thiswillreadallyoursourcefiles(donotprovidetheincludefilesinthecommandlinebecauseTapenadewillreadthem automaticallywhenneeded).Yoursourcecodewillbeparsedandanalyzed,andthiswillresultinafile n d f e e t a e _ . s .Itcontainsa(verbous)listofallwarninganderrormessages,thatyoushouldbrowse oifrnitdpmg through,withthehelpofthedocumentationonmessages.Theexplanationsinthereferencemanualwillhelpyoufindwhich messagescanbediscarded(e.g.,veryoften,theimplicitconversionsbetweenR A * andD U L R C S O ),and EL8 OBEPEIIN whichonescan't.Tapenadeshouldhavegeneratedbynowanewsourcefilen d f e e t a e _ . f f 0 c ,that oifrnitdp[/9/] containsaduplicateofyouroriginalsource,with" P appendedtoprocedurenames.Youmaylookatittocheckthatitis _" identicaltoyoursource,andyoumaycallitinsteadoftheoriginalcodeandcheckyougetthesameresults.Yourcodecould nowlooklike: ..waeeoeieoe.. .htvrcdsbfr. CBgnigohucinIwnodfeetae einnftefntoattifrnit ALMFN_(12..12.. CLYUCPx,x,.,y,y,.) CEdohucinIwnodfeetae nftefntoattifrnit ..waeeoecmsatr.. .htvrcdoefe.

THEFOLLOWINGPREPARATIONSTEPSWEREUSEFUL FORTHEOLDVALIDATIONMETHODONLY

Wekeepthemherefortherecordonly,buttheyarenotnecessary!
1. Writeaprocedurethatsavesallthese"parameters"ofM F N intosomestorage(canbestaticarrays),andaprocedurethat YUC restoresthesavedvaluesintotheseparameters.Youmaytesttheminsideyourmodifiedcode,justtomakesure!Yourcode couldnowlooklike: ..waeeoeieoe.. .htvrcdsbfr. ALSV_EESR_AASx,x,.,y,y,.) CLAENCSAYPRM(12..12.. ALRSOENCSAYPRM(12..12.. CLETR_EESR_AASx,x,.,y,y,.) CBgnigohucinIwnodfeetae einnftefntoattifrnit ALMFN(12..12.. CLYUCx,x,.,y,y,.) CEdohucinIwnodfeetae nftefntoattifrnit ..waeeoecmsatr.. .htvrcdoefe. 2. Callthenewproceduretwiceinarow,insidethesameexecution.checkyouobtainthesameresults.Soyouwillbesure thatthereisnounwantedsideeffectorundefinedvariableusedinyourcodetobedifferentiated.Yourcodecouldnowlook like: .htvrcdsbfr. ..waeeoeieoe.. CSvhtieesrntecretsae aewasncsayihurntt ALSV_EESR_AASx,x,.,y,y,.) CLAENCSAYPRM(12..12.. CBgnigohucinIwnodfeetae einnftefntoattifrnit ALMFN(12..12.. CLYUCx,x,.,y,y,.) e1y1 rs_1=y e1y2 rs_2=y . .. CPeaefrtescnu rproheodrn ALRSOENCSAYPRM(12..12.. CLETR_EESR_AASx,x,.,y,y,.) CClYUCaan alMFNgi ALMFN(12..12.. CLYUCx,x,.,y,y,.) e2y1 rs_1=y e2y2 rs_2=y . .. aesrle1yy=e2yy mkuealrs_y=rs_y CEdohucinIwnodfeetae nftefntoattifrnit .htvrcdoefe. ..waeeoecmsatr.. 3. Nowchoosewhichonesamongtheabove"parameters"youwanttodifferentiate(the"dependents"),withrespecttowhich ones(the"independents").Bewarethesemustallbeelementsofthesetof"parameters".Wedonotdifferentiateaprocedure withrespecttoitslocalvariables,constants,orwithrespecttovariablesthattheproceduredoesnotsee.Inparticular,it happensthatsomeoftheseparametersaredynamicallyallocated(resp.freed).Thenitisessentialthattheallocation(resp. free)isoutsideoftheproceduretobedifferentiated.Althoughthisisnotasessential,itisalsogoodpoliticstoavoiddynamic allocation/freeinganywhereinsidetheprocedurestobedifferentiated.Thismaycausehardproblemsespeciallywiththe reversemode. 4. Whynottryandcomputea(approached)derivativebydivideddifferencesthen?It'smaybealittleearlybutyouwillneedto doitanywayfortheoldvalidationstep.Defineadifferentiatedvariableforeachofthedependentandindependent.Initialize themallto1.0.Modifyeachindependentinputsby"epsilon"timesitsderivativevariable.Runtheprocedureonceandkeep theresult.Resettheinputs,thistimewithoutthe"epsilon"modification,andrunagain.Thenevaluatethedifferenceof outputs,dividedby"epsilon".ItisthesumofallcolumnsoftheJacobian.Forinstance,ifyouwantallinputstobe independentandalloutputstobedependent,andsetting"ddeps"theepsilon,yourcodecouldnowlooklike: .htvrcdsbfr. ..waeeoeieoe.. CIiilzheiaiefteidpnetaddpnetprmtr ntaietedrvtvsohneednneednaaees C(o'ogttelrhsaibe) dntfreodcaeteevrals!: 1. xd=10 2. xd=10 . .. 1. yd=00 2. yd=00 . .. CSvhtieesrntecretsae aewasncsayihurntt ALSV_EESR_AASx,x,.,y,y,.) CLAENCSAYPRM(12..12.. CMdfhurntt: oiytecretsae dp. des=1d5 1=xdp*1 x1+desxd 2=xdp*2 x2+desxd . .. CCl(pio*d alFX+eslnX) ALMFN(12..12.. CLYUCx,x,.,y,y,.) e1y1 rs_1=y e1y2 rs_2=y . .. CPeaefrtescnu rproheodrn ALRSOENCSAYPRM(12..12.. CLETR_EESR_AASx,x,.,y,y,.) CCl() alFX ALMFN(12..12.. CLYUCx,x,.,y,y,.) e2y1 rs_1=y e2y2 rs_2=y . .. CCmueapo_d=((+pio*d()/pio optprxYFXeslnX)FX)esln

prxyd=(e1ye2y)des apo_1rs_1rs_1/dp . .. aesrhseiaieaesne mkueteedrvtvsmkes! CEdohucinIwnodfeetae nftefntoattifrnit .htvrcdoefe. ..waeeoecmsatr..

IndependentandDependentsets.Initializationandusageofthedifferentiatedparameters
WhatvariablesshouldIlistinthe" a s v r "and" u v r "argumentsofthedifferentiationcommand?(alsoknownasthe otas independentanddependentparameters) Alsorelated:formyrootprocedure,thedifferentiatedprocedurethatIobtainhasadditionaldifferentiatedparameters. WhatvalueshouldIgivetothesevariablesuponcallingthedifferentiatedrootprocedure? WhatvaluewillIobtaininthesevariablesuponreturn? Tobeginwith,whatwerefertoas"parameters"hereisnotlimitedtoformalparametersintheprocedure'sargumentlist.Thefollowingappliesjust aswelltoglobalparameterssuchasthoseinaC M O orinthevariablesofaU E OMN S 'dmodule.Ifthesevariablesareactive,differentiatedvariables willappearclosetothem,andiftheseactivevariablesareinputsoroutputs,youwillhavetothinkaboutprovidingtheirinitialvalueandretrieving theirresultvalue. Nowsomenotation.SupposewearecallingTapenadeonsomerootprocedureF ,andcallV someparameterofF .Althoughweadvocate differentiatinginasinglebatchallthesubroutinesthatdefinethemathematicalfunctionofinterest,itmayhelpinthefollowingexplanationtothink ofF asonlyapartofthisbigmathematicalfunction.TheningeneraltheremaybesomeupstreamcodeU thatrunsbeforeF iscalled,andsome downstreamcodeD thatrunsafterF iscalled.Thecompletemathematicalfunctionisimplementedby{ F } U D .Thiscodesequencehasan inputthatwecallthe"mainentry",andanoutputthatwecallthe"mainresult".Thegeneralgoalistocomputethederivativesofthemainresultwith respecttothemainentry.AssumingthatU andD arealsodifferentiatedinsomeway,thequestionsare:shouldV gointotheindependentand dependentsets,whatisthenewparameterV (orV )ofthedifferentiatedF D d b _ (orF B _ ),whatvalueshouldbeprovidedintoit,andwhatvaluewill itcontainuponexit?Theheaderofeachdifferentiatedrootprocedureanswerssomeofthesequestions.However,someexplanationsmaybe useful.

Question1:Whatshouldtheindependentanddependentparametersbesetas?
ItiswisenottothinkofthoseasthesetsofinputsandoutputsofF thatareconnecteddifferentiablythroughF .Tapenadecanfindthis byitself! WhatTapenadedoesexpectfromyouisinformationaboutthecontextoutsideF .Asaruleofthumb: ThinkoftheindependentsastheinputsofF thatareinfluencedbythe"mainentry",i.e.thataredifferentiablyconnectedtothemain entrythroughU .InthedegeneratecasewherethereisnoU (emptyU ),theindependentsmustbetheinputsofF forwhichyou,the enduser,promisetogiveatangentderivativeorexpectareversederivative. ThinkofthedependentsastheoutputsofF thatinfluencethe"mainresult",i.e.thataredifferentiablyconnectedtothemainresult throughD .InthedegeneratecasewherethereisnoD (emptyD ),thedependentsmustbetheoutputsofF forwhichyou,theend user,expectatangentderivativeorpromisetogiveareversederivative. Forexample,ifthecontextoutsideF is (naotbFi:,u:)(nbi:,u:) Ui:,u:)(nbotcDi:,ncotd b isuseddifferentiablylaterinD .Therefore,evenifF onlyreadsb anddoesnotmodifyit,b mustbeinthesetofdependents. Anotherpossiblemistakecomeswithloops.Ifthecodelookslike (naotbloFi:,u:)}Di:,u:) Ui:,u:)op{(nbotc(ncotd thedownstreamcodeafterF includesF itself,becauseoftheloop,andthereforeb mustbeinthedependents. PleasenoticethatTapenademayremovevariablesfromtheindependentsordependentssetsthatyouprovided.Tapenadedetectssuperfluous independents,i.e.inputsthathavenodifferentiableinfluenceonanydependent.Similarlyasuperfluousdependentisanoutputthatdoesn't dependdifferentiablyonanyindependentandisn'tindependentitself.Superfluousvariablesareremovedfromtheirset,anda"command" messageisissued.Ifyoudon'tspecifyanyindependents(resp.dependents),Tapenadechoosesforyouthelargestpossibleset,usingits dependenceanalysistoremovesuperfluouselements. TherearealsosituationswhereTapenadeaddsvariablestotheindependentordependentsetsthatyouprovidedforarootprocedure.Intangent mode,ifV isintheindependentsetandV maybeoverwritteninF D d _ ,thenV isaddedintothedependentset.Converselyinreversemode,ifV is inthedependentsetandV maybeoverwritteninF B b _ ,thenV isaddedintotheindependentset.Inbothcasesa"command"messageisissued. ThisisTapenade'swayofwarningtheenduserthat,althoughV wasnotinthedependent(resp.independent)setprovided,theoriginalV (resp. d V )providedwillbemodifiedbythederivativeprocedure,andthereforewilllooseitsoriginalvalueandshouldbeusedwithcareafterthat. b

Question2:Whatistheeffectofavariablebeingintheindependentand/ordependentsets,ontherequired initializationandreturnedvalueofitsderivative?
Whateverindependentsanddependentssetsyouprovided,itiswisetocheckthesesetsmodifiedandactuallyusedbyTapenade,aslistedinthe commentheaderofthedifferentiatedprocedure.Thesesetsdeterminehowthederivativesmustbeinitializedandwhattheywillcontainuponexit. Thisinformationisalsosummarizedinthecommentheaderofeverydifferentiatedrootprocedure. Thefollowingtableshowsandexplainsthepossiblecases.Intangentmode,thingsarestraightforward.Inreversemodethingsprovealittlemore surprising,buttheyare"logical"consequencesofthetranspositionoftheJacobianmatrixthatisbehindthereversemode.Togivearoughidea,in reversemodetheflowofdatagoesinthereversedirection:IfV isanoutputofF ,thenF B _ mustbecalledwithanadditionalinputV ,and B converselyifV isaninputofF ,thenF B _ willreturnanadditionaloutputV .WhenavariableispurelyreadbyF B ,itsderivativeispurely incremented(andviceversa)...ohyes,you'rerightit'scomplicated!Butweneversaidthatthereversemodeissimple,it'sonlyextremelyefficient

andworththeeffort! IfV islistedas... thenintangentmode,V ... d ...isarequiredinput,andanunspecifiedoutput.Uponentry, FD _ expectsavalueinV ,whichshouldbeingeneralthe d derivativeofV uponF entrywithrespecttothe"mainentry". Intheparticularcasewherethisderivativeisnull,itisthe responsibilityofthecallingcontexttosetV to0 0 d . before thecalltoF D _ . ...anindependent (i.e."v r i g a y n ") inputonly DependingofwhatF doestoV d ,V mayverywellbe modifiedbyF D _ :sinceV isnotinthedependentoutputs, thevalueofV uponF D d _ exithasnomeaningandshould notbeused.ThisisokifV uponF exithasnoinfluenceon the"mainresult". WhenF isadifferentiationrootprocedure,thisappearsin theRWstatusintheheaderofF D _ as"V i "or"V i :n :n k l e "tostresswhenV ismodifiedbyF D ild d _. theninreversemode,V ... b ...isaresultonly.UponF B _ exit,V containsthederivativeof b the"mainresult"withrespecttotheV uponF entry.Thisis possiblebecauseF B _ expectsasinputthederivativeofthe "mainresult"withrespecttoeachofF 'sdependentoutputs. SinceV isnotinthedependentoutputs,F B _ expectsno meaningfulvalueprovidedatcalltimeinV .Actually, b whateverisintherewillbehappilydiscarded,resettozero, andoverwritten.ThereforeitsvalueuponF B _ entrywillbe lost. WhenF isadifferentiationrootprocedure,thisappearsinthe RWstatusintheheaderofF B _ as"V o t : u "or"V z r " :eo whentheV isnulluponreturnfromF B b _.

...isarequiredinput,andanunspecifiedoutput.Uponentry, FB _ expectsavalueinV ,whichshouldbeingeneralthe b ...isaresultonly.UponF D _ exit,V containsthederivative derivativeofthe"mainresult"withrespecttoV d uponF exit.In ofV uponF exitwithrespecttothe"mainentry".Thisis theparticularcasewherethisderivativeisnull,itisthe possiblebecauseF D _ expectsasinputthederivativesof responsibilityofthecallingcontexttosetV to0 0 b . beforethe F 'sindependentinputswithrespecttothe"mainentry". calltoF B _ . ...adependent(i.e. "u e u ")output sfl only SinceV isnotintheindependentinputs,thevalueofV d uponF entryismeaningless.Actually,whateverisinthere willbehappilydiscardedandoverwritten,Thereforeits valueuponF D _ entrywillbelost. WhenF isadifferentiationrootprocedure,thisappearsin theRWstatusintheheaderofF D _ as"V o t : u "or "V z r "whentheexitV isnull. :eo d DependingofwhatF doestoV b ,V mayverywellbemodified byF B _ :sinceV isnotintheindependentinputs,thevalueof V uponF B b _ exithasnomeaningandshouldnotbeused. ThisisokifV uponF entrydoesnotdependonthe"main entry". WhenF isadifferentiationrootprocedure,thisappearsinthe RWstatusintheheaderofF B _ as"V i "or"V i :n :n k l e "tostresswhenV ismodifiedbyF B ild b _. ...isarequiredinputandaresult.Uponentry,F B _ expectsa valueinV ,whichshouldbeingeneralthederivativeofthe b "mainresult"withrespecttoV uponF exit.Intheparticular casewherethisderivativeisnull,itistheresponsibilityofthe callingcontexttosetV to0 0 b . beforethecalltoF B _ . UponF B _ exit,V containsthederivativeofthe"mainresult" b withrespecttotheV uponF entry.Thisispossiblebecause FB _ expectsasinputthederivativeofthe"mainresult"with respecttoeachofF 'sdependentoutputs.

...isarequiredinputandaresult.Uponentry,F D _ expectsa valueinV ,whichshouldbeingeneralthederivativeofV d uponF entrywithrespecttothe"mainentry".Inthe particularcasewherethisderivativeisnull,itisthe responsibilityofthecallingcontexttosetV to0 0 d . before thecalltoF D _ . UponF D _ exit,V containsthederivativeofV d uponF exit withrespecttothe"mainentry".Thisispossiblebecause FD _ expectsasinputthederivativesofF 'sindependent inputswithrespecttothe"mainentry".

...independentand dependent

WhenF isadifferentiationrootprocedure,thisissummarized intheRWstatusofV intheheaderofF B _ .Thisstatuscan WhenF isadifferentiationrootprocedure,thisis startwith"i "whentheentryvalueofV isused,and n b summarizedintheRWstatusofV intheheaderofF D _ .This continueswith statuscanstartwith"i "whentheentryvalueofV isused, n d andcontinueswith "o t u "whenV ismodifiedbyF B b _ "z r "whenV isresettozerobyF B eo b _ "o t u "whenV ismodifiedbyF D d _ ornothingwhenV isnotmodifiedbyF B b _ "z r "whenV isresettozerobyF D eo d _ ornothingwhenV isnotmodifiedbyF D d _ TheRWstatuscanalsobe"i c ",meaningthatV issimply nr b incrementedandnototherwisereadnorwrittenbyF B _.

Example:considerforinstanceprocedureF below,thatshouldillustratemanyofthepossiblesituations. sbotn(2v,3w) uruieFu,1v,2 ra1u,3v,2v,1w,3 elu,2u,1v,3w,2w cmoc/1u,2w,3 omn/cu,3v,1w c u3v*1 3=u+1u u3u*2 3=u2v u.*1v 2=30u*1 u3Gu) 3=u+(2 c v320v 3=v+.*1 v3v 3=v2 v.*1 2=30v v3Gv) 3=v+(2 c w3v*2 3=w+1v w3w*1 3=w2u

w.*1 2=30w w3Gw) 3=w+(2 ed n raucinGx elfnto() ra elx G=xx * ed n FunctionG issomepureuserdefinedfunction.Webuiltthiscodesothatall* parametersareonlyreadbyF 2 1 ,* parametersareread,then overwrittenandmaybereadagain,and* parametersareonlyincremented,whichisaninterestingcaseforreversedifferentiation.Parameters 3 arescatteredasformalargumentsorglobalvariables,justtoshowitdoesn'tmatter. Supposethattheenduserdecidedthatallv andw parametersareindependents,andallu andw aredependents,e.g.usingthecommand * * * * lineoption: ed"(1v3w2w)(1u3w2w) haFv2v1w3>u2u1w3" Intangentmode,procedureF D _ is Inreversemode,procedureF B _ is

Cifrnitoffiowrtnetoe Dfeetainonfrad(agn)md: Cvrainfueueut:u2w2u aitososflrsls3v3w2 Cwtepcovrigipt:v1w1w ihrsettaynnus2w3v2 CRttsoifvrals3otv:not Cifrnitoffieesajitoe Wsaufdfaibe:u:u2iu Dfeetainonrvre(don)md: Cw:nw:notv:nw:notu:u Cgainfueueut:u1w2u 1i3iu1i2iu2ot rdetosflrsls3w3w2 S B O T N _ ( 2 2 , v , v d 3 2 2 )C w t e p c o v r i g i p t : u 2 w 3 v URUIEFDu,ud11,v,w,wd ihrsettaynnus3v1w1 IPIIOE MLCTNN C2u w2 C CRttsoifvrals3iu2ot Wsaufdfaibe:u:notv:u RA123123123 ELu,u,u,v,v,v,w,w,w C1ic3iu1otw:notu:not w:nrw:notv:u2iu2iu RA2,ud1,vd1,wd3 ELud3,vd2,wd2,wd SBOTN_(22,v,vb322) URUIEFBu,ub11,v,w,wb CMOc/u,u,v,w,w OMN/c13213 IPIIOE MLCTNN RA ELG C RA_ ELGD RA123123123 ELu,u,u,v,v,v,w,w,w RAeut ELrsl1 RA2,ub1,vb1,wb3 ELub3,vb2,wb2,wb RAeutd ELrsl1 CMOc/u,u,v,w,w OMN/c13213 CMOc_/ud2,wd3 OMN/cd3,vd1,wd RA ELG C RAeut ELrsl1 ud=u*1 31vd RAeutb ELrsl1 u3+v*1 3=u1u CMOc_/ub2,wb3 OMN/cb3,vb1,wb ud=udu*2 332vd CLUHEL(2 ALPSRA4u) u3u*2 3=u2v C ud=30u*1 2.*1vd u.*1v 2=30u*1 u.*1v 2=30u*1 CLUHEL(2 ALPSRA4v) rsl1_(22,rsl1 eutd=GDu,udeut) C ud=ud+rsl1 33eutd v.*1 2=30v u3+rsl1 3=ueut C C w.*1 2=30w v3+20v 3=v.*1 rsl13 eutb=wb v3v 3=v2 CL_(22,rsl1) ALGBw,wbeutb vd=30vd 2.*1 wb=wb+30wb 11.*2 v.*1 2=30v wb=u*3) 2(1wb rsl1=Gv) eut(2 vb=v*3 12wb v3+rsl1 3=veut vb=v*3 21wb C CLORA4v) ALPPEL(2 wd=wd+vdv1vd 331*2+v*2 rsl13 eutb=ub w3+v*2 3=w1v CL_(22,rsl1) ALGBu,ubeutb wd=wdu*2 331wd vb=vb+u*.*21ub+30vb 11130ub+u*3.*2 w3w*1 3=w2u CLORA4u) ALPPEL(2 wd=30wd 2.*1 ub=v*3) 2(2ub w.*1 2=30w vb=u*3) 2(2ub rsl1_(22,rsl1 eutd=GDw,wdeut) ED N wd=wd+rsl1 33eutd w3+rsl1 3=weut ED N YoucanseethatTapenadetooku andw outofthesetofdependents 1 1 becausetheyarenotinfluenceddifferentiablybyanyindependent.It alsotookv outoftheindependentsbecauseithasnodifferentiable 3 influenceonanydependentandisnotdependentitself.Also,Tapenade addedv tothesetofdependentsbecauseitsderivativehastobe 2 modifiedbyF D _ . Sincev , v , w , w ,andw areindependent(i.e.inthecomment 1212 3 "v r i g i p t "),procedureF D aynnus _ expectstobeprovidedwiththeir derivativesv d 2 , w d 2 ,andw d 1,vd1,wd 3 withrespecttothe"main entry".Aftermodificationoftheindependentanddependentsetsby Tapenade,thereissimplynou d 1 norv d 3 sincetheyareneither independentnordependent.ThevalueuponF D _ entryoftheother derivativevariablesu d 2 andu d 3 aremeaninglessandarehappily YoucanseethatTapenadetooku outofthesetofdependents 1 becauseitisnotinfluenceddifferentiablybyanyindependentanditis notanindependentitself.Italsotookv outoftheindependents 3 becauseithasnodifferentiableinfluenceonanydependentandisnot dependentitself.Also,Tapenadeaddedu andu tothesetof 2 3 independentsbecausetheirderivativehastobemodifiedbyF B _. Actuallythisisnotthecaseforu becauseu isonlyincrementedinF 3 3 , butTapenadedoesnotdetectthis(yet?). Sinceu , u , w , w ,andw aredependent(i.e.inthecomment 2312 3 "u e u e u t "),procedureF B sflrsls _ expectstobeprovidedwiththe derivativesu b 3 , w b 2 ,andw b 2,ub1,wb 3 ofthe"mainresult"with respecttothem.Aftermodificationoftheindependentanddependent setsbyTapenade,thereissimplynou b 1 norv b 3 sincetheyareneither independentnordependent.Theinputvaluesoftheotherderivatives

overwritteninsideF D _ .Sobewarethatwhateverwasintherebefore callingF D _ maybelost. Thedependentarevariablesu , u , v , w ,andw (i.e.inthe 2322 3 comment"u e u e u t "),sothattheycontainuponF D sflrsls _ exitthe derivativeofthecorrespondingoriginalvariableuponF D _ exitwith respecttothe"mainentry".Variablesu , v , v ,andw arenot 113 1 dependent,thereforetheexitvalueoftheirderivativeisnotspecified.In factthesederivativesarenotoverwrittensothereisnorisk.

vb 1 andv b 2 aremeaninglessandmaybehappilyoverwritteninside FB _ .SobewarethatwhateverwasintherebeforecallingF B _ maybe lost. AttheotherendofF ,theindependentarevariablesu , u , v , 231 v , w , w ,andw ("v r i g i p t ").Thereforetheycontain 212 3 aynnus uponexitfromF B _ thederivativeofthe"mainresult"withrespectto theircorrespondingoriginalvariable,uponF entry.

Noticefurthermorethatvariablesu andw areonlyincrementedinside 3 3 Inotherwords,variablesu , v , v ,andw areonlyreadinsideF F 113 1 andthereforetheirderivativesu b 3 andw b 3 areonlyreadbyF B _. andthereforetheirderivativeseitherdon'texistorareonlyreadbyF D Theyretaintheirinputvalue.Still,Tapenadewasnotabletodetectthat _. Theyretaintheirinputvalue. andreturnedthemoregeneralRWstatus"i u " not

Validationofthedifferentiatedprograms
Help,thedifferentiatedprogramcrashes! HowcanIbesurethatthedifferentiatedprogramreallycomputeswhatIwant? HowcanIcheckthederivativesandfindtheplacewherethedifferentiatedprogramgoeswrong? Thisisadifficultissue.Differentiatedprogramsareverylongandcomplexandyoumayfindithardtocheckthembyhand,evenif wetrytokeepthemreadable.Hereisaproposedmethodtovalidate/debugthecodeproducedbyTapenade.ItusesTapenade optionsthatareavailableonlyinthecommandlineversion(localinstallation).Italsousestwofilesthatareinthe AFrtiKd D i s A d i directoryofyourlocalinstalledTapenadefiles.Ifyoudon'thavealocallyinstalledTapenade,pleasefollow thesesteps.Also,toeaseourvalidationmechanism,westronglyadviseyoutofollowafewpreparationsteps,beforeyoueven starttodifferentiateanything.Validationisbynomeanseasy.Althoughwetrytoautomatetheprocess,itstillrequiresgood knowledgeoftheprinciplesofAD,especiallyfortheadjointmode.ThesadthingwithTapenade'svalidationmechanismisthatyou mayrunintobugsofTapenadeitself(otherwiseyouwouldn'tbereadingthissectionanyway!)butyoumayalsorunintobugsofthe validationmechanismitself. Inthefollowing,wordsbetweendoublequotesanddrawninthiscolor(suchas"SHAPE")areindeedtemplates,thatmustbe instantiatedbyactualstringsdependingonthecontext.Squarebracketsrepresentadditionalarguments,andthestarrepresents repetition. Intheexamplepiecesofprograms,actualpiecesofcodearewritteninthissortofdarkred,shortstringsindoublequotesare "patterns"asabove,and "nirieeweobeqoenrwnteptenclr eteelnsbtendulutsaddanihatroo" describeapieceofcodethatyoumustwriteatthatplace. ThechoiceofIndependentandDependentparametersmayinterferewiththevalidationprocess.Thisisexplainedattheendofthis section,butwe'dratherwarnyourightnow.Makesureyoureadthispartifvalidationfailsclosetotheentryintoorexitfromyour code.

CHECKTHESIZESOFPRIMITIVETYPES:
Iftheprogramcrashes,thereisasmallchancethatthiscomesfromthesizesofbasictypesonyoursystem.Totestthesesizes,go intotheA F r t i K d D i s A d i directory,then: 1)adaptthecompilationcommands$(FC)and$(CC)intheMakefile 2)maketestMemSize 3)executetestMemSize 4)lookatthesizesprinted. ThestandardTapenadeexpects:integers:4integers*8:8reals:4reals*8:8doubles:8complexes:8complexes*8:8 complexes*16:16doublecomplexes:16logicals:4characters:1 iftestMemSizeonyoursystemgivesdifferentsizes,usethetypesizeoptionsthatchangethedefaultsizestowhatyoursystem expects.

CHECKTHETANGENTDERIVATIVESWITHRESPECTTODIVIDEDDIFFERENCES:
Comparisonwithdivideddifferencesisthestandardwaytotestthetangentderivatives.Itamountstorunningyouroriginal subroutinePwithaperturbedinputX * d + X ,thenwithX ,thencomputing:

YoushouldgetnearlythesameresultaswhenrunningthetangentdifferentiatedsubroutineP d X X , ) _ ( , d Y Witha"good"epsilon, differencesshouldbeonlyonafewlastdecimals.Thereisalottosayonthechoiceofagoodepsilon.Itdependsonthealgorithm, thedata...andmaybealsothephaseofthemoon!Also,itiscertainlymoreaccuratetocomputethecentereddivideddifferences, butwegenerallydon'tneedthisdegreeofaccuracyforourvalidation.Torunthetest,startfromthepreparedcodeaccordingtothe recommendedcodepreparation.Supposethe"central"partlookslike: ..waeeoeieoe.. .htvrcdsbfr. CBgnigohrcdrodfeetae einnftepoeuetifrnit ALMFN(12..12.. CLYUCx,x,.,y,y,.) CEdohrcdrodfeetae nftepoeuetifrnit

..waeeoecmsatr.. .htvrcdoefe. Definetwocopiesofyourcompletecode,thatwilldifferonlyinthis"central"part. Thefirstcopy,thatwewillcall" o e D " c d D 1 ,compilesintoanexecutablethatwewillcall" x D 1 .Insteadofcalling eeD" M F N ,itcallsitstangentdifferentiatedversionM F N _ ,callingafewdebuggingprimitivesbeforeandafter.Thecentral YUC YUCD partof" o e D " c d D 1 isthus: ..waeeoeieoe.. .htvrcdsbfr. Nwiiilzhnuifrnitdvralso ontaieteiptdfeetaeaibe,t dfnhhsndfeetaindrcin.. eietecoeifrnitoieto,eg: 1.d xd=150 2/1d,.d,050..) xd=(.0230.d,./ .. . Nwpeaefrtets: orprohet ALDBGTTII11d8.,.d) CLEU_G_NT(.,1d510 ALDBGTTIIRA8x,xd CLEU_G_NTEL(11) ALDBGTTIIRA8RA(22,n) CLEU_G_NTELARYx,xds .. . ALDBGTTCL(MFN'tu.fle) CLEU_G_AL'YUC,.re,.as. CBgnigoifrnitdpr einnfdfeetaeat ALMFN_(11,x,xd.,y,yd22,.. CLYUCDx,xd22,..11,y,yd.) CEdoifrnitdpr nfdfeetaeat Nwtriaetets: oemnthet ALDBGTTEI( CLEU_G_XT) ALDBGTTCNLDRA8'1,y,yd CLEU_G_OCUEEL(y'11) ALDBGTTCNLDRA8RA(y'22,2) CLEU_G_OCUEELARY'2,y,yd0 .. . ..waeeoecmsatr.. .htvrcdoefe. Thesecondcopy,thatwewillcall" o e D " c d D 2 ,compilesintoanexecutablethatwewillcall" x D 2 .Itisstrictlyidentical eeD" to" o e D " c d D 1 exceptthatitcallsD B G T T I I 2 E U _ G _ N T insteadofD B G T T I I 1 EU_G_NT Afewremarksandexplanations: Eachcopy" o e D " c d D 1 and" o e D " c d D 2 willbeexecutedindependently.Eachmustuseorreadfromthesamedatafilesas theoriginalcode,andshouldbehaveexactlythesameuntilreachingthecentralpart.Youdon'tneedtoduplicateevery sourcefiletodefinethesecopies" o e D " c d D 1 and" o e D " c d D 2 .Youprobablyneedonlytoduplicatethefilethatcontains the"central"part.Therestshouldbetakencareofbythecompilationandlinkcommands. ThecodeforM F N _ mustbebuiltbyTapenadebytangentdifferentiationofthesourcecode,withtheonlyadditional YUCD commandlineoption e u T T d b g G .Thesamedifferentiatedcodeisusedby" x D 1 and" x D 2 . eeD" eeD" Eachexecutable" x D 1 and" x D 2 mustbelinkedwiththedebugginglibraryprovidedinthe"FirstAidKit".This eeD" eeD" libraryconsistsoffiled b g D f e u A . anditsincludefiled b g D i c euA.n ThecallstoD B G T T I I 1 E U _ G _ N T ortoD B G T T I I 2 E U _ G _ N T musthavethesamearguments.Thefirstisthe"epsilon"used fordivideddifferences.Thesecondiscalled"almostzero"(seebelow).Thethirdisthe"threshold"(seebelow)error percentageabovewhichamessageisprinted. Inthepreparedcodeabove,noticethatthereisastatementthatinitializeseachinputdifferentiatedvariable.Youarefreeto choosethevalues,thereforechoosingthedirectionofyourtangent.Naturally,arraysrequireseveralvalues. Afewlineslater,beforecallingM F N _ ,youmustinsertoneinitializationcallforeachinputdifferentiatedvariable.For YUCD arrays,initializationprocedureD B G T T I I R A 8 R A takesanextraargument,whichisthesizeofthearray.For EU_G_NTELARY singleprecisionREAL's,replaceR A 8 E L byR A 4 EL. ThesameremarksapplytotheD B G T T C N L D . . E U _ G _ O C U E . debugterminationcalls. Don'tforgettodeclarealldifferentiatedvariablesinthedeclarationpartofyourcodes" o e D " c d D 1 and" o e D " cdD2. When" x D 1 and" x D 2 areready,runthedivideddifferencestestbyjusttypingthecommandline: eeD" eeD" >eeDxD2 $xD1|eeD usingaUNIXpipe"|"(orequivalent).Whathappensbehindthesceneis:e e D runsontheinput"plusepsilontimesdirection". xD1 Atselectedmoments("points"inexecution),itsendsthevalueofselectedvariablesintostdout,thereforeintothepipe. Inparallel,e e D runsontheinput"withoutepsilon".Atthesameselectedpoints,itreadsthevalueofthesameselected xD2 variablesfromstdin,thereforefromthepipe.Usingitsownvalueofthesamevariable,e e D computesdederivative xD2 approximationbydivideddifferences(the"DDderivative")andcomparesitwiththeanalyticalderivative.IftheDDderivativeis almostzero(i.e.absolutevaluebelowthegiven"almostzero"),thecomparisonisnotdonebecauseTapenadecodesometimes doesn'treinitializezeroderivatives(tosavetime).Otherwise,ifthepercentageofdifferenceismorethatthe"threshold",amessage isprintedonthescreen.Whatyouobtainonthescreenlookslikewhatfollows.Inthisexample,weintroducedanerrorintothe tangentdifferentiatedcode,insidethetangentderivativeofsomefunctioncalled"P L S R ": OYUF $xD1|eeD >eeDxD2 SatnGetpio=.E0,zr=.E0,era=00 trigTTts,esln017eo014rmx1.% ============================== ============================= A:nrFMFN TetyOYUC A:nrFOYEI TetyOPLPRM A:ideOPLPRM TmdlFOYEI A:nrFICSR TetyONRQT A:xtONRQT TeiFICSR A:nrFICSR TetyONRQT

A:xtONRQT TeiFICSR A:xtOPLPRM TeiFOYEI A:nrFOYEI TetyOPLPRM A:ideOPLPRM TmdlFOYEI .osoieeeeee.. ..ltflnsdltdhr. A:xtOPLPRM TeiFOYEI A:ideOYUC TmdlFMFN A:nrFPLSR TetyOOYUF A:ideOOYUF TmdlFPLSR p:.00000000+1(d00IFWTd).00001807+2 p010000000E0a)9.%DFIH(d010001669E0 A:nrFICSR TetyONRQT p:.01951433+1(d100IFWTd).98067093+1 p030970335E0a)5.%DFIH(d059022945E0 A:xtONRQT TeiFICSR plsr:.01951433+1(d100IFWTd).98067093+1 oyuf030970335E0a)5.%DFIH(d059022945E0 A:xtOOYUF TeiFPLSR y:.77471519+3(d23IFWTd)020011650E1 2038477439E1a)9.%DFIH(d.91405200+2 A:xtOYUC TeiFMFN Fnlrsl2.77471519+3(d2308IFWTd)020011650E1 iaeuty:038477439E1a)9.43%DFIH(d.91405200+2 ============================== ============================= Fromthislisting,youcandeducethatDDandADderivativesstarttodivergebetweenthe"middle"pointofsubroutineP L S R OYUF andthefollowingcalltosubroutineI C S R ,andthefaultyderivativeisthatofvariablep .Theerrorinthedifferentiatedcodeis NRQT p probablynearby.Nowyoucanstarttoinspectthecodefromthatpoint,identifythebug,andmaybesendusabugreport! Itisclearwhatthe"e t y n r "and"e i "debugpointsofaprocedureare.The"m d l "debugpointisplacedautomaticallyby xt ide Tapenadedebugmode.Youcaneasilyfinditinthedifferentiatedcode.Formoreaccuracy,youcanadddebugpointsbyplacing $ D D B G H R directivesinyouroriginalsource.Forinstance: AEUEE F(pE.)p=s Ip.Q0pn CAEUEEC4c.q4 $DDBGHRPpe. x=Xc)Xp) d(p(p definesanewdebugpointnamed"C 4 P ".Thesecondargument(optional,defaultistrue)ofthedirectiveisatest(withnowhite spaceinit,please),anddebugpointisactiveonlywhenthetestistrue.Ifnotestisgiven,thedebugpointisalwaysactive.There canbeathirdargument(optional,defaultisfalse)that,whentrue,forcesthedebugpointtobeactiveevenifdebugoftheenclosing procedureisnotactive(seebelow). Bydefault,debuggingoccursonevery"active"variableateverydebugpointofeverysubroutinecalled.Youcancontroldebugging forthecompletecalltreebelowaparticularsubroutinecall,byplacingthe$ D D B G C L directiveinyouroriginalsource AEUAL beforethisparticularcall.Forinstance: CAEUALfle $DDBGCLas eiei+iPLPRMXYn) prm=prm*OYEI(,,s turnsdebuggingoffforthecalltreebelowthisfunctioncall.Thef l e a s canbereplacedbysomethingmoreelaborate(nowhite spaceinthetestplease!).Whendebuggingisturnedoffinsideaparticularsubtreeofthecalltree,notestismadeandnomessage isprintedforthissubcalltree.However,youhavethepossibilitytoforcedebugginginsidethissubcalltreebyaddingathird argumenttosome$ D D B G H R or$ D D B G C L inside.Thisthirdargumentmustbeatest.Whenitevaluatestotrue, AEUEE AEUAL debuggingistemporarilyturnedonagainforthedebugpointorinsidethecallinquestion.

CHECKTHEREVERSEDERIVATIVESUSINGTHEDOTPRODUCTTEST:
Thedotproducttestreliesonthefactthatforeveryprogram,procedure,orpartP ofaprocedure,foreveryvectorX inthespaceof d theinputsofP andforeveryvectorY inthespaceoftheoutputsofP b ,thefollowingscalar: Y * . J d b P . X (where*istranspositionandJ istheJacobian(derivativematrix)ofthefunctionimplementedbyP )canbecomputedfromleftto P rightorfromrighttoleft: ( b P d = Y * . ( P . X ) Y * . J) . X b J d Since( b P isinfactX * Y * . J) b ,theoutputofthereversedifferentiatedprogramP B _ appliedtoY ,andconversely( P . X ) b J d isinfactY ,theoutputofthetangentdifferentiatedprogramP D d _ appliedtoX ,wefindthatcorrecttangentandreverse d differentiatedcodesP D _ andP B _ mustbesuchthat: X * . X b d b d = Y * . Y orintermsofdotproduct (b.X)=(b.Y) XdYd Weassumethatthetangentdifferentiatedcodeiscorrect,forinstancebecauseitwasvalidatedbythedivideddifferencesmethod. WeproposeamethodtovalidatethereversedifferentiatedprogramP B _ bycheckingthatthisdotproductequalityholdsandifit doesn't,tolocatethepartofthereversedifferentiatedprogramwheretheequalitydoesn'thold. Supposeagainthatthe"central"partlookslike: ..waeeoeieoe.. .htvrcdsbfr.

CBgnigohrcdrodfeetae einnftepoeuetifrnit ALMFN(12..12.. CLYUCx,x,.,y,y,.) CEdohrcdrodfeetae nftepoeuetifrnit ..waeeoecmsatr.. .htvrcdoefe. Defineagaintwocopiesofyourcompletecode,thatwilldifferonlyinthis"central"part. Thefirstcopy,thatwewillcall" o e B " c d B 1 ,compilesintoanexecutablethatwewillcall" x B 1 .Insteadofcalling eeB" M F N ,itcallsitsreversedifferentiatedversionM F N _ ,callingafewdebuggingprimitivesbeforeandafter.Thecentral YUC YUCB partof" o e B " c d B 1 isthus: ..waeeoeieoe.. .htvrcdsbfr. NwiiilzhEESifrnitdvrals.. ontaieteRVREdfeetaeaibe,eg: (hssesNTasltleesroeu) titpiOboueyncsayfrdbg! 1.0 xb=0d Oi1n D=,s 2()=0d xbi.0 ND EDO 1.0 yb=1d Oi12 D=,0 2()=1d ybi.0 ND EDO .. . Nwpeaefrtets: orprohet ALDBGBDII(.,010.70 CLEU_W_NT1d5.d,08d) ALDBGBDCL(MFN'tu. CLEU_W_AL'YUC,.re) CBgnigoifrnitdpr einnfdfeetaeat ALMFN_(11,x,xb.,y,yb22,.. CLYUCBx,xb22,..11,y,yb.) CEdoifrnitdpr nfdfeetaeat Nwtriaetets: oemnthet ALDBGBDEI( CLEU_W_XT) ALDBGBDCNLD( CLEU_W_OCUE) ..waeeoecmsatr.. .htvrcdoefe. Thesecondcopy,thatwewillcall" o e B " c d B 2 ,compilesintoanexecutablethatwewillcall" x B 2 .Again,onlyits eeB" centralpartismodifiedintothefollowingcode: ..waeeoeieoe.. .htvrcdsbfr. NwiiilzhAGNifrnitdvrals.. ontaieteTNETdfeetaeaibe,eg: (hssesNTasltleesroeu) titpiOboueyncsayfrdbg! 1.0 xd=1d Oi1n D=,s 2()=1d xdi.0 ND EDO 1.0 yd=0d Oi12 D=,0 2()=0d ydi.0 ND EDO .. . Nwpeaefrtets: orprohet ALDBGFDII(.,010.70 CLEU_W_NT1d5.d,08d) ALDBGFDCL(MFN' CLEU_W_AL'YUC) CBgnigoifrnitdpr einnfdfeetaeat alMFN_(11,x,xd.,y,yd22,.. clYUCDx,xd22,..11,y,yd.) CEdoifrnitdpr nfdfeetaeat Nwtriaetets: oemnthet ALDBGFDEI( CLEU_W_XT) ALDBGFDCNLD( CLEU_W_OCUE) ..waeeoecmsatr.. .htvrcdoefe. Afewremarksandexplanations: Eachcopy" o e B " c d B 1 and" o e B " c d B 2 willbeexecutedindependently.Eachmustuseorreadfromthesamedatafilesas theoriginalcode,andshouldbehaveexactlythesameuntilreachingthecentralpart.Youdon'tneedtoduplicateevery sourcefiletodefinethesecopies" o e B " c d B 1 and" o e B " c d B 2 .Youprobablyneedonlytoduplicatethefilethatcontains the"central"part.Therestshouldbetakencareofbythecompilationandlinkcommands. Notethat" o e B " c d B 1 makesuseofthereversedifferentiatedcodeM F N _ andoftheassociateddifferentiatedvariables YUCB xb 1 etc.ThecodeforM F N _ mustbebuiltbyTapenadebyreversedifferentiationofthesourcecode,withtheonly YUCB additionalcommandlineoption e u A J dbgD. Conversely," o e B " c d B 2 makesuseofthetangentdifferentiatedcodeM F N _ andoftheassociateddifferentiated YUCD variablesx d 1 etc.ThecodeforM F N _ mustbebuiltbyTapenadebytangentdifferentiationofthesourcecode,withthe YUCD onlyadditionalcommandlineoption e u A J dbgD. Eachexecutable" x D 1 and" x D 2 mustbelinkedwiththedebugginglibraryprovidedinthe"FirstAidKit".This eeD" eeD" libraryconsistsoffiled b g D f e u A . anditsincludefiled b g D i c euA.n ThecallstoD B G B D I I ortoD B G F D I I musthavethesamearguments.Thefirstiscalled"almostzero" EU_W_NT EU_W_NT (seebelow).Thesecondisthe"threshold"(seebelow)errorpercentageabovewhichamessageisprinted.Thethirdisthe "seed"thatinfluencestherandomchoiceofvectorsX andY duringtesting. d b

Inthepreparedcodeabove,noticethatdebugstatementsdonotconcernindividualdifferentiatedvariables.Thisisbecause, unlikethedivideddifferencestest,thisdotproducttestisglobalandcannotbeappliedtoasinglevariable. Don'tforgettodeclarealldifferentiatedvariablesinthedeclarationpartofyourcodes" o e B " c d B 1 and" o e B " cdB2. When" x B 1 and" x B 2 areready,runthedivideddifferencestestbyjusttypingthecommandline: eeB" eeB" >eeBxB2 $xB1|eeB usingaUNIXpipe"|"(orequivalent).Whathappensbehindthesceneis:e e B callsM F N _ ,whichisconceptuallysplitinto xB1 YUCB asequenceofcodepiecesbetweensuccessivepoints.Atthebeginningofeach(reversedifferentiated)piece(actuallyit'swhatwe callitsdownstreamend,becausecontrolflowisreversed...),pseudorandomvalues(using"seed")areassignedtothecurrent reversedifferentiatedvariablesY .Attheendofthesamepiece(actuallyit'swhatwecallitsupstreamend,becausecontrolflowis b reversed...),thecurrentreversedifferentiatedvariablesX aremultipliedbydotproductwitharandomlyfilledX .Theresultissent b d intostdout,thereforeintothepipe. Inparallel,e e B readsfromstdin,thereforefromthepipe,andreversesthesequenceofincomingdotproducts.e e B then xB2 xB2 callsM F N _ ,whichissplitinawaythatmatchesthesplittingofM F N _ .Atthebeginningofeach(tangentdifferentiated) YUCD YUCB piece,i.e.theupstreamendofthispiece,thesamerandomvaluesareassignedtothecurrenttangentdifferentiatedvariablesX . d Attheendofthesamepiece,i.e.thedownstreamendofthispiece,thecurrenttangentdifferentiatedvariablesY aremultipliedby d dotproductwiththesamerandomlyfilledY .Thedotproductiscomparedwiththeonecomingfrome e B .Ifthetwodot b xB1 productsarealmostzero(i.e.absolutevaluebelowthegiven"almostzero"),nocomparisonisdone.Otherwise,ifthepercentageof differenceismorethatthe"threshold",amessageisprintedonthescreen.Whatyouobtainonthescreenlookslikewhatfollows. Inthisexample,weintroducedanerrorintothereversedifferentiatedcode,insidethereversederivativeofsomefunctioncalled "P L S R ": OYUF $xB1|eeB >eeBxB2 SatnDeteo014rmx1.%admic=.7+0 trigAJts,zr=.E0,era=00,rno_nr08E0 ============================== ============================= A:nrFMFN TetyOYUC A:nrFOYEI TetyOPLPRM A:ideOPLPRM TmdlFOYEI .osoieeeeee.. ..ltflnsdltdhr. A:xtOPLPRM TeiFOYEI A:ideOYUC TmdlFMFN A:nrFPLSR TetyOOYUF A:ideOOYUF TmdlFPLSR 4.%DFEEC!fd.16900001+2w:026800000E0 90IFRNE!w:015700000E0bd.28200001+2 A:nrFICSR TetyONRQT A:xtONRQT TeiFICSR A:xtOOYUF TeiFPLSR A:xtOYUC TeiFMFN EdoDetro()fudANN:tsigatreiaie! nfAJts.1errson.WRIGetnlesdrvtvs ============================== ============================= Fromthislisting,youcandeducethatthereisanerrorinthereversedifferentiationofthepieceofcodebetweenthe"middle"point ofsubroutineP L S R andthefollowingcalltosubroutineI C S R .Nowyoucanstarttoinspectthecodefromthatpoint, OYUF NRQT identifythebug,andmaybesendusabugreport! Asforthedivideddifferencestest,the"e t y n r "and"e i "debugpointsareclearenough,andthe"m d l "debugpointis xt ide placedarbitrarilybyTapenade.Formoreaccuracy,youcanadddebugpointsbyplacingthesame$ D D B G H R directives AEUEE inyouroriginalsource.Youcanalsoplace$ D D B G C L directivesinyouroriginalsourcetocontroldebuggingmorefinely AEUAL insidethecalltree.Noticehoweverthatthethirdargumentofthesedirectivesissimplyignoredbythedotproducttest.Inother words,ifdebuggingisturnedoffforagivensubcalltree,thereisnowaytoturnitonlocallyinsidethesubcalltree.Noticefinally thatitisdelicateanddangeroustotryandaddnewdebugpointsbyhandintothereversedifferentiatedcode.Ontheotherhand,it israthereasyandsafetoturnoff/onexistingdebugpointsinsidethereversedifferentiatedcode,withoutneedingtoredifferentiate withdifferent$ D D B G * * directives.ThismayprovehandyasrunningTapenadeonabigcodecanbelong.Todoso,you AEU** justneedtoeditthereversedifferentiatedcode.Inityouwillfindthedebugpointsbytheirnames. Toturnoff/ontestinginsidethecalltreebelowaparticularcalltoprocedureF O O ,findthiscalltoF O B O _ ,moveupthe debuggingifthenelsebeforeit,toreachthecallto CLEU_W_AL'O'boenepeso" ALDBGBDCL(FO,"olaxrsin) youmaychangethebooleansecondargumentatwill. Toturnoff/onadebugpointnamed"m p i t y o n ",findthetest: I"olaxrsinADEU_W_EE'yon')TE F(boenepeso".N.DBGBDHR(mpit)HN andchangethebooleanbeforeA D N atwill.Debugpointsnamed"e t y x t e o e a l n r ","e i ","b f r C l ",and"a t r a l f e C l "can notandmustnotbeturnedoffindividually.

PROBLEMATICINTERACTIONWITHINDEPENDENTANDDEPENDENTPARAMETERSETS:
ThisproblemwasidentifiedandstudiedwiththehelpofNicolasHuneeus,LSCEIPSL,UniversityofVersaillesSaintQuentin. TapenadesometimestakesthelibertytomodifytheIndependentandDependentparametersprovidedbytheenduser.The reasonsaredescribedhereand,althoughtheyarecomplex,webelievetheyarejustified.Thismodificationcanbedifferentin tangentandadjointmodes. Thismaycauseproblemsinthetangentvalidation("divideddifferences")becauseitaffectsthewayderivativesare initialized.

Thiswillcertainlycauseproblemsintheadjointvalidation("dotproduct")ifthetangentandadjointcodesarebuiltwith (in)dependentparamatersthatdon'tmatch Asaresult,thevalidationresultsmaydisplayunjustified"Differences",locatedattheentryorattheexitoftheheadprocedure. Thesignyoumustlookforistheoccurrenceofoneofthemessages: Cmadnuaibesxxhvodfeetalnlecnpp:rmvdfo omn:Iptvral()xxaenifrnibeifuneippeoerm idpnet needns Cmaduptvral()xxronlecddfeetalnpp:rmvdfo omn:Otuaibesxxaentifuneifrnibyippeoerm dpnet eedns Cmadnuaibesxxhvhieiaiemdfenpp:addt omn:Iptvral()xxaeterdrvtvoiidippdeo idpnet needns Cmaduptvral()xxaeterdrvtvoiidippdeodpnet omn:Otuaibesxxhvhieiaiemdfenpp:addteedns whensettingthemessagelevelhighenough(commandlineoption s l v l 2 ). mgee0 Therearetwowaystoturnaroundthisproblem: eitheraddthecommandlineoption i i t r a e f x n e f c andtheIndependentandDependentparameterswillnotbemodified, ormodifybyhandyourIndependentandDependentparametersets,accordingtothevariablesindicatedbytheabove messages.Thenthesetswillbecoherentagainbetweenthetangentandtheadjointcodes.

Problemswithwebbrowsers(Macintosh):
I'musingInternetExplorerforMacintosh,andTapenadecomplainsaboutsyntaxerrorsinmyFortranfiles.HoweverI'msurethe syntaxiscorrect,and/oritworksfinethroughthecutandpasteinterface. ThisbugwasfoundandfixedbyDavidPearson,UniversityofReadingU.K.ThereasonmightbethattheFortranfilesthatyouare uploadingtousaresentas"application/xmacbinary"insteadof"application/octetstream".Thiscanbefixedbymodifyingyour browser'spreferences: Open"Preferences",thenclickeitheron"FileHelpers"ifvisible,oron"ReceivingFiles/FileHelpers"submenu.Youshould getsomethinglikethis. Clickthe"Add..."button. Enteradescription,e.g."Fortranfiles",andanextension,e.g..for.f90. EnteraMIMEtypeapplication/octetstream. EnteraFiletype.Fourcharactersexactlyarerequiredhere.Typee.g.FORT. EnteraFilecreator(asfor"Filetype"above). Makesurethe"Plaintext"radiobuttonisselected. Tick"Useforincoming"and"Useforoutgoing"checkboxes. Thisshouldlooklikethis.Thenclick"OK".

OtherADtools:
WhataretheotherAutomaticDifferentiationtools?HowdotheycomparetoTapenade? ThisisonlyourpartialvisionoftheotherADtools.Maybeabettersourceisthewww.autodiff.org sitefortheAutomatic Differentiationcommunity,managedbyourcolleaguesinAachenandArgonne. ThereareADtoolsthatrelyonprogramoverloadingratherthanprogramtransformation.Ingeneralthismakesthetooleasierto implement.HoweversomeoverloadingbasedADtoolscanbecomeverysophisticatedandefficient,andrepresentafairbitofhard worktoo.OverloadingbasedADtoolsexistonlyfortargetlanguagesthatpermitsomeformofoverloading,e.g.C++andFortran95. OverloadingbasedADtoolsareparticularlyadaptedfordifferentiationsthataremostlylocaltoeachstatement,i.e.nofancycontrol flowreschedulingisallowed.Ontheotherhand,theselocaltransformationscanbeverycomplex,morethanwhattransformation basedADtoolsgenerallyprovide.Forinstance,overloadingbasedADtoolscangenerallycomputenotonlyfirst,butalsosecond, thirdderivativesandsoon,aswellasTaylorexpansionsorintervalarithmetic.AdolC,fromTUDresden,isanexcellentexample ofoverloadingbasedADtool.AlsoFADBAD/TADIFF ThereareADtoolsthattransformtheoriginalsourceintoadifferentiatedsource.Tapenadeisoneofthose.Thesetoolssharetheir generalarchitecture,withafrontendverymuchlikeacompiler,followedwithananalysiscomponent,adifferentiationcomponent, andfinallyabackendthatregeneratesthedifferentiatedsource.Theydifferinparticularinthelanguagethattheyrecognizeand differentiate,theADmodesthattheyprovide,andsomedifferencesinADstrategies,mostlyaboutthereversemode.Others transformationbasedADtoolsare: AdifordifferentiatesFortran77codesintangent(direct)mode.Adiforoncewasextendedtowardstheadjoint(reverse)mode (Adjfor),butwebelievethisknowhowhasnowbeenreinjectedintotheOpenADframework. AdiccanbeseenastheCequivalentofAdifor.Howeveritisbasedonacompletelydifferentarchitecture,fromtheOpenAD framework.Thisframework,verysimilartoTapenade'sarchitecture,claimsthatonlyfrontendandbackendshoulddepend ontheparticularlanguage,whereastheanalysisanddifferentiationpartshouldworkonalanguageindependentprogram representation.AdicdifferentiatesANSICprogramsintangentmode,withthepossibilitytoobtainsecondderivatives. OpenAD/FdifferentiatesFortrancodesintangentandadjointmodes.ItsstrategytorestoreintermediatevaluesinreverseAD isextremelyclosetoTapenade's.OpenAD/FismadeofAdiforandAdjforcomponentsintegratedintotheOpenAD framework. TAMC,anditscommercialoffspringTAFdifferentiateFortranfiles.TafalsodifferentiatesFortran95files,undercertain restrictions.TAFiscommercializedbytheFastOptcompanyinHamburg,Germany.TAFdifferentiatesintangentandreverse mode,withtherecomputeallapproachtorestoreintermediatevaluesinreverseAD.Checkpointingandanalgorithmtoavoid uselessrecomputations(ERA)areusedtoavoidexplosionofruntime.TAFalsoprovidesamodethatefficientlycomputes thesparsitypatternofJacobianmatrices,usingbitsets.

TAC++istheCversionofTAF.ItisalsodeveloppedbyFastOpt.Presently,TAC++onlyhandlesalargesubsetofC,anditis stillinitsdevelopmentphase,althoughmakingsignificantprogress.LikeTAF,itwillprovidetangentandadjointmodes,with thesamestrategies,e.g.therecomputeallapproachwithcheckpointingforthereversemode.

ThereareADtoolsthatdirectlyinterfacetoanexistingcompiler.Infact,theseareextensionstothecompilersothatdifferentiated codeisaddedatcompiletime.ForinstancetheNAGWareFortran95compilerhasADfacilitiesinside,thataretriggeredbyuser directivesintheFortransource.Itsofarprovidestangentmodedifferentiation. ThereareADtoolsthattargethigherlevellanguages,suchasMATLAB.WeknowofADiMat,MAD,andINTLAB.Evenwhenthey relyonoperatoroverloading,theymayembedafairbitofprogramanalysistoproduceefficientdifferentiatedcode.

Directoryseparatorlinux/windows:
Ineedtouse"\"asaseparatorinmypaths,andTapenadeusestheUNIX"/"separator. ThishappenswhenyourfilesarestoredinaWINDOWS(DOS)filesystemandyourefertothemfromalinuxsystemsuchas cygwin.Usethe" a s r i e e a a o "optionontheTapenadecommandlinetomodifythepathseparator.Forinstance: preflsprtr tpnd.preflsprtr"".. aeae..asrieeaao\. or tpnd.preflsprtr"".. aeae..asrieeaao/.

JAVAerrors:NullPointerException,ArrayOutOfBoundsException,etc:
WhatdoesitmeanwhenIget Sse:jv.agNlPitrxeto ytmaaln.uloneEcpin asamessage,orsimilarlyanA r y u O B u d E c p i n raOtfonsxeto ItjustmeansyouranintooneoftheremainingbugsinTapenade.Pleasesendusabugreport,andwewillfixit.

Differentiatingmultipletop/headprocedures,DifferentiatingLibraries:
Iwanttodifferentiatemanytop/headprocedures Iwanttodifferentiateatop/headprocedurefordifferentactivitycontexts IwanttoforcetheactivitycontextofagivenproceduretosomethinglargerthanTapenade'schoice Iwanttodifferentiateeveryentryofalibrary TherecentversionsofTapenadeallowyoutodifferentiateseveraltop/headproceduresatthesametime.Thisisconvenientfor differentiatinglibraries.Theadvantageisthefollowing:whentwoheadprocedurescallthesameinternalprocedure,theinternal procedureisdifferentiatedonlyonce,withanactivitycontextwhichistheunionofthecontextscomingfromeachheadprocedure. IntheolderversionsofTapenade,whereonlyoneheadprocedurecouldbedifferentiated,usershadtowriteadummytop procedurethatcalledeveryentryofthelibrary,arrangingforthedesiredactivitycontextforeachcall.Thiscouldprovetedious. Toavoidthistedioustask,thesyntaxofthecommandlinehaschangedasfollows(theoldsyntaxstillworks,though):The e d ha optionnowacceptsastringbetweendoublequotes,inwhichyoumayspecifyanynumberoftriplets: Subroutinename Dependentoutputs Independentinputs Forinstance tpndo.a.oof..ed"1y3/x)f(2b)(1a3" aeaefofbrftt..haf(1y)(22b4/a2a) willdifferentiateheadsubroutinef ,differentiatingoutputsy andy withrespecttox andatthesametimeheadsubroutinef 1 1 3 2 2 willbedifferentiatedfordependentoutputsb andb withrespecttoindependentinputsa ,a ,anda .Youmaysplitthesehead 2 4 1 2 3 subroutinesintoseveral e d h a options.Ifyouusethissyntax,theoptions u v r and a s otas v r becomeuseless.Thereisan alternativesyntaxforpeoplewhowanttosayittheotherwayaround,i.e.forinstance ed"1x)(1y) haf(2>y3" InFortran90,youmaywanttodesignateasubroutineoravariablenamefromagivenmodule.Youmaydosousinga% ora. character,equivalently.Forinstance: ed"5f(1y2v%4/x) ham%1y3m%2c)(2" designatestheprocedurenamedf inmodulem ,andinthelistofdependentoutputs,youcanseethecomponentc ofthe 1 5 4 globalvariablev frommodulem .Ofcourseweassumethatv haspublicaccessanditisofastructuredtypewithafieldnamed 2 2 2 c. 4 Finally,ithappensthatTapenadefindsthatsomedifferentiationheadprocedureactuallyhasadifferentiationcontextsmallerthan theonegivenbytheuser.Supposeforinstancethatyouspecify

ed"5f(1y2v)(2x) ham.1y3m.2/x4" andactuallynodependentoutputdependsonx ,andatthesametimey actuallydependsonnoindependentinput.Tapenade 4 3 will"simplify"thecommandlineoptions,neglectingvariablesx andy .Thisisokingeneral,butannoyingsometimes.Notice 4 3 thatamessageisdisplayedtoexplainthesituation,butthedifferentiatedsubroutinewilllackthedifferentiatedargumentsforx 4 andy .Thecommandlineoption i i t r a e 3 f x n e f c willforcethedifferentiatedargumentstobepresent.Hopefullytheyare useless,buttheywillbethere.ThisisusefulwhenfeedingprogramsproducedbyTapenadeintooptimizationplatformsthatrequire afixedinterface.

DownloadTapenade
WherecanIdownloadalocalversionofTapenade? YoumaydownloadthelatestversionofTapenadeonyourlocalsystem.Firstreadourdownloadingpolicy,andregister.Youwill getalinktoaREADME.htmlfile.

OldversionsofTapenade
HowcanIgetapreviousversionofTapenade? Well,toputitbluntly,youcan't! Sorrybutkeepingseveralversionsavailableisalotofwork.Moreimportantly,itcontradictsourbelief(hope,dream,?)that successiveversionsofTapenadeonlyaddnewfunctionalities,improveexistingones,andfixbugs.Inthevastmajorityofcases, thisisjustified. However,itdidhappeninthepastthatswitchingtoanewversionposedaproblemtoauser.Forinstancewhenweintroducedthe PSCNRL) U H O T O ( inversion3.5,ortheautomaticmodificationoftheuserprovidedindependentanddependentsetsinversion 2.2.Inthesecases,theproblemscouldbesolvedbyasmallchangeintheenduser'senvironment,oranewcommandlineoption. Wethinkitwasforthebetter. Soifyoufindyourselfinthissituation,weencourageyoutoanalyzetheproblemandtelluswhichisthedifferenceinthe differentiatedprogramthatposesaproblem.Asmallstandalonesourcefilewouldbegreat.Hopefullywecanfindawaysothat youcanusethenewTapenadeandbenefitfromnextimprovements.

Systemerrorsofthe"notfound"sort:
Somethingwentwrongafterinstallation:Igeta Ecpinihed"anaaln.olsDfonErrolvlDfeetao xetontrami"jv.agNCaseFudro:tpee/ifrnitr oramessage To:Preoon olasrntfud Checkyourenvironmentvariable(e.g.shellvariable)T P N D _ O E A E A E H M . Thisvariableisusedinfileb n t p n d (forthe"LINUX"case)andinfileb n t p n d . a (forthe"WINDOWS"case). i/aeae i/aeaebt AlsowhenswitchingtoanewversionofTapenade,don'tforgettoupdateyourT P N D _ O E A E A E H M variabletorefertothenew version,e.g. T P N D _ O E "install_dir"/ a e a e . AEAEHM= tpnd35 SeeREADME.htmlformoredetails.

Systemerrorsofthe"outofmemory"sort:
Igeta Ecpinihed"anaaln.uOMmrErr xetontrami"jv.agOtfeoyro TheTapenadeservercomplainsmysourcefileistoolarge! Youareprobablydifferentiatingalargeprogram! TheTapenadeserverwillrejectyourfilesifthetotalsizeoffilessubmittedisover300000characters,asmanyuserscanaccess theserversimultaneouslyandtoolargefilesmakeisswapandslowdown,orevencrash.Ifyourunintothislimit,youmust probablydownloadaTapenadeexecutableonyourlocalsystem.Itisfreeforresearchnoncommercialuse. Bytheway,itstillhappensthattheservercrashesbecauseoftoomanyusersdifferentiatingtoolargefiles.Inthiscasetheserver shouldrestartautomatically,sopleasewaitforamomentandretry. ForalocallyinstalledTapenade,ifthejavaheapoverflows(j v . a g O t f e o y r o ),increasetheheapsizeinthe aaln.uOMmrErr launchingscriptfileb n t p n d (forthe"LINUX"case)orinfileb n t p n d . a (forthe"WINDOWS"case). i/aeae i/aeaebt SetH A _ I E x 0 8 orevenH A _ I E x 0 6 EPSZ=m24m EPSZ=m49m

Sendingremarks,questions,andbugreports:
WhereshouldIsubmitquestionsaboutTapenade? HowdoIsendbugreportstotheTapenadedevelopmentteam? Firstofall,thisisaverygoodidea.Thanks! BugreportsshouldbesenttothedevelopersofTapenadeatthisemailaddress:tapenade@listssop.inria.fr

Ontheotherhand,questionsandremarksshouldratherbesenttothetapenadeusersmailinglist.Youwillhavetosubscribe first,thenyouwillbeabletosubmityourquestionsandremarksbymailingtotapenadeusers@listssop.inria.fr.Maybesome otheruserofTapenadewillanswerevenbeforewedo!Naturally,frequentlyaskedquestionsshouldeventuallybeincludedhere. Ifyoureallydon'twanttosubscribe,thenyoumayjustsendanemailtothedevelopersofTapenade Morespecificallyaboutbugs,therearesomedatathatweneedinordertotrackthebugefficiently.Iftheerroryougotcontainedthe Javaexecutionstack,pleaseforwardittous.Pleasesendustheexactdifferentiationcommandthatyoutypedwithallits commandlinearguments,andthelogtextthatTapenadeproducedbeforeitcrashes. Ifyoucansendus(areducedversionof)thefilethatcausesthebug,thatwouldbejustgreat! IfyouwereusingtheonlineTapenade,thenyougotan"Error"webpage.Attheendisausefullink(forusdeveloppers):"I f eeyhnemorc,ispoalnitrau.Ihtcs,peseduhs vrtigsescrettirbbyanenlbgntaaelaesnsti bgrpr,wtotmdfigtecnetftemsaeewlxmntpopl. u e o t i h u o i y n h o t n s o h e s g . W i l e a i e i r m t y ". Pleasesendusthisbugreportusingthebutton"s n s e d u ".Itwillsendusthesessionreferencesothatwecanfinditstrackinour verylargelocallog.Mostimportantforthatarethesessionnumberandthedate/timeofthesession,forexample: Session:ac3eQiTVfzwb,Time:WedMar2907:59:01MEST2006

JAVAVersionrequired:
IinstalledthenewversionofTapenadeandJavanowcomplains. WhatminimalversionofJavajdkisrequired? AlocallyinstalledversionofTapenaderequiresareasonablyrecentversionofJava.SeetheinstallationREADMEfordetails.For instance,asofJan2008,itrequiresJavajdk1.6.

Wrongexternal's:
Mydifferentiatedprogramdoesnotcompilebecauseitcontainsa etraEL xenlRA declaration. Tapenadedoesn'twellrecognizetheR A Fortranintrinsic,andthereforeconsidersitasanexternal.Thismayalsohappenfora EL coupleothersintrinsics.Thisbughasstillnotbeenfixed.Theworkaroundissimple:justdeletethefaultydeclaration.

Controllingcheckpointinginthereversemode:
Myreversedifferentiatedprogramistooslow,andIthinkitcomesfromcheckpointing. HowcanItellTapenadetorefrainfromcheckpointingeachandeveryprocedurecall?HowcanItellTapenadetouse checkpointingonagivenpieceofcode? Bydefault,Tapenadewillapplycheckpointingtoeachandeveryprocedurecall,i.e.subroutineorfunctioncall.Therearevarious directivesthatlettheusercontrolcheckpointing.Whenusedwisely,thesetechniquescanvastlyimprovetheperformanceofthe adjointdifferentiatedcodebymodifyingthetradeoffbetweenmemoryuseandextrarecomputing. Therearethreewaystospecifyplaces(i.e.procedurecalls)wherecheckpointingmustnotbedone. Youcanplaceadirectiveinthesourcejustbeforeaparticularprocedurecall.Forexample: CAOHCPIT $DNCEKON w=tt(,y) oox,z tellsTapenadenottoplaceacheckpointaroundthisparticularcalltofunctiont t . oo Youcanplaceadirectiveinthesourcejustbeforethedefinitionofaprocedure.Forexample: CAOHCPIT $DNCEKON uruiett(,) sbotniiab tellsTapenadenottoplaceacheckpointaroundanycalltosubroutinet t . ii Youcanusethecommandlineargument o h c p i t n c e k o n thattakesthelistofproceduresthatmustneverbe checkpointed.Forexample: >tpndbohcpit"oott"pormf $aeaencekonttiirga. willcheckpointnocalltosubroutinest t andt t . oo ii ConverselyyoucanaskTapenadetoapplycheckpointingonanarbitrarypieceofcode. Youcanplacethepairsofdirectivesdescribedhere. Anotherway,withmoreeffortbutthatgivesyoumorecontrol,istomakethispieceofcodeatruesubroutine:sinceTapenade appliescheckpointingsystematicallytoeachsubroutinecall,youwillobtainthedesiredeffect. Finallytherearesituationswhereyoucanusespecialpredefinedcheckpointingstrategies: loopswithindependentiterations, binomialcheckpointing,whichapplieswelltolongchainsofsimilariterationssuchastimesteppingloops.

IndependentIterationsLoops($ADIILOOP):
Whatisthe$ADIILOOPdirective?WhencanIuseit? TheIILOOPdirective($ADIILOOP)tellsTapenadethatthefollowingloophas"IndependentIterations",i.e.roughlyspeakingcan beparallelized.Withthisinformation,Tapenadeisabletoproduceabettercodeinthereversemode. AloopcanbedeclaredIILOOPifithasnoloopcarried"true"or"flow"datadependence,i.e.ifnoloopiterationdependsona valuethatiscomputedbyanotheriteration.ThefollowingloopisanIILOOP: CAILO $DIOP D=,, Oi1N2 Ai.* ()=20z Ai1.* (+)=30z tp(() m1=BTi) tp(() m2=BKi) vQTtp*m2 v=SR(m1tp) Ci()v ()=Ci+v EDO ND because,asyoucancheck,thereisnoloopcarriedtruedependence.Thisimpliesthattheiterationordercanbefreelyshuffled.On theotherhand,thefollowingloopisnotanIILOOP: D=, Oi3N2 Ai() ()=Ai2 Bi(+) ()=Bi2 pordCi rd=po*() EDO ND foratleastthreereasons: Thereisaloopcarriedflowdependenceofdistance2onA Thereisaloopcarriedantidependenceofdistance2onB,anditwillbecomeaflowdependenceiftheiterationorderis reversed. Thereisaloopcarrieddependenceontheproductprod Finally,thefollowingisalsoanIILOOP: CAILO $DIOP D=,, Oi1N2 tp(() m1=BTi) tp(() m2=BKi) vQTtp*m2 v=SR(m1tp) sm=smv uu+v EDO ND Becausetheloopcarrieddependenceonsumindeedcomesfroma"sum"reduction,whichisOK. Ifyouaddthedirective$ADIILOOPasacommentjustbeforeanIILOOP,youwillseethatreversedifferentiationwillproducea differentcode,whichusessignificantlylessPUSH/POPmemoryspace("tape")thanwithoutthedirective. MoreexplanationsinthepaperonAdjoiningIndependentComputations

Userdefinedadditionalcheckpoints($ADCHECKPOINTSTARTand$ADCHECKPOINTEND):
HowcanIaddreversemodecheckpointsintomysourcecode? HowcanIforcecheckpointingofagivenpieceofcode? JustplaceTapenadedirectives$ADCHECKPOINTSTARTand$ADCHECKPOINTENDaroundthepieceofcodeyouwantto checkpoint.ThisallowsyoutodefineacheckpointaroundapieceofcodeP.Thismayproveusefulinverylongsubroutines,that consumealotofmemorytostoretheirintermediatevariablesduringtheforwardsweep. AddingauserdefinedcheckpointaroundapiecofcodePisapplicablewhenPcouldindeedbeextractedasaseparate procedure.InthepreviousversionsofTapenade,youhadtoactuallymakePaproceduretotriggercheckpointingonit.These directivesachievethesameeffectwithoutforcingyoutomakethiserrorpronetransformation. The$ADCHECKPOINTSTARTmustbeplacedimmediatelybeforethefirststatementthatisinP. The$ADCHECKPOINTENDmustbeplacedimmediatelybeforethefirststatementoutofP,i.e.thefirststatementthat followstheendofP. Makesurethese"firststatementin"and"firststatementout"areunambiguous.Allpredecessorsofthe"firststatementin"mustbe outofP.Allpredecessorsofthe"firststatementout"mustbeinP.Ifnecessary,introduceanoopstatement(e.g.C N I U )tolift OTNE ambiguities.TheremustbenoentryintoPthatdoesnotgothroughthe"firststatementin",andnoexitfromPthatdoesnotgo throughthe"firststatementout".Finally,"firststatementin"and"firststatementout"mustbeexecutablestatements,notdeclaration statementsnorpurecontrol(e.g.closingbraces,E S ,E D L E N ,...)Userdefinedcheckpointscanbenested.Thefollowingpieceof codegivesanexample: ixg.00hn f(.t1.)te

CAHCPITSAT $DCEKONTR * x=xy f(.t00 ixg..)x=x CAHCPITSAT $DCEKONTR * y=yxx * y=yy CAHCPITED $DCEKONN i() x=x+sny CAHCPITED $DCEKONN otne cniu . .. edf ni Itcontainstwonestedcheckpoints,bothlocatedinsidethe"true"branchofaconditional.Thegeneratedcodeshowstheeffectof theseadditionalcheckpoints.Intheforwardsweep: Ix.T00HN F(G.1.)TE CLUHEL() ALPSRA4x CLUHEL() ALPSRA4y x=xy * Ix.T.)x= F(G.00x y=yxx * y=yy * x=x+SNy I() .. . CLUHOTO1() ALPSCNRLB1 ES LE CLUHOTO1() ALPSCNRLB0 EDI NF andinthereversesweep: CLOCNRLBbac) ALPPOTO1(rnh IbacN.0HN F(rnh.E)TE .. . CLORA4y ALPPEL() CLORA4x ALPPEL() CLUHEL() ALPSRA4x x=xy * Ix.T.)TE F(G.00HN x= x CLUHOTO1() ALPSCNRLB1 ES LE CLUHOTO1() ALPSCNRLB0 EDI NF CLUHEL() ALPSRA4y y=yxx * y=yy * yb+CSy*b b=yO()x CLOKEL() ALLORA4y y=yxx * y**b b=2yy xb2xy b=x**b CLORA4y ALPPEL() CLOCNRLBbac) ALPPOTO1(rnh IbacN.0b=b F(rnh.E)xx CLORA4x ALPPEL() yb+xx b=y*b x*b b=yx EDI NF

BinomialCheckpointing($ADBINOMIALCKP):
Mytimesteppingloopfillsupthememoryspaceduringtheforwardsweep,andthismakesmyadjointcodecrash. Whatisthe$ADBINOMIALCKPdirective?WhencanIuseit? Binomialcheckpointingistheoptimalmemory/runtimetradeoffforiterativeloops,i.e.loopswherethecomputationofiterationn dependsontheresultsofpreviousiterations,andthereforethereisnohopeofmakingtheseiterationsindependentand/orrun theminparallel.Atypicalexampleisthetimesteppingloopinaninstationnarysimulation. AssumingthatthefinalnumberofiterationsNisknown,andassumingthateachiterationhasthesameruntimecost,thenA. Griewankhasgivenacheckpointingstrategyfortheiterativeloop,whichisoptimalbothintermsofmemoryusedtostorethe checkpoints,andintermsofruntimei.e.thenumberoftimesagiveniterationisrepeated.A.Griewankhasshownthatthissocalled "binomial"strategyhasacostinmemoryandacostinexecutiontimethatgrowlikelog(N).Moreprecisely Themaximumnumberofcheckpointsnapshots(i.e.bunchesofvaluesrequiredtorestartexecutionfromagiveniteration) presentinmemoryatthesametimegrowslikelog(N).Thereforethemaximummemoryrequiredislog(N)timesthesizeof onesnapshot. Themaximumnumberoftimesagiveniterationwithberepeatedlyexecutedduetothisstrategyalsogrowslikelog(N). ThereforethetimerequiredtocomputetheadjointoftheNiterationswillbe(log(N)+Constant)timeslongerthanthetimeto

runtheoriginalNiterations. Inpractice,onehaslessflexibilityonthenumberofsnapshotsthanontheruntime.Thereforeonepreferstofixthemaximum numberofsnapshotstoafixedq.Thenthecostsofthebinomialschemechangeto: Themaximummemoryrequiredisqtimesthesizeofonesnapshot.ItisindependentofN. ThetimerequiredtocomputetheadjointoftheNiterationswillgrowliketheqthrootofN. ThiswaspublishedbyA.GriewankandA.Walther,andfirstimplementedinAdolC.Tapenadeisnowabletoproduceanadjoint codethatappliesthisstrategyautomatically,triggeredbytheB N M A K directive($ D B N M A K ).Considerthis IOILCP AIOILCP example: CAIOILCPntt124sp $DBNMAKspsp+t1 dt=t1nt ospsp,sp + x=xy * y=yxx 3=10 v. oi39 d=,8 j=isp1 t/0 v()Bi 1=Aj*() v()Ai1+3 2=Bj/(+)v Ai*()snCj) ()=2Ai+i(() Bi1()Bi2(+)Ai1 ()=Bi*(+)Ai1*() Ai1()v*2 (+)=Ai+1v Bi1()v*3 ()=Bi+2v v3v1 3=v+2v nd edo ()=Ai*() Aj()Bj edo nd Theouterloopisagoodcandidateforbinomialcheckpointing.Itsiterationsaredefinitelynotindependent,sotheI O P ILO directiveisoutofquestion.Also,thenumberofiterations,althoughdynamic,isknownbeforetheloopstarts. Theinnerloopcouldalsobeappliedbinomialcheckpointing,butthiswouldbelessprofitableasitisasmallerpieceofcode.The threevaluesprovidedaftertheB N M A K directiveare IOILCP Thenumberofiterationsteps,oranexpressiontocomputeitjustbeforetheloopstarts.Foroptimalresults,alsocountthe iterationduringwhichloopexitoccurs.Thisiswhywewroten t t 1 2 s p s p + insteadofn t t 1 1 s p s p + .Ifthegivennumberof iterationsiswrong,thedifferentiatedcodeshouldstillwork,butwillbelessefficientintimeormemory. Themaximumnumberqofallowedsnapshots,chosenbytheenduser Theindexofthefirststepinthesequence Warning:asTapenadeparserisnotperfectinsidedirectives,becarefulnottoputanywhitespaceinsidethesethreeexpressions. Theseexpressionswillwecopiedverbatimintheadjointcode. Ifyoulookattheproducedreversecode,youwillfindthattheadjointcodeisinfactadriver,initializedwiththeparametersofthe B N M A K directive,andthatwilltriggerthecorrectsequenceofactionstocomputetheadjointoftheiterativeloop.These IOILCP actionsare: PushSnapshot:Pushthecurrentmemorystateonthestack. LookSnapshot:Restorethememorystatefromthestack,butleaveitonthestack. PopSnapshot:Restorethememorystatefromthestackandremoveitfromthestack Advance:runthenextiterationinitsoriginalform,withoutanyderivativecomputednorvaluepushed. FirstTurn:Computetheadjointofthelastiterationandoftherestofthecode. Turn:Computetheadjointofanordinaryiteration(notthelastone) Tobeexecuted,thedifferentiatedcodemustbelinkedwiththedefinitionsoftheexternalroutinesusedbythedriver: TVII( R_NT) TVNX_CIN) R_ETATO( TVRSZ( R_EIE) Theyareprovidedinfilet e v r e f r e e s . (alternativelyt e v r e c r e e s . andt e v r e h r e e s . )fromtheA F r t i K d D i s A d i .
DownloadFirstAidKit

LinearSolvers:
Myoriginalcodesolvesalinearsystemandthedifferentiatedcodeforthispartofthecodeisslow. Myoriginalcodeusesaniterativelinearsolverandthedifferentiatedcodeproducesderivativeswhoseaccuracygetspoorerand poorerasthedifferentiatedlinearsolverisused. LinearsolversareoftensophisticatedanddonotlendthemselvestoAutomaticDifferentiationefficiently.Evenifyouhaveaccess tothesourceofthelinearsolver,itmaybewisernottousetheADgeneratedcodeforthissolver,andwriteabetteronebyhand. Moreover,whenthelinearsolverisiterative,itoftenhappensthatthenumberofiterationsrequiredtofullyconvergethesolutiony ofA isnotenoughtoconvergeitsderivativey ory .SinceADusesthesamecontrolastheoriginalprogram,itstops y=b d b iterationwhentheoriginalprogramdoes.Thereforey ory maynothavereachedconvergencewheny d b has!

Ofcourseifyouuseasolverfromalibraryandyoudon'thaveaccesstothesource,thenyouareforcedtowritethedifferentiated solverbyhand.Youhavenochoice.Itwilljustcomfortyoutolearnthatthisisthebestthingtodoanyway!JustrefertotheFAQ itemaboutBlackboxprocedurestotellTapenadeaboutthislibrarycall(Wewouldadviseyoutoapplythedummyprocedure method),andcomebackhereforhintsabouthowtowritethishandcodeddifferentiatedlinearsolver. AssumethatthecodetobedifferentiatedhasfirstbuiltamatrixA andarighthandsidevectorb ,thatmaybebothactive.SizeofA isn n * andsizeofb isn .Assumethatsomesolveriscalled,e.g. clOV(,,,) alSLEAybn thatcomputesthey (ofsizen )suchthat:

y isalsoactive.Intangentmode,wecandifferentiatetheaboveequationbyhandandweobtain:

sothatwefindy is: d

Therefore,sinceTapenadeprobablybuiltatangentdifferentiatedcalllookinglike: clOV_(,dyy,,dn alSLEDAA,,dbb,) youcanwriteyourhandcodedS L E D O V _ alongthelinesof: sbotnOV_(,dyy,,dn uruieSLEDAA,,dbb,) ITGRn NEE RA(,)dnn ELAnn,A(,) RA()dn,bn,b() ELyn,y()()dn ITGRij NEE, RAHdn ELRS() clOV(,,,) alSLEAybn D=, Oi1n RS()=b() Hdidi D=, Oj1n RS()=RS()A(,)yj HdiHdidij*() EDO ND EDO ND clOV(,dRS,) alSLEAY,Hdn ED N

Inreversemode,it'snosurprisethingsgetalittleharder!TheinputsofthelinearsolvercallareA andb ,andtheoutputisy .By definitionoftheadjointvariablesA ,b ,andy ,wehavethedotproductsequality(dotproductnotedas(_|_))foranyAdandb : b b b d

Bewarethatthesedotproductsareelementwise:thedotproductofAdandAbisthesumofallAd(i,j)*Ab(i,j)foralliandj. Replacingy byitsabovedefinition,weget: d

foranyAdandb .Therefore: d

Thefirstequationgivesdirectlyb as: b

ThesecondequationgivesA butthisislessobvious:wemustgodowntotheindiceslevel: b

andwegetthevalueofeachelementofA .Finally,sinceTapenadeprobablybuiltareversedifferentiatedcalllookinglike: b clOV_(,byy,,bn alSLEBAA,,bbb,) youcanwriteyourhandcodedS L E B O V _ alongthelinesof: sbotnOV_(,byy,,bn uruieSLEBAA,,bbb,) ITGRn NEE RA(,)bnn ELAnn,A(,) RA()bn,bn,b() ELyn,y()()bn ITGRij NEE, RATnn,icb() ELA(,)nrbn D=, Oi1n D=, Oj1n A(,)=Aji Tij(,) EDO ND EDO ND clOV(Tnrbb) alSLEA,icb,y,n D=, Oj1n b()=b()+icb() bjbjnrbj EDO ND clOV(,,,) alSLEAybn D=, Oi1n D=, Oj1n A(,)=A(,)yj*nrbi bijbij()icb() EDO ND EDO ND ED N Tworemarks: first,noticethatthiscodeincrementsA andb insteadofjustsettingthem.Thisistheusualadjointbehaviorthatcomes b b fromthefactthatA andb mayverywellbeusedelsewhereintheprogramandthereforemayhaveotherinfluencesonthe dependentoutputs. Second,wehavebeenconsideringintheseexamplesthatbothA andb areactive.Ifeitheroneisnotactive,thenthecodes abovemaybesimplified.

Multiplesessionsinparallel:
Thedifferentiatedcodecontainspartsthatdonotcorrespondtoanythinginmyoriginalcode. MaybeyouhavetwowebTapenadesessionsactiveatthesametime.WhenusingtheTapenadeserverontheweb,therecanbe onlyonedifferentiationpageopenedatatime.Donottrytoopentwodifferentwebbrowserpages,andcalloneTapenadeineach, becauseyourTapenadesessionisunique:thereforethefilesyouuploadwouldallgointothesamedirectoryhere,causing strangebehavior.

Approximativesourcecodecorrespondence:
Youclaimthatthegraphicalinterfaceshowsthecorrespondencebetweensourceanddifferentiatedcode.Ican'tseeit! Thissourcecodecorrespondenceisimplementedthroughhtmllinksandanchors,soitsbehaviorisapproximate.Allitguarantees isthatthetargetofthelinkissomewherevisibleonthepage,butnotnecessarilyinfrontofthesource. Moreovertherearesituationswheresourcecodecorrespondenceislost,forexampleondeclarations.Sointhiscase,thebest targetwecanshowistheenclosinggeneratedsubroutine.

ToomanyWarningandErrormessages:
Help!Tapenadesendsmeplentyoferrormessages! YesTapenadesendsplentyoferrormessages!Onemaythinktherearetoomanyofthem.Thisisbecausewebelievethese messagescanbeveryhelpfulinthecasewhenyouobtainwrongdifferentiatedvalues.Sometimesoneofthesemessagesisan indicationofarealproblemfordifferentiation,soitcansaveyoualotoftimetolookatthem.

However,maybeyoushouldnottrytoohardtogetridofallthesemessages.Afterallmanyofthemarepurelywarnings.Veryoften, Tapenadewillproduceacorrectdifferentiatedcode,evenifithascomplained.Westronglyadviseyoutolookatthedetailed discussiononmessageswhichindicates,foreachmessage,whetheritisreallyimportant(ornot),andwhy,especiallyfromtheAD pointofview. Tapenadeletsyoudefineinthecommandlineanerrorlevelthreshold,belowwhicherrorsarenotshownanymore. Finally,tosummarize,pleasedonotoverlookTapenadeerrorsbecausetheypotentiallyindicateharddifferentiationproblems,and atthesametimedonotbetoodemandingonthesourceprogram:somewarningsaretheretostay!

IncludesandComments:
Whatamess!theniceincludefilesofmyoriginalprogramhaveallbeenexpandedinthedifferentiatedprogram! Whatamess!allmynicecommentsarounddeclarationshavebeendisplacedorlost! Thisstillhappensforsomeprograms.Sorryaboutthat. Foraverylongtime,youcanunderstandthishasnotbeenourtopmostpriority.Afterall,itwasmoreurgenttomakedifferentiated programstharrun,right? AlsothiswasnotsoeasybecausetheinternalrepresentationofTapenadeusedtodigesteverydeclarationintosymboltablesthat didnotrememberwheredeclarationscamefrom.Forthesamereason,wehadnowheretokeepcommentsattachedto declarations.Thishasbeenimprovedforversion3.Sohopefully,youshouldnoticeanimprovementinthatrespect.Sometimes whenthecodeusesthei p i i featureofFortran77,thei c u e mlct n l d "calls"can'tbeplacedback.Usei p i i o e m l c t n n to avoidthat. Despiteourefforts,itmayhappenthatsomecommentsarelost.Thisisabugandwewillbegratefulifyousignalittous.

Defaultsizesofbasicdatatypes:
Mysystem/compilerhasadefaultsizeof8bytesforR A ,andTapenadehasadefaultsizeof4bytes. EL Bydefault,Tapenadeassumesthattheactualsizesinmemoryofbasicdatatypesareasfollows: ITGR N E E :4bytes, R A :4bytes, EL D U L R C S O :8bytes, OBEPEIIN CMLX O P E :8bytes, D U L O P E :16bytes, OBECMLX LGCL O I A :4byte, CAATR H R C E :1byte, ThesevaluesareimportanttosolveE U V L N E Q I A E C s,ortomatchdifferentdeclarationsofthesameC M O block,ormore OMN importantlyforthePUSH/POPmechanismofthereversemode. Pleasecheckthatyoursystem/compilerusesthesamesizes.Incaseofdoubt,weprovideasmalltooltocheckthis.Tapenadelets youdeclarenewdefaultsizeswiththefollowingoptions: 24848 iiirr otheroptionsmightbecreatedasnecessary...

Parameterizedarraysizes:
MyoriginalfiledeclaresarrayA N 1 ,andthecodegeneratedbyTapenadedeclaresitasA 2 1 (+) (0) Thisistrue,andweunderstanditmakesthenewcodelesseasytomaintain.WhenN isaconstant,whosevalueisknown statically,e.g.2 0 0 ,TapenadeinternallysolvestheN 1 + intoitsvalue,here2 1 0 .Thisisonourlistofthingstoimprove.

Somecasesof"segmentationfault":
Mydifferentiatedprogramcrasheswitha"segmentationfault". Yes.Thismaystillhappen.Sigh... Herearetwo(notsofrequent)cases,thathappeninthereversemode: TherearenotasmanyPUSH'esandPOP'sbecausethecontrolflowisnotinversedwell.Thesymptomisthatthecrash occursinsideaPOP,andthestackisempty.Ifthisoccurstoyou,itisaTapenadebug.Pleasesignalittous. IfyourprogrampassesaFORTRANP R M T R A A E E (i.e.aconstant)asanargumenttoanotherprocedure,Tapenadetriesto P S / O thisconstant,whichresultsinasegmentationfaultattheP P UHPP O .Untilwefixthat,don'tpassaP R M T R A A E E asan argument.

Multidirectionalmodes(i.e.option u t ): mli
WhatdoIdowiththecodegeneratedwiththeoption u t ? mli WhatisthisD F S Z S i c I F I E . n includefileusedinthedifferentiatedcode? Whatisthisextradimensioninthedifferentiatedvariables? Whatistherelationbetweenvariablesn ,n d r andn d r m x d bis bisa? Theoption u t turnsonmultidirectionalmode,whichproducesacodethatwillcomputethederivativesatasinglepointinthe mli inputspacebutalongseveraldirectionsinthisinputspace.Thisdefinitionisforthetangentmode.

Actually,therecouldbea u t behaviorintheadjointmodetoo.Thiswouldcomputethegradientforseveralweightingsofthe mli outputs.AtpresentthismultidirectionaladjointmodeisnotavailableinTapenade,butthismightbethecaseinthefuture. Thewayitworksisbymodifyingthestructureofthederivativevariables.Instandard,singledirectionmode,aderivativevariable retainsthestructureofitsoriginalvariable(exceptforstructures,inwhichnonactivecomponentsdisappearfromthedifferentiated structure).Inmultidirectionalmode,derivativevariablesreceiveoneextradimension,thatcorrespondstothemultiplepossible directions(orweightings).Forinstanceascalarvariable el8x ra*x willbedifferentiatedifactiveinto: el8xdndrmx ra*x(bisa) andanarrayvariable el8a(0,) ra*a10N willbedifferentiatedifactiveinto: el8adndrmx10N ra*a(bisa,0,) Noticethattheextradimensionisaddedontheleft,i.e.inthedeepestplaceintheFORTRANdimensionsorder.Thisisusefulto isolatethearrayofderivativesofagivenelementofa d a .Alsonoticethatthe(static)sizeofthisnewdimensionisfixedto ndrmx b i s a ,whichmustbeatleastequaltothemaximumnumberofdirectionsyouplantoexploreatatime. Thedifferentiatedinstructionsaremodifiedaccordingly.Astatementsuchas (12*(12 ai,i)=2bi,i) willbedifferentiatedas On=,bis Dd1ndr dn,i,i)=2b(d12 a(d12*dn,i,i) ND EDO Noticethattheloopondifferentiationdirectionsisplacedatthedeepestlevelofthecode'sloopnesting.Alsotheseloopsiterate n d r times,thusallowingyoutoselectatruntimeanumberofdirectionslessorequalton d r m x d bis b i s a .n isjusttheindexthat spansonthen d r directions.Whenpossible,Tapenadetriestomergetheseloopsondimensions,toreduceloopoverhead. bis Noticefinallythatthereisnosuchlooparounddifferentiatedprocedurecalls,astheseloopswilldoneinsidethedifferentiated procedure. n d r isdynamic,soitisanextraparametertobepassedtothedifferentiatedprocedures.ontheotherhand,n d r m x bis b i s a is static:youmustprovideitatcompiletime.Tapenadestronglysuggeststhatyoudefineitinanincludefilenamed DFSZSic I F I E . n .Tapenadeaddstheincludecommandthatincludesthisfile.Allyouhavetodoistocreatethisfile.Ingeneral, youneedtodothatonlyonce.Forinstance,thisfilemaycontain neebisa itgrndrmx aaeendrmx=5) prmtr(bisa0 InFortran90,amoduleD F S Z S I F I E isusedinsteadofanincludefile.Youhavetowriteitsothatitdefinestheconstant ndrmx bisa.

GettingtheJacobianusingtheMultidirectionalmode:
HowcanIusetheMultidirectionalmodetocomputeaJacobian? Someeasytheorytobeginwith. ConsideraprocedureP X Y ,thatevaluatesamathfunctionF:X>Y.TangentmodeADofP (,) producesatangentcode P D X X , , D thatcomputesandreturns: _(,DYY) thesameY=F(X) andinadditionYD=J*XD,whereJistheJacobianofFatpointX. IfyourgoalistoobtaintheJacobianJ,andknowingthatthetangentcodegivesyouJ*XDforanyXD,youseethatthetangent codewillgiveyouanycolumnofJbyjustprovidingaXDwhichisthecorrespondingvectoroftheCartesianbasisofthespaceof X's.Inotherwords,ifyouprovideXDfullof0.0exceptfora1.0atranki,thenyouwillobtainaYD=J*XDthatistheithcolumnof J.Ifyoudothisrepeatedly,youwillgetallthecolumnsofJ,andthereforeJ. SoyouneednothingmorethantheplaintangentmodeofADtoobtainJ.However,youseethatthisamountstorunningP D _ severaltimesfordifferentX 'sbutforthesameX D .Someidenticalcomputationsaregoingtobeduplicated.Multidirectionaltangent modejustletsyoufactorouttheseidenticalcomputations:justaskTapenadetoproducethemultidirectionaltangentcodeofP , andthenrunthismultidirectionaltangentcodewithanX whichhasnowanextradimension(atthedeepestlevel)sothatit D containsallthevectorsoftheCartesianbasisoftheinputspace(thespaceofX's).

Technicaldetailsofthemultidirectionalmodeareexplainedhere. WealsohaveinthisFAQashortdiscussionaboutJacobiansandHessiansbyAD.Pleasetakealookatit,especiallyinitssecond partdevotedtotheJacobian:itcontainsfurtherdiscussionaboutJacobiansparsityandthemultidirectionaladjoint/reversemode. AfinalwarningabouttheJacobianbuiltbymultidirectionalmode:Inmultidirectionaltangent,allderivativevariablesreceive naturallyanextradimension,whichrangesoverthedifferentdirectionsofdifferentiation,i.e.overthedimensionsoftheinputspace. Forimplementationreasons,thisextradimensionmustbethe"deepest"dimensioninthenewdifferentiatedvariable.Thisis becausewemustbeabletoeasilyextractthederivativesofanyelementofanyarray.Thishasaconsequenceontheshapeofthe differentiatedoutput.SupposethattheoutputY isactuallyanarrayY 1 1 ) ( : 0 .Thederivativeresult,whichyoumaythinkofasthe Jacobian,willhaveanextradimensionatthedeepestlocation. ForinstanceinFortran,thiswillbeY ( : b , : 0 wheren d D1nd11) b isthenumberofdirections,whichis(forgettingaboutsparsity) thedimensionoftheinputspace.Ontheotherhand,theseconddimensionislinkedtothedimensionoftheoutputspace. YoumusttakethisintoaccoutwhengivingthisY toaprocedurethatasksfortheJacobian:theconventionsondimensionsmust D matchandiftheydon'tyoumayhavetobuildthetransposeofY ormodifytheusingprocedure. D

SecondDerivativesandHessians:
CanTapenadereturnsecondderivatives? CanTapenadecreateafiletocalculateHessianalso? Basicallyyes!Howeverthisrequiresabitmoreinteractionwiththeenduser.Alsotheresultingdifferentiatedprogramscouldbe improvedfurtherifTapenadehadaspecialmodeforsecondderivatives.Thereissomeongoingresearchonthesequestionsin ourresearchteam. TheideatoobtainsecondderivativesistoapplyAutomaticDifferentiationtwice.StartingfromaprocedureP infilep f . that computesy=f(x),afirstrunofTapenadee.g.intangentmodethroughthecommandline: >tpnddedPas""uvry. $aeaehavrxotas""pf returnsinfilep d f _ . aprocedureP D _ thatcomputesyd=f'(x).xd.NowanewrunofTapenadeontheresultingfilee.g.intangent modeagainthroughthecommandline: >tpnddedPDas""uvry"pdf $aeaeha_vrxotas"d_. returnsinfilep d d f _ _ . aprocedureP D D _ _ thatcomputesydd=f''(x).xd.xd0 SpecificallyifyoucallP D D _ _ withinputsx = . andx 0 1 0 d10 d = . inadditiontothecurrentx ,youobtaininoutputy d d thesecond derivativef''(x).

TANGENTONTANGENTAPPROACH:
Inmorerealisticcases,theinputxcanbemultidimensional.Theoutputycanbemultidimensionaltoo,butthisisnotsomuchofa problemhere.Ifyoulookforthesecondpartialderivatived2y/dxidxj,youmayapplythesametwosteps,differentiatingP with respecttoinputxifirst,thendifferentiatingtheresultingP D _ withrespecttoinputxj.Ifinrealityxiorxjareindeedelementsofa largervariable,e.g.elementsofanarrayx ,youmustdifferentiatewithrespecttothelargervariable.ThisisbecauseTapenade doesnotletyoudifferentiatewithrespecttoindividualarrayelements.ThenifyoucalltheresultingprocedureP D D _ _ with"correct" inputsx andx 0 d d ,youobtainthesecondderivative(s)thatyouwantinoutputy d d .Finallywhatarethe"correct"valuesto initializex andx 0 d d ?Inthespecialcaseofsimplescalars,itisjust1 0 . .Inthegeneralcaseofarrayelements,setarraysx and d x0 d to0 0 . ,exceptforelementsx ( ) d i andx 0 j whichmustbesetto1 0 d() .. Goingonestepfurther,onemaywantseveralsuchderivativesd2y/dxidxjforabunchofxiorxj.Thenaturalansweristhentouse tapenademultidirectionalmodetocomputealthesederivativesinjustonerun.Thisworksfinewhenmultidirectionalmodeis calledforonlyoneofthetwosteps.Inotherwords,whatyougettheniseitheraroworacolumnofthecompleteHessianmatrix.In thegeneralcasewhereonewantstocomputethecompleteHessian,oneneedstocallbothdifferentiationsinmultidirectional mode.Doingthis,youmightencouteracoupleofsimpleproblemsthatyouwillneedtofixbyhandlikeweusuallydo: ThefirstmultidirectionaldifferentiationcreatesaprogramthatincludesanewfileD F S Z S i c I F I E . n ,containinginformation aboutarraysizes.Precisely,theincludefilemustdeclaretheintegerconstantn d r m x b i s a whichisthemaximumnumberof differentiationdirectionsthatyouplantocomputeinasinglerun.n d r m x b i s a isusedinthedeclarationsofthesizeofthe differentiatedarrays.YoumustcreatethisfileD F S Z S i c I F I E . n beforestartingtheseconddifferentiationstep.Forinstance, thisfilemaycontain neebisa itgrndrmx aaeendrmx=5) prmtr(bisa0 if50isthemaxnumberofdifferentiationdirections.IfwhatyouwantistheHessian,thismaxnumberofdifferentiation directionsisthecumulatedsizesofalltheinputsx . Thesecondmultidirectionaldifferentiationrequiresanewmaximumsizevaluen d r m x ,whichisaprioridifferentfrom bisa0 ndrmx b i s a .FortheHessiancase,itisprobablyequalton d r m x b i s a .What'smore,Tapenadehasinlinedthe1stlevel includefile,sowhatyougetisastrangelookingpieceofdeclarations: .. .

ICUE'IFIE.n' NLDDFSZSic Citbisahudbhaiu. Hn:ndrmxsoletemxmm.. ITGRndrmx NEEbisa PRMTR(bisa=0 AAEEndrmx5) .. . WesuggestyoujustremovetheI C U E ' I F I E . n ' N L D D F S Z S i c line,andhandreplaceeachoccurrenceofn d r m x bisa0 byeithern d r m x b i s a oreven5 ! 0 AcompleteHessianisingeneralsymmetric.Tapenadewasnotabletotakeadvantageofthis.Youmaydosobyreplacing everysuccessivelinesoftheform .. . Dd1ndr On=,bis Dd=,bis On01ndr0 .. . by .. . Dd1ndr On=,bis Dd=dndr0 On0n,bis .. .

TANGENTONREVERSEAPPROACH:
Supposetheoutputy isscalar.ThenyouknowthatthereversemodeofADcanreturnyouwiththeJacobian,whichisasinglerow matrix,i.e.agradient,ataverylowcost.Thenaturalextensionofthisistodifferentiatethereversedifferentiatedprogram,thistime inmultidirectionaltangentmode.Hencethename"TangentonReverse".Intheory,thecostofcomputingthegradientisafixed smallmultiple(say7*)ofthecostoftheoriginalprogramP .Similarly,thecostofcomputingthetangentderivativeisalsoasmall multiple(say4*)ofthecostofP .Ifoneusesthemultidirectionaltangentmode,fornsimultaneousdirections,thecostofthemulti directionaltangentderivativeisproportionalton(say4*n).Tosummarize,thecostofcomputingthecompleteHessianthroughthe TangentonReverseapproachshouldberoughly 4n7P *** whereasthecostofcomputingthecompleteHessianthroughtheTangentonTangentapproachshouldberoughly 4n4nP **** ThereforeTangentonReverseisappealing,althoughinrealitythingsarenotsoclearcut. TangentonReverseraisesanumberofnewproblems,becauseoftheP S andP P UH O thatareinsertedbyreversedifferentiation. Toobtainancorrecttangentdifferentiation,onemusttellthesecondstepTapenadeaboutthebehavioroftheseP S andP P UH O addedbythefirststepTapenade.ThiscouldbeimprovedifTapenademadethetwostepsjointly,butthisisstillnotthecase... FirstthingistodeclaretheinputsandoutputsofP S andP P UH O .Inadditiontotheirargument,theyhaveanotherhiddenparameter, whichisthestack.Youmustdeclarethisinaspecialfile,usingthesyntaxforblackboxroutinesdescribedhere.Weprovidesuch afile,namedP S P P e e a L b U H O G n r l i ,intheA F r t i K d D i s A d i .Youmayneedtoextenditifitdoesn'tcoveraP S orP P UH O functionthatyourcodeuses.
DownloadFirstAidKit

YouwilltellTapenadetousethisfileP S P P e e a L b U H O G n r l i inthecommandline,usingthecommandlineoption ..xUHOGnrli. .etPSPPeeaLb.. ThesecondthingistolinkthefinalTangentonReversedifferentiatedcodewithP S P P i f o U H O D f . obtainedbycompiling PSPPiff UHODf.. PSPPeeaLb U H O G n r l i declarestheinputsandoutputsoftheP S andP P UH O procedures.Forexamplethelines sbotnora8 uruieppel: etra: xenl sae(aa, hp:prm1 omn/dtc/0*) cmoasak[,[ tp:mdfeTpNm(oiir(nCt8,ietra) ye(oiidyeaemdfesits)dnel, aryyemdfeTpNm(oiir(nCt8,ietra) raTp(oiidyeaemdfesits)dnel, dmoosdmoo(oe)nn()) iCln(iClnnn(,oe))) R:(,1 0) W:(,1 1) meanthatP P E L hasindeedtwoarguments.Oneistheformalparameter,whichisaR A * ,andtheotheristhestack, ORA8 EL8 whosetypeisanarrayofR A * values.Theformalparameterisnotreadbutoverwritten,thestackisreadandoverwritten EL8 i.e.modified.YoumaywanttoaddnewsuchdeclarationsifyourcodeusesotherP S andP P UH O 's,suchasP S R A 4 UHEL. PSPPeeaLb U H O G n r l i alsodeclaresthedependenciesthroughtheP S andP P UH O procedures.Forexamplethelines

concerningP P E L ORA8 dp:(,1 es0, ,1 0) meanthatP P E L outputsavalueinitsfirstargumentthatonlydependsonthegivenvalueofthestack,andalsosetsa ORA8 newvalueforthestackthatonlydependsonthegivenvalueofthestack.Youmayneedtoaddnewsuchdeclarationsifyour codeusesotherP S andP P UH O 's,suchasP S R A 4 UHEL. PSPPiff U H O D f . definesthedifferentiatedproceduresfortheP S andP P UH O procedures,inplaintangentmodeaswellas inmultidirectionaltangentmode.ThesedifferentiatedroutinesexistonlyfortheP S andP P UH O ofactivevariables.In particulartherearenosuchdifferentiatedroutinesforP S I T G R U H N E E norP P N E E .Thetrickistopushandpopthe OITGR originalvariableandthedifferentiatedvariable,andsinceweuseastack,theorderoftheP S 'esandP P UH O 'sisreversed. YoumayneedtoaddnewsuchdefinitionsifyourcodeusesotherP S andP P UH O 's,suchasP S R A 4 UHEL. Thiscompleteprocessisstillverymuchexperimental,sobeawarethaterrorsmightoccur.Don'thesitatetocontactusthroughthe Tapenademailinglistincaseoftrouble(subscribe tothe"tapenadeusers"mailinglist).

Declarationsimproperlydifferentiated:
Thearraydimensionsarelostinthedifferentiateddeclaration. Commoncontainingarrays: Ifanarrayhasitsboundsdeclaredinthecommonstatement,thedifferentiateddeclarationsmaylostthesebounds.Ifitisthecase, youshoulddeclaretheseboundsinthetypedeclarationstatementinsteadofthecommonstatement. Thepointer,target,andallocatablestatements: alctbe:(,) loaal:a:: pitr:o one:sn tre:a(0 agt:,y1) AsdescribedIn"Fortran95/2003explained","Webelievethatitismuchclearertospecifytheseattributesonthetypedeclaration statements,andthereforedonotusetheseforms." Inparticular,ifthesestatementsuseimplicittyping,somedeclarationsmaybedifferentiatedimproperly. Untilwefixthesebugs,youshouldcompletlydeclarethearraysorallocatablearraysinthetypingdeclaration.

Errorsoncomplicatedstatements:
Igotthemessage( C 3 o l o i d t p f . . l a e c e k T4)Cudntfnyeo.,peshc Fordifferentiationpurposes,Tapenadesometimesneedstosplitalongandcomplexexpressionwithsubroutinecalls,introducing aanumberofnewintermediatevariables.Usuallythisworksfine,butinsomesituations,Tapenadecannotguessthetypeand sizeofthenewintermediatevariable.IfTapenadefailstofindthecorrecttypeREALofthetemporaryvariable,putsINTEGER instead,thisloosesactivity.Weputthewarningmessage( C 3 tosignaltheproblem.Theworkaroundistodothesplitting T4) beforehand,onthesourceprogram,andtodeclaretheintermediatevariableswiththecorrecttype.

Copiedproceduresinthedifferentiatedcode:
TapenadegeneratesalargebodyofwhatlookslikesuperfluouscodethatsimplyduplicatesthecodegoingintoTapenade. Therearenearlycopiesoftheoriginalproceduresinthedifferentiatedcode.Theyappearingeneralwithanewnameobtainedby appending"_C".ThishappensinFORTRAN90whenanondifferentiatedsubroutineUSE'sadifferentiatedMODULE. Supposetheoriginalcodeis: MDL OUEM .. . EDMDL NOUEM SBOTN URUIES UEM S .. . EDSBOTN NURUIES Thiscodeisdifferentiatedas: MDL_ OUEMB .. . EDMDL_ NOUEMB SBOTN_ URUIESB UEMB S_ .. . EDSBOTN_ NURUIESB SBOTN_B URUIESC UEMB S_ .. . EDSBOTN_B NURUIESC

Subroutine"S_CB"isacopyofsubroutine"S",withaUSEofthedifferentiatedmodule"M_B"insteadoftheoriginalmodule"M". Sothisdoesnotcreateaclashforuserswhousetheoriginalcodetogetherwiththedifferentiatedcode:wegenerateadifferent procedurefor"S". Sowechosetomakeadifferencebetween "S",theoriginal"S"thatworkswiththeoriginal"M",and "S_CB",thecopyof"S"forthereversemode,whichworkswiththesamemodule"M_B"asthedifferentiated"S_B".(the_CB suffixisbuiltbyadding_Cinfrontofthesuffixfordifferentiatedvariables.) Thiscausescodealmostduplicationbutfewercompilationconflicts. However,wearenotverysatisfiedwiththisstrategyandwemaymodifyitsomeday,fromourdiscussionswithendusers...

Buildingthecontextforcallingthedifferentiatedcode:
Ihavedifferentiatedmyrootprocedure.HowshouldIorganizetheprogramthatcallsit? I'mlostwiththesealltheseF O O andF O D O _ andF O C proceduresandmodules.HowcanIincludethemintomybig O_D application? Mydifferentiatedprogramdoesasegmentationfaultbecausesomedifferentiatedglobalsarenotallocated! Thefollowingisyetsubjecttodiscussionsandmodifications,whichmeansitmaychangesorry! Tapenadedifferentiatesthecalltreethatisundertherootprocedurethatyouhavespecified.Theresultisasetofdifferentiated modulesand/ordifferentiatedstandaloneprocedures,thatdonotexecutealonebutmustbeused/calledfromacertaincontext.Itis uptoyoutowritethiscallingcontext,andTapenadecanhelpyouonlytoacertainpoint.Thefollowingdiscussionfocusesonthe tangentmode.Beawarethatthingscanbeslightlyharderintheadjointmode. Youmayuseasabasisacallingcontextdesignedfortheoriginal,nondifferentiatedrootprocedure.Noticethatevenifyou providedTapenadewiththecompletecallingcontextcodeaswell,itwilldifferentiateonlythecallgraphbelowtherootprocedure. Thecallingcontextwillnotbemodifiednoradaptedtothedifferentiatedroot.Soevenifthereisacallingcontextavailableforthe root,youwillhavetomodifyitbyhandsothatitcancallthedifferentiatedroot. Therearemanypossibilecases,sowe'dratheruseanexample.We'lltrytoextractgeneralruleslater.Here'sasmallexample sourceprogram.ThedifferentiationrootprocedureisR O .Thefirstcolumncontainsthecodethatisnotinthecallgraphbelow OT RO. OT

PORMMI RGAAN UEMA SM UEMB SM RA,DMNIN:,ALCTBE: ELIESO()LOAAL:x ITGR: NEE:i RA:ca2 EL:v() CMOc1v OMN/c/ca ETRAEFA XENLGTLG CLEFA(lg ALGTLGfa) ALCT((0) LOAEx3) CLNT( ALIIA) D=,0 Oi12 a()=100i ai0./ xi.* ()=33i EDD NO ca1. v()=18 ca2. v()=81 CLOTx ALRO() PIT,'z'z RN*z=,z DALCT(a ELOAEa) DALCT() ELOAEx EDPORMMI NRGAAN

MDLM OUEMA RA,DMNIN2)LOAAL:a ELIESO(0,ALCTBE:a CNAN OTIS SBOTNNT( URUIEIIA) ALCT(a LOAEa) EDSBOTNNT NURUIEIIA EDMDLM NOUEMA MDLM OUEMB ITGR:lg NEE:fa RA:z EL:z EDMDLM NOUEMB SBOTNOTx URUIERO() UEMA SM UEMB SM RA,DMNIN*:x ELIESO(): RA:v EL: RA:c1v EL:v,c2 CMOc1v,c2 OMN/c/c1v zv*v z=c1c2 v=SMa) U(a IfaG.2z=z* F(lg.T)zzv zz+SMx z=zU() EDSBOTNOT NURUIERO

Wewilldifferentiateintangentmode,withheadR O .Wewillnotspecifythenameoftheoutputfile,sothateverymoduleor OT isolatedprocedureofthedifferentiatedcodewillbeputintoaseparatefile.Thedifferentiatedcallgraphisreturnedinfiles r o _ . 9 ,m a d f 0 o t d f 0 m _ . 9 ,andm b d f 0 m_.9. r o _ . 9 definesprocedureR O _ otdf0 OTD !iffrontnetmd: Df.ootiagnoe !vrainfueueut:z aitososflrslsz !wtepcovrigipt: ihrsettaynnus !avv *ac1c2x !Rttsoifvrals Wsaufdfaibe: !zot*aiv:nc2i:n z:ua:nc1iv:nxi SBOTNOTDxd URUIERO_(,x) UEMBD SM_ UEMAD SM_ IPIIOE MLCTNN RA,DMNIN*:x ELIESO(): m a d f 0 m _ . 9 definemodulesM A D M _ m_.9,mbdf0 M_,MBD

MDLM_ OUEMAD IPIIOE MLCTNN RA,DMNIN:,ALCTBE:a ELIESO()LOAAL:a RA,DMNIN:,ALCTBE:a ELIESO()LOAAL:ad

RA,DMNIN*:x ELIESO():d RA:v EL: RA:v EL:d RA:c1v EL:v,c2 RA:c1,c2 EL:vdvd CMOc1v,c2 OMN/c/c1v ITISCSM NRNIU CMOc1dvdvd OMN/c_/c1,c2 zd=c1*vv*vd zvdc2+c1c2 zv*v z=c1c2 vU(a) d=SMad v=SMa) U(a IfaG.2HN F(lg.T)TE zd=zdv+z*d zz*zv zzv z=z* EDI NF zd=zd+SMx) zzU(d zz+SMx z=zU() EDSBOTNOTD NURUIERO_

CNAN OTIS SBOTNNT( URUIEIIA) IPIIOE MLCTNN ALCT(a(0) LOAEad2) ALCT(a2) LOAEa(0) EDSBOTNNT NURUIEIIA EDMDLM_ NOUEMAD MDLM_ OUEMBD IPIIOE MLCTNN ITGR:lg NEE:fa RA:z EL:z RA:zd EL:z EDMDLM_ NOUEMBD

ThecallingcontextoftheoriginalR O isnotsufficienttocallR O _ andmustbeadapted.Herethiscontextconsistsofprogram OT OTD M I ,buttherecanbemoremodulesandproceduresinit.ThenewprogramM I _ D AN A N C producedbyTapenadeinfile mi_df0 a n c . 9 isapossiblestartingpoint,butitmustbemodifiedbyhand.Oryoucanwriteitfromscratch.Forinstance:

PORMMI_D RGAANC UEMBD SM_ UEMAD SM_ RA,DMNIN:,ALCTBE: ELIESO()LOAAL:x RA,DMNIN:,ALCTBE:d ELIESO()LOAAL:x ITGR: NEE:i RA:ca() EL:vd2 CMOc1dvd OMN/c_/ca RA:ca2 EL:v() CMOc1v OMN/c/ca ETRAEFA XENLGTLG CLEFA(lg ALGTLGfa) ALCT(d3) LOAEx(0) ALCT((0) LOAEx3) CLNT( ALIIA) iiilza,dca,z ntaieadx,vdzd D=,0 Oi12 a()=100i ai0./ xi.* ()=33i EDD NO ca1. v()=18 ca2. v()=81 CLOTDxd ALRO_(,x) PIT,'z'zadzd'z RN*z=,z,'nz=,zd DALCT(a) ELOAEad DALCT(a ELOAEa) DALCT(d ELOAEx) DALCT() ELOAEx EDPORMMI_D NRGAANC

Placesofinterestareinblue.Somecomments: thecontextmustcallthedifferentiatedroot,hencethecalltoR O _ . OTD thecontextmustprovideR O _ withallthederivativesofitsarguments,whetherformalparametersorglobals.Hencex , OTD d butalsoa d c 1 d , z d a,/c_/z. Thesenewx , a d c 1 d , z d d a , / c _ / z mustbedeclared,allocatediftheiroriginalvariableis,andinitializedwiththe derivativesyouwant. AfterreturnfromR O _ ,thederivativesinthesevariablescanbeused,andthevariablesmustbedeallocatediftheir OTD originalvariableis. OnemustmakesurethatthesevariablesareeffectivelythesameinthecontextandinR O _ ,whichmeansthatwhenthey OTD areglobalsfrommodules,thesamemodulesmustbeusedonbothsides.HencetheU E M A D S M _ andU E M B D S M _ .This alsoensuresthatthe"good"I I _ ( iscalled. NTA) AlltheseoperationsneednotbedoneonlyinM I _ D A N C callsutilityprocedures,youmaychoosetomodifythese A N C .IfM I _ D proceduresinstead.Makesuretokeepallthesemodifiedproceduresseparatedfromtheiroriginaldefinition,andlinkwiththe originalcodeonlywhenitwasnotmodified.ThisalsoappliestoM I _ D A N C itself. Tomakesureyoudon'tmixtheoriginalcontextproceduresandthenewcontextproceduresadaptedtoR O _ ,whynot OTD givethemthesameprocedurename?Thisway,thecompilershouldcomplainifyoumixthem.HereM I _ D A N C couldjust remainM I .ThesamecouldbesaidformodulenamesM A D AN M _ andM B D M _ butthat'sanotherstoryascurrentlyTapenade systematicallyappendsa_ tothem. D

TapenadeandWindows:

WhatshouldIdotouseTapenadeonWindows? BeforeinstallingTapenade,youmustcheckthatanuptodateJavaRuntimeEnvironmentisinstalled.Ifthereisnone,youcan downloadandinstallthelatestJavaRuntimeEnvironment(JavaSE6orJDK6)fromthejava.sun.comsite.Tapenadewillnotrun withanolderJavaRuntimeEnvironment. Then firstreadourdownloadingpolicy,andregistereitherforacademicusageorforevaluationusage.Youwillgetalinktoa README.htmlfile. LookingattheREADME.html,goingtothesectionaboutWindows,downloadthetapenadeziparchiveintoyourchosen installationdirectory"install_dir" Gotoyourchoseninstallationdirectory"install_dir",anduncompressTAPENADE. Copy"install_dir"\bin\tapenade.batandupdatetheinstallationparameters:JAVA_HOME,TAPENADE_HOME,and BROWSER. Copythefile"tapenade.bat"inanewfile"mytapenade.bat"inthesubdirectory"bin"locatedwhereyouhaveinstalledthetapenade files.Inthefile"mytapenade.bat"updatethevariableJAVA_HOMEaccordingyourjavainstallation,updatethevariable TAPENADE_HOME:itiswhereyouhaveinstalledthetapenadefiles,updatethevariableBROWSER,thedefaultisinterner explorer.

Tapenade,Windowsandcygwinproblem:
Whenparsingafortranfilewithtapenade,Igettheseerrormessages: Yuhvutpecpefcgi1dloorsse. oaemliloisoywn.lnyuytm Sacoywn.lsn. erhfrcgi1dluig.. addltluhotrcneso.Temseetvrin*hud neeealbttemseetvrinhotrcnesosol* rsdnx\ywnbnhrxstedieohcoae eiei:cgi\i,wee''ihrvnwihyuhv isaldtecgiitiuin ntlehywndsrbto. Sse:Prigerr ytmasnro Ifitisthecase,justremovetheoldcygwin1.dllthatisinthedirectorytapenade/bin/windows.

TapenadewithCprogramsonWindows,problemwithcpp:
Preprocessorerror BeforeparsingaCprogram,TapenadecallscpptopreprocesstheCprogram.Onaunixmachine,cppisfoundinthestandard PATH.Onawindowsmachine,youcanspecifythecppcommandusingtheoptioncpp"\...\...\cpp_command".Ifyouhavecygwin, youshouldfindthecorrectpath. Ifyoudon'tfindcpp,andifyourCprogramdoesn'tcontain#define,#include,youcancalltapenadecpp"nocpp"program.c.Ifyour programcontains#includetheywillbeignore.

Validationofthedifferentiatedprograms(OLDMETHOD)
Help,thedifferentiatedprogramcrashes! HowcanIbesurethatthedifferentiatedprogramreallycomputeswhatIwant? HowcanIcheckthederivativesandfindtheplacewherethedifferentiatedprogramgoeswrong? Thissectioniskepthereonlyfortherecord.ItdescribestheoldmethodthatTapenadeprovidedbefore2011.Wenow haveasimpler,morepowerfulmethod,describedhere. Inanycase,toeaseourvalidationmechanism,westronglyadviseyoutofollowafewpreparationsteps,beforeyouevenstartto differentiateanything.Youwillunderstandthereasonforthesestepsifyougothroughthevalidationprocessthatfollows.

CHECKTHETANGENTDERIVATIVESWITHRESPECTTODIVIDEDDIFFERENCES:
Inthefollowing,wordsbetweendoublequotesanddrawninthiscolor(suchas"SHAPE")areindeedtemplates,thatmustbe instantiatedbyactualstringsdependingonthecontext.Squarebracketsrepresentadditionalarguments,andthestarrepresents repetition. Intheexamplepiecesofprograms,actualpiecesofcodearewritteninthissortofdarkred,shortstringsindoublequotesare "patterns"asabove,and "nirieeweobeqoenrwnteptenclr eteelnsbtendulutsaddanihatroo" representapieceofcodethatyoumustwriteatthatplace. Thisisthestandardwaytotestthetangentderivatives.RunyouroriginalsubroutinePwithinputX ,thenwithX * d + X ,then compute:

YoushouldgetnearlythesameresultaswhenrunningthetangentdifferentiatedsubroutineP d X X , ) _ ( , d Y Differencesshouldbe onlyonafewlastdecimals. Ifthisisnotthecase,hereisawaytofindwhereitgoeswrong.Usethegivenlibraryd T s . fromtheADFirstAidkit. detf d T s . definessubroutineswhosenameisoftheform detf

DTP"SAE((TIGaibenm" D"YE"HP""SRN)vralae, (OTAYErgnlvral" "FRRNTP)oiiaaibe, (AEFRRNTP)dfeetaeaibe "SMOTAYEifrnitdvral" ,(NEE)arylnt") ["ITGRraegh] Theseprocedurescomparedynamically(atruntime)thederivativesobtainedbydivideddifferences(seebelowhowtheyarebuilt) withthederivativesobtainednormallybythetangentdifferentiatedprogram.Ifthereisadifference,theprocedureprintsamessage. ThistestisdoneeachtimeoneoftheseD * D proceduresisencountered,onthevariablepassedasargument. Syntaxdetails: "TYPE"isoneofr a 4 e l ,etc e l ,r a 8 "SHAPE"isA R Y R A foranarray,andnothingforascalar. Thefirstargumentisanystring(nowhitespaceplease!)inprinciple,youshouldputthenameofthevariableyouaretesting. Thisnameisonlyusedtobeprinted! Thesecondargumentistheoriginalvariablethatyouwanttotest. Thethirdargumentisitscorrespondingdifferentiatedvariable. Forarrays,anextraargumentgivesthetotalsizeofthearray,i.e.theproductofthesizesofalldimensions. Forexample,ifatagivenpoint,youwanttotestthederivativeoftheREAL*8x,justinsertthecall: ALDRA8"",x) CLDEL(x,xd Totestawholearray,e.g.REALT(10,0:2),andREALmeansREAL*4onyoursystem ALDRA4RA(T,td0 CLDELARY"",t,3) Youmaytestonlyonecellofanarray,andthenyoudon'tneedthedimension: ALDRA4"__"(,)d32) CLDEL(T32,t32,t(,) Youmayevencheatalittlewitharraysizes,tocheckonlyafragmentofit: ALDRA4RA(smproT,t31,t(,)) CLDELARY"oeatf"(,)d31,5 Noticethatthesecallsmustbeinsertedinthecodeafterthetracedvariableanditsderivativehavebeencomputed,andbeforethe tracedvariableoritsderivativegetoverwrittenlateron. Ofcourse,theaboveworksonlyonapreparedprogram.Hereishowtoprepareit:Weassumeyouhavebeendifferentiatinga subroutineF.ThenthefilethatusesF_Dprobablyhastheshape: stalipt12.oteriiilvle" "elnusx,x,..thintaaus sttedfeetaintnetdrcini1,xd.. "ehifrnitoagnietonxd2,." ALFDx,xd22,.,y,yd22,.) CL_(11,x,xd..11,y,yd.. uetersligdrvtvsyd2,.. "sheutneiaie1,yd." Caution:By"inputs"(resp."outputs")ofF,wemeanlogicalinputs(resp.outputs),i.e.allvaluesthatwillbeusedinside(resp. returnedby)F.Thiscomprisesnotonlythearguments,butanycommon,global,etc...ofanysortthatholdsavaluethatisused inside(resp.returnedby)F.SincethisverificationmechanismneedstoperformtworunsofF_Donthesameinputs,youwillneed tostorealltheseinputs(wecallthis"takingasnapshot")beforethefirstrunofF_D,andrestorethembeforethesecondrun.Ifyou knowyourcodeverywellandyouarecertainthatsomeinputsarenevermodifiedbetweenthe1standthe2ndrunofF_D,then youmayomitthemfromthesnapshot. Youshouldpreparetheabovecallsitesothatitnowlookslike: el8despzr ra*dp,eseo needhs,dfl itgrdpaedie omn/DETOMNdhs,dfl,despzr cmoDTSCMO/dpaediedp,eseo stteipt12.oteriiilvle" "ehnusx,x,..thintaaus tknpht""ohntaauso12.. "aeasasoSfteiiilvlefx,x,." sttedfeetaintnetdrciniaibe1,xd.. "ehifrnitoagnietonvralsxd2,." tknpht"Dftintaieto1,xd.. "aeasasoS"ohsiiildrcinxd2,." dp. des=1d5 stteipt12.oavle"ls " "ehnusx,x,..taupu " 1=xdp*1 x1+desxd . .. pzr.0001 eseo=00000 dhs dpae=1 die=3 dfl7 PN3,FL=esaus) OE(7IE'pvle' ALFDx,xd22,.,y,yd22,.) CL_(11,x,xd..11,y,yd.. LS(7 COE3) rstteipt12.oteriiilvle,uig"" "eehnusx,x,..thintaaussnS" rsttedfeetaintnetdrcini1,xd..snS" "eehifrnitoagnietonxd2,.uig"D" dhs dpae=2

PN3,FL=esaus) OE(7IE'pvle' ALFDx,xd22,.,y,yd22,.) CL_(11,x,xd..11,y,yd.. LS(7 COE3) ThisprogramwillencounteralltheD * D callsthatarepresentinF_D.Onthefirstsweep(ddphase=1),itwilljuststorethecurrent valueofthevariable,whichisthevalue"with "intoatemporaryfilee s a u s p v l e .Ifthisfilereallygrowstoobig,youmight considermodifyingthepresentstrategy,forexamplebuildingtwoseparateprocessesthatcommunicatethroughaUNIXpipe.On thesecondsweep(ddphase=2),thevalue"with "isretrieved,andcombinedwiththecurrentvaluetobuildthe"divided differences"derivative.ThisderivativeisinturncomparedwiththeADanalyticderivative.ifthedivideddifferenceisfarfromtheAD derivative(thisisparametrizedinsided T s . ,wherethepercentageofdifferenceisinvariabled f ),thentheword detf if "D F E E C "isprintedonthestandardoutputclosetothefaultyderivatives.Pleasenotethatnothingisprintedwhenboth IFRNE derivatives(DDandAD)arezero.TapenadehasanoptionthatplacesautomaticallythecallstoallnecessaryD * D procedures,at thebeginningandendofuserspecifiedprocedures.ThisisofcourseanoptionofthetangentmodeofADanditssyntaxis: >tpnddrcatvvruis. $aeaetaecieas"nt".. Thefollowingdoesitonallunits: >tpnddrcatvvral. $aeaetaecieas%l%.. Ofcourse,youmayaddextraD * D tracecallsbyhandafterthat. Thenplayingwithadding/movingcallstoD * D procedures,youwillhopefullycloseintotheplacewheretheF_Dfileisincorrect!

CHECKTHEREVERSEDERIVATIVESUSINGTHEDOTPRODUCTTEST:
Inthefollowing,wordsbetweendoublequotesanddrawninthiscolor(suchas"SHAPE")areindeedtemplates,thatmustbe instantiatedbyactualstringsdependingonthecontext.Squarebracketsrepresentadditionalarguments,andthestarrepresents repetition. Intheexamplepiecesofprograms,actualpiecesofcodearewritteninthissortofdarkred,shortstringsindoublequotesare "patterns"asabove,and "nirieeweobeqoenrwnteptenclr eteelnsbtendulutsaddanihatroo" representapieceofcodethatyoumustwriteatthatplace. Thedotproducttestassumesthat,ifyoucallJthejacobianofF,andJtitstranspose,F_D(X,Xd)computesactuallyYd=J.Xdand F_B(X,Yb)computesactuallyXb=Jt.YbTherefore,foragivenXd,F_DreturnsYd=J.Xd,andifwesetYb=YdF_B(X,Yb)then returnsXb=Jt.J.XdWeobservethatiftheprogramiscorrect,thenthedotproduct (b|X)=J..d|X) Xd=(tJXd shouldbeequalto (.d|JX)=Yd JX.d=(d|Y) Tapenadewillautomaticallyplacethenecessarycallsinsidethedifferentiatedcodeforyou.Usetheoption p e t d t s providedin thedifferentiationcommand.Logically,youshouldfirsttrytovalidatethecompletedifferentiatedprogram,whoserootishere procedureF .Also,thedotproducttestrequiresapreliminaryrunofatangentdifferentiatedcode,soyoushoulduseTapenadeto generatebothatangentandareversedifferentiatedcode,bothbeingawareofthedotproducttestonF throughthe p e t dts option.Finally,the p e t d t s optionisincompatiblewiththe r c a t v v r optionusedtovalidatethetangentmode,so taecieas makesurenottouseitnow.Allinall,youshouldproducevalidationcodesforF D _ andF B _ withcommands: >tpnddedFpet"".. $aeaehadtsF. >tpndbedFpet"".. $aeaehadtsF. Lookoutfor(AD14)messagessuchas:( D 4 a n t i s r m n i d n a t v a i b e V V .Thesetellyou A1)CnontuethdecievralV thatthepreparationofthecodeforthedotproducttestcouldnotbecompleted,becausethetestneedstomaipulatevariableVVV whichisaglobalnotvisiblebyF .Tosolvethis,youneedtomakethisvariablevisibleinsideF ,anddifferentiateagain. Then,torunthedotproducttest,youmustalsoprepareyourcallingprogramasfollows: NEEpntalon,prgecut ITGRduiclcutdtigron OMN/PALON/pntalon,prgecut CMODCLCUTduiclcutdtigron stteipt12.oteriiilvle" "ehnusx,x,..thintaaus tknpht""ohntaauso12.. "aeasasoSfteiiilvlefx,x,." sttedfeetaintnetdrcini1,xd.. "ehifrnitoagnietonxd2,." prgecut=1 dtigron ALFDx,xd22,.,y,yd22,.) CL_(11,x,xd..11,y,yd.. rstteipt12.oteriiilvle,uig"" "eehnusx,x,..thintaaussnS" asgh1,yd.noyb2,." "sinteyd2,..it1,yb.. ALFBx,xb22,.,y,yb22,.) CL_(11,x,xb..11,y,yb.. ALDPITUMR( CLPRNSMAY) BecarefultocheckthatthecallpatternsoftheF D _ andF B _ subroutinesmatchexactlytheirdefinition,whichyouwillfindinthe differentiatedfiles.Infact,usingthe p e t d t s optionmightchangethenumberofargumentsofthesedifferentiatedsubroutines,so

itisimportanttocheckthatdefinitionsandcallmatch. Herestartsthe"CompileandRun"step,towhichwewillcyclelaterinthisexplanation... Compilethetestcodeandlinkitwiththetwolibrariesspecifictothedotproducttest,d t s . andd S a k c petf p t c . .Sincethisuses thereversemode,itshouldalsobelinkedwiththeusuala B f e . anda S a k c dufrf d t c . .AllthesefilesareprovidedintheAD FirstAidkit. Runningthetestcodewillprintonthestandardoutputthevaluesofthedotproductsforeachfragmentbetweenthemilestone pointsplacedinthecode,printingtheproducts( d | Y ) Y d whengoingforward,andtheproducts( d | X ) X b whengoing backward.ThereisasummaryprintedattheendoftheprocessbythecalltoD P I T U M R ( inthetestsite,whichshould PRNSMAY) makeiteasiertoread.Ifthereisnobug,correspondingforwardandbackwarddotproductsshouldreturnthesamevalue,andthe validationprocessiscomplete.Ifthisisnotthecase,assumingthatyouhavevalidatedthetangentmodeasindicatedabove,this showsthereisaproblemwiththereversemodeofthefragmentsforwhichthedotproductsdiffer. Thenthequestionis:wheredoestheproblemoccur?Tofindthisout,weproposethatyouuseadivideandconquermethod,by movingoraddingextramilestonesforthedotproducttest.Forexamplethe p e t " " d t s F hassetonlytheminimaltwo milestones,attheentryandattheexitofF ,plusone"somewhereinthemiddle".Thesemilestonesappearinthesummarywith defaultnames:E t y n o ,I f n r I t f n ,andE i F o f x t r m respectively.Chooseone(ormore...)pointsin(thecontrolflowof)subroutine FD _ .FindtheEXACTLYcorrespondingpointinthereversesweepofsubroutineF B _ .Thenmovethepieceofcodethatdefines theI f n milestoneofF D _ (itisaninthenendifstatementcontrolledbyI D F D E T O N ( f ) H N F ( P W T S C U T ' ' ) T E )tothechosen newpointinF D _ .Alternativelyyoumayduplicatethiscode(giveitanewnamee.g' o n A n ' P i t I f )toplaceanadditional milestone.DothesameforF B _ .Don'tcallTapenadeagainatthisstep,becauseitwouldoverwriteyourhandmodifiedF D _ and FB _ codes.Compileandrunthetestagainasexplainedabove,fromthis"CompileandRun"step. Ifthefaultyzoneeventuallygetsreducedtoahandfulofsimplestatements,thenyoushouldbeabletofindthebug.Otherwisethe faultyzonemaybereducedtoasinglecalltoaprocedurecalled,say,G .Youmustnowrestartthedivideandconquerprocessthis timeonG .UseTapenadetoplacethenecessarydotproductcalls,withthe p e t d t s option. >tpnddedFpet"".. $aeaehadtsG. >tpndbedFpet"".. $aeaehadtsG. Lookoutfor(AD14)messagesagain! Also,thebugmayappearnotonthefirstruntimecalltoG ,butonsomelatercall,forinstancethe12th.Inthiscase,replace prgecut=1 dtigron inthecallingtoplevelby prgecut=1 dtigron2 thenresumefromthe"CompileandRun"step. Weadviseyounottocreatenewmilestonesbyhand,exceptbycopypasteasexplainedabove,becausethisisratherdelicate. TheorderofP S andP P UH O 'smatters,anditisextremelyimportantthatnoactivevariablegoesthroughamilestonewithoutbeing caughtbyoneP S / O .Thisdotproductvalidationprocessismuchmorefragilethanthedivideddifferencesvalidationofthe UHPP tangentderivatives. Lastly,whenallgoeswell,don'tforgettoremovethevalidationoptionsfromyourdifferentiationcommandlines!

AllWarningorErrormessages:
WhatdoalltheseWarningandErrormessagesmean? TAPENADEissuesanumberofmessagesresultingfromthepreliminaryanalysesdonebeforeactualdifferentiation.Althoughthe temptationisstrong,thesemessagesshouldnotbeignoredrightaway.EspeciallywhenADisconcerned,thesemessagescanbe theindicationthattheprogramrunsintoonelimitationoftheADtechnology.Soeveniftheoriginalprogramcompilesandruns correctlywithyourcompiler,thesemessageswarnyouthatdifferntiationmayproduceafaultyprogram. Somemessagesarerequestsforhelp:whenTAPENADEneedssomehelpfromtheuser,e.g.becauseitneedsanddoesnot knowthesizeofanarray,itissuesamessagethatasksyoutogivethissizeafterdifferentiationisdone.Otherwisetheresulting programwillnotrun. HereisthelistofallWarningandErrormessagesofTapenade,groupedbycategory,andpointingtoadiscussionontheir meaningandimportance.Thislistmaynotbeuptodate,unfortunately... Declarationsmessages DD01,DD02,DD03,DD04,DD05,DD06,DD07,DD08,DD09,DD10,DD11,DD12,DD13 Typemessages TC01,TC02,TC03,TC04,TC05,TC06,TC07,TC08,TC09,TC10,TC11,TC12,TC13,TC14,TC15,TC16,TC17,TC18,TC19, TC20,TC21,TC22,TC23,TC24,TC25,TC26,TC27,TC28,TC29,TC30,TC31,TC32,TC33,TC34,TC35,TC36,TC37,TC38, TC40,TC41,TC42,TC50,TC51,TC52 Controlflowmessages CF01,CF02,CF03,CF04,CF05 Dataflowmessages

DF01,DF02,DF03,DF04 AutomaticDifferentiationmessages AD01,AD02,AD03,AD04,AD05,AD06,AD07,AD08,AD09,AD10 Declarationsproblems: (DD01) Cannotcombinesuccessivedeclarationsofvariablevar:Type1andType2(ignorednew) (DD02) Cannotcombinesuccessivedeclarationsofcharacterarray:Type1andType2(ignorednew) (DD03) Cannotcombinesuccessivedeclarationsofconstantconsttype:Type1andType2(ignorednew) (DD04) DoubledefinitionoffunctionFunc(ignorednew) (DD05) CannotcombinesuccessivedeclarationsoffunctionFunc(ignorednew) (DD06) CannotcombinesuccessivedeclarationsoffunctionFuncreturntype:Type1andType2(overwrittenprevious) (DD07) CannotcombinesuccessivedeclarationsoffunctionFunckind:kind1andkind2(ignorednew) (DD08) DoubledeclarationofexternalfunctionFunc (DD09) DoubledeclarationofintrinsicfunctionFunc (DD10) Doubledeclarationofthemainprogram:Func1andFunc2(overwrittenprevious) (DD11) DoubledeclarationoffunctionFuncinlibraryfileLibraryFile (DD12) CannotcombinesuccessivedeclarationsoftypeType(ignorednew) (DD13) Doubledefinitionoflabellabel(ignorednew) Therearetwosuccessivedefinitionsorpartialdeclarationsofsomesymbol,asavariable,constant,type,labelorfunction name.Thesesuccessivedeclarationsoccurinthesamescopeanddonotcombineintoacoherentdeclarationforthe symbol.TAPENADEthereforechooseseithertoignorethenewpieceofdeclaration,ortooverwritethepreviouspieceof declaration,asindicatedbythemessage.ForDD08andDD09,thetwochoicesareequivalent. Whyshouldyoucare?TAPENADEchoosestoignoretheindicateddeclarationorpartofyouroriginalprogram.Therefore thetypesconsideredbyTAPENADEmaynotbetheonesyouexpect,ortheprogramactuallydifferentiatedmaydifferfrom theprogramyouthink.Inparticularwhentypesareconcerned,rememberthatdifferentiationoccursonlyonREAL(or COMPLEX)typedvariables.Thereforesomevariablemayhavenoderivativewhereasyouthinkitshouldhaveone.Alsofor DD13orDD10,theflowofcontrolmaybedifferentfromwhatyouthink,orfromwhatyourlocalcompilerusedtobuild.Also forDD11,someinformationthatyougiveaboutanexternalfunctionmaybeignoredandreplacedbyanotherone. Whatcanyoudo?Removethedeclarationordefinitioninexcess,ormodifyconflictingdeclarationssothattheycombine wellintothetypethatyouexpect.ForDD11,cleanupthelibraryfilebymergingthedataaboutagivenfunctionintoone singleentry. Typeproblems: (TC01) (TC02) (TC03) (TC04) (TC05) (TC06) (TC07) (TC08) (TC09) (TC10) (TC11) (TC12) (TC13) (TC14) (TC15) Expressionisexpectedtobenumeric,andishereType Expressionisexpectedtobeboolean,andishereType Expressionisexpectedtobearecord,andishereType Expressionisexpectedtobeapointer,andishereType Expressionvarisexpectedtobeanarray,andishereType Loopindexisexpectedtobeinteger,andishereType Loopboundisexpectedtobeinteger,andishereType Looptimesexpressionisexpectedtobeinteger,andishereType Realpartofcomplexconstructorisexpectedtobenumeric,andishereType Imaginarypartofcomplexconstructorisexpectedtobenumeric,andishereType Argumentofarithmeticoperatorisexpectedtobenumeric,andishereType Arrayindexisexpectedtobenumeric,andishereType Tripletelementisexpectedtobenumeric,andishereType Substringindexisexpectedtobenumeric,andishereType Argumentoflogicaloperatorisexpectedtobeboolean,andishereType

Theindicatedexpressionisusedinacontextwhichrequiressometype.Usingtheavailabledeclarations,thetypechecker foundthatthisexpressionactuallyhasadifferenttype,indicatedinthemessage. Whyshouldyoucare?Thismaybethesignofanerrorintheprogram.Evenifthisoriginalprogramactuallycompilesand runswell,rememberthatdifferentiationoccursonlyonREAL(orCOMPLEX)typedvariables.Ifsomesubexpression becomesofanothertype,therewillbenoderivativeassociated,evenifyouthinkthereshouldbeone.Rememberalsothat theseproblemsmaketheprogramnonportable. Whatcanyoudo?Checkthetypes.Youmayalsousetheappropriateconversionfunctions.Beawarethatwhenyou convertaREALintoanINTEGER,thedifferentialsareresettozero. (TC16) (TC17) (TC18) Typemismatchinassignment:Type1receivesType2 Typemismatchincase:Type1comparedtoType2 Typemismatchincomparison:Type1comparedtoType2

Thetwotermsintheassignmentorcomparison,ortheswitchandcaseexpressions,musthavecompatibletypes.Otherwise, themeaningoftheinstructionisnotwelldefined,andnotportable. Whyshouldyoucare?Thiscanbetheindicationforastandardtypeerrorand,likeabove,thiscanlosederivatives. Whatcanyoudo?Checkthetypes.Useconversionfunctionswhenappropriate. (TC19) Equalitytestonrealsisnotreliable

EqualityandnonequalityarenotwelldefinedbetweenREAL's,becausetheyaredefinedonlyuptoagiven"machine precision".Equalitytestcanyielddifferentresultsondifferentmachines. Whyshouldyoucare?AsfarasADisconcerned,equalitytestsonREAL'saresometimesusedasthestoppingcriterionfor someiterativeprocess.Thenitrelatestoawellknownproblem:dothederivativesconvergewhenthefunctiondoes,andif yes,dotheyconvergeatthesamespeed? Whatcanyoudo?Checkthatthisdoesnotimpactdifferentiation.Takecaretoperformtheusualvalidationtestsonthe computedderivatives. (TC20) Variablevarisnotdeclared (TC21) Noimplicitruleavailabletocompletethedeclarationofvariablevar (TC22) Variablevar,declaredimplicitlyasType1,isusedasType2 (TC23) Symbolvar,formerlyusedasavariable,nowusedforanotherobject (TC24) WrongnumberofdimensionsforarrayArray:n2expected,andn1given (TC25) Dimensionsmismatchinvectorexpression,Type1combinedwithType2 (TC26) Incorrectequivalence (TC27) Endofcommon/COMMON/reachedbeforevariablevar (TC28) Variablevarcannotbeaddedtocommon/COMMON1/becauseitisalreadyincommon/COMMON2/ (TC29) Common/COMMON1/declaredwithtwodifferentsizes:s1vss2 (TC30) TypemismatchinargumentArgoffunctionFunc,expectsType1,receivesType2 (TC31) TypemismatchinargumentArgoffunctionFunc,declaredType1,previouslygivenType2 (TC32) WrongnumberofargumentscallingFunc:n2expected,andn1given Thesemessagesindicateconflictsbetweendeclarationandusageofsomevariable.MessageTC25occursinthecontextof vectorizedprograms.MessagesTC26toTC29indicateaprobleminEQUIVALENCE'sandCOMMON's.Recallthat variablesinaCOMMONarestoredcontiguously,sothattheirrelativeordercannotbechanged,andextravariablescannot beinsertedoccasionaly.AlsoavariablecannotbeintwodifferentCOMMON's,andonecannotsetanEQUIVALENCE betweentwovariablesthatalreadyhavetheirown,different,memoryspaceallocated.MessagesTC30toTC32indicatea mismatchbetweentheformalandactualparametersofafunction.Recallthatthestandardsrequirethatformalandactual parametersmatchinnumberandsize. Whyshouldyoucare?Ingeneral,ifyouletthesystemchoosethetypeofavariable,youruntheriskthatitbecomesREAL whileyoudon'twant,ortheotherwayround.Andthisinturnimpactsdifferentiation.MessagesTC24,andTC26toTC32are evenmoreserious:TheycanindicatethatthecodereliesonFORTRANweaktypingtoperformhiddenequivalence's betweentwoarrays,ortoresize,reshape,orsplitanarrayintopieces.SinceTAPENADEcannotpossiblyunderstandwhatis intended,itmightnotpropagatedifferentiabilityandderivativesfromtheoldarrayshapetothenewarrayshape. Whatcanyoudo?Declarevariablesexplicitly.Trytoavoidthe"implicit"declarationfacility.Declarearrayswiththeiractual size(not"1").Neverresize,reshape,orsplitarraysimplicitlythroughparameterpassing,andavoidtodosoacross COMMON's (TC33) (TC34) IntrinsicfunctionFuncisnotdeclared ExternalfunctionFuncisnotdeclared

ThesemessagesindicatethatyoudidnotprovideTAPENADEwiththesourcesoftheindicatedprocedures. Whyshouldyoucare?IfTAPENADEdoesn'tseethesourceofacalledprocedure,itisobligedtomakeseveralworstcase assumptionsaboutit.Moreover,itthisproceduremustbedifferentiated,TAPENADEcan'tdoit.Youwillhavetodoit yourself.InsomecasesthiscanbeprofitablebecauseanexpertusercandoabetterjobthanTAPENADE. Whatcanyoudo?Ifyoujustforgottheprocedure'ssourcefile,addittothelistofsourcefiles.Iftheprocedureismissingfor agoodreason,firstgiveTAPENADEtypeanddataflowinformationaboutthemissingprocedures,usingeitheradummy sourceprocedure,orusingourlibrarymechanism.Thiscanonlymakethingsbetter,astheconservativeworstcase assumptionsoftenhaveanegativeimpact.Thenthinkaboutthesemissingprocedures:ifitturnsouttheywillhavea derivative,youwillhavetoprovidethisderivative:messageAD09willremindyouofthatlater.Inthebestsituation,youare abletowriteananalyticdifferentiatedcodeyourself.Intheworstsituation,youmayusedivideddifferencesforthederivative ofthisunknownprocedure,butthiswillloosesomeaccuracy! (TC35) (TC36) SubroutineFuncusedasafunction TypefunctionFuncusedasasubroutine

(TC37)

ReturntypeofFuncsetbyimplicitruletoType

Thesemessagesindicateconflictsbetweendeclarationandusageofsomefunction.ProgramswithmessagesTC35and TC36shouldnotcompile. Whyshouldyoucare?Ingeneral,ifyouletthesystemchoosethereturntypeofafunction,youruntheriskthatitbecomes REALwhileyoudon'twant,ortheotherwayround.Andthisinturnimpactsdifferentiation. Whatcanyoudo?Insertthecorrectdeclaration. (TC38) Recursivity:Func1>Func2>...>FuncN

RecursivityiswhenasubroutineFunc1callsasubroutineFunc2,whichitselfcallsasubroutineFunc3,andsoon,andwhen thiseventuallyleadstocallingFunc1again.Inotherwordsthisisacycleinthecallgraph.Basically,thisisnotanerror, althoughFORTRAN77usedtoforbidrecursiveprograms. Whyshouldyoucare?Itisjustamatteroftime.Recursiveprogramsarehardertoanalyze.Weareprogressivelyupdating TAPENADEanalysessothattheycopewithrecursivity. Whatcanyoudo?Inthemeantime,checkthattheresultingderivativesarecorrect,usingtheclassicaltests. (TC40) (TC41) NofieldnamednameinType Recursivetypedefinition

Theseareillegalusesoftypes.TC40saystheprogramlooksforafieldnamewhichdoesnotexist,andTC41thatsome typesdirectlycontainsafieldofthesametype,thusyieldinganinfinitesize! Whyshouldyoucare?Itissurprisingistheoriginalprogramactuallycompiles!Differentiationisundefined. Whatcanyoudo?Rewritethedefinitionsofthesetypes. (TC42) DefinitionoftypeTypecomesalittletoolate

Thedefinitionofthetypecameafteritwasused.Thisisjustawarning,becauseTAPENADEwilldoasifthedefinitionwas doneinduetime.Butyoushouldbetterfixthis. (TC50) (TC51) (TC52) Labellabelisnotdefined LabelvariablevarmustbewrittenbyanASSIGNTOconstruct VariablevarusedinassignedGOTOisnotalabelvariable

Theseareillegalusesoflabels,orlabelvariables. Whyshouldyoucare?Iflabelsaremisunderstood,thefinalflowofcontrolthatTAPENADEunderstandsandregenerates mightnotbetheoneyouexpect.ThereforeTAPENADEdoesnotdifferentiatetheprogramyouexpect. Whatcanyoudo?Definethelabelsyouuse.Usethecorrectconstructstomanipulatelabelvariables.

Controlflowproblems: (CF01) (CF02) (CF03) (CF04) (CF05) Irreducibleentryintoloop ComputedGOTOwithoutlegaldestinationslist ThisassignedGOTOtovargoesnowhere ThisassignedGOTOtovaronlygoestolabellabel ThisassignedGOTOtovarispossiblyundefined

Thesemessagesindicateproblemsintheflowofcontroloftheprogram.MessageCF01comesfromjumpsthatgofrom outsidetoinsidealoop,withoutgoingthroughtheloopcontrolheader.Themeaningofthisisnotwelldefinedandis probablyimplementationdependent. Whyshouldyoucare?Ofcourseabadcontrolofaprogramyieldsabadcontrolofitsdifferentiatedprograms. Whatcanyoudo?Avoidirreducibleloops.Avoidjumpsintoloopsandintobranchesofaconditionalstructure.Trytouse structuredprogramminginstead.IfanassignedGOTOgoestoonlyoneplace,atleastreplaceitbyaGOTO. Dataflowproblems: (DF01) (DF02) (DF03) (DF04) IllegalnonreferencevalueforoutputargumentArgoffunctionFunc PotentialaliasingincallingfunctionFunc,betweenargumentsarguments Variablevarisusedbeforeinitialized Variablevarmaybeusedbeforeinitialized

TheseproblemsaredetectedthroughthebuiltininterproceduraldataflowanalysisofTAPENADE.

Whyshouldyoucare?ProgramswithDF01usuallyprovokesegmentationfaults.Aliasing(DF02)isamajorsourceof errorsforDIfferentiationinthereversemode,asdescribedinmoredetailinthe"Aliasing"section.Uninitializedvariableslead touninitializedderivatives. Whatcanyoudo?Outputformalargumentsofasubroutinemustbepassedreference(i.e.writable)actualarguments.Itis bettertoavoidaliasing.Whenaliasingisreallyaproblem,onecaneasilyavoiditbyintroducingtemporaryvariables,sothat memorylocationsoftheparametersdonotoverlap.Variablesshouldbeinitializedbeforetheyareused. AutomaticDifferentiationproblems: (AD01) (AD02) ActualargumentArgofFuncisactivewhileformalargumentisnondifferentiable ActualoutputArgofFuncisusefulwhileformalresultisnondifferentiable

Thesetwomessagesareusuallyassociated.Theyindicatethefollowingsituation:SomeprogramsimplicitlyconvertREAL's toINTEGER'sduringafunctioncall,thentheseINTEGERSfollowtheirwaythroughthecode,andfinallyareconvertedback intoREAL'sduringanotherfunctioncall. Whyshouldyoucare?IftheoriginalREAL'sareactive,i.e.dependontheindependentinputvariables,andthefinal REAL'sareuseful,i.e.influencethedependentoutputvariables,thenthistempraryconversionintoINTEGER'shaslost differentiabilityandderivatives.Differentiabilityislost,derivativesarewrong! Whatcanyoudo?REAL'smustremainREAL's.Ifabsolutelynecessary,thereisa"secret"optioninTAPENADEtoactually differentiatethesestrangeREALINTEGER's... (AD03) (AD04) ActivevariablevarwrittenbyIOtofilefile UsefulvariablevarreadbyIOfromfilefile

ThisiscomparabletotheaboveAD01andAD02.Thesetwomessagesareusuallyassociatedor,toputitdifferently,ifthe twomessagesoccurforthesamedifferentiationsession,thenthereisarisk.Theriskoccursinthefollowingsituation:Some programswriteintermediateREALvaluesintoafileandlateron,readbacktheseREALvaluesfromthisfile.Thisprocess loosesdifferentiability. Whyshouldyoucare?Ifthewrittenvalueisactive,i.e.dependsontheindependentinputvariables,andthecorresponding (equal)valuereadisuseful,i.e.influencesthedependentoutputvariables,thenthistemporarystorageintoafilehaslost differentiabilityandderivatives.Differentiabilityislost,derivativesarewrong! Forinstanceifthecodelookslike: sbotnO(,,ieo uruieTPxyfln) ra,,m elxytp itgrfln neeieo wiefln,)x rt(ieo* .. . ra(ieo*m edfln,)tp y=20x+tp .*m ed n Tapenadecannotfindthedependencypaththatgoesfromtheinputxthroughdefile"fileno"andfinallytotheoutputy.ifxdis 1.0ydwillbe2.0,whereasthecorrectanswershouldbe3.0. Whatcanyoudo?Donotusefilesastemporarystorage.Usearraysinstead.YoumayalsouseTAPENADE"Blackbox" mechanismtodisconnectdifferentiationonthecallingsubroutines,anddifferentiatethembyhand,withanadhoc mechanismtopropagatederivatives. (AD05) ActivevariablevaroverwrittenbyIO

ThisremindsyouthatavariablethatwasactivehasbeenoverwrittenbyanIOoperation Whyshouldyoucare?Thederivativeisofcourseresettozero.Maybethisisnotwhatyouexpected? Whatcanyoudo?CheckthatitisOK.OtherwisetrytoputallIOoperationsoutsidethepartoftheprogramyouareactually differentiating. (AD06) (AD07) (AD10) DifferentiateoffunctionFuncneedstosaveundeclaredsideeffectvariables:variables Thiscallneedstosaveundeclaredsideeffectvariable:externvar calltoProc2indifferentiationofprocedureProc1needtosaveandrestoretheIOstate

Thesemessagesallindicatethatthecheckpointingmechanismisnotcertainthatitcandoacorrectjob.Thesemessages pointat(acollection)ofprocedurecallsintheprogram,e.g.allcallstoFunc,oroneparticularcall,orprocedure(i.e. subroutineorfunction)Proc1callinganotherprocedure(i.e.subroutineorfunction)Proc2. Inreversemode,thestandardstrategyofTapenadeistocheckpointallprocedurecalls(cfaquickdescriptionof checkpointing,orTapenadeUser'sguideforsomethingmoretechnical).CheckpointingacalltoProc2insideProc1means thatthereversedifferentiatedsubroutineofProc1isgoingtocallProc2twice,ormorepreciselytheoriginalProc2onceand thenthedifferentiatedProc2.Roughlyspeking,thesetwocallsmustbemadewiththesameinputs,sothatthe2ndcallisan exactduplicationofthe1stcall.Todothis,aminimal"environment"needstobestoredjustbeforethe1stcalltoProc2and

restoredjustbeforethe2ndcall.Weknowhowtosaveandrestoremostvariables,e.g.theformalparametersofProc2,but alsotheglobalsknownbyProc1andProc2.Ontheotherhand,wedon'tknowhowtosaveandrestorehiddenSAVE variables,orprivateglobalvariablesofamodule,oraCOMMONwhichisnotdeclaredatthecallingsubroutinelevel,orthe positionoftheread/writepointerinsideanopenedfilewhichisbeingread/written,ortheveryopen/closestatusofafile,and severalotherthings...Thesemessagesjustsignalsuch(possibly)nastysituations. Whyshouldyoucare?CheckpointingcannotrestoretheenvironmentbeforethesecondcalltoProc2.Thereforethe differentiatedprogrammightbewrong. Whatcanyoudo?Eithermakeabsolutelysurethatthisisnotaprobleminthepresentcase,becauseyouknowthatthe secondcalltoProc2willbehavejustlikethe1stone,atleastasfarasdifferentiationisconcerned.Youneedtoknowthe codeforthatbecauseerrorswillbehardtotrackdown.Forinstance,onefavorablesituationiswhenthe(AD10)message correspondsonlytoprintsofmessagesonthescreen.Themessageswillappeartwice,butderivativesshouldbecorrect. Ormodifyyourcodesothattheproblemdisappears,forinstanceinserttheCOMMONdeclarationatthelevelofthecalling subroutine.ortransformtheSAVEvariablesintoelementsofaCOMMONthateveryonecansee,ortakethefileI/Oor dynamicmemoryallocationordeallocationoutsidethecodetodifferentiate, Or(morehightech!)tellTapenadenottocheckpointthecallsthatraisetheproblemusingthe"nocheckpoint"directive. (AD08) Don'tknowthesizeofdimensionnofarrayarray,whichmustbesaved

Insomecases,forexampleinreversemodeAD,thedifferentiatedfunctionmustsavesomevariables,torestorethemlater.If onesuchvariableisanarray,andthesizeofthisarrayisdynamic,TAPENADEmustknowthissizetoactuallydothesave. Whyshouldyoucare?Ifyoudon'tgivethissize,thedifferentiatedprogramwillnotcompile. Whatcanyoudo?DefinethevalueofthePARAMETERSthatTAPENADErequests,inaspecialincludefile.Youmayalso explicitlygivethissizeintheoriginalfile,sothatthemessagedoesnotshowup! (AD09) PleaseprovideadifferentialoffunctionFuncforargumentsarguments

Youmustprovidethedifferentiatedprogramwithanewfunction,differentiatedfromFuncwithrespecttotheinputandoutput parametersofFuncspecifiedinthearguments.ProbablyFuncisanexternalfunction,forwhichTAPENADEhasnosource, andthereforeitcouldn'tdifferentiateit. Whyshouldyoucare?Ifyoudon'tgivethisnewfunction,thedifferentiatedprogramwillprobablynotcompile. Whatcanyoudo?Definethisnewfunction.Youhavethechoiceofthemethod.Maybeyouknowexplicitlythederivative mathematicalfunction,whichcanbeimplementedefficiently.MaybeyouhavethesourceandyoumayuseADagain,with TAPENADEorwithanothertool.Maybeyoujustwanttorunalocal"divideddifferences"method,knowingthatthismight slightlydegradetheprecisionofthederivatives.Inanycase,makesurethatyouprovideadifferentiatedfunctioninthecorrect mode(tangent,reverse...),andwhichadherestotheconventionsofTAPENADEaboutthenameofthefunctionandthe orderofitsparameters.

...Tobecontinued...