Vous êtes sur la page 1sur 278

APythonBook

APythonBook:BeginningPython,Advanced
Python,andPythonExercises
Author:
DaveKuhlman
Contact:
dkuhlman@davekuhlman.org
Address:
http://www.davekuhlman.org

Page1

APythonBook

Revision
1.3a
Date
December15,2013
Copyright
Copyright(c)2009DaveKuhlman.AllRightsReserved.Thisdocumentissubject
totheprovisionsoftheOpenSourceMITLicense
http://www.opensource.org/licenses/mitlicense.php.
Abstract
ThisdocumentisaselflearningdocumentforacourseinPythonprogramming.
Thiscoursecontains(1)apartforbeginners,(2)adiscussionofseveraladvanced
topicsthatareofinteresttoPythonprogrammers,and(3)aPythonworkbookwith
lotsofexercises.

Page2

APythonBook

Contents
1Part1BeginningPython...........................................................................................10
1.1IntroductionsEtc...................................................................................................10
1.1.1Resources.......................................................................................................11
1.1.2AgeneraldescriptionofPython....................................................................12
1.1.3InteractivePython..........................................................................................15
1.2Lexicalmatters......................................................................................................15
1.2.1Lines..............................................................................................................15
1.2.2Comments......................................................................................................16
1.2.3Namesandtokens..........................................................................................16
1.2.4Blocksandindentation..................................................................................16
1.2.5Docstrings.....................................................................................................17
1.2.6Programstructure..........................................................................................17
1.2.7Operators.......................................................................................................18
1.2.8Alsosee.........................................................................................................19
1.2.9Codeevaluation.............................................................................................19
1.3Statementsandinspectionpreliminaries...........................................................20
1.4Builtindatatypes.................................................................................................21
1.4.1Numerictypes................................................................................................21
1.4.2Tuplesandlists..............................................................................................21
1.4.3Strings............................................................................................................24
1.4.3.1Thenewstring.formatmethod...............................................................26
1.4.3.2Unicodestrings......................................................................................27
1.4.4Dictionaries....................................................................................................29
1.4.5Files...............................................................................................................32
1.4.6Otherbuiltintypes........................................................................................35
1.4.6.1TheNonevalue/type..............................................................................35
1.4.6.2Booleanvalues.......................................................................................36
1.4.6.3Setsandfrozensets.................................................................................36
1.5FunctionsandClassesAPreview......................................................................36
1.6Statements.............................................................................................................37
1.6.1Assignmentstatement....................................................................................37
1.6.2importstatement............................................................................................39
1.6.3printstatement...............................................................................................41
1.6.4if:elif:else:statement...................................................................................43
1.6.5for:statement.................................................................................................44
1.6.6while:statement.............................................................................................48
Page3

APythonBook
1.6.7continueandbreakstatements.......................................................................48
1.6.8try:except:statement.....................................................................................49
1.6.9raisestatement...............................................................................................51
1.6.10with:statement.............................................................................................52
1.6.10.1Writingacontextmanager...................................................................52
1.6.10.2Usingthewith:statement....................................................................53
1.6.11del................................................................................................................54
1.6.12casestatement..............................................................................................55
1.7Functions,Modules,Packages,andDebugging....................................................55
1.7.1Functions.......................................................................................................55
1.7.1.1Thedefstatement...................................................................................55
1.7.1.2Returningvalues....................................................................................55
1.7.1.3Parameters..............................................................................................56
1.7.1.4Arguments..............................................................................................56
1.7.1.5Localvariables.......................................................................................57
1.7.1.6Otherthingstoknowaboutfunctions....................................................57
1.7.1.7Globalvariablesandtheglobalstatement.............................................58
1.7.1.8Docstringsforfunctions.......................................................................60
1.7.1.9Decoratorsforfunctions........................................................................60
1.7.2lambda...........................................................................................................61
1.7.3Iteratorsandgenerators.................................................................................62
1.7.4Modules.........................................................................................................67
1.7.4.1Docstringsformodules.........................................................................68
1.7.5Packages........................................................................................................68
1.8Classes...................................................................................................................69
1.8.1Asimpleclass................................................................................................69
1.8.2Definingmethods..........................................................................................70
1.8.3Theconstructor..............................................................................................70
1.8.4Membervariables..........................................................................................70
1.8.5Callingmethods.............................................................................................71
1.8.6Addinginheritance........................................................................................71
1.8.7Classvariables...............................................................................................72
1.8.8Classmethodsandstaticmethods.................................................................72
1.8.9Properties.......................................................................................................74
1.8.10Interfaces.....................................................................................................75
1.8.11Newstyleclasses.........................................................................................75
1.8.12Docstringsforclasses.................................................................................77
1.8.13Privatemembers..........................................................................................77
1.9SpecialTasks.........................................................................................................77
1.9.1Debuggingtools.............................................................................................77
Page4

APythonBook
1.9.2Fileinputandoutput......................................................................................78
1.9.3Unittests........................................................................................................80
1.9.3.1Asimpleexample..................................................................................80
1.9.3.2Unittestsuites........................................................................................81
1.9.3.3Additionalunittestfeatures....................................................................83
1.9.3.4GuidanceonUnitTesting......................................................................85
1.9.4doctest............................................................................................................85
1.9.5ThePythondatabaseAPI..............................................................................87
1.9.6InstallingPythonpackages............................................................................88
1.10MorePythonFeaturesandExercises..................................................................89
2Part2AdvancedPython............................................................................................90
2.1IntroductionPython201(Slightly)AdvancedPythonTopics.......................90
2.2RegularExpressions..............................................................................................90
2.2.1Definingregularexpressions.........................................................................90
2.2.2Compilingregularexpressions......................................................................91
2.2.3Usingregularexpressions..............................................................................91
2.2.4Usingmatchobjectstoextractavalue..........................................................92
2.2.5Extractingmultipleitems..............................................................................93
2.2.6Replacingmultipleitems...............................................................................94
2.3IteratorObjects......................................................................................................96
2.3.1ExampleAgeneratorfunction....................................................................98
2.3.2ExampleAclasscontainingageneratormethod......................................100
2.3.3ExampleAniteratorclass.........................................................................102
2.3.4ExampleAniteratorclassthatusesyield.................................................104
2.3.5ExampleAlistcomprehension.................................................................105
2.3.6ExampleAgeneratorexpression..............................................................105
2.4UnitTests............................................................................................................106
2.4.1Definingunittests........................................................................................106
2.4.1.1Createatestclass.................................................................................106
2.5ExtendingandembeddingPython......................................................................109
2.5.1Introductionandconcepts............................................................................109
2.5.2Extensionmodules.......................................................................................110
2.5.3SWIG...........................................................................................................112
2.5.4Pyrex............................................................................................................115
2.5.5SWIGvs.Pyrex...........................................................................................120
2.5.6Cython.........................................................................................................120
2.5.7Extensiontypes............................................................................................122
2.5.8Extensionclasses.........................................................................................122
2.6Parsing.................................................................................................................122
2.6.1Specialpurposeparsers...............................................................................123
Page5

APythonBook
2.6.2Writingarecursivedescentparserbyhand.................................................124
2.6.3Creatingalexer/tokenizerwithPlex...........................................................131
2.6.4Asurveyofexistingtools............................................................................141
2.6.5CreatingaparserwithPLY.........................................................................141
2.6.6Creatingaparserwithpyparsing.................................................................148
2.6.6.1Parsingcommadelimitedlines............................................................148
2.6.6.2Parsingfunctors...................................................................................149
2.6.6.3Parsingnames,phonenumbers,etc.....................................................150
2.6.6.4Amorecomplexexample....................................................................151
2.7GUIApplications................................................................................................153
2.7.1Introduction.................................................................................................153
2.7.2PyGtk...........................................................................................................153
2.7.2.1Asimplemessagedialogbox..............................................................153
2.7.2.2Asimpletextinputdialogbox.............................................................156
2.7.2.3Afileselectiondialogbox...................................................................158
2.7.3EasyGUI......................................................................................................160
2.7.3.1AsimpleEasyGUIexample................................................................161
2.7.3.2AnEasyGUIfileopendialogexample................................................161
2.8GuidanceonPackagesandModules...................................................................161
2.8.1Introduction.................................................................................................161
2.8.2ImplementingPackages...............................................................................162
2.8.3UsingPackages............................................................................................162
2.8.4DistributingandInstallingPackages...........................................................162
2.9EndMatter...........................................................................................................164
2.9.1AcknowledgementsandThanks..................................................................164
2.9.2SeeAlso.......................................................................................................164
3Part3PythonWorkbook.........................................................................................165
3.1Introduction.........................................................................................................165
3.2LexicalStructures................................................................................................165
3.2.1Variablesandnames....................................................................................165
3.2.2Linestructure...............................................................................................167
3.2.3Indentationandprogramstructure...............................................................168
3.3ExecutionModel.................................................................................................169
3.4BuiltinDataTypes.............................................................................................170
3.4.1Numbers......................................................................................................170
3.4.1.1Literalrepresentationsofnumbers......................................................171
3.4.1.2Operatorsfornumbers.........................................................................173
3.4.1.3Methodsonnumbers............................................................................175
3.4.2Lists.............................................................................................................175
3.4.2.1Literalrepresentationoflists...............................................................176
Page6

APythonBook
3.4.2.2Operatorsonlists.................................................................................178
3.4.2.3Methodsonlists...................................................................................178
3.4.2.4Listcomprehensions............................................................................180
3.4.3Strings..........................................................................................................182
3.4.3.1Characters............................................................................................183
3.4.3.2Operatorsonstrings.............................................................................184
3.4.3.3Methodsonstrings...............................................................................185
3.4.3.4Rawstrings..........................................................................................187
3.4.3.5Unicodestrings....................................................................................188
3.4.4Dictionaries..................................................................................................190
3.4.4.1Literalrepresentationofdictionaries...................................................190
3.4.4.2Operatorsondictionaries.....................................................................191
3.4.4.3Methodsondictionaries.......................................................................192
3.4.5Files.............................................................................................................195
3.4.6Afewmiscellaneousdatatypes..................................................................197
3.4.6.1None.....................................................................................................197
3.4.6.2ThebooleansTrueandFalse...............................................................197
3.5Statements...........................................................................................................198
3.5.1Assignmentstatement..................................................................................198
3.5.2printstatement.............................................................................................200
3.5.3if:statementexercises..................................................................................201
3.5.4for:statementexercises...............................................................................202
3.5.5while:statementexercises...........................................................................205
3.5.6breakandcontinuestatements.....................................................................206
3.5.7Exceptionsandthetry:except:andraisestatements...................................207
3.6Functions.............................................................................................................210
3.6.1Optionalargumentsanddefaultvalues.......................................................211
3.6.2Passingfunctionsasarguments...................................................................213
3.6.3Extraargsandkeywordargs.......................................................................214
3.6.3.1Orderofarguments(positional,extra,andkeywordargs)..................216
3.6.4Functionsandducktypingandpolymorphism...........................................216
3.6.5Recursivefunctions.....................................................................................217
3.6.6Generatorsanditerators...............................................................................219
3.7Objectorientedprogrammingandclasses..........................................................223
3.7.1Theconstructor............................................................................................224
3.7.2InheritanceImplementingasubclass.......................................................225
3.7.3Classesandpolymorphism..........................................................................227
3.7.4Recursivecallstomethods..........................................................................228
3.7.5Classvariables,classmethods,andstaticmethods.....................................230
3.7.5.1Decoratorsforclassmethodandstaticmethod.....................................233
Page7

APythonBook
3.8AdditionalandAdvancedTopics........................................................................234
3.8.1Decoratorsandhowtoimplementthem......................................................234
3.8.1.1Decoratorswitharguments..................................................................235
3.8.1.2Stackeddecorators...............................................................................236
3.8.1.3Morehelpwithdecorators...................................................................238
3.8.2Iterables.......................................................................................................239
3.8.2.1AfewpreliminariesonIterables..........................................................239
3.8.2.2Morehelpwithiterables......................................................................240
3.9ApplicationsandRecipes....................................................................................240
3.9.1XMLSAX,minidom,ElementTree,Lxml..............................................241
3.9.2Relationaldatabaseaccess...........................................................................249
3.9.3CSVcommaseparatedvaluefiles...........................................................255
3.9.4YAMLandPyYAML..................................................................................256
3.9.5Json..............................................................................................................258
4Part4GeneratingPythonBindingsforXML.........................................................260
4.1Introduction.........................................................................................................260
4.2Generatingthecode.............................................................................................261
4.3UsingthegeneratedcodetoparseandexportanXMLdocument.....................263
4.4Somecommandlineoptionsyoumightwanttoknow.......................................263
4.5Thegraphicalfrontend.......................................................................................264
4.6Addingapplicationspecificbehavior.................................................................265
4.6.1Implementingcustomsubclasses................................................................265
4.6.2Usingthegenerated"API"fromyourapplication......................................266
4.6.3Acombinedapproach..................................................................................267
4.7Specialsituationsanduses..................................................................................269
4.7.1Generic,typeindependentprocessing.........................................................269
4.7.1.1Step1generatethebindings............................................................270
4.7.1.2Step2addapplicationspecificcode................................................270
4.7.1.3Step3writeatest/driverharness.....................................................274
4.7.1.4Step4runthetestapplication..........................................................276
4.8Somehints...........................................................................................................276
4.8.1ChildrendefinedwithmaxOccursgreaterthan1........................................276
4.8.2Childrendefinedwithsimplenumerictypes...............................................277
4.8.3Thetypeofanelement'scharactercontent..................................................277
4.8.4Constructorsandtheirdefaultvalues..........................................................277

Page8

APythonBook

Preface
ThisbookisacollectionofmaterialsthatI'veusedwhenconductingPythontrainingand
alsomaterialsfrommyWebsitethatareintendedforselfinstruction.
Youmaypreferamachinereadablecopyofthisbook.Youcanfinditinvariousformats
here:
HTMLhttp://www.davekuhlman.org/python_book_01.html
PDFhttp://www.davekuhlman.org/python_book_01.pdf
ODF/OpenOfficehttp://www.davekuhlman.org/python_book_01.odt
And,letmethankthestudentsinmyPythonclasses.Theirquestionsandsuggestions
wereagreathelpinthepreparationofthesematerials.

Page9

APythonBook

1Part1BeginningPython
1.1IntroductionsEtc
Introductions
Practicalmatters:restrooms,breakroom,lunchandbreaktimes,etc.
StartingthePythoninteractiveinterpreter.Also,IPythonandIdle.
Runningscripts
EditorsChooseaneditorwhichyoucanconfiguresothatitindentswith4spaces,not
tabcharacters.ForalistofeditorsforPython,see:
http://wiki.python.org/moin/PythonEditors.Afewpossibleeditors:
SciTEhttp://www.scintilla.org/SciTE.html.
MSWindowsonly(1)TextPadhttp://www.textpad.com;(2)UltraEdit
http://www.ultraedit.com/.
JedSeehttp://www.jedsoft.org/jed/.
EmacsSeehttp://www.gnu.org/software/emacs/and
http://www.xemacs.org/faq/xemacsfaq.html.
jEditRequiresabitofcustomizationforPythonSeehttp://jedit.org.
Vimhttp://www.vim.org/
Geanyhttp://www.geany.org/
Andmanymore.
Interactiveinterpreters:

python
ipython
Idle
IDEsAlsosee
http://en.wikipedia.org/wiki/List_of_integrated_development_environments_for_Python:

PyWinMSWindowsonly.Availableat:
http://sourceforge.net/projects/pywin32/.
WingIDESeehttp://wingware.com/wingide/.
Eclipsehttp://eclipse.org/.ThereisapluginthatsupportsPython.
KdevelopLinux/KDESeehttp://www.kdevelop.org/.
EricLinuxKDE?Seehttp://ericide.pythonprojects.org/index.html
EmacsandSciTEwillevaluateaPythonbufferwithintheeditor.
Page10

APythonBook

1.1.1Resources
Whereelsetogethelp:
Pythonhomepagehttp://www.python.org
Pythonstandarddocumentationhttp://www.python.org/doc/.
Youwillalsofindlinkstotutorialsthere.
FAQshttp://www.python.org/doc/faq/.
ThePythonWikihttp://wiki.python.org/
ThePythonPackageIndexLotsofPythonpackages
https://pypi.python.org/pypi
Specialinterestgroups(SIGs)http://www.python.org/sigs/
Otherpythonrelatedmailinglistsandlistsforspecificapplications(forexample,
Zope,Twisted,etc).Try:http://dir.gmane.org/search.php?match=python.
http://sourceforge.netLotsofprojects.Searchfor"python".
USENETcomp.lang.python.CanalsobeaccessedthroughGmane:
http://dir.gmane.org/gmane.comp.python.general.
ThePythontutoremaillisthttp://mail.python.org/mailman/listinfo/tutor
Localdocumentation:

OnMSWindows,thePythondocumentationisinstalledwiththestandard
installation.
InstallthestandardPythondocumentationonyourmachinefrom
http://www.python.org/doc/.
pydoc.Example,onthecommandline,type:pydocre.
Importamodule,thenviewits.__doc__attribute.
Attheinteractiveprompt,usehelp(obj).Youmightneedtoimportitfirst.
Example:
>>>importurllib
>>>help(urllib)

InIPython,thequestionmarkoperatorgiveshelp.Example:
In[13]:open?
Type:builtin_function_or_method
BaseClass:<type'builtin_function_or_method'>
StringForm:<builtinfunctionopen>
Namespace:Pythonbuiltin
Docstring:
open(name[,mode[,buffering]])>fileobject
Openafileusingthefile()type,returnsafile
object.
ConstructorDocstring:
x.__init__(...)initializesx;see
x.__class__.__doc__forsignature

Page11

APythonBook
Callable:Yes
Calldef:Callingdefinitionnotavailable.Call
docstring:
x.__call__(...)<==>x(...)

1.1.2AgeneraldescriptionofPython
Pythonisahighlevelgeneralpurposeprogramminglanguage:
Becausecodeisautomaticallycompiledtobytecodeandexecuted,Pythonis
suitableforuseasascriptinglanguage,Webapplicationimplementation
language,etc.
BecausePythoncanbeextendedinCandC++,Pythoncanprovidethespeed
neededforevencomputeintensivetasks.
Becauseofitsstrongstructuringconstructs(nestedcodeblocks,functions,
classes,modules,andpackages)anditsconsistentuseofobjectsand
objectorientedprogramming,Pythonenablesustowriteclear,logical
applicationsforsmallandlargetasks.
ImportantfeaturesofPython:

Builtinhighleveldatatypes:strings,lists,dictionaries,etc.
Theusualcontrolstructures:if,ifelse,ifelifelse,while,plusapowerful
collectioniterator(for).
Multiplelevelsoforganizationalstructure:functions,classes,modules,and
packages.Theseassistinorganizingcode.Anexcellentandlargeexampleisthe
Pythonstandardlibrary.
CompileontheflytobytecodeSourcecodeiscompiledtobytecodewithouta
separatecompilestep.Sourcecodemodulescanalsobe"precompiled"tobyte
codefiles.
ObjectorientedPythonprovidesaconsistentwaytouseobjects:everythingis
anobject.And,inPythonitiseasytoimplementnewobjecttypes(calledclasses
inobjectorientedprogramming).
ExtensionsinCandC++Extensionmodulesandextensiontypescanbewritten
byhand.Therearealsotoolsthathelpwiththis,forexample,SWIG,sip,Pyrex.
JythonisaversionofPythonthat"playswellwith"Java.See:TheJythonProject
http://www.jython.org/Project/.
Somethingsyouwillneedtoknow:

Pythonusesindentationtoshowblockstructure.Indentoneleveltoshowthe
beginningofablock.Outdentoneleveltoshowtheendofablock.Asan
example,thefollowingCstylecode:
if(x)
{

Page12

APythonBook
if(y)
{
f1()
}
f2()
}

inPythonwouldbe:
ifx:
ify:
f1()
f2()

And,theconventionistousefourspaces(andnohardtabs)foreachlevelofindentation.
Actually,it'smorethanaconvention;it'spracticallyarequirement.Followingthat
"convention"willmakeitsomucheasiertomergeyourPythoncodewithcodefrom
othersources.
AnoverviewofPython:

AscriptinglanguagePythonissuitable(1)forembedding,(2)forwritingsmall
unstructuredscripts,(3)for"quickanddirty"programs.
Notascriptinglanguage(1)Pythonscales.(2)Pythonencouragesustowrite
codethatisclearandwellstructured.
Interpreted,butalsocompiledtobytecode.Modulesareautomaticallycompiled
(to.pyc)whenimported,butmayalsobeexplicitlycompiled.
Providesaninteractivecommandlineandinterpretershell.Infact,thereare
several.
DynamicForexample:
Typesareboundtovalues,nottovariables.
Functionandmethodlookupisdoneatruntime.
Valuesareinspectable.
Thereisaninteractiveinterpreter,morethanone,infact.
Youcanlistthemethodssupportedbyanygivenobject.
Stronglytypedatruntime,notcompiletime.Objects(values)haveatype,but
variablesdonot.
ReasonablyhighlevelHighlevelbuiltindatatypes;highlevelcontrol
structures(forwalkinglistsanditerators,forexample).
ObjectorientedAlmosteverythingisanobject.Simpleobjectdefinition.Data
hidingbyagreement.Multipleinheritance.Interfacesbyconvention.
Polymorphism.
HighlystructuredStatements,functions,classes,modules,andpackagesenable
ustowritelarge,wellstructuredapplications.Whystructure?Readability,
locateability,modifiability.
Explicitness
Page13

APythonBook

Firstclassobjects:
Definition:Can(1)passtofunction;(2)returnfromfunction;(3)stuffintoa
datastructure.
Operatorscanbeappliedtovalues(notvariables).Example:f(x)[3]
Indentedblockstructure"Pythonispseudocodethatruns."
EmbeddingandextendingPythonPythonprovidesawelldocumentedand
supportedway(1)toembedthePythoninterpreterinC/C++applicationsand(2)
toextendPythonwithmodulesandobjectsimplementedinC/C++.
Insomecases,SWIGcangeneratewrappersforexistingC/C++code
automatically.Seehttp://www.swig.org/
CythonenablesustogenerateCcodefromPythonandto"easily"create
wrappersforC/C++functions.See
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
ToembedandextendPythonwithJava,thereisJython.See
http://www.jython.org/
Automaticgarbagecollection.(But,thereisagcmoduletoallowexplicitcontrol
ofgarbagecollection.)
Comparisonwithotherlanguages:compiledlanguages(e.g.C/C++);Java;Perl,
Tcl,andRuby.Pythonexcellsat:developmentspeed,executionspeed,clarityand
maintainability.
VarietiesofPython:
CPythonStandardPython2.ximplementedinC.
JythonPythonfortheJavaenvironmenthttp://www.jython.org/
PyPyPythonwithaJITcompilerandstacklessmodehttp://pypy.org/
StacklessPythonwithenhancedthreadsupportandmicrothreadsetc.
http://www.stackless.com/
IronPythonPythonfor.NETandtheCLRhttp://ironpython.net/
Python3Thenew,newPython.Thisisintendedasareplacementfor
Python2.x.http://www.python.org/doc/.Afewdifferences(fromPython
2.x):
Theprintstatementchangedtotheprintfunction.
Stringsareunicodebydefault.
Classesareall"newstyle"classes.
Changestosyntaxforcatchingexceptions.
Changestointegersnolonginteger;integerdivisionwithautomatic
converttofloat.
Morepervasiveuseofiterables(ratherthancollections).
Etc.
ForamoreinformationaboutdifferencesbetweenPython2.xandPython3.x,
seethedescriptionofthevariousfixesthatcanbeappliedwiththe2to3tool:
Page14

APythonBook

http://docs.python.org/3/library/2to3.html#fixers
Themigrationtool,2to3,easestheconversionof2.xcodeto3.x.
AlsoseeTheZenofPythonhttp://www.python.org/peps/pep0020.html.Or,at
thePythoninteractiveprompt,type:
>>>importthis

1.1.3InteractivePython
IfyouexecutePythonfromthecommandlinewithnoscript(noarguments),Python
givesyouaninteractiveprompt.ThisisanexcellentfacilityforlearningPythonandfor
tryingsmallsnippetsofcode.Manyoftheexamplesthatfollowweredevelopedusing
thePythoninteractiveprompt.
StartthePythoninteractiveinterpreterbytypingpythonwithnoargumentsatthe
commandline.Forexample:
$python
Python2.6.1(r261:67515,Jan112009,15:19:23)
[GCC4.3.2]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>print'hello'
hello
>>>

YoumayalsowanttoconsiderusingIDLE.IDLEisagraphicalintegrateddevelopment
environmentforPython;itcontainsaPythonshell.ItislikelythatIdlewasinstalledfor
youwhenyouinstalledPython.YouwillfindascripttostartupIDLEinthe
Tools/scriptsdirectoryofyourPythondistribution.IDLErequiresTkinter.
Inaddition,therearetoolsthatwillgiveyouamorepowerfulandfancyPython
interactiveinterpreter.OneexampleisIPython,whichisavailableat
http://ipython.scipy.org/.

1.2Lexicalmatters
1.2.1Lines

Pythondoeswhatyouwantittodomostofthetimesothatyouonlyhavetoadd
extracharacterssomeofthetime.
Statementseparatorisasemicolon,butisonlyneededwhenthereismorethan
onestatementonaline.And,writingmorethanonestatementonthesamelineis
consideredbadform.
ContinuationlinesAbackslashaslastcharacterofthelinemakesthe
Page15

APythonBook
followinglineacontinuationofthecurrentline.But,notethatanopening
"context"(parenthesis,squarebracket,orcurlybracket)makesthebackslash
unnecessary.

1.2.2Comments
Everythingafter"#"onalineisignored.Noblockcomments,butdocstringsarea
commentinquotesatthebeginningofamodule,class,methodorfunction.Also,editors
withsupportforPythonoftenprovidetheabilitytocommentoutselectedblocksofcode,
usuallywith"##".

1.2.3Namesandtokens

Allowedcharacters:azAZ09underscore,andmustbeginwithaletteror
underscore.
Namesandidentifiersarecasesensitive.
Identifierscanbeofunlimitedlength.
Specialnames,customizing,etc.Usuallybeginandendindoubleunderscores.
SpecialnameclassesSingleanddoubleunderscores.
SingleleadingsingleunderscoreSuggestsa"private"methodorvariable
name.Notimportedby"frommoduleimport*".
SingletrailingunderscoreUseittoavoidconflictswithPythonkeywords.
DoubleleadingunderscoresUsedinaclassdefinitiontocausename
mangling(weakhiding).But,notoftenused.
NamingconventionsNotrigid,but:
Modulesandpackagesalllowercase.
GlobalsandconstantsUppercase.
ClassesBumpycapswithinitialupper.
MethodsandfunctionsAlllowercasewithwordsseparatedbyunderscores.
LocalvariablesLowercase(withunderscorebetweenwords)orbumpy
capswithinitialloweroryourchoice.
GoodadviceFollowtheconventionsusedinthecodeonwhichyouare
working.
Names/variablesinPythondonothaveatype.Valueshavetypes.

1.2.4Blocksandindentation
Pythonrepresentsblockstructureandnestedblockstructurewithindentation,notwith
beginandendbrackets.
TheemptyblockUsethepassnoopstatement.
Benefitsoftheuseofindentationtoindicatestructure:
Page16

APythonBook
Reducestheneedforacodingstandard.Onlyneedtospecifythatindentationis4
spacesandnohardtabs.
Reducesinconsistency.Codefromdifferentsourcesfollowthesameindentation
style.Ithasto.
Reduceswork.Onlyneedtogettheindentationcorrect,notbothindentationand
brackets.
Reducesclutter.Eliminatesallthecurlybrackets.
Ifitlookscorrect,itiscorrect.Indentationcannotfoolthereader.
EditorconsiderationsThestandardis4spaces(nohardtabs)foreachindentationlevel.
Youwillneedatexteditorthathelpsyourespectthat.

1.2.5Docstrings
Docstringsarelikecomments,buttheyarecarriedwithexecutingcode.Docstringscan
beviewedwithseveraltools,e.g.help(),obj.__doc__,and,inIPython,aquestion
mark(?)afteranamewillproducehelp.
Adocstringiswrittenasaquotedstringthatisatthetopofamoduleorthefirstlines
aftertheheaderlineofafunctionorclass.
Wecanusetriplequotingtocreatedocstringsthatspanmultiplelines.
Therearealsotoolsthatextractandformatdocstrings,forexample:
pydocDocumentationgeneratorandonlinehelpsystem
http://docs.python.org/lib/modulepydoc.html.
epydocEpydoc:AutomaticAPIDocumentationGenerationforPython
http://epydoc.sourceforge.net/index.html
SphinxCanalsoextractdocumentationfromPythondocstrings.See
http://sphinxdoc.org/index.html.
Seethefollowingforsuggestionsandmoreinformationondocstrings:Docstring
conventionshttp://www.python.org/dev/peps/pep0257/.

1.2.6Programstructure

Executiondef,class,etcareexecutablestatementsthataddsomethingtothe
currentnamespace.Modulescanbebothexecutableandimportable.
Statements,datastructures,functions,classes,modules,packages.
Functions
Classes
Modulescorrespondtofileswitha"*.py"extension.Packagescorrespondtoa
directory(orfolder)inthefilesystem;apackagecontainsafilenamed
"__init__.py".Bothmodulesandpackagescanbeimported(seesectionimport
Page17

APythonBook

statement).
PackagesAdirectorycontainingafilenamed"__init__.py".Canprovide
additionalinitializationwhenthepackageoramoduleinitisloaded(imported).

1.2.7Operators

See:http://docs.python.org/ref/operators.html.Pythondefinesthefollowing
operators:
+***///%
<<>>&|^~
<><=>===!=<>

Thecomparisonoperators<>and!=arealternatespellingsofthesameoperator.
!=isthepreferredspelling;<>isobsolescent.
Logicaloperators:
andorisnotin

Therearealso(1)thedotoperator,(2)thesubscriptoperator[],andthe
function/methodcalloperator().
Forinformationontheprecedencesofoperators,seethetableat
http://docs.python.org/2/reference/expressions.html#operatorprecedence,which
isreproducedbelow.
Forinformationonwhatthedifferentoperatorsdo,thesectioninthe"Python
LanguageReference"titled"Specialmethodnames"maybeofhelp:
http://docs.python.org/2/reference/datamodel.html#specialmethodnames
ThefollowingtablesummarizestheoperatorprecedencesinPython,fromlowest
precedence(leastbinding)tohighestprecedence(mostbinding).Operatorsonthe
samelinehavethesameprecedence.Unlessthesyntaxisexplicitlygiven,
operatorsarebinary.Operatorsonthesamelinegrouplefttoright(exceptfor
comparisons,includingtests,whichallhavethesameprecedenceandchainfrom
lefttorightseesection5.9andexponentiation,whichgroupsfromrightto
left):
OperatorDescription
==========================================
lambdaLambdaexpression
orBooleanOR
andBooleanAND
notxBooleanNOT
in,notinMembershiptests
is,isnotIdentitytests
<,<=,>,>=,<>,!=,==Comparisons
|BitwiseOR
^BitwiseXOR
&BitwiseAND
<<,>>Shifts

Page18

APythonBook
+,Additionandsubtraction
*,/,%Multiplication,division,
remainder
+x,xPositive,negative
~xBitwisenot
**Exponentiation
x.attributeAttributereference
x[index]Subscription
x[index:index]Slicing
f(arguments...)Functioncall
(expressions...)Bindingortupledisplay
[expressions...]Listdisplay
{key:datum...}Dictionarydisplay
`expressions...`Stringconversion

Notethatmostoperatorsresultincallstomethodswithspecialnames,for
example__add__,__sub__,__mul__,etc.SeeSpecialmethodnames
http://docs.python.org/2/reference/datamodel.html#specialmethodnames
Later,wewillseehowtheseoperatorscanbeemulatedinclassesthatyoudefine
yourself,throughtheuseofthesespecialnames.

1.2.8Alsosee
FormoreonlexicalmattersandPythonstyles,see:

CodeLikeaPythonista:IdiomaticPython
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html.
StyleGuideforPythonCodehttp://www.python.org/dev/peps/pep0008/
Theflake8stylecheckingprogram.Seehttps://pypi.python.org/pypi/flake8.Also
seethepylintcodechecker:https://pypi.python.org/pypi/pylint.

1.2.9Codeevaluation
UnderstandingthePythonexecutionmodelHowPythonevaluatesandexecutesyour
code.
Evaluatingexpressions.
Creatingnames/variablesBindingThefollowingallcreatenames(variables)and
bindvalues(objects)tothem:(1)assignment,(2)functiondefinition,(3)classdefinition,
(4)functionandmethodcall,(5)importingamodule,...
FirstclassobjectsAlmostallobjectsinPythonarefirstclass.Definition:Anobjectis
firstclassif:(1)wecanputitinastructuredobject;(2)wecanpassittoafunction;(3)
wecanreturnitfromafunction.
ReferencesObjects(orreferencestothem)canbeshared.Whatdoesthismean?

Theobject(s)satisfytheidentitytestoperatoris.
Page19

APythonBook

Thebuiltinfunctionid()returnsthesamevalue.
Theconsequencesformutableobjectsaredifferentfromthoseforimmutable
objects.
Changing(updating)amutableobjectreferencedthroughonevariableor
containeralsochangesthatobjectreferencedthroughothervariablesor
containers,becauseitisthesameobject.
del()Thebuiltinfunctiondel()removesareference,not(necessarily)the
objectitself.

1.3Statementsandinspectionpreliminaries
printExample:
printobj
print"one","two",'three'

for:Example:
stuff=['aa','bb','cc']
foriteminstuff:
printitem

LearnwhatthetypeofanobjectisExample:
type(obj)

Learnwhatattributesanobjecthasandwhatit'scapabilitiesareExample:
dir(obj)
value="amessage"
dir(value)

GethelponaclassoranobjectExample:
help(str)
help("")
value="abc"
help(value)
help(value.upper)

InIPython(butnotstandardPython),youcanalsogethelpattheinteractivepromptby
typing"?"and"??"afteranobject.Example:
In[48]:a=''
In[49]:a.upper?
Type:builtin_function_or_method
StringForm:<builtinmethodupperofstrobjectat0x7f1c426e0508>
Docstring:
S.upper()>string

Page20

APythonBook
ReturnacopyofthestringSconvertedtouppercase.

1.4Builtindatatypes
Forinformationonbuiltindatatypes,seesectionBuiltinTypes
http://docs.python.org/lib/types.htmlinthePythonstandarddocumentation.

1.4.1Numerictypes
Thenumerictypesare:
PlainintegersSameprecisionasaClong,usuallya32bitbinarynumber.
LongintegersDefinewith100L.But,plainintegersareautomatically
promotedwhenneeded.
FloatsImplementedasaCdouble.Precisiondependsonyourmachine.See
sys.float_info.
ComplexnumbersDefinewith,forexample,3jorcomplex(3.0,2.0).
See2.3.4NumericTypesint,float,long,complex
http://docs.python.org/lib/typesnumeric.html.

Pythondoesmixedarithmetic.
Integerdivisiontruncates.ThisischangedinPython3.Usefloat(n)toforcecoercion
toafloat.Example:
In[8]:a=4
In[9]:b=5
In[10]:a/b
Out[10]:0#possiblywrong?
In[11]:float(a)/b
Out[11]:0.8

Applyingthefunctioncalloperator(parentheses)toatypeorclasscreatesaninstanceof
thattypeorclass.
ScientificandheavilynumericprogrammingHighlevelPythonisnotveryefficientfor
numericalprogramming.But,therearelibrariesthathelpNumpyandSciPySee:
SciPy:ScientificToolsforPythonhttp://scipy.org/

1.4.2Tuplesandlists
ListAlistisadynamicarray/sequence.Itisorderedandindexable.Alistismutable.
Listconstructors:[],list().
range()andxrange():
Page21

APythonBook
range(n)createsalistofnintegers.Optionalargumentsarethestartinginteger
andastride.
xrangeislikerange,exceptthatitcreatesaniteratorthatproducestheitems
inthelistofintegersinsteadofthelistitself.
TuplesAtupleisasequence.Atupleisimmutable.

Tupleconstructors:(),butreallyacomma;alsotuple().
Tuplesarelikelists,butarenotmutable.
Pythonlistsare(1)heterogeneous(2)indexable,and(3)dynamic.Forexample,wecan
addtoalistandmakeitlonger.
Notesonsequenceconstructors:
Toconstructatuplewithasingleelement,use(x,);atuplewithasingle
elementrequiresacomma.
Youcanspreadelementsacrossmultiplelines(andnoneedforbackslash
continuationcharacter"\").
Acommacanfollowthelastelement.
Thelengthofatupleorlist(orothercontainer):len(mylist).

Operatorsforlists:

Try:list1+list2,list1*n,list1+=list2,etc.
Comparisonoperators:<,==,>=,etc.
Testformembershipwiththeinoperator.Example:
In[77]:a=[11,22,33]
In[78]:a
Out[78]:[11,22,33]
In[79]:22ina
Out[79]:True
In[80]:44ina
Out[80]:False

Subscription:
Indexingintoasequence
NegativeindexesEffectively,lengthofsequenceplus(minus)index.
SlicingExample:data[2:5].Defaultvalues:beginningandendoflist.
SlicingwithstridesExample:data[::2].
OperationsontuplesNooperationsthatchangethetuple,sincetuplesareimmutable.
Wecandoiterationandsubscription.Wecando"contains"(theinoperator)andgetthe
length(thelen()operator).Wecanusecertainbooleanoperators.

OperationsonlistsOperationssimilartotuplesplus:

Appendmylist.append(newitem).
Page22

APythonBook
Insertmylist.insert(index,newitem).Noteonefficiency:The
insertmethodisnotasfastastheappendmethod.Ifyoufindthatyouneed
todoalargenumberofmylist.insert(0,obj)(thatis,insertingatthe
beginningofthelist)considerusingadequeinstead.See:
http://docs.python.org/2/library/collections.html#collections.deque.Or,use
appendandreverse.
Extendmylist.extend(anotherlist).Alsocanuse+and+=.
Removemylist.remove(item)andmylist.pop().Notethat
append()togetherwithpop()implementsastack.
Deletedelmylist[index].
PopGetlast(rightmost)itemandremovefromlistmylist.pop().
Listoperators+,*,etc.

Formoreoperationsandoperatorsonsequences,see:
http://docs.python.org/2/library/stdtypes.html#sequencetypesstrunicodelisttuplebyte
arraybufferxrange.
Exercises:

Createanemptylist.Append4stringstothelist.Thenpoponeitemofftheend
ofthelist.Solution:
In[25]:a=[]
In[26]:a.append('aaa')
In[27]:a.append('bbb')
In[28]:a.append('ccc')
In[29]:a.append('ddd')
In[30]:printa
['aaa','bbb','ccc','ddd']
In[31]:a.pop()
Out[31]:'ddd'

Usetheforstatementtoprinttheitemsinthelist.Solution:
In[32]:foritemina:
....:printitem
....:
aaa
bbb
ccc

Usethestringjoinoperationtoconcatenatetheitemsinthelist.Solution:
In[33]:'||'.join(a)
Out[33]:'aaa||bbb||ccc'

Uselistscontainingthree(3)elementstocreateandshowatree:
In[37]:b=['bb',None,None]
In[38]:c=['cc',None,None]
In[39]:root=['aa',b,c]

Page23

APythonBook
In[40]:
In[40]:
In[40]:defshow_tree(t):
....:ifnott:
....:return
....:printt[0]
....:show_tree(t[1])
....:show_tree(t[2])
....:
....:
In[41]:show_tree(root)
aa
bb
cc

Notethatwewilllearnabetterwaytorepresenttreestructureswhenwecover
implementingclassesinPython.

1.4.3Strings
Stringsaresequences.Theyareimmutable.Theyareindexable.Theyareiterable.
Foroperationsonstrings,seehttp://docs.python.org/lib/stringmethods.htmloruse:
>>>help(str)

Or:
>>>dir("abc")

Stringoperations(methods).
Stringoperators,e.g.+,<,<=,==,etc..
Constructors/literals:
Quotes:singleanddouble.Escapingquotesandotherspecialcharacterswitha
backslash.
TriplequotingUsetriplesinglequotesordoublequotestodefinemultiline
strings.
str()Theconstructorandthenameofthetype/class.
'aSeparator'.join(aList)
Manymore.
Escapecharactersinstrings\t,\n,\\,etc.

StringformattingSee:
http://docs.python.org/2/library/stdtypes.html#stringformattingoperations
Examples:
In[18]:name='dave'

Page24

APythonBook
In[19]:size=25
In[20]:factor=3.45
In[21]:print'Name:%sSize:%dFactor:%3.4f'%(name,size,
factor,)
Name:daveSize:25Factor:3.4500
In[25]:print'Name:%sSize:%dFactor:%08.4f'%(name,size,
factor,)
Name:daveSize:25Factor:003.4500

Iftherighthandargumenttotheformattingoperatorisadictionary,thenyoucan
(actually,must)usethenamesofkeysinthedictionaryinyourformatstrings.Examples:
In[115]:values={'vegetable':'chard','fruit':'nectarine'}
In[116]:'Ilove%(vegetable)sandIlove%(fruit)s.'%values
Out[116]:'IlovechardandIlovenectarine.'

Alsoconsiderusingtherightjustifyandleftjustifyoperations.Examples:
mystring.rjust(20),mystring.ljust(20,':').
InPython3,thestr.formatmethodispreferredtothestringformattingoperator.
ThismethodisalsoavailableinPython2.7.Ithasbenefitsandadvantagesoverthestring
formattingoperator.Youcanstartlearningaboutithere:
http://docs.python.org/2/library/stdtypes.html#stringmethods
Exercises:

Usealiteraltocreateastringcontaining(1)asinglequote,(2)adoublequote,(3)
bothasingleanddoublequote.Solutions:
"Some'quoted'text."
'Some"quoted"text.'
'Some"quoted"\'extra\'text.'

Writeastringliteralthatspansmultiplelines.Solution:
"""Thisstring
spansseverallines
becauseitisalittlelong.
"""

Usethestringjoinoperationtocreateastringthatcontainsacolonasa
separator.Solution:
>>>content=[]
>>>content.append('finch')
>>>content.append('sparrow')
>>>content.append('thrush')
>>>content.append('jay')
>>>contentstr=':'.join(content)
>>>printcontentstr
finch:sparrow:thrush:jay

Usestringformattingtoproduceastringcontainingyourlastandfirstnames,
Page25

APythonBook
separatedbyacomma.Solution:
>>>first='Dave'
>>>last='Kuhlman'
>>>full='%s,%s'%(last,first,)
>>>printfull
Kuhlman,Dave

IncrementallybuildinguplargestringsfromlotsofsmallstringstheoldwaySince
stringsinPythonareimmutable,appendingtoastringrequiresareallocation.So,itis
fastertoappendtoalist,thenusejoin.Example:
In[25]:strlist=[]
In[26]:strlist.append('Line#1')
In[27]:strlist.append('Line#2')
In[28]:strlist.append('Line#3')
In[29]:str='\n'.join(strlist)
In[30]:printstr
Line#1
Line#2
Line#3

IncrementallybuildinguplargestringsfromlotsofsmallstringsthenewwayThe
+=operationonstringshasbeenoptimized.So,whenyoudothisstr1+=str2,
evenmanytimes,itisefficient.
Thetranslatemethodenablesustomapthecharactersinastring,replacingthosein
onetablebythoseinanother.And,themaketransfunctioninthestringmodule,
makesiteasytocreatethemappingtable:
importstring
deftest():
a='axbycz'
t=string.maketrans('abc','123')
printa
printa.translate(t)
test()

1.4.3.1Thenewstring.formatmethod

Thenewwaytodostringformatting(whichisstandardinPython3andperhaps
preferredfornewcodeinPython2)istousethestring.formatmethod.Seehere:
http://docs.python.org/2/library/stdtypes.html#str.format
http://docs.python.org/2/library/string.html#formatstringsyntax
http://docs.python.org/2/library/string.html#formatspecificationminilanguage
Someexamples:

Page26

APythonBook
In[1]:'aaa{1}bbb{0}ccc{1}ddd'.format('xx','yy',)
Out[1]:'aaayybbbxxcccyyddd'
In[2]:'number:{0:05d}ok'.format(25)
Out[2]:'number:00025ok'
In[4]:'n1:{num1}n2:{num2}'.format(num2=25,num1=100)
Out[4]:'n1:100n2:25'
In[5]:'n1:{num1}n2:{num2}again:{num1}'.format(num2=25,
num1=100)
Out[5]:'n1:100n2:25again:100'
In[6]:'number:{:05d}ok'.format(25)
Out[6]:'number:00025ok'
In[7]:values={'name':'dave','hobby':'birding'}
In[8]:'user:{name}activity:{hobby}'.format(**values)
Out[8]:'user:daveactivity:birding'

1.4.3.2Unicodestrings

Representingunicode:
In[96]:a=u'abcd'
In[97]:a
Out[97]:u'abcd'
In[98]:b=unicode('efgh')
In[99]:b
Out[99]:u'efgh'

Converttounicode:a_string.decode(encoding).Examples:
In[102]:'abcd'.decode('utf8')
Out[102]:u'abcd'
In[103]:
In[104]:'abcd'.decode(sys.getdefaultencoding())
Out[104]:u'abcd'

Convertoutofunicode:a_unicode_string.encode(encoding).Examples:
In[107]:a=u'abcd'
In[108]:a.encode('utf8')
Out[108]:'abcd'
In[109]:a.encode(sys.getdefaultencoding())
Out[109]:'abcd'
In[110]:b=u'Sel\xe7uk'
In[111]:printb.encode('utf8')
Seluk

TestforunicodetypeExample:
In[122]:importtypes
In[123]:a=u'abcd'
In[124]:type(a)istypes.UnicodeType
Out[124]:True
In[125]:

Page27

APythonBook
In[126]:type(a)istype(u'')
Out[126]:True

Orbetter:
In[127]:isinstance(a,unicode)
Out[127]:True

Anexamplewithacharacter"c"withahachek:
In[135]:name='IvanKrsti\xc4\x87'
In[136]:name.decode('utf8')
Out[136]:u'IvanKrsti\u0107'
In[137]:
In[138]:len(name)
Out[138]:12
In[139]:len(name.decode('utf8'))
Out[139]:11

Youcanalsocreateaunicodecharacterbyusingtheunichr()builtinfunction:
In[2]:a='aa'+unichr(170)+'bb'
In[3]:a
Out[3]:u'aa\xaabb'
In[6]:b=a.encode('utf8')
In[7]:b
Out[7]:'aa\xc2\xaabb'
In[8]:printb
aabb

GuidanceforuseofencodingsandunicodeIfyouareworkingwithamultibyte
characterset:
1. Convert/decodefromanexternalencodingtounicodeearly
(my_string.decode(encoding)).
2. Doyourworkinunicode.
3. Convert/encodetoanexternalencodinglate
(my_string.encode(encoding)).
Formoreinformation,see:
UnicodeInPython,CompletelyDemystifiedhttp://farmdev.com/talks/unicode/
PEP100:PythonUnicodeIntegration
http://www.python.org/dev/peps/pep0100/
InthePythonstandardlibrary:
codecsCodecregistryandbaseclasses
http://docs.python.org/2/library/codecs.html#modulecodecs
StandardEncodings
http://docs.python.org/2/library/codecs.html#standardencodings
Ifyouarereadingandwritingmultibytecharacterdatafromortoafile,thenlookatthe

Page28

APythonBook
codecs.open()inthecodecsmodule
http://docs.python.org/2/library/codecs.html#codecs.open.
HandlingmultibytecharactersetsinPython3iseasier,Ithink,butdifferent.Onehintis
tousetheencodingkeywordparametertotheopenbuiltinfunction.Hereisan
example:
deftest():
infile=open('infile1.txt','r',encoding='utf8')
outfile=open('outfile1.txt','w',encoding='utf8')
forlineininfile:
line=line.upper()
outfile.write(line)
infile.close()
outfile.close()
test()

1.4.4Dictionaries
Adictionaryisacollection,whosevaluesareaccessiblebykey.Itisacollectionof
namevaluepairs.
Theorderofelementsinadictionaryisundefined.But,wecaniterateover(1)thekeys,
(2)thevalues,and(3)theitems(keyvaluepairs)inadictionary.Wecansetthevalueof
akeyandwecangetthevalueassociatedwithakey.
Keysmustbeimmutableobjects:ints,strings,tuples,...
Literalsforconstructingdictionaries:
d1={}
d2={key1:value1,key2:value2,}

Constructorfordictionariesdict()canbeusedtocreateinstancesoftheclassdict.
Someexamples:
dict({'one':2,'two':3})
dict({'one':2,'two':3}.items())
dict({'one':2,'two':3}.iteritems())
dict(zip(('one','two'),(2,3)))
dict([['two',3],['one',2]])
dict(one=2,two=3)
dict([(['one','two'][i2],i)foriin(2,3)])

Foroperationsondictionaries,seehttp://docs.python.org/lib/typesmapping.htmloruse:
>>>help({})

Or:
Page29

APythonBook
>>>dir({})

IndexingAccessoradditemstoadictionarywiththeindexingoperator[].Example:
In[102]:dict1={}
In[103]:dict1['name']='dave'
In[104]:dict1['category']=38
In[105]:dict1
Out[105]:{'category':38,'name':'dave'}

Someoftheoperationsproducethekeys,thevalues,andtheitems(pairs)inadictionary.
Examples:
In[43]:d={'aa':111,'bb':222}
In[44]:d.keys()
Out[44]:['aa','bb']
In[45]:d.values()
Out[45]:[111,222]
In[46]:d.items()
Out[46]:[('aa',111),('bb',222)]

Wheniteratingoverlargedictionaries,usemethodsiterkeys(),itervalues(),
anditeritems().Example:
In[47]:
In[47]:d={'aa':111,'bb':222}
In[48]:forkeyind.iterkeys():
....:printkey
....:
....:
aa
bb

Totestfortheexistenceofakeyinadictionary,usetheinoperatororthe
mydict.has_key(k)method.Theinoperatorispreferred.Example:
>>>d={'tomato':101,'cucumber':102}
>>>k='tomato'
>>>kind
True
>>>d.has_key(k)
True

Youcanoftenavoidtheneedforatestbyusingmethodget.Example:
>>>d={'tomato':101,'cucumber':102}
>>>d.get('tomato',1)
101
>>>d.get('chard',1)
1
>>>ifd.get('eggplant')isNone:
...print'missing'

Page30

APythonBook
...
missing

Dictionary"view"objectsprovidedynamic(automaticallyupdated)viewsofthekeysor
thevaluesortheitemsinadictionary.Viewobjectsalsosupportsetoperations.Create
viewswithmydict.viewkeys(),mydict.viewvalues(),and
mydict.viewitems().See:
http://docs.python.org/2/library/stdtypes.html#dictionaryviewobjects.
Thedictionarysetdefaultmethodprovidesawaytogetthevalueassociatedwitha
keyfromadictionaryandtosetthatvalueifthekeyismissing.Example:
In[106]:a
Out[106]:{}
In[108]:a.setdefault('cc',33)
Out[108]:33
In[109]:a
Out[109]:{'cc':33}
In[110]:a.setdefault('cc',44)
Out[110]:33
In[111]:a
Out[111]:{'cc':33}

Exercises:

Writealiteralthatdefinesadictionaryusingbothstringliteralsandvariables
containingstrings.Solution:
>>>first='Dave'
>>>last='Kuhlman'
>>>name_dict={first:last,'Elvis':'Presley'}
>>>printname_dict
{'Dave':'Kuhlman','Elvis':'Presley'}

Writestatementsthatiterateover(1)thekeys,(2)thevalues,and(3)theitemsin
adictionary.(Note:Requiresintroductionoftheforstatement.)Solutions:
>>>d={'aa':111,'bb':222,'cc':333}
>>>forkeyind.keys():
...printkey
...
aa
cc
bb
>>>forvalueind.values():
...printvalue
...
111
333
222
>>>foritemind.items():
...printitem

Page31

APythonBook
...
('aa',111)
('cc',333)
('bb',222)
>>>forkey,valueind.items():
...printkey,'::',value
...
aa::111
cc::333
bb::222

Additionalnotesondictionaries:

Youcanuseiterkeys(),itervalues(),iteritems()toobtain
iteratorsoverkeys,values,anditems.
Adictionaryitselfisiterable:ititeratesoveritskeys.So,thefollowingtwolines
areequivalent:
forkinmyDict:printk
forkinmyDict.iterkeys():printk

Theinoperatortestsforakeyinadictionary.Example:
In[52]:mydict={'peach':'sweet','lemon':'tangy'}
In[53]:key='peach'
In[54]:ifkeyinmydict:
....:printmydict[key]
....:
sweet

1.4.5Files
Openafilewiththeopenfactorymethod.Example:
In[28]:f=open('mylog.txt','w')
In[29]:f.write('message#1\n')
In[30]:f.write('message#2\n')
In[31]:f.write('message#3\n')
In[32]:f.close()
In[33]:f=file('mylog.txt','r')
In[34]:forlineinf:
....:printline,
....:
message#1
message#2
message#3
In[35]:f.close()

Notes:

Usethe(builtin)open(path,mode)functiontoopenafileandcreateafile
object.Youcouldalsousefile(),butopen()isrecommended.
Page32

APythonBook

Afileobjectthatisopenforreadingatextfilesupportstheiteratorprotocoland,
therefore,canbeusedinaforstatement.Ititeratesoverthelinesinthefile.This
ismostlikelyonlyusefulfortextfiles.
openisafactorymethodthatcreatesfileobjects.Useittoopenfilesforreading,
writing,andappending.Examples:
infile=open('myfile.txt','r')#openforreading
outfile=open('myfile.txt','w')#openfor(over)
writing
log=open('myfile.txt','a')#openfor
appendingtoexistingcontent

Whenyouhavefinishedwithafile,closeit.Examples:
infile.close()
outfile.close()

Youcanalsousethewith:statementtoautomaticallyclosethefile.Example:
withopen('tmp01.txt','r')asinfile:
forxininfile:
printx,

Theaboveworksbecauseafileisacontextmanager:itobeysthecontext
managerprotocol.Afilehasmethods__enter__and__exit__,andthe
__exit__methodautomaticallyclosesthefileforus.Seethesectiononthe
with:statement.
Inordertoopenmultiplefiles,youcannestwith:statements,oruseasingle
with:statementwithmultiple"expressionastarget"clauses.Example:
deftest():
#
#usemultiplenestedwith:statements.
withopen('small_file.txt','r')asinfile:
withopen('tmp_outfile.txt','w')asoutfile:
forlineininfile:
outfile.write('line:%s'%
line.upper())
printinfile
printoutfile
#
#useasinglewith:statement.
withopen('small_file.txt','r')asinfile,\
open('tmp_outfile.txt','w')asoutfile:
forlineininfile:
outfile.write('line:%s'%line.upper())
printinfile
printoutfile
test()

fileisthefiletypeandcanbeusedasaconstructortocreatefileobjects.But,
Page33

APythonBook
openispreferred.
Linesreadfromatextfilehaveanewline.Stripitoffwithsomethinglike:
line.rstrip('\n').
Forbinaryfilesyoushouldaddthebinarymode,forexample:rb,wb.Formore
aboutmodes,seethedescriptionoftheopen()functionatBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.html.
Learnmoreaboutfileobjectsandthemethodstheyprovideat:2.3.9FileObjects
http://docs.python.org/2/library/stdtypes.html#fileobjects.
Youcanalsoappendtoanexistingfile.Notethe"a"modeinthefollowingexample:
In[39]:f=open('mylog.txt','a')
In[40]:f.write('message#4\n')
In[41]:f.close()
In[42]:f=file('mylog.txt','r')
In[43]:forlineinf:
....:printline,
....:
message#1
message#2
message#3
message#4
In[44]:f.close()

Forbinaryfiles,add"b"tothemode.NotstrictlynecessaryonUNIX,butneededonMS
Windows.And,youwillwanttomakeyourcodeportableacrossplatforms.Example:
In[62]:importzipfile
In[63]:outfile=open('tmp1.zip','wb')
In[64]:zfile=zipfile.ZipFile(outfile,'w',zipfile.ZIP_DEFLATED)
In[65]:zfile.writestr('entry1','myheroeshavealwaysbeen
cowboys')
In[66]:zfile.writestr('entry2','andtheystillareitseems')
In[67]:zfile.writestr('entry3','sadlyinsearchofand')
In[68]:zfile.writestr('entry4','onstepinbackof')
In[69]:
In[70]:zfile.writestr('entry4','onestepinbackof')
In[71]:zfile.writestr('entry5','themselvesandtheirslowmoving
ways')
In[72]:zfile.close()
In[73]:outfile.close()
In[75]:
$
$unziplvtmp1.zip
Archive:tmp1.zip
LengthMethodSizeRatioDateTimeCRC32Name

34Defl:N366%05290817:04f6b7d921entry1
27Defl:N297%05290817:0710da8f3dentry2
22Defl:N249%05290817:073fd17fdaentry3
18Defl:N2011%05290817:08d55182e6entry4

Page34

APythonBook
19Defl:N2111%05290817:081a892acdentry4
37Defl:N395%05290817:09e213708centry5

1571698%6files

Exercises:

Readallofthelinesofafileintoalist.Printthe3rdand5thlinesinthefile/list.
Solution:
In[55]:f=open('tmp1.txt','r')
In[56]:lines=f.readlines()
In[57]:f.close()
In[58]:lines
Out[58]:['the\n','big\n','brown\n','dog\n',
'had\n','long\n','hair\n']
In[59]:printlines[2]
brown
In[61]:printlines[4]
had

Morenotes:

Stripnewlines(andotherwhitespace)fromastringwithmethodsstrip(),
lstrip(),andrstrip().
Getthecurrentpositionwithinafilebyusingmyfile.tell().
Setthecurrentpositionwithinafilebyusingmyfile.seek().Itmaybe
helpfultouseos.SEEK_CURandos.SEEK_END.Forexample:
f.seek(2,os.SEEK_CUR)advancesthepositionbytwo
f.seek(3,os.SEEK_END)setsthepositiontothethirdtolast.
f.seek(25)setsthepositionrelativetothebeginningofthefile.

1.4.6Otherbuiltintypes
OtherbuiltindatatypesaredescribedinsectionBuiltinTypes
http://docs.python.org/lib/types.htmlinthePythonstandarddocumentation.
1.4.6.1TheNonevalue/type

TheuniquevalueNoneisusedtoindicate"novalue","nothing","nonexistence",etc.
ThereisonlyoneNonevalue;inotherwords,it'sasingleton.
UseistotestforNone.Example:
>>>flag=None
>>>
>>>ifflagisNone:
...print'clear'

Page35

APythonBook
...
clear
>>>ifflagisnotNone:
...print'hello'
...
>>>

1.4.6.2Booleanvalues

TrueandFalsearethebooleanvalues.
Thefollowingvaluesalsocountasfalse,forexample,inanif:statement:False,
numericzero,None,theemptystring,anemptylist,anemptydictionary,anyempty
container,etc.Allothervalues,includingTrue,actastruevalues.
1.4.6.3Setsandfrozensets

Asetisanunorderedcollectionofimmutableobjects.Asetdoesnotcontainduplicates.
Setssupportseveralsetoperations,forexample:union,intersection,difference,...
Afrozensetislikeaset,exceptthatafrozensetisimmutable.Therefore,afrozensetis
hashableandcanbeusedasakeyinadictionary,anditcanbeaddedtoaset.
Createasetwiththesetconstructor.Examples:
>>>a=set()
>>>a
set([])
>>>a.add('aa')
>>>a.add('bb')
>>>a
set(['aa','bb'])
>>>b=set([11,22])
>>>b
set([11,22])
>>>c=set([22,33])
>>>b.union(c)
set([33,11,22])
>>>b.intersection(c)
set([22])

Formoreinformationonsets,see:SetTypesset,frozenset
http://docs.python.org/lib/typesset.html

1.5FunctionsandClassesAPreview
StructuredcodePythonprogramsaremadeupofexpressions,statements,functions,
classes,modules,andpackages.
Page36

APythonBook
Pythonobjectsarefirstclassobjects.
Expressionsareevaluated.
Statementsareexecuted.
Functions(1)areobjectsand(2)arecallable.
ObjectorientedprogramminginPython.Modeling"realworld"objects.(1)
Encapsulation;(2)datahiding;(3)inheritance.Polymorphism.
Classes(1)encapsulation;(2)datahiding;(3)inheritance.
Anoverviewofthestructureofatypicalclass:(1)methods;(2)theconstructor;(3)class
(static)variables;(4)super/subclasses.

1.6Statements
1.6.1Assignmentstatement
Formtarget=expression.
Possibletargets:

Identifier
TupleorlistCanbenested.Leftandrightsidesmusthaveequivalentstructure.
Example:
>>>x,y,z=11,22,33
>>>[x,y,z]=111,222,333
>>>a,(b,c)=11,(22,33)
>>>a,B=11,(22,33)

Thisfeaturecanbeusedtosimulateanenum:
In[22]:LITTLE,MEDIUM,LARGE=range(1,4)
In[23]:LITTLE
Out[23]:1
In[24]:MEDIUM
Out[24]:2

Subscriptionofasequence,dictionary,etc.Example:
In[10]:a=range(10)
In[11]:a
Out[11]:[0,1,2,3,4,5,6,7,8,9]
In[12]:a[3]='abc'
In[13]:a
Out[13]:[0,1,2,'abc',4,5,6,7,8,9]
In[14]:
In[14]:b={'aa':11,'bb':22}
In[15]:b

Page37

APythonBook
Out[15]:{'aa':11,'bb':22}
In[16]:b['bb']=1000
In[17]:b['cc']=2000
In[18]:b
Out[18]:{'aa':11,'bb':1000,'cc':2000}

AsliceofasequenceNotethatthesequencemustbemutable.Example:
In[1]:a=range(10)
In[2]:a
Out[2]:[0,1,2,3,4,5,6,7,8,9]
In[3]:a[2:5]=[11,22,33,44,55,66]
In[4]:a
Out[4]:[0,1,11,22,33,44,55,66,5,6,7,8,9]

AttributereferenceExample:
>>>classMyClass:
...pass
...
>>>anObj=MyClass()
>>>anObj.desc='pretty'
>>>printanObj.desc
pretty

Thereisalsoaugmentedassignment.Examples:
>>>index=0
>>>index+=1
>>>index+=5
>>>index+=f(x)
>>>index=1
>>>index*=3

Thingstonote:

Assignmenttoanamecreatesanewvariable(ifitdoesnotexistinthe
namespace)andabinding.Specifically,itbindsavaluetothenewname.Calling
afunctionalsodoesthistothe(formal)parameterswithinthelocalnamespace.
InPython,alanguagewithdynamictyping,thedatatypeisassociatedwiththe
value,notthevariable,asisthecaseinstaticallytypedlanguages.
Assignmentcanalsocausesharingofanobject.Example:
obj1=A()
obj2=obj1

Checktodeterminethatthesameobjectissharedwithid(obj)ortheis
operator.Example:
In[23]:a=range(10)
In[24]:a
Out[24]:[0,1,2,3,4,5,6,7,8,9]
In[25]:b=a
In[26]:b

Page38

APythonBook
Out[26]:[0,1,2,3,4,5,6,7,8,9]
In[27]:b[3]=333
In[28]:b
Out[28]:[0,1,2,333,4,5,6,7,8,9]
In[29]:a
Out[29]:[0,1,2,333,4,5,6,7,8,9]
In[30]:aisb
Out[30]:True
In[31]:printid(a),id(b)
3103792031037920

Youcanalsodomultipleassignmentinasinglestatement.Example:
In[32]:a=b=123
In[33]:a
Out[33]:123
In[34]:b
Out[34]:123
In[35]:
In[35]:
In[35]:a=b=[11,22]
In[36]:aisb
Out[36]:True

Youcaninterchange(swap)thevalueoftwovariablesusingassignmentand
packing/unpacking:
>>>a=111
>>>b=222
>>>a,b=b,a
>>>a
222
>>>b
111

1.6.2importstatement
Makemodule(orobjectsinthemodule)available.
Whatimportdoes:

Evaluatethecontentofamodule.
Likelytocreatevariablesinthelocal(module)namespace.
Evaluationofaspecificmoduleonlyhappensonceduringagivenrunofthe
program.Therefore,amoduleissharedacrossanapplication.
Amoduleisevaluatedfromtoptobottom.Laterstatementscanreplacevalues
createdearlier.Thisistrueoffunctionsandclasses,aswellas(other)variables.
Whichstatementsareevaluated?Assignment,class,def,...
Usethefollowingidiomtomakeamodulebothrunableandimportable:
if__name__=='__main__':

Page39

APythonBook
#importpdb;pdb.set_trace()
main()#or"test()"orsomeotherfunction
definedinmodule

Notes:
Theaboveconditionwillbetrueonlywhenthemoduleisrunasascriptand
willnotbetruewhenthemoduleisimported.
Thelinecontainingpdbcanbecopiedanyplaceinyourprogramand
uncommented,andthentheprogramwilldropintothePythondebugger
whenthatlocationisreached.
Whereimportlooksformodules:

sys.pathshowswhereitlooks.
Therearesomestandardplaces.
AddadditionaldirectoriesbysettingtheenvironmentvariablePYTHONPATH.
Youcanalsoaddpathsbymodifyingsys.path,forexample:
importsys
sys.path.insert(0,'/path/to/my/module')

Packagesneedafilenamed__init__.py.
ExtensionsTodeterminewhatextensionsimportlooksfor,do:
>>>importimp
>>>imp.get_suffixes()
[('.so','rb',3),('module.so','rb',3),('.py','U',
1),('.pyc','rb',2)]

Formsoftheimportstatement:
importANamesinthelocal(module)namespaceareaccessiblewiththedot
operator.
importAasBImportthemoduleA,butbindthemoduleobjecttothe
variableB.
importA1,A2Notrecommended
fromAimportB
fromAimportB1,B2
fromAimportBasC
fromAimport*Notrecommended:cluttersandmixesnamespaces.
fromA.BimportC(1)PossiblyimportobjectCfrommoduleBin
packageAor(2)possiblyimportmoduleCfromsubpackageBinpackageA.
importA.B.CToreferenceattributesinC,mustusefullyqualifiedname,
forexampleuseA.B.C.DtoreferenceDinsideofC.
Morenotesontheimportstatement:

TheimportstatementandpackagesAfilenamed__init__.pyisrequiredin
apackage.Thisfileisevaluatedthefirsttimeeitherthepackageisimportedora
Page40

APythonBook

fileinthepackageisimported.Question:Whatismadeavailablewhenyoudo
importaPackage?Answer:Allvariables(names)thatareglobalinsidethe
__init__.pymoduleinthatpackage.But,seenotesontheuseof__all__:
Theimportstatementhttp://docs.python.org/ref/import.html
Theuseofif__name__=="__main__":Makesamoduleboth
importableandexecutable.
UsingdotsintheimportstatementFromthePythonlanguagereferencemanual:
"Hierarchicalmodulenames:whenthemodulenamescontains
oneormoredots,themodulesearchpathiscarriedout
differently.Thesequenceofidentifiersuptothelastdotisused
tofindapackage;thefinalidentifieristhensearchedinside
thepackage.Apackageisgenerallyasubdirectoryofa
directoryonsys.paththathasafile__init__.py."

See:Theimportstatementhttp://docs.python.org/ref/import.html
Exercises:

Importamodulefromthestandardlibrary,forexamplere.
Importanelementfromamodulefromthestandardlibrary,forexampleimport
compilefromtheremodule.
CreateasimplePythonpackagewithasinglemoduleinit.Solution:
1. Createadirectorynamedsimplepackageinthecurrentdirectory.
2. Createan(empty)__init__.pyinthenewdirectory.
3. Createansimple.pyinthenewdirectory.
4. Addasimplefunctionnametest1insimple.py.
5. Importusinganyofthefollowing:
>>>importsimplepackage.simple
>>>fromsimplepackageimportsimple
>>>fromsimplepackage.simpleimporttest1
>>>fromsimplepackage.simpleimporttest1asmytest

1.6.3printstatement
printsendsoutputtosys.stdout.Itaddsanewline,unlessanextracommais
added.
Argumentstoprint:

MultipleitemsSeparatedbycommas.
Endwithcommatosuppresscarriagereturn.
Usestringformattingformorecontroloveroutput.
Alsoseevarious"prettyprinting"functionsandmethods,inparticular,pprint.
See3.27pprintDataprettyprinter
Page41

APythonBook
http://docs.python.org/lib/modulepprint.html.
StringformattingArgumentsareatuple.Reference:2.3.6.2StringFormatting
Operationshttp://docs.python.org/lib/typesseqstrings.html.
Canalsousesys.stdout.Notethatacarriagereturnisnotautomaticallyadded.
Example:
>>>importsys
>>>sys.stdout.write('hello\n')

ControllingthedestinationandformatofprintReplacesys.stdoutwithaninstance
ofanyclassthatimplementsthemethodwritetakingoneparameter.Example:
importsys
classWriter:
def__init__(self,file_name):
self.out_file=file(file_name,'a')
defwrite(self,msg):
self.out_file.write('[[%s]]'%msg)
defclose(self):
self.out_file.close()
deftest():
writer=Writer('outputfile.txt')
save_stdout=sys.stdout
sys.stdout=writer
print'hello'
print'goodbye'
writer.close()
#Showtheoutput.
tmp_file=file('outputfile.txt')
sys.stdout=save_stdout
content=tmp_file.read()
tmp_file.close()
printcontent
test()

Thereisanalternativeformoftheprintstatementthattakesafilelikeobject,in
particularanobjectthathasawritemethod.Forexample:
In[1]:outfile=open('tmp.log','w')
In[2]:print>>outfile,'Message#1'
In[3]:print>>outfile,'Message#2'
In[4]:print>>outfile,'Message#3'
In[5]:outfile.close()
In[6]:
In[6]:infile=open('tmp.log','r')
In[7]:forlineininfile:
...:print'Line:',line.rstrip('\n')
...:

Page42

APythonBook
Line:Message#1
Line:Message#2
Line:Message#3
In[8]:infile.close()

FuturedeprecationwarningThereisnoprintstatementinPython3.Thereisaprint
builtinfunction.

1.6.4if:elif:else:statement
Atemplatefortheif:statement:
ifcondition1:
statements
elifcondition2:
statements
elifcondition3:
statements
else:
statements

Theelifandelseclausesareoptional.
ConditionsExpressionsAnythingthatreturnsavalue.Comparewitheval()and
exec.
Truthvalues:
FalseFalse,None,numericzero,theemptystring,anemptycollection(list
ortupleordictionaryor...).
TrueTrueandeverythingelse.
Operators:

andandorNotethatbothandandordoshortcircuitevaluation.
not
isandisnotTheidenticalobject.Cf.aisbandid(a)==id(b).
UsefultotestforNone,forexample:
ifxisNone:
...
ifxisnotNone:
...

inandnotinCanbeusedtotestforexistenceofakeyinadictionaryorfor
thepresenceofavalueinacollection.
Theinoperatortestsforequality,notidentity.
Example:
>>>d={'aa':111,'bb':222}
>>>'aa'ind

Page43

APythonBook
True
>>>'aa'notind
False
>>>'xx'ind
False

Comparisonoperators,forexample==,!=,<,<=,...
Thereisanifexpression.Example:

>>>a='aa'
>>>b='bb'
>>>x='yes'ifa==belse'no'
>>>x
'no'

Notes:
Theelif:clausesandtheelse:clauseareoptional.
Theif:,elif:,andelse:clausesareallheaderlinesinthesensethatthey
areeachfollowedbyanindentedblockofcodeandeachoftheseheaderlines
endswithacolon.(Toputanemptyblockafteroneofthese,oranyother,
statementheaderline,usethepassstatement.It'seffectivelyanoop.)
Parenthesesaroundtheconditioninanif:orelif:arenotrequiredandare
consideredbadform,unlesstheconditionextendsovermultiplelines,inwhich
caseparenthesesarepreferredoveruseofalinecontinuationcharacter(backslash
attheendoftheline).
Exercises:

Writeanifstatementwithanandoperator.
Writeanifstatementwithanoroperator.
Writeanifstatementcontainingbothandandoroperators.

1.6.5for:statement
Iterateoverasequenceoran"iterable"object.
Form:
forxiny:
block

IteratorSomenotesonwhatitmeanstobeiterable:

Aniterableissomethingthatcanbeusedinaniteratorcontext,forexample,ina
for:statement,inalistcomprehension,andinageneratorexpression.
Sequencesandcontainersareiterable.Examples:tuples,lists,strings,
dictionaries.
Instancesofclassesthatobeytheiteratorprotocolareiterable.See
Page44

APythonBook
http://docs.python.org/lib/typeiter.html.
Wecancreateaniteratorobjectwithbuiltinfunctionssuchasiter()and
enumerate().SeeBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.htmlinthePythonstandardlibrary
reference.
Functionsthatusetheyieldstatement,produceaniterator,althoughit'sactually
calledagenerator.
Aniterableimplementstheiteratorinterfaceandsatisfiestheiteratorprotocol.
Theiteratorprotocol:__iter__()andnext()methods.See2.3.5Iterator
Types(http://docs.python.org/lib/typeiter.html).
Testingfor"iterability":
Ifyoucanuseanobjectinafor:statement,it'siterable.
Iftheexpresioniter(obj)doesnotproduceaTypeErrorexception,it's
iterable.
Somewaystoproduceiterators:

iter()andenumerate()See:
http://docs.python.org/lib/builtinfuncs.html.
some_dict.iterkeys(),some_dict.itervalues(),
some_dict.iteritems().
Useasequenceinaniteratorcontext,forexampleinaforstatement.Lists,
tuples,dictionaries,andstringscanbeusedinaniteratorcontexttoproducean
iterator.
GeneratorexpressionsLatestPythononly.Syntacticallylikelist
comprehensions,but(1)surroundedbyparenthesesinsteadofsquarebracketsand
(2)uselazyevaluation.
AclassthatimplementstheiteratorprotocolExample:
classA(object):
def__init__(self):
self.data=[11,22,33]
self.idx=0
def__iter__(self):
returnself
defnext(self):
ifself.idx<len(self.data):
x=self.data[self.idx]
self.idx+=1
returnx
else:
raiseStopIteration
deftest():
a=A()
forxina:

Page45

APythonBook
printx
test()

NotethattheiteratorprotocolchangesinPython3.
Afunctioncontainingayieldstatement.See:
Yieldexpressions
http://docs.python.org/2/reference/expressions.html#yieldexpressions
Theyieldstatement
http://docs.python.org/2/reference/simple_stmts.html#theyieldstatement
AlsoseeitertoolsmoduleinthePythonstandardlibraryformuchmorehelp
withiterators:itertoolsFunctionscreatingiteratorsforefficientlooping
http://docs.python.org/2/library/itertools.html#moduleitertools
Thefor:statementcanalsodounpacking.Example:

In[25]:items=['apple','banana','cherry','date']
In[26]:foridx,iteminenumerate(items):
....:print'%d.%s'%(idx,item,)
....:
0.apple
1.banana
2.cherry
3.date

Theforstatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executediftheforstatementcompletesnormally,thatisifabreakstatementisnot
executed.
Helpfulfunctionswithfor:

enumerate(iterable)Returnsaniterablethatproducespairs(tuples)
containingcount(index)andvalue.Example:
>>>foridx,valueinenumerate([11,22,33]):
...printidx,value
...
011
122
233

range([start,]stop[,step])andxrange([start,]stop[,
step]).
ListcomprehensionsSincelistcomprehensionscreatelists,theyareusefulinfor
statements,although,whenthenumberofelementsislarge,youshouldconsiderusinga
generatorexpressioninstead.Alistcomprehensionlooksabitlikeafor:statement,but
isinsidesquarebrackets,anditisanexpression,notastatement.Twoforms(among
others):

[f(x)forxiniterable]
Page46

APythonBook
[f(x)forxiniterableift(x)]
GeneratorexpressionsAgeneratorexpressionlookssimilartoalistcomprehension,
exceptthatitissurroundedbyparenthesesratherthansquarebrackets.Example:

In[28]:items=['apple','banana','cherry','date']
In[29]:gen1=(item.upper()foriteminitems)
In[30]:forxingen1:
....:print'x:',x
....:
x:APPLE
x:BANANA
x:CHERRY
x:DATE

Exercises:

Writealistcomprehensionthatreturnsallthekeysinadictionarywhose
associatedvaluesaregreaterthanzero.
Thedictionary:{'aa':11,'cc':33,'dd':55,'bb':22}
Solution:[x[0]forxinmy_dict.iteritems()ifx[1]>
0]
Writealistcomprehensionthatproducesevenintegersfrom0to10.Useafor
statementtoiterateoverthosevalues.Solution:
forxin[yforyinrange(10)ify%2==0]:
print'x:%s'%x

Writealistcomprehensionthatiteratesovertwolistsandproducesallthe
combinationsofitemsfromthelists.Solution:
In[19]:a=range(4)
In[20]:b=[11,22,33]
In[21]:a
Out[21]:[0,1,2,3]
In[22]:b
Out[22]:[11,22,33]
In[23]:c=[(x,y)forxinaforyinb]
In[24]:printc
[(0,11),(0,22),(0,33),(1,11),(1,22),(1,33),
(2,11),(2,22),(2,33),(3,11),(3,22),(3,33)]

But,notethatinthepreviousexercise,ageneratorexpressionwouldoftenbebetter.A
generatorexpressionislikealistcomprehension,exceptthat,insteadofcreatingthe
entirelist,itproducesageneratorthatcanbeusedtoproduceeachoftheelements.
Thebreakandcontinuestatementsareoftenusefulinaforstatement.Seecontinue
andbreakstatements
Theforstatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executediftheforstatementcompletesnormally,thatisifabreakstatementisnot
executed.Example:
Page47

APythonBook
foritemindata1:
ifitem>100:
value1=item
break
else:
value1='notfound'
print'value1:',value1

Whenrun,itprints:
value1:notfound

1.6.6while:statement
Form:
whilecondition:
block

Thewhile:statementisnotoftenusedinPythonbecausethefor:statementis
usuallymoreconvenient,moreidiomatic,andmorePythonic.
Exercises:

Writeawhilestatementthatprintsintegersfromzeroto5.Solution:
count=0
whilecount<5:
count+=1
printcount

Thebreakandcontinuestatementsareoftenusefulinawhilestatement.See
continueandbreakstatements
Thewhilestatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executedifthewhilestatementcompletesnormally,thatisifabreakstatementisnot
executed.

1.6.7continueandbreakstatements
Thebreakstatementexitsfromaloop.
Thecontinuestatementcausesexecutiontoimmediatelycontinueatthestartofthe
loop.
Canbeusedinfor:andwhile:.
Whenthefor:statementorthewhile:statementhasanelse:clause,theblockin
theelse:clauseisexecutedonlyifabreakstatementisnotexecuted.
Exercises:
Page48

APythonBook

Usingbreak,writeawhilestatementthatprintsintegersfromzeroto5.
Solution:
count=0
whileTrue:
count+=1
ifcount>5:
break
printcount

Notes:
Aforstatementthatusesrange()orxrange()wouldbebetterthana
whilestatementforthisuse.
Usingcontinue,writeawhilestatementthatprocessesonlyevenintegers
from0to10.Note:%isthemodulooperator.Solution:
count=0
whilecount<10:
count+=1
ifcount%2==0:
continue
printcount

1.6.8try:except:statement
Exceptionsareasystematicandconsistentwayofprocessingerrorsand"unusual"events
inPython.
CaughtanduncaughtexceptionsUncaughtexceptionsterminateaprogram.
Thetry:statementcatchesanexception.
AlmostallerrorsinPythonareexceptions.
Evaluation(executionmodel)ofthetrystatementWhenanexceptionoccursinthe
tryblock,evenifinsideanestedfunctioncall,executionofthetryblockendsandthe
exceptclausesaresearchedforamatchingexception.
TracebacksAlsoseethetracebackmodule:
http://docs.python.org/lib/moduletraceback.html
Exceptionsareclasses.
Exceptionclassessubclassing,args.
Anexceptionclassinanexcept:clausecatchesinstancesofthatexceptionclassand
allsubclasses,butnotsuperclasses.
BuiltinexceptionclassesSee:

Moduleexceptions.
Page49

APythonBook
Builtinexceptionshttp://docs.python.org/lib/moduleexceptions.html.
UserdefinedexceptionclassessubclassesofException.

Example:
try:
raiseRuntimeError('thissillyerror')
exceptRuntimeError,exp:
print"[[[%s]]]"%exp

Reference:http://docs.python.org/lib/moduleexceptions.html
Youcanalsogettheargumentspassedtotheconstructorofanexceptionobject.Inthe
aboveexample,thesewouldbe:
exp.args

Whywouldyoudefineyourownexceptionclass?Oneanswer:Youwantauserofyour
codetocatchyourexceptionandnoothers.
Catchinganexceptionbyexceptionclasscatchesexceptionsofthatclassandallits
subclasses.So:
exceptSomeExceptionClass,exp:

matchesandcatchesanexceptionifSomeExceptionClassistheexceptionclassorabase
class(superclass)oftheexceptionclass.Theexceptionobject(usuallyaninstanceof
someexceptionclass)isboundtoexp.
Amore"modern"syntaxis:
exceptSomeExceptionClassasexp:

So:
classMyE(ValueError):
pass
try:
raiseMyE()
exceptValueError:
print'caughtexception'

willprint"caughtexception",becauseValueErrorisabaseclassofMyE.
Alsoseetheentriesfor"EAFP"and"LBYL"inthePythonglossary:
http://docs.python.org/3/glossary.html.
Exercises:

Writeaverysimple,emptyexceptionsubclass.Solution:
classMyE(Exception):

Page50

APythonBook
pass

Writeatry:except:statementthatraisesyourexceptionandalsocatchesit.
Solution:
try:
raiseMyE('hellotheredave')
exceptMyE,e:
printe

1.6.9raisestatement
Throworraiseanexception.
Forms:
raiseinstance
raiseMyExceptionClass(value)preferred.
raiseMyExceptionClass,value
Theraisestatementtakes:

An(instanceof)abuiltinexceptionclass.
AninstanceofclassExceptionor
AninstanceofabuiltinsubclassofclassExceptionor
AninstanceofauserdefinedsubclassofclassExceptionor
Oneoftheaboveclassesand(optionally)avalue(forexample,astringora
tuple).
Seehttp://docs.python.org/ref/raise.html.

Foralistofbuiltinexceptions,seehttp://docs.python.org/lib/moduleexceptions.html.
Thefollowingexampledefinesanexceptionsubclassandthrowsaninstanceofthat
subclass.Italsoshowshowtopassandcatchmultipleargumentstotheexception:
classNotsobadError(Exception):
pass
deftest(x):
try:
ifx==0:
raiseNotsobadError('amoderatelybaderror','nottoo
bad')
exceptNotsobadError,e:
print'Errorargs:%s'%(e.args,)
test(0)

Whichprintsoutthefollowing:
Errorargs:('amoderatelybaderror','nottoobad')

Page51

APythonBook
Notes:
Inordertopassinmultipleargumentswiththeexception,weuseatuple,orwe
passmultipleargumentstotheconstructor.
Thefollowingexampledoesasmallamountofprocessingofthearguments:

classNotsobadError(Exception):
"""Anexceptionclass.
"""
defget_args(self):
return'::::'.join(self.args)
deftest(x):
try:
ifx==0:
raiseNotsobadError('amoderatelybaderror','nottoo
bad')
exceptNotsobadError,e:
print'Errorargs:{{{%s}}}'%(e.get_args(),)
test(0)

1.6.10with:statement
Thewithstatementenablesustouseacontextmanager(anyobjectthatsatisfiesthe
contextmanagerprotocol)toaddcodebefore(onentryto)andafter(onexitfrom)a
blockofcode.
1.6.10.1Writingacontextmanager

Acontextmanagerisaninstanceofaclassthatsatisfiesthisinterface:
classContext01(object):
def__enter__(self):
pass
def__exit__(self,exc_type,exc_value,traceback):
pass

Hereisanexamplethatusestheabovecontextmanager:
classContext01(object):
def__enter__(self):
print'in__enter__'
return'somevalueorother'#usuallywewanttoreturn
self
def__exit__(self,exc_type,exc_value,traceback):
print'in__exit__'

Notes:

Page52

APythonBook

The__enter__methodiscalledbeforeourblockofcodeisentered.
Usually,butnotalways,wewillwantthe__enter__methodtoreturnself,
thatis,theinstanceofourcontextmanagerclass.Wedothissothatwecanwrite:
withMyContextManager()asobj:
pass

andthenusetheinstance(objinthiscase)inthenestedblock.
The__exit__methodiscalledwhenourblockofcodeisexitedeither
normallyorbecauseofanexception.
Ifanexceptionissupplied,andthemethodwishestosuppresstheexception(i.e.,
preventitfrombeingpropagated),itshouldreturnatruevalue.Otherwise,the
exceptionwillbeprocessednormallyuponexitfromthismethod.
Iftheblockexitsnormally,thevalueofexc_type,exc_value,and
tracebackwillbeNone.
Formoreinformationonthewith:statement,seeContextManagerTypes
http://docs.python.org/2/library/stdtypes.html#contextmanagertypes.

Seemodulecontextlibforstrangewaysofwritingcontextmanagers:
http://docs.python.org/2/library/contextlib.html#modulecontextlib
1.6.10.2Usingthewith:statement

Hereareexamples:
#example1
withContext01():
print'inbody'
#example2
withContext02()asa_value:
print'inbody'
print'a_value:"%s"'%(a_value,)
a_value.some_method_in_Context02()
#example3
withopen(infilename,'r')asinfile,open(outfilename,'w')as
outfile:
forlineininfile:
line=line.rstrip()
outfile.write('%s\n'%line.upper())

Notes:

Intheformwith...asval,thevaluereturnedbythe__enter__
methodisassignedtothevariable(valinthiscase).
Inordertousemorethanonecontextmanager,youcannestwith:statements,
orseparateusesofofthecontextmanagerswithcommas,whichisusually
Page53

APythonBook
preferred.Seeexample3above.

1.6.11del
Thedelstatementcanbeusedto:
Removenamesfromnamespace.
Removeitemsfromacollection.
Ifnameislistedinaglobalstatement,thendelremovesnamefromtheglobal
namespace.

Namescanbea(nested)list.Examples:
>>>dela
>>>dela,b,c

Wecanalsodeleteitemsfromalistordictionary(andperhapsfromotherobjectsthatwe
cansubscript).Examples:
In[9]:d={'aa':111,'bb':222,'cc':333}
In[10]:printd
{'aa':111,'cc':333,'bb':222}
In[11]:deld['bb']
In[12]:printd
{'aa':111,'cc':333}
In[13]:
In[13]:a=[111,222,333,444]
In[14]:printa
[111,222,333,444]
In[15]:dela[1]
In[16]:printa
[111,333,444]

And,wecandeleteanattributefromaninstance.Example:
In[17]:classA:
....:pass
....:
In[18]:a=A()
In[19]:a.x=123
In[20]:dir(a)
Out[20]:['__doc__','__module__','x']
In[21]:printa.x
123
In[22]:dela.x
In[23]:dir(a)
Out[23]:['__doc__','__module__']
In[24]:printa.x

exceptions.AttributeErrorTraceback(mostrecentcalllast)

Page54

APythonBook
/home/dkuhlman/a1/Python/Test/<console>
AttributeError:Ainstancehasnoattribute'x'

1.6.12casestatement
ThereisnocasestatementinPython.Usetheif:statementwithasequenceofelif:
clauses.Or,useadictionaryoffunctions.

1.7Functions,Modules,Packages,andDebugging
1.7.1Functions
1.7.1.1Thedefstatement

Thedefstatementisusedtodefinefunctionsandmethods.
Thedefstatementisevaluated.Itproducesafunction/method(object)andbindsittoa
variableinthecurrentnamespace.
Althoughthedefstatementisevaluated,thecodeinitsnestedblockisnotexecuted.
Therefore,manyerrorsmaynotbedetecteduntileachandeverypaththroughthatcodeis
tested.Recommendations:(1)UseaPythoncodechecker,forexampleflake8or
pylint;(2)DothoroughtestingandusethePythonunittestframework.Pythonic
wisdom:Ifit'snottested,it'sbroken.
1.7.1.2Returningvalues

Thereturnstatementisusedtoreturnvaluesfromafunction.
Thereturnstatementtakeszeroormorevalues,separatedbycommas.Usingcommas
actuallyreturnsasingletuple.
ThedefaultvalueisNone.
Toreturnmultiplevalues,useatupleorlist.Don'tforgetthat(assignment)unpacking
canbeusedtocapturemultiplevalues.Returningmultipleitemsseparatedbycommasis
equivalenttoreturningatuple.Example:
In[8]:deftest(x,y):
...:returnx*3,y*4
...:
In[9]:a,b=test(3,4)
In[10]:printa
9

Page55

APythonBook
In[11]:printb
16

1.7.1.3Parameters

DefaultvaluesExample:
In[53]:deft(max=5):
....:forvalinrange(max):
....:printval
....:
....:
In[54]:t(3)
0
1
2
In[55]:t()
0
1
2
3
4

Givingaparameteradefaultvaluemakesthatparameteroptional.
Note:Ifafunctionhasaparameterwithadefaultvalue,thenall"normal"arguments
mustproceedtheparameterswithdefaultvalues.Morecompletely,parametersmustbe
givenfromlefttorightinthefollowingorder:
1. Normalarguments.
2. Argumentswithdefaultvalues.
3. Argumentlist(*args).
4. Keywordarguments(**kwargs).
Listparameters*args.It'satuple.
Keywordparameters**kwargs.It'sadictionary.
1.7.1.4Arguments

Whencallingafunction,valuesmaybepassedtoafunctionwithpositionalargumentsor
keywordarguments.
Positionalargumentsmustplacedbefore(totheleftof)keywordarguments.
Passingliststoafunctionasmultipleargumentssome_func(*aList).Effectively,
thissyntaxcausesPythontounrollthearguments.Example:
deffn1(*args,**kwargs):
fn2(*args,**kwargs)

Page56

APythonBook
1.7.1.5Localvariables

CreatinglocalvariablesAnybindingoperationcreatesalocalvariable.Examplesare
(1)parametersofafunction;(2)assignmenttoavariableinafunction;(3)theimport
statement;(4)etc.Contrastwithaccessingavariable.
VariablelookupTheLGB/LEGBruleThelocal,enclosing,global,builtinscopes
aresearchedinthatorder.See:http://www.python.org/dev/peps/pep0227/
TheglobalstatementInsideafunction,wemustuseglobalwhenwewanttoset
thevalueofaglobalvariable.Example:
deffn():
globalSome_global_variable,Another_global_variable
Some_global_variable='hello'
...

1.7.1.6Otherthingstoknowaboutfunctions

FunctionsarefirstclassYoucanstoretheminastructure,passthemtoa
function,andreturnthemfromafunction.
Functioncallscantakekeywordarguments.Example:
>>>test(size=25)

Formalparameterstoafunctioncanhavedefaultvalues.Example:
>>>deftest(size=0):
...

Donotusemutableobjectsasdefaultvalues.
Youcan"capture"remainingargumentswith*args,and**kwargs.(Spelling
isnotsignificant.)Example:
In[13]:deftest(size,*args,**kwargs):
....:printsize
....:printargs
....:printkwargs
....:
....:
In[14]:test(32,'aa','bb',otherparam='xyz')
32
('aa','bb')
{'otherparam':'xyz'}

Normalargumentsmustcomebeforedefaultargumentswhichmustcomebefore
keywordarguments.
Afunctionthatdoesnotexplicitlyreturnavalue,returnsNone.
Inordertosetthevalueofaglobalvariable,declarethevariablewithglobal.
Exercises:

Page57

APythonBook

Writeafunctionthattakesasingleargument,printsthevalueoftheargument,
andreturnstheargumentasastring.Solution:
>>>deft(x):
...print'x:%s'%x
...return'[[%s]]'%x
...
>>>t(3)
x:3
'[[3]]'

Writeafunctionthattakesavariablenumberofargumentsandprintsthemall.
Solution:
>>>deft(*args):
...forarginargs:
...print'arg:%s'%arg
...
>>>t('aa','bb','cc')
arg:aa
arg:bb
arg:cc

Writeafunctionthatprintsthenamesandvaluesofkeywordargumentspassedto
it.Solution:
>>>deft(**kwargs):
...forkeyinkwargs.keys():
...print'key:%svalue:%s'%(key,
kwargs[key],)
...
>>>t(arg1=11,arg2=22)
key:arg1value:11
key:arg2value:22

1.7.1.7Globalvariablesandtheglobalstatement

Bydefault,assignmentinafunctionormethodcreateslocalvariables.
Reference(notassignment)toavariable,accessesalocalvariableifithasalreadybeen
created,elseaccessesaglobalvariable.
Inordertoassignavaluetoaglobalvariable,declarethevariableasglobalatthe
beginningofthefunctionormethod.
Ifinafunctionormethod,youbothreferenceandassigntoavariable,thenyoumust
either:
1. Assigntothevariablefirst,or
2. Declarethevariableasglobal.
Theglobalstatementdeclaresoneormorevariables,separatedbycommas,tobe
global.
Page58

APythonBook
Someexamples:
In[1]:
In[1]:X=3
In[2]:deft():
...:printX
...:
In[3]:
In[3]:t()
3
In[4]:defs():
...:X=4
...:
In[5]:
In[5]:
In[5]:s()
In[6]:t()
3
In[7]:X=1
In[8]:defu():
...:globalX
...:X=5
...:
In[9]:
In[9]:u()
In[10]:t()
5
In[16]:defv():
....:x=X
....:X=6
....:returnx
....:
In[17]:
In[17]:v()

Traceback(mostrecentcalllast):
File"<ipythonconsole>",line1,in<module>
File"<ipythonconsole>",line2,inv
UnboundLocalError:localvariable'X'referencedbeforeassignment
In[18]:defw():
....:globalX
....:x=X
....:X=7
....:returnx
....:
In[19]:
In[19]:w()
Out[19]:5
In[20]:X
Out[20]:7

Page59

APythonBook
1.7.1.8Docstringsforfunctions

Adddocstringsasatriplequotedstringbeginningwiththefirstlineofafunctionor
method.Seeepydocforasuggestedformat.
1.7.1.9Decoratorsforfunctions

Adecoratorperformsatransformationonafunction.Examplesofdecoratorsthatare
builtinfunctionsare:@classmethod,@staticmethod,and@property.See:
http://docs.python.org/2/library/functions.html#builtinfunctions
Adecoratorisappliedusingthe"@"characteronalineimmediatelypreceedingthe
functiondefinitionheader.Examples:
classSomeClass(object):
@classmethod
defHelloClass(cls,arg):
pass
##HelloClass=classmethod(HelloClass)
@staticmethod
defHelloStatic(arg):
pass
##HelloStatic=staticmethod(HelloStatic)
#
#Define/implementadecorator.
defwrapper(fn):
definner_fn(*args,**kwargs):
print'>>'
result=fn(*args,**kwargs)
print'<<'
returnresult
returninner_fn
#
#Applyadecorator.
@wrapper
deffn1(msg):
pass
##fn1=wrapper(fn1)

Notes:

Thedecoratorform(withthe"@"character)isequivalenttotheform
(commentedout)thatcallsthedecoratorfunctionexplicitly.
Theuseofclassmethodsandstaticmethodwillbeexplainedlaterinthe
sectiononobjectorientedprogramming.
Adecoratorisimplementedasafunction.Therefore,tolearnaboutsomespecific
Page60

APythonBook

decorator,youshouldsearchforthedocumentationonortheimplementationof
thatfunction.Rememberthatinordertouseafunction,itmustbedefinedinthe
currentmoduleorimportedbythecurrentmoduleorbeabuiltin.
Theformthatexplicitlycallsthedecoratorfunction(commentedoutinthe
exampleabove)isequivalenttotheformusingthe"@"character.

1.7.2lambda
Usealambda,asaconvenience,whenyouneedafunctionthatboth:
isanonymous(doesnotneedaname)and
containsonlyanexpressionandnostatements.
Example:

In[1]:fn=lambdax,y,z:(x**2)+(y*2)+z
In[2]:fn(4,5,6)
Out[2]:32
In[3]:
In[3]:format=lambdaobj,category:'The"%s"isa"%s".'%(obj,
category,)
In[4]:format('pinetree','conifer')
Out[4]:'The"pinetree"isa"conifer".'

Alambdacantakemultipleargumentsandcanreturn(likeafunction)multiplevalues.
Example:
In[79]:a=lambdax,y:(x*3,y*4,(x,y))
In[80]:
In[81]:a(3,4)
Out[81]:(9,16,(3,4))

Suggestion:Insomecases,alambdamaybeusefulasaneventhandler.
Example:
classTest:
def__init__(self,first='',last=''):
self.first=first
self.last=last
deftest(self,formatter):
"""
Testforlambdas.
formatterisafunctiontaking2arguments,firstandlast
names.Itshouldreturntheformattedname.
"""
msg='Mynameis%s'%(formatter(self.first,self.last),)
printmsg
deftest():
t=Test('Dave','Kuhlman')

Page61

APythonBook
t.test(lambdafirst,last:'%s%s'%(first,last,))
t.test(lambdafirst,last:'%s,%s'%(last,first,))
test()

Alambdaenablesustodefine"functions"wherewedonotneednamesforthose
functions.Example:
In[45]:a=[
....:lambdax:x,
....:lambdax:x*2,
....:]
In[46]:
In[46]:a
Out[46]:[<function__main__.<lambda>>,<function__main__.<lambda>>]
In[47]:a[0](3)
Out[47]:3
In[48]:a[1](3)
Out[48]:6

Reference:http://docs.python.org/2/reference/expressions.html#lambda

1.7.3Iteratorsandgenerators
Concepts:
iterator
Anditeratorissomethingthatsatisfiestheiteratorprotocol.Clue:Ifit'saniterator,
youcanuseitinafor:statement.
generator
Ageneratorisaclassorfunctionthatimplementsaniterator,i.e.thatimplementsthe
iteratorprotocol.
theiteratorprotocol
Anobjectsatisfiestheiteratorprotocolifitdoesthefollowing:

Itimplementsa__iter__method,whichreturnsaniteratorobject.
Itimplementsanextfunction,whichreturnsthenextitemfromthe
collection,sequence,stream,etcofitemstobeiteratedover
ItraisestheStopIterationexceptionwhentheitemsareexhaustedand
thenext()methodiscalled.

yield
Theyieldstatementenablesustowritefunctionsthataregenerators.Such
functionsmaybesimilartocoroutines,sincetheymay"yield"multipletimesandare
resumed.
Page62

APythonBook
Formoreinformationoniterators,seethesectiononiteratortypesinthePythonLibrary
Referencehttp://docs.python.org/2/library/stdtypes.html#iteratortypes.
Formoreontheyieldstatement,see:
http://docs.python.org/2/reference/simple_stmts.html#theyieldstatement
Actually,yieldisanexpression.Formoreonyieldexpressionsandonthenext()
andsend()generatormethods,aswellasothers,see:Yieldexpression
http://docs.python.org/2/reference/expressions.html#yieldexpressionsinthePython
languagereferencemanual.
Afunctionormethodcontainingayieldstatementimplementsagenerator.Addingthe
yieldstatementtoafunctionormethodturnsthatfunctionormethodintoonewhich,
whencalled,returnsagenerator,i.e.anobjectthatimplementstheiteratorprotocol.
Agenerator(afunctioncontainingyield)providesaconvenientwaytoimplementa
filter.But,alsoconsider:
Thefilter()builtinfunction
Listcomprehensionswithanifclause
Hereareafewexamples:

defsimplegenerator():
yield'aaa'#Note1
yield'bbb'
yield'ccc'
deflist_tripler(somelist):
foriteminsomelist:
item*=3
yielditem
deflimit_iterator(somelist,max):
foriteminsomelist:
ifitem>max:
return#Note2
yielditem
deftest():
print'1.',''*30
it=simplegenerator()
foriteminit:
printitem
print'2.',''*30
alist=range(5)
it=list_tripler(alist)
foriteminit:
printitem
print'3.',''*30
alist=range(8)

Page63

APythonBook
it=limit_iterator(alist,4)
foriteminit:
printitem
print'4.',''*30
it=simplegenerator()
try:
printit.next()#Note3
printit.next()
printit.next()
printit.next()
exceptStopIteration,exp:#Note4
print'reachedendofsequence'
if__name__=='__main__':
test()

Notes:
1. Theyieldstatementreturnsavalue.Whenthenextitemisrequestedandthe
iteratoris"resumed",executioncontinuesimmediatelyaftertheyield
statement.
2. Wecanterminatethesequencegeneratedbyaniteratorbyusingareturn
statementwithnovalue.
3. Toresumeagenerator,usethegenerator'snext()orsend()methods.
send()islikenext(),butprovidesavaluetotheyieldexpression.
4. Wecanalternativelyobtaintheitemsinasequencebycallingtheiterator's
next()method.Sinceaniteratorisafirstclassobject,wecansaveitinadata
structureandcanpassitaroundforuseatdifferentlocationsandtimesinour
program.
1. Whenaniteratorisexhaustedorempty,itthrowstheStopIteration
exception,whichwecancatch.
Andhereistheoutputfromrunningtheaboveexample:
$pythontest_iterator.py
1.
aaa
bbb
ccc
2.
0
3
6
9
12
3.
0
1
2
3

Page64

APythonBook
4
4.
aaa
bbb
ccc
reachedendofsequence

Aninstanceofaclasswhichimplementsthe__iter__method,returninganiterator,is
iterable.Forexample,itcanbeusedinaforstatementorinalistcomprehension,orin
ageneratorexpression,orasanargumenttotheiter()builtinmethod.But,notice
thattheclassmostlikelyimplementsageneratormethodwhichcanbecalleddirectly.
ExamplesThefollowingcodeimplementsaniteratorthatproducesalltheobjectsina
treeofobjects:
classNode:
def__init__(self,data,children=None):
self.initlevel=0
self.data=data
ifchildrenisNone:
self.children=[]
else:
self.children=children
defset_initlevel(self,initlevel):self.initlevel=initlevel
defget_initlevel(self):returnself.initlevel
defaddchild(self,child):
self.children.append(child)
defget_data(self):
returnself.data
defget_children(self):
returnself.children
defshow_tree(self,level):
self.show_level(level)
print'data:%s'%(self.data,)
forchildinself.children:
child.show_tree(level+1)
defshow_level(self,level):
print''*level,
#
#Generatormethod#1
#Thisgeneratorturnsinstancesofthisclassintoiterable
objects.
#
defwalk_tree(self,level):
yield(level,self,)
forchildinself.get_children():
forlevel1,tree1inchild.walk_tree(level+1):
yieldlevel1,tree1
def__iter__(self):
returnself.walk_tree(self.initlevel)

Page65

APythonBook
#
#Generatormethod#2
#Thisgeneratorusesasupportfunction(walk_list)whichcalls
#thisfunctiontorecursivelywalkthetree.
#Ifeffect,thisiteratesoverthesupportfunction,which
#iteratesoverthisfunction.
#
defwalk_tree(tree,level):
yield(level,tree)
forchildinwalk_list(tree.get_children(),level+1):
yieldchild
defwalk_list(trees,level):
fortreeintrees:
fortreeinwalk_tree(tree,level):
yieldtree
#
#Generatormethod#3
#Thisgeneratorislikemethod#2,butcallsitself(asan
iterator),
#ratherthancallingasupportfunction.
#
defwalk_tree_recur(tree,level):
yield(level,tree,)
forchildintree.get_children():
forlevel1,tree1inwalk_tree_recur(child,level+1):
yield(level1,tree1,)
defshow_level(level):
print''*level,
deftest():
a7=Node('777')
a6=Node('666')
a5=Node('555')
a4=Node('444')
a3=Node('333',[a4,a5])
a2=Node('222',[a6,a7])
a1=Node('111',[a2,a3])
initLevel=2
a1.show_tree(initLevel)
print'='*40
forlevel,iteminwalk_tree(a1,initLevel):
show_level(level)
print'item:',item.get_data()
print'='*40
forlevel,iteminwalk_tree_recur(a1,initLevel):
show_level(level)
print'item:',item.get_data()

Page66

APythonBook
print'='*40
a1.set_initlevel(initLevel)
forlevel,itemina1:
show_level(level)
print'item:',item.get_data()
iter1=iter(a1)
printiter1
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
##printiter1.next()
returna1
if__name__=='__main__':
test()

Notes:

AninstanceofclassNodeis"iterable".Itcanbeuseddirectlyinafor
statement,alistcomprehension,etc.So,forexample,whenaninstanceofNode
isusedinaforstatement,itproducesaniterator.
WecouldalsocalltheNode.walk_methoddirectlytoobtainaniterator.
MethodNode.walk_treeandfunctionswalk_treeand
walk_tree_recuraregenerators.Whencalled,theyreturnaniterator.They
dothisbecausetheyeachcontainayieldstatement.
Thesemethods/functionsarerecursive.Theycallthemselves.Sincetheyare
generators,theymustcallthemselvesinacontextthatusesaniterator,for
exampleinaforstatement.

1.7.4Modules
AmoduleisaPythonsourcecodefile.
Amodulecanbeimported.Whenimported,themoduleisevaluated,andamoduleobject
iscreated.Themoduleobjecthasattributes.Thefollowingattributesareofspecial
interest:
__doc__Thedocstringofthemodule.
__name__Thenameofthemodulewhenthemoduleisimported,butthe
string"__main__"whenthemoduleisexecuted.
Othernamesthatarecreated(bound)inthemodule.
Amodulecanberun.

Tomakeamodulebothimportableandrunable,usethefollowingidiom(attheendof
Page67

APythonBook
themodule):
defmain():
o
o
o
if__name__=='__main__':
main()

WherePythonlooksformodules:
Seesys.path.
Standardplaces.
EnvironmentvariablePYTHONPATH.
Notesaboutmodulesandobjects:

Amoduleisanobject.
Amodule(object)canbeshared.
Aspecificmoduleisimportedonlyonceinasinglerun.Thismeansthatasingle
moduleobjectissharedbyallthemodulesthatimportit.

1.7.4.1Docstringsformodules

Adddocstringsasatriplequotedstringatornearthetopofthefile.Seeepydocfora
suggestedformat.

1.7.5Packages
Apackageisadirectoryonthefilesystemwhichcontainsafilenamed__init__.py.
The__init__.pyfile:

Whyisitthere?Itmakesmodulesinthedirectory"importable".
Can__init__.pybeempty?Yes.Or,justincludeacomment.
Whenisitevaluated?Itisevaluatedthefirsttimethatanapplicationimports
anythingfromthatdirectory/package.
Whatcanyoudowithit?Anycodethatshouldbeexecutedexactlyonceand
duringimport.Forexample:
Performinitializationneededbythepackage.
Makevariables,functions,classes,etcavailable.Forexample,whenthe
packageisimportedratherthanmodulesinthepackage.Youcanalsoexpose
objectsdefinedinmodulescontainedinthepackage.
Defineavariablenamed__all__tospecifythelistofnamesthatwillbe
importedbyfrommy_packageimport*.Forexample,ifthefollowingis
presentinmy_package/__init__.py:
Page68

APythonBook
__all__=['func1','func2',]

Then,frommy_packageimport*willimportfunc1andfunc2,but
notothernamesdefinedinmy_package.
Notethat__all__canbeusedatthemodulelevel,aswellasatthepackage
level.
Formoreinformation,seethesectiononpackagesinthePythontutorial:
http://docs.python.org/2/tutorial/modules.html#packages.
Guidanceandsuggestions:

"Flatisbetter"Usethe__init__.pyfiletopresenta"flat"viewoftheAPI
foryourcode.Enableyouruserstodoimportmypackageandthen
reference:
mypackage.item1
mypackage.item2
mypackage.item3
Etc.
Whereitem1,item2,etccomposetheAPIyouwantyouruserstouse,even
thoughtheimplementationoftheseitemsmaybeburieddeepinyourcode.
Usethe__init__.pymoduletopresenta"clean"API.Presentonlytheitems
thatyouintendyouruserstouse,andbydoingso,"hide"itemsyoudonotintend
themtouse.

1.8Classes
Classesmodelthebehaviorofobjectsinthe"real"world.Methodsimplementthe
behaviorsofthesetypesofobjects.Membervariableshold(current)state.Classesenable
ustoimplementnewdatatypesinPython.
Theclass:statementisusedtodefineaclass.Theclass:statementcreatesaclass
objectandbindsittoaname.

1.8.1Asimpleclass
In[104]:classA:
.....:pass
.....:
In[105]:a=A()

Todefineanewstyleclass(recommended),inheritfromobjectorfromanotherclass
thatdoes.Example:
In[21]:classA(object):
....:pass
....:

Page69

APythonBook
In[22]:
In[22]:a=A()
In[23]:a
Out[23]:<__main__.Aobjectat0x82fbfcc>

1.8.2Definingmethods
Amethodisafunctiondefinedinclassscopeandwithfirstparameterself:
In[106]:classB(object):
.....:defshow(self):
.....:print'hellofromB'
.....:
In[107]:b=B()
In[108]:b.show()
hellofromB

Amethodaswedescribeithereismoreproperlycalledaninstancemethod,inorderto
distinguishitfromclassmethodsandstaticmethods.

1.8.3Theconstructor
Theconstructorisamethodnamed__init__.
Exercise:Defineaclasswithamembervariablenameandashowmethod.Useprint
toshowthename.Solution:
In[109]:classA(object):
.....:def__init__(self,name):
.....:self.name=name
.....:defshow(self):
.....:print'name:"%s"'%self.name
.....:
In[111]:a=A('dave')
In[112]:a.show()
name:"dave"

Notes:

Theselfvariableisexplicit.Itreferencesthecurrentobject,thatistheobject
whosemethodiscurrentlyexecuting.
Thespelling("self")isoptional,buteveryonespellsitthatway.

1.8.4Membervariables
DefiningmembervariablesMembervariablesarecreatedwithassignment.Example:
classA(object):
def__init__(self,name):

Page70

APythonBook
self.name=name

AsmallgotchaDothis:
In[28]:classA(object):
....:def__init__(self,items=None):
....:ifitemsisNone:
....:self.items=[]
....:else:
....:self.items=items

Donotdothis:
In[29]:classB:
....:def__init__(self,items=[]):#wrong.listctor
evaluatedonlyonce.
....:self.items=items

Inthesecondexample,thedefstatementandthelistconstructorareevaluatedonly
once.Therefore,theitemmembervariableofallinstancesofclassB,willsharethesame
value,whichismostlikelynotwhatyouwant.

1.8.5Callingmethods

Usetheinstanceandthedotoperator.
Callingamethoddefinedinthesameclassorasuperclass:
FromoutsidetheclassUsetheinstance:
some_object.some_method()
an_array_of_of_objects[1].another_method()

FromwithinthesameclassUseself:
self.a_method()

Fromwithasubclasswhenthemethodisinthesuperclassandthereisa
methodwiththesamenameinthecurrentclassUsetheclass(name)oruse
super:
SomeSuperClass.__init__(self,arg1,arg2)
super(CurrentClass,
self).__init__(arg1,arg2)

CallingamethoddefinedinaspecificsuperclassUsetheclass(name).

1.8.6Addinginheritance
ReferencingsuperclassesUsethebuiltinsuperortheexplicitnameofthe
superclass.Useofsuperispreferred.Forexample:
In[39]:classB(A):

Page71

APythonBook
....:def__init__(self,name,size):
....:super(B,self).__init__(name)
....:#A.__init__(self,name)#anolderalternative
form
....:self.size=size

Theuseofsuper()maysolveproblemssearchingforthebaseclasswhenusing
multipleinheritance.Abettersolutionistonotusemultipleinheritance.
Youcanalsousemultipleinheritance.But,itcancauseconfusion.Forexample,inthe
following,classCinheritsfrombothAandB:
classC(A,B):
...

PythonsearchessuperclassesMRO(methodresolutionorder).Ifonlysingleinheritance
isinvolved,thereislittleconfusion.Ifmultipleinheritanceisbeingused,thesearchorder
ofsuperclassescangetcomplexseehere:
http://www.python.org/download/releases/2.3/mro
Formoreinformationoninheritance,seethetutorialinthestandardPython
documentationset:9.5Inheritanceand9.5.1MultipleInheritance.
Watchoutforproblemswithinheritingfrommultipleclassesthathaveacommonbase
class.

1.8.7Classvariables

Alsocalledstaticdata.
Aclassvariableissharedbyinstancesoftheclass.
Defineatclasslevelwithassignment.Example:
classA(object):
size=5
defget_size(self):
returnA.size

Referencewithclassname.variable.
Caution:self.variable=xcreatesanewmembervariable.

1.8.8Classmethodsandstaticmethods
Instance(plain)methods:
Aninstancemethodreceivestheinstanceasitsfirstargument.
Classmethods:

Aclassmethodreceivestheclassasitsfirstargument.
Defineclassmethodswithbuiltinfunctionclassmethod()orwithdecorator
Page72

APythonBook
@classmethod.
Seethedescriptionofclassmethod()builtinfunctionat"Builtin
Functions":http://docs.python.org/2/library/functions.html#classmethod
Staticmethods:
Astaticmethodreceivesneithertheinstancenortheclassasitsfirstargument.
Definestaticmethodswithbuiltinfunctionstaticmethod()orwith
decorator@staticmethod.
Seethedescriptionofstaticmethod()builtinfunctionat"Builtin
Functions":http://docs.python.org/2/library/functions.html#staticmethod
Notesondecorators:

Adecoratoroftheform@afuncisthesameasm=afunc(m).So,this:
@afunc
defm(self):pass

isthesameas:
defm(self):pass
m=afunc(m)

Youcanusedecorators@classmethodand@staticmethod(insteadofthe
classmethod()andstaticmethod()builtinfunctions)todeclareclass
methodsandstaticmethods.
Example:

classB(object):
Count=0
defdup_string(x):
s1='%s%s'%(x,x,)
returns1
dup_string=staticmethod(dup_string)
@classmethod
defshow_count(cls,msg):
print'%s%d'%(msg,cls.Count,)
deftest():
printB.dup_string('abcd')
B.show_count('hereisthecount:')

Analternativewaytoimplement"staticmethods"Usea"plain",modulelevel
function.Forexample:
In[1]:definc_count():
...:A.count+=1
...:
In[2]:

Page73

APythonBook
In[2]:defdec_count():
...:A.count=1
...:
In[3]:
In[3]:classA:
...:count=0
...:defget_count(self):
...:returnA.count
...:
In[4]:
In[4]:a=A()
In[5]:a.get_count()
Out[5]:0
In[6]:
In[6]:
In[6]:inc_count()
In[7]:inc_count()
In[8]:a.get_count()
Out[8]:2
In[9]:
In[9]:b=A()
In[10]:b.get_count()
Out[10]:2

1.8.9Properties
Thepropertybuiltinfunctionenablesustowriteclassesinawaythatdoesnotrequirea
useroftheclasstousegettersandsetters.Example:
classTestProperty(object):
def__init__(self,description):
self._description=description
def_set_description(self,description):
print'settingdescription'
self._description=description
def_get_description(self):
print'gettingdescription'
returnself._description
description=property(_get_description,_set_description)

Thepropertybuiltinfunctionisalsoadecorator.So,thefollowingisequivalentto
theaboveexample:
classTestProperty(object):
def__init__(self,description):
self._description=description
@property
defdescription(self):
print'gettingdescription'
returnself._description

Page74

APythonBook
@description.setter
defdescription(self,description):
print'settingdescription'
self._description=description

Notes:
Wemarktheinstancevariableasprivatebyprefixingitwithandunderscore.
Thenameoftheinstancevariableandthenameofthepropertymustbedifferent.
Iftheyarenot,wegetrecursionandanerror.
Formoreinformationonproperties,seeBuiltinFunctionsproperties
http://docs.python.org/2/library/functions.html#property

1.8.10Interfaces
InPython,toimplementaninterfaceistoimplementamethodwithaspecificnameanda
specificarguments.
"Ducktyping"Ifitwalkslikeaduckandquackslikeaduck...
Onewaytodefinean"interface"istodefineaclasscontainingmethodsthathavea
headerandadocstringbutnoimplementation.
Additionalnotesoninterfaces:

Interfacesarenotenforced.
Aclassdoesnothavetoimplementallofaninterface.

1.8.11Newstyleclasses
Anewstyleclassisonethatsubclassesobjectoraclassthatsubclassesobject(that
is,anothernewstyleclass).
YoucansubclassPython'sbuiltindatatypes.

Asimpleexamplethefollowingclassextendsthelistdatatype:
classC(list):
defget_len(self):
returnlen(self)
c=C((11,22,33))
c.get_len()
c=C((11,22,33,44,55,66,77,88))
printc.get_len()
#Prints"8".

Aslightlymorecomplexexamplethefollowingclassextendsthedictionary
Page75

APythonBook
datatype:
classD(dict):
def__init__(self,data=None,name='no_name'):
ifdataisNone:
data={}
dict.__init__(self,data)
self.name=name
defget_len(self):
returnlen(self)
defget_keys(self):
content=[]
forkeyinself:
content.append(key)
contentstr=','.join(content)
returncontentstr
defget_name(self):
returnself.name
deftest():
d=D({'aa':111,'bb':222,'cc':333})
#Prints"3"
printd.get_len()
#Prints"'aa,cc,bb'"
printd.get_keys()
#Prints"no_name"
printd.get_name()

Somethingstorememberaboutnewstyleclasses:
Inordertobenewstyle,aclassmustinherit(directlyorindirectly)from
object.Notethatifyouinheritfromabuiltintype,yougetthisautomatically.
Newstyleclassesunifytypesandclasses.
Youcansubclass(builtin)typessuchasdict,str,list,file,etc.
Thebuiltintypesnowprovidefactoryfunctions:dict(),str(),int(),
file(),etc.
ThebuiltintypesareintrospectableUsex.__class__,
dir(x.__class__),isinstance(x,list),etc.
Newstyleclassesgiveyoupropertiesanddescriptors.
Newstyleclassesenableyoutodefinestaticmethods.Actually,allclassesenable
youtodothis.
Anewstyleclassisauserdefinedtype.Foraninstanceofanewstyleclassx,
type(x)isthesameasx.__class__.
Formoreonnewstyleclasses,see:http://www.python.org/doc/newstyle/

Exercises:

Writeaclassandasubclassofthisclass.
Givethesuperclassonemembervariable,aname,whichcanbeenteredwhen
Page76

APythonBook
aninstanceisconstructed.
Givethesubclassonemembervariable,adescription;thesubclassconstructor
shouldallowentryofbothnameanddescription.
Putashow()methodinthesuperclassandoverridetheshow()methodin
thesubclass.
Solution:
classA(object):
def__init__(self,name):
self.name=name
defshow(self):
print'name:%s'%(self.name,)
classB(A):
def__init__(self,name,desc):
A.__init__(self,name)
self.desc=desc
defshow(self):
A.show(self)
print'desc:%s'%(self.desc,)

1.8.12Docstringsforclasses
Adddocstringsasa(triplequoted)stringbeginningwiththefirstlineofaclass.See
epydocforasuggestedformat.

1.8.13Privatemembers
Addanleadingunderscoretoamembername(methodordatavariable)to"suggest"that
thememberisprivate.

1.9SpecialTasks
1.9.1Debuggingtools
pdbThePythondebugger:

Startthedebuggerbyrunninganexpression:
pdb.run('expression')

Example:
if__name__=='__main__':
importpdb
pdb.run('main()')

Startupthedebuggerataspecificlocationwiththefollowing:
Page77

APythonBook
importpdb;pdb.set_trace()

Example:
if__name__=='__main__':
importpdb
pdb.set_trace()
main()

Gethelpfromwithinthedebugger.Forexample:
(Pdb)help
(Pdb)helpnext

CanalsoembedIPythonintoyourcode.See
http://ipython.scipy.org/doc/manual/manual.html.
ipdbAlsoconsiderusingipdb(andIPython).Theipdbdebuggerinteractive
prompthassomeadditionalfeatures,forexample,itdoestabnamecompletion.
Inspecting:
importinspect
Seehttp://docs.python.org/lib/moduleinspect.html.
Don'tforgettotrydir(obj)andtype(obj)andhelp(obj),first.
Miscellaneoustools:

id(obj)
globals()andlocals().
dir(obj)Returnsinterestingnames,butlistisnotnecessarilycomplete.
obj.__class__
cls.__bases__
obj.__class__.__bases__
obj.__doc__.Butusually,help(obj)isbetter.Itproducesthedocstring.
Customizetherepresentationofyourclass.Definethefollowingmethodsinyour
class:
__repr__()Calledby(1)repr(),(2)interactiveinterpreterwhen
representationisneeded.
__str__()Calledby(1)str(),(2)stringformatting.
pdbisimplementedwiththecmdmoduleinthePythonstandardlibrary.Youcan
implementsimilarcommandlineinterfacesbyusingcmd.See:cmdSupportfor
lineorientedcommandinterpretershttp://docs.python.org/lib/modulecmd.html.

1.9.2Fileinputandoutput
Createafileobject.Useopen().
Thisexamplereadsandprintseachlineofafile:
Page78

APythonBook
deftest():
f=file('tmp.py','r')
forlineinf:
print'line:',line.rstrip()
f.close()
test()

Notes:

Atextfileisaniterable.Ititeratesoverthelinesinafile.Thefollowingisa
commonidiom:
infile=file(filename,'r')
forlineininfile:
process_a_line(line)
infile.close()

string.rstrip()stripsnewlineandotherwhitespacefromtherightsideof
eachline.Tostripnewlinesonly,butnototherwhitespace,tryrstrip('\n').
Otherwaysofreadingfromafile/streamobject:my_file.read(),
my_file.readline(),my_file.readlines(),
Thisexamplewriteslinesoftexttoafile:

deftest():
f=file('tmp.txt','w')
forchin'abcdefg':
f.write(ch*10)
f.write('\n')
f.close()
test()

Notes:
Thewritemethod,unliketheprintstatement,doesnotautomaticallyadd
newlinecharacters.
Mustclosefileinordertoflushoutput.Or,usemy_file.flush().
And,don'tforgetthewith:statement.Itmakesclosingfilesautomatic.Thefollowing
exampleconvertsallthevowelsinaninputfiletouppercaseandwritestheconverted
linestoanoutputfile:

importstring
defshow_file(infilename,outfilename):
tran_table=string.maketrans('aeiou','AEIOU')
withopen(infilename,'r')asinfile,open(outfilename,'w')as
outfile:
forlineininfile:
line=line.rstrip()
outfile.write('%s\n'%line.translate(tran_table))

Page79

APythonBook

1.9.3Unittests
Formoredocumentationontheunittestframework,seeunittestUnittesting
frameworkhttp://docs.python.org/2/library/unittest.html#moduleunittest
ForhelpandmoreinformationdothefollowingatthePythoninteractiveprompt:
>>>importunittest
>>>help(unittest)

And,youcanreadthesource:Lib/unittest.pyinthePythonstandardlibrary.
1.9.3.1Asimpleexample

Hereisaverysimpleexample.Youcanfindmoreinformationaboutthisprimitiveway
ofstructuringunittestsinthelibrarydocumentationfortheunittestmoduleBasic
examplehttp://docs.python.org/lib/minimalexample.html
importunittest
classUnitTests02(unittest.TestCase):
deftestFoo(self):
self.failUnless(False)
classUnitTests01(unittest.TestCase):
deftestBar01(self):
self.failUnless(False)
deftestBar02(self):
self.failUnless(False)
defmain():
unittest.main()
if__name__=='__main__':
main()

Notes:

Thecalltounittest.main()runsalltestsinalltestfixturesinthemodule.It
actuallycreatesaninstanceofclassTestPrograminmodule
Lib/unittest.py,whichautomaticallyrunstests.
Testfixturesareclassesthatinheritfromunittest.TestCase.
Withinatestfixture(aclass),thetestsareanymethodswhosenamesbeginwith
theprefix"test".
Inanytest,wecheckforsuccessorfailurewithinheritedmethodssuchas
failIf(),failUnless(),assertNotEqual(),etc.Formoreonthese
Page80

APythonBook

methods,seethelibrarydocumentationfortheunittestmoduleTestCase
Objectshttp://docs.python.org/lib/testcaseobjects.html.
Ifyouwanttochange(1)thetestmethodprefixor(2)thefunctionusedtosort
(theorderof)executionoftestswithinatestfixture,thenyoucancreateyourown
instanceofclassunittest.TestLoaderandcustomizeit.Forexample:
defmain():
my_test_loader=unittest.TestLoader()
my_test_loader.testMethodPrefix='check'
my_test_loader.sortTestMethodsUsing=my_cmp_func
unittest.main(testLoader=my_test_loader)
if__name__=='__main__':
main()

But,seethenotesinsectionAdditionalunittestfeaturesforinstructionsona
(possibly)betterwaytodothis.
1.9.3.2Unittestsuites

Hereisanother,notquitesosimple,example:
#!/usr/bin/envpython
importsys,popen2
importgetopt
importunittest
classGenTest(unittest.TestCase):
deftest_1_generate(self):
cmd='python../generateDS.pyfoout2sup.pysout2sub.py
people.xsd'
outfile,infile=popen2.popen2(cmd)
result=outfile.read()
outfile.close()
infile.close()
self.failUnless(len(result)==0)
deftest_2_compare_superclasses(self):
cmd='diffout1sup.pyout2sup.py'
outfile,infile=popen2.popen2(cmd)
outfile,infile=popen2.popen2(cmd)
result=outfile.read()
outfile.close()
infile.close()
#print'len(result):',len(result)
#Ignorethedifferinglinescontainingthedate/time.
#self.failUnless(len(result)<130and
result.find('Generated')>1)

Page81

APythonBook
self.failUnless(check_result(result))
deftest_3_compare_subclasses(self):
cmd='diffout1sub.pyout2sub.py'
outfile,infile=popen2.popen2(cmd)
outfile,infile=popen2.popen2(cmd)
result=outfile.read()
outfile.close()
infile.close()
#Ignorethedifferinglinescontainingthedate/time.
#self.failUnless(len(result)<130and
result.find('Generated')>1)
self.failUnless(check_result(result))
defcheck_result(result):
flag1=0
flag2=0
lines=result.split('\n')
len1=len(lines)
iflen1<=5:
flag1=1
s1='\n'.join(lines[:4])
ifs1.find('Generated')>1:
flag2=1
returnflag1andflag2
#Makethetestsuite.
defsuite():
#Thefollowingisobsolete.SeeLib/unittest.py.
#returnunittest.makeSuite(GenTest)
loader=unittest.TestLoader()
#oralternatively
#loader=unittest.defaultTestLoader
testsuite=loader.loadTestsFromTestCase(GenTest)
returntestsuite
#Makethetestsuiteandrunthetests.
deftest():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
runner.run(testsuite)
USAGE_TEXT="""
Usage:
pythontest.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythontest.py

Page82

APythonBook
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=0:
usage()
test()
if__name__=='__main__':
main()
#importpdb
#pdb.run('main()')

Notes:

GenTestisourtestsuiteclass.Itinheritsfromunittest.TestCase.
EachmethodinGenTestwhosenamebeginswith"test"willberunasatest.
Thetestsareruninalphabeticorderbymethodname.
DefaultsinclassTestLoaderforthetestnameprefixandsortcomparison
functioncanbeoverridden.See5.3.8TestLoaderObjects
http://docs.python.org/lib/testloaderobjects.html.
AtestcaseclassmayalsoimplementmethodsnamedsetUp()and
tearDown()toberunbeforeandaftertests.See5.3.5TestCaseObjects
http://docs.python.org/lib/testcaseobjects.html.Actually,thefirsttestmethodin
ourexampleshould,perhaps,beasetUp()method.
Thetestsusecallssuchasself.failUnless()toreporterrors.Theseare
inheritedfromclassTestCase.See5.3.5TestCaseObjects
http://docs.python.org/lib/testcaseobjects.html.
Functionsuite()createsaninstanceofthetestsuite.
Functiontest()runsthetests.

1.9.3.3Additionalunittestfeatures

And,thefollowingexampleshowsseveraladditionalfeatures.Seethenotesthatfollow
Page83

APythonBook
thecode:
importunittest
classUnitTests02(unittest.TestCase):
deftestFoo(self):
self.failUnless(False)
defcheckBar01(self):
self.failUnless(False)
classUnitTests01(unittest.TestCase):
#Note1
defsetUp(self):
print'settingupUnitTests01'
deftearDown(self):
print'tearingdownUnitTests01'
deftestBar01(self):
print'testingtestBar01'
self.failUnless(False)
deftestBar02(self):
print'testingtestBar02'
self.failUnless(False)
deffunction_test_1():
name='mona'
assertnotname.startswith('mo')
defcompare_names(name1,name2):
ifname1<name2:
return1
elifname1>name2:
return1
else:
return0
defmake_suite():
suite=unittest.TestSuite()
#Note2
suite.addTest(unittest.makeSuite(UnitTests01,
sortUsing=compare_names))
#Note3
suite.addTest(unittest.makeSuite(UnitTests02,prefix='check'))
#Note4
suite.addTest(unittest.FunctionTestCase(function_test_1))
returnsuite
defmain():
suite=make_suite()
runner=unittest.TextTestRunner()
runner.run(suite)
if__name__=='__main__':

Page84

APythonBook
main()

Notes:
1. Ifyourunthiscode,youwillnoticethatthesetUpandtearDownmethodsin
classUnitTests01arerunbeforeandaftereachtestinthatclass.
2. Wecancontroltheorderinwhichtestsarerunbypassingacomparefunctionto
themakeSuitefunction.Thedefaultisthecmpbuiltinfunction.
3. Wecancontrolwhichmethodsinatestfixtureareselectedtoberunbypassing
theoptionalargumentprefixtothemakeSuitefunction.
4. Ifwehaveanexistingfunctionthatwewantto"wrap"andrunasaunittest,we
cancreateatestcasefromafunctionwiththeFunctionTestCasefunction.If
wedothat,noticethatweusetheassertstatementtotestandpossiblycause
failure.
1.9.3.4GuidanceonUnitTesting

Whyshouldweuseunittests?Manyreasons,including:
Withoutunittests,cornercasesmaynotbechecked.Thisisespeciallyimportant,
sincePythondoesrelativelylittlecompiletimeerrorchecking.
Unittestsfacilitateafrequentandshortdesignandimplementandrelease
developmentcycle.SeeONLamp.comExtremePython
http://www.onlamp.com/pub/a/python/2001/03/28/pythonnews.htmlandWhatis
XPhttp://www.xprogramming.com/what_is_xp.htm.
Designingthetestsbeforewritingthecodeis"agoodidea".
Additionalnotes:

Inatestclass,instancemethodssetUpandtearDownarerunautomatically
beforeeachandaftereachindividualtest.
Inatestclass,classmethodssetUpClassandtearDownClassarerun
automaticallyoncebeforeandafterallthetestsinaclass.
ModulelevelfunctionssetUpModuleandtearDownModulearerunbefore
andafteranytestsinamodule.
Insomecasesyoucanalsoruntestsdirectlyfromthecommandline.Dothe
followingforhelp:
$pythonmunittesthelp

1.9.4doctest
Forsimpletestharnesses,considerusingdoctest.Withdoctestyoucan(1)runa
testatthePythoninteractiveprompt,then(2)copyandpastethattestintoadocstringin
yourmodule,andthen(3)runthetestsautomaticallyfromwithinyourmoduleunder
Page85

APythonBook
doctest.
ThereareexamplesandexplanationinthestandardPythondocumentation:5.2doctest
TestinteractivePythonexampleshttp://docs.python.org/lib/moduledoctest.html.
Asimplewaytousedoctestinyourmodule:
1. RunseveraltestsinthePythoninteractiveinterpreter.Notethatbecause
doctestlooksfortheinterpreter's">>>"prompt,youmustusethestandard
interpreter,andnot,forexample,IPython.Also,makesurethatyouincludealine
withthe">>>"promptaftereachsetofresults;thisenablesdoctestto
determinetheextentofthetestresults.
2. Usecopyandpaste,toinsertthetestsandtheirresultsfromyourinteractive
sessionintothedocstrings.
3. Addthefollowingcodeatthebottomofyourmodule:
def_test():
importdoctest
doctest.testmod()
if__name__=="__main__":
_test()

Hereisanexample:
deff(n):
"""
Printsomethingfunny.
>>>f(1)
10
>>>f(2)
10
>>>f(3)
0
"""
ifn==1:
return10
elifn==2:
return10
else:
return0
deftest():
importdoctest,test_doctest
doctest.testmod(test_doctest)
if__name__=='__main__':
test()

And,hereistheoutputfromrunningtheabovetestwiththevflag:
Page86

APythonBook
$pythontest_doctest.pyv
Runningtest_doctest.__doc__
0of0examplesfailedintest_doctest.__doc__
Runningtest_doctest.f.__doc__
Trying:f(1)
Expecting:10
ok
Trying:f(2)
Expecting:10
ok
Trying:f(3)
Expecting:0
ok
0of3examplesfailedintest_doctest.f.__doc__
Runningtest_doctest.test.__doc__
0of0examplesfailedintest_doctest.test.__doc__
2itemshadnotests:
test_doctest
test_doctest.test
1itemspassedalltests:
3testsintest_doctest.f
3testsin3items.
3passedand0failed.
Testpassed.

1.9.5ThePythondatabaseAPI
PythondatabaseAPIdefinesastandardinterfaceforaccesstoarelationaldatabase.
InordertousethisAPIyoumustinstallthedatabaseadapter(interfacemodule)foryour
particulardatabase,e.g.PostgreSQL,MySQL,Oracle,etc.
YoucanlearnmoreaboutthePythonDBAPIhere:
http://www.python.org/dev/peps/pep0249/
Thefollowingsimpleexampleusessqlite3http://docs.python.org/2/library/sqlite3.html
#!/usr/bin/envpython
"""
Createarelationaldatabaseandatableinit.
Addsomerecords.
Readanddisplaytherecords.
"""
importsys
importsqlite3
defcreate_table(db_name):
con=sqlite3.connect(db_name)
cursor=con.cursor()
cursor.execute('''CREATETABLEplants

Page87

APythonBook
(nametext,desctext,catint)''')
cursor.execute(
'''INSERTINTOplantsVALUES('tomato','redandjuicy',
1)''')
cursor.execute(
'''INSERTINTOplantsVALUES('pepper','greenandcrunchy',
2)''')
cursor.execute('''INSERTINTOplantsVALUES('pepper','purple',
2)''')
con.commit()
con.close()
defretrieve(db_name):
con=sqlite3.connect(db_name)
cursor=con.cursor()
cursor.execute('''select*fromplants''')
rows=cursor.fetchall()
printrows
print''*40
cursor.execute('''select*fromplants''')
forrowincursor:
printrow
con.close()
deftest():
args=sys.argv[1:]
iflen(args)!=1:
sys.stderr.write('\nusage:test_db.py<db_name>\n\n')
sys.exit(1)
db_name=args[0]
create_table(db_name)
retrieve(db_name)
test()

1.9.6InstallingPythonpackages
Simple:
$pythonsetup.pybuild
$pythonsetup.pyinstall#asroot

Morecomplex:

LookforaREADMEorINSTALLfileattherootofthepackage.
Typethefollowingforhelp:
$pythonsetup.pycmdhelp
$pythonsetup.pyhelpcommands
$pythonsetup.pyhelp[cmd1cmd2...]

And,forevenmoredetails,seeInstallingPythonModules
Page88

APythonBook
http://docs.python.org/inst/inst.html
pipisbecomingpopularforinstallingandmanagingPythonpackages.See:
https://pypi.python.org/pypi/pip
Also,considerusingvirtualenv,especiallyifyoususpectorworrythatinstalling
somenewpackagewillalterthebehaviorofapackagecurrentlyinstalledonyour
machine.See:https://pypi.python.org/pypi/virtualenv.virtualenvcreatesadirectory
andsetsupaPythonenvironmentintowhichyoucaninstallandusePythonpackages
withoutchangingyourusualPythoninstallation.

1.10MorePythonFeaturesandExercises
[Astimepermits,explainmorefeaturesanddomoreexercisesasrequestedbyclass
members.]
ThankstoDavidGoodgerforthefollowinglistorreferences.His"CodeLikea
Pythonista:IdiomaticPython"
(http://python.net/~goodger/projects/pycon/2007/idiomatic/)isworthacarefulreading:

"PythonObjects",FredrikLundh,http://www.effbot.org/zone/pythonobjects.htm
"HowtothinklikeaPythonista",MarkHammond,
http://python.net/crew/mwh/hacks/objectthink.html
"Pythonmain()functions",GuidovanRossum,
http://www.artima.com/weblogs/viewpost.jsp?thread=4829
"PythonIdiomsandEfficiency",http://jaynes.colorado.edu/PythonIdioms.html
"Pythontrack:pythonidioms",
http://www.cs.caltech.edu/courses/cs11/material/python/misc/python_idioms.html
"BePythonic",ShalabhChaturvedi,http://shalabh.infogami.com/Be_Pythonic2
"PythonIsNotJava",PhillipJ.Eby,
http://dirtsimple.org/2004/12/pythonisnotjava.html
"WhatisPythonic?",MartijnFaassen,
http://faassen.ntree.net/blog/view/weblog/2005/08/06/0
"SortingMiniHOWTO",AndrewDalke,
http://wiki.python.org/moin/HowTo/Sorting
"PythonIdioms",http://www.gungfu.de/facts/wiki/Main/PythonIdioms
"PythonFAQs",http://www.python.org/doc/faq/

Page89

APythonBook

2Part2AdvancedPython
2.1IntroductionPython201(Slightly)AdvancedPythonTopics
Thisdocumentisintendedasnotesforacourseon(slightly)advancedPythontopics.

2.2RegularExpressions
Formorehelponregularexpressions,see:

reRegularexpressionoperationshttp://docs.python.org/library/re.html
RegularExpressionHOWTOhttp://docs.python.org/howto/regex.html

2.2.1Definingregularexpressions
Aregularexpressionpatternisasequenceofcharactersthatwillmatchsequencesof
charactersinatarget.
Thepatternsorregularexpressionscanbedefinedasfollows:

Literalcharactersmustmatchexactly.Forexample,"a"matches"a".
Concatenatedpatternsmatchconcatenatedtargets.Forexample,"ab"("a"
followedby"b")matches"ab".
Alternatepatterns(separatedbyaverticalbar)matcheitherofthealternative
patterns.Forexample,"(aaa)|(bbb)"willmatcheither"aaa"or"bbb".
Repeatingandoptionalitems:
"abc*"matches"ab"followedbyzeroormoreoccurancesof"c",forexample,
"ab","abc","abcc",etc.
"abc+"matches"ab"followedbyoneormoreoccurancesof"c",forexample,
"abc","abcc",etc,butnot"ab".
"abc?"matches"ab"followedbyzerooroneoccurancesof"c",forexample,
"ab"or"abc".
SetsofcharactersCharactersandsequencesofcharactersinsquarebrackets
formaset;asetmatchesanycharacterinthesetorrange.Forexample,"[abc]"
matches"a"or"b"or"c".And,forexample,"[_az09]"matchesanunderscore
oranylowercaseletteroranydigit.
GroupsParenthesesindicateagroupwithapattern.Forexample,"ab(cd)*ef"is
apatternthatmatches"ab"followedbyanynumberofoccurancesof"cd"
followedby"ef",forexample,"abef","abcdef","abcdcdef",etc.
Therearespecialnamesforsomesetsofcharacters,forexample"\d"(anydigit),
Page90

APythonBook
"\w"(anyalphanumericcharacter),"\W"(anynonalphanumericcharacter),etc.
Moremoreinformation,seePythonLibraryReference:RegularExpression
Syntaxhttp://docs.python.org/library/re.html#regularexpressionsyntax
Becauseoftheuseofbackslashesinpatterns,youareusuallybetteroffdefiningregular
expressionswithrawstrings,e.g.r"abc".

2.2.2Compilingregularexpressions
Whenaregularexpressionistobeusedmorethanonce,youshouldconsidercompiling
it.Forexample:
importsys,re
pat=re.compile('aa[bc]*dd')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
ifpat.search(line):
print'matched:',line
else:
print'nomatch:',line

Comments:

Weimportmodulereinordertouseregularexpresions.
re.compile()compilesaregularexpressionsothatwecanreusethe
compiledregularexpressionwithoutcompilingitrepeatedly.

2.2.3Usingregularexpressions
Usematch()tomatchatthebeginningofastring(ornotatall).
Usesearch()tosearchastringandmatchthefirststringfromtheleft.
Herearesomeexamples:
>>>importre
>>>pat=re.compile('aa[09]*bb')
>>>x=pat.match('aa1234bbccddee')
>>>x
<_sre.SRE_Matchobjectat0x401e9608>
>>>x=pat.match('xxxxaa1234bbccddee')
>>>x
>>>type(x)
<type'NoneType'>
>>>x=pat.search('xxxxaa1234bbccddee')
>>>x

Page91

APythonBook
<_sre.SRE_Matchobjectat0x401e9608>

Notes:

Whenamatchorsearchissuccessful,itreturnsamatchobject.Whenitfails,it
returnsNone.
Youcanalsocallthecorrespondingfunctionsmatchandsearchintheremodule,
e.g.:
>>>x=re.search(pat,'xxxxaa1234bbccddee')
>>>x
<_sre.SRE_Matchobjectat0x401e9560>

Foralistoffunctionsintheremodule,seeModuleContents
http://docs.python.org/library/re.html#modulecontents.

2.2.4Usingmatchobjectstoextractavalue
Matchobjectsenableyoutoextractmatchedsubstringsafterperformingamatch.A
matchobjectisreturnedbysuccessfulmatch.Thepartofthetargetavailableinthematch
objectistheportionmatchedbygroupsinthepattern,thatistheportionofthepattern
insideparentheses.Forexample:
In[69]:mo=re.search(r'height:(\d*)width:(\d*)','height:123
width:456')
In[70]:mo.groups()
Out[70]:('123','456')

Hereisanotherexample:
importsys,re
Targets=[
'Thereare<<25>>sparrows.',
'Isee<<15>>finches.',
'Thereisnothinghere.',
]
deftest():
pat=re.compile('<<([09]*)>>')
forlineinTargets:
mo=pat.search(line)
ifmo:
value=mo.group(1)
print'value:%s'%value
else:
print'nomatch'
test()

Whenweruntheabove,itprintsoutthefollowing:
Page92

APythonBook
value:25
value:15
nomatch

Explanation:
Intheregularexpression,putparenthesesaroundtheportionoftheregular
expressionthatwillmatchwhatyouwanttoextract.Eachpairofparentheses
marksoffagroup.
Afterthesearch,checktodetermineiftherewasasuccessfulmatchbychecking
foramatchingobject."pat.search(line)"returnsNoneifthesearchfails.
Ifyouspecifymorethanonegroupinyourregularexpression(morethatonepair
ofparentheses),thenyoucanuse"value=mo.group(N)"toextractthevalue
matchedbytheNthgroupfromthematchingobject."value=mo.group(1)"
returnsthefirstextractedvalue;"value=mo.group(2)"returnsthesecond;etc.An
argumentof0returnsthestringmatchedbytheentireregularexpression.
Inaddition,youcan:

Use"values=mo.groups()"togetatuplecontainingthestringsmatchedbyall
groups.
Use"mo.expand()"tointerpolatethegroupvaluesintoastring.Forexample,
"mo.expand(r'value1:\1value2:\2')"insertsthevaluesofthefirstandsecond
groupintoastring.Ifthefirstgroupmatched"aaa"andthesecondmatched
"bbb",thenthisexamplewouldproduce"value1:aaavalue2:bbb".Forexample:
In[76]:mo=re.search(r'h:(\d*)w:(\d*)','h:123
w:456')
In[77]:mo.expand(r'Height:\1Width:\2')
Out[77]:'Height:123Width:456'

2.2.5Extractingmultipleitems
Youcanextractmultipleitemswithasinglesearch.Hereisanexample:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
print'value1:%svalue2:%s'%(value1,value2)
else:
print'nomatch'

Page93

APythonBook
Comments:

Usemultipleparenthesizedsubstringsintheregularexpressiontoindicatethe
portions(groups)tobeextracted.
"mo.group(1,2)"returnsthevaluesofthefirstandsecondgroupinthestring
matched.
Wecouldalsohaveused"mo.groups()"toobtainatuplethatcontainsboth
values.
Yetanotheralternativewouldhavebeentousethefollowing:print
mo.expand(r'value1:\1value2:\2').

2.2.6Replacingmultipleitems
Asimplewaytoperformmultiplereplacementsusingaregularexpressionistousethe
re.subn()function.Hereisanexample:
In[81]:re.subn(r'\d+','***','thereare203birdssittingin2
trees')
Out[81]:('thereare***birdssittingin***trees',2)

Formorecomplexreplacements,useafunctioninsteadofaconstantreplacementstring:
importre
defrepl_func(mo):
s1=mo.group(1)
s2='*'*len(s1)
returns2
deftest():
pat=r'(\d+)'
in_str='thereare2034birdsin21trees'
out_str,count=re.subn(pat,repl_func,in_str)
print'in:"%s"'%in_str
print'out:"%s"'%out_str
print'count:%d'%count
test()

Andwhenweruntheabove,itproduces:
in:"thereare2034birdsin21trees"
out:"thereare****birdsin**trees"
count:2

Notes:

Thereplacementfunctionreceivesoneargument,amatchobject.
There.subn()functionreturnsatuplecontainingtwovalues:(1)thestring
afterreplacementsand(2)thenumberofreplacementsperformed.
Page94

APythonBook
HereisanevenmorecomplexexampleYoucanlocatesubstrings(slices)ofamatch
andreplacethem:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
start1=mo.start(1)
end1=mo.end(1)
start2=mo.start(2)
end2=mo.end(2)
print'value1:%sstart1:%dend1:%d'%(value1,start1,
end1)
print'value2:%sstart2:%dend2:%d'%(value2,start2,
end2)
repl1=raw_input('Enterreplacement#1:')
repl2=raw_input('Enterreplacement#2:')
newline=(line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:])
print'newline:%s'%newline
else:
print'nomatch'

Explanation:

Alternatively,use"mo.span(1)"insteadof"mo.start(1)"and"mo.end(1)"inorder
togetthestartandendofasubmatchinasingleoperation."mo.span(1)"returnsa
tuple:(start,end).
Puttogetheranewstringwithstringconcatenationfrompiecesoftheoriginal
stringandreplacementvalues.Youcanusestringslicestogetthesubstringsof
theoriginalstring.Inourcase,thefollowinggetsthestartofthestring,addsthe
firstreplacement,addsthemiddleoftheoriginalstring,addsthesecond
replacement,andfinally,addsthelastpartoftheoriginalstring:
newline=line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:]

Youcanalsousethesubfunctionormethodtodosubstitutions.Hereisanexample:
importsys,re
pat=re.compile('[09]+')
print'Replacingdecimaldigits.'

Page95

APythonBook
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
repl=raw_input('Enterareplacement:')
result=pat.sub(repl,target)
print'result:%s'%result

Hereisanotherexampleoftheuseofafunctiontoinsertcalculatedreplacements.
importsys,re,string
pat=re.compile('[am]+')
defreplacer(mo):
returnstring.upper(mo.group(0))
print'Uppercasingam.'
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
result=pat.sub(replacer,target)
print'result:%s'%result

Notes:
Ifthereplacementargumenttosubisafunction,thatfunctionmusttakeone
argument,amatchobject,andmustreturnthemodified(orreplacement)value.
Thematchedsubstringwillbereplacedbythevaluereturnedbythisfunction.
Inourcase,thefunctionreplacerconvertsthematchedvaluetouppercase.
Thisisalsoaconvenientuseforalambdainsteadofanamedfunction,forexample:

importsys,re,string
pat=re.compile('[am]+')
print'Uppercasingam.'
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
result=pat.sub(
lambdamo:string.upper(mo.group(0)),
target)
print'result:%s'%result

2.3IteratorObjects
Note1:YouwillneedasufficientlyrecentversionofPythoninordertouseiteratorsand
generators.IbelievethattheywereintroducedinPython2.2.
Page96

APythonBook
Note2:TheiteratorprotocolhaschangedslightlyinPythonversion3.0.
Goalsforthissection:
Learnhowtoimplementageneratorfunction,thatis,afunctionwhich,when
called,returnsaniterator.
Learnhowtoimplementaclasscontainingageneratormethod,thatis,amethod
which,whencalled,returnsaniterator.
Learntheiteratorprotocol,specificallywhatmethodsaniteratormustsupportand
whatthosemethodsmustdo.
Learnhowtoimplementaniteratorclass,thatis,aclasswhoseinstancesare
iteratorobjects.
Learnhowtoimplementrecursiveiteratorgenerators,thatis,aniteratorgenerator
whichrecursivelyproducesiteratorgenerators.
Learnthatyourimplementationofaniteratorobject(aniteratorclass)can
"refresh"itselfandlearnatleastonewaytodothis.
Definitions:

IteratorAnditeratorisanobjectthatsatisfies(implements)theiteratorprotocol.
IteratorprotocolAnobjectimplementstheiteratorprotocolifitimplementsboth
anext()andan__iter__()methodwhichsatisfytheserules:(1)the
__iter__()methodmustreturntheiterator;(2)thenext()methodshould
returnthenextitemtobeiteratedoverandwhenfinished(therearenomore
items)shouldraisetheStopIterationexception.Theiteratorprotocolis
describedatIteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
IteratorclassAclassthatimplements(satisfies)theiteratorprotocol.In
particular,theclassimplementsnext()and__iter__()methodsas
describedaboveandinIteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
(Iterator)generatorfunctionAfunction(ormethod)which,whencalled,returns
aniteratorobject,thatis,anobjectthatsatisfiestheiteratorprotocol.Afunction
containingayieldstatementautomaticallybecomesagenerator.
GeneratorexpressionAnexpressionwhichproducesaniteratorobject.
Generatorexpressionshaveaformsimilartoalistcomprehension,butare
enclosedinparenthesesratherthansquarebrackets.Seeexamplebelow.
Afewadditionalbasicpoints:

Afunctionthatcontainsayieldstatementisageneratorfunction.Whencalled,it
returnsaniterator,thatis,anobjectthatprovidesnext()and__iter__()
methods.
Theiteratorprotocolisdescribedhere:PythonStandardLibrary:IteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
Page97

APythonBook
Aclassthatdefinesbothanext()methodanda__iter__()methodsatisfies
theiteratorprotocol.So,instancesofsuchaclasswillbeiterators.
Pythonprovidesavarietyofwaystoproduce(implement)iterators.Thissection
describesafewofthoseways.Youshouldalsolookattheiter()builtin
function,whichisdescribedinThePythonStandardLibrary:BuiltinFunctions:
iter()http://docs.python.org/library/functions.html#iter.
Aniteratorcanbeusedinaniteratorcontext,forexampleinaforstatement,ina
listcomprehension,andinageneratorexpression.Whenaniteratorisusedinan
iteratorcontext,theiteratorproducesitsvalues.
Thissectionattemptstoprovideexamplesthatillustratethegenerator/iteratorpattern.

Whyisthisimportant?
Oncemastered,itisasimple,convenient,andpowerfulprogrammingpattern.
Ithasmanyandpervasiveuses.
Ithelpstolexicallyseparatetheproducercodefromtheconsumercode.Doingso
makesiteasiertolocateproblemsandtomodifyorfixcodeinawaythatis
localizedanddoesnothaveunwantedsideeffects.
Implementingyourowniterators(andgenerators)enablesyoutodefineyourown
abstractsequences,thatis,sequenceswhosecompositionaredefinedbyyour
computationsratherthanbytheirpresenceinacontainer.Infact,youriteratorcan
calculateorretrievevaluesaseachoneisrequested.
ExamplesTheremainderofthissectionprovidesasetofexampleswhichimplement
anduseiterators.

2.3.1ExampleAgeneratorfunction
Thisfunctioncontainsayieldstatement.Therefore,whenwecallit,itproducesan
iterator:
defgenerateItems(seq):
foriteminseq:
yield'item:%s'%item
anIter=generateItems([])
print'dir(anIter):',dir(anIter)
anIter=generateItems([111,222,333])
forxinanIter:
printx
anIter=generateItems(['aaa','bbb','ccc'])
printanIter.next()
printanIter.next()
printanIter.next()
printanIter.next()

Runningthisexampleproducesthefollowingoutput:
Page98

APythonBook
dir(anIter):['__class__','__delattr__','__doc__',
'__getattribute__',
'__hash__','__init__','__iter__','__new__','__reduce__',
'__reduce_ex__','__repr__','__setattr__','__str__','gi_frame',
'gi_running','next']
item:111
item:222
item:333
item:aaa
item:bbb
item:ccc
Traceback(mostrecentcalllast):
File"iterator_generator.py",line14,in?
printanIter.next()
StopIteration

Notesandexplanation:
Thevaluereturnedbythecalltothegenerator(function)isaniterator.Itobeys
theiteratorprotocol.Thatis,dir(anIter)showsthatithasboth
__iter__()andnext()methods.
Becausethisobjectisaniterator,wecanuseaforstatementtoiterateoverthe
valuesreturnedbythegenerator.
Wecanalsogetitsvaluesbyrepeatedlycallingthenext()method,untilit
raisestheStopIterationexception.Thisabilitytocallthenextmethodenablesus
topasstheiteratorobjectaroundandgetvaluesatdifferentlocationsinourcode.
Oncewehaveobtainedallthevaluesfromaniterator,itis,ineffect,"empty"or
"exhausted".Theiteratorprotocol,infact,specifiesthatonceaniteratorraisesthe
StopIterationexception,itshouldcontinuetodoso.Anotherwaytosaythisis
thatthereisno"rewind"operation.But,youcancallthethegeneratorfunction
againtogeta"fresh"iterator.
Analternativeandperhapssimplerwaytocreateaninteratoristouseagenerator
expression.Thiscanbeusefulwhenyoualreadyhaveacollectionoriteratortowork
with.

Thenfollowingexampleimplementsafunctionthatreturnsageneratorobject.Theeffect
istogeneratetheobjectsinacollectionwhichexcludingitemsinasepartecollection:
DATA=[
'lemon',
'lime',
'grape',
'apple',
'pear',
'watermelon',
'canteloupe',
'honeydew',
'orange',

Page99

APythonBook
'grapefruit',
]
defmake_producer(collection,excludes):
gen=(itemforitemincollectionifitemnotinexcludes)
returngen
deftest():
iter1=make_producer(DATA,('apple','orange','honeydew',))
print'%s'%iter1
forfruitiniter1:
printfruit
test()

Whenrun,thisexampleproducesthefollowing:
$pythonworkbook063.py
<generatorobject<genexpr>at0x7fb3d0f1bc80>
lemon
lime
grape
pear
watermelon
canteloupe
grapefruit

Notes:

Ageneratorexpressionlooksalmostlikealistcomprehension,butissurrounded
byparenthesesratherthansquarebrackets.Formoreonlistcomprehensionssee
sectionExampleAlistcomprehension.
Themake_producerfunctionreturnstheobjectproducedbythegenerator
expression.

2.3.2ExampleAclasscontainingageneratormethod
Eachtimethismethodiscalled,itproducesa(new)iteratorobject.Thismethodis
analogoustotheiterkeysanditervaluesmethodsinthedictionarybuiltinobject:
#
#Aclassthatprovidesaniteratorgeneratormethod.
#
classNode:
def__init__(self,name='<noname>',value='<novalue>',
children=None):
self.name=name
self.value=value
self.children=children
ifchildrenisNone:
self.children=[]

Page100

APythonBook
else:
self.children=children
defset_name(self,name):self.name=name
defget_name(self):returnself.name
defset_value(self,value):self.value=value
defget_value(self):returnself.value
defiterchildren(self):
forchildinself.children:
yieldchild
#
#Printinformationonthisnodeandwalkoverallchildrenand
#grandchildren...
defwalk(self,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),self.get_name(),self.get_value(),)
forchildinself.iterchildren():
child.walk(level+1)
#
#Anfunctionthatistheequivalentofthewalk()methodin
#classNode.
#
defwalk(node,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),node.get_name(),node.get_value(),)
forchildinnode.iterchildren():
walk(child,level+1)
defget_filler(level):
return''*level
deftest():
a7=Node('gilbert','777')
a6=Node('fred','666')
a5=Node('ellie','555')
a4=Node('daniel','444')
a3=Node('carl','333',[a4,a5])
a2=Node('bill','222',[a6,a7])
a1=Node('alice','111',[a2,a3])
#Usethewalkmethodtowalktheentiretree.
print'Usingthemethod:'
a1.walk()
print'='*30
#Usethewalkfunctiontowalktheentiretree.
print'Usingthefunction:'
walk(a1)
test()

Runningthisexampleproducesthefollowingoutput:
Usingthemethod:
name:alicevalue:111

Page101

APythonBook
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555
==============================
Usingthefunction:
name:alicevalue:111
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555

Notesandexplanation:

Thisclasscontainsamethoditerchildrenwhich,whencalled,returnsaniterator.
Theyieldstatementinthemethoditerchildrenmakesitintoagenerator.
Theyieldstatementreturnsoneitemeachtimeitisreached.Thenexttimethe
iteratorobjectis"called"itresumesimmediatelyaftertheyieldstatement.
Afunctionmayhaveanynumberofyieldstatements.
Aforstatementwilliterateoveralltheitemsproducedbyaniteratorobject.
Thisexampleshowstwowaystousethegenerator,specifically:(1)thewalk
methodintheclassNodeand(2)thewalkfunction.Bothcallthegenerator
iterchildrenandbothdoprettymuchthesamething.

2.3.3ExampleAniteratorclass
Thisclassimplementstheiteratorprotocol.Therefore,instancesofthisclassareiterators.
Thepresenceofthenext()and__iter__()methodsmeansthatthisclass
implementstheiteratorprotocolandmakesinstancesofthisclassiterators.
Notethatwhenaniteratoris"exhausted"it,normally,cannotbereusedtoiterateoverthe
sequence.However,inthisexample,weprovidearefreshmethodwhichenablesusto
"rewind"andreusetheiteratorinstance:
#
#Aniteratorclassthatdoes*not*use``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classIteratorExample:
def__init__(self,seq):
self.seq=seq
self.idx=0
defnext(self):
self.idx+=1
ifself.idx>=len(self.seq):

Page102

APythonBook
raiseStopIteration
value=self.seq[self.idx]
self.idx+=1
returnvalue
def__iter__(self):
returnself
defrefresh(self):
self.idx=0
deftest_iteratorexample():
a=IteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=IteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
test_iteratorexample()

Runningthisexampleproducesthefollowingoutput:
d
b

d
b
==============================
b
d
stopping

Notesandexplanation:

Thenextmethodmustkeeptrackofwhereitisandwhatitemitshouldproduce
next.
Alert:TheiteratorprotocolhaschangedslightlyinPython3.0.Inparticular,the
next()methodhasbeenrenamedto__next__().See:PythonStandard
Library:IteratorTypes
http://docs.python.org/3.0/library/stdtypes.html#iteratortypes.
Page103

APythonBook

2.3.4ExampleAniteratorclassthatusesyield
Theremaybetimeswhenthenextmethodiseasierandmorestraightforwardto
implementusingyield.Ifso,thenthisclassmightserveasanmodel.Ifyoudonotfeel
theneedtodothis,thenyoushouldignorethisexample:
#
#Aniteratorclassthatuses``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classYieldIteratorExample:
def__init__(self,seq):
self.seq=seq
self.iterator=self._next()
self.next=self.iterator.next
def_next(self):
flag=0
forxinself.seq:
ifflag:
flag=0
yieldx
else:
flag=1
def__iter__(self):
returnself.iterator
defrefresh(self):
self.iterator=self._next()
self.next=self.iterator.next
deftest_yielditeratorexample():
a=YieldIteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=YieldIteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
test_yielditeratorexample()

Runningthisexampleproducesthefollowingoutput:
Page104

APythonBook
d
b

d
b
==============================
b
d
stopping

Notesandexplanation:

Becausethe_nextmethodusesyield,callingit(actually,callingtheiterator
objectitproduces)inaniteratorcontextcausesittobe"resumed"immediately
aftertheyieldstatement.Thisreducesbookkeepingabit.
However,withthisstyle,wemustexplicitlyproduceaniterator.Wedothisby
callingthe_nextmethod,whichcontainsayieldstatement,andisthereforea
generator.Thefollowingcodeinourconstructor(__init__)completesthe
setupofourclassasaniteratorclass:
self.iterator=self._next()
self.next=self.iterator.next

Rememberthatweneedboth__iter__()andnext()methodsin
YieldIteratorExampletosatisfytheiteratorprotocol.The__iter__()
methodisalreadythereandtheabovecodeintheconstructorcreatesthenext()
method.

2.3.5ExampleAlistcomprehension
Alistcomprehensionlooksabitlikeaniterator,butitproducesalist.See:ThePython
LanguageReference:Listdisplays
http://docs.python.org/reference/expressions.html#listdisplaysformoreonlist
comprehensions.
Hereisanexample:
In[4]:deff(x):
...:returnx*3
...:
In[5]:list1=[11,22,33]
In[6]:list2=[f(x)forxinlist1]
In[7]:printlist2
[33,66,99]

2.3.6ExampleAgeneratorexpression
Ageneratorexpressionlooksquitesimilartoalistcomprehension,butisenclosedin
Page105

APythonBook
parenthesesratherthansquarebrackets.Unlikealistcomprehension,agenerator
expressiondoesnotproducealist;itproducesangeneratorobject.Ageneratorobjectis
aniterator.
Formoreongeneratorexpressions,seeThePythonLanguageReference:Generator
expressionshttp://docs.python.org/reference/expressions.html#generatorexpressions.
Thefollowingexampleusesageneratorexpressiontoproduceaniterator:
mylist=range(10)
deff(x):
returnx*3
genexpr=(f(x)forxinmylist)
forxingenexpr:
printx

Notesandexplanation:

Thegeneratorexpression(f(x)forxinmylist)producesaniteratorobject.
Noticethatwecanusetheiteratorobjectlaterinourcode,cansaveitinadata
structure,andcanpassittoafunction.

2.4UnitTests
UnittestandthePythonunittestframeworkprovideaconvenientwaytodefineandrun
teststhatensurethataPythonapplicationproducesspecifiedresults.
Thissection,whileitwillnotattempttoexplaineverythingabouttheunittestframework,
willprovideexamplesofseveralstraightforwardwaystoconstructandruntests.
Someassumptions:

Wearegoingtodevelopasoftwareprojectincrementally.Wewillnotimplement
andreleaseallatonce.Therefore,eachtimeweaddtoourexistingcodebase,we
needawaytoverifythatouradditions(andfixes)havenotcausednewproblems
inoldcode.
Addingnewcodetoexistingcodewillcauseproblems.Weneedtobeableto
check/testforthoseproblemsateachstep.
Asweaddcode,weneedtobeabletoaddtestsforthatnewcode,too.

2.4.1Definingunittests
2.4.1.1Createatestclass.

Inthetestclass,implementanumberofmethodstoperformyourtests.Nameyourtest
Page106

APythonBook
methodswiththeprefix"test".Hereisanexample:
importunittest
classMyTest(unittest.TestCase):
deftest_one(self):
#sometestcode
pass
deftest_two(self):
#sometestcode
pass

Createatestharness.Hereisanexample:
importunittest
#makethetestsuite.
defsuite():
loader=unittest.TestLoader()
testsuite=loader.loadTestsFromTestCase(MyTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
result=runner.run(testsuite)

Hereisamorecompleteexample:
importsys,StringIO,string
importunittest
importwebserv_example_heavy_sub
#Acomparisonfunctionforcaseinsenstivesorting.
defmycmpfunc(arg1,arg2):
returncmp(string.lower(arg1),string.lower(arg2))
classXmlTest(unittest.TestCase):
deftest_import_export1(self):
inFile=file('test1_in.xml','r')
inContent=inFile.read()
inFile.close()
doc=webserv_example_heavy_sub.parseString(inContent)
outFile=StringIO.StringIO()
outFile.write('<?xmlversion="1.0"?>\n')
doc.export(outFile,0)
outContent=outFile.getvalue()
outFile.close()
self.failUnless(inContent==outContent)
#makethetestsuite.
defsuite():

Page107

APythonBook
loader=unittest.TestLoader()
#Changethetestmethodprefix:test>trial.
#loader.testMethodPrefix='trial'
#Changethecomparisonfunctionthatdeterminestheorderof
tests.
#loader.sortTestMethodsUsing=mycmpfunc
testsuite=loader.loadTestsFromTestCase(XmlTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest_main():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
result=runner.run(testsuite)
if__name__=="__main__":
test_main()

Runningtheabovescriptproducesthefollowingoutput:
test_import_export(__main__.XmlTest)...ok

Ran1testin0.035s
OK

Afewnotesonthisexample:

Thisexampleteststheabilitytoparseanxmldocumenttest1_in.xmlandexport
thatdocumentbacktoXML.ThetestsucceedsiftheinputXMLdocumentand
theexportedXMLdocumentarethesame.
ThecodewhichisbeingtestedparsesanXMLdocumentreturnedbyarequestto
AmazonWebservices.YoucanlearnmoreaboutAmazonWebservicesat:
http://www.amazon.com/webservices.ThiscodewasgeneratedfromanXML
SchemadocumentbygenerateDS.py.Soweareineffect,testinggenerateDS.py.
YoucanfindgenerateDS.pyat:
http://http://www.davekuhlman.org/#generatedspy.
Testingforsuccess/failureandreportingfailuresUsethemethodslistedat
http://www.python.org/doc/current/lib/testcaseobjects.htmltotestforandreport
successandfailure.Inourexample,weused"self.failUnless(inContent==
outContent)"toensurethatthecontentweparsedandthecontentthatwe
exportedwerethesame.
Addadditionaltestsbyaddingmethodswhosenameshavetheprefix"test".If
youpreferadifferentprefixfortestsnames,addsomethinglikethefollowingto
theabovescript:
loader.testMethodPrefix='trial'

Page108

APythonBook

Bydefault,thetestsarerunintheorderoftheirnamessortedbythecmp
function.So,ifneeded,youcancontroltheorderofexecutionoftestsby
selectingtheirnames,forexample,usingnamesliketest_1_checkderef,
test_2_checkcalc,etc.Or,youcanchangethecomparisonfunctionbyadding
somethinglikethefollowingtotheabovescript:
loader.sortTestMethodsUsing=mycmpfunc

Asabitofmotivationforcreatingandusingunittests,whiledevelopingthisexample,I
discoveredseveralerrors(ormaybe"specialfeatures")ingenerateDS.py.

2.5ExtendingandembeddingPython
2.5.1Introductionandconcepts
Extendingvs.embeddingTheyaredifferentbutrelated:
ExtendingPythonmeanstoimplementanextensionmoduleoranextensiontype.
AnextensionmodulecreatesanewPythonmodulewhichisimplementedin
C/C++.FromPythoncode,anextensionmoduleappearstobejustlikeamodule
implementedinPythoncode.AnextensiontypecreatesanewPython(builtin)
typewhichisimplementedinC/C++.FromPythoncode,anextensiontype
appearstobejustlikeabuiltintype.
EmbeddingPython,bycontrast,istoputthePythoninterpreterwithinan
application(i.e.linkitin)sothattheapplicationcanrunPythonscripts.The
scriptscanbeexecutedortriggeredinavarietyofways,e.g.theycanbeboundto
keysonthekeyboardortomenuitems,theycanbetriggeredbyexternalevents,
etc.Usually,inordertomaketheembeddedPythoninterpreteruseful,Pythonis
alsoextendedwithfunctionsfromtheembeddingapplication,sothatthescripts
cancallfunctionsthatareimplementedbytheembeddingC/C++application.
DocumentationThetwoimportantsourcesforinformationaboutextendingand
embeddingarethefollowing:

ExtendingandEmbeddingthePythonInterpreter
http://www.python.org/doc/current/ext/ext.html
Python/CAPIReferenceManual
http://www.python.org/doc/current/api/api.html
Typesofextensions:

ExtensionmodulesFromthePythonside,itappearstobeaPythonmodule.
Usuallyitexportsfunctions.
ExtensiontypesUsedtoimplementanewPythondatatype.
ExtensionclassesFromthePythonside,itappearstobeaclass.
Page109

APythonBook
ToolsThereareseveraltoolsthatsupportthedevelopmentofPythonextensions:

SWIGLearnaboutSWIGat:http://www.swig.org
PyrexLearnaboutPyrexat:
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
ThereisalsoCython,whichseemstobeanadvancedversionof,oratleastan
alternativetoPyrex.See:CythonCExtensionsforPython
http://www.cython.org/

2.5.2Extensionmodules
WritinganextensionmodulebyhandWhattodo:
Createthe"init"functionThenameofthisfunctionmustbe"init"followedby
thenameofthemodule.Everyextensionmodulemusthavesuchafunction.
CreatethefunctiontableThistablemapsfunctionnames(referencedfrom
Pythoncode)tofunctionpointers(implementedinC/C++).
Implementeachwrapperfunction.
ImplementingawrapperfunctionWhattodo:

1. CapturetheargumentswithPyArg_ParseTuple.Theformatstringspecifieshow
argumentsaretobeconvertedandcaptured.See1.7ExtractingParametersin
ExtensionFunctions.Herearesomeofthemostcommonlyusedtypes:
Use"i","s","f",etctoconvertandcapturesimpletypessuchasintegers,
strings,floats,etc.
Use"O"togetapointertoPython"complex"typessuchaslists,tuples,
dictionaries,etc.
Useitemsinparenthesestocaptureandunpacksequences(e.g.listsand
tuples)offixedlength.Example:
if(!PyArg_ParseTuple(args,"(ii)(ii)",&x,&y,
&width,&height))
{
returnNULL;
}/*if*/

Asamplecallmightbe:
lowerLeft=(x1,y1)
extent=(width1,height1)
scan(lowerLeft,extent)

Use":aName"(colon)attheendoftheformatstringtoprovideafunction
nameforerrormessages.Example:
if(!PyArg_ParseTuple(args,"O:setContentHandler",
&pythonInstance))
{

Page110

APythonBook
returnNULL;
}/*if*/

Use";anerrormessage"(semicolon)attheendoftheformatstringtoprovide
astringthatreplacesthedefaulterrormessage.
Docsareavailableat:http://www.python.org/doc/current/ext/parseTuple.html.
2. Writethelogic.
3. HandleerrorsandexceptionsYouwillneedtounderstandhowto(1)clearing
errorsandexceptionsand(2)Raiseerrors(exceptions).
ManyfunctionsinthePythonCAPIraiseexceptions.Youwillneedtocheck
forandcleartheseexceptions.Hereisanexample:

char*message;
intmessageNo;
message=NULL;
messageNo=1;
/*Istheargumentastring?
*/
if(!PyArg_ParseTuple(args,"s",&message))
{
/*It'snotastring.Cleartheerror.
*Thentrytogetamessagenumber(an
integer).
*/
PyErr_Clear();
if(!PyArg_ParseTuple(args,"i",&messageNo))
{
o
o
o

YoucanalsoraiseexceptionsinyourCcodethatcanbecaught(ina
"try:except:"block)backinthecallingPythoncode.Hereisanexample:
if(n==0)
{
PyErr_SetString(PyExc_ValueError,"Valuemust
notbezero");
returnNULL;
}

SeeInclude/pyerrors.hinthePythonsourcedistributionformore
exception/errortypes.
And,youcantestwhetherafunctioninthePythonCAPIthatyouhavecalled
hasraisedanexception.Forexample:
if(PyErr_Occurred())
{
/*Anexceptionwasraised.
*Dosomethingaboutit.

Page111

APythonBook
*/
o
o
o

Formoredocumentationonerrorsandexceptions,see:
http://www.python.org/doc/current/api/exceptionHandling.html.
4. Createandreturnavalue:
ForeachbuiltinPythontypethereisasetofAPIfunctionstocreateand
manipulateit.Seethe"Python/CAPIReferenceManual"foradescriptionof
thesefunctions.Forexample,see:
http://www.python.org/doc/current/api/intObjects.html
http://www.python.org/doc/current/api/stringObjects.html
http://www.python.org/doc/current/api/tupleObjects.html
http://www.python.org/doc/current/api/listObjects.html
http://www.python.org/doc/current/api/dictObjects.html
Etc.
ThereferencecountYouwillneedtofollowPython'srulesforreference
countingthatPythonusestogarbagecollectobjects.Youcanlearnabout
theserulesathttp://www.python.org/doc/current/ext/refcounts.html.Youwill
notwantPythontogarbagecollectobjectsthatyoucreatetooearlyortoolate.
WithrespecttoPythonobjectscreatedwiththeabovefunctions,thesenew
objectsareownedandmaybepassedbacktoPythoncode.However,there
aresituationswhereyourC/C++codewillnotautomaticallyownareference,
forexamplewhenyouextractanobjectfromacontainer(alist,tuple,
dictionary,etc).Inthesecasesyoushouldincrementthereferencecountwith
Py_INCREF.

2.5.3SWIG
Note:OurdiscussionandexamplesareforSWIGversion1.3
SWIGwilloftenenableyoutogeneratewrappersforfunctionsinanexistingCfunction
library.SWIGdoesnotunderstandeverythinginCheaderfiles.Butitdoesafairly
impressivejob.Youshouldtryitfirstbeforeresortingtothehardworkofwriting
wrappersbyhand.
MoreinformationonSWIGisathttp://www.swig.org.
Herearesomestepsthatyoucanfollow:
1. CreateaninterfacefileEvenwhenyouarewrappingfunctionsdefinedinan
existingheaderfile,creatinganinterfacefileisagoodidea.Includeyourexisting
headerfileintoit,thenaddwhateverelseyouneed.Hereisanextremelysimple
exampleofaSWIGinterfacefile:
Page112

APythonBook
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
%include"MyLibrary.h"

Comments:
The"%{"and"%}"bracketsaredirectivestoSWIG.Theysay:"Addthecode
betweenthesebracketstothegeneratedwrapperfilewithoutprocessingit.
The"%include"statementsays:"Copythefileintotheinterfacefilehere.In
effect,youareaskingSWIGtogeneratewrappersforallthefunctionsinthis
headerfile.Ifyouwantwrappersforonlysomeofthefunctionsinaheader
file,thencopyorreproducefunctiondeclarationsforthedesiredfunctions
here.Anexample:
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
intcalcArea(intwidth,intheight);
intcalcVolume(intradius);

Thisexamplewillgeneratewrappersforonlytwofunctions.
YoucanfindmoreinformationaboutthedirectivesthatareusedinSWIG
interfacefilesintheSWIGUserManual,inparticularat:
http://www.swig.org/Doc1.3/Preprocessor.html
http://www.swig.org/Doc1.3/Python.html
2. Generatethewrappers:

swigpythonMyLibrary.i

3. Compileandlinkthelibrary.OnLinux,youcanusesomethinglikethefollowing:
gcccMyLibrary.c
gcccI/usr/local/include/python2.3MyLibrary_wrap.c
gccsharedMyLibrary.oMyLibrary_wrap.oo
_MyLibrary.so

Notethatweproduceasharedlibrarywhosenameisthemodulenameprefixed
withanunderscore.SWIGalsogeneratesa.pyfile,withouttheleading
underscore,whichwewillimportfromourPythoncodeandwhich,inturn,
importsthesharedlibrary.
4. Usetheextensionmoduleinyourpythoncode:
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2

Page113

APythonBook
Type"help","copyright","credits"or"license"for
moreinformation.
>>>importMyLibrary
>>>MyLibrary.calcArea(4.0,5.0)
20.0

Hereisamakefilethatwillexecuteswigtogeneratewrappers,thencompileandlinkthe
extension.
CFLAGS=I/usr/local/include/python2.3
all:_MyLibrary.so
_MyLibrary.so:MyLibrary.oMyLibrary_wrap.o
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so
MyLibrary.o:MyLibrary.c
gcccMyLibrary.coMyLibrary.o
MyLibrary_wrap.o:MyLibrary_wrap.c
gccc${CFLAGS}MyLibrary_wrap.coMyLibrary_wrap.o
MyLibrary_wrap.c:MyLibrary.i
swigpythonMyLibrary.i
clean:
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c
MyLibrary_wrap.o_MyLibrary.so
Hereisanexampleofrunningthismakefile:
$makefMyLibrary_makefileclean
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c\
MyLibrary_wrap.o_MyLibrary.so
$makefMyLibrary_makefile
gcccMyLibrary.coMyLibrary.o
swigpythonMyLibrary.i
gcccI/usr/local/include/python2.3MyLibrary_wrap.co
MyLibrary_wrap.o
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so

And,hereareCsourcefilesthatcanbeusedinourexample.
MyLibrary.h:
/*MyLibrary.h
*/

Page114

APythonBook
floatcalcArea(floatwidth,floatheight);
floatcalcVolume(floatradius);
intgetVersion();
intgetMode();

MyLibrary.c:
/*MyLibrary.c
*/
floatcalcArea(floatwidth,floatheight)
{
return(width*height);
}
floatcalcVolume(floatradius)
{
return(3.14*radius*radius);
}
intgetVersion()
{
return123;
}
intgetMode()
{
return1;
}

2.5.4Pyrex
PyrexisausefultoolforwritingPythonextensions.BecausethePyrexlanguageis
similartoPython,writingextensionsinPyrexiseasierthandoingsoinC.Cython
appearstobetheanewerversionofPyrex.
MoreinformationisonPyrexandCythonisat:
Pyrexhttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
CythonCExtensionsforPythonhttp://www.cython.org/
HereisasimplefunctiondefinitioninPyrex:

#python_201_pyrex_string.pyx
importstring
defformatString(objects1,objects2):
s1=string.strip(s1)
s2=string.strip(s2)

Page115

APythonBook
s3='<<%s||%s>>'%(s1,s2)
s4=s3*4
returns4

And,hereisamakefile:
CFLAGS=DNDEBUGO3WallWstrictprototypesfPIC\
I/usr/local/include/python2.3
all:python_201_pyrex_string.so
python_201_pyrex_string.so:python_201_pyrex_string.o
gccsharedpython_201_pyrex_string.oo
python_201_pyrex_string.so
python_201_pyrex_string.o:python_201_pyrex_string.c
gccc${CFLAGS}python_201_pyrex_string.co
python_201_pyrex_string.o
python_201_pyrex_string.c:python_201_pyrex_string.pyx
pyrexcpython_201_pyrex_string.pyx
clean:
rmfpython_201_pyrex_string.sopython_201_pyrex_string.o\
python_201_pyrex_string.c

Hereisanotherexample.Inthisone,onefunctioninthe.pyxfilecallsanother.Hereis
theimplementationfile:
#python_201_pyrex_primes.pyx
defshowPrimes(intkmax):
plist=primes(kmax)
forpinplist:
print'prime:%d'%p
cdefprimes(intkmax):
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1

Page116

APythonBook
returnresult

And,hereisamakefile:
#CFLAGS=DNDEBUGgO3WallWstrictprototypesfPIC#
I/usr/local/include/python2.3CFLAGS=DNDEBUG
I/usr/local/include/python2.3
all:python_201_pyrex_primes.so
python_201_pyrex_primes.so:python_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so
python_201_pyrex_primes.o:python_201_pyrex_primes.c
gccc${CFLAGS}python_201_pyrex_primes.copython_201_pyrex_primes.o
python_201_pyrex_primes.c:python_201_pyrex_primes.pyx
pyrexcpython_201_pyrex_primes.pyx
clean:
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o
python_201_pyrex_primes.c
Hereistheoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeprimesclean
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o\
python_201_pyrex_primes.c
$makefpython_201_pyrex_makeprimes
pyrexcpython_201_pyrex_primes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_primes.copython_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so

Hereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>importpython_201_pyrex_primes
>>>dir(python_201_pyrex_primes)
['__builtins__','__doc__','__file__','__name__','showPrimes']
>>>python_201_pyrex_primes.showPrimes(5)
prime:2
prime:3
prime:5
prime:7

Page117

APythonBook
prime:11

ThisnextexampleshowshowtousePyrextoimplementanewextensiontype,thatisa
newPythonbuiltintype.Noticethattheclassisdeclaredwiththecdefkeyword,which
tellsPyrextogeneratetheCimplementationofatypeinsteadofaclass.
Hereistheimplementationfile:
#python_201_pyrex_clsprimes.pyx
"""Animplementationofprimeshandlingclass
forademonstrationofPyrex.
"""
cdefclassPrimes:
"""Aclasscontainingfunctionsfor
handlingprimes.
"""
defshowPrimes(self,intkmax):
"""Showarangeofprimes.
Usethemethodprimes()togeneratetheprimes.
"""
plist=self.primes(kmax)
forpinplist:
print'prime:%d'%p
defprimes(self,intkmax):
"""Generatetheprimesintherange0kmax.
"""
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1
returnresult

And,hereisamakefile:
CFLAGS=DNDEBUGI/usr/local/include/python2.3
all:python_201_pyrex_clsprimes.so

Page118

APythonBook
python_201_pyrex_clsprimes.so:python_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o:python_201_pyrex_clsprimes.c
gccc${CFLAGS}python_201_pyrex_clsprimes.co
python_201_pyrex_clsprimes.o
python_201_pyrex_clsprimes.c:python_201_pyrex_clsprimes.pyx
pyrexcpython_201_pyrex_clsprimes.pyx
clean:
rmfpython_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c

Hereisoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeclsprimesclean
rmfpython_201_pyrex_clsprimes.sopython_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c
$makefpython_201_pyrex_makeclsprimes
pyrexcpython_201_pyrex_clsprimes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_clsprimes.copython_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so

Andhereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>importpython_201_pyrex_clsprimes
>>>dir(python_201_pyrex_clsprimes)
['Primes','__builtins__','__doc__','__file__','__name__']
>>>primes=python_201_pyrex_clsprimes.Primes()
>>>dir(primes)
['__class__','__delattr__','__doc__','__getattribute__',
'__hash__',
'__init__','__new__','__reduce__','__reduce_ex__','__repr__',
'__setattr__','__str__','primes','showPrimes']
>>>primes.showPrimes(4)
prime:2
prime:3
prime:5
prime:7

DocumentationAlsonoticethatPyrexpreservesthedocumentationforthemodule,the
class,andthemethodsintheclass.Youcanshowthisdocumentationwithpydoc,as
Page119

APythonBook
follows:
$pydocpython_201_pyrex_clsprimes

Or,inPythoninteractivemode,use:
$python
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>importpython_201_pyrex_clsprimes
>>>help(python_201_pyrex_clsprimes)

2.5.5SWIGvs.Pyrex
ChooseSWIGwhen:
YoualreadyhaveanexistingCorC++implementationofthecodeyouwantto
callfromPython.InthiscaseyouwantSWIGtogeneratethewrappers.Butnote
thatCythonpromisestoenableyoutoquicklywrapandcallfunctions
implementedinC.
YouwanttowritetheimplementationinCorC++byhand.Perhaps,becauseyou
thinkyoucandosoquickly,forexample,orbecauseyoubelievethatyoucan
makeithighlyoptimized.Then,youwanttobeabletogeneratethePython
(extension)wrappersforitquickly.
ChoosePyrexwhen:

YoudonothaveaC/C++implementationandyouwantaneasierwaytowrite
thatCimplementation.WritingPyrexcode,whichisalotlikePython,iseasier
thanwritingCorC++codebyhand).
YoustarttowritetheimplementationinC,thenfindthatitrequireslotsofcallsto
thePythonCAPI,andyouwanttoavoidhavingtolearnhowtodothat.

2.5.6Cython
HereisasimpleexamplethatusesCythontowrapafunctionimplementedinC.
FirsttheCheaderfile:
/*test_c_lib.h*/
intcalculate(intwidth,intheight);

And,theCimplementationfile:
/*test_c_lib.c*/

Page120

APythonBook
#include"test_c_lib.h"
intcalculate(intwidth,intheight)
{
intresult;
result=width*height*3;
returnresult;
}

HereisaCythonfilethatcallsourCfunction:
#test_c.pyx
#DeclaretheexternalCfunction.
cdefexternfrom"test_c_lib.h":
intcalculate(intwidth,intheight)
deftest(w,h):
#CalltheexternalCfunction.
result=calculate(w,h)
print'resultfromcalculate:%d'%result

Wecancompileourcodeusingthisscript(onLinux):
#!/bin/bashx
cythontest_c.pyx
gcccfPICI/usr/local/include/python2.6otest_c.otest_c.c
gcccfPICI/usr/local/include/python2.6otest_c_lib.o
test_c_lib.c
gccsharedfPICI/usr/local/include/python2.6otest_c.so
test_c.otest_c_lib.o

HereisasmallPythonfilethatusesthewrapperthatwewroteinCython:
#run_test_c.py
importtest_c
deftest():
test_c.test(4,5)
test_c.test(12,15)
if__name__=='__main__':
test()

And,whenwerunit,weseethefollowing:
$pythonrun_test_c.py
resultfromcalculate:60
resultfromcalculate:540

Page121

APythonBook

2.5.7Extensiontypes
ThegoalAnewbuiltindatatypeforPython.
ExistingexamplesObjects/listobject.c,Objects/stringobject.c,Objects/dictobject.c,etc
inthePythonsourcecodedistribution.
InolderversionsofthePythonsourcecodedistribution,atemplatefortheCcodewas
providedinObjects/xxobject.c.Objects/xxobject.cisnolongerincludedinthePython
sourcecodedistribution.However:
Thediscussionandexamplesforcreatingextensiontypeshavebeenexpanded.
See:ExtendingandEmbeddingthePythonInterpreter,2.DefiningNewTypes
http://docs.python.org/extending/newtypes.html.
IntheTools/framerdirectoryofthePythonsourcecodedistributionthereisan
applicationthatwillgenerateaskeletonforanextensiontypefromaspecification
objectwritteninPython.RunTools/framer/example.pytoseeitinaction.
And,youcanusePyrextogenerateanewbuiltintype.Todoso,implementa
Python/PyrexclassanddeclaretheclasswiththePyrexkeywordcdef.Infact,youmay
wanttousePyrextogenerateaminimalextensiontype,andtheneditthatgeneratedcode
toinsertandaddfunctionalitybyhand.SeethePyrexsectionforanexample.

Pyrexalsogoessomewaytowardgivingyouaccessto(existing)Cstructsandfunctions
fromPython.

2.5.8Extensionclasses
ExtensionclassestheeasywaySWIGshadowclasses.
StartwithanimplementationofaC++classanditsheaderfile.
UsethefollowingSWIGflags:
swigc++pythonmymodule.i

MoreinformationisavailablewiththeSWIGdocumentationat:
http://www.swig.org/Doc1.3/Python.html.
ExtensionclassesthePyrexwayAnalternatieistousePyrextocompileaclass
definitionthatdoesnothavethecdefkeyword.UsingcdefontheclasstellsPyrexto
generateanextensiontypeinsteadofaclass.Youwillhavetodeterminewhetheryou
wantanextensionclassoranextensiontype.

2.6Parsing
Pythonisanexcellentlanguagefortextanalysis.
Page122

APythonBook
Insomecases,simplysplittinglinesoftextintowordswillbeenough.Inthesecasesuse
string.split().
Inothercases,regularexpressionsmaybeabletodotheparsingyouneed.Ifso,seethe
sectiononregularexpressionsinthisdocument.
However,insomecases,morecomplexanalysisofinputtextisrequired.Thissection
describessomeofthewaysthatPythoncanhelpyouwiththiscomplexparsingand
analysis.

2.6.1Specialpurposeparsers
ThereareanumberofspecialpurposeparserswhichyouwillfindinthePythonstandard
library:
ConfigParserparserConfigurationfileparser
http://docs.python.org/library/configparser.html
getoptParserforcommandlineoptions
http://docs.python.org/library/getopt.html
optparseMorepowerfulcommandlineoptionparser
http://docs.python.org/library/optparse.html
urlparseParseURLsintocomponents
http://docs.python.org/library/urlparse.html
csvCSV(commaseparatedvalues)FileReadingandWriting
http://docs.python.org/library/csv.html#modulecsv
os.pathCommonpathnamemanipulations
http://docs.python.org/library/os.path.html
XMLparsersandXMLtoolsThereislotsofsupportforparsingandprocessingXML
inPython.Hereareafewplacestolookforsupport:

ThePythonstandardlibraryStructuredMarkupProcessingTools
http://docs.python.org/library/markup.html.
Inparticular,youmaybeinterestedinxml.dom.minidomLightweightDOM
implementationhttp://docs.python.org/library/xml.dom.minidom.html.
ElementTreeYoucanthinkofElementTreeasanenhancedDOM(document
objectmodel).Manyfinditeasiertousethanminidom.ElementTreeisinthe
Pythonstandardlibrary,anddocumentationishere:ElementTreeOverview
http://effbot.org/zone/elementindex.htm.
LxmlmimicstheElementTreeAPI,buthasadditionalcapabilities.Findoutabout
Lxmlatlxmlhttp://codespeak.net/lxml/index.htmlNotethatlxmlalsohas
supportforXPathandXSLT.
Dave'ssupportforPythonandXMLhttp://www.rexx.com/~dkuhlman.

Page123

APythonBook

2.6.2Writingarecursivedescentparserbyhand
Forsimplegrammars,thisisnotsohard.
Youwillneedtoimplement:
Arecognizermethodorfunctionforeachproductionruleinyourgrammar.Each
recognizermethodbeginslookingatthecurrenttoken,thenconsumesasmany
tokensasneededtorecognizeit'sownproductionrule.Itcallstherecognizer
functionsforanynonterminalsonitsrighthandside.
AtokenizerSomethingthatwillenableeachrecognizerfunctiontogettokens,
onebyone.Thereareavarietyofwaystodothis,e.g.(1)afunctionthat
producesalistoftokensfromwhichrecognizerscanpoptokens;(2)agenerator
whosenextmethodreturnsthenexttoken;etc.
Asanexample,we'llimplementarecursivedescentparserwritteninPythonforthe
followinggrammer:

Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>

Hereisanimplementationofarecursivedescentparserfortheabovegrammar:
#!/usr/bin/envpython
"""
Arecursivedescentparserexample.
Usage:
pythonrparser.py[options]<inputfile>
Options:
h,helpDisplaythishelpmessage.
Example:
pythonrparser.pymyfile.txt
Thegrammar:
Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>
"""
importsys
importstring
importtypes
importgetopt

Page124

APythonBook
#
#TousetheIPythoninteractiveshelltoinspectyourrunning
#application,uncommentthefollowinglines:
#
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Thenaddthefollowinglineatthepointinyourcodewhere
#youwanttoinspectruntimevalues:
#
#ipshell('somemessagetoidentifywhereweare')
#
#Formoreinformationsee:http://ipython.scipy.org/moin/
#
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
FuncCallListNodeType=4
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType

Page125

APythonBook
self.children=[]
foriteminargs:
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
forchildinself.children:
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
item.show(level)
else:
self.showLevel(level)
print'Child:',child
defshowLevel(self,level):
foridxinrange(level):
print'',
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
pass
defparseFile(self,infileName):
self.infileName=infileName
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=file(self.infileName,'r')
self.tokens=genTokens(self.infile)
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=genTokens(instream,'<instream>')
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:

Page126

APythonBook
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
result=self.func_call_reco()
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokenType,self.token,self.lineNo=
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missingright
paren')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleftparen')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
result=self.func_call_reco()
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:

Page127

APythonBook
break
self.tokenType,self.token,self.lineNo=
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
defis_word(token):
forletterintoken:
ifletternotinstring.ascii_letters:
returnNone
return1
#
#Generatethetokens.
#Usage:
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile):
lineNo=0
while1:
lineNo+=1
try:
line=infile.next()
except:
yield(EOFTokType,None,lineNo)
toks=line.split()
fortokintoks:
ifis_word(tok):
tokType=WordTokType
eliftok=='(':
tokType=LParTokType
eliftok==')':
tokType=RParTokType
eliftok==',':
tokType=CommaTokType
yield(tokType,tok,lineNo)
deftest(infileName):
parser=ProgParser()
#ipshell('(test)#1\nCtrlDtoexit')

Page128

APythonBook
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=1:
usage()
inputfile=args[0]
test(inputfile)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Commentsandexplanation:

ThetokenizerisaPythongenerator.ItreturnsaPythongeneratorthatcan
produce"(tokType,tok,lineNo)"tuples.Ourtokenizerissosimplemindedthat
wehavetoseparateallofourtokenswithwhitespace.(Alittlelater,we'llseehow
tousePlextoovercomethislimitation.)
Theparserclass(ProgParser)containstherecognizermethodsthatimplementthe
productionrules.Eachofthesemethodsrecognizesasyntacticconstructdefined
byarule.Inourexample,thesemethodshavenamesthatendwith"_reco".
Wecouldhave,alternatively,implementedourrecognizersasglobalfunctions,
insteadofasmethodsinaclass.However,usingaclassgivesusaplaceto"hang"
thevariablesthatareneededacrossmethodsandsavesusfromhavingtouse
("evil")globalvariables.
Arecognizermethodrecognizesterminals(syntacticelementsontherighthand
sideofthegrammarruleforwhichthereisnogrammarrule)by(1)checkingthe
tokentypeandthetokenvalue,andthen(2)callingthetokenizertogetthenext
token(becauseithasconsumedatoken).
Page129

APythonBook
Arecognizermethodchecksforandprocessesanonterminal(syntacticelements
ontherighthandsideforwhichthereisagrammarrule)bycallingtherecognizer
methodthatimplementsthatnonterminal.
Ifarecognizermethodfindsasyntaxerror,itraisesanexceptionofclass
ParserError.
SinceourexamplerecursivedescentparsercreatesanAST(anabstractsyntax
tree),wheneverarecognizermethodsuccessfullyrecognizesasyntacticconstruct,
itcreatesaninstanceofclassASTNodetorepresentitandreturnsthatinstanceto
itscaller.TheinstanceofASTNodehasanodetypeandcontainschildnodes
whichwereconstructedbyrecognizermethodscalledbythisone(i.e.that
representnonterminalsontherighthandsideofagrammarrule).
Eachtimearecognizermethod"consumesatoken",itcallsthetokenizertoget
thenexttoken(andtokentypeandlinenumber).
Thetokenizerreturnsatokentypeinadditiontothetokenvalue.Italsoreturnsa
linenumberforerrorreporting.
ThesyntaxtreeisconstructedfrominstancesofclassASTNode.
TheASTNodeclasshasashowmethod,whichwalkstheASTandproduces
output.Youcanimaginethatasimilarmethodcoulddocodegeneration.And,
youshouldconsiderthepossibilityofwritinganalogoustreewalkmethodsthat
performtaskssuchasoptimization,annotationoftheAST,etc.
And,hereisasampleofthedatawecanapplythisparserto:

aaa()
bbb(ccc())
ddd(eee(),fff(ggg(),hhh(),iii()))

And,ifweruntheparseronthethisinputdata,wesee:
$pythonworkbook045.pyworkbook045.data
NodeTypeProgNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:aaa
NodeTypeFuncCallListNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:bbb
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ccc
NodeTypeFuncCallListNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType

Page130

APythonBook
Child:ddd
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:eee
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:fff
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ggg
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:hhh
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:iii
NodeTypeFuncCallListNodeType

2.6.3Creatingalexer/tokenizerwithPlex
LexicalanalysisThetokenizerinourrecursivedescentparserexamplewas(for
demonstrationpurposes)overlysimple.Youcanalwayswritemorecomplextokenizers
byhand.However,formorecomplex(andreal)tokenizers,youmaywanttouseatoolto
buildyourtokenizer.
Inthissectionwe'lldescribePlexanduseittoproduceatokenizerforourrecursive
descentparser.
YoucanobtainPlexathttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inordertouseit,youmaywanttoaddPlex1.1.4/PlextoyourPYTHONPATH.
HereisasimpleexamplefromthePlextutorial:
#!/usr/bin/envpython
"""
SamplePlexlexer
Usage:
pythonplex_example.pyinputfile
"""
importsys
importPlex

Page131

APythonBook
defcount_lines(scanner,text):
scanner.line_count+=1
print''*60
deftest(infileName):
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
number=Plex.Rep1(digit)
space=Plex.Any("\t")
endline=Plex.Str('\n')
#comment=Plex.Str('"')+Plex.Rep(Plex.AnyBut('"'))+
Plex.Str('"')
resword=Plex.Str("if","then","else","end")
lexicon=Plex.Lexicon([
(endline,count_lines),
(resword,'keyword'),
(name,'ident'),
(number,'int'),
(Plex.Any("+*/=<>"),'operator'),
(space,Plex.IGNORE),
#(comment,'comment'),
(Plex.Str('('),'lpar'),
(Plex.Str(')'),'rpar'),
#commentssurroundedby(*and*)
(Plex.Str("(*"),Plex.Begin('comment')),
Plex.State('comment',[
(Plex.Str("*)"),Plex.Begin('')),
(Plex.AnyChar,Plex.IGNORE),
]),
])
infile=open(infileName,"r")
scanner=Plex.Scanner(lexicon,infile,infileName)
scanner.line_count=0
whileTrue:
token=scanner.read()
iftoken[0]isNone:
break
position=scanner.position()
posstr=('(%d,%d)'%(position[1],
position[2],)).ljust(10)
tokstr='"%s"'%token[1]
tokstr=tokstr.ljust(20)
print'%stok:%stokType:%s'%(posstr,tokstr,token[0],)
print'line_count:%d'%scanner.line_count
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]

Page132

APythonBook
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Hereisabitofdataonwhichwecanusetheabovelexer:
mass=(height*(*somecomment*)width*depth)/density
totalmass=totalmass+mass

And,whenweapplytheabovetestprogramtothisdata,hereiswhatwesee:
$pythonplex_example.pyplex_example.data
(1,0)tok:"mass"tokType:ident
(1,5)tok:"="tokType:operator
(1,7)tok:"("tokType:lpar
(1,8)tok:"height"tokType:ident
(1,15)tok:"*"tokType:operator
(1,36)tok:"width"tokType:ident
(1,42)tok:"*"tokType:operator
(1,44)tok:"depth"tokType:ident
(1,49)tok:")"tokType:rpar
(1,51)tok:"/"tokType:operator
(1,53)tok:"density"tokType:ident

(2,0)tok:"totalmass"tokType:ident
(2,10)tok:"="tokType:operator
(2,12)tok:"totalmass"tokType:ident
(2,22)tok:"+"tokType:operator
(2,24)tok:"mass"tokType:ident

line_count:2

Commentsandexplanation:

Createalexiconfromscanningpatterns.
SeethePlextutorialandreference(andbelow)formoreinformationonhowto
constructthepatternsthatmatchvarioustokens.
Createascannerwithalexicon,aninputfile,andaninputfilename.
Thecall"scanner.read()"getsthenexttoken.Itreturnsatuplecontaining(1)the
tokenvalueand(2)thetokentype.
Thecall"scanner.position()"getsthepositionofthecurrenttoken.Itreturnsa
tuplecontaining(1)theinputfilename,(2)thelinenumber,and(3)thecolumn
number.
Wecanexecuteamethodwhenagiventokenisfoundbyspecifyingthefunction
asthetokenaction.Inourexample,thefunctioniscount_lines.Maintainingaline
Page133

APythonBook
countisactuallyunneeded,sincethepositiongivesusthisinformation.However,
noticehowweareabletomaintainavalue(inourcaseline_count)asan
attributeofthescanner.
And,herearesomecommentsonconstructingthepatternsusedinalexicon:
Plex.Rangeconstructsapatternthatmatchesanycharacterintherange.
Plex.Repconstructsapatternthatmatchesasequenceofzeroormoreitems.
Plex.Rep1constructsapatternthatmatchesasequenceofoneormoreitems.
pat1+pat2constructsapatternthatmatchesasequencecontainingpat1
followedbypat2.
pat1|pat2constructsapatternthatmatcheseitherpat1orpat2.
Plex.Anyconstructsapatternthatmatchesanyonecharacterinitsargument.
Nowlet'srevisitourrecursivedescentparser,thistimewithatokenizerbuiltwithPlex.
Thetokenizeristrivial,butwillserveasanexampleofhowtohookitintoaparser:

#!/usr/bin/envpython
"""
ArecursivedescentparserexampleusingPlex.
ThisexampleusesPlextoimplementatokenizer.
Usage:
pythonpython_201_rparser_plex.py[options]<inputfile>
Options:
h,helpDisplaythishelpmessage.
Example:
pythonpython_201_rparser_plex.pymyfile.txt
Thegrammar:
Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>
"""
importsys,string,types
importgetopt
importPlex
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Constants
#

Page134

APythonBook
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
FuncCallListNodeType=4
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType
self.children=[]
foriteminargs:
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
forchildinself.children:
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
item.show(level)
else:
self.showLevel(level)
print'Child:',child
defshowLevel(self,level):
foridxinrange(level):
print'',

Page135

APythonBook
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=None
self.tokens=None
defparseFile(self,infileName):
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=file(infileName,'r')
self.tokens=genTokens(self.infile,infileName)
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.tokens=genTokens(self.instream,'<stream>')
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptystream'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break

Page136

APythonBook
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
result=self.func_call_reco()
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokenType,self.token,self.lineNo=
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missingright
paren')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleftparen')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
result=self.func_call_reco()
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:
break
self.tokenType,self.token,self.lineNo=
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):

Page137

APythonBook
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
#
#Generatethetokens.
#Usageexample
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile,infileName):
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
lpar=Plex.Str('(')
rpar=Plex.Str(')')
comma=Plex.Str(',')
comment=Plex.Str("#")+Plex.Rep(Plex.AnyBut("\n"))
space=Plex.Any("\t\n")
lexicon=Plex.Lexicon([
(name,'word'),
(lpar,'lpar'),
(rpar,'rpar'),
(comma,'comma'),
(comment,Plex.IGNORE),
(space,Plex.IGNORE),
])
scanner=Plex.Scanner(lexicon,infile,infileName)
while1:
tokenType,token=scanner.read()
name,lineNo,columnNo=scanner.position()
iftokenType==None:
tokType=EOFTokType
token=None
eliftokenType=='word':
tokType=WordTokType
eliftokenType=='lpar':
tokType=LParTokType
eliftokenType=='rpar':
tokType=RParTokType
eliftokenType=='comma':
tokType=CommaTokType
else:
tokType=NoneTokType
tok=token
yield(tokType,tok,lineNo)
deftest(infileName):
parser=ProgParser()

Page138

APythonBook
#ipshell('(test)#1\nCtrlDtoexit')
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

And,hereisasampleofthedatawecanapplythisparserto:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
bbb(ccc())#Anendoflinecomment.
#Command#3
ddd(eee(),fff(ggg(),hhh(),iii()))
#Endoftest

And,whenwerunourparser,itproducesthefollowing:
$pythonplex_recusive.pyplex_recusive.data
NodeTypeProgNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:aaa
NodeTypeFuncCallListNodeType
NodeTypeCommandNodeType

Page139

APythonBook
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:bbb
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ccc
NodeTypeFuncCallListNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ddd
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:eee
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:fff
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:ggg
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:hhh
NodeTypeFuncCallListNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:iii
NodeTypeFuncCallListNodeType

Comments:

Wecannowputcommentsinourinput,andtheywillbeignored.Comments
beginwitha"#"andcontinuetotheendofline.Seethedefinitionofcommentin
functiongenTokens.
Thistokenizerdoesnotrequireustoseparatetokenswithwhitespaceasdidthe
simpletokenizerintheearlierversionofourrecursivedescentparser.
Thechangeswemadeovertheearlierversionwereto:
1. ImportPlex.
2. ReplacethedefinitionofthetokenizerfunctiongenTokens.
3. ChangethecalltogenTokenssothatthecallpassesinthefilename,whichis
neededtocreatethescanner.
OurnewversionofgenTokensdoesthefollowing:
1. Createpatternsforscanning.
2. Createalexicon(aninstanceofPlex.Lexicon),whichusesthepatterns.
Page140

APythonBook
3. Createascanner(aninstanceofPlex.Scanner),whichusesthelexicon.
4. Executealoopthatreadstokens(fromthescanner)and"yields"eachone.

2.6.4Asurveyofexistingtools
Forcomplexparsingtasks,youmaywanttoconsiderthefollowingtools:
kwParsingAparsergeneratorinPython
http://gadfly.sourceforge.net/kwParsing.html
PLYPythonLexYacchttp://systems.cs.uchicago.edu/ply/
PyLRFastLRparsinginpython
http://starship.python.net/crew/scott/PyLR.html
YappsTheYappsParserGeneratorSystem
http://theory.stanford.edu/~amitp/Yapps/
And,forlexicalanalysis,youmayalsowanttolookhere:

UsingRegularExpressionsforLexicalAnalysis
http://effbot.org/zone/xmlscanner.htm
Plexhttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inthesectionsbelow,wegiveexamplesandnotesabouttheuseofPLYandpyparsing.

2.6.5CreatingaparserwithPLY
InthissectionwewillshowhowtoimplementourparserexamplewithPLY.
FirstdownloadPLY.Itisavailablehere:PLY(PythonLexYacc)
http://www.dabeaz.com/ply/
ThenaddthePLYdirectorytoyourPYTHONPATH.
LearnhowtoconstructlexersandparserswithPLYbyreadingdoc/ply.htmlinthe
distributionofPLYandbylookingattheexamplesinthedistribution.
Forthoseofyouwhowantamorecomplexexample,seeAPythonParserforthe
RELAXNGCompactSyntax,whichisimplementedwithPLY.
Now,hereisourexampleparser.Commentsandexplanationsarebelow:
#!/usr/bin/envpython
"""
Aparserexample.
ThisexampleusesPLYtoimplementalexerandparser.
Thegrammar:
Prog::=Command*
Command::=Func_call

Page141

APythonBook
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call*
Term=<word>
Hereisasample"program"touseasinput:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
bbb(ccc())#Anendoflinecomment.
#Command#3
ddd(eee(),fff(ggg(),hhh(),iii()))
#Endoftest
"""
importsys
importtypes
importgetopt
importply.lexaslex
importply.yaccasyacc
#
#Globals
#
startlinepos=0
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
CommandListNodeType=3
FuncCallNodeType=4
FuncCallListNodeType=5
TermNodeType=6
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
CommandListNodeType:'CommandListNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#

Page142

APythonBook
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType
self.children=[]
foriteminargs:
self.children.append(item)
defappend(self,item):
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType:%s'%NodeTypeDict[self.nodeType]
level+=1
forchildinself.children:
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
item.show(level)
else:
self.showLevel(level)
print'Value:',child
defshowLevel(self,level):
foridxinrange(level):
print'',
#
#Exceptionclasses
#
classLexerError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Lexererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
classParserError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Parsererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
#
#Lexerspecification
#
tokens=(
'NAME',

Page143

APythonBook
'LPAR','RPAR',
'COMMA',
)
#Tokens
t_LPAR=r'\('
t_RPAR=r'\)'
t_COMMA=r'\,'
t_NAME=r'[azAZ_][azAZ09_]*'
#Ignorewhitespace
t_ignore='\t'
#Ignorecomments('#'toendofline)
deft_COMMENT(t):
r'\#[^\n]*'
pass
deft_newline(t):
r'\n+'
globalstartlinepos
startlinepos=t.lexer.lexpos1
t.lineno+=t.value.count("\n")
deft_error(t):
globalstartlinepos
msg="Illegalcharacter'%s'"%(t.value[0])
columnno=t.lexer.lexposstartlinepos
raiseLexerError(msg,t.lineno,columnno)
#
#Parserspecification
#
defp_prog(t):
'prog:command_list'
t[0]=ASTNode(ProgNodeType,t[1])
defp_command_list_1(t):
'command_list:command'
t[0]=ASTNode(CommandListNodeType,t[1])
defp_command_list_2(t):
'command_list:command_listcommand'
t[1].append(t[2])
t[0]=t[1]
defp_command(t):
'command:func_call'
t[0]=ASTNode(CommandNodeType,t[1])
defp_func_call_1(t):
'func_call:termLPARRPAR'

Page144

APythonBook
t[0]=ASTNode(FuncCallNodeType,t[1])
defp_func_call_2(t):
'func_call:termLPARfunc_call_listRPAR'
t[0]=ASTNode(FuncCallNodeType,t[1],t[3])
defp_func_call_list_1(t):
'func_call_list:func_call'
t[0]=ASTNode(FuncCallListNodeType,t[1])
defp_func_call_list_2(t):
'func_call_list:func_call_listCOMMAfunc_call'
t[1].append(t[3])
t[0]=t[1]
defp_term(t):
'term:NAME'
t[0]=ASTNode(TermNodeType,t[1])
defp_error(t):
globalstartlinepos
msg="Syntaxerrorat'%s'"%t.value
columnno=t.lexer.lexposstartlinepos
raiseParserError(msg,t.lineno,columnno)
#
#ParsetheinputanddisplaytheAST(abstractsyntaxtree)
#
defparse(infileName):
startlinepos=0
#Buildthelexer
lex.lex(debug=1)
#Buildtheparser
yacc.yacc()
#Readtheinput
infile=file(infileName,'r')
content=infile.read()
infile.close()
try:
#Dotheparse
result=yacc.parse(content)
#DisplaytheAST
result.show(0)
exceptLexerError,exp:
exp.show()
exceptParserError,exp:
exp.show()
USAGE_TEXT=__doc__
defusage():
printUSAGE_TEXT
sys.exit(1)

Page145

APythonBook
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=1:
usage()
infileName=args[0]
parse(infileName)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Applyingthisparsertothefollowinginput:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
bbb(ccc())#Anendoflinecomment.
#Command#3
ddd(eee(),fff(ggg(),hhh(),iii()))
#Endoftest

producesthefollowingoutput:
NodeType:ProgNodeType
NodeType:CommandListNodeType
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:aaa
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:bbb
NodeType:FuncCallListNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:ccc
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:ddd
NodeType:FuncCallListNodeType
NodeType:FuncCallNodeType

Page146

APythonBook
NodeType:TermNodeType
Value:eee
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:fff
NodeType:FuncCallListNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:ggg
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:hhh
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:iii

Commentsandexplanation:

CreatingthesyntaxtreeBasically,eachrule(1)recognizesanonterminal,(2)
createsanode(possiblyusingthevaluesfromtherighthandsideoftherule),and
(3)returnsthenodebysettingthevalueoft[0].Adeviationfromthisisthe
processingofsequences,discussedbelow.
Sequencesp_command_list_1andp_command_list_1showhowtohandle
sequencesofitems.Inthiscase:
p_command_list_1recognizesacommandandcreatesaninstanceof
ASTNodewithtypeCommandListNodeTypeandaddsthecommandtoitasa
child,and
p_command_list_2recognizesanadditionalcommandandaddsit(asachild)
totheinstanceofASTNodethatrepresentsthelist.
DistinguishingbetweendifferentformsofthesameruleInordertoprocess
alternativestothesameproductionruledifferently,weusedifferentfunctions
withdifferentimplementations.Forexample,weuse:
p_func_call_1torecognizeandprocess"func_call:termLPARRPAR"(a
functioncallwithoutarguments),and
p_func_call_2torecognizeandprocess"func_call:termLPARfunc_call_list
RPAR"(afunctioncallwitharguments).
ReportingerrorsOurparserreportsthefirsterrorandquits.We'vedonethisby
raisinganexceptionwhenwefindanerror.Weimplementtwoexceptionclasses:
LexerErrorandParserError.Implementingmorethanoneexceptionclassenables
ustodistinguishbetweendifferentclassesoferrors(notethemultipleexcept:
clausesonthetry:statementinfunctionparse).And,weuseaninstanceofthe
exceptionclassasacontainerinorderto"bubbleup"informationabouttheerror
(e.g.amessage,alinenumber,andacolumnnumber).

Page147

APythonBook

2.6.6Creatingaparserwithpyparsing
pyparsingisarelativelynewparsingpackageforPython.Itwasimplementedandis
supportedbyPaulMcGuireanditshowspromise.Itappearsespeciallyeasytouseand
seemsespeciallyappropriateinparticularforquickparsingtasks,althoughithasfeatures
thatmakesomecomplexparsingtaskseasy.ItfollowsaverynaturalPythonstylefor
constructingparsers.
Gooddocumentationcomeswiththepyparsingdistribution.Seefile
HowToUseParsing.html.So,Iwon'ttrytorepeatthathere.Whatfollowsisanattemptto
provideseveralquickexamplestohelpyousolvesimpleparsingtasksasquicklyas
possible.
Youwillalsowanttolookatthesamplesintheexamplesdirectory,whicharevery
helpful.Myexamplesbelowarefairlysimple.Youcanseemoreoftheabilityof
pyparsingtohandlecomplextasksintheexamples.
WheretogetitYoucanfindpyparsingat:PyparsingWikiHome
http://pyparsing.wikispaces.com/
HowtoinstallitPutthepyparsingmodulesomewhereonyourPYTHONPATH.
Andnow,hereareafewexamples.
2.6.6.1Parsingcommadelimitedlines

Note:Thisexampleisfordemonstrationpurposesonly.Ifyoureallytoneedtoparse
commadelimitedfields,youcanprobablydosomuchmoreeasilywiththeCSV(comma
separatedvalues)moduleinthePythonstandardlibrary.
Hereisasimplegrammarforlinescontainingfieldsseparatedbycommas:
importsys
frompyparsingimportalphanums,ZeroOrMore,Word
fieldDef=Word(alphanums)
lineDef=fieldDef+ZeroOrMore(","+fieldDef)
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test1.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
infile=file(infilename,'r')
forlineininfile:
fields=lineDef.parseString(line)
printfields

Page148

APythonBook
test()

Hereissomesampledata:
abcd,defg
11111,22222,33333

And,whenwerunourparseronthisdatafile,hereiswhatwesee:
$pythoncomma_parser.pysample1.data
['abcd',',','defg']
['11111',',','22222',',','33333']

Notesandexplanation:

NotehowthegrammarisconstructedfromnormalPythoncallstofunctionand
object/classconstructors.I'veconstructedtheparserinlinebecausemyexample
issimple,butconstructingtheparserinafunctionorevenamodulemightmake
senseformorecomplexgrammars.pyparsingmakesiteasytousethesethese
differentstyles.
Use"+"tospecifyasequence.Inourexample,alineDefisafieldDef
followedby....
UseZeroOrMoretospecifyrepetition.Inourexample,alineDefisa
fieldDeffollowedbyzeroormoreoccurancesofcommaandfieldDef.
ThereisalsoOneOrMorewhenyouwanttorequireatleastoneoccurance.
Parsingcommadelimitedtexthappenssofrequentlythatpyparsingprovidesa
shortcut.Replace:
lineDef=fieldDef+ZeroOrMore(","+fieldDef)

with:
lineDef=delimitedList(fieldDef)

AndnotethatdelimitedListtakesanoptionalargumentdelimusedtospecify
thedelimiter.Thedefaultisacomma.
2.6.6.2Parsingfunctors

Thisexampleparsesexpressionsoftheformfunc(arg1,arg2,arg3):
frompyparsingimportWord,alphas,alphanums,nums,ZeroOrMore,
Literal
lparen=Literal("(")
rparen=Literal(")")
identifier=Word(alphas,alphanums+"_")
integer=Word(nums)
functor=identifier
arg=identifier|integer

Page149

APythonBook
args=arg+ZeroOrMore(","+arg)
expression=functor+lparen+args+rparen
deftest():
content=raw_input("Enteranexpression:")
parsedContent=expression.parseString(content)
printparsedContent
test()

Explanation:

UseLiteraltospecifyafixedstringthatistobematchedexactly.Inourexample,
alparenisa(.
Wordtakesanoptionalsecondargument.Withasingle(string)argument,it
matchesanycontiguouswordmadeupofcharactersinthestring.Withtwo
(string)argumentsitmatchesawordwhosefirstcharacterisinthefirststringand
whoseremainingcharactersareinthesecondstring.So,ourdefinitionof
identifiermatchesawordwhosefirstcharacterisanalphaandwhoseremaining
charactersarealphanumericsorunderscore.Asanotherexample,youcanthink
ofWord("0123456789")asanalogoustoaregexpcontainingthepattern"[09]+".
Useaverticalbarforalternation.Inourexample,anargcanbeeitheranidentifier
oraninteger.

2.6.6.3Parsingnames,phonenumbers,etc.

Thisexampleparsesexpressionshavingthefollowingform:
Inputformat:
[name][phone][city,statezip]
Last,first1112223333city,ca99999

Hereistheparser:
importsys
frompyparsingimportalphas,nums,ZeroOrMore,Word,Group,
Suppress,Combine
lastname=Word(alphas)
firstname=Word(alphas)
city=Group(Word(alphas)+ZeroOrMore(Word(alphas)))
state=Word(alphas,exact=2)
zip=Word(nums,exact=5)
name=Group(lastname+Suppress(",")+firstname)
phone=Combine(Word(nums,exact=3)+""+Word(nums,exact=3)+""
+Word(nums,exact=4))
location=Group(city+Suppress(",")+state+zip)
record=name+phone+location

Page150

APythonBook
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test3.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
infile=file(infilename,'r')
forlineininfile:
line=line.strip()
iflineandline[0]!="#":
fields=record.parseString(line)
printfields
test()

And,hereissomesampleinput:
Jabberer,Jerry1112223333Bakersfield,CA95111
Kackler,Kerry1112223334Fresno,CA95112
Louderdale,Larry1112223335LosAngeles,CA94001

Hereisoutputfromparsingtheaboveinput:
[['Jabberer','Jerry'],'1112223333',[['Bakersfield'],'CA',
'95111']]
[['Kackler','Kerry'],'1112223334',[['Fresno'],'CA','95112']]
[['Louderdale','Larry'],'1112223335',[['Los','Angeles'],'CA',
'94001']]

Comments:

Weusethelen=nargumenttotheWordconstructortoresticttheparserto
acceptingaspecificnumberofcharacters,forexampleinthezipcodeandphone
number.Wordalsoacceptsmin=n''and``max=ntoenableyoutorestrict
thelengthofawordtowithinarange.
WeuseGrouptogrouptheparsedresultsintosublists,forexampleinthe
definitionofcityandname.Groupenablesustoorganizetheparseresultsinto
simpleparsetrees.
WeuseCombinetojoinparsedresultsbackintoasinglestring.Forexample,in
thephonenumber,wecanrequiredashesandyetjointheresultsbackintoa
singlestring.
WeuseSuppresstoremoveunneededsubelementsfromparsedresults.For
example,wedonotneedthecommabetweenlastandfirstname.

2.6.6.4Amorecomplexexample

Thisexample(thankstoPaulMcGuire)parsesamorecomplexstructureandproducesa
dictionary.
Page151

APythonBook
Hereisthecode:
frompyparsingimportLiteral,Word,Group,Dict,ZeroOrMore,alphas,
nums,\
delimitedList
importpprint
testData="""
++++++++++
||A1|B1|C1|D1|A2|B2|C2|D2|
+=======+======+======+======+======+======+======+======+======+
|min|7|43|7|15|82|98|1|37|
|max|11|52|10|17|85|112|4|39|
|ave|9|47|8|16|84|106|3|38|
|sdev|1|3|1|1|1|3|1|1|
++++++++++
"""
#Definegrammarfordatatable
heading=(Literal(
"++++++++++")
+
"||A1|B1|C1|D1|A2|B2|C2|D2|"+
"+=======+======+======+======+======+======+======+======+======+").
suppress()
vert=Literal("|").suppress()
number=Word(nums)
rowData=Group(vert+Word(alphas)+vert+
delimitedList(number,"|")+
vert)
trailing=Literal(
"++++++++++").
suppress()
datatable=heading+Dict(ZeroOrMore(rowData))+trailing
defmain():
#Nowparsedataandprintresults
data=datatable.parseString(testData)
print"data:",data
print"data.asList():",
pprint.pprint(data.asList())
print"datakeys:",data.keys()
print"data['min']:",data['min']
print"data.max:",data.max
if__name__=='__main__':
main()

Whenwerunthis,itproducesthefollowing:
data:[['min','7','43','7','15','82','98','1','37'],

Page152

APythonBook
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
data.asList():[['min','7','43','7','15','82','98','1','37'],
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
datakeys:['ave','min','sdev','max']
data['min']:['7','43','7','15','82','98','1','37']
data.max:['11','52','10','17','85','112','4','39']

Notes:

NotetheuseofDicttocreateadictionary.Theprintstatementsshowhowtoget
attheitemsinthedictionary.
NotehowwecanalsogettheparseresultsasalistbyusingmethodasList.
Again,weusesuppresstoremoveunneededitemsfromtheparseresults.

2.7GUIApplications
2.7.1Introduction
ThissectionwillhelpyoutoputaGUI(graphicaluserinterface)inyourPython
program.
WewilluseaparticularGUIlibrary:PyGTK.We'vechosenthisbecauseitisreasonably
lightweightandourgoalistoembedlightweightGUIinterfacesinan(possibly)
existingapplication.
ForsimplerGUIneeds,considerEasyGUI,whichisalsodescribedbelow.
FormoreheavyweightGUIneeds(forexample,completeGUIapplications),youmay
wanttoexploreWxPython.SeetheWxPythonhomepageat:http://www.wxpython.org/

2.7.2PyGtk
InformationaboutPyGTKishere:ThePyGTKhomepagehttp://www.pygtk.org//.
2.7.2.1Asimplemessagedialogbox

InthissectionweexplainhowtopopupasimpledialogboxfromyourPython
application.
Todothis,dothefollowing:
1. ImportgtkintoyourPythonmodule.
2. Definethedialoganditsbehavior.
Page153

APythonBook
3. Createaninstanceofthedialog.
4. Runtheeventloop.
Hereisasamplethatdisplaysamessagebox:
#!/usr/bin/envpython
importsys
importgetopt
importgtk
classMessageBox(gtk.Dialog):
def__init__(self,message="",buttons=(),pixmap=None,
modal=True):
gtk.Dialog.__init__(self)
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
hbox=gtk.HBox(spacing=5)
hbox.set_border_width(5)
self.vbox.pack_start(hbox)
hbox.show()
ifpixmap:
self.realize()
pixmap=Pixmap(self,pixmap)
hbox.pack_start(pixmap,expand=False)
pixmap.show()
label=gtk.Label(message)
hbox.pack_start(label)
label.show()
fortextinbuttons:
b=gtk.Button(text)
b.set_flags(gtk.CAN_DEFAULT)
b.set_data("user_data",text)
b.connect("clicked",self.click)
self.action_area.pack_start(b)
b.show()
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=button.get_data("user_data")
self.quit()
#createamessagebox,andreturnwhichbuttonwaspressed
defmessage_box(title="MessageBox",message="",buttons=(),
pixmap=None,
modal=True):
win=MessageBox(message,buttons,pixmap=pixmap,modal=modal)
win.set_title(title)

Page154

APythonBook
win.show()
gtk.main()
returnwin.ret
deftest():
result=message_box(title='Test#1',
message='Hereisyourmessage',
buttons=('Ok','Cancel'))
print'result:',result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythonsimple_dialog.py
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=0:
usage()
test()
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Someexplanation:

First,weimportgtk
NextwedefineaclassMessageBoxthatimplementsamessagebox.Herearea
fewimportantthingstoknowaboutthatclass:
Itisasubclassofgtk.Dialog.
Itcreatesalabelandpacksitintothedialog'sclientarea.NotethataDialogis
aWindowthatcontainsavboxatthetopofandanaction_areaatthebottom
ofitsclientarea.Theintensionisforustopackmiscellaneouswidgetsinto
thevboxandtoputbuttonssuchas"Ok","Cancel",etcintotheaction_area.
Page155

APythonBook
Itcreatesonebuttonforeachbuttonlabelpassedtoitsconstructor.The
buttonsareallconnectedtotheclickmethod.
Theclickmethodsavesthevalueoftheuser_dataforthebuttonthatwas
clicked.Inourexample,thisvaluewillbeeither"Ok"or"Cancel".
And,wedefineafunction(message_box)that(1)createsaninstanceofthe
MessageBoxclass,(2)setsitstitle,(3)showsit,(4)startsitseventloopsothatit
cangetandprocesseventsfromtheuser,and(5)returnstheresulttothecaller(in
thiscase"Ok"or"Cancel").
Ourtestingfunction(test)callsfunctionmessage_boxandprintstheresult.
Thislookslikequiteabitofcode,untilyounoticethattheclassMessageBoxand
thefunctionmessage_boxcouldbeputitautilitymoduleandreused.

2.7.2.2Asimpletextinputdialogbox

And,hereisanexamplethatdisplaysantextinputdialog:
#!/usr/bin/envpython
importsys
importgetopt
importgtk
classEntryDialog(gtk.Dialog):
def__init__(self,message="",default_text='',modal=True):
gtk.Dialog.__init__(self)
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
box=gtk.VBox(spacing=10)
box.set_border_width(10)
self.vbox.pack_start(box)
box.show()
ifmessage:
label=gtk.Label(message)
box.pack_start(label)
label.show()
self.entry=gtk.Entry()
self.entry.set_text(default_text)
box.pack_start(self.entry)
self.entry.show()
self.entry.grab_focus()
button=gtk.Button("OK")
button.connect("clicked",self.click)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
button.grab_default()
button=gtk.Button("Cancel")

Page156

APythonBook
button.connect("clicked",self.quit)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
self.ret=None
defquit(self,w=None,event=None):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=self.entry.get_text()
self.quit()
definput_box(title="InputBox",message="",default_text='',
modal=True):
win=EntryDialog(message,default_text,modal=modal)
win.set_title(title)
win.show()
gtk.main()
returnwin.ret
deftest():
result=input_box(title='Test#2',
message='Enteravaluexxx:',
default_text='adefaultvalue')
ifresultisNone:
print'Canceled'
else:
print'result:"%s"'%result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythonsimple_dialog.py
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()

Page157

APythonBook
iflen(args)!=0:
usage()
test()
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Mostoftheexplanationforthemessageboxexampleisrelevanttothisexample,too.
Herearesomedifferences:

OurEntryDialogclassconstructorcreatesinstanceofgtk.Entry,setsitsdefault
value,andpacksitintotheclientarea.
Theconstructoralsoautomaticallycreatestwobuttons:"OK"and"Cancel".The
"OK"buttonisconnecttotheclickmethod,whichsavesthevalueoftheentry
field.The"Cancel"buttonisconnecttothequitmethod,whichdoesnotsavethe
value.
And,ifclassEntryDialogandfunctioninput_boxlookusableanduseful,add
themtoyourutilityguimodule.

2.7.2.3Afileselectiondialogbox

Thisexampleshowsafileselectiondialogbox:
#!/usr/bin/envpython
importsys
importgetopt
importgtk
classFileChooser(gtk.FileSelection):
def__init__(self,modal=True,multiple=True):
gtk.FileSelection.__init__(self)
self.multiple=multiple
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
self.cancel_button.connect('clicked',self.quit)
self.ok_button.connect('clicked',self.ok_cb)
ifmultiple:
self.set_select_multiple(True)
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defok_cb(self,b):
ifself.multiple:
self.ret=self.get_selections()

Page158

APythonBook
else:
self.ret=self.get_filename()
self.quit()
deffile_sel_box(title="Browse",modal=False,multiple=True):
win=FileChooser(modal=modal,multiple=multiple)
win.set_title(title)
win.show()
gtk.main()
returnwin.ret
deffile_open_box(modal=True):
returnfile_sel_box("Open",modal=modal,multiple=True)
deffile_save_box(modal=True):
returnfile_sel_box("SaveAs",modal=modal,multiple=False)
deftest():
result=file_open_box()
print'openresult:',result
result=file_save_box()
print'saveresult:',result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythonsimple_dialog.py
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=0:
usage()
test()
if__name__=='__main__':
main()
#importpdb
#pdb.run('main()')

Page159

APythonBook
Alittleguidance:
Thereisapredefinedfileselectiondialog.Wesubclassit.
Thisexampledisplaysthefileselectiondialogtwice:oncewithatitle"Open"and
oncewithatitle"SaveAs".
Notehowwecancontrolwhetherthedialogallowsmultiplefileselections.And,
ifweselectthemultipleselectionmode,thenweuseget_selectionsinsteadof
get_filenameinordertogettheselectedfilenames.
Thedialogcontainsbuttonsthatenabletheuserto(1)createanewfolder,(2)
deleteafile,and(3)renameafile.Ifyoudonotwanttheusertoperformthese
operations,thencallhide_fileop_buttons.Thiscalliscommentedoutinour
samplecode.
Notethattherearealsopredefineddialogsforfontselection(FontSelectionDialog)and
colorselection(ColorSelectionDialog)

2.7.3EasyGUI
IfyourGUIneedsareminimalist(maybeapopupdialogortwo)andyourapplicationis
imperativeratherthaneventdriven,thenyoumaywanttoconsiderEasyGUI.Asthe
namesuggests,itisextremelyeasytouse.
HowtoknowwhenyoumightbeabletouseEasyGUI:
Yourapplicationdoesnotneedtoruninawindowcontainingmenusandamenu
bar.
YourGUIneedsamounttolittlemorethandisplayingadialognowandthento
getresponsesfromtheuser.
Youdonotwanttowriteaneventdrivenapplication,thatis,oneinwhichyour
codesitsandwaitsforthetheusertoinitiateoperation,forexample,withmenu
items.
EasyGUIplusdocumentationandexamplesareavailableatEasyGUIhomepageat
SourceForgehttp://easygui.sourceforge.net/

EasyGUIprovidesfunctionsforavarietyofcommonlyneededdialogboxes,including:

Amessageboxdisplaysamessage.
Ayes/nomessageboxdisplays"Yes"and"No"buttons.
Acontinue/cancelmessageboxdisplays"Continue"and"Cancel"buttons.
Achoiceboxdisplaysaselectionlist.
Anenterboxallowsentryofalineoftext.
Anintegerboxallowsentryofaninterger.
Amultipleentryboxallowsentryintomultiplefields.
Codeandtextboxessupportthedisplayoftextinmonospacedorporportional
Page160

APythonBook
fonts.
Fileanddirectoryboxesenabletheusertoselectafileoradirectory.
SeethedocumentationattheEasyGUIWebsiteformorefeatures.
ForademonstrationofEasyGUI'scapabilities,runtheeasygui.pyasaPythonscript:
$pythoneasygui.py

2.7.3.1AsimpleEasyGUIexample

Hereisasimpleexamplethatpromptstheuserforanentry,thenshowstheresponseina
messagebox:
importeasygui
deftesteasygui():
response=easygui.enterbox(msg='Enteryourname:',title='Name
Entry')
easygui.msgbox(msg=response,title='YourResponse')
testeasygui()

2.7.3.2AnEasyGUIfileopendialogexample

Thisexamplepresentsadialogtoallowtheusertoselectafile:
importeasygui
deftest():
response=easygui.fileopenbox(msg='Selectafile')
print'filename:%s'%response
test()

2.8GuidanceonPackagesandModules
2.8.1Introduction
Pythonhasanexcellentrangeofimplementationorganizationstructures.Theserange
fromstatementsandcontrolstructures(atalowlevel)throughfunctions,methods,and
classes(atanintermediatelevel)andmodulesandpackagesatanupperlevel.
Thissectionprovidessomeguidancewiththeuseofpackages.Inparticular:

Howtoconstructandimplementthem.
Howtousethem.
Howtodistributeandinstallthem.
Page161

APythonBook

2.8.2ImplementingPackages
APythonpackageisacollectionofPythonmodulesinadiskdirectory.
Inordertobeabletoimportindividualmodulesfromadirectory,thedirectorymust
containafilenamed__init__.py.(Notethatrequirementdoesnotapplytodirectoriesthat
arelistedinPYTHONPATH.)The__init__.pyservesseveralpurposes:

Thepresenceofthefile__init__.pyinadirectorymarksthedirectoryasaPython
package,whichenablesimportingmodulesfromthedirectory.
Thefirsttimeanapplicationimportsanymodulefromthedirectory/package,the
codeinthemodule__init__isevaluated.
Ifthepackageitselfisimported(asopposedtoanindividualmodulewithinthe
directory/package),thenitisthe__init__thatisimported(andevaluated).

2.8.3UsingPackages
Onesimplewaytoenabletheusertoimportanduseapackageistoinstructtheuseto
importindividualmodulesfromthepackage.
Asecond,slightlymoreadvancedwaytoenabletheusertoimportthepackageisto
exposethosefeaturesofthepackageinthe__init__module.Supposethatmodulemod1
containsfunctionsfun1aandfun1bandsupposethatmodulemod2containsfunctions
fun2aandfun2b.Thenfile__init__.pymightcontainthefollowing:
frommod1importfun1a,fun1b
frommod2importfun2a,fun2b

Then,ifthefollowingisevaluatedintheuser'scode:
importtestpackages

Thentestpackageswillcontainfun1a,fun1b,fun2a,andfun2b.
Forexample,hereisaninteractivesessionthatdemostratesimportingthepackage:
>>>importtestpackages
>>>printdir(testpackages)
[`__builtins__',`__doc__',`__file__',`__name__',
`__path__',
`fun1a',`fun1b',`fun2a',`fun2b',`mod1',`mod2']

2.8.4DistributingandInstallingPackages
Distutils(PythonDistributionUtilities)hasspecialsupportfordistrubutingandinstalling
packages.Learnmorehere:DistributingPythonModules
http://docs.python.org/distutils/index.html.
Page162

APythonBook
Asourexample,imaginethatwehaveadirectorycontainingthefollowing:
Testpackages
Testpackages/README
Testpackages/MANIFEST.in
Testpackages/setup.py
Testpackages/testpackages/__init__.py
Testpackages/testpackages/mod1.py
Testpackages/testpackages/mod2.py

NoticethesubdirectoryTestpackages/testpackagescontainingthefile__init__.py.
ThisisthePythonpackagethatwewillinstall.
We'lldescribehowtoconfiguretheabovefilessothattheycanbepackagedasasingle
distributionfileandsothatthePythonpackagetheycontaincanbeinstalledasapackage
byDistutils.
TheMANIFEST.infileliststhefilesthatwewantincludedinourdistribution.Hereis
thecontentsofourMANIFEST.infile:
includeREADMEMANIFESTMANIFEST.in
includesetup.py
includetestpackages/*.py

Thesetup.pyfiledescribestoDistutils(1)howtopackagethedistributionfileand(2)
howtoinstallthedistribution.Hereisthecontentsofoursamplesetup.py:
#!/usr/bin/envpython
fromdistutils.coreimportsetup#[1]
long_description='TestsforinstallinganddistributingPython
packages'
setup(name='testpackages',#[2]
version='1.0a',
description='TestsforPythonpackages',
maintainer='DaveKuhlman',
maintainer_email='dkuhlman@rexx.com',
url='http://www.rexx.com/~dkuhlman',
long_description=long_description,
packages=['testpackages']#[3]
)

Explanation:
1. WeimportthenecessarycomponentfromDistutils.
2. Wedescribethepackageanditsdeveloper/maintainer.
3. Wespecifythedirectorythatistobeinstalledasapackage.Whentheuser
installsourdistribution,thisdirectoryandallthemodulesinitwillbeinstalledas
apackage.
Page163

APythonBook
Now,tocreateadistributionfile,werunthefollowing:
pythonsetup.pysdistformats=gztar

whichwillcreateafiletestpackages1.0a.tar.gzunderthedirectorydist.
Then,youcangivethisdistributionfiletoapotentialuser,whocaninstallitbydoingthe
following:
$tarxvzftestpackages1.0a.tar.gz
$cdtestpackages1.0a
$pythonsetup.pybuild
$pythonsetup.pyinstall#asroot

2.9EndMatter
2.9.1AcknowledgementsandThanks

ThankstotheimplementorsofPythonforproducinganexceptionallyusableand
enjoyableprogramminglanguage.
ThankstoDaveBeazleyandothersforSWIGandPLY.
ThankstoGregEwingforPyrexandPlex.
ThankstoJamesHenstridgeforPyGTK.

2.9.2SeeAlso

ThemainPythonWebSitehttp://www.python.orgformoreinformationon
Python.
PythonDocumentationhttp://www.python.org/doc/forlotsofdocumentation
onPython
Dave'sWebSitehttp://http://www.davekuhlman.orgformoresoftwareand
informationonusingPythonforXMLandtheWeb.
TheSWIGhomepagehttp://www.swig.orgformoreinformationonSWIG
(SimplifiedWrapperandInterfaceGenerator).
ThePyrexhomepagehttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
formoreinformationonPyrex.
PLY(PythonLexYacc)homepagehttp://www.dabeaz.com/ply/formore
informationonPLY.
ThePlexhomepagehttp://www.cosc.canterbury.ac.nz/greg.ewing/python/Plex/
formoreinformationonPlex.
DistributingPythonModuleshttp://docs.python.org/distutils/index.htmlfor
informationonthePythonDistributionUtilities(Distutils).

Page164

APythonBook

3Part3PythonWorkbook
3.1Introduction
ThisdocumenttakesaworkbookandexercisewithsolutionsapproachtoPython
training.Itishopedthatthosewhofeelaneedforlessexplanationandmorepractical
exerciseswillfindthisuseful.
Afewnotesabouttheexercises:
I'vetriedtoincludesolutionsformostoftheexercises.Hopefully,youwillbe
abletocopyandpastethesesolutionsintoyourtexteditor,thenextendand
experimentwiththem.
IusetwointeractivePythoninterpreters(althoughtheyarethesamePython
underneath).Whenyouseethisprompt>>>,it'sthestandardPythoninterpreter.
And,whenyouseethispromptIn[1]:,it'sIPython
http://ipython.scipy.org/moin/.
ThelatestversionofthisdocumentisatmyWebsite(URLabove).

Ifyouhavecommentsorsuggestions,pleasesendthemmyway.

3.2LexicalStructures
3.2.1Variablesandnames
Anameisanycombinationofletters,digits,andtheunderscore,butthefirstcharacter
mustbealetteroranunderscore.Namesmaybeofanylength.
Caseissignificant.
Exercises:
1. Whichofthefollowingarevalidnames?
1. total
2. total_of_all_vegetables
3. bigtitle1
4. _inner_func
5. 1bigtitle
6. bigtitle1
2. Whichorthefollowingpairsarethesamename:
1. the_last_itemandthe_last_item
Page165

APythonBook
2. the_last_itemandThe_Last_Item
3. itemianditemj
4. item1anditeml
Solutions:
1. Items1,2,4,and6arevalid.Item3isnotasinglename,butisthreeitems
separatedbytheminusoperator.Item5isnotvalidbecauseitbeginswithadigit.
2. Pythonnamesarecasesensitive,whichmeans:
1. the_last_itemandthe_last_itemarethesame.
2. the_last_itemandThe_Last_ItemaredifferentThesecondname
hasanuppercasecharacters.
3. itemianditemjaredifferent.
4. item1anditemlaredifferentThisonemaybedifficulttosee,
dependingonthefontyouareviewing.Onenameendswiththedigitone;the
otherendswiththealphacharacter"el".Andthisexampleprovidesagood
reasontouse"1"and"l"judiciouslyinnames.
ThefollowingarekeywordsinPythonandshouldnotbeusedasvariablenames:
anddelfromnotwhile
aselifglobalorwith
assertelseifpassyield
breakexceptimportprint
classexecinraise
continuefinallyisreturn
defforlambdatry

Exercises:
1. WhichofthefollowingarevalidnamesinPython?
1. _global
2. global
3. file
Solutions:
1. Donotusekeywordsforvariablenames:
1. Valid
2. Notavalidname."global"isakeyword.
3. Valid,however,"file"isthenameofabuiltintype,asyouwilllearnlater,so
youareadvisednottoredefineit.Hereareafewofthenamesofbuiltin
types:"file","int","str","float","list","dict",etc.SeeBuiltinTypes
http://docs.python.org/lib/types.htmlformorebuiltintypes..
ThefollowingareoperatorsinPythonandwillseparatenames:
+***///%
<<>>&|^~

Page166

APythonBook
<><=>===!=<>
andorisnotin
Also:()[].(dot)

But,notethatthePythonstyleguidesuggeststhatyouplaceblanksaroundbinary
operators.Oneexceptiontothisruleisfunctionargumentsandparametersforfunctions:
itissuggestedthatyounotputblanksaroundtheequalsign(=)usedtospecifykeyword
argumentsanddefaultparameters.
Exercises:
1. Whichofthefollowingaresinglenamesandwhicharenamesseparatedby
operators?
1. fruit_collection
2. fruitcollection
Solutions:
1. Donotuseadash,orotheroperator,inthemiddleofaname:
1. fruit_collectionisasinglename
2. fruitcollectionistwonamesseparatedbyadash.

3.2.2Linestructure
InPython,normallywewriteonestatementperline.Infact,Pythonassumesthis.
Therefore:
Statementseparatorsarenotnormallyneeded.
But,ifwewantmorethanonestatementonaline,weuseastatementseparator,
specificallyasemicolon.
And,ifwewanttoextendastatementtoasecondorthirdlineandsoon,we
sometimesneedtodoabitextra.
ExtendingaPythonstatementtoasubsequentlineFollowthesetworules:

1. Ifthereisanopencontext,nothingspecialneedbedonetoextendastatement
acrossmultiplelines.Anopencontextisanopenparenthesis,anopensquare
bracket,oranopencurlybracket.
2. Wecanalwaysextendastatementonafollowinglinebyplacingabackslashas
thelastcharacteroftheline.
Exercises:
1. Extendthefollowingstatementtoasecondlineusingparentheses:
total_count=tree_count+vegetable_count+
fruit_count

2. Extendthefollowingstatementtoasecondlineusingthebackslashline
Page167

APythonBook
continuationcharacter:
total_count=tree_count+vegetable_count+
fruit_count

Solutions:
1. ParenthesescreateanopencontextthattellsPythonthatastatementextendsto
thenextline:
total_count=(tree_count+
vegetable_count+fruit_count)

2. AbackslashasthelastcharacteronlinetellsPythonthatthecurrentstatement
extendstothenextline:
total_count=tree_count+\
vegetable_count+fruit_count

Forextendingalineonasubsequentline,whichisbetter,parenthesesorabackslash?
Hereisaquote:
"ThepreferredwayofwrappinglonglinesisbyusingPython'simplied
linecontinuationinsideparentheses,bracketsandbraces.Ifnecessary,
youcanaddanextrapairofparenthesesaroundanexpression,but
sometimesusingabackslashlooksbetter."
PEP8:StyleGuideforPythonCode
http://www.python.org/dev/peps/pep0008/

3.2.3Indentationandprogramstructure
Pythonusesindentationtoindicateprogramstructure.Thatistosay,inordertonesta
blockofcodeinsideacompoundstatement,youindentthatnestedcode.Thisisdifferent
frommanyprogramminglanguageswhichusesomesortofbeginandendmarkers,for
examplecurlybrackets.
ThestandardcodingpracticeforPythonistousefourspacesperindentationlevelandto
notusehardtabs.(SeetheStyleGuideforPythonCode.)Becauseofthis,youwillwant
touseatexteditorthatyoucanconfiguresothatitwillusefourspacesforindentation.
SeehereforalistofPythonfriendlytexteditors:PythonEditors.
Exercises:
1. Giventhefollowing,nesttheprintstatementinsidetheifstatement:
ifx>0:
printx

2. Nestthesetwolines:
Page168

APythonBook
z=x+y
printz

insidethefollowingfunctiondefinitionstatement:
defshow_sum(x,y):

Solutions:
1. Indentationindicatesthatonestatementisnestedinsideanotherstatement:
ifx>0:
printx

2. Indentationindicatesthatablockofstatementsisnestedinsideanotherstatement:
defshow_sum(x,y):
z=x+y
printz

3.3ExecutionModel
Hereareafewrules:
1. PythonevaluatesPythoncodefromthetopofamoduledowntothebottomofa
module.
2. Bindingstatementsattoplevelcreatenames(andbindvaluestothosenames)as
Pythonevaluatescode.Furthermore,anameisnotcreateduntilitisboundtoa
value/object.
3. Anestedreferencetoaname(forexample,insideafunctiondefinitionorinthe
nestedblockofanifstatement)isnotuseduntilthatnestedcodeisevaluated.
Exercises:
1. Willthefollowingcodeproduceanerror?
show_version()
defshow_version():
print'Version1.0a'

2. Willthefollowingcodeproduceanerror?
deftest():
show_version()
defshow_version():
print'Version1.0a'
test()

3. Willthefollowingcodeproduceanerror?Assumethatshow_configisnot
defined:
x=3

Page169

APythonBook
ifx>5:
show_config()

Solutions:
1. Answer:Yes,itgeneratesanerror.Thenameshow_versionwouldnotbe
createdandboundtoavalueuntilthedeffunctiondefinitionstatementbindsa
functionobjecttoit.Thatisdoneaftertheattempttouse(call)thatobject.
2. Answer:No.Thefunctiontest()doescallthefunctionshow_version(),
butsincetest()isnotcalleduntilaftershow_version()isdefined,thatis
OK.
3. Answer:No.It'sbadcode,butinthiscasewillnotgenerateanerror.Sincexis
lessthan5,thebodyoftheifstatementisnotevaluated.
N.B.Thisexampleshowswhyitisimportantduringtestingthateverylineof
codeinyourPythonprogrambeevaluated.HereisgoodPythonicadvice:"Ifit's
nottested,it'sbroken."

3.4BuiltinDataTypes
Eachofthesubsectionsinthissectiononbuiltindatatypeswillhaveasimilarstructure:
1. Abriefdescriptionofthedatatypeanditsuses.
2. RepresentationandconstructionHowtorepresentaninstanceofthedatatype.
Howtocodealiteralrepresentationthatcreatesanddefinesaninstance.Howto
createaninstanceofthebuiltintype.
3. Operatorsthatareapplicabletothedatatype.
4. Methodsimplementedandsupportedbythedatatype.

3.4.1Numbers
Thenumbersyouwillusemostcommonlyarelikelytobeintegersandfloats.Python
alsohaslongintegersandcomplexnumbers.
Afewfactsaboutnumbers(inPython):

Pythonwillconverttousingalongintegerautomaticallywhenneeded.Youdo
notneedtoworryaboutexceedingthesizeofa(standard)integer.
ThesizeofthelargestintegerinyourversionofPythonisinsys.maxint.To
learnwhatitis,do:
>>>importsys
>>>printsys.maxint
9223372036854775807

Theaboveshowthemaximumsizeofanintegerona64bitversionofPython.
Youcanconvertfromintegertofloatbyusingthefloatconstructor.Example:
Page170

APythonBook
>>>x=25
>>>y=float(x)
>>>printy
25.0

Pythondoes"mixedarithmetic".Youcanadd,multiply,anddivideintegersand
floats.Whenyoudo,Python"promotes"theresulttoafloat.

3.4.1.1Literalrepresentationsofnumbers

Anintegerisconstructedwithaseriesofdigitsortheintegerconstructor(int(x)).Be
awarethatasequenceofdigitsbeginningwithzerorepresentsanoctalvalue.Examples:
>>>x1=1234
>>>x2=int('1234')
>>>x3=25
>>>x1
1234
>>>x2
1234
>>>x3
25

Afloatisconstructedeitherwithdigitsandadot(example,12.345)orwith
engineering/scientificnotationorwiththefloatconstructor(float(x)).Examples:
>>>x1=2.0e3
>>>x1=1.234
>>>x2=1.234
>>>x3=float('1.234')
>>>x4=2.0e3
>>>x5=2.0e3
>>>printx1,x2,x3,x4,x5
1.2341.2341.2342000.00.002

Exercises:
Constructthesenumericvalues:
1.
2.
3.
4.
5.
6.
7.

Integerzero
Floatingpointzero
Integeronehundredandone
Floatingpointonethousand
Floatingpointonethousandusingscientificnotation
Createapositiveinteger,anegativeinteger,andzero.Assignthemtovariables
Writeseveralarithmeticexpressions.Bindthevaluestovariables.Useavariety
ofoperators,e.g.+,,/,*,etc.Useparenthesestocontroloperatorscope.
8. Createseveralfloatsandassignthemtovariables.
9. Writeseveralarithmeticexpressionscontainingyourfloatvariables.
Page171

APythonBook
10. Writeseveralexpressionsusingmixedarithmetic(integersandfloats).Obtaina
floatasaresultofdivisionofoneintegerbyanother;dosobyexplicitly
convertingoneintegertoafloat.
Solutions:
1.
2.
3.
4.
5.
6.

0
0.0,0.,or.0
101
1000.0
1e3or1.0e3
Asigningintegervaluestovariables:
In[7]:value1=23
In[8]:value2=14
In[9]:value3=0
In[10]:value1
Out[10]:23
In[11]:value2
Out[11]:14
In[12]:value3
Out[12]:0

7. Assigningexpressionvaluestovariables:
value1=4*(3+5)
value2=(value1/3.0)2

8. Assigningfloatstovariables:
value1=0.01
value2=3.0
value3=3e4

9. Assigningexpressionscontainingvarialbes:
value4=value1*(value2value3)
value4=value1+value2+value3value4

10. Mixedarithmetic:
x=5
y=8
z=float(x)/y

Youcanalsoconstructintegersandfloatsusingtheclass.Callingaclass(using
parenthesesafteraclassname,forexample)producesaninstanceoftheclass.
Exercises:
1. Constructanintegerfromthestring"123".
2. Constructafloatfromtheinteger123.
3. Constructanintegerfromthefloat12.345.
Solutions:
Page172

APythonBook
1. Usetheintdatatypetoconstructanintegerinstancefromastring:
int("123")

2. Usethefloatdatatypetoconstructafloatinstancefromaninteger:
float(123)

3. Usetheintdatatypetoconstructanintegerinstancefromafloat:
int(12.345)#>12

Noticethattheresultistruncatedtotheintegerpart.
3.4.1.2Operatorsfornumbers

Youcanusemostofthefamiliaroperatorswithnumbers,forexample:
+***///%
<<>>&|^~
<><=>===!=<>

Lookhereforanexplanationoftheseoperatorswhenappliedtonumbers:Numeric
Typesint,float,long,complexhttp://docs.python.org/lib/typesnumeric.html.
Someoperatorstakeprecedenceoverothers.ThetableintheWebpagejustreferenced
abovealsoshowsthatorderofpriority.
Hereisabitofthattable:
Allnumerictypes(exceptcomplex)supportthefollowingoperations,
sortedbyascendingpriority(operationsinthesameboxhavethe
same
priority;allnumericoperationshaveahigherprioritythan
comparison
operations):
OperationResult

x+ysumofxandy
xydifferenceofxandy
x*yproductofxandy
x/yquotientofxandy
x//y(floored)quotientofxandy
x%yremainderofx/y
xxnegated
+xxunchanged
abs(x)absolutevalueormagnitudeofx
int(x)xconvertedtointeger
long(x)xconvertedtolonginteger
float(x)xconvertedtofloatingpoint
complex(re,im)acomplexnumberwithrealpartre,imaginarypart
im.imdefaultstozero.
c.conjugate()conjugateofthecomplexnumberc

Page173

APythonBook
divmod(x,y)thepair(x//y,x%y)
pow(x,y)xtothepowery
x**yxtothepowery

Noticealsothatthesameoperatormayperformadifferentfunctiondependingonthe
datatypeofthevaluetowhichitisapplied.
Exercises:
1. Addthenumbers3,4,and5.
2. Add2totheresultofmultiplying3by4.
3. Add2plus3andmultiplytheresultby4.
Solutions:
1. Arithmeticexpressionsarefollowstandardinfixalgebraicsyntax:
3+4+5

2. Useanotherinfixexpression:
2+3*4

Or:
2+(3*4)

But,inthiscasetheparenthesesarenotnecessarybecausethe*operatorbinds
moretightlythanthe+operator.
3. Useparenthesestocontrolorderofevaluation:
(2+3)*4

Notethatthe*operatorhasprecedenceover(bindstighterthan)the+operator,
sotheparenthesesareneeded.
Pythondoesmixedarithemetic.Whenyouapplyanoperationtoanintegerandafloat,it
promotestheresulttothe"higher"datatype,afloat.
Ifyouneedtoperformanoperationonseveralintegers,butwantuseafloatingpoint
operation,firstconvertoneoftheintegerstoafloatusingfloat(x),whicheffectively
createsaninstanceofclassfloat.
TrythefollowingatyourPythoninteractiveprompt:
1. 1.0+2
2. 2/3Noticethattheresultistruncated.
3. float(2)/3Noticethattheresultisnottruncated.
Exercises:
1. Giventhefollowingassignments:
x=20
y=50

Page174

APythonBook
Dividexbyygivingafloatresult.
Solutions:
1. Promoteoneoftheintegerstofloatbeforeperformingthedivision:
z=float(x)/y

3.4.1.3Methodsonnumbers

Mostofthemethodsimplementedbythedatatypes(classes)intandfloatarespecial
methodsthatarecalledthroughtheuseofoperators.Specialmethodsoftenhavenames
thatbeginandendwithadoubleunderscore.Toseealistofthespecialnamesandabit
ofanindicationofwheneachiscalled,doanyofthefollowingatthePythoninteractive
prompt:
>>>help(int)
>>>help(32)
>>>help(float)
>>>help(1.23)
>>>dir(1)
>>>dir(1.2)

3.4.2Lists
Listsareacontainerdatatypethatactsasadynamicarray.Thatistosay,alistisa
sequencethatcanbeindexedintoandthatcangrowandshrink.
Atupleisanindexablecontainer,likealist,exceptthatatupleisimmutable.
Afewcharacteristicsoflistsandtuples:

Alisthasa(current)lengthGetthelengthofalistwithlen(mylist).
AlisthasanorderTheitemsinalistareordered,andyoucanthinkofthat
orderasgoingfromlefttoright.
AlistisheterogeneousYoucaninsertdifferenttypesofobjectsintothesame
list.
Listsaremutable,buttuplesarenot.Thus,thefollowingaretrueoflists,butnot
oftuples:
Youcanextendedoraddtoalist.
Youcanshrinkalistbydeletingitemsfromit.
Youcaninsertitemsintothemiddleofalistoratthebeginningofalist.You
canadditemstotheendofalist.
Youcanchangewhichitemisatagivenpositioninalist.

Page175

APythonBook
3.4.2.1Literalrepresentationoflists

Theliteralrepresentationofalistissquarebracketscontainingzeroormoreitems
separatedbycommas.
Examples:
1. TrytheseatthePythoninteractiveprompt:
>>>[11,22,33]
>>>['aa','bb','cc',]
>>>[100,'apple',200,'banana',]#Thelastcomma
is
>>>optional.

2. Alistcancontainlists.Infactalistcancontainanykindofobject:
>>>[1,[2,3],4,[5,6,7,],8]

3. Listsareheterogenous,thatis,differentkindsofobjectscanbeinthesamelist.
Hereisalistthatcontainsanumber,astring,andanotherlist:
>>>[123,'abc',[456,789]]

Exercises:
1. Create(define)thefollowingtuplesandlistsusingaliteral:
1. Atupleofintegers
2. Atupleofstrings
3. Alistofintegers
4. Alistofstrings
5. Alistoftuplesortupleoflists
6. Alistofintegersandstringsandtuples
7. Atuplecontainingexactlyoneitem
8. Anemptytuple
2. Doeachofthefollowing:
1. Printthelengthofalist.
2. PrinteachiteminthelistIterateovertheitemsinoneofyourlists.Print
eachitem.
3. Appendanitemtoalist.
4. Insertanitematthebeginningofalist.Insertaniteminthemiddleofalist.
5. Addtwoliststogether.Dosobyusingboththeextendmethodandtheplus
(+)operator.Whatisthedifferencebetweenextendingalistandaddingtwo
lists?
6. Retrievethe2nditemfromoneofyourtuplesorlists.
7. Retrievethe2nd,3rd,and4thitems(aslice)fromoneofyourtuplesorlists.
8. Retrievethelast(rightmost)iteminoneofyourlists.
9. Replaceaniteminalistwithanewitem.
Page176

APythonBook
10. Poponeitemofftheendofyourlist.
11. Deleteanitemfromalist.
12. Dothefollowinglistmanipulations:
1. Writeafunctionthattakestwoarguments,alistandanitem,andthat
appendstheitemtothelist.
2. Createanemptylist,
3. Callyourfunctionseveraltimestoappenditemstothelist.
4. Then,printouteachiteminthelist.
Solutions:
1. WecandefinelistliteralsatthePythonorIPythoninteractiveprompt:
1. Createatupleusingcommas,optionallywithparentheses:
In[1]:a1=(11,22,33,)
In[2]:a1
Out[2]:(11,22,33)

2. Quotedcharactersseparatedbycommascreateatupleofstrings:
In[3]:a2=('aaa','bbb','ccc')
In[4]:a2
Out[4]:('aaa','bbb','ccc')

3. Itemsseparatedbycommasinsidesquarebracketscreatealist:
In[26]:a3=[100,200,300,]
In[27]:a3
Out[27]:[100,200,300]

4. Stringsseparatedbycommasinsidesquarebracketscreatealistofstrings:
In[5]:a3=['basil','parsley','coriander']
In[6]:a3
Out[6]:['basil','parsley','coriander']
In[7]:

5. Atupleoralistcancontaintuplesandlists:
In[8]:a5=[(11,22),(33,44),(55,)]
In[9]:a5
Out[9]:[(11,22),(33,44),(55,)]

6. Alistortuplecancontainitemsofdifferenttypes:
In[10]:a6=[101,102,'abc',"def",(201,202),
('ghi','jkl')]
In[11]:a6
Out[11]:[101,102,'abc','def',(201,202),
('ghi','jkl')]

7. Inordertocreateatuplecontainingexactlyoneitem,wemustuseacomma:
In[13]:a7=(6,)
In[14]:a7

Page177

APythonBook
Out[14]:(6,)

8. Inordertocreateanemptytuple,usethetupleclass/typetocreateaninstance
ofaemptytuple:
In[21]:a=tuple()
In[22]:a
Out[22]:()
In[23]:type(a)
Out[23]:<type'tuple'>

3.4.2.2Operatorsonlists

Thereareseveraloperatorsthatareapplicabletolists.Hereishowtofindoutabout
them:
Dodir([])ordir(any_list_instance).Someoftheitemswith
specialnames(leadingandtrainingdoubleunderscores)willgiveyoucluesabout
operatorsimplementedbythelisttype.
Dohelp([])orhelp(list)atthePythoninteractiveprompt.
Dohelp(any_list_instance.some_method),wheresome_method
isoneoftheitemslistedusingdir(any_list_instance).
SeeSequenceTypesstr,unicode,list,tuple,buffer,xrange
http://docs.python.org/lib/typesseq.html
Exercises:

1. Concatenate(add)twoliststogether.
2. Createasinglelistthatcontainstheitemsinaninitiallistrepeated3times.
3. Comparetwolists.
Solutions:
1. Theplusoperator,appliedtotwolistsproducesanewlistthatisaconcatenation
oftwolists:
>>>[11,22]+['aa','bb']

2. Multiplyingalistbyanintegerncreatesanewlistthatrepeatstheoriginallistn
times:
>>>[11,'abc',4.5]*3

3. Thecomparisonoperatorscanbeusedtocomparelists:
>>>[11,22]==[11,22]
>>>[11,22]<[11,33]

3.4.2.3Methodsonlists

Again,usedir()andhelp()tolearnaboutthemethodssupportedbylists.
Page178

APythonBook
Examples:
1. Createtwo(small)lists.Extendthefirstlistwiththeitemsinthesecond.
2. Appendseveralindividualitemstotheendofalist.
3. (a)Insertaitematthebeginningofalist.(b)Insertanitemsomewhereinthe
middleofalist.
4. Popanitemofftheendofalist.
Solutions:
1. Theextendmethodaddselementsfromanotherlist,orotheriterable:
>>>a=[11,22,33,44,]
>>>b=[55,66]
>>>a.extend(b)
>>>a
[11,22,33,44,55,66]

2. Usetheappendmethodonalisttoadd/appendanitemtotheendofalist:
>>>a=['aa',11]
>>>a.append('bb')
>>>a.append(22)
>>>a
['aa',11,'bb',22]

3. Theinsertmethodonalistenablesustoinsertitemsatagivenpositionina
list:
>>>a=[11,22,33,44,]
>>>a.insert(0,'aa')
>>>a
['aa',11,22,33,44]
>>>a.insert(2,'bb')
>>>a
['aa',11,'bb',22,33,44]

But,notethatweuseappendtoadditemsattheendofalist.
4. Thepopmethodonalistreturnsthe"rightmost"itemfromalistandremoves
thatitemfromthelist:
>>>a=[11,22,33,44,]
>>>
>>>b=a.pop()
>>>a
[11,22,33]
>>>b
44
>>>b=a.pop()
>>>a
[11,22]
>>>b
33

Page179

APythonBook
Notethattheappendandpopmethodstakentogethercanbeusedtoimplement
astack,thatisaLIFO(lastinfirstout)datastructure.
3.4.2.4Listcomprehensions

Alistcomprehensionisaconvenientwaytoproducealistfromaniterable(asequence
orotherobjectthatcanbeiteratedover).
Initssimplestform,alistcomprehensionresemblestheheaderlineofaforstatement
insidesquarebrackets.However,inalistcomprehension,theforstatementheaderis
prefixedwithanexpressionandsurroundedbysquarebrackets.Hereisatemplate:
[expr(x)forxiniterable]

where:
expr(x)isanexpression,usually,butnotalways,containingx.
iterableissomeiterable.Aniterablemaybeasequence(forexample,alist,a
string,atuple)oranunorderedcollectionoraniterator(somethingoverwhichwe
caniterateorapplyaforstatementto).
Hereisanexample:

>>>a=[11,22,33,44]
>>>b=[x*2forxina]
>>>b
[22,44,66,88]

Exercises:
1. Giventhefollowinglistofstrings:
names=['alice','bertrand','charlene']

producethefollowinglists:(1)alistofalluppercasenames;(2)alistof
capitalized(firstletteruppercase);
2. Giventhefollowingfunctionwhichcalculatesthefactorialofanumber:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)

andthefollowinglistofnumbers:
numbers=[2,3,4,5]

createalistofthefactorialsofeachofthenumbersinthelist.
Solutions:
1. Forourexpressioninalistcomprehension,usetheupperandcapitalize
Page180

APythonBook
methods:
>>>names=['alice','bertrand','charlene']
>>>[name.upper()fornameinnames]
['ALICE','BERTRAND','CHARLENE']
>>>[name.capitalize()fornameinnames]
['Alice','Bertrand','Charlene']

2. Theexpressioninourlistcomprehensioncallsthefactorialfunction:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)
deftest():
numbers=[2,3,4,5]
factorials=[t(n)forninnumbers]
print'factorials:',factorials
if__name__=='__main__':
test()

Alistcomprehensioncanalsocontainanifclause.Hereisatemplate:
[expr(x)forxiniterableifpred(x)]

where:
pred(x)isanexpressionthatevaluatestoatrue/falsevalue.Valuesthatcount
asfalsearenumericzero,False,None,andanyemptycollection.Allother
valuescountastrue.
Onlyvaluesforwhichtheifclauseevaluatestotrueareincludedincreatingtheresulting
list.

Examples:
>>>a=[11,22,33,44]
>>>b=[x*3forxinaifx%2==0]
>>>b
[66,132]

Exercises:
1. Giventwolists,generatealistofallthestringsinthefirstlistthatarenotinthe
secondlist.Herearetwosamplelists:
names1=['alice','bertrand','charlene','daniel']
names2=['bertrand','charlene']

Solutions:
1. Theifclauseofourlistcomprehensionchecksforcontainmentinthelistnames2:
Page181

APythonBook
deftest():
names1=['alice','bertrand','charlene',
'daniel']
names2=['bertrand','charlene']
names3=[namefornameinnames1ifnamenotin
names2]
print'names3:',names3
if__name__=='__main__':
test()

Whenrun,thisscriptprintsoutthefollowing:
names3:['alice','daniel']

3.4.3Strings
Astringisanorderedsequenceofcharacters.Hereareafewcharacteristicsofstrings:
Astringhasalength.Getthelengthwiththelen()builtinfunction.
Astringisindexable.Getasinglecharacteratapositioninastringwiththe
squarebracketoperator,forexamplemystring[5].
Youcanretrieveaslice(substring)ofastringwithasliceoperation,forexample
mystring[5:8].
Createstringswithsinglequotesordoublequotes.Youcanputsinglequotesinside
doublequotesandyoucanputdoublequotesinsidesinglequotes.Youcanalsoescape
characterswithabackslash.

Exercises:
1. Createastringcontainingasinglequote.
2. Createastringcontainingadoublequote.
3. Createastringcontainingbothasinglequoteadoublequote.
Solutions:
1. Createastringwithdoublequotestoincludesinglequotesinsidethestring:
>>>str1="thatisjerry'sball"

2. Createastringenclosedwithsinglequotesinordertoincludedoublequotes
insidethestring:
>>>str1='say"goodbye",bullwinkle'

3. Takeyourchoice.Escapeeitherthesinglequotesorthedoublequoteswitha
backslash:
>>>str1='say"hello"tojerry\'smom'
>>>str2="say\"hello\"tojerry'smom"
>>>str1
'say"hello"tojerry\'smom'

Page182

APythonBook
>>>str2
'say"hello"tojerry\'smom'

Triplequotesenableyoutocreateastringthatspansmultiplelines.Usethreesingle
quotesorthreedoublequotestocreateasinglequotedstring.
Examples:
1. Createatriplequotedstringthatcontainssingleanddoublequotes.
Solutions:
1. Usetriplesinglequotesortripledoublequotestocreatemultilinestrings:
String1='''Thisstringextends
acrossseverallines.And,soithas
endoflinecharactersinit.
'''
String2="""
Thisstringbeginsandendswithanendofline
character.Itcanhaveboth'single'
quotesand"double"quotesinit.
"""
deftest():
printString1
printString2
if__name__=='__main__':
test()

3.4.3.1Characters

Pythondoesnothaveadistinctcharactertype.InPython,acharacterisastringoflength
1.Youcanusetheord()andchr()builtinfunctionstoconvertfromcharacterto
integerandback.
Exercises:
1. Createacharacter"a".
2. Createacharacter,thenobtainitsintegerrepresentation.
Solutions:
1. Thecharacter"a"isaplainstringoflength1:
>>>x='a'

2. Theintegerequivalentoftheletter"A":
>>>x="A"
>>>ord(x)
65

Page183

APythonBook
3.4.3.2Operatorsonstrings

Youcanconcatenatestringswiththe"+"operator.
Youcancreatemultipleconcatenatedcopiesofastringwiththe"*"operator.
And,augmentedassignment(+=and*=)alsowork.
Examples:
>>>'cat'+'and'+'dog'
'catanddog'
>>>'#'*40
'########################################'
>>>
>>>s1='flower'
>>>s1+='s'
>>>s1
'flowers'

Exercises:
1. Giventhesestrings:
>>>s1='abcd'
>>>s2='efgh'

createanewstringcomposedofthefirststringfollowedby(concatenatedwith)
thesecond.
2. Createasinglestringcontaining5copiesofthestring'abc'.
3. Usethemultiplicationoperatortocreatea"line"of50dashes.
4. Herearethecomponentsofapathtoafileonthefilesystem:"home",
"myusername","Workdir","notes.txt".Concatenatethesetogetherseparating
themwiththepathseparatortoformacompletepathtothatfile.(Notethatifyou
usethebackslashtoseparatecomponentsofthepath,youwillneedtousea
doublebackslash,becausethebackslashistheescapecharacterinstrings.
Solutions:
1. Theplus(+)operatorappliedtoastringcanbeusedtoconcatenatestrings:
>>>s3=s1+s2
>>>s3
'abcdefgh'

2. Themultiplicationoperator(*)appliedtoastringcreatesanewstringthat
concatenatesastringwithitselfsomenumberoftimes:
>>>s1='abc'*5
>>>s1
'abcabcabcabcabc'

3. Themultiplicationoperator(*)appliedtoastringcanbeusedtocreatea
Page184

APythonBook
"horizontaldividerline":
>>>s1=''*50
>>>prints1

4. Thesepmemberoftheosmodulegivesusaplatformindependentwayto
constructpaths:
>>>importos
>>>
>>>a=["home","myusername","Workdir","notes.txt"]
>>>path=a[0]+os.sep+a[1]+os.sep+a[2]+
os.sep+a[3]
>>>path
'home/myusername/Workdir/notes.txt'

And,amoreconcisesolution:
>>>importos
>>>a=["home","myusername","Workdir","notes.txt"]
>>>os.sep.join(a)
'home/myusername/Workdir/notes.txt'

Notes:
Notethatimportingtheosmoduleandthenusingos.sepfromthatmodule
givesusaplatformindependentsolution.
Ifyoudodecidetocodethepathseparatorcharacterexplicitlyandifyouare
onMSWindowswherethepathseparatoristhebackslash,thenyouwillneed
touseadoublebackslash,becausethatcharacteristheescapecharacter.
3.4.3.3Methodsonstrings

Stringsupportavarietyofoperations.Youcanobtainalistofthesemethodsbyusingthe
dir()builtinfunctiononanystring:
>>>dir("")
['__add__','__class__','__contains__','__delattr__','__doc__',
'__eq__','__ge__','__getattribute__','__getitem__',
'__getnewargs__','__getslice__','__gt__','__hash__','__init__',
'__le__','__len__','__lt__','__mod__','__mul__','__ne__',
'__new__','__reduce__','__reduce_ex__','__repr__','__rmod__',
'__rmul__','__setattr__','__str__','capitalize','center',
'count','decode','encode','endswith','expandtabs','find',
'index','isalnum','isalpha','isdigit','islower','isspace',
'istitle','isupper','join','ljust','lower','lstrip',
'partition','replace','rfind','rindex','rjust','rpartition',
'rsplit','rstrip','split','splitlines','startswith','strip',
'swapcase','title','translate','upper','zfill']

And,youcangethelponanyspecificmethodbyusingthehelp()builtinfunction.
Hereisanexample:
Page185

APythonBook
>>>help("".strip)
Helponbuiltinfunctionstrip:
strip(...)
S.strip([chars])>stringorunicode
ReturnacopyofthestringSwithleadingandtrailing
whitespaceremoved.
IfcharsisgivenandnotNone,removecharactersinchars
instead.
Ifcharsisunicode,Swillbeconvertedtounicodebefore
stripping

Exercises:
1. Stripallthewhitespacecharactersofftherightendofastring.
2. Centerashortstringwithinalongerstring,thatis,padashortstringwithblank
charactersonbothrightandlefttocenterit.
3. Convertastringtoalluppercase.
4. Splitastringintoalistof"words".
5. (a)Jointhestringsinalistofstringstoformasinglestring.(b)Ditto,butputa
newlinecharacterbetweeneachoriginalstring.
Solutions:
1. Therstrip()methodstripswhitespaceofftherightsideofastring:
>>>s1='sometext\n'
>>>s1
'sometext\n'
>>>s2=s1.rstrip()
>>>s2
'sometext'

2. Thecenter(n)methodcentersastringwithinapaddedstringofwidthn:
>>>s1='Dave'
>>>s2=s1.center(20)
>>>s2
'Dave'

3. Theupper()methodproducesanewstringthatconvertsallalphacharactersin
theoriginaltouppercase:
>>>s1='Banana'
>>>s1
'Banana'
>>>s2=s1.upper()
>>>s2
'BANANA'

4. Thesplit(sep)methodproducesalistofstringsthatareseparatedbysepin
theoriginalstring.Ifsepisomitted,whitespaceistreatedastheseparator:
Page186

APythonBook
>>>s1="""howdoesitfeel
...tobeonyourown
...nodirectionsknown
...likearollingstone
..."""
>>>words=s1.split()
>>>words
['how','does','it','feel','to','be','on','your',
'own','no',
'directions','known','like','a','rolling','stone']

Notethatthesplit()functioninthere(regularexpression)moduleisuseful
whentheseparatorismorecomplexthanwhitespaceorasinglecharacter.
5. Thejoin()methodconcatenatesstringsfromalistofstringstoformasingle
string:
>>>lines=[]
>>>lines.append('howdoesitfeel')
>>>lines.append('tobeonyourown')
>>>lines.append('nodirectionsknown')
>>>lines.append('likearollingstone')
>>>lines
['howdoesitfeel','tobeonyourown','no
directionsknown',
'likearollingstone']
>>>s1=''.join(lines)
>>>s2=''.join(lines)
>>>s3='\n'.join(lines)
>>>s1
'howdoesitfeeltobeonyourownnodirections
knownlikearollingstone'
>>>s2
'howdoesitfeeltobeonyourownnodirectionsknown
likearollingstone'
>>>s3
'howdoesitfeel\ntobeonyourown\nnodirections
known\nlikearollingstone'
>>>prints3
howdoesitfeel
tobeonyourown
nodirectionsknown
likearollingstone

3.4.3.4Rawstrings

Rawstringsgiveusaconvenientwaytoincludethebackslashcharacterinastring
withoutescaping(withanadditionalbackslash).Rawstringslooklikeplainliteral
strings,butareprefixedwithan"r"or"R".SeeStringliterals
http://docs.python.org/reference/lexical_analysis.html#stringliterals
Excercises:
Page187

APythonBook
1. Createastringthatcontainsabackslashcharacterusingbothplainliteralstring
andarawstring.
Solutions:
1. Weusean"r"prefixtodefinearawstring:
>>>print'abc\\def'
abc\def
>>>printr'abc\def'
abc\def

3.4.3.5Unicodestrings

Unicodestringsgiveusaconsistentwaytoprocesscharacterdatafromavarietyof
characterencodings.
Excercises:
1. Createseveralunicodestrings.Useboththeunicodeprefixcharacter("u")andthe
unicodetype(unicode(some_string)).
2. Convertastring(possiblyfromanothernonasciiencoding)tounicode.
3. Convertaunicodestringtoanotherencoding,forexample,utf8.
4. Testastringtodetermineifitisunicode.
5. Createastringthatcontainsaunicodecharacter,thatis,acharacteroutsidethe
asciicharacterset.
Solutions:
1. Wecanrepresentunicodestringwitheitherthe"u"prefixorwithacalltothe
unicodetype:
defexercise1():
a=u'abcd'
printa
b=unicode('efgh')
printb

2. Weconvertastringfromanothercharacterencodingintounicodewiththe
decode()stringmethod:
importsys
defexercise2():
a='abcd'.decode('utf8')
printa
b='abcd'.decode(sys.getdefaultencoding())
printb

3. Wecanconvertaunicodestringtoanothercharacterencodingwiththe
encode()stringmethod:
importsys

Page188

APythonBook
defexercise3():
a=u'abcd'
printa.encode('utf8')
printa.encode(sys.getdefaultencoding())

4. Herearetwowaystocheckthetypeofastring:
importtypes
defexercise4():
a=u'abcd'
printtype(a)istypes.UnicodeType
printtype(a)istype(u'')

5. Wecanencodeunicodecharactersinastringinseveralways,forexample,(1)by
definingautf8stringandconvertingittounicodeor(2)definingastringwithan
embeddedunicodecharacteror(3)concatenatingaunicodecharacherintoa
string:
defexercise5():
utf8_string='IvanKrsti\xc4\x87'
unicode_string=utf8_string.decode('utf8')
printunicode_string.encode('utf8')
printlen(utf8_string)
printlen(unicode_string)
unicode_string=u'aa\u0107bb'
printunicode_string.encode('utf8')
unicode_string='aa'+unichr(263)+'bb'
printunicode_string.encode('utf8')

Guidanceforuseofencodingsandunicode:
1. Convert/decodefromanexternalencodingtounicodeearly:
my_source_string.decode(encoding)

2. Doyourwork(Pythonprocessing)inunicode.
3. Convert/encodetoanexternalencodinglate(forexample,justbeforesavingtoan
externalfile):
my_unicode_string.encode(encoding)

Formoreinformation,see:

UnicodeInPython,CompletelyDemystifiedhttp://farmdev.com/talks/unicode/
UnicodeHowtohttp://www.amk.ca/python/howto/unicode.
PEP100:PythonUnicodeIntegration
http://www.python.org/dev/peps/pep0100/
4.8codecsCodecregistryandbaseclasses
http://docs.python.org/lib/modulecodecs.html
4.8.2EncodingsandUnicode
Page189

APythonBook

http://docs.python.org/lib/encodingsoverview.html
4.8.3StandardEncodingshttp://docs.python.org/lib/standardencodings.html
ConvertingUnicodeStringsto8bitStrings
http://effbot.org/zone/unicodeconvert.htm

3.4.4Dictionaries
Adictionaryisanunorderedcollectionofkeyvaluepairs.
Adictionaryhasalength,specificallythenumberofkeyvaluepairs.
Adictionaryprovidesfastlookupbykey.
Thekeysmustbeimmutableobjecttypes.
3.4.4.1Literalrepresentationofdictionaries

Curleybracketsareusedtorepresentadictionary.Eachpairinthedictionaryis
representedbyakeyandvalueseparatedbyacolon.Multiplepairsareseparatedby
comas.Forexample,hereisanemptydictionaryandseveraldictionariescontaining
key/valuepairs:
In[4]:d1={}
In[5]:d2={'width':8.5,'height':11}
In[6]:d3={1:'RED',2:'GREEN',3:'BLUE',}
In[7]:d1
Out[7]:{}
In[8]:d2
Out[8]:{'height':11,'width':8.5}
In[9]:d3
Out[9]:{1:'RED',2:'GREEN',3:'BLUE'}

Notes:
Acommaafterthelastpairisoptional.SeetheREDGREENBLUEexample
above.
Stringsandintegersworkaskeys,sincetheyareimmutable.Youmightalsowant
tothinkabouttheuseoftuplesofintegersaskeysinadictionaryusedto
representasparsearray.
Exercises:

1. Defineadictionarythathasthefollowingkeyvaluepairs:
2. Defineadictionarytorepresentthe"enum"daysoftheweek:Sunday,Monday,
Tuesday,...
Solutions:
1. Adictionarywhosekeysandvaluesarestringscanbeusedtorepresentthistable:

Page190

APythonBook
vegetables={
'Eggplant':'Purple',
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}

Notethattheopencurlybracketenablesustocontinuethisstatementacross
multiplelineswithoutusingabackslash.
2. Wemightusestringsforthenamesofthedaysoftheweekaskeys:
DAYS={
'Sunday':1,
'Monday':2,
'Tuesday':3,
'Wednesday':4,
'Thrusday':5,
'Friday':6,
'Saturday':7,
}

3.4.4.2Operatorsondictionaries

Dictionariessupportthefollowing"operators":

Lengthlen(d)returnsthenumberofpairsinadictionary.
IndexingYoucanbothsetandgetthevalueassociatedwithakeybyusingthe
indexingoperator[].Examples:
In[12]:d3[2]
Out[12]:'GREEN'
In[13]:d3[0]='WHITE'
In[14]:d3[0]
Out[14]:'WHITE'

TestforkeyTheinoperatortestsfortheexistenceofakeyinadictionary.
Example:
In[6]:trees={'poplar':'deciduous','cedar':
'evergreen'}
In[7]:if'cedar'intrees:
...:print'Thecedaris%s'%
(trees['cedar'],)
...:
Thecedarisevergreen

Exercises:
1. Createanemptydictionary,thenusetheindexingoperator[]toinsertthe
followingnamevaluepairs:
"red""255:0:0"

Page191

APythonBook
"green""0:255:0"
"blue""0:0:255"

2. Printoutthenumberofitemsinyourdictionary.
Solutions:
1. Wecanuse"[]"tosetthevalueofakeyinadictionary:
deftest():
colors={}
colors["red"]="255:0:0"
colors["green"]="0:255:0"
colors["blue"]="0:0:255"
print'Thevalueofredis"%s"'%
(colors['red'],)
print'Thecolorsdictionarycontains%ditems.'%
(len(colors),)
test()

Whenwerunthis,wesee:
Thevalueofredis"255:0:0"
Thecolorsdictionarycontains3items.

2. Thelen()builtinfunctiongivesusthenumberofitemsinadictionary.Seethe
previoussolutionforanexampleofthis.
3.4.4.3Methodsondictionaries

Hereisatablethatdescribesthemethodsapplicabletodictionarys:
Operation

Result

len(a)

thenumberofitemsina

a[k]

theitemofawithkeyk

a[k]=v

seta[k]tov

dela[k]

removea[k]froma

a.clear()

removeallitemsfroma

a.copy()

a(shallow)copyofa

kina

Trueifahasakeyk,elseFalse

knotina

equivalenttonotkina

a.has_key(k)

equivalenttokina,usethatforminnewcode

a.items()

acopyofa'slistof(key,value)pair
Page192

APythonBook
Operation

Result

a.keys()

acopyofa'slistofkeys

a.update([b])

updatesawithkey/valuepairsfromb,overwritingexisting
keys,returnsNone

a.fromkeys(seq[,value])

createsanewdictionarywithkeysfromseqandvaluessetto
value

a.values()

acopyofa'slistofvalues

a.get(k[,x])

a[k]ifkina,elsex)

a.setdefault(k[,x])

a[k]ifkina,elsex(alsosettingit)

a.pop(k[,x])

a[k]ifkina,elsex(andremovek)(8)

a.popitem()

removeandreturnanarbitrary(key,value)pair

a.iteritems()

returnaniteratorover(key,value)pairs

a.iterkeys()

returnaniteratoroverthemapping'skeys

a.itervalues()

returnaniteratoroverthemapping'svalues

YoucanalsofindthistableatthestandarddocumentationWebsiteinthe"Python
LibraryReference":MappingTypesdicthttp://docs.python.org/lib/typesmapping.html
Exercises:
1. Printthekeysandvaluesintheabove"vegetable"dictionary.
2. Printthekeysandvaluesintheabove"vegetable"dictionarywiththekeysin
alphabeticalorder.
3. Testfortheoccuranceofakeyinadictionary.
Solutions:
1. Wecanusethed.items()methodtoretrievealistoftuplescontaining
keyvaluepairs,thenuseunpackingtocapturethekeyandvalue:
Vegetables={
'Eggplant':'Purple',
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}

Page193

APythonBook
deftest():
forkey,valueinVegetables.items():
print'key:',key,'value:',value
test()

2. Weretrievealistofkeyswiththekeys()method,thesortitwiththelist
sort()method:
Vegetables={
'Eggplant':'Purple',
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
keys=Vegetables.keys()
keys.sort()
forkeyinkeys:
print'key:',key,'value:',Vegetables[key]
test()

3. Totestfortheexistenceofakeyinadictionary,wecanuseeitherthein
operator(preferred)orthed.has_key()method(oldstyle):
Vegetables={
'Eggplant':'Purple',
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
if'Eggplant'inVegetables:
print'wehave%segplants'%
Vegetables['Eggplant']
if'Banana'notinVegetables:
print'yeswehavenobananas'
ifVegetables.has_key('Parsley'):
print'wehaveleafy,%sparsley'%
Vegetables['Parsley']
test()

Whichwillprintout:
wehavePurpleegplants
yeswehavenobananas
wehaveleafy,Greenparsley

Page194

APythonBook

3.4.5Files
APythonfileobjectrepresentsafileonafilesystem.
Afileobjectopenforreadingatextfileisiterable.Whenweiterateoverit,itproduces
thelinesinthefile.
Afilemaybeopenedinthesemodes:
'r'readmode.Thefilemustexist.
'w'writemode.Thefileiscreated;anexistingfileisoverwritten.
'a'appendmode.Anexistingfileisopenedforwriting(attheendofthefile).A
fileiscreatedifitdoesnotexist.
Theopen()builtinfunctionisusedtocreateafileobject.Forexample,thefollowing
code(1)opensafileforwriting,then(2)forreading,then(3)forappending,andfinally
(4)forreadingagain:

deftest(infilename):
#1.Openthefileinwritemode,whichcreatesthefile.
outfile=open(infilename,'w')
outfile.write('line1\n')
outfile.write('line2\n')
outfile.write('line3\n')
outfile.close()
#2.Openthefileforreading.
infile=open(infilename,'r')
forlineininfile:
print'Line:',line.rstrip()
infile.close()
#3.Openthefileinappendmode,andaddalinetotheendof
#thefile.
outfile=open(infilename,'a')
outfile.write('line4\n')
outfile.close()
print''*40
#4.Openthefileinreadmodeoncemore.
infile=open(infilename,'r')
forlineininfile:
print'Line:',line.rstrip()
infile.close()
test('tmp.txt')

Exercises:
1. Openatextfileforreading,thenreadtheentirefileasasinglestring,andthen
splitthecontentonnewlinecharacters.
2. Openatextfileforreading,thenreadtheentirefileasalistofstrings,whereeach
stringisonelineinthefile.
3. Openatextfileforreading,theniterateofeachlineinthefileandprintitout.
Page195

APythonBook
Solutions:
1. Usetheopen()builtinfunctiontoopenthefileandcreateafileobject.Usethe
read()methodonthefileobjecttoreadtheentirefile.Usethesplit()or
splitlines()methodstosplitthefileintolines:
>>>infile=open('tmp.txt','r')
>>>content=infile.read()
>>>infile.close()
>>>lines=content.splitlines()
>>>printlines
['line1','line2','line3','']

2. Thef.readlines()methodreturnsalistoflinesinafile:
>>>infile=open('tmp.txt','r')
>>>lines=infile.readlines()
>>>infile.close()
>>>printlines
['line1\n','line2\n','line3\n']

3. Sinceafileobject(openforreading)isitselfaniterator,wecaniterateoveritina
forstatement:
"""
Testiterationoveratextfile.
Usage:
pythontest.pyin_file_name
"""
importsys
deftest(infilename):
infile=open(infilename,'r')
forlineininfile:
#Stripoffthenewlinecharacterandany
whitespaceon
#theright.
line=line.rstrip()
#Printonlynonblanklines.
ifline:
printline
infile.close()
defmain():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
infilename=args[0]
test(infilename)
if__name__=='__main__':
main()

Page196

APythonBook
Notes:
Thelasttwolinesofthissolutioncheckthe__name__attributeofthe
moduleitselfsothatthemodulewillrunasascriptbutwillnotrunwhenthe
moduleisimportedbyanothermodule.
The__doc__attributeofthemodulegivesusthemodule'sdocstring,which
isthestringdefinedatthetopofthemodule.
sys.argvgivesusthecommandline.And,sys.argv[1:]chopsoffthe
programname,leavinguswiththecommanlinearguments.

3.4.6Afewmiscellaneousdatatypes
3.4.6.1None

Noneisasingleton.ThereisonlyoneinstanceofNone.Usethisvaluetoindicatethe
absenceofanyother"real"value.
TestforNonewiththeidentityoperatoris.
Exercises:
1. Createalist,someofwhoseelementsareNone.Thenwriteaforloopthat
countsthenumberofoccurancesofNoneinthelist.
Solutions:
1. TheidentityoperatorsisandisnotcanbeusedtotestforNone:
>>>a=[11,None,'abc',None,{}]
>>>a
[11,None,'abc',None,{}]
>>>count=0
>>>foritemina:
...ifitemisNone:
...count+=1
...
>>>
>>>printcount
2

3.4.6.2ThebooleansTrueandFalse

PythonhasthetwobooleanvaluesTrueandFalse.Manycomparisonoperatorsreturn
TrueandFalse.
Examples:
1. Whatvalueisreturnedby3>2?
Answer:ThebooleanvalueTrue.
2. Giventhesevariabledefinitions:
Page197

APythonBook
x=3
y=4
z=5

Whatdoesthefollowingprintout:
printy>xandz>y

AnswerPrintsout"True"

3.5Statements
3.5.1Assignmentstatement
Theassignmentstatementusestheassignmentoperator=.
Theassignmentstatementisabindingstatement:itbindsavaluetoanamewithina
namespace.
Exercises:
1. Bindthevalue"eggplant"tothevariablevegetable.
Solutions:
1.The=operatorisanassignmentstatementthatbindsavaluetoavariable:
>>>vegetable="eggplant"

Thereisalsoaugmentedassignmentusingtheoperators+=,=,*=,/=,etc.
Exercises:
1. Useaugmentedassignmenttoincrementthevalueofaninteger.
2. Useaugmentedassignmenttoappendcharacterstotheendofastring.
3. Useaugmentedassignmenttoappendtheitemsinonelisttoanother.
4. Useaugmentedassignmenttodecrementavariablecontaininganintegerby1.
Solutions:
1. The+=operatorincrementsthevalueofaninteger:
>>>count=0
>>>count+=1
>>>count
1
>>>count+=1
>>>count
2

2. The+=operatorappendscharacterstotheendofastring:
>>>buffer='abcde'
>>>buffer+='fgh'

Page198

APythonBook
>>>buffer
'abcdefgh'

3. The+=operatorappendsitemsinonelisttoanother:
In[20]:a=[11,22,33]
In[21]:b=[44,55]
In[22]:a+=b
In[23]:a
Out[23]:[11,22,33,44,55]

1. The=operatordecrementsthevalueofaninteger:
>>>count=5
>>>count
5
>>>count=1
>>>count
4

Youcanalsoassignavalueto(1)anelementofalist,(2)aniteminadictionary,(3)an
attributeofanobject,etc.
Exercises:
1. Createalistofthreeitems,thenassignanewvaluetothe2ndelementinthelist.
2. Createadictionary,thenassignvaluestothekeys"vegetable"and"fruit"inthat
dictionary.
3. Usethefollowingcodetocreateaninstanceofaclass:
classA(object):
pass
a=A()

Thenassignvaluestoanattribuenamedcategoryinthatinstance.
Solutions:
1. Assignmentwiththeindexingoperator[]assignsavaluetoanelementinalist:
>>>trees=['pine','oak','elm']
>>>trees
['pine','oak','elm']
>>>trees[1]='cedar'
>>>trees
['pine','cedar','elm']

2. Assignmentwiththeindexingoperator[]assignsavaluetoanitem(akeyvalue
pair)inadictionary:
>>>foods={}
>>>foods
{}
>>>foods['vegetable']='greenbeans'
>>>foods['fruit']='nectarine'
>>>foods

Page199

APythonBook
{'vegetable':'greenbeans','fruit':'nectarine'}

3. Assignmentalongwiththedereferencingoperator.(dot)enablesustoassigna
valuetoanattributeofanobject:
>>>classA(object):
...pass
...
>>>a=A()
>>>a.category=25
>>>a.__dict__
{'category':25}
>>>a.category
25

3.5.2printstatement
Warning:BeawarethattheprintstatementwillgoawayinPythonversion3.0.Itwill
bereplacedbythebuiltinprint()function.
Theprintstatementsendsoutputtostandardoutput.Itprovidesasomewhatmore
convenientwayofproducingoutputthanusingsys.stdout.write().
Theprintstatementtakesaseriesofzeroormoreobjectsseparatedbycommas.Zero
objectsproducesablankline.
Theprintstatementnormallyaddsanewlineattheendofitsoutput.Toeliminatethat,
addacommaattheend.
Exercises:
1. Printasinglestring.
2. Printthreestringsusingasingleprintstatement.
3. Givenavariablenamecontainingastring,printoutthestringMynameis
"xxxx".,wherexxxxisreplacebythevalueofname.Usethestringformatting
operator.
Solutions:
1. Wecanprintaliteralstring:
>>>print'Hello,there'
Hello,there

2. Wecanprintliteralsandthevalueofvariables:
>>>description='cute'
>>>print'Iama',description,'kid.'
Iamacutekid.

3. Thestringformattingoperatorgivesmorecontroloverformattingoutput:
>>>name='Alice'

Page200

APythonBook
>>>print'Mynameis"%s".'%(name,)
Mynameis"Alice".

3.5.3if:statementexercises
Theifstatementisacompoundstatementthatenablesustoconditionallyexecute
blocksofcode.
Theifstatementalsohasoptionalelif:andelse:clauses.
Theconditioninanif:orelif:clausecanbeanyPythonexpression,inotherwords,
somethingthatreturnsavalue(evenifthatvalueisNone).
Intheconditioninanif:orelif:clause,thefollowingvaluescountas"false":
False
None
Numericzero
Anemptycollection,forexampleanemptylistordictionary
Anemptystring(astringoflengthzero)
Allothervaluescountastrue.

Exercises:
1. Giventhefollowinglist:
>>>bananas=['banana1','banana2','banana3',]

Printonemessageifitisanemptylistandanothermessgeifitisnot.
2. HereisonewayofdefiningaPythonequivalentofan"enum":
NO_COLOR,RED,GREEN,BLUE=range(4)

Writeanif:statementwhichimplementstheeffectofa"switch"statementin
Python.Printoutauniquemessageforeachcolor.
Solutions:
1. Wecantestforanemptyornonemptylist:
>>>bananas=['banana1','banana2','banana3',]
>>>ifnotbananas:
...print'yes,wehavenobananas'
...else:
...print'yes,wehavebananas'
...
yes,wehavebananas

2. Wecansimulatea"switch"statementusingif:elif:...:
NO_COLOR,RED,GREEN,BLUE=range(4)
deftest(color):

Page201

APythonBook
ifcolor==RED:
print"It'sred."
elifcolor==GREEN:
print"It'sgreen."
elifcolor==BLUE:
print"It'sblue."
defmain():
color=BLUE
test(color)
if__name__=='__main__':
main()

Which,whenrunprintsoutthefollowing:
It'sblue.

3.5.4for:statementexercises
Thefor:statementisthePythonwaytoiterateoverandprocesstheelementsofa
collectionorotheriterable.
Thebasicformofthefor:statementisthefollowing:
forXinY:
statement
o
o
o

where:
Xissomethingthatcanbeassignedto.ItissomethingtowhichPythoncanbinda
value.
Yissomecollectionorotheriterable.
Exercises:

1. Createalistofintegers.Useafor:statementtoprintouteachintegerinthelist.
2. Createastring.printouteachcharacterinthestring.
Solutions:
1. Thefor:statementcaniterateovertheitemsinalist:
In[13]:a=[11,22,33,]
In[14]:forvalueina:
....:print'value:%d'%value
....:
....:
value:11
value:22
value:33

Page202

APythonBook
2. Thefor:statementcaniterateoverthecharactersinastring:
In[16]:b='chocolate'
In[17]:forchr1inb:
....:print'character:%s'%chr1
....:
....:
character:c
character:h
character:o
character:c
character:o
character:l
character:a
character:t
character:e

Notes:
Inthesolution,Iusedthevariablenamechr1ratherthanchrsoasnotto
overwritethenameofthebuiltinfunctionchr().
Whenweneedasequentialindex,wecanusetherange()builtinfunctiontocreatea
listofintegers.And,thexrange()builtinfunctionproducesaninteratorthatproduces
asequenceofintegerswithoutcreatingtheentirelist.Toiterateoveralargesequenceof
integers,usexrange()insteadofrange().
Exercises:
1. Printouttheintegersfrom0to5insequence.
2. Computethesumofalltheintegersfrom0to99999.
3. Giventhefollowinggeneratorfunction:
importurllib
Urls=[
'http://yahoo.com',
'http://python.org',
'http://gimp.org',#TheGNUimagemanipulation
program
]
defwalk(url_list):
forurlinurl_list:
f=urllib.urlopen(url)
stuff=f.read()
f.close()
yieldstuff

Writeafor:statementthatusesthisiteratorgeneratortoprintthelengthsofthe
contentateachoftheWebpagesinthatlist.
Solutions:
1. Therange()builtinfunctiongivesusasequencetoiterateover:
Page203

APythonBook
In[5]:foridxinrange(6):
...:print'idx:%d'%idx
...:
...:
idx:0
idx:1
idx:2
idx:3
idx:4
idx:5

2. Sincethatsequenceisabitlarge,we'llusexrange()insteadofrange():
In[8]:count=0
In[9]:forninxrange(100000):
...:count+=n
...:
...:
In[10]:count
Out[10]:4999950000

3. Thefor:statementenablesustoiterateoveriterablesaswellascollections:
importurllib
Urls=[
'http://yahoo.com',
'http://python.org',
'http://gimp.org',#TheGNUimagemanipulation
program
]
defwalk(url_list):
forurlinurl_list:
f=urllib.urlopen(url)
stuff=f.read()
f.close()
yieldstuff
deftest():
forurlinwalk(Urls):
print'length:%d'%(len(url),)
if__name__=='__main__':
test()

WhenIranthisscript,itprintsthefollowing:
length:9562
length:16341
length:12343

Ifyouneedanindexwhileiteratingoverasequence,considerusingtheenumerate()
builtinfunction.
Page204

APythonBook
Exercises:
1. Giventhefollowingtwolistsofintegersofthesamelength:
a=[1,2,3,4,5]
b=[100,200,300,400,500]

Addthevaluesinthefirstlisttothecorrespondingvaluesinthesecondlist.
Solutions:
1. Theenumerate()builtinfunctiongivesusanindexandvaluesfroma
sequence.Sinceenumerate()givesusaninteratorthatproducesasequenceof
twotuples,wecanunpackthosetuplesintoindexandvaluevariablesinthe
headerlineoftheforstatement:
In[13]:a=[1,2,3,4,5]
In[14]:b=[100,200,300,400,500]
In[15]:
In[16]:foridx,valueinenumerate(a):
....:b[idx]+=value
....:
....:
In[17]:b
Out[17]:[101,202,303,404,505]

3.5.5while:statementexercises
Awhile:statementexecutesablockofcoderepeatedlyaslongasaconditionistrue.
Hereisatemplateforthewhile:statement:
whilecondition:
statement
o
o
o

Where:
conditionisanexpression.Theexpressionissomethingthatreturnsavalue
whichcanbeinterpretedastrueorfalse.
Exercises:

1. Writeawhile:loopthatdoublesallthevaluesinalistofintegers.
Solutions:
1. Awhile:loopwithanindexvariablecanbeusedtomodifyeachelementofa
list:
deftest_while():
numbers=[11,22,33,44,]
print'before:%s'%(numbers,)

Page205

APythonBook
idx=0
whileidx<len(numbers):
numbers[idx]*=2
idx+=1
print'after:%s'%(numbers,)

But,noticethatthistaskiseasierusingthefor:statementandthebuiltin
enumerate()function:
deftest_for():
numbers=[11,22,33,44,]
print'before:%s'%(numbers,)
foridx,iteminenumerate(numbers):
numbers[idx]*=2
print'after:%s'%(numbers,)

3.5.6breakandcontinuestatements
Thecontinuestatementskipstheremainderofthestatementsinthebodyofaloop
andstartsimmediatelyatthetopoftheloopagain.
Abreakstatementinthebodyofaloopterminatestheloop.Itexitsfromthe
immediatelycontainingloop.
breakandcontinuecanbeusedinbothfor:andwhile:statements.
Exercises:
1. Writeafor:loopthattakesalistofintegersandtripleseachintegerthatiseven.
Usethecontinuestatement.
2. Writealoopthattakesalistofintegersandcomputesthesumofalltheintegers
upuntilazeroisfoundinthelist.Usethebreakstatement.
Solutions:
1. Thecontinuestatementenablesusto"skip"itemsthatsatisfyaconditionor
test:
deftest():
numbers=[11,22,33,44,55,66,]
print'before:%s'%(numbers,)
foridx,iteminenumerate(numbers):
ifitem%2!=0:
continue
numbers[idx]*=3
print'after:%s'%(numbers,)
test()

2. Thebreakstatementenablesustoexitfromaloopwhenwefindazero:
deftest():
numbers=[11,22,33,0,44,55,66,]

Page206

APythonBook
print'numbers:%s'%(numbers,)
sum=0
foriteminnumbers:
ifitem==0:
break
sum+=item
print'sum:%d'%(sum,)
test()

3.5.7Exceptionsandthetry:except:andraisestatements
Thetry:except:statementenablesustocatchanexceptionthatisthrownfrom
withinablockofcode,orfromcodecalledfromanydepthwithingthatblock.
Theraisestatementenablesustothrowanexception.
Anexceptionisaclassoraninstanceofanexceptionclass.Ifanexceptionisnotcaught,
itresultsinatracebackandterminationoftheprogram.
Thereisasetofstandardexceptions.Youcanlearnaboutthemhere:BuiltinExceptions
http://docs.python.org/lib/moduleexceptions.html.
Youcandefineyourownexceptionclasses.Todoso,createanemptysubclassofthe
classException.Definingyourownexceptionwillenableyou(orothers)tothrow
andthencatchthatspecificexceptiontypewhileignoreothersexceptions.
Exercises:
1. Writeatry:except:statementthatattemptstoopenafileforreadingand
catchestheexceptionthrownwhenthefiledoesnotexist.
Question:Howdoyoufindoutthenameoftheexceptionthatisthrownforan
input/outputerrorsuchasthefailuretoopenafile?
2. Defineanexceptionclass.Thenwriteatry:except:statementinwhichyou
throwandcatchthatspecificexception.
3. Defineanexceptionclassanduseittoimplementamultilevelbreakfroman
innerloop,bypassinganouterloop.
Solutions:
1. UsethePythoninteractiveinterpretertolearntheexceptiontypethrownwhena
I/Oerroroccurs.Example:
>>>infile=open('xx_nothing__yy.txt','r')
Traceback(mostrecentcalllast):
File"<stdin>",line1,in<module>
IOError:[Errno2]Nosuchfileordirectory:
'xx_nothing__yy.txt'
>>>

Page207

APythonBook
Inthiscase,theexceptiontypeisIOError.
Now,writeatry:except:blockwhichcatchesthatexception:
deftest():
infilename='nothing_noplace.txt'
try:
infile=open(infilename,'r')
forlineininfile:
printline
exceptIOError,exp:
print'cannotopenfile"%s"'%infilename
test()

2. WedefineaexceptionclassasasubclassofclassException,thenthrowit
(withtheraisestatement)andcatchit(withatry:except:statement):
classSizeError(Exception):
pass
deftest_exception(size):
try:
ifsize<=0:
raiseSizeError,'sizemustbegreaterthan
zero'
#Produceadifferenterrortoshowthatit
willnotbecaught.
x=y
exceptSizeError,exp:
print'%s'%(exp,)
print'goodbye'
deftest():
test_exception(1)
print''*40
test_exception(1)
test()

Whenwerunthisscript,itproducesthefollowingoutput:
$pythonworkbook027.py
sizemustbegreaterthanzero
goodbye

Traceback(mostrecentcalllast):
File"workbook027.py",line20,in<module>
test()
File"workbook027.py",line18,intest
test_exception(1)
File"workbook027.py",line10,intest_exception
x=y
NameError:globalname'y'isnotdefined

Page208

APythonBook
Notes:
Ourexcept:clausecaughttheSizeError,butallowedtheNameError
tobeuncaught.
3. WedefineasubclassofofclassException,thenraiseitinaninnerloopand
catchitoutsideofanouterloop:
classBreakException1(Exception):
pass
deftest():
a=[11,22,33,44,55,66,]
b=[111,222,333,444,555,666,]
try:
forxina:
print'outerx:%d'%x
foryinb:
ifx>22andy>444:
raiseBreakException1('leaving
innerloop')
print'innery:%d'%y
print'outerafter'
print''*40
exceptBreakException1,exp:
print'outofloopexp:%s'%exp
test()

Hereiswhatthisprintsoutwhenrun:
outerx:11
innery:111
innery:222
innery:333
innery:444
innery:555
innery:666
outerafter

outerx:22
innery:111
innery:222
innery:333
innery:444
innery:555
innery:666
outerafter

outerx:33
innery:111
innery:222
innery:333
innery:444
outofloopexp:leavinginnerloop

Page209

APythonBook

3.6Functions
Afunctionhasthesecharacteristics:
Itgroupsablockofcodetogethersothatwecancallitbyname.
Itenablesustopassvaluesintothethefunctionwhenwecallit.
Itcanreturnsavalue(evenifNone).
Whenafunctioniscalled,ithasitsownnamespace.Variablesinthefunctionare
localtothefunction(anddisappearwhenthefunctionexits).
Afunctionisdefinedwiththedef:statement.Hereisasimpleexample/template:

deffunction_name(arg1,arg2):
local_var1=arg1+1
local_var2=arg2*2
returnlocal_var1+local_var2

And,hereisanexampleofcallingthisfunction:
result=function_name(1,2)

Hereareafewnotesofexplanation:
Theabovedefinesafunctionwhosenameisfunction_name.
Thefunctionfunction_namehastwoarguments.Thatmeansthatwecanand
mustpassinexactlytwovalueswhenwecallit.
Thisfunctionhastwolocalvariables,local_var1andlocal_var2.These
variablesarelocalinthesensethatafterwecallthisfunction,thesetwovariables
arenotavailableinthelocationofthecaller.
Whenwecallthisfunction,itreturnsonevalue,specificallythesumof
local_var1andlocal_var2.
Exercises:

1. Writeafunctionthattakesalistofintegersasanargument,andreturnsthesum
oftheintegersinthatlist.
Solutions:
1. Thereturnstatementenablesustoreturnavaluefromafunction:
deflist_sum(values):
sum=0
forvalueinvalues:
sum+=value
returnsum
deftest():
a=[11,22,33,44,]
printlist_sum(a)
if__name__=='__main__':

Page210

APythonBook
test()

3.6.1Optionalargumentsanddefaultvalues
Youcanprovideadefaultvalueforanargumenttoafunction.
Ifyoudo,thatargumentisoptional(whenthefunctioniscalled).
Hereareafewthingstolearnaboutoptionalarguments:

Provideadefaultvaluewithanequalsignandavalue.Example:
defsample_func(arg1,arg2,arg3='empty',arg4=0):

Allparameterswithdefaultvaluesmustbeafter(totherightof)normal
parameters.
Donotuseamutableobjectasadefaultvalue.Becausethedef:statementis
evaluatedonlyonceandnoteachtimethefunctioniscalled,themutableobject
mightbesharedacrossmultiplecallstothefunction.Donotdothis:
defsample_func(arg1,arg2=[]):

Instead,dothis:
defsample_func(arg1,arg2=None):
ifarg2isNone:
arg2=[]

Hereisanexamplethatillustrateshowthismightgowrong:
defadder(a,b=[]):
b.append(a)
returnb
deftest():
printadder('aaa')
printadder('bbb')
printadder('ccc')
test()

Which,whenexecuted,displaysthefollowing:
['aaa']
['aaa','bbb']
['aaa','bbb','ccc']

Exercises:
1. Writeafunctionthatwritesastringtoafile.Thefunctiontakestwoarguments:
(1)afilethatisopenforoutputand(2)astring.Givethesecondargument(the
string)adefaultvaluesothatwhenthesecondargumentisomitted,anempty,
blanklineiswrittentothefile.
Page211

APythonBook
2. Writeafunctionthattakesthefollowingarguments:(1)aname,(2)avalue,and
(3)andoptionaldictionary.Thefunctionaddsthevaluetothedictionaryusingthe
nameasakeyinthedictionary.
Solutions:
1. Wecanpassafileaswewouldanyotherobject.And,wecanuseanewline
characterasadefaultparametervalue:
importsys
defwriter(outfile,msg='\n'):
outfile.write(msg)
deftest():
writer(sys.stdout,'aaaaa\n')
writer(sys.stdout)
writer(sys.stdout,'bbbbb\n')
test()

Whenrunfromthecommandline,thisprintsoutthefollowing:
aaaaa
bbbbb

2. Inthissolutionwearecarefulnottouseamutableobjectasadefaultvalue:
defadd_to_dict(name,value,dic=None):
ifdicisNone:
dic={}
dic[name]=value
returndic
deftest():
dic1={'albert':'cute',}
printadd_to_dict('barry','funny',dic1)
printadd_to_dict('charlene','smart',dic1)
printadd_to_dict('darryl','outrageous')
printadd_to_dict('eddie','friendly')
test()

Ifwerunthisscript,wesee:
{'barry':'funny','albert':'cute'}
{'barry':'funny','albert':'cute','charlene':
'smart'}
{'darryl':'outrageous'}
{'eddie':'friendly'}

Notes:
It'simportantthatthedefaultvalueforthedictionaryisNoneratherthanan
emptydictionary,forexample({}).Rememberthatthedef:statementis
Page212

APythonBook
evaluatedonlyonce,whichresultsinasingledictionary,whichwouldbe
sharedbyallcallersthatdonotprovideadictionaryasanargument.

3.6.2Passingfunctionsasarguments
Afunction,likeanyotherobject,canbepassedasanargumenttoafunction.Thisisdue
thethefactthatalmostall(maybeall)objectsinPythonare"firstclassobjects".Afirst
classobjectisonewhichwecan:
1. Storeinadatastructure(e.g.alist,adictionary,...).
2. Passtoafunction.
3. Returnfromafunction.
Exercises:
1. Writeafunctionthattakesthreearguments:(1)aninputfile,(2)anoutputfile,
and(3)afilterfunction:
Argument1isafileopenedforreading.
Argument2isafileopenedforwriting.
Argument3isafunctionthattakesasingleargument(astring),performsa
transformationonthatstring,andreturnsthetransformedstring.
Theabovefunctionshouldreadeachlineintheinputtextfile,passthatline
throughthefilterfunction,thenwritethat(possibly)transformedlinetothe
outputfile.
Now,writeoneormore"filterfunctions"thatcanbepassedtothefunction
describedabove.
Solutions:
1. Thisscriptaddsorremovescommentcharacterstothelinesofafile:
importsys
deffilter(infile,outfile,filterfunc):
forlineininfile:
line=filterfunc(line)
outfile.write(line)
defadd_comment(line):
line='##%s'%(line,)
returnline
defremove_comment(line):
ifline.startswith('##'):
line=line[3:]
returnline
defmain():
filter(sys.stdin,sys.stdout,add_comment)

Page213

APythonBook
if__name__=='__main__':
main()

Runningthismightproducesomethinglikethefollowing(noteforMSWindows
users:usetypeinsteadofcat):
$cattmp.txt
line1
line2
line3
$cattmp.txt|pythonworkbook005.py
##line1
##line2
##line3

3.6.3Extraargsandkeywordargs
Additionalpositionalargumentspassedtoafunctionthatarenotspecifiedinthefunction
definition(thedef:statement``),arecollectedinanargumentprecededbyasingle
asterisk.Keywordargumentspassedtoafunctionthatarenotspecifiedinthefunction
definitioncanbecollectedinadictionaryandpassedtoanargumentprecededbya
doubleasterisk.
Examples:
1. Writeafunctionthattakesonepositionalargument,oneargumentwithadefault
value,andalsoextraargsandkeywordargs.
2. Writeafunctionthatpassesallitsarguments,nomatterhowmany,toacallto
anotherfunction.
Solutions:
1. Weuse*argsand**kwargstocollectextraargumentsandextrakeyword
arguments:
defshow_args(x,y=1,*args,**kwargs):
print''*40
print'x:',x
print'y:',y
print'args:',args
print'kwargs:',kwargs
deftest():
show_args(1)
show_args(x=2,y=3)
show_args(y=5,x=4)
show_args(4,5,6,7,8)
show_args(11,y=44,a=55,b=66)
test()

Page214

APythonBook
Runningthisscriptproducesthefollowing:
$pythonworkbook006.py

x:1
y:1
args:()
kwargs:{}

x:2
y:3
args:()
kwargs:{}

x:4
y:5
args:()
kwargs:{}

x:4
y:5
args:(6,7,8)
kwargs:{}

x:11
y:44
args:()
kwargs:{'a':55,'b':66}

Notes:
Thespellingofargsandkwargsisnotfixed,butthe
2. Weuseargsandkwargstocatchandpassonallarguments:
deffunc1(*args,**kwargs):
print'args:%s'%(args,)
print'kwargs:%s'%(kwargs,)
deffunc2(*args,**kwargs):
print'before'
func1(*args,**kwargs)
print'after'
deftest():
func2('aaa','bbb','ccc',arg1='ddd',arg2='eee')
test()

Whenwerunthis,itprintsthefollowing:
before
args:('aaa','bbb','ccc')
kwargs:{'arg1':'ddd','arg2':'eee'}
after

Page215

APythonBook
Notes:
Inafunctioncall,the*operatorunrollsalistintoindividualpositional
arguments,andthe**operatorunrollsadictionaryintoindividualkeyword
arguments.
3.6.3.1Orderofarguments(positional,extra,andkeywordargs)

Inafunctiondefinition,argumentsmustappearinthefollowingorder,fromlefttoright:
1. Positional(normal,plain)arguments
2. Argumentswithdefaultvalues,ifany
3. Extraargumentsparameter(procededbysingleasterisk),ifpresent
4. Keywordargumentsparameter(procededbydoubleasterisk),ifpresent
Inafunctioncall,argumentsmustappearinthefollowingorder,fromlefttoright:
1. Positional(plain)arguments
2. Extraarguments,ifpresent
3. Keywordarguments,ifpresent

3.6.4Functionsandducktypingandpolymorphism
Iftheargumentsandreturnvalueofafunctionsatisfysomedescription,thenwecansay
thatthefunctionispolymorphicwithrespecttothatdescription.
Ifthesomeofthemethodsofanobjectsatisfysomedescription,thenwecansaythatthe
objectispolymorphicwithrespecttothatdescription.
Basically,whatthisdoesistoenableustouseafunctionoranobjectanywherethat
functionsatisfiestherequirementsgivenbyadescription.
Exercises:
1. Implementafunctionthattakestwoarguments:afunctionandanobject.It
appliesthefunctionargumenttotheobject.
2. Implementafunctionthattakestwoarguments:alistoffunctionsandanobject.
Itapplieseachfunctioninthelisttotheargument.
Solutions:
1. Wecanpassafunctionasanargumenttoafunction:
deffancy(obj):
print'fancyfancy%sfancyfancy'%(obj,)
defplain(obj):
print'plain%splain'%(obj,)
defshow(func,obj):
func(obj)

Page216

APythonBook
defmain():
a={'aa':11,'bb':22,}
show(fancy,a)
show(plain,a)
if__name__=='__main__':
main()

2. Wecanalsoputfunctions(functionobjects)inadatastructure(forexample,a
list),andthenpassthatdatastructuretoafunction:
deffancy(obj):
print'fancyfancy%sfancyfancy'%(obj,)
defplain(obj):
print'plain%splain'%(obj,)
Func_list=[fancy,plain,]
defshow(funcs,obj):
forfuncinfuncs:
func(obj)
defmain():
a={'aa':11,'bb':22,}
show(Func_list,a)
if__name__=='__main__':
main()

NoticethatPythonsupportspolymorphism(withor)withoutinheritance.Thistypeof
polymorphismisenabledbywhatiscalledducktyping.Formoreonthissee:Duck
typinghttp://en.wikipedia.org/wiki/Duck_typingatWikipedia.

3.6.5Recursivefunctions
Arecursivefunctionisafunctionthatcallsitself.
Arecursivefunctionmusthavealimitingcondition,orelseitwillloopendlessly.
Eachrecursivecallconsumesspaceonthefunctioncallstack.Therefore,thenumberof
recursionsmusthavesomereasonableupperbound.
Exercises:
1. Writearecursivefunctionthatprintsinformationabouteachnodeinthe
followingtreestructuredatastructure:
Tree={
'name':'animals',
'left_branch':{

Page217

APythonBook
'name':'birds',
'left_branch':{
'name':'seedeaters',
'left_branch':{
'name':'housefinch',
'left_branch':None,
'right_branch':None,
},
'right_branch':{
'name':'whitecrownedsparrow',
'left_branch':None,
'right_branch':None,
},
},
'right_branch':{
'name':'insecteaters',
'left_branch':{
'name':'hermitthrush',
'left_branch':None,
'right_branch':None,
},
'right_branch':{
'name':'blackheadedphoebe',
'left_branch':None,
'right_branch':None,
},
},
},
'right_branch':None,
}

Solutions:
1. Wewritearecursivefunctiontowalkthewholetree.Therecursivefunctioncalls
itselftoprocesseachchildofanodeinthetree:
Tree={
'name':'animals',
'left_branch':{
'name':'birds',
'left_branch':{
'name':'seedeaters',
'left_branch':{
'name':'housefinch',
'left_branch':None,
'right_branch':None,
},
'right_branch':{
'name':'whitecrownedsparrow',
'left_branch':None,
'right_branch':None,
},
},

Page218

APythonBook
'right_branch':{
'name':'insecteaters',
'left_branch':{
'name':'hermitthrush',
'left_branch':None,
'right_branch':None,
},
'right_branch':{
'name':'blackheadedphoebe',
'left_branch':None,
'right_branch':None,
},
},
},
'right_branch':None,
}
Indents=[''*idxforidxinrange(10)]
defwalk_and_show(node,level=0):
ifnodeisNone:
return
print'%sname:%s'%(Indents[level],node['name'],
)
level+=1
walk_and_show(node['left_branch'],level)
walk_and_show(node['right_branch'],level)
deftest():
walk_and_show(Tree)
if__name__=='__main__':
test()

Notes:
Later,youwilllearnhowtocreateequivalentdatastructuresusingclassesand
OOP(objectorientedprogramming).FormoreonthatseeRecursivecallsto
methodsinthisdocument.

3.6.6Generatorsanditerators
The"iteratorprotocol"defineswhataniteratorobjectmustdoinordertobeusableinan
"iteratorcontext"suchasaforstatement.Theiteratorprotocolisdescribedinthe
standardlibraryreference:IteratorTypeshttp://docs.python.org/lib/typeiter.html
Aneasywaytodefineanobjectthatobeystheiteratorprotocolistowriteagenerator
function.Ageneratorfunctionisafunctionthatcontainsoneormoreyieldstatements.
Ifafunctioncontainsatleastoneyieldstatement,thenthatfunctionwhencalled,
returnsgeneratoriterator,whichisanobjectthatobeystheiteratorprotocol,i.e.it'san
iteratorobject.
Page219

APythonBook
NotethatinrecentversionsofPython,yieldisanexpression.Thisenablestheconsumer
tocommunicatebackwiththeproducer(thegeneratoriterator).Formoreonthis,see
PEP:342CoroutinesviaEnhancedGenerators
http://www.python.org/dev/peps/pep0342/.
Exercises:
1. ImplementageneratorfunctionThegeneratorproducedshouldyieldall
valuesfromalist/iterablethatsatisfyapredicate.Itshouldapplythetransforms
beforereturneachvalue.Thefunctiontakesthesearguments:
1. valuesAlistofvalues.Actually,itcouldbeanyiterable.
2. predicateAfunctionthattakesasingleargument,performsateston
thatvalue,andreturnsTrueorFalse.
3. transforms(optional)Alistoffunctions.Applyeachfunctioninthislist
andreturnstheresultingvalue.So,forexample,ifthefunctioniscalledlike
this:
result=transforms([11,22],p,[f,g])

thentheresultinggeneratormightreturn:
g(f(11))

2. ImplementageneratorfunctionthattakesalistofURLsasitsargumentand
generatesthecontentsofeachWebpage,onebyone(thatis,itproducesa
sequenceofstrings,theHTMLpagecontents).
Solutions:
1. Hereistheimplementationofafunctionwhichcontainsyield,and,therefore,
producesagenerator:
#!/usr/bin/envpython
"""
filter_and_transform
filter_and_transform(content,test_func,
transforms=None)
Returnageneratorthatreturnsitemsfromcontent
afterapplying
thefunctionsintransformsiftheitemsatisfies
test_func.
Arguments:
1.``values``Alistofvalues
2.``predicate``Afunctionthattakesasingle
argument,
performsatestonthatvalue,andreturnsTrue

Page220

APythonBook
orFalse.
3.``transforms``(optional)Alistoffunctions.
Applyeach
functioninthislistandreturnstheresulting
value.So,
forexample,ifthefunctioniscalledlike
this::
result=filter_and_transforms([11,22],p,[f,
g])
thentheresultinggeneratormightreturn::
g(f(11))
"""
deffilter_and_transform(content,test_func,
transforms=None):
forxincontent:
iftest_func(x):
iftransformsisNone:
yieldx
elifisiterable(transforms):
forfuncintransforms:
x=func(x)
yieldx
else:
yieldtransforms(x)
defisiterable(x):
flag=True
try:
x=iter(x)
exceptTypeError,exp:
flag=False
returnflag
defiseven(n):
returnn%2==0
deff(n):
returnn*2
defg(n):
returnn**2
deftest():
data1=[11,22,33,44,55,66,77,]
forvalinfilter_and_transform(data1,iseven,f):
print'val:%d'%(val,)
print''*40
forvalinfilter_and_transform(data1,iseven,[f,

Page221

APythonBook
g]):
print'val:%d'%(val,)
print''*40
forvalinfilter_and_transform(data1,iseven):
print'val:%d'%(val,)
if__name__=='__main__':
test()

Notes:
Becausefunctionfilter_and_transformcontainsyield,when
called,itreturnsaniteratorobject,whichwecanuseinaforstatement.
Thesecondparameteroffunctionfilter_and_transformtakesany
functionwhichtakesasingleargumentandreturnsTrueorFalse.Thisisan
exampleofpolymorphismand"ducktyping"(seeDuckTyping
http://en.wikipedia.org/wiki/Duck_typing).Ananalogousclaimcanbemade
aboutthethirdparameter.
2. Thefollowingfunctionusestheurllibmoduleandtheyieldfunctionto
generatethecontentsofasequenceofWebpages:
importurllib
Urls=[
'http://yahoo.com',
'http://python.org',
'http://gimp.org',#TheGNUimagemanipulation
program
]
defwalk(url_list):
forurlinurl_list:
f=urllib.urlopen(url)
stuff=f.read()
f.close()
yieldstuff
deftest():
forxinwalk(Urls):
print'length:%d'%(len(x),)
if__name__=='__main__':
test()

WhenIrunthis,Isee:
$pythongenerator_example.py
length:9554
length:16748
length:11487

Page222

APythonBook

3.7Objectorientedprogrammingandclasses
ClassesprovidePython'swaytodefinenewdatatypesandtodoOOP(objectoriented
programming).
Ifyouhavemadeitthisfar,youhavealreadyusedlotsofobjects.Youhavebeena
"consumer"ofobjectsandtheirservices.Now,youwilllearnhowtodefineand
implementnewkindsofobjects.Youwillbecomea"producer"ofobjects.Youwill
definenewclassesandyouwillimplementthecapabilities(methods)ofeachnewclass.
Aclassisdefinedwiththeclassstatement.Thefirstlineofaclassstatementisa
header(ithasacolonattheend),anditspecifiesthenameoftheclassbeingdefinedand
an(optional)superclass.Andthatheaderintroducesacompoundstatement:specifically,
thebodyoftheclassstatementwhichcontainsindented,nestedstatements,
importantly,defstatementsthatdefinethemethodsthatcanbecalledoninstancesofthe
objectsimplementedbythisclass.
Exercises:
1. Defineaclasswithonemethodshow.Thatmethodshouldprintout"Hello".
Then,createaninstanceofyourclass,andcalltheshowmethod.
Solutions:
1. Asimpleinstancemethodcanhavetheselfparameterandnoothers:
classDemo(object):
defshow(self):
print'hello'
deftest():
a=Demo()
a.show()
test()

Notes:
Noticethatweuseobjectasasuperclass,becausewewanttodefinean
"newstyle"classandbecausethereisnootherclassthatwewantasa
superclass.Seethefollowingformoreinformationonnewstyleclasses:
NewstyleClasseshttp://www.python.org/doc/newstyle/.
InPython,wecreateaninstanceofaclassbycallingtheclass,thatis,we
applythefunctioncalloperator(parentheses)totheclass.

3.7.1Theconstructor
Aclasscandefinemethodswithspecialnames.Youhaveseemsomeofthesebefore.
Thesenamesbeginandendwithadoubleunderscore.
Page223

APythonBook
Oneimportantspecialnameis__init__.It'stheconstructorforaclass.Itiscalled
eachtimeaninstanceoftheclassiscreated.Implementingthismethodinaclassgivesus
achancetoinitializeeachinstanceofourclass.
Exercises:
1. ImplementaclassnamedPlantthathasaconstructorwhichinitializestwo
instancevariables:nameandsize.Also,inthisclass,implementamethod
namedshowthatprintsoutthevaluesoftheseinstancevariables.Createseveral
instancesofyourclassand"show"them.
2. ImplementaclassnameNodethathastwoinstancevariables:dataand
children,wheredataisany,arbitraryobjectandchildrenisalistofchild
Nodes.Alsoimplementamethodnamedshowthatrecursivelydisplaysthe
nodesina"tree".Createaninstanceofyourclassthatcontainsseveralchild
instancesofyourclass.Calltheshowmethodontheroot(topmost)objectto
showthetree.
Solutions:
1. Theconstructorforaclassisamethodwiththespecialname__init__:
classPlant(object):
def__init__(self,name,size):
self.name=name
self.size=size
defshow(self):
print'name:"%s"size:%d'%(self.name,
self.size,)
deftest():
p1=Plant('Eggplant',25)
p2=Plant('Tomato',36)
plants=[p1,p2,]
forplantinplants:
plant.show()
test()

Notes:
Ourconstructortakestwoarguments:nameandsize.Itsavesthosetwo
valuesasinstancevariables,thatisinattributesoftheinstance.
Theshow()methodprintsoutthevalueofthosetwoinstancevariables.
2. Itisagoodideatoinitializeallinstancevariablesintheconstructor.Thatenables
someonereadingourcodetolearnaboutalltheinstancevariablesofaclassby
lookinginasinglelocation:
#simple_node.py
Indents=[''*nforninrange(10)]

Page224

APythonBook
classNode(object):
def__init__(self,name=None,children=None):
self.name=name
ifchildrenisNone:
self.children=[]
else:
self.children=children
defshow_name(self,indent):
print'%sname:"%s"'%(Indents[indent],
self.name,)
defshow(self,indent=0):
self.show_name(indent)
indent+=1
forchildinself.children:
child.show(indent)
deftest():
n1=Node('N1')
n2=Node('N2')
n3=Node('N3')
n4=Node('N4')
n5=Node('N5',[n1,n2,])
n6=Node('N6',[n3,n4,])
n7=Node('N7',[n5,n6,])
n7.show()
if__name__=='__main__':
test()

Notes:
Noticethatwedonotusetheconstructorforalist([])asadefaultvaluefor
thechildrenparameteroftheconstructor.Alistismutableandwouldbe
createdonlyonce(whentheclassstatementisexecuted)andwouldbeshared.

3.7.2InheritanceImplementingasubclass
Asubclassextendsorspecializesasuperclassbyaddingadditionalmethodstothe
superclassandbyoverridingmethods(withthesamename)thatalreadyexistinthe
superclass.
Exercises:
1. ExtendyourNodeexerciseabovebyaddingtwoadditionalsubclassesofthe
Nodeclass,onenamedPlantandtheothernamedAnimal.ThePlantclass
alsohasaheightinstancevariableandtheAnimalclassalsohasacolor
instancevariable.
Solutions:
1. WecanimportourpreviousNodescript,thenimplementclassesthathavethe
Nodeclassasasuperclass:
Page225

APythonBook
fromsimple_nodeimportNode,Indents
classPlant(Node):
def__init__(self,name,height=1,children=None):
Node.__init__(self,name,children)
self.height=height
defshow(self,indent=0):
self.show_name(indent)
print'%sheight:%s'%(Indents[indent],
self.height,)
indent+=1
forchildinself.children:
child.show(indent)
classAnimal(Node):
def__init__(self,name,color='nocolor',
children=None):
Node.__init__(self,name,children)
self.color=color
defshow(self,indent=0):
self.show_name(indent)
print'%scolor:"%s"'%(Indents[indent],
self.color,)
indent+=1
forchildinself.children:
child.show(indent)
deftest():
n1=Animal('scrubjay','grayblue')
n2=Animal('raven','black')
n3=Animal('americankestrel','brown')
n4=Animal('redshoulderedhawk','brownand
gray')
n5=Animal('corvid','none',[n1,n2,])
n6=Animal('raptor',children=[n3,n4,])
n7a=Animal('bird',children=[n5,n6,])
n1=Plant('valleyoak',50)
n2=Plant('canyonliveoak',40)
n3=Plant('jefferypine',120)
n4=Plant('ponderosapine',140)
n5=Plant('oak',children=[n1,n2,])
n6=Plant('conifer',children=[n3,n4,])
n7b=Plant('tree',children=[n5,n6,])
n8=Node('birdsandtrees',[n7a,n7b,])
n8.show()
if__name__=='__main__':
test()

Notes:
TheshowmethodinclassPlantcallstheshow_namemethodinits
superclassusingself.show_name(...).Pythonsearchesupthe
Page226

APythonBook

inheritancetreetofindtheshow_namemethodinclassNode.
Theconstructor(__init__)inclassesPlantandAnimaleachcallthe
constructorinthesuperclassbyusingthenameofthesuperclass.Whythe
difference?Because,if(inthePlantclass,forexample)itused
self.__init__(...)itwouldbecallingthe__init__inthePlant
class,itself.So,itbypassesitselfbyreferencingtheconstructorinthe
superclassdirectly.
Thisexercisealsodemonstrates"polymorphism"Theshowmethodis
calledanumberoftimes,butwhichimplementationexecutesdependson
whichinstanceitiscalledon.Callingontheshowmethodonaninstanceof
classPlantresultsinacalltoPlant.show.Callingtheshowmethodon
aninstanceofclassAnimalresultsinacalltoAnimal.show.Andsoon.It
isimportantthateachshowmethodtakesthecorrectnumberofarguments.

3.7.3Classesandpolymorphism
Pythonalsosupportsclassbasedpolymorphism,whichwas,bytheway,demonstratedin
thepreviousexample.
Exercises:
1. Writethreeclasses,eachofwhichimplementashow()methodthattakesone
argument,astring.Theshowmethodshouldprintoutthenameoftheclassand
themessage.Thencreatealistofinstancesandcalltheshow()methodoneach
objectinthelist.
Solution:
1. Weimplementthreesimpleclassesandthencreatealistofinstancesofthese
classes:
classA(object):
defshow(self,msg):
print'classAmsg:"%s"'%(msg,)
classB(object):
defshow(self,msg):
print'classBmsg:"%s"'%(msg,)
classC(object):
defshow(self,msg):
print'classCmsg:"%s"'%(msg,)
deftest():
objs=[A(),B(),C(),A(),]
foridx,objinenumerate(objs):
msg='message#%d'%(idx+1,)
obj.show(msg)

Page227

APythonBook
if__name__=='__main__':
test()

Notes:
Wecancalltheshow()methodinanyobjectinthelistobjsaslongaswe
passinasingleparameter,thatis,aslongasweobeytherequirementsof
ducktyping.Wecandothisbecauseallobjectsinthatlistimplementa
show()method.
Inastaticallytypedlanguage,thatisalanguagewherethetypeis(also)
presentinthevariable,alltheinstancesinexamplewouldhavetodescend
fromacommonsuperclassandthatsuperclasswouldhavetoimplementa
show()method.Pythondoesnotimposethisrestriction.And,because
variablesarenotnottypedinPython,perhapsthatwouldnotevenpossible.
Noticethatthisexampleofpolymorphismworkseventhoughthesethree
classes(A,B,andC)arenotrelated(forexample,inaclasshierarchy).All
thatisrequiredforpolymorphismtoworkinPythonisforthemethodnames
tobethesameandtheargumentstobecompatible.

3.7.4Recursivecallstomethods
Amethodinaclasscanrecusivelycallitself.Thisisverysimilartothewayinwhichwe
implementedrecursivefunctionssee:Recursivefunctions.
Exercises:
1. ReimplementthebinarytreeofanimalsandbirdsdescribedinRecursive
functions,butthistime,useaclasstorepresenteachnodeinthetree.
2. Solvethesameproblem,butthistimeimplementatreeinwhicheachnodecan
haveanynumberofchildren(ratherthanexactly2children).
Solutions:
1. Weimplementaclasswiththreeinstancevariables:(1)name,(2)leftbranch,and
(3)rightbranch.Then,weimplementashow()methodthatdisplaysthename
andcallsitselftoshowthechildrenineachsubtree:
Indents=[''*idxforidxinrange(10)]
classAnimalNode(object):
def__init__(self,name,left_branch=None,
right_branch=None):
self.name=name
self.left_branch=left_branch
self.right_branch=right_branch
defshow(self,level=0):
print'%sname:%s'%(Indents[level],

Page228

APythonBook
self.name,)
level+=1
ifself.left_branchisnotNone:
self.left_branch.show(level)
ifself.right_branchisnotNone:
self.right_branch.show(level)
Tree=AnimalNode('animals',
AnimalNode('birds',
AnimalNode('seedeaters',
AnimalNode('housefinch'),
AnimalNode('whitecrownedsparrow'),
),
AnimalNode('insecteaters',
AnimalNode('hermitthrush'),
AnimalNode('blackheadedphoebe'),
),
),
None,
)
deftest():
Tree.show()
if__name__=='__main__':
test()

2. Insteadofusingaleftbranchandarightbranch,inthissolutionweusealistto
representthechildrenofanode:
classAnimalNode(object):
def__init__(self,data,children=None):
self.data=data
ifchildrenisNone:
self.children=[]
else:
self.children=children
defshow(self,level=''):
print'%sdata:%s'%(level,self.data,)
level+=''
forchildinself.children:
child.show(level)
Tree=AnimalNode('animals',[
AnimalNode('birds',[
AnimalNode('seedeaters',[
AnimalNode('housefinch'),
AnimalNode('whitecrownedsparrow'),
AnimalNode('lessergoldfinch'),
]),
AnimalNode('insecteaters',[
AnimalNode('hermitthrush'),

Page229

APythonBook
AnimalNode('blackheadedphoebe'),
]),
])
])
deftest():
Tree.show()
if__name__=='__main__':
test()

Notes:
Werepresentthechildrenofanodeasalist.Eachnode"hasa"listof
children.
Noticethatbecausealistismutable,wedonotusealistconstructor([])in
theinitializerofthemethodheader.Instead,weuseNone,thenconstructan
emptylistinthebodyofthemethodifnecessary.SeesectionOptional
argumentsanddefaultvaluesformoreonthis.
We(recursively)calltheshowmethodforeachnodeinthechildrenlist.
Sinceanodewhichhasnochildren(aleafnode)willhaveanempty
childrenlist,thisprovidesalimitconditionforourrecursion.

3.7.5Classvariables,classmethods,andstaticmethods
Aclassvariableisonewhosesinglevalueissharedbyallinstancesoftheclassand,in
fact,issharedbyallwhohaveaccesstotheclass(object).
"Normal"methodsareinstancemethods.Aninstancemethodreceivestheinstanceasits
firstargument.Ainstancemethodisdefinedbyusingthedefstatementinthebodyofa
classstatement.
Aclassmethodreceivestheclassasitsfirstargument.Aclassmethodisdefinedby
defininganormal/instancemethod,thenusingtheclassmethodbuiltinfunction.For
example:
classASimpleClass(object):
description='asimpleclass'
defshow_class(cls,msg):
print'%s:%s'%(cls.description,msg,)
show_class=classmethod(show_class)

Astaticmethoddoesnotreceiveanythingspecialasitsfirstargument.Astaticmethodis
definedbydefininganormal/instancemethod,thenusingthestaticmethodbuiltin
function.Forexample:
classASimpleClass(object):
description='asimpleclass'
defshow_class(msg):

Page230

APythonBook
print'%s:%s'%(ASimpleClass.description,msg,)
show_class=staticmethod(show_class)

Ineffect,bothclassmethodsandstaticmethodsaredefinedbycreatinganormal
(instance)method,thencreatingawrapperobject(aclassmethodorstaticmethod)using
theclassmethodorstaticmethodbuiltinfunction.
Exercises:
1. Implementaclassthatkeepsarunningtotalofthenumberofinstancescreated.
2. Implementanothersolutiontothesameproblem(aclassthatkeepsarunning
totalofthenumberofinstances),butthistimeuseastaticmethodinsteadofa
classmethod.
Solutions:
1. Weuseaclassvariablenamedinstance_count,ratherthananinstance
variable,tokeeparunningtotalofinstances.Then,weincrementthatvariable
eachtimeaninstanceiscreated:
classCountInstances(object):
instance_count=0
def__init__(self,name='noname'):
self.name=name
CountInstances.instance_count+=1
defshow(self):
print'name:"%s"'%(self.name,)
defshow_instance_count(cls):
print'instancecount:%d'%
(cls.instance_count,)
show_instance_count=
classmethod(show_instance_count)
deftest():
instances=[]
instances.append(CountInstances('apple'))
instances.append(CountInstances('banana'))
instances.append(CountInstances('cherry'))
instances.append(CountInstances())
forinstanceininstances:
instance.show()
CountInstances.show_instance_count()
if__name__=='__main__':
test()

Notes:
Page231

APythonBook

Whenwerunthisscript,itprintsoutthefollowing:
name:"apple"
name:"banana"
name:"cherry"
name:"noname"
instancecount:4

Thecalltotheclassmethodbuiltinfunctioneffectivelywrapsthe
show_instance_countmethodinaclassmethod,thatis,inamethod
thattakesaclassobjectasitsfirstargumentratherthananinstanceobject.To
readmoreaboutclassmethod,gotoBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.htmlandsearchfor"classmethod".
2. Astaticmethodtakesneitheraninstance(self)noraclassasitsfirst
paramenter.And,staticmethodiscreatedwiththestaticmethod()builtin
function(ratherthanwiththeclassmethod()builtin):

classCountInstances(object):
instance_count=0
def__init__(self,name='noname'):
self.name=name
CountInstances.instance_count+=1
defshow(self):
print'name:"%s"'%(self.name,)
defshow_instance_count():
print'instancecount:%d'%(
CountInstances.instance_count,)
show_instance_count=
staticmethod(show_instance_count)
deftest():
instances=[]
instances.append(CountInstances('apple'))
instances.append(CountInstances('banana'))
instances.append(CountInstances('cherry'))
instances.append(CountInstances())
forinstanceininstances:
instance.show()
CountInstances.show_instance_count()
if__name__=='__main__':
test()

3.7.5.1Decoratorsforclassmethodandstaticmethod

Adecoratorenablesustodowhatwedidinthepreviousexamplewithasomewhat
simplersyntax.
Page232

APythonBook
Forsimplecases,thedecoratorsyntaxenablesustodothis:
@functionwrapper
defmethod1(self):
o
o
o

insteadofthis:
defmethod1(self):
o
o
o
method1=functionwrapper(method1)

So,wecanwritethis:
@classmethod
defmethod1(self):
o
o
o

insteadofthis:
defmethod1(self):
o
o
o
method1=classmethod(method1)

Exercises:
1. ImplementtheCountInstancesexampleabove,butuseadecoratorrather
thantheexplicitcalltoclassmethod.
Solutions:
1. Adecoratorisaneasierandcleanerwaytodefineaclassmethod(orastatic
method):
classCountInstances(object):
instance_count=0
def__init__(self,name='noname'):
self.name=name
CountInstances.instance_count+=1
defshow(self):
print'name:"%s"'%(self.name,)
@classmethod

Page233

APythonBook
defshow_instance_count(cls):
print'instancecount:%d'%
(cls.instance_count,)
#Notethatthefollowinglinehasbeenreplacedby
#theclassmethoddecorator,above.
#show_instance_count=
classmethod(show_instance_count)
deftest():
instances=[]
instances.append(CountInstances('apple'))
instances.append(CountInstances('banana'))
instances.append(CountInstances('cherry'))
instances.append(CountInstances())
forinstanceininstances:
instance.show()
CountInstances.show_instance_count()
if__name__=='__main__':
test()

3.8AdditionalandAdvancedTopics
3.8.1Decoratorsandhowtoimplementthem
Decoratorscanbeusedto"wrap"afunctionwithanotherfunction.
Whenimplementingadecorator,itishelpfultorememberthatthefollowingdecorator
application:
@dec
deffunc(arg1,arg2):
pass

isequivalentto:
deffunc(arg1,arg2):
pass
func=dec(func)

Therefore,toimplementadecorator,wewriteafunctionthatreturnsafunctionobject,
sincewereplacethevalueoriginallyboundtothefunctionwiththisnewfunctionobject.
Itmaybehelpfultotaketheviewthatwearecreatingafunctionthatisawrapperforthe
originalfunction.
Exercises:
1. Writeadecoratorthatwritesamessagebeforeandafterexecutingafunction.
Solutions:
Page234

APythonBook
1. Afunctionthatcontainsandreturnsaninnerfunctioncanbeusedtowrapa
function:
deftrace(func):
definner(*args,**kwargs):
print'>>'
func(*args,**kwargs)
print'<<'
returninner
@trace
deffunc1(x,y):
print'x:',x,'y:',y
func2((x,y))
@trace
deffunc2(content):
print'content:',content
deftest():
func1('aa','bb')
test()

Notes:
Yourinnerfunctioncanuse*argsand**kwargstoenableittocall
functionswithanynumberofarguments.
3.8.1.1Decoratorswitharguments

Decoratorscanalsotakearguments.
Thefollowingdecoratorwitharguments:
@dec(argA,argB)
deffunc(arg1,arg2):
pass

isequivalentto:
deffunc(arg1,arg2):
pass
func=dec(argA,argB)(func)

Becausethedecorator'sargumentsarepassedtotheresultofcallingthedecoratoronthe
decoratedfunction,youmayfinditusefultoimplementadecoratorwithargumentsusing
afunctioninsideafunctioninsideafunction.
Exercises:
1. Writeandtestadecoratorthattakesoneargument.Thedecoratorprintsa
messagealongwiththevalueoftheargumentbeforeandafterenteringthe
Page235

APythonBook
decoratedfunction.
Solutions:
1. Implementthisdecoratorthattakesargumentswithafunctioncontaininganested
functionwhichinturncontainsanestedfunction:
deftrace(msg):
definner1(func):
definner2(*args,**kwargs):
print'>>[%s]'%(msg,)
retval=func(*args,**kwargs)
print'<<[%s]'%(msg,)
returnretval
returninner2
returninner1
@trace('tracingfunc1')
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@trace('tracingfunc2')
deffunc2(content):
print'content:',content
returncontent*3
deftest():
result=func1('aa','bb')
print'result:',result
test()

3.8.1.2Stackeddecorators

Decoratorscanbe"stacked".
Thefollowingstackeddecorators:
@dec2
@dec1
deffunc(arg1,arg2,...):
pass

areequivalentto:
deffunc(arg1,arg2,...):
pass
func=dec2(dec1(func))

Exercises:
1. Implementadecorator(asabove)thattracescallstoadecoratedfunction.Then
Page236

APythonBook
"stack"thatwithanotherdecoratorthatprintsahorizontallineofdashesbefore
andaftercallingthefunction.
2. Modifyyoursolutiontotheaboveexercisesothatthedecoratorthatprintsthe
horizontallinetakesoneargument:acharacter(orcharacters)thatcanberepeated
toproduceahorizontalline/separator.
Solutions:
1. Reuseyourtracingfunctionfromthepreviousexercise,thenwriteasimple
decoratorthatprintsarowofdashes:
deftrace(msg):
definner1(func):
definner2(*args,**kwargs):
print'>>[%s]'%(msg,)
retval=func(*args,**kwargs)
print'<<[%s]'%(msg,)
returnretval
returninner2
returninner1
defhorizontal_line(func):
definner(*args,**kwargs):
print''*50
retval=func(*args,**kwargs)
print''*50
returnretval
returninner
@trace('tracingfunc1')
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@horizontal_line
@trace('tracingfunc2')
deffunc2(content):
print'content:',content
returncontent*3
deftest():
result=func1('aa','bb')
print'result:',result
test()

2. Onceagain,adecoratorwithargumentscanbeimplementedwithafunction
nestedinsideafunctionwhichisnestedinsideafunction.Thisremainsthesame
whetherthedecoratorisusedasastackeddecoratorornot.Hereisasolution:
deftrace(msg):

Page237

APythonBook
definner1(func):
definner2(*args,**kwargs):
print'>>[%s]'%(msg,)
retval=func(*args,**kwargs)
print'<<[%s]'%(msg,)
returnretval
returninner2
returninner1
defhorizontal_line(line_chr):
definner1(func):
definner2(*args,**kwargs):
printline_chr*15
retval=func(*args,**kwargs)
printline_chr*15
returnretval
returninner2
returninner1
@trace('tracingfunc1')
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@horizontal_line('<**>')
@trace('tracingfunc2')
deffunc2(content):
print'content:',content
returncontent*3
deftest():
result=func1('aa','bb')
print'result:',result
test()

3.8.1.3Morehelpwithdecorators

Thereismoreaboutdecoratorshere:

Pythonsyntaxandsemantics
http://en.wikipedia.org/wiki/Python_syntax_and_semantics#Decoratorsat
Wikipedia.
PythonDecoratorLibraryhttp://wiki.python.org/moin/PythonDecoratorLibrary
atthePythonWikihaslotsofsamplecode.
PEP318DecoratorsforFunctionsandMethods
http://www.python.org/dev/peps/pep0318/istheformalproposaland
specificationforPythondecorators.

Page238

APythonBook

3.8.2Iterables
3.8.2.1AfewpreliminariesonIterables

Definition:iterable(adjective)thatwhichcanbeiteratedover.
Agoodtestofwhethersomethingisiterableiswhetheritcanbeusedinafor:
statement.Forexample,ifwecanwriteforiteminX:,thenXisiterable.Hereis
anothersimpletest:
defisiterable(x):
try:
y=iter(x)
exceptTypeError,exp:
returnFalse
returnTrue

Somekindsofiterables:
ContainersWecaniterateoverlists,tuples,dictionaries,sets,strings,andother
containers.
Somebuiltin(noncontainer)typesExamples:
Atextfileopeninreadmodeisiterable:ititeratesoverthelinesinthefile.
ThexrangetypeSeeXRangeType
http://docs.python.org/lib/typesseqxrange.html.It'susefulwhenyouwanta
largesequenceofintegerstoiterateover.
Instancesofclassesthatobeytheiteratorprotocol.Foradescriptionoftheiterator
protocol,seeIteratorTypeshttp://docs.python.org/lib/typeiter.html.Hint:Type
dir(obj)andlookfor"__iter__"and"next".
GeneratorsAnobjectreturnedbyanyfunctionormethodthatcontainsyield.
Exercises:

1. Implementaclasswhoseinstancesareinterable.Theconstructortakesalistof
URLsasitsargument.Aninstanceofthisclass,wheniteratedover,generatesthe
contentoftheWebpageatthataddress.
Solutions:
1. Weimplementaclassthathas__iter__()andnext()methods:
importurllib
classWebPages(object):
def__init__(self,urls):
self.urls=urls
self.current_index=0
def__iter__(self):
self.current_index=0
returnself

Page239

APythonBook
defnext(self):
ifself.current_index>=len(self.urls):
raiseStopIteration
url=self.urls[self.current_index]
self.current_index+=1
f=urllib.urlopen(url)
content=f.read()
f.close()
returncontent
deftest():
urls=[
'http://www.python.org',
'http://en.wikipedia.org/',

'http://en.wikipedia.org/wiki/Python_(programming_langu
age)',
]
pages=WebPages(urls)
forpageinpages:
print'length:%d'%(len(page),)
pages=WebPages(urls)
print''*50
page=pages.next()
print'length:%d'%(len(page),)
page=pages.next()
print'length:%d'%(len(page),)
page=pages.next()
print'length:%d'%(len(page),)
page=pages.next()
print'length:%d'%(len(page),)
test()

3.8.2.2Morehelpwithiterables

TheitertoolsmoduleinthePythonstandardlibraryhashelpersforiterators:
http://docs.python.org/library/itertools.html#moduleitertools

3.9ApplicationsandRecipes
3.9.1XMLSAX,minidom,ElementTree,Lxml
Exercises:
1. SAXParseanXMLdocumentwithSAX,thenshowsomeinformation(tag,
attributes,characterdata)foreachelement.
2. MinidomParseanXMLdocumentwithminidom,thenwalktheDOMtree
andshowsomeinformation(tag,attributes,characterdata)foreachelement.
Page240

APythonBook
HereisasampleXMLdocumentthatyoucanuseforinput:
<?xmlversion="1.0"?>
<people>
<personid="1"value="abcd"ratio="3.2">
<name>Alberta</name>
<interest>gardening</interest>
<interest>reading</interest>
<category>5</category>
</person>
<personid="2">
<name>Bernardo</name>
<interest>programming</interest>
<category></category>
<agent>
<firstname>Darren</firstname>
<lastname>Diddly</lastname>
</agent>
</person>
<personid="3"value="efgh">
<name>Charlie</name>
<interest>people</interest>
<interest>cats</interest>
<interest>dogs</interest>
<category>8</category>
<promoter>
<firstname>David</firstname>
<lastname>Donaldson</lastname>
<client>
<fullname>ArnoldApplebee</fullname>
<refid>10001</refid>
</client>
</promoter>
<promoter>
<firstname>Edward</firstname>
<lastname>Eddleberry</lastname>
<client>
<fullname>ArnoldApplebee</fullname>
<refid>10001</refid>
</client>
</promoter>
</person>
</people>

3. ElementTreeParseanXMLdocumentwithElementTree,thenwalktheDOM
treeandshowsomeinformation(tag,attributes,characterdata)foreachelement.
4. lxmlParseanXMLdocumentwithlxml,thenwalktheDOMtreeandshow
someinformation(tag,attributes,characterdata)foreachelement.
5. ModifydocumentwithElementTreeUseElementTreetoreadadocument,then
modifythetree.Showthecontentsofthetree,andthenwriteoutthemodified
document.
6. XPathlxmlsupportsXPath.UsetheXPathsupportinlxmltoaddresseachof
Page241

APythonBook
thefollowingintheaboveXMLinstancedocument:
Thetextinallthenameelements
Thevaluesofalltheidattributes
Solutions:
1. WecanusetheSAXsupportinthePythonstandardlibrary:
#!/usr/bin/envpython
"""
ParseandXMLwithSAX.Displayinfoabouteach
element.
Usage:
pythontest_sax.pyinfilename
Examples:
pythontest_sax.pypeople.xml
"""
importsys
fromxml.saximportmake_parser,handler
classTestHandler(handler.ContentHandler):
def__init__(self):
self.level=0
defshow_with_level(self,value):
print'%s%s'%(''*self.level,value,)
defstartDocument(self):
self.show_with_level('Documentstart')
self.level+=1
defendDocument(self):
self.level=1
self.show_with_level('Documentend')
defstartElement(self,name,attrs):
self.show_with_level('startelementname:
"%s"'%(name,))
self.level+=1
defendElement(self,name):
self.level=1
self.show_with_level('endelementname:
"%s"'%(name,))
defcharacters(self,content):
content=content.strip()
ifcontent:
self.show_with_level('characters:"%s"'%
(content,))

Page242

APythonBook
deftest(infilename):
parser=make_parser()
handler=TestHandler()
parser.setContentHandler(handler)
parser.parse(infilename)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
test(infilename)
if__name__=='__main__':
main()

2. Theminidommodulecontainsaparse()functionthatenablesustoreadan
XMLdocumentandcreateaDOMtree:
#!/usr/bin/envpython
"""ProcessanXMLdocumentwithminidom.
Showthedocumenttree.
Usage:
pythonminidom_walk.py[options]infilename
"""
importsys
fromxml.domimportminidom
defshow_tree(doc):
root=doc.documentElement
show_node(root,0)
defshow_node(node,level):
count=0
ifnode.nodeType==minidom.Node.ELEMENT_NODE:
show_level(level)
print'tag:%s'%(node.nodeName,)
forkeyinnode.attributes.keys():
attr=node.attributes.get(key)
show_level(level+1)
print'attributename:%svalue:"%s"'%
(attr.name,
attr.value,)
if(len(node.childNodes)==1and
node.childNodes[0].nodeType==

Page243

APythonBook
minidom.Node.TEXT_NODE):
show_level(level+1)
print'data:"%s"'%
(node.childNodes[0].data,)
forchildinnode.childNodes:
count+=1
show_node(child,level+1)
returncount
defshow_level(level):
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=minidom.parse(docname)
show_tree(doc)
if__name__=='__main__':
#importpdb;pdb.set_trace()
test()

3. ElementTreeenablesustoparseanXMLdocumentandcreateaDOMtree:
#!/usr/bin/envpython
"""ProcessanXMLdocumentwithelementtree.
Showthedocumenttree.
Usage:
pythonelementtree_walk.py[options]infilename
"""
importsys
fromxml.etreeimportElementTreeasetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
defshow_node(node,level):
show_level(level)
print'tag:%s'%(node.tag,)
forkey,valueinnode.attrib.iteritems():
show_level(level+1)
print'attributename:%svalue:"%s"'%
(key,value,)
ifnode.text:
text=node.text.strip()

Page244

APythonBook
show_level(level+1)
print'text:"%s"'%(node.text,)
ifnode.tail:
tail=node.tail.strip()
show_level(level+1)
print'tail:"%s"'%(tail,)
forchildinnode.getchildren():
show_node(child,level+1)
defshow_level(level):
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=etree.parse(docname)
show_tree(doc)
if__name__=='__main__':
#importpdb;pdb.set_trace()
test()

4. lxmlenablesustoparseanXMLdocumentandcreateaDOMtree.Infact,since
lxmlattemptstomimictheElementTreeAPI,ourcodeisverysimilartothatin
thesolutiontotheElementTreeexercise:
#!/usr/bin/envpython
"""ProcessanXMLdocumentwithelementtree.
Showthedocumenttree.
Usage:
pythonlxml_walk.py[options]infilename
"""
#
#Imports:
importsys
fromlxmlimportetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
defshow_node(node,level):
show_level(level)
print'tag:%s'%(node.tag,)
forkey,valueinnode.attrib.iteritems():

Page245

APythonBook
show_level(level+1)
print'attributename:%svalue:"%s"'%
(key,value,)
ifnode.text:
text=node.text.strip()
show_level(level+1)
print'text:"%s"'%(node.text,)
ifnode.tail:
tail=node.tail.strip()
show_level(level+1)
print'tail:"%s"'%(tail,)
forchildinnode.getchildren():
show_node(child,level+1)
defshow_level(level):
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=etree.parse(docname)
show_tree(doc)
if__name__=='__main__':
#importpdb;pdb.set_trace()
test()

5. WecanmodifytheDOMtreeandwriteitouttoanewfile:
#!/usr/bin/envpython
"""ProcessanXMLdocumentwithelementtree.
Showthedocumenttree.
Modifythedocumenttreeandthenshowitagain.
WritethemodifiedXMLtreetoanewfile.
Usage:
pythonelementtree_walk.py[options]infilename
outfilename
Options:
h,helpDisplaythishelpmessage.
Example:
pythonelementtree_walk.pymyxmldoc.xml
myotherxmldoc.xml
"""
importsys
importos
importgetopt

Page246

APythonBook
importtime
#UseElementTree.
fromxml.etreeimportElementTreeasetree
#OruncommenttouseLxml.
#fromlxmlimportetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
defshow_node(node,level):
show_level(level)
print'tag:%s'%(node.tag,)
forkey,valueinnode.attrib.iteritems():
show_level(level+1)
print'attributename:%svalue:"%s"'%
(key,value,)
ifnode.text:
text=node.text.strip()
show_level(level+1)
print'text:"%s"'%(node.text,)
ifnode.tail:
tail=node.tail.strip()
show_level(level+1)
print'tail:"%s"'%(tail,)
forchildinnode.getchildren():
show_node(child,level+1)
defshow_level(level):
forxinrange(level):
print'',
defmodify_tree(doc,tag,attrname,attrvalue):
root=doc.getroot()
modify_node(root,tag,attrname,attrvalue)
defmodify_node(node,tag,attrname,attrvalue):
ifnode.tag==tag:
node.attrib[attrname]=attrvalue
forchildinnode.getchildren():
modify_node(child,tag,attrname,attrvalue)
deftest(indocname,outdocname):
doc=etree.parse(indocname)
show_tree(doc)
print''*50
date=time.ctime()
modify_tree(doc,'person','date',date)
show_tree(doc)
write_output=False
ifos.path.exists(outdocname):
response=raw_input('Outputfile(%s)exists.

Page247

APythonBook
Overwrite?(y/n):'%
outdocname)
ifresponse=='y':
write_output=True
else:
write_output=True
ifwrite_output:
doc.write(outdocname)
print'WrotemodifiedXMLtreeto%s'%
outdocname
else:
print'Didnotwriteoutputfile.'
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help',
])
except:
usage()
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=2:
usage()
indocname=args[0]
outdocname=args[1]
test(indocname,outdocname)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Notes:
TheabovesolutioncontainsanimportstatementforElementTreeand
anotherforlxml.Theoneforlxmliscommentedout,butyoucouldchange
thatifyouwishtouselxmlinsteadofElementTree.Thissolutionwillwork
thesamewaywitheitherElementTreeorlxml.
6. WhenweparseandXMLdocumentwithlxml,eachelement(node)hasan
xpath()method.
#test_xpath.py
fromlxmlimportetree
deftest():
doc=etree.parse('people.xml')
root=doc.getroot()

Page248

APythonBook
printroot.xpath("//name/text()")
printroot.xpath("//@id")
test()

And,whenweruntheabovecode,hereiswhatwesee:
$pythontest_xpath.py
['Alberta','Bernardo','Charlie']
['1','2','3']

FormoreonXPathsee:XMLPathLanguage(XPath)
http://www.w3.org/TR/xpath

3.9.2Relationaldatabaseaccess
YoucanfindinformationaboutdatabaseprogramminginPythonhere:Database
Programminghttp://wiki.python.org/moin/DatabaseProgramming/.
FordatabaseaccessweusethePythonDatabaseAPI.Youcanfindinformationaboutit
here:PythonDatabaseAPISpecificationv2.0
http://www.python.org/dev/peps/pep0249/.
TousethedatabaseAPIwedothefollowing:
1. Usethedatabaseinterfacemoduletocreateaconnectionobject.
2. Usetheconnectionobjecttocreateacursorobject.
3. UsethecursorobjecttoexecuteanSQLquery.
4. Retrieverowsfromthecursorobject,ifneeded.
5. Optionally,commitresultstothedatabase.
6. Closetheconnectionobject.
Ourexamplesusethegadflydatabase,whichiswritteninPython.Ifyouwanttouse
gadfly,youcanfindithere:http://gadfly.sourceforge.net/.gadflyisareasonable
choiceifyouwantaneasytousedatabaseonyourlocalmachine.
Anotherreasonablechoiceforalocaldatabaseissqlite3,whichisinthePython
standardlibrary.HereisadescriptivequotefromtheSQLiteWebsite:
"SQLiteisasoftwarelibrarythatimplementsaselfcontained,
serverless,zeroconfiguration,transactionalSQLdatabaseengine.
SQLiteisthemostwidelydeployedSQLdatabaseengineintheworld.
ThesourcecodeforSQLiteisinthepublicdomain."
Youcanlearnaboutithere:

sqlite3DBAPI2.0interfaceforSQLitedatabases
http://docs.python.org/library/sqlite3.html
SQLitehomepagehttp://www.sqlite.org/
Page249

APythonBook
Thepysqlitewebpagehttp://oss.itsystementwicklung.de/trac/pysqlite/
Ifyouwantorneedtouseanother,enterpriseclassdatabase,forexamplePostgreSQL,
MySQL,Oracle,etc.,youwillneedaninterfacemoduleforyourspecificdatabase.You
canfindinformationaboutdatabaseinterfacemoduleshere:Databaseinterfaces
http://wiki.python.org/moin/DatabaseInterfaces

Excercises:
1. Writeascriptthatretrievesalltherowsinatableandprintseachrow.
2. Writeascriptthatretrievesalltherowsinatable,thenusesthecursorasan
iteratortoprinteachrow.
3. Writeascriptthatusesthecursor'sdescriptionattributetoprintoutthename
andvalueofeachfieldineachrow.
4. Writeascriptthatperformsseveraloftheabovetasks,butusessqlite3instead
ofgadfly.
Solutions:
1. WecanexecuteaSQLqueryandthenretrievealltherowswith
fetchall():
importgadfly
deftest():
connection=gadfly.connect("dbtest1",
"plantsdbdir")
cur=connection.cursor()
cur.execute('select*fromplantsdborderby
p_name')
rows=cur.fetchall()
forrowinrows:
print'2.row:',row
connection.close()
test()

2. Thecursoritselfisaniterator.Ititeratesovertherowsreturnedbyaquery.So,
weexecuteaSQLqueryandthenweusethecursorinafor:statement:
importgadfly
deftest():
connection=gadfly.connect("dbtest1",
"plantsdbdir")
cur=connection.cursor()
cur.execute('select*fromplantsdborderby
p_name')
forrowincur:
printrow
connection.close()

Page250

APythonBook
test()

3. Thedescriptionattributeinthecursorisacontainerthathasanitemdescribing
eachfield:
importgadfly
deftest():
cur.execute('select*fromplantsdborderby
p_name')
forfieldincur.description:
print'field:',field
rows=cur.fetchall()
forrowinrows:
foridx,fieldinenumerate(row):
content='%s:"%s"'%
(cur.description[idx][0],field,)
printcontent,
print
connection.close()
test()

Notes:
ThecommaattheendoftheprintstatementtellsPythonnottoprinta
newline.
Thecur.descriptionisasequencecontaininganitemforeachfield.
Afterthequery,wecanextractadescriptionofeachfield.
4. Thesolutionsusingsqlite3areverysimilartothoseusinggadfly.For
informationonsqlite3,see:sqlite3DBAPI2.0interfaceforSQLite
databaseshttp://docs.python.org/library/sqlite3.html#modulesqlite3.
#!/usr/bin/envpython
"""
Performoperationsonsqlite3(plants)database.
Usage:
pythonpy_db_api.pycommand[arg1,...]
Commands:
createcreatenewdatabase.
showshowcontentsofdatabase.
addaddrowtodatabase.Requires3args(name,
descrip,rating).
deleteremoverowfromdatabase.Requires1arg
(name).
Examples:
pythontest1.pycreate
pythontest1.pyshow
pythontest1.pyaddcrenshaw"Themostsucculent
melon"10
pythontest1.pydeletelemon

Page251

APythonBook
"""
importsys
importsqlite3
Values=[
('lemon','brightandyellow','7'),
('peach','succulent','9'),
('banana','smoothandcreamy','8'),
('nectarine','tangyandtasty','9'),
('orange','sweetandtangy','8'),
]
Field_defs=[
'p_namevarchar',
'p_descripvarchar',
#'p_ratinginteger',
'p_ratingvarchar',
]
defcreatedb():
connection=sqlite3.connect('sqlite3plantsdb')
cursor=connection.cursor()
q1="createtableplantsdb(%s)"%(',
'.join(Field_defs))
print'createq1:%s'%q1
cursor.execute(q1)
q1="createindexindex1onplantsdb(p_name)"
cursor.execute(q1)
q1="insertintoplantsdb(p_name,p_descrip,
p_rating)values('%s','%s',%s)"
forspecinValues:
q2=q1%spec
print'q2:"%s"'%q2
cursor.execute(q2)
connection.commit()
showdb1(cursor)
connection.close()
defshowdb():
connection,cursor=opendb()
showdb1(cursor)
connection.close()
defshowdb1(cursor):
cursor.execute("select*fromplantsdborderby
p_name")
hr()
description=cursor.description

Page252

APythonBook
printdescription
print'description:'
forrowdescriptionindescription:
print'%s'%(rowdescription,)
hr()
rows=cursor.fetchall()
printrows
print'rows:'
forrowinrows:
print'%s'%(row,)
hr()
print'content:'
forrowinrows:
descrip=row[1]
name=row[0]
rating='%s'%row[2]
print'%s%s%s'%(
name.ljust(12),descrip.ljust(30),
rating.rjust(4),)
defaddtodb(name,descrip,rating):
try:
rating=int(rating)
exceptValueError,exp:
print'Error:ratingmustbeinteger.'
return
connection,cursor=opendb()
cursor.execute("select*fromplantsdbwherep_name
='%s'"%name)
rows=cursor.fetchall()
iflen(rows)>0:
ql="updateplantsdbsetp_descrip='%s',
p_rating='%s'wherep_name='%s'"%(
descrip,rating,name,)
print'ql:',ql
cursor.execute(ql)
connection.commit()
print'Updated'
else:
cursor.execute("insertintoplantsdbvalues
('%s','%s','%s')"%(
name,descrip,rating))
connection.commit()
print'Added'
showdb1(cursor)
connection.close()
defdeletefromdb(name):
connection,cursor=opendb()
cursor.execute("select*fromplantsdbwherep_name
='%s'"%name)

Page253

APythonBook
rows=cursor.fetchall()
iflen(rows)>0:
cursor.execute("deletefromplantsdbwhere
p_name='%s'"%name)
connection.commit()
print'Plant(%s)deleted.'%name
else:
print'Plant(%s)doesnotexist.'%name
showdb1(cursor)
connection.close()
defopendb():
connection=sqlite3.connect("sqlite3plantsdb")
cursor=connection.cursor()
returnconnection,cursor
defhr():
print''*60
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)<1:
usage()
cmd=args[0]
ifcmd=='create':
iflen(args)!=1:
usage()
createdb()
elifcmd=='show':
iflen(args)!=1:
usage()
showdb()
elifcmd=='add':
iflen(args)<4:
usage()
name=args[1]
descrip=args[2]
rating=args[3]
addtodb(name,descrip,rating)
elifcmd=='delete':
iflen(args)<2:
usage()
name=args[1]
deletefromdb(name)
else:

Page254

APythonBook
usage()
if__name__=='__main__':
main()

3.9.3CSVcommaseparatedvaluefiles
ThereissupportforparsingandgeneratingCSVfilesinthePythonstandardlibrary.See:
csvCSVFileReadingandWriting
http://docs.python.org/library/csv.html#modulecsv.
Exercises:
1. ReadaCSVfileandprintthefieldsincolumns.Hereisasamplefiletouseas
input:
#namedescriptionrating
Lemon,Brightyellowandtart,5
Eggplant,Purpleandshiny,6
Tangerine,Succulent,8

Solutions:
1. UsetheCSVmoduleinthePythonstandardlibrarytoreadaCSVfile:
"""
ReadaCSVfileandprintthecontentsincolumns.
"""
importcsv
deftest(infilename):
infile=open(infilename)
reader=csv.reader(infile)
print'===============
======'
print'NameDescription
Rating'
print'===============
======'
forfieldsinreader:
iflen(fields)==3:
line='%s%s%s'%(fields[0].ljust(20),
fields[1].ljust(40),
fields[2].ljust(4))
printline
infile.close()
defmain():
infilename='csv_report.csv'
test(infilename)

Page255

APythonBook
if__name__=='__main__':
main()

And,whenrun,hereiswhatitdisplays:
===============
======
NameDescription
Rating
===============
======
LemonBrightyellowandtart
5
EggplantPurpleandshiny
6
TangerineSucculent
8

3.9.4YAMLandPyYAML
YAMLisastructuredtextdatarepresentationformat.Itusesindentationtoindicate
nesting.HereisadescriptionfromtheYAMLWebsite:
"YAML:YAMLAin'tMarkupLanguage
"WhatItIs:YAMLisahumanfriendlydataserializationstandardfor
allprogramminglanguages."
YoucanlearnmoreaboutYAMLandPyYAMLhere:
TheOfficialYAMLWebSitehttp://yaml.org/
PyYAML.orgthehomeofvariousYAMLimplementationsforPython
http://pyyaml.org/
TheYAML1.2specificationhttp://yaml.org/spec/1.2/
Exercises:

1. ReadthefollowingsampleYAMLdocument.Printouttheinformationinit:
american:
BostonRedSox
DetroitTigers
NewYorkYankees
national:
NewYorkMets
ChicagoCubs
AtlantaBraves

2. LoadtheYAMLdatausedinthepreviousexercise,thenmakeamodification(for
example,add"SanFranciscoGiants"totheNationalLeague),thendumpthe
modifieddatatoanewfile.
Solutions:
Page256

APythonBook
1. PrintingoutinformationfromYAMLisas"simple"asprintingoutaPythondata
structure.Inthissolution,weusetheprettyprinterfromthePythonstandard
library:
importyaml
importpprint
deftest():
infile=open('test1.yaml')
data=yaml.load(infile)
infile.close()
pprint.pprint(data)
test()

Wecould,alternatively,readinandthen"load"fromastring:
importyaml
importpprint
deftest():
infile=open('test1.yaml')
data_str=infile.read()
infile.close()
data=yaml.load(data_str)
pprint.pprint(data)
test()

2. TheYAMLdump()functionenablesustodumpdatatoafile:
importyaml
importpprint
deftest():
infile=open('test1.yaml','r')
data=yaml.load(infile)
infile.close()
data['national'].append('SanFranciscoGiants')
outfile=open('test1_new.yaml','w')
yaml.dump(data,outfile)
outfile.close()
test()

Notes:
IfwewanttoproducethestandardYAML"block"styleratherthanthe"flow"
format,thenwecoulduse:
yaml.dump(data,outfile,default_flow_style=False)

Page257

APythonBook

3.9.5Json
HereisaquotefromWikipediaentryforJson:
"JSON(pronounced'Jason'),shortforJavaScriptObjectNotation,isa
lightweightcomputerdatainterchangeformat.Itisatextbased,
humanreadableformatforrepresentingsimpledatastructuresand
associativearrays(calledobjects)."
TheJsontextrepresentationlooksverysimilartoPythonliteralrepresentationofPython
builtindatatypes(forexample,lists,dictionaries,numbers,andstrings).
LearnmoreaboutJsonandPythonsupportforJsonhere:
IntroducingJSONhttp://json.org/
JsonatWikipediahttp://en.wikipedia.org/wiki/Json
pythonjsonhttp://pypi.python.org/pypi/pythonjson
simplejsonhttp://pypi.python.org/pypi/simplejson
Excercises:

1. WriteaPythonscript,usingyourfavoritePythonJsonimplementation(for
examplepythonjsonorsimplejson),thatdumpsthefollowingdata
structuretoafile:
Data={
'rockandroll':
['Elis','TheBeatles','TheRollingStones',],
'country':
['WillieNelson','HankWilliams',]
}

2. WriteaPythonscriptthatreadsJsondatafromafileandloadsitintoPythondata
structures.
Solutions:
1. ThissolutionusessimplejsontostoreaPythondatastructureencodedasJson
inafile:
importsimplejsonasjson
Data={
'rockandroll':
['Elis','TheBeatles','TheRollingStones',],
'country':
['WillieNelson','HankWilliams',]
}
deftest():
fout=open('tmpdata.json','w')
content=json.dumps(Data)
fout.write(content)

Page258

APythonBook
fout.write('\n')
fout.close()
test()

2. Wecanreadthefileintoastring,thendecodeitfromJson:
importsimplejsonasjson
deftest():
fin=open('tmpdata.json','r')
content=fin.read()
fin.close()
data=json.loads(content)
printdata
test()

Notethatyoumaywantsomecontroloverindentation,characterencoding,etc.For
simplejson,youcanlearnaboutthathere:simplejsonJSONencoderanddecoder
http://simplejson.googlecode.com/svn/tags/simplejson2.0.1/docs/index.html.

Page259

APythonBook

4Part4GeneratingPythonBindingsforXML
ThissectiondiscussesaspecificPythontool,specificallyaPythoncodegeneratorthat
generatesPythonbindingsforXMLfiles.
Thus,thissectionwillhelpyouinthefollowingways:
1. Itwillhelpyoulearntouseaspecifictool,namelygenerateDS.py,that
generatesPythoncodetobeusedtoprocessXMLinstancedocumentsofa
particulardocumenttype.
2. Itwillhelpyougainmoreexperiencewithreading,modifyingandusingPython
code.

4.1Introduction
Additionalinformation:

Ifyouplantoworkthroughthistutorial,youmayfindithelpfultolookatthe
samplecodethataccompaniesthistutorial.Youcanfinditinthedistribution
under:
tutorial/
tutorial/Code/

YoucanfindadditionalinformationaboutgenerateDS.pyhere:
http://http://www.davekuhlman.org/#generatedspy

Thatdocumentationisalsoincludedinthedistribution.
generateDS.pygeneratesPythondatastructures(forexample,classdefinitions)from
anXMLschemadocument.ThesedatastructuresrepresenttheelementsinanXML
documentdescribedbytheXMLschema.generateDS.pyalsogeneratesparsersthat
loadanXMLdocumentintothosedatastructures.Inaddition,aseparatefilecontaining
subclasses(stubs)isoptionallygenerated.Theusercanaddmethodstothesubclassesin
ordertoprocessthecontentsofanXMLdocument.
ThegeneratedPythoncodecontains:

AclassdefinitionforeachelementdefinedintheXMLschemadocument.
Amainanddriverfunctionthatcanbeusedtotestthegeneratedcode.
AparserthatwillreadanXMLdocumentwhichsatisfiestheXMLschemafrom
whichtheparserwasgenerated.Theparsercreatesandpopulatesatreestructure
ofinstancesofthegeneratedPythonclasses.
MethodsineachclasstoexporttheinstancebackouttoXML(methodexport)
andtoexporttheinstancetoaliteralrepresentingthePythondatastructure
Page260

APythonBook
(methodexportLiteral).
Eachgeneratedclasscontainsthefollowing:
Aconstructormethod(__init__),withmembervariableinitializers.
Methodswithnamesget_xyzandset_xyzforeachmembervariable"xyz"
or,ifthemembervariableisdefinedwithmaxOccurs="unbounded",
methodswithnamesget_xyz,set_xyz,add_xyz,andinsert_xyz.
(Note:Ifyouusetheuseoldgettersetter,thenyouwillget
methodswithnameslikegetXyzandsetXyz.)
Abuildmethodthatcanbeusedtopopulateaninstanceoftheclassfroma
nodeinanElementTreeorLxmltree.
Anexportmethodthatwillwritetheinstance(andanynestedsubinstances)to
afileobjectasXMLtext.
AnexportLiteralmethodthatwillwritetheinstance(andanynested
subinstances)toafileobjectasPythonliterals(text).
Thegeneratedsubclassfilecontainsone(sub)classdefinitionforeachdata
representationclass.Ifthesubclassfileisused,thentheparsercreatesinstancesofthe
subclasses(insteadofcreatinginstancesofthesuperclasses).Thisenablestheuserto
extendthesubclasseswith"treewalk"methods,forexample,thatprocessthecontentsof
theXMLfile.Theusercanalsogenerateandextendmultiplesubclassfileswhichusea
single,commonsuperclassfile,thusimplementinganumberofdifferentprocessesonthe
sameXMLdocumenttype.

ThisdocumentintroducestheusertogenerateDS.pyandwalkstheuserthrough
severalexamplesthatshowhowtogeneratePythoncodeandhowtousethatgenerated
code.

4.2Generatingthecode
Note:Thesamplefilesusedbelowareunderthetutorial/Code/directory.
Usethefollowingtogethelp:
$generateDS.pyhelp

I'llassumethatgenerateDS.pyisinadirectoryonyourpath.Ifnot,youshoulddo
whateverisnecessarytomakeitaccessibleandexecutable.
HereisasimpleXMLschemadocument:
And,hereishowyoumightgenerateclassesandsubclassesthatprovidedatabindings(a
PythonAPI)forthedefinitionsinthatschema:
$generateDS.pyopeople_api.pyspeople_sub.pypeople.xsd

Page261

APythonBook
And,ifyouwanttoautomaticallyoverwritethegeneratedPythonfiles,usethef
commandlineflagtoforceoverwritewithoutasking:
$generateDS.pyfopeople_api.pyspeople_sub.pypeople.xsd

And,tohardwirethesubclassfilesothatitimportstheAPImodule,usethesuper
commandlinefile.Example:
$generateDS.pyopeople_api.pypeople.xsd
$generateDS.pyspeople_appl1.pysuper=people_apipeople.xsd

Or,dobothatthesametimewiththefollowing:
$generateDS.pyopeople_api.pyspeople_appl1.py
super=people_apipeople.xsd

And,foryoursecondapplication:
$generateDS.pyspeople_appl2.pysuper=people_apipeople.xsd

Ifyoutakealookinsidethesetwo"application"files,youwillseeandimportstatement
likethefollowing:
import???assupermod

Ifyouhadnotusedthesupercommandlineoptionwhengeneratingthe
"application"files,thenyoucouldmodifythatstatementyourself.Thesuper
commandlineoptiondoesthisforyou.
YoucanalsousetheThegraphicalfrontendtoconfigureoptionsandsavethemina
sessionfile,thenusethatsessionfilewithgenerateDS.pytospecifyyourcommand
lineoptions.Forexample:
$generateDS.pysession=test01.session

Youcantestthegeneratedcodebyrunningit.Trysomethinglikethefollowing:
$pythonpeople_api.pypeople.xml

or:
$pythonpeople_appl1.pypeople.xml

Whydoesthiswork?WhycanwerunthegeneratedcodeasaPythonscript?Ifyou
lookatthegeneratedcode,downneartheendofthefileyou'llfindamain()function
thatcallsafunctionnamedparse().Theparsefunctiondoesthefollowing:
1. ParsesyourXMLinstancedocument.
2. UsesyourgeneratedAPItobuildatreeofinstancesofthegeneratedclasses.
3. Usestheexport()methodsinthattreeofinstancestoprintout(export)XML
Page262

APythonBook
thatrepresentsyourgeneratedtreeofinstances.
Exceptforsomeindentation(ignorablewhitespace),thisexportedXMLshouldbethe
sameastheoriginalXMLdocument.So,thatgivesyouareasonablythoroughtestof
yourgeneratedcode.
And,thecodeinthatparse()functiongivesyouahintofhowyoumightbuildyour
ownapplicationspecificcodethatusesthegeneratedAPI(thosegeneratedPython
classes).

4.3UsingthegeneratedcodetoparseandexportanXMLdocument
Nowthatyouhavegeneratedcodeforyourdatamodel,youcantestitbyrunningitasan
application.SupposethatyouhaveanXMLinstancedocumentpeople1.xmlthat
satisfiesyourschema.Thenyoucanparsethatinstancedocumentandexportit(printit
out)withsomethinglikethefollowing:
$pythonpeople_api.pypeople1.xml

And,ifyouhaveusedthesupercommandlineoption,asIhaveabove,toconnect
yoursubclassfilewiththesuperclass(API)file,thenyoucouldusethefollowingtodo
thesamething:
$pythonpeople_appl1.pypeople1.xml

4.4Somecommandlineoptionsyoumightwanttoknow
Youmaywanttomerelyskimthissectionfornow,thenlaterreferbacktoitwhensome
oftheseoptionsareareusedlaterinthistutorial.Also,rememberthatyoucanget
informationaboutmorecommandlineoptionsusedbygenerateDS.pybytyping:
$pythongenerateDS.pyhelp

andbyreadingthedocumentathttp://www.davekuhlman.org/#generatedspy
o
Generatethesuperclassmodule.Thisisthemodulethatcontainstheimplementation
ofeachclassforeachelementtype.So,youcanthinkofthisastheimplementationof
the"databindings"ortheAPIforXMLdocumentsofthetypedefinedbyyourXML
schema.
s
Generatethesubclassmodule.Youmightormightnotneedthese.Ifyouintendto
writesomeapplicationspecificcode,youmightwanttoconsiderstartingwiththese
skeletonclassesandaddyourapplicationcodethere.
Page263

APythonBook
super
Thisoptioninsertsthenameofthesuperclassmoduleintoanimportstatementin
thesubclassfile(generatedwith"s").Ifyouknowthenameofthesuperclassfilein
advance,youcanusethisoptiontoenablethesubclassfiletoimportthesuperclass
moduleautomatically.Ifyoudonotusethisoption,youwillneedtoeditthesubclass
modulewithyourtexteditorandmodifytheimportstatementnearthetop.
rootelement="elementname"
UsethisoptiontotellgenerateDS.pywhichoftheelementsdefinedinyourXM
schemaisthe"root"element.Therootelementistheoutermost(toplevel)element
inXMLinstancedocumentsdefinedbythisschema.Ineffect,thistellsyour
generatedmoduleswhichelementtouseastherootelementwhenparsingand
exportingdocuments.
generateDS.pyattemptstoguesstherootelement,usuallythefirstelement
definedinyourXMLschema.Usethisoptionwhenthatdefaultisnotwhatyouwant.
memberspecs=list|dict
Supposeyouwanttowritesomecodethatcanbegenericallyappliedtoelementsof
differentkinds(elementtypesimplementedbyseveraldifferentgeneratedclasses.If
so,itmightbehelpfultohavealistordictionaryspecifyinginformationabouteach
memberdataitemineachclass.Thisoptiondoesthatbygeneratingalistora
dictionary(withthememberdataitemnameaskey)ineachgeneratedclass.Takea
lookatthegeneratedcodetolearnaboutit.Inparticular,lookatthegeneratedlistor
dictionaryinaclassforanyelementtypeandalsoatthedefinitionoftheclass
_MemberSpecgeneratednearthetopoftheAPImodule.
version
AskgenerateDS.pytotellyouwhatversionitis.Thisishelpfulwhenyouwant
toaskaboutaproblem,forexampleatthegeneratedsusersemaillist
(https://lists.sourceforge.net/lists/listinfo/generatedsusers),andwanttospecifywhich
versionyouareusing.

4.5Thegraphicalfrontend
ThereisalsoapointandclickwaytorungenerateDS.Itenablesyoutospecifythe
optionsneededbygenerateDS.pythroughagraphicalinterface,thentorun
generateDS.pywiththoseoptions.Italso
Youcanrunit,ifyouhaveinstalledgenerateDS,bytypingthefollowingata
commandline:
Page264

APythonBook
$generateds_gui.py

Afterconfiguringoptions,youcansavethoseoptionsina"session"file,whichcanbe
loadedlater.LookundertheFilemenuforsaveandloadcommandsandalsoconsider
usingthe"session"commandlineoption.
AlsonotethatgenerateDS.pyitselfsupportsa"session"commandlineoptionthat
enablesyoutorungenerateDS.pywiththeoptionsthatyouspecifiedandsavedwith
thegraphicalfrontend.

4.6Addingapplicationspecificbehavior
generateDS.pygeneratesPythoncodewhich,withnomodification,willparseand
thenexportanXMLdocumentdefinedbyyourschema.However,youarelikelytowant
togobeyondthat.Inmanysituationsyouwillwanttoconstructacustomapplicationthat
processesyourXMLdocumentsusingthegeneratedcode.

4.6.1Implementingcustomsubclasses
Onestrategyistogenerateasubclassfileandtoaddyourapplicationspecificcodeto
that.Generatethesubclassfilewiththe"s"commandlineflag:
$generateDS.pysmyapp.pypeople.xsd

Nowaddsomeapplicationspecificcodetomyapp.py,forexample,ifyouareusingthe
included"people"samplefiles:
classpeopleTypeSub(supermod.people):
def__init__(self,comments=None,person=None,programmer=None,
python_programmer=None,java_programmer=None):
supermod.people.__init__(self,comments,person,programmer,
python_programmer,
java_programmer)
deffancyexport(self,outfile):
outfile.write('Startingfancyexport')
forpersoninself.get_person():
person.fancyexport(outfile)
supermod.people.subclass=peopleTypeSub
#endclasspeopleTypeSub
classpersonTypeSub(supermod.person):
def__init__(self,vegetable=None,fruit=None,ratio=None,
id=None,value=None,
name=None,interest=None,category=None,agent=None,
promoter=None,
description=None):
supermod.person.__init__(self,vegetable,fruit,ratio,id,
value,

Page265

APythonBook
name,interest,category,agent,promoter,description)
deffancyexport(self,outfile):
outfile.write('Fancypersonexportname:%s'%
self.get_name(),)
supermod.person.subclass=personTypeSub
#endclasspersonTypeSub

4.6.2Usingthegenerated"API"fromyourapplication
Inthisapproachyoumightdothingslikethefollowing:
importyourgeneratedclasses.
Createinstancesofthoseclasses.
Linkthoseinstances,forexampleput"children"insideofaparent,oraddoneor
moreinstancestoaparentthatcancontainalistofobjects(think"maxOccurs"
greaterthan1inyourschema)
GettoknowthegeneratedexportAPIbyinspectingthegeneratedcodeinthesuperclass
file.That'sthefilegeneratedwiththe"o"commandlineflag.

Whattolookfor:
Lookattheargumentstotheconstructor(__init__)tolearnhowtoinitialize
aninstance.
Lookatthe"getters"and"setters"(methodsnamegetxxxandsetxxx,tolearn
howtomodifymembervariables.
Lookforamethodnamedaddxxxformembersthatarelists.Thesecorrespond
tomembersdefinedwithmaxOccurs="n",wheren>1.
Lookatthebuildmethods:build,buildChildren,and
buildAttributes.Thesewillgiveyouinformationabouthowtoconstruct
eachofthemembersofagivenelement/class.
Now,youcanimportyourgeneratedAPImodule,anduseittoconstructandmanipulate
objects.Hereisanexampleusingcodegeneratedwiththe"people"schema:

importsys
importpeople_apiasapi
deftest(names):
people=api.peopleType()
forcount,nameinenumerate(names):
id='%d'%(count+1,)
person=api.personType(name=name,id=id)
people.add_person(person)
people.export(sys.stdout,0)
test(['albert','betsy','charlie'])

Runthisandyoumightseesomethinglikethefollowing:
Page266

APythonBook
$pythontmp.py
<people>
<personid="1">
<name>albert</name>
</person>
<personid="2">
<name>betsy</name>
</person>
<personid="3">
<name>charlie</name>
</person>
</people>

4.6.3Acombinedapproach
Note:Youcanfindexamplesofthecodeinthissectioninthesefiles:
tutorial/Code/upcase_names.py
tutorial/Code/upcase_names_appl.py

Herearetherelevant,modifiedsubclasses(upcase_names_appl.py):
importpeople_apiassupermod
classpeopleTypeSub(supermod.peopleType):
def__init__(self,comments=None,person=None,
specialperson=None,programmer=None,python_programmer=None,
java_programmer=None):
super(peopleTypeSub,self).__init__(comments,person,
specialperson,programmer,python_programmer,java_programmer,)
defupcase_names(self):
forpersoninself.get_person():
person.upcase_names()
supermod.peopleType.subclass=peopleTypeSub
#endclasspeopleTypeSub
classpersonTypeSub(supermod.personType):
def__init__(self,vegetable=None,fruit=None,ratio=None,
id=None,value=None,name=None,interest=None,category=None,
agent=None,promoter=None,description=None,range_=None,
extensiontype_=None):
super(personTypeSub,self).__init__(vegetable,fruit,ratio,
id,value,name,interest,category,agent,promoter,description,
range_,extensiontype_,)
defupcase_names(self):
self.set_name(self.get_name().upper())
supermod.personType.subclass=personTypeSub
#endclasspersonTypeSub

Notes:

Theseclassesweregeneratedwiththe"s"commandlineoption.Theyare
Page267

APythonBook
subclassesofclassesinthemodulepeople_api,whichwasgeneratedwiththe
"o"commandlineoption.
Theonlymodificationtotheskeletonsubclassesistheadditionofthetwo
methodsnamedupcase_names().
InthesubclasspeopleTypeSub,themethodupcase_names()merelywalk
overitsimmediatechildren.
InthesubclasspersonTypeSub,themethodupcase_names()justconverts
thevalueofits"name"membertouppercase.
Hereistheapplicationitself(upcase_names.py):
importsys
importupcase_names_applasappl
defcreate_people(names):
people=appl.peopleTypeSub()
forcount,nameinenumerate(names):
id='%d'%(count+1,)
person=appl.personTypeSub(name=name,id=id)
people.add_person(person)
returnpeople
defmain():
names=['albert','betsy','charlie']
people=create_people(names)
print'Before:'
people.export(sys.stdout,1)
people.upcase_names()
print''*50
print'After:'
people.export(sys.stdout,1)
main()

Notes:
Thecreate_people()functioncreatesapeopleTypeSubinstancewith
severalpersonTypeSubinstancesinsideit.
And,whenyourunthisminiapplication,hereiswhatyoumightsee:

$pythonupcase_names.py
Before:
<people>
<personid="1">
<name>albert</name>
</person>
<personid="2">
<name>betsy</name>
</person>
<personid="3">
<name>charlie</name>

Page268

APythonBook
</person>
</people>

After:
<people>
<personid="1">
<name>ALBERT</name>
</person>
<personid="2">
<name>BETSY</name>
</person>
<personid="3">
<name>CHARLIE</name>
</person>
</people>

4.7Specialsituationsanduses
4.7.1Generic,typeindependentprocessing
Therearetimeswhenyouwouldliketoimplementafunctionormethodthatcanperform
operationsonavarietyofmembersandthatneedstypeinformationabouteachmember.
Youcangethelpwiththisbygeneratingyourcodewiththe"memberspecs"command
lineoption.Whenyouusethisoption,generateDS.pyaddalistoradictionary
containinganitemforeachmember.Ifyouwantalist,thenuse"memberspecs=list",
andifyouwantadictionary,withmembernamesaskeys,thenuse
"memberspecs=dict".
HereisanexampleInthisexample,wewalkthedocument/instancetreeandconvert
allstringsimpletypestouppercase.
Hereisaschema(Code/member_specs.xsd):
<?xmlversion="1.0"?>
<xs:schemaxmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:elementname="contactlist"type="contactlistType"/>
<xs:complexTypename="contactlistType">
<xs:sequence>
<xs:elementname="description"type="xs:string"/>
<xs:elementname="contact"type="contactType"
maxOccurs="unbounded"/>
</xs:sequence>
<xs:attributename="locator"type="xs:string"/>
</xs:complexType>
<xs:complexTypename="contactType">

Page269

APythonBook
<xs:sequence>
<xs:elementname="firstname"type="xs:string"/>
<xs:elementname="lastname"type="xs:string"/>
<xs:elementname="interest"type="xs:string"
maxOccurs="unbounded"/>
<xs:elementname="category"type="xs:integer"/>
</xs:sequence>
<xs:attributename="id"type="xs:integer"/>
<xs:attributename="priority"type="xs:float"/>
<xs:attributename="colorcode"type="xs:string"/>
</xs:complexType>
</xs:schema>

4.7.1.1Step1generatethebindings

Wegeneratecodewiththefollowingcommandline:
$generateDS.pyf\
omember_specs_api.py\
smember_specs_upper.py\
super=member_specs_api\
memberspecs=list\
member_specs.xsd

Notes:

Wegeneratethememberspecificationsasalistwiththecommandlineoption
memberspecs=list.
Wegeneratean"application"modulewiththescommandlineoption.We'llput
ourapplicationspecificcodeinmember_specs_upper.py.

4.7.1.2Step2addapplicationspecificcode

And,hereisthesubclassfile(member_specs_upper.py,generatedwiththe"s"
commandlineoption),towhichwehaveaddedabitofcodethatconvertsanystringtype
memberstouppercase.Youcanthinkofthismoduleasaspecial"application"ofthe
generatedclasses.
#!/usr/bin/envpython
#
#member_specs_upper.py
#
#
#GeneratedTueNov915:54:472010bygenerateDS.pyversion2.2a.
#
importsys

Page270

APythonBook
importmember_specs_apiassupermod
etree_=None
Verbose_import_=False
(XMLParser_import_none,XMLParser_import_lxml,
XMLParser_import_elementtree
)=range(3)
XMLParser_import_library=None
try:
#lxml
fromlxmlimportetreeasetree_
XMLParser_import_library=XMLParser_import_lxml
ifVerbose_import_:
print("runningwithlxml.etree")
exceptImportError:
try:
#cElementTreefromPython2.5+
importxml.etree.cElementTreeasetree_
XMLParser_import_library=XMLParser_import_elementtree
ifVerbose_import_:
print("runningwithcElementTreeonPython2.5+")
exceptImportError:
try:
#ElementTreefromPython2.5+
importxml.etree.ElementTreeasetree_
XMLParser_import_library=XMLParser_import_elementtree
ifVerbose_import_:
print("runningwithElementTreeonPython2.5+")
exceptImportError:
try:
#normalcElementTreeinstall
importcElementTreeasetree_
XMLParser_import_library=
XMLParser_import_elementtree
ifVerbose_import_:
print("runningwithcElementTree")
exceptImportError:
try:
#normalElementTreeinstall
importelementtree.ElementTreeasetree_
XMLParser_import_library=
XMLParser_import_elementtree
ifVerbose_import_:
print("runningwithElementTree")
exceptImportError:
raiseImportError("FailedtoimportElementTree
fromanyknownplace")
defparsexml_(*args,**kwargs):
if(XMLParser_import_library==XMLParser_import_lxmland
'parser'notinkwargs):
#UsethelxmlElementTreecompatibleparsersothat,e.g.,

Page271

APythonBook
#weignorecomments.
kwargs['parser']=etree_.ETCompatXMLParser()
doc=etree_.parse(*args,**kwargs)
returndoc
#
#Globals
#
ExternalEncoding='ascii'
#
#Utilityfuntionsneededineachgeneratedclass.
#
defupper_elements(obj):
foriteminobj.member_data_items_:
ifitem.get_data_type()=='xs:string':
name=remap(item.get_name())
val1=getattr(obj,name)
ifisinstance(val1,list):
foridx,val2inenumerate(val1):
val1[idx]=val2.upper()
else:
setattr(obj,name,val1.upper())
defremap(name):
newname=name.replace('','_')
returnnewname
#
#Datarepresentationclasses
#
classcontactlistTypeSub(supermod.contactlistType):
def__init__(self,locator=None,description=None,contact=None):
super(contactlistTypeSub,self).__init__(locator,
description,contact,)
defupper(self):
upper_elements(self)
forchildinself.get_contact():
child.upper()
supermod.contactlistType.subclass=contactlistTypeSub
#endclasscontactlistTypeSub
classcontactTypeSub(supermod.contactType):
def__init__(self,priority=None,color_code=None,id=None,
first_name=None,last_name=None,interest=None,category=None):
super(contactTypeSub,self).__init__(priority,color_code,
id,first_name,last_name,interest,category,)
defupper(self):

Page272

APythonBook
upper_elements(self)
supermod.contactType.subclass=contactTypeSub
#endclasscontactTypeSub
defget_root_tag(node):
tag=supermod.Tag_pattern_.match(node.tag).groups()[1]
rootClass=None
ifhasattr(supermod,tag):
rootClass=getattr(supermod,tag)
returntag,rootClass
defparse(inFilename):
doc=parsexml_(inFilename)
rootNode=doc.getroot()
rootTag,rootClass=get_root_tag(rootNode)
ifrootClassisNone:
rootTag='contactlist'
rootClass=supermod.contactlistType
rootObj=rootClass.factory()
rootObj.build(rootNode)
#EnablePythontocollectthespaceusedbytheDOM.
doc=None
sys.stdout.write('<?xmlversion="1.0"?>\n')
rootObj.export(sys.stdout,0,name_=rootTag,
namespacedef_='')
doc=None
returnrootObj
defparseString(inString):
fromStringIOimportStringIO
doc=parsexml_(StringIO(inString))
rootNode=doc.getroot()
rootTag,rootClass=get_root_tag(rootNode)
ifrootClassisNone:
rootTag='contactlist'
rootClass=supermod.contactlistType
rootObj=rootClass.factory()
rootObj.build(rootNode)
#EnablePythontocollectthespaceusedbytheDOM.
doc=None
sys.stdout.write('<?xmlversion="1.0"?>\n')
rootObj.export(sys.stdout,0,name_=rootTag,
namespacedef_='')
returnrootObj
defparseLiteral(inFilename):
doc=parsexml_(inFilename)
rootNode=doc.getroot()
rootTag,rootClass=get_root_tag(rootNode)

Page273

APythonBook
ifrootClassisNone:
rootTag='contactlist'
rootClass=supermod.contactlistType
rootObj=rootClass.factory()
rootObj.build(rootNode)
#EnablePythontocollectthespaceusedbytheDOM.
doc=None
sys.stdout.write('#frommember_specs_apiimport*\n\n')
sys.stdout.write('importmember_specs_apiasmodel_\n\n')
sys.stdout.write('rootObj=model_.contact_list(\n')
rootObj.exportLiteral(sys.stdout,0,name_="contact_list")
sys.stdout.write(')\n')
returnrootObj
USAGE_TEXT="""
Usage:python???.py<infilename>
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
root=parse(infilename)
if__name__=='__main__':
#importpdb;pdb.set_trace()
main()

Notes:

Weaddthefunctionsupper_elementsandremapthatweuseineach
generatedclass.
Noticehowthefunctionupper_elementscallsthefunctionremaponlyon
thosememberswhosetypeisxs:string.
Ineachgenerated(sub)class,weaddthemethodsthatwalktheDOMtreeand
applythemethod(upper)thattransformseachxs:stringvalue.

4.7.1.3Step3writeatest/driverharness

Hereisatestdriver(member_specs_test.py)forour(mini)application:
#!/usr/bin/envpython

Page274

APythonBook
#
#member_specs_test.py
#
importsys
importmember_specs_apiassupermod
importmember_specs_upper
defprocess(inFilename):
doc=supermod.parsexml_(inFilename)
rootNode=doc.getroot()
rootClass=member_specs_upper.contactlistTypeSub
rootObj=rootClass.factory()
rootObj.build(rootNode)
#EnablePythontocollectthespaceusedbytheDOM.
doc=None
sys.stdout.write('<?xmlversion="1.0"?>\n')
rootObj.export(sys.stdout,0,name_="contactlist",
namespacedef_='')
rootObj.upper()
sys.stdout.write(''*60)
sys.stdout.write('\n')
rootObj.export(sys.stdout,0,name_="contactlist",
namespacedef_='')
returnrootObj
USAGE_MSG="""\
Synopsis:
Sampleapplicationusingclassesandsubclassesgeneratedby
generateDS.py
Usage:
pythonmember_specs_test.pyinfilename
"""
defusage():
printUSAGE_MSG
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
process(infilename)
if__name__=='__main__':
main()

Notes:

Wecopythefunctionparse()fromourgeneratedcodetoserveasamodelfor
Page275

APythonBook

ourfunctionprocess().
AfterparsinganddisplayingtheXMLinstancedocument,wecallmethod
upper()inthegeneratedclasscontactlistTypeSubinordertowalkthe
DOMtreeandtransformeachxs:stringtouppercase.

4.7.1.4Step4runthetestapplication

Wecanusethefollowingcommandlinetorunourapplication:
$pythonmember_specs_test.pymember_specs_data.xml

Whenwerunourapplication,hereistheoutput:
$pythonmember_specs_test.pymember_specs_data.xml
<?xmlversion="1.0"?>
<contactlistlocator="http://www.rexx.com/~dkuhlman">
<description>Mylistofcontacts</description>
<contactpriority="0.050000"colorcode="red"id="1">
<firstname>arlene</firstname>
<lastname>Allen</lastname>
<interest>traveling</interest>
<category>2</category>
</contact>
</contactlist>

<contactlistlocator="HTTP://WWW.REXX.COM/~DKUHLMAN">
<description>MYLISTOFCONTACTS</description>
<contactpriority="0.050000"colorcode="RED"id="1">
<firstname>ARLENE</firstname>
<lastname>ALLEN</lastname>
<interest>TRAVELING</interest>
<category>2</category>
</contact>
</contactlist>

Notes:

Theoutputaboveshowsbothbeforeandafterversionofexportingtheparsed
XMLinstancedocument.

4.8Somehints
Thefollowinghintsareofferedforconvenience.Youcandiscoverthemforyourself
rathereasilybyinspectingthegeneratedcode.

4.8.1ChildrendefinedwithmaxOccursgreaterthan1
IfachildelementisdefinedintheXMLschemawithmaxOccurs="unbounded"or
avalueofmaxOccursgreaterthan1,thenaccesstothechildisthroughalist.
Page276

APythonBook

4.8.2Childrendefinedwithsimplenumerictypes
Ifachildelementisdefinedasanumerictypesuchasxs:integer,xs:float,or
xs:doubleorasasimpletypethatis(ultimately)basedonanumerictype,thenthe
valueisstored(inthePythonobject)asaPythondatatype(int,float,etc).

4.8.3Thetypeofanelement'scharactercontent
But,whentheelementitselfisdefinedasmixed="true"ortheelementarestrictionof
andhasasimple(numeric)asabase,thenthevalueOf_instancevariableholdsthe
charactercontentanditisalwaysastring,thatisitisnotconverted.

4.8.4Constructorsandtheirdefaultvalues
Allparameterstotheconstructorsofgeneratedclasseshavedefaultparameters.
Therefore,youcancreatean"empty"instanceofanyelementbycallingtheconstructor
withnoparameters.
Forexample,supposewehavethefollowingXMLschema:
<?xmlversion="1.0"?>
<xs:schemaxmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:elementname="plantlist"type="PlantList"/>
<xs:complexTypename="PlantType">
<xs:sequence>
<xs:elementname="description"type="xs:string"/>
<xs:elementname="catagory"type="xs:integer"/>
<xs:elementname="fertilizer"type="FertilizerType"
maxOccurs="unbounded"/>
</xs:sequence>
<xs:attributename="identifier"type="xs:string"/>
</xs:complexType>
<xs:complexTypename="FertilizerType">
<xs:sequence>
<xs:elementname="name"type="xs:string"/>
<xs:elementname="description"type="xs:string"/>
</xs:sequence>
<xs:attributename="id"type="xs:integer"/>
</xs:complexType>
</xs:schema>

And,supposewegenerateamodulewiththefollowingcommandline:
$./generateDS.pyogarden_api.pygarden.xsd

Page277

APythonBook
Then,fortheelementnamedPlantTypeinthegeneratedmodulenamed
garden_api.py,youcancreateaninstanceasfollows:
>>>importgarden_api
>>>plant=garden_api.PlantType()
>>>importsys
>>>plant.export(sys.stdout,0)
<PlantType/>

Page278